2024-05-08T17:53:03.146 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T17:53:03.146 DEBUG:teuthology.run:Teuthology command: teuthology --name ivancich-2024-05-08_17:43:43-rgw-wip-eric-testing-1-distro-default-smithi --archive /home/teuthworker/archive/ivancich-2024-05-08_17:43:43-rgw-wip-eric-testing-1-distro-default-smithi/7697857 --verbose --description rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_latest}} --owner scheduled_ivancich@teuthology -- /home/teuthworker/archive/ivancich-2024-05-08_17:43:43-rgw-wip-eric-testing-1-distro-default-smithi/7697857/orig.config.yaml 2024-05-08T17:53:03.179 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T17:53:03.255 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/ivancich-2024-05-08_17:43:43-rgw-wip-eric-testing-1-distro-default-smithi/7697857 branch: wip-eric-testing-1 description: rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_latest}} email: ivancich@redhat.com first_in_suite: false job_id: '7697857' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: ivancich-2024-05-08_17:43:43-rgw-wip-eric-testing-1-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-eric-testing-1 ceph: conf: client: debug rgw: 20 setgroup: ceph setuser: ceph mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled sha1: 27d371685027569bcfa4efc661d3d3186f3a1776 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 27d371685027569bcfa4efc661d3d3186f3a1776 rgw: ec-data-pool: true erasure_code_profile: crush-failure-domain: osd k: 3 m: 1 name: testprofile frontend: beast s3tests: slow_backend: true selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-eric-testing-1 sha1: 27d371685027569bcfa4efc661d3d3186f3a1776 owner: scheduled_ivancich@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - - mgr.x - client.0 - osd.1 - osd.2 - osd.3 seed: 831 sha1: 27d371685027569bcfa4efc661d3d3186f3a1776 sleep_before_teardown: 0 subset: 1/50 suite: rgw suite_branch: wip-eric-testing-1 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_27d371685027569bcfa4efc661d3d3186f3a1776/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 27d371685027569bcfa4efc661d3d3186f3a1776 targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJr4zEpCGn//LIxU7aGC/o5MVrcwMvj4d93v+lvCqZIHfWrWTiAHriNdCD5BWjC8YWvxXSHcVyN9fyPIQXuI6Ag= smithi050.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEGj53oQeZ4et4D00re4FlURSj4VUvY3hlLD8oeqLymTYvDOiX4DMqjF3HRHVKwY+g2DKmAoy+04IF8euge4zm0= tasks: - install: null - ceph: conf: client: debug ms: 1 rgw gc obj min wait: 15 osd: debug ms: 1 debug objclass: 20 - rgw: client.0: null - radosgw-admin: null - radosgw-admin-rest: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_17:43:43 tube: smithi user: ivancich verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T17:53:03.256 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_27d371685027569bcfa4efc661d3d3186f3a1776/qa; will attempt to use it 2024-05-08T17:53:03.256 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_27d371685027569bcfa4efc661d3d3186f3a1776/qa/tasks 2024-05-08T17:53:03.256 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T17:53:03.258 INFO:teuthology.task.internal:Checking packages... 2024-05-08T17:53:03.281 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '27d371685027569bcfa4efc661d3d3186f3a1776' 2024-05-08T17:53:03.281 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T17:53:03.281 INFO:teuthology.packaging:ref: None 2024-05-08T17:53:03.281 INFO:teuthology.packaging:tag: None 2024-05-08T17:53:03.281 INFO:teuthology.packaging:branch: wip-eric-testing-1 2024-05-08T17:53:03.281 INFO:teuthology.packaging:sha1: 27d371685027569bcfa4efc661d3d3186f3a1776 2024-05-08T17:53:03.282 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-eric-testing-1 2024-05-08T17:53:03.496 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3525.g27d37168 2024-05-08T17:53:03.498 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T17:53:03.504 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T17:53:03.505 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T17:53:03.545 INFO:teuthology.task.internal:Saving configuration 2024-05-08T17:53:03.558 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T17:53:03.605 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T17:53:03.630 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/ivancich-2024-05-08_17:43:43-rgw-wip-eric-testing-1-distro-default-smithi/7697857', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 17:46:24.312126', 'locked_by': 'scheduled_ivancich@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJr4zEpCGn//LIxU7aGC/o5MVrcwMvj4d93v+lvCqZIHfWrWTiAHriNdCD5BWjC8YWvxXSHcVyN9fyPIQXuI6Ag='} 2024-05-08T17:53:03.653 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/ivancich-2024-05-08_17:43:43-rgw-wip-eric-testing-1-distro-default-smithi/7697857', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 17:46:24.310583', 'locked_by': 'scheduled_ivancich@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEGj53oQeZ4et4D00re4FlURSj4VUvY3hlLD8oeqLymTYvDOiX4DMqjF3HRHVKwY+g2DKmAoy+04IF8euge4zm0='} 2024-05-08T17:53:03.653 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T17:53:03.660 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'osd.0'] 2024-05-08T17:53:03.660 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3'] 2024-05-08T17:53:03.660 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T17:53:03.807 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f78bcde00d0>, signals=[15]) 2024-05-08T17:53:03.808 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T17:53:03.914 INFO:teuthology.task.internal:Opening connections... 2024-05-08T17:53:03.914 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T17:53:03.916 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:53:03.993 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2024-05-08T17:53:03.994 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:53:04.065 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T17:53:04.071 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-05-08T17:53:04.090 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-05-08T17:53:04.090 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-05-08T17:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2024-05-08T17:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="9" 2024-05-08T17:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2024-05-08T17:53:04.148 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2024-05-08T17:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="9" 2024-05-08T17:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el9" 2024-05-08T17:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T17:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2024-05-08T17:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout:LOGO="fedora-logo-icon" 2024-05-08T17:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T17:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2024-05-08T17:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T17:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T17:53:04.149 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T17:53:04.150 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-05-08T17:53:04.174 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2024-05-08T17:53:04.192 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2024-05-08T17:53:04.192 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2024-05-08T17:53:04.250 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream" 2024-05-08T17:53:04.250 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="9" 2024-05-08T17:53:04.250 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2024-05-08T17:53:04.250 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2024-05-08T17:53:04.250 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="9" 2024-05-08T17:53:04.250 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el9" 2024-05-08T17:53:04.250 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T17:53:04.250 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2024-05-08T17:53:04.250 INFO:teuthology.orchestra.run.smithi050.stdout:LOGO="fedora-logo-icon" 2024-05-08T17:53:04.251 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T17:53:04.251 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2024-05-08T17:53:04.251 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T17:53:04.251 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T17:53:04.251 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T17:53:04.251 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2024-05-08T17:53:04.274 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T17:53:04.283 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T17:53:04.399 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T17:53:04.399 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-05-08T17:53:04.401 DEBUG:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2024-05-08T17:53:04.418 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T17:53:04.476 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T17:53:04.476 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-05-08T17:53:04.479 DEBUG:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2024-05-08T17:53:04.508 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T17:53:04.583 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T17:53:04.598 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-08T17:53:04.598 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-08T17:53:04.598 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T17:53:04.599 DEBUG:teuthology.orchestra.run.smithi050:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T17:53:04.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T17:53:04.616 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-05-08T17:53:04.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T17:53:04.617 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-05-08T17:53:04.673 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-437.el9.x86_64 2024-05-08T17:53:04.673 INFO:teuthology.task.kernel:Running kernel on smithi046: 5.14.0-437.el9.x86_64 2024-05-08T17:53:04.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum install -y kernel 2024-05-08T17:53:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:5.14.0-437.el9.x86_64 2024-05-08T17:53:04.675 INFO:teuthology.task.kernel:Running kernel on smithi050: 5.14.0-437.el9.x86_64 2024-05-08T17:53:04.675 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum install -y kernel 2024-05-08T17:53:07.134 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.1 MB 00:00 2024-05-08T17:53:07.618 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 5.9 MB/s | 8.1 MB 00:01 2024-05-08T17:53:10.851 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 7.4 MB/s | 19 MB 00:02 2024-05-08T17:53:11.142 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 19 MB 00:02 2024-05-08T17:53:16.086 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 6.2 MB 00:01 2024-05-08T17:53:16.634 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 6.2 MB 00:01 2024-05-08T17:53:17.836 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-05-08T17:53:18.409 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-05-08T17:53:20.013 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-05-08T17:53:25.488 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-05-08T17:53:27.736 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T17:53:27.736 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T17:53:27.826 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T17:53:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T17:53:27.836 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-05-08T17:53:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T17:53:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T17:53:27.836 INFO:teuthology.orchestra.run.smithi046.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T17:53:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-05-08T17:53:27.836 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T17:53:27.836 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T17:53:27.836 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T17:53:27.836 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T17:53:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T17:53:27.836 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T17:53:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:Install 4 Packages 2024-05-08T17:53:27.837 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T17:53:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 78 M 2024-05-08T17:53:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 123 M 2024-05-08T17:53:27.837 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T17:53:28.064 INFO:teuthology.orchestra.run.smithi046.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 4.4 MB/s | 484 kB 00:00 2024-05-08T17:53:28.573 INFO:teuthology.orchestra.run.smithi046.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 25 MB/s | 15 MB 00:00 2024-05-08T17:53:29.181 INFO:teuthology.orchestra.run.smithi046.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 28 MB/s | 34 MB 00:01 2024-05-08T17:53:29.440 INFO:teuthology.orchestra.run.smithi046.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 20 MB/s | 28 MB 00:01 2024-05-08T17:53:29.440 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:53:29.440 INFO:teuthology.orchestra.run.smithi046.stdout:Total 49 MB/s | 78 MB 00:01 2024-05-08T17:53:30.152 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T17:53:30.570 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T17:53:30.570 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T17:53:31.532 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T17:53:31.532 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T17:53:34.114 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T17:53:34.763 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T17:53:34.856 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T17:53:37.683 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T17:53:37.823 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T17:53:42.425 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 906 kB/s | 21 MB 00:23 2024-05-08T17:53:42.896 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T17:53:42.967 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:53:47.667 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-05-08T17:53:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:53:49.665 INFO:teuthology.orchestra.run.smithi050.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T17:53:49.665 INFO:teuthology.orchestra.run.smithi050.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T17:53:49.763 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T17:53:49.771 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T17:53:49.771 INFO:teuthology.orchestra.run.smithi050.stdout: Package Architecture Version Repository Size 2024-05-08T17:53:49.771 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T17:53:49.771 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T17:53:49.771 INFO:teuthology.orchestra.run.smithi050.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T17:53:49.771 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-05-08T17:53:49.771 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T17:53:49.771 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T17:53:49.771 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T17:53:49.772 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T17:53:49.772 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T17:53:49.772 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T17:53:49.772 INFO:teuthology.orchestra.run.smithi050.stdout:Install 4 Packages 2024-05-08T17:53:49.772 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T17:53:49.772 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 78 M 2024-05-08T17:53:49.772 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 123 M 2024-05-08T17:53:49.772 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T17:53:49.927 INFO:teuthology.orchestra.run.smithi050.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 5.5 MB/s | 484 kB 00:00 2024-05-08T17:53:50.510 INFO:teuthology.orchestra.run.smithi050.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 23 MB/s | 15 MB 00:00 2024-05-08T17:53:51.002 INFO:teuthology.orchestra.run.smithi050.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 29 MB/s | 34 MB 00:01 2024-05-08T17:53:51.286 INFO:teuthology.orchestra.run.smithi050.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 21 MB/s | 28 MB 00:01 2024-05-08T17:53:51.286 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:53:51.286 INFO:teuthology.orchestra.run.smithi050.stdout:Total 51 MB/s | 78 MB 00:01 2024-05-08T17:53:51.998 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T17:53:52.397 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T17:53:52.398 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T17:53:53.373 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T17:53:53.373 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T17:53:55.919 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T17:53:56.577 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T17:53:56.654 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T17:53:59.440 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T17:53:59.584 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T17:54:04.658 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T17:54:04.703 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:54:09.798 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:54:19.154 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:54:19.195 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:54:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:54:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T17:54:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T17:54:19.558 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T17:54:20.407 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:54:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T17:54:20.408 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T17:54:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T17:54:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T17:54:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T17:54:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T17:54:20.408 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T17:54:20.408 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T17:54:20.757 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T17:54:20.834 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T17:54:20.835 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T17:54:20.835 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T17:54:20.835 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T17:54:20.835 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T17:54:20.836 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-08T17:54:20.836 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi046.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T17:54:20.836 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T17:54:20.886 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T17:54:20.887 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T17:54:20.887 DEBUG:teuthology.orchestra.run.smithi046:> sudo rpm -qi grub2-tools 2024-05-08T17:54:20.921 INFO:teuthology.orchestra.run.smithi046.stdout:Name : grub2-tools 2024-05-08T17:54:20.921 INFO:teuthology.orchestra.run.smithi046.stdout:Epoch : 1 2024-05-08T17:54:20.921 INFO:teuthology.orchestra.run.smithi046.stdout:Version : 2.06 2024-05-08T17:54:20.921 INFO:teuthology.orchestra.run.smithi046.stdout:Release : 68.el9 2024-05-08T17:54:20.921 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: x86_64 2024-05-08T17:54:20.921 INFO:teuthology.orchestra.run.smithi046.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T17:54:20.921 INFO:teuthology.orchestra.run.smithi046.stdout:Group : Unspecified 2024-05-08T17:54:20.921 INFO:teuthology.orchestra.run.smithi046.stdout:Size : 8264254 2024-05-08T17:54:20.921 INFO:teuthology.orchestra.run.smithi046.stdout:License : GPLv3+ 2024-05-08T17:54:20.922 INFO:teuthology.orchestra.run.smithi046.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T17:54:20.922 INFO:teuthology.orchestra.run.smithi046.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T17:54:20.922 INFO:teuthology.orchestra.run.smithi046.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T17:54:20.922 INFO:teuthology.orchestra.run.smithi046.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T17:54:20.922 INFO:teuthology.orchestra.run.smithi046.stdout:Packager : builder@centos.org 2024-05-08T17:54:20.922 INFO:teuthology.orchestra.run.smithi046.stdout:Vendor : CentOS 2024-05-08T17:54:20.922 INFO:teuthology.orchestra.run.smithi046.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T17:54:20.922 INFO:teuthology.orchestra.run.smithi046.stdout:Summary : Support tools for GRUB. 2024-05-08T17:54:20.922 INFO:teuthology.orchestra.run.smithi046.stdout:Description : 2024-05-08T17:54:20.922 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T17:54:20.922 INFO:teuthology.orchestra.run.smithi046.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T17:54:20.922 INFO:teuthology.orchestra.run.smithi046.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T17:54:20.922 INFO:teuthology.orchestra.run.smithi046.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T17:54:20.922 INFO:teuthology.orchestra.run.smithi046.stdout:hardware devices. 2024-05-08T17:54:20.923 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T17:54:20.923 INFO:teuthology.orchestra.run.smithi046.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T17:54:20.923 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T17:54:20.923 INFO:teuthology.task.kernel:Updating grub on smithi046 to boot 5.14.0-444.el9.x86_64 2024-05-08T17:54:20.923 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T17:54:21.594 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-05-08T17:54:22.997 INFO:teuthology.orchestra.run.smithi046.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T17:54:23.033 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-05-08T17:54:23.036 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-05-08T17:54:23.052 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.Xwatxd0SfC 2024-05-08T17:54:23.052 DEBUG:teuthology.orchestra.run.smithi046:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Xwatxd0SfC 2024-05-08T17:54:23.117 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.Xwatxd0SfC 2024-05-08T17:54:23.259 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.Xwatxd0SfC is 6KB 2024-05-08T17:54:23.307 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.Xwatxd0SfC 2024-05-08T17:54:23.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T17:54:23.385 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T17:54:23.385 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T17:54:23.385 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T17:54:23.385 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T17:54:23.387 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T17:54:23.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo shutdown -r now 2024-05-08T17:54:40.868 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:54:40.908 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:54:41.234 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:54:41.235 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T17:54:41.235 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T17:54:41.235 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T17:54:42.026 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:54:42.026 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T17:54:42.026 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T17:54:42.026 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T17:54:42.026 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T17:54:42.026 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T17:54:42.027 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T17:54:42.027 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T17:54:42.027 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T17:54:42.362 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T17:54:42.433 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T17:54:42.434 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T17:54:42.434 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T17:54:42.434 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T17:54:42.434 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T17:54:42.434 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-05-08T17:54:42.434 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi050.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T17:54:42.435 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T17:54:42.485 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T17:54:42.485 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T17:54:42.485 DEBUG:teuthology.orchestra.run.smithi050:> sudo rpm -qi grub2-tools 2024-05-08T17:54:42.520 INFO:teuthology.orchestra.run.smithi050.stdout:Name : grub2-tools 2024-05-08T17:54:42.521 INFO:teuthology.orchestra.run.smithi050.stdout:Epoch : 1 2024-05-08T17:54:42.521 INFO:teuthology.orchestra.run.smithi050.stdout:Version : 2.06 2024-05-08T17:54:42.521 INFO:teuthology.orchestra.run.smithi050.stdout:Release : 68.el9 2024-05-08T17:54:42.521 INFO:teuthology.orchestra.run.smithi050.stdout:Architecture: x86_64 2024-05-08T17:54:42.521 INFO:teuthology.orchestra.run.smithi050.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T17:54:42.521 INFO:teuthology.orchestra.run.smithi050.stdout:Group : Unspecified 2024-05-08T17:54:42.521 INFO:teuthology.orchestra.run.smithi050.stdout:Size : 8264254 2024-05-08T17:54:42.521 INFO:teuthology.orchestra.run.smithi050.stdout:License : GPLv3+ 2024-05-08T17:54:42.521 INFO:teuthology.orchestra.run.smithi050.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T17:54:42.521 INFO:teuthology.orchestra.run.smithi050.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T17:54:42.521 INFO:teuthology.orchestra.run.smithi050.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T17:54:42.521 INFO:teuthology.orchestra.run.smithi050.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T17:54:42.522 INFO:teuthology.orchestra.run.smithi050.stdout:Packager : builder@centos.org 2024-05-08T17:54:42.522 INFO:teuthology.orchestra.run.smithi050.stdout:Vendor : CentOS 2024-05-08T17:54:42.522 INFO:teuthology.orchestra.run.smithi050.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T17:54:42.522 INFO:teuthology.orchestra.run.smithi050.stdout:Summary : Support tools for GRUB. 2024-05-08T17:54:42.522 INFO:teuthology.orchestra.run.smithi050.stdout:Description : 2024-05-08T17:54:42.522 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T17:54:42.522 INFO:teuthology.orchestra.run.smithi050.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T17:54:42.522 INFO:teuthology.orchestra.run.smithi050.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T17:54:42.522 INFO:teuthology.orchestra.run.smithi050.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T17:54:42.522 INFO:teuthology.orchestra.run.smithi050.stdout:hardware devices. 2024-05-08T17:54:42.522 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T17:54:42.522 INFO:teuthology.orchestra.run.smithi050.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T17:54:42.523 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T17:54:42.523 INFO:teuthology.task.kernel:Updating grub on smithi050 to boot 5.14.0-444.el9.x86_64 2024-05-08T17:54:42.524 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T17:54:43.113 INFO:teuthology.orchestra.run.smithi050.stderr:Generating grub configuration file ... 2024-05-08T17:54:44.675 INFO:teuthology.orchestra.run.smithi050.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T17:54:44.702 INFO:teuthology.orchestra.run.smithi050.stderr:done 2024-05-08T17:54:44.704 DEBUG:teuthology.orchestra.run.smithi050:> mktemp 2024-05-08T17:54:44.719 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.ZTRHXL2HbB 2024-05-08T17:54:44.720 DEBUG:teuthology.orchestra.run.smithi050:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ZTRHXL2HbB 2024-05-08T17:54:44.787 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.ZTRHXL2HbB 2024-05-08T17:54:44.923 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.ZTRHXL2HbB is 6KB 2024-05-08T17:54:44.971 DEBUG:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.ZTRHXL2HbB 2024-05-08T17:54:44.986 DEBUG:teuthology.orchestra.run.smithi050:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T17:54:45.054 INFO:teuthology.orchestra.run.smithi050.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T17:54:45.054 INFO:teuthology.orchestra.run.smithi050.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T17:54:45.054 INFO:teuthology.orchestra.run.smithi050.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T17:54:45.054 INFO:teuthology.orchestra.run.smithi050.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T17:54:45.055 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T17:54:45.217 DEBUG:teuthology.orchestra.run.smithi050:> sudo shutdown -r now 2024-05-08T17:54:45.292 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-05-08T17:54:53.538 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-08T17:54:53.538 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T17:54:53.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:55:12.031 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-05-08T17:55:15.222 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-05-08T17:55:15.223 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-08T17:55:15.224 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:55:21.037 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T17:55:21.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:55:24.094 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-05-08T17:55:33.794 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.50 2024-05-08T17:55:36.098 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T17:55:36.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:55:42.802 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-08T17:55:42.803 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:55:54.526 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-05-08T17:56:09.539 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T17:56:09.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:56:09.896 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-05-08T17:56:10.403 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T17:56:10.403 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-444.el9.x86_64"... 2024-05-08T17:56:10.404 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-05-08T17:56:10.419 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-444.el9.x86_64 2024-05-08T17:56:10.419 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T17:56:10.419 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T17:56:10.419 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T17:56:11.420 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-08T17:56:11.420 DEBUG:teuthology.orchestra.run.smithi046:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T17:56:11.499 INFO:teuthology.orchestra.run.smithi046.stdout:ttyS1 2024-05-08T17:56:11.532 DEBUG:teuthology.parallel:result is None 2024-05-08T17:56:42.805 DEBUG:teuthology.orchestra.remote:timed out 2024-05-08T17:56:54.807 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-08T17:56:54.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:56:55.114 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-05-08T17:56:55.487 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-08T17:56:55.487 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-444.el9.x86_64"... 2024-05-08T17:56:55.488 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-05-08T17:56:55.505 INFO:teuthology.orchestra.run.smithi050.stdout:5.14.0-444.el9.x86_64 2024-05-08T17:56:55.506 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T17:56:55.506 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T17:56:55.506 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T17:56:56.506 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-05-08T17:56:56.506 DEBUG:teuthology.orchestra.run.smithi050:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T17:56:56.573 INFO:teuthology.orchestra.run.smithi050.stdout:ttyS1 2024-05-08T17:56:56.607 DEBUG:teuthology.parallel:result is None 2024-05-08T17:56:56.607 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T17:56:56.618 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T17:56:56.618 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T17:56:56.625 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T17:56:56.644 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T17:56:56.673 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T17:56:56.743 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T17:56:56.744 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T17:56:56.747 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T17:56:56.779 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T17:56:56.871 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T17:56:56.871 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T17:56:56.874 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T17:56:56.910 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T17:56:56.915 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T17:56:56.922 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T17:56:56.927 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T17:56:56.929 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T17:56:56.982 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T17:56:56.982 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T17:56:56.985 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T17:56:57.015 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T17:56:57.086 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T17:56:57.087 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T17:56:57.090 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T17:56:57.109 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T17:56:57.193 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T17:56:57.237 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T17:56:57.321 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T17:56:57.365 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T17:56:57.365 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T17:56:57.432 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T17:56:57.481 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T17:56:57.532 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T17:56:57.619 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T17:56:57.662 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T17:56:57.662 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T17:56:57.731 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-05-08T17:56:57.734 DEBUG:teuthology.orchestra.run.smithi050:> sudo service rsyslog restart 2024-05-08T17:56:57.784 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T17:56:57.816 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T17:56:58.173 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T17:56:58.181 INFO:teuthology.task.internal:Starting timer... 2024-05-08T17:56:58.181 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T17:56:58.271 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T17:56:58.342 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-08T17:56:58.343 DEBUG:teuthology.orchestra.run.smithi046:> sudo service auditd rotate 2024-05-08T17:56:58.404 INFO:teuthology.orchestra.run.smithi046.stdout:Rotating logs: 2024-05-08T17:56:58.406 DEBUG:teuthology.orchestra.run.smithi050:> sudo service auditd rotate 2024-05-08T17:56:58.467 INFO:teuthology.orchestra.run.smithi050.stdout:Rotating logs: 2024-05-08T17:56:58.469 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T17:56:58.469 DEBUG:teuthology.orchestra.run.smithi046:> /usr/sbin/getenforce 2024-05-08T17:56:58.501 INFO:teuthology.orchestra.run.smithi046.stdout:Permissive 2024-05-08T17:56:58.501 DEBUG:teuthology.orchestra.run.smithi050:> /usr/sbin/getenforce 2024-05-08T17:56:58.533 INFO:teuthology.orchestra.run.smithi050.stdout:Permissive 2024-05-08T17:56:58.534 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi046.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi050.front.sepia.ceph.com': 'permissive'} 2024-05-08T17:56:58.534 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T17:56:58.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T17:56:58.570 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T17:56:58.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T17:56:58.605 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T17:56:58.605 DEBUG:teuthology.orchestra.run.smithi046:> sudo /usr/sbin/setenforce permissive 2024-05-08T17:56:58.639 DEBUG:teuthology.orchestra.run.smithi050:> sudo /usr/sbin/setenforce permissive 2024-05-08T17:56:58.679 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T17:56:58.689 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T17:56:58.708 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T17:56:58.710 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi050.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T18:01:27.304 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi050.front.sepia.ceph.com')] 2024-05-08T18:01:27.305 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T18:01:27.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T18:01:27.388 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-05-08T18:01:27.466 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-08T18:01:27.467 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-08T18:01:27.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T18:01:27.548 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-05-08T18:01:27.628 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-05-08T18:01:27.628 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T18:01:27.638 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T18:01:27.639 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T18:01:27.639 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T18:01:27.641 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T18:01:27.641 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T18:01:27.670 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T18:01:27.686 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T18:01:27.703 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T18:01:27.711 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: ntpd: command not found 2024-05-08T18:01:27.718 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T18:01:27.723 INFO:teuthology.orchestra.run.smithi046.stdout:506 Cannot talk to daemon 2024-05-08T18:01:27.738 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T18:01:27.745 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: ntpd: command not found 2024-05-08T18:01:27.752 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T18:01:27.757 INFO:teuthology.orchestra.run.smithi050.stdout:506 Cannot talk to daemon 2024-05-08T18:01:27.773 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T18:01:27.787 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T18:01:27.808 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-05-08T18:01:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T18:01:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-05-08T18:01:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T18:01:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T18:01:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T18:01:27.813 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T18:01:27.842 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-05-08T18:01:27.846 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T18:01:27.846 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-05-08T18:01:27.846 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T18:01:27.846 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T18:01:27.846 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T18:01:27.846 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T18:01:27.847 INFO:teuthology.run_tasks:Running task install... 2024-05-08T18:01:27.856 DEBUG:teuthology.task.install:project ceph 2024-05-08T18:01:27.856 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '27d371685027569bcfa4efc661d3d3186f3a1776'}} 2024-05-08T18:01:27.856 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '27d371685027569bcfa4efc661d3d3186f3a1776'} 2024-05-08T18:01:27.856 INFO:teuthology.task.install:Using flavor: default 2024-05-08T18:01:27.863 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T18:01:27.863 INFO:teuthology.task.install:extra packages: [] 2024-05-08T18:01:27.863 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '27d371685027569bcfa4efc661d3d3186f3a1776', 'tag': None, 'wait_for_package': False} 2024-05-08T18:01:27.864 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=27d371685027569bcfa4efc661d3d3186f3a1776 2024-05-08T18:01:27.866 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '27d371685027569bcfa4efc661d3d3186f3a1776', 'tag': None, 'wait_for_package': False} 2024-05-08T18:01:27.867 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=27d371685027569bcfa4efc661d3d3186f3a1776 2024-05-08T18:01:28.018 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-eric-testing-1/27d371685027569bcfa4efc661d3d3186f3a1776/centos/9/flavors/default/ 2024-05-08T18:01:28.018 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3525.g27d37168 2024-05-08T18:01:28.033 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-eric-testing-1/27d371685027569bcfa4efc661d3d3186f3a1776/centos/9/flavors/default/ 2024-05-08T18:01:28.034 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3525.g27d37168 2024-05-08T18:01:28.131 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-eric-testing-1/27d371685027569bcfa4efc661d3d3186f3a1776/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-eric-testing-1/27d371685027569bcfa4efc661d3d3186f3a1776/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-eric-testing-1/27d371685027569bcfa4efc661d3d3186f3a1776/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T18:01:28.131 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T18:01:28.131 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T18:01:28.146 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-eric-testing-1/27d371685027569bcfa4efc661d3d3186f3a1776/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-eric-testing-1/27d371685027569bcfa4efc661d3d3186f3a1776/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-eric-testing-1/27d371685027569bcfa4efc661d3d3186f3a1776/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T18:01:28.146 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T18:01:28.147 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T18:01:28.159 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-08T18:01:28.159 DEBUG:teuthology.orchestra.run.smithi046:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/27d371685027569bcfa4efc661d3d3186f3a1776/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T18:01:28.173 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-08T18:01:28.173 DEBUG:teuthology.orchestra.run.smithi050:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/27d371685027569bcfa4efc661d3d3186f3a1776/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T18:01:28.226 DEBUG:teuthology.orchestra.run.smithi046:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T18:01:28.241 DEBUG:teuthology.orchestra.run.smithi050:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T18:01:28.303 DEBUG:teuthology.orchestra.run.smithi046:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T18:01:28.318 DEBUG:teuthology.orchestra.run.smithi050:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T18:01:28.369 INFO:teuthology.orchestra.run.smithi046.stdout:check_obsoletes = 1 2024-05-08T18:01:28.371 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum clean all 2024-05-08T18:01:28.383 INFO:teuthology.orchestra.run.smithi050.stdout:check_obsoletes = 1 2024-05-08T18:01:28.387 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum clean all 2024-05-08T18:01:28.612 INFO:teuthology.orchestra.run.smithi046.stdout:54 files removed 2024-05-08T18:01:28.633 INFO:teuthology.orchestra.run.smithi050.stdout:54 files removed 2024-05-08T18:01:28.639 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-radosgw 2024-05-08T18:01:28.660 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-radosgw 2024-05-08T18:01:29.342 INFO:teuthology.orchestra.run.smithi046.stdout:ceph packages for x86_64 202 kB/s | 84 kB 00:00 2024-05-08T18:01:29.366 INFO:teuthology.orchestra.run.smithi050.stdout:ceph packages for x86_64 201 kB/s | 84 kB 00:00 2024-05-08T18:01:29.800 INFO:teuthology.orchestra.run.smithi046.stdout:ceph noarch packages 29 kB/s | 12 kB 00:00 2024-05-08T18:01:29.808 INFO:teuthology.orchestra.run.smithi050.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-05-08T18:01:30.193 INFO:teuthology.orchestra.run.smithi050.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-05-08T18:01:30.209 INFO:teuthology.orchestra.run.smithi046.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-05-08T18:01:30.885 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-05-08T18:01:32.059 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 4.5 MB/s | 8.1 MB 00:01 2024-05-08T18:01:34.977 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-05-08T18:01:41.662 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.2 MB 00:02 2024-05-08T18:01:43.054 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 95 kB/s | 16 kB 00:00 2024-05-08T18:01:45.967 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 7.6 MB/s | 21 MB 00:02 2024-05-08T18:01:46.143 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-05-08T18:01:51.281 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-08T18:01:51.868 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.2 MB 00:02 2024-05-08T18:01:53.528 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-05-08T18:01:53.732 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:01:53.740 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:01:53.740 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-05-08T18:01:53.740 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:01:53.740 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T18:01:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw x86_64 2:19.0.0-3525.g27d37168.el9 ceph 11 M 2024-05-08T18:01:53.741 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrading: 2024-05-08T18:01:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 x86_64 2:19.0.0-3525.g27d37168.el9 ceph 3.4 M 2024-05-08T18:01:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1 x86_64 2:19.0.0-3525.g27d37168.el9 ceph 3.1 M 2024-05-08T18:01:53.741 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-05-08T18:01:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T18:01:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base x86_64 2:19.0.0-3525.g27d37168.el9 ceph 5.4 M 2024-05-08T18:01:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common x86_64 2:19.0.0-3525.g27d37168.el9 ceph 21 M 2024-05-08T18:01:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux x86_64 2:19.0.0-3525.g27d37168.el9 ceph 25 k 2024-05-08T18:01:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T18:01:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T18:01:53.741 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T18:01:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2 x86_64 2:19.0.0-3525.g27d37168.el9 ceph 747 k 2024-05-08T18:01:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T18:01:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T18:01:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T18:01:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 x86_64 2:19.0.0-3525.g27d37168.el9 ceph 501 k 2024-05-08T18:01:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T18:01:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2 x86_64 2:19.0.0-3525.g27d37168.el9 ceph 5.4 M 2024-05-08T18:01:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T18:01:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T18:01:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T18:01:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse x86_64 2:19.0.0-3525.g27d37168.el9 ceph 45 k 2024-05-08T18:01:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common x86_64 2:19.0.0-3525.g27d37168.el9 ceph 135 k 2024-05-08T18:01:53.742 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs x86_64 2:19.0.0-3525.g27d37168.el9 ceph 162 k 2024-05-08T18:01:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T18:01:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados x86_64 2:19.0.0-3525.g27d37168.el9 ceph 320 k 2024-05-08T18:01:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd x86_64 2:19.0.0-3525.g27d37168.el9 ceph 298 k 2024-05-08T18:01:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw x86_64 2:19.0.0-3525.g27d37168.el9 ceph 100 k 2024-05-08T18:01:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T18:01:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T18:01:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T18:01:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T18:01:53.743 INFO:teuthology.orchestra.run.smithi046.stdout:Installing weak dependencies: 2024-05-08T18:01:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T18:01:53.743 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:01:53.743 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T18:01:53.743 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:01:53.743 INFO:teuthology.orchestra.run.smithi046.stdout:Install 30 Packages 2024-05-08T18:01:53.743 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrade 2 Packages 2024-05-08T18:01:53.744 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:01:53.744 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 60 M 2024-05-08T18:01:53.744 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T18:01:55.023 INFO:teuthology.orchestra.run.smithi046.stdout:(1/32): ceph-radosgw-19.0.0-3525.g27d37168.el9. 10 MB/s | 11 MB 00:01 2024-05-08T18:01:55.038 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-05-08T18:01:55.099 INFO:teuthology.orchestra.run.smithi046.stdout:(2/32): ceph-base-19.0.0-3525.g27d37168.el9.x86 4.9 MB/s | 5.4 MB 00:01 2024-05-08T18:01:55.124 INFO:teuthology.orchestra.run.smithi046.stdout:(3/32): ceph-selinux-19.0.0-3525.g27d37168.el9. 251 kB/s | 25 kB 00:00 2024-05-08T18:01:55.191 INFO:teuthology.orchestra.run.smithi046.stdout:(4/32): libradosstriper1-19.0.0-3525.g27d37168. 7.4 MB/s | 501 kB 00:00 2024-05-08T18:01:55.242 INFO:teuthology.orchestra.run.smithi046.stdout:(5/32): libcephfs2-19.0.0-3525.g27d37168.el9.x8 5.1 MB/s | 747 kB 00:00 2024-05-08T18:01:55.341 INFO:teuthology.orchestra.run.smithi046.stdout:(6/32): python3-ceph-argparse-19.0.0-3525.g27d3 452 kB/s | 45 kB 00:00 2024-05-08T18:01:55.442 INFO:teuthology.orchestra.run.smithi046.stdout:(7/32): python3-ceph-common-19.0.0-3525.g27d371 1.3 MB/s | 135 kB 00:00 2024-05-08T18:01:55.592 INFO:teuthology.orchestra.run.smithi046.stdout:(8/32): librgw2-19.0.0-3525.g27d37168.el9.x86_6 13 MB/s | 5.4 MB 00:00 2024-05-08T18:01:55.617 INFO:teuthology.orchestra.run.smithi046.stdout:(9/32): python3-cephfs-19.0.0-3525.g27d37168.el 924 kB/s | 162 kB 00:00 2024-05-08T18:01:55.643 INFO:teuthology.orchestra.run.smithi046.stdout:(10/32): python3-rados-19.0.0-3525.g27d37168.el 6.3 MB/s | 320 kB 00:00 2024-05-08T18:01:55.701 INFO:teuthology.orchestra.run.smithi046.stdout:(11/32): python3-rgw-19.0.0-3525.g27d37168.el9. 1.7 MB/s | 100 kB 00:00 2024-05-08T18:01:55.727 INFO:teuthology.orchestra.run.smithi046.stdout:(12/32): python3-rbd-19.0.0-3525.g27d37168.el9. 2.7 MB/s | 298 kB 00:00 2024-05-08T18:01:55.943 INFO:teuthology.orchestra.run.smithi046.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 442 kB/s | 107 kB 00:00 2024-05-08T18:01:55.977 INFO:teuthology.orchestra.run.smithi046.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 640 kB/s | 160 kB 00:00 2024-05-08T18:01:56.011 INFO:teuthology.orchestra.run.smithi046.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-05-08T18:01:56.295 INFO:teuthology.orchestra.run.smithi046.stdout:(16/32): ceph-common-19.0.0-3525.g27d37168.el9. 9.2 MB/s | 21 MB 00:02 2024-05-08T18:01:56.320 INFO:teuthology.orchestra.run.smithi046.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 947 kB/s | 292 kB 00:00 2024-05-08T18:01:56.395 INFO:teuthology.orchestra.run.smithi046.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 1.6 MB/s | 662 kB 00:00 2024-05-08T18:01:56.470 INFO:teuthology.orchestra.run.smithi046.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-05-08T18:01:56.487 INFO:teuthology.orchestra.run.smithi046.stdout:(20/32): python3-prettytable-0.7.2-27.el9.noarc 220 kB/s | 42 kB 00:00 2024-05-08T18:01:56.504 INFO:teuthology.orchestra.run.smithi046.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 329 kB/s | 36 kB 00:00 2024-05-08T18:01:56.529 INFO:teuthology.orchestra.run.smithi046.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 807 kB/s | 47 kB 00:00 2024-05-08T18:01:58.829 INFO:teuthology.orchestra.run.smithi046.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 132 kB/s | 309 kB 00:02 2024-05-08T18:01:59.038 INFO:teuthology.orchestra.run.smithi046.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 10 kB/s | 26 kB 00:02 2024-05-08T18:01:59.338 INFO:teuthology.orchestra.run.smithi046.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 95 kB/s | 49 kB 00:00 2024-05-08T18:01:59.705 INFO:teuthology.orchestra.run.smithi046.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 101 kB/s | 67 kB 00:00 2024-05-08T18:02:00.039 INFO:teuthology.orchestra.run.smithi046.stdout:(27/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.2 MB/s | 839 kB 00:00 2024-05-08T18:02:00.270 INFO:teuthology.orchestra.run.smithi046.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 343 kB/s | 192 kB 00:00 2024-05-08T18:02:00.363 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-08T18:02:02.106 INFO:teuthology.orchestra.run.smithi046.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 784 kB/s | 1.6 MB 00:02 2024-05-08T18:02:02.631 INFO:teuthology.orchestra.run.smithi046.stdout:(30/32): librbd1-19.0.0-3525.g27d37168.el9.x86_ 6.0 MB/s | 3.1 MB 00:00 2024-05-08T18:02:02.799 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:02:02.806 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:02:02.806 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-05-08T18:02:02.806 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:02:02.806 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T18:02:02.806 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw x86_64 2:19.0.0-3525.g27d37168.el9 ceph 11 M 2024-05-08T18:02:02.806 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrading: 2024-05-08T18:02:02.806 INFO:teuthology.orchestra.run.smithi050.stdout: librados2 x86_64 2:19.0.0-3525.g27d37168.el9 ceph 3.4 M 2024-05-08T18:02:02.806 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1 x86_64 2:19.0.0-3525.g27d37168.el9 ceph 3.1 M 2024-05-08T18:02:02.806 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-05-08T18:02:02.807 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T18:02:02.807 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base x86_64 2:19.0.0-3525.g27d37168.el9 ceph 5.4 M 2024-05-08T18:02:02.807 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common x86_64 2:19.0.0-3525.g27d37168.el9 ceph 21 M 2024-05-08T18:02:02.807 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux x86_64 2:19.0.0-3525.g27d37168.el9 ceph 25 k 2024-05-08T18:02:02.807 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T18:02:02.807 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T18:02:02.807 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T18:02:02.807 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2 x86_64 2:19.0.0-3525.g27d37168.el9 ceph 747 k 2024-05-08T18:02:02.807 INFO:teuthology.orchestra.run.smithi050.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T18:02:02.807 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T18:02:02.807 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T18:02:02.807 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1 x86_64 2:19.0.0-3525.g27d37168.el9 ceph 501 k 2024-05-08T18:02:02.807 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T18:02:02.807 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2 x86_64 2:19.0.0-3525.g27d37168.el9 ceph 5.4 M 2024-05-08T18:02:02.808 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T18:02:02.808 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T18:02:02.808 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T18:02:02.808 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse x86_64 2:19.0.0-3525.g27d37168.el9 ceph 45 k 2024-05-08T18:02:02.808 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common x86_64 2:19.0.0-3525.g27d37168.el9 ceph 135 k 2024-05-08T18:02:02.808 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs x86_64 2:19.0.0-3525.g27d37168.el9 ceph 162 k 2024-05-08T18:02:02.808 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T18:02:02.808 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados x86_64 2:19.0.0-3525.g27d37168.el9 ceph 320 k 2024-05-08T18:02:02.808 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd x86_64 2:19.0.0-3525.g27d37168.el9 ceph 298 k 2024-05-08T18:02:02.808 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw x86_64 2:19.0.0-3525.g27d37168.el9 ceph 100 k 2024-05-08T18:02:02.808 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T18:02:02.808 INFO:teuthology.orchestra.run.smithi050.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T18:02:02.808 INFO:teuthology.orchestra.run.smithi050.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T18:02:02.808 INFO:teuthology.orchestra.run.smithi050.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T18:02:02.809 INFO:teuthology.orchestra.run.smithi050.stdout:Installing weak dependencies: 2024-05-08T18:02:02.809 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T18:02:02.809 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:02.809 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T18:02:02.809 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:02:02.809 INFO:teuthology.orchestra.run.smithi050.stdout:Install 30 Packages 2024-05-08T18:02:02.809 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrade 2 Packages 2024-05-08T18:02:02.809 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:02.809 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 60 M 2024-05-08T18:02:02.810 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T18:02:04.715 INFO:teuthology.orchestra.run.smithi050.stdout:(1/32): ceph-radosgw-19.0.0-3525.g27d37168.el9. 7.3 MB/s | 11 MB 00:01 2024-05-08T18:02:04.767 INFO:teuthology.orchestra.run.smithi050.stdout:(2/32): ceph-selinux-19.0.0-3525.g27d37168.el9. 505 kB/s | 25 kB 00:00 2024-05-08T18:02:04.899 INFO:teuthology.orchestra.run.smithi050.stdout:(3/32): libcephfs2-19.0.0-3525.g27d37168.el9.x8 5.5 MB/s | 747 kB 00:00 2024-05-08T18:02:04.982 INFO:teuthology.orchestra.run.smithi050.stdout:(4/32): libradosstriper1-19.0.0-3525.g27d37168. 5.9 MB/s | 501 kB 00:00 2024-05-08T18:02:05.258 INFO:teuthology.orchestra.run.smithi050.stdout:(5/32): ceph-base-19.0.0-3525.g27d37168.el9.x86 2.7 MB/s | 5.4 MB 00:01 2024-05-08T18:02:05.440 INFO:teuthology.orchestra.run.smithi046.stdout:(31/32): librados2-19.0.0-3525.g27d37168.el9.x8 672 kB/s | 3.4 MB 00:05 2024-05-08T18:02:05.500 INFO:teuthology.orchestra.run.smithi050.stdout:(6/32): ceph-common-19.0.0-3525.g27d37168.el9.x 9.5 MB/s | 21 MB 00:02 2024-05-08T18:02:05.525 INFO:teuthology.orchestra.run.smithi050.stdout:(7/32): python3-ceph-argparse-19.0.0-3525.g27d3 169 kB/s | 45 kB 00:00 2024-05-08T18:02:05.550 INFO:teuthology.orchestra.run.smithi050.stdout:(8/32): python3-ceph-common-19.0.0-3525.g27d371 2.6 MB/s | 135 kB 00:00 2024-05-08T18:02:05.592 INFO:teuthology.orchestra.run.smithi050.stdout:(9/32): python3-cephfs-19.0.0-3525.g27d37168.el 2.4 MB/s | 162 kB 00:00 2024-05-08T18:02:05.668 INFO:teuthology.orchestra.run.smithi050.stdout:(10/32): librgw2-19.0.0-3525.g27d37168.el9.x86_ 7.8 MB/s | 5.4 MB 00:00 2024-05-08T18:02:05.693 INFO:teuthology.orchestra.run.smithi050.stdout:(11/32): python3-rados-19.0.0-3525.g27d37168.el 2.2 MB/s | 320 kB 00:00 2024-05-08T18:02:05.718 INFO:teuthology.orchestra.run.smithi050.stdout:(12/32): python3-rbd-19.0.0-3525.g27d37168.el9. 2.3 MB/s | 298 kB 00:00 2024-05-08T18:02:05.743 INFO:teuthology.orchestra.run.smithi050.stdout:(13/32): python3-rgw-19.0.0-3525.g27d37168.el9. 1.3 MB/s | 100 kB 00:00 2024-05-08T18:02:05.902 INFO:teuthology.orchestra.run.smithi050.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 285 kB/s | 45 kB 00:00 2024-05-08T18:02:05.936 INFO:teuthology.orchestra.run.smithi050.stdout:(15/32): boost-program-options-1.75.0-8.el9.x86 440 kB/s | 107 kB 00:00 2024-05-08T18:02:05.961 INFO:teuthology.orchestra.run.smithi050.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 660 kB/s | 160 kB 00:00 2024-05-08T18:02:06.007 INFO:teuthology.orchestra.run.smithi046.stdout:(32/32): libarrow-9.0.0-11.el9.x86_64.rpm 476 kB/s | 4.4 MB 00:09 2024-05-08T18:02:06.009 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:02:06.010 INFO:teuthology.orchestra.run.smithi046.stdout:Total 4.9 MB/s | 60 MB 00:12 2024-05-08T18:02:06.020 INFO:teuthology.orchestra.run.smithi050.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 725 kB/s | 42 kB 00:00 2024-05-08T18:02:06.087 INFO:teuthology.orchestra.run.smithi050.stdout:(18/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-05-08T18:02:06.120 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T18:02:06.129 INFO:teuthology.orchestra.run.smithi050.stdout:(19/32): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2024-05-08T18:02:06.154 INFO:teuthology.orchestra.run.smithi050.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-05-08T18:02:06.160 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T18:02:06.160 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T18:02:06.171 INFO:teuthology.orchestra.run.smithi050.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 427 kB/s | 36 kB 00:00 2024-05-08T18:02:06.197 INFO:teuthology.orchestra.run.smithi050.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 706 kB/s | 47 kB 00:00 2024-05-08T18:02:06.305 INFO:teuthology.orchestra.run.smithi050.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-05-08T18:02:06.397 INFO:teuthology.orchestra.run.smithi050.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-05-08T18:02:06.422 INFO:teuthology.orchestra.run.smithi050.stdout:(25/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.1 MB/s | 309 kB 00:00 2024-05-08T18:02:06.481 INFO:teuthology.orchestra.run.smithi050.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 806 kB/s | 67 kB 00:00 2024-05-08T18:02:06.547 INFO:teuthology.orchestra.run.smithi050.stdout:(27/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.6 MB/s | 839 kB 00:00 2024-05-08T18:02:06.573 INFO:teuthology.orchestra.run.smithi050.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-05-08T18:02:06.640 INFO:teuthology.orchestra.run.smithi050.stdout:(29/32): libarrow-9.0.0-11.el9.x86_64.rpm 9.4 MB/s | 4.4 MB 00:00 2024-05-08T18:02:06.690 INFO:teuthology.orchestra.run.smithi050.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-05-08T18:02:06.804 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T18:02:06.804 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T18:02:06.915 INFO:teuthology.orchestra.run.smithi050.stdout:(31/32): librbd1-19.0.0-3525.g27d37168.el9.x86_ 11 MB/s | 3.1 MB 00:00 2024-05-08T18:02:07.266 INFO:teuthology.orchestra.run.smithi050.stdout:(32/32): librados2-19.0.0-3525.g27d37168.el9.x8 4.9 MB/s | 3.4 MB 00:00 2024-05-08T18:02:07.267 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:02:07.268 INFO:teuthology.orchestra.run.smithi050.stdout:Total 14 MB/s | 60 MB 00:04 2024-05-08T18:02:07.374 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T18:02:07.414 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T18:02:07.414 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T18:02:08.055 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T18:02:08.056 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T18:02:08.224 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T18:02:08.301 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-08T18:02:08.337 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-08T18:02:08.362 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-08T18:02:08.385 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T18:02:08.412 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T18:02:08.571 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T18:02:08.630 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T18:02:09.043 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-08T18:02:09.105 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librados2-2:19.0.0-3525.g27d37168.el9.x86_64 7/34 2024-05-08T18:02:09.180 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:19.0.0-3525.g27d37168.el9.x86_64 7/34 2024-05-08T18:02:09.270 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rados-2:19.0.0-3525.g27d37168.el9.x86_64 8/34 2024-05-08T18:02:09.298 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs2-2:19.0.0-3525.g27d37168.el9.x86_64 9/34 2024-05-08T18:02:09.361 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs2-2:19.0.0-3525.g27d37168.el9.x86_64 9/34 2024-05-08T18:02:09.398 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-08T18:02:09.467 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-argparse-2:19.0.0-3525.g27d37168.el9. 11/34 2024-05-08T18:02:09.537 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cephfs-2:19.0.0-3525.g27d37168.el9.x86_64 12/34 2024-05-08T18:02:09.558 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libradosstriper1-2:19.0.0-3525.g27d37168.el9.x86_6 13/34 2024-05-08T18:02:09.623 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3525.g27d37168.el9.x86_6 13/34 2024-05-08T18:02:09.649 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-08T18:02:09.739 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T18:02:09.761 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-08T18:02:09.791 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-08T18:02:09.818 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-08T18:02:09.846 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-08T18:02:09.854 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-08T18:02:09.871 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-08T18:02:09.881 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-08T18:02:09.894 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T18:02:09.910 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-08T18:02:09.915 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T18:02:09.943 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-08T18:02:10.064 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T18:02:10.121 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T18:02:10.274 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-08T18:02:10.305 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librbd1-2:19.0.0-3525.g27d37168.el9.x86_64 22/34 2024-05-08T18:02:10.380 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:19.0.0-3525.g27d37168.el9.x86_64 22/34 2024-05-08T18:02:10.411 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rbd-2:19.0.0-3525.g27d37168.el9.x86_64 23/34 2024-05-08T18:02:10.447 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-08T18:02:10.493 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-08T18:02:10.598 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librados2-2:19.0.0-3525.g27d37168.el9.x86_64 7/34 2024-05-08T18:02:10.699 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:19.0.0-3525.g27d37168.el9.x86_64 7/34 2024-05-08T18:02:10.789 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rados-2:19.0.0-3525.g27d37168.el9.x86_64 8/34 2024-05-08T18:02:10.816 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs2-2:19.0.0-3525.g27d37168.el9.x86_64 9/34 2024-05-08T18:02:10.896 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephfs2-2:19.0.0-3525.g27d37168.el9.x86_64 9/34 2024-05-08T18:02:10.925 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-08T18:02:10.961 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-argparse-2:19.0.0-3525.g27d37168.el9. 11/34 2024-05-08T18:02:10.964 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-08T18:02:10.991 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librgw2-2:19.0.0-3525.g27d37168.el9.x86_64 26/34 2024-05-08T18:02:11.031 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cephfs-2:19.0.0-3525.g27d37168.el9.x86_64 12/34 2024-05-08T18:02:11.047 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librgw2-2:19.0.0-3525.g27d37168.el9.x86_64 26/34 2024-05-08T18:02:11.061 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libradosstriper1-2:19.0.0-3525.g27d37168.el9.x86_6 13/34 2024-05-08T18:02:11.125 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3525.g27d37168.el9.x86_6 13/34 2024-05-08T18:02:11.126 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rgw-2:19.0.0-3525.g27d37168.el9.x86_64 27/34 2024-05-08T18:02:11.159 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-08T18:02:11.160 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-common-2:19.0.0-3525.g27d37168.el9.x8 28/34 2024-05-08T18:02:11.271 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-08T18:02:11.302 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-08T18:02:11.343 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-08T18:02:11.375 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-08T18:02:11.404 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-08T18:02:11.437 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-08T18:02:11.767 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-08T18:02:11.808 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librbd1-2:19.0.0-3525.g27d37168.el9.x86_64 22/34 2024-05-08T18:02:11.891 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:19.0.0-3525.g27d37168.el9.x86_64 22/34 2024-05-08T18:02:11.922 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rbd-2:19.0.0-3525.g27d37168.el9.x86_64 23/34 2024-05-08T18:02:11.958 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-08T18:02:12.476 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-08T18:02:12.503 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librgw2-2:19.0.0-3525.g27d37168.el9.x86_64 26/34 2024-05-08T18:02:12.567 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librgw2-2:19.0.0-3525.g27d37168.el9.x86_64 26/34 2024-05-08T18:02:12.646 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rgw-2:19.0.0-3525.g27d37168.el9.x86_64 27/34 2024-05-08T18:02:12.672 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-common-2:19.0.0-3525.g27d37168.el9.x8 28/34 2024-05-08T18:02:13.597 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.0.0-3525.g27d37168.el9.x86_64 29/34 2024-05-08T18:02:13.640 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-common-2:19.0.0-3525.g27d37168.el9.x86_64 29/34 2024-05-08T18:02:14.257 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.0.0-3525.g27d37168.el9.x86_64 29/34 2024-05-08T18:02:14.294 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-base-2:19.0.0-3525.g27d37168.el9.x86_64 30/34 2024-05-08T18:02:14.390 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-base-2:19.0.0-3525.g27d37168.el9.x86_64 30/34 2024-05-08T18:02:14.390 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T18:02:14.391 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T18:02:14.391 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-selinux-2:19.0.0-3525.g27d37168.el9.x86_64 31/34 2024-05-08T18:02:15.255 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-3525.g27d37168.el9.x86_64 29/34 2024-05-08T18:02:15.293 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-common-2:19.0.0-3525.g27d37168.el9.x86_64 29/34 2024-05-08T18:02:15.874 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-3525.g27d37168.el9.x86_64 29/34 2024-05-08T18:02:15.914 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-base-2:19.0.0-3525.g27d37168.el9.x86_64 30/34 2024-05-08T18:02:16.006 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-base-2:19.0.0-3525.g27d37168.el9.x86_64 30/34 2024-05-08T18:02:16.006 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T18:02:16.006 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T18:02:16.007 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:16.049 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-selinux-2:19.0.0-3525.g27d37168.el9.x86_64 31/34 2024-05-08T18:02:24.348 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3525.g27d37168.el9.x86_64 31/34 2024-05-08T18:02:24.348 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /sys 2024-05-08T18:02:24.348 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /proc 2024-05-08T18:02:24.348 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /mnt 2024-05-08T18:02:24.348 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /var/tmp 2024-05-08T18:02:24.348 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /home 2024-05-08T18:02:24.348 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /root 2024-05-08T18:02:24.349 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /tmp 2024-05-08T18:02:24.349 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:25.363 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-radosgw-2:19.0.0-3525.g27d37168.el9.x86_64 32/34 2024-05-08T18:02:25.390 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3525.g27d37168.el9.x86_64 32/34 2024-05-08T18:02:25.390 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T18:02:25.390 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T18:02:25.390 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T18:02:25.390 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T18:02:25.390 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:25.392 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T18:02:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T18:02:25.491 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T18:02:26.192 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3525.g27d37168.el9.x86_64 31/34 2024-05-08T18:02:26.192 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /sys 2024-05-08T18:02:26.192 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /proc 2024-05-08T18:02:26.192 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /mnt 2024-05-08T18:02:26.192 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /var/tmp 2024-05-08T18:02:26.193 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /home 2024-05-08T18:02:26.193 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /root 2024-05-08T18:02:26.193 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /tmp 2024-05-08T18:02:26.193 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:26.555 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T18:02:26.555 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-base-2:19.0.0-3525.g27d37168.el9.x86_64 1/34 2024-05-08T18:02:26.555 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-common-2:19.0.0-3525.g27d37168.el9.x86_64 2/34 2024-05-08T18:02:26.555 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-radosgw-2:19.0.0-3525.g27d37168.el9.x86_64 3/34 2024-05-08T18:02:26.555 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-selinux-2:19.0.0-3525.g27d37168.el9.x86_64 4/34 2024-05-08T18:02:26.555 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs2-2:19.0.0-3525.g27d37168.el9.x86_64 5/34 2024-05-08T18:02:26.555 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libradosstriper1-2:19.0.0-3525.g27d37168.el9.x86_6 6/34 2024-05-08T18:02:26.555 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librgw2-2:19.0.0-3525.g27d37168.el9.x86_64 7/34 2024-05-08T18:02:26.555 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3525.g27d37168.el9. 8/34 2024-05-08T18:02:26.555 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-common-2:19.0.0-3525.g27d37168.el9.x8 9/34 2024-05-08T18:02:26.555 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cephfs-2:19.0.0-3525.g27d37168.el9.x86_64 10/34 2024-05-08T18:02:26.556 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rados-2:19.0.0-3525.g27d37168.el9.x86_64 11/34 2024-05-08T18:02:26.556 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rbd-2:19.0.0-3525.g27d37168.el9.x86_64 12/34 2024-05-08T18:02:26.556 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rgw-2:19.0.0-3525.g27d37168.el9.x86_64 13/34 2024-05-08T18:02:26.556 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-08T18:02:26.556 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-08T18:02:26.556 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-08T18:02:26.556 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-08T18:02:26.556 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-08T18:02:26.556 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-08T18:02:26.556 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-08T18:02:26.556 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-08T18:02:26.557 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-08T18:02:26.557 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-08T18:02:26.557 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-08T18:02:26.557 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-08T18:02:26.557 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-08T18:02:26.557 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-08T18:02:26.557 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-08T18:02:26.557 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-08T18:02:26.557 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-08T18:02:26.558 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:19.0.0-3525.g27d37168.el9.x86_64 31/34 2024-05-08T18:02:26.558 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-08T18:02:26.558 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:19.0.0-3525.g27d37168.el9.x86_64 33/34 2024-05-08T18:02:27.227 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-radosgw-2:19.0.0-3525.g27d37168.el9.x86_64 32/34 2024-05-08T18:02:27.254 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3525.g27d37168.el9.x86_64 32/34 2024-05-08T18:02:27.254 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T18:02:27.254 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T18:02:27.255 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T18:02:27.255 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T18:02:27.255 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:27.256 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T18:02:27.339 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T18:02:27.339 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T18:02:27.474 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T18:02:27.475 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:27.475 INFO:teuthology.orchestra.run.smithi046.stdout:Upgraded: 2024-05-08T18:02:27.475 INFO:teuthology.orchestra.run.smithi046.stdout: librados2-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:27.475 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:27.475 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T18:02:27.475 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T18:02:27.475 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:27.475 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:27.475 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:27.475 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:27.475 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T18:02:27.475 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T18:02:27.475 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T18:02:27.476 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:27.476 INFO:teuthology.orchestra.run.smithi046.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T18:02:27.476 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T18:02:27.476 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T18:02:27.476 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:27.476 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T18:02:27.476 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:27.476 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T18:02:27.476 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T18:02:27.476 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T18:02:27.476 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:27.476 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:27.476 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:27.476 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T18:02:27.476 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:27.477 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:27.477 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:27.477 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T18:02:27.477 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T18:02:27.477 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T18:02:27.477 INFO:teuthology.orchestra.run.smithi046.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T18:02:27.477 INFO:teuthology.orchestra.run.smithi046.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T18:02:27.477 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:27.477 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:02:27.852 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-test 2024-05-08T18:02:28.443 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:37 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:02:28.535 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T18:02:28.535 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-base-2:19.0.0-3525.g27d37168.el9.x86_64 1/34 2024-05-08T18:02:28.535 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-common-2:19.0.0-3525.g27d37168.el9.x86_64 2/34 2024-05-08T18:02:28.535 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-radosgw-2:19.0.0-3525.g27d37168.el9.x86_64 3/34 2024-05-08T18:02:28.535 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-selinux-2:19.0.0-3525.g27d37168.el9.x86_64 4/34 2024-05-08T18:02:28.535 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs2-2:19.0.0-3525.g27d37168.el9.x86_64 5/34 2024-05-08T18:02:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libradosstriper1-2:19.0.0-3525.g27d37168.el9.x86_6 6/34 2024-05-08T18:02:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librgw2-2:19.0.0-3525.g27d37168.el9.x86_64 7/34 2024-05-08T18:02:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3525.g27d37168.el9. 8/34 2024-05-08T18:02:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-common-2:19.0.0-3525.g27d37168.el9.x8 9/34 2024-05-08T18:02:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cephfs-2:19.0.0-3525.g27d37168.el9.x86_64 10/34 2024-05-08T18:02:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rados-2:19.0.0-3525.g27d37168.el9.x86_64 11/34 2024-05-08T18:02:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rbd-2:19.0.0-3525.g27d37168.el9.x86_64 12/34 2024-05-08T18:02:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rgw-2:19.0.0-3525.g27d37168.el9.x86_64 13/34 2024-05-08T18:02:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-08T18:02:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-08T18:02:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-08T18:02:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-08T18:02:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-08T18:02:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-08T18:02:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-08T18:02:28.536 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-08T18:02:28.537 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-08T18:02:28.537 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-08T18:02:28.537 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-08T18:02:28.537 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-08T18:02:28.537 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-08T18:02:28.537 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-08T18:02:28.537 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-08T18:02:28.537 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-08T18:02:28.537 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-08T18:02:28.537 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:19.0.0-3525.g27d37168.el9.x86_64 31/34 2024-05-08T18:02:28.537 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-08T18:02:28.537 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:19.0.0-3525.g27d37168.el9.x86_64 33/34 2024-05-08T18:02:28.568 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:02:28.568 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:02:28.568 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-05-08T18:02:28.568 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:02:28.568 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T18:02:28.569 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test x86_64 2:19.0.0-3525.g27d37168.el9 ceph 49 M 2024-05-08T18:02:28.569 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-05-08T18:02:28.569 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite x86_64 2:19.0.0-3525.g27d37168.el9 ceph 163 k 2024-05-08T18:02:28.569 INFO:teuthology.orchestra.run.smithi046.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T18:02:28.569 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T18:02:28.569 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:28.569 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T18:02:28.569 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:02:28.569 INFO:teuthology.orchestra.run.smithi046.stdout:Install 4 Packages 2024-05-08T18:02:28.569 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:28.570 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 50 M 2024-05-08T18:02:28.570 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 211 M 2024-05-08T18:02:28.570 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T18:02:28.862 INFO:teuthology.orchestra.run.smithi046.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.2 MB/s | 305 kB 00:00 2024-05-08T18:02:28.920 INFO:teuthology.orchestra.run.smithi046.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-08T18:02:29.020 INFO:teuthology.orchestra.run.smithi046.stdout:(3/4): libcephsqlite-19.0.0-3525.g27d37168.el9. 404 kB/s | 163 kB 00:00 2024-05-08T18:02:29.327 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T18:02:29.328 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:29.328 INFO:teuthology.orchestra.run.smithi050.stdout:Upgraded: 2024-05-08T18:02:29.328 INFO:teuthology.orchestra.run.smithi050.stdout: librados2-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:29.328 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:29.328 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T18:02:29.328 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T18:02:29.328 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:29.328 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:29.328 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:29.328 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:29.328 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T18:02:29.328 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T18:02:29.329 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T18:02:29.329 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:29.329 INFO:teuthology.orchestra.run.smithi050.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T18:02:29.329 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T18:02:29.329 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T18:02:29.329 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:29.329 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T18:02:29.329 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:29.329 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T18:02:29.329 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T18:02:29.329 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T18:02:29.329 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:29.329 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:29.329 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:29.329 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T18:02:29.329 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:29.330 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:29.330 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:29.330 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T18:02:29.330 INFO:teuthology.orchestra.run.smithi050.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T18:02:29.330 INFO:teuthology.orchestra.run.smithi050.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T18:02:29.330 INFO:teuthology.orchestra.run.smithi050.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T18:02:29.330 INFO:teuthology.orchestra.run.smithi050.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T18:02:29.330 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:29.330 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:02:29.664 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-test 2024-05-08T18:02:30.251 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:00:30 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:02:30.377 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:02:30.378 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:02:30.378 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-05-08T18:02:30.378 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:02:30.378 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T18:02:30.378 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test x86_64 2:19.0.0-3525.g27d37168.el9 ceph 49 M 2024-05-08T18:02:30.378 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-05-08T18:02:30.378 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite x86_64 2:19.0.0-3525.g27d37168.el9 ceph 163 k 2024-05-08T18:02:30.378 INFO:teuthology.orchestra.run.smithi050.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T18:02:30.379 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T18:02:30.379 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:30.379 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T18:02:30.379 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:02:30.379 INFO:teuthology.orchestra.run.smithi050.stdout:Install 4 Packages 2024-05-08T18:02:30.379 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:30.379 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 50 M 2024-05-08T18:02:30.380 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 211 M 2024-05-08T18:02:30.380 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T18:02:30.853 INFO:teuthology.orchestra.run.smithi050.stdout:(1/4): libcephsqlite-19.0.0-3525.g27d37168.el9. 641 kB/s | 163 kB 00:00 2024-05-08T18:02:31.011 INFO:teuthology.orchestra.run.smithi050.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 737 kB/s | 305 kB 00:00 2024-05-08T18:02:31.045 INFO:teuthology.orchestra.run.smithi050.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 332 kB/s | 64 kB 00:00 2024-05-08T18:02:34.012 INFO:teuthology.orchestra.run.smithi050.stdout:(4/4): ceph-test-19.0.0-3525.g27d37168.el9.x86_ 14 MB/s | 49 MB 00:03 2024-05-08T18:02:34.013 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:02:34.014 INFO:teuthology.orchestra.run.smithi050.stdout:Total 14 MB/s | 50 MB 00:03 2024-05-08T18:02:34.042 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T18:02:34.065 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T18:02:34.065 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T18:02:34.228 INFO:teuthology.orchestra.run.smithi046.stdout:(4/4): ceph-test-19.0.0-3525.g27d37168.el9.x86_ 8.8 MB/s | 49 MB 00:05 2024-05-08T18:02:34.230 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:02:34.231 INFO:teuthology.orchestra.run.smithi046.stdout:Total 8.8 MB/s | 50 MB 00:05 2024-05-08T18:02:34.258 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T18:02:34.281 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T18:02:34.281 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T18:02:34.456 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T18:02:34.456 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T18:02:34.687 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T18:02:34.687 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T18:02:35.007 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T18:02:35.080 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-08T18:02:35.128 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-08T18:02:35.156 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephsqlite-2:19.0.0-3525.g27d37168.el9.x86_64 3/4 2024-05-08T18:02:35.233 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T18:02:35.306 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-08T18:02:35.345 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-08T18:02:35.372 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephsqlite-2:19.0.0-3525.g27d37168.el9.x86_64 3/4 2024-05-08T18:02:40.083 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3525.g27d37168.el9.x86_64 3/4 2024-05-08T18:02:40.157 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-test-2:19.0.0-3525.g27d37168.el9.x86_64 4/4 2024-05-08T18:02:40.301 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3525.g27d37168.el9.x86_64 3/4 2024-05-08T18:02:40.390 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-test-2:19.0.0-3525.g27d37168.el9.x86_64 4/4 2024-05-08T18:02:42.684 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-test-2:19.0.0-3525.g27d37168.el9.x86_64 4/4 2024-05-08T18:02:42.685 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-test-2:19.0.0-3525.g27d37168.el9.x86_64 1/4 2024-05-08T18:02:42.685 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephsqlite-2:19.0.0-3525.g27d37168.el9.x86_64 2/4 2024-05-08T18:02:42.685 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-08T18:02:42.839 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-test-2:19.0.0-3525.g27d37168.el9.x86_64 4/4 2024-05-08T18:02:42.839 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-test-2:19.0.0-3525.g27d37168.el9.x86_64 1/4 2024-05-08T18:02:42.839 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephsqlite-2:19.0.0-3525.g27d37168.el9.x86_64 2/4 2024-05-08T18:02:42.839 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-08T18:02:42.953 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-08T18:02:42.953 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:42.953 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T18:02:42.953 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:42.953 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:42.954 INFO:teuthology.orchestra.run.smithi050.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T18:02:42.954 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T18:02:42.954 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:42.954 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:02:43.137 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-08T18:02:43.137 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:43.137 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T18:02:43.137 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:43.137 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:43.137 INFO:teuthology.orchestra.run.smithi046.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T18:02:43.137 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T18:02:43.137 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:43.137 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:02:43.267 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph 2024-05-08T18:02:43.431 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph 2024-05-08T18:02:43.856 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:00:43 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:02:43.985 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:02:43.988 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:02:43.988 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-05-08T18:02:43.988 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:02:43.988 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T18:02:43.988 INFO:teuthology.orchestra.run.smithi050.stdout: ceph x86_64 2:19.0.0-3525.g27d37168.el9 ceph 6.6 k 2024-05-08T18:02:43.989 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-05-08T18:02:43.989 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds x86_64 2:19.0.0-3525.g27d37168.el9 ceph 2.4 M 2024-05-08T18:02:43.989 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr x86_64 2:19.0.0-3525.g27d37168.el9 ceph 1.5 M 2024-05-08T18:02:43.989 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 274 k 2024-05-08T18:02:43.989 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon x86_64 2:19.0.0-3525.g27d37168.el9 ceph 4.7 M 2024-05-08T18:02:43.989 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd x86_64 2:19.0.0-3525.g27d37168.el9 ceph 17 M 2024-05-08T18:02:43.989 INFO:teuthology.orchestra.run.smithi050.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T18:02:43.989 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T18:02:43.989 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T18:02:43.989 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T18:02:43.989 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T18:02:43.989 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T18:02:43.989 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T18:02:43.989 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T18:02:43.989 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T18:02:43.990 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T18:02:43.990 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections 2024-05-08T18:02:43.990 INFO:teuthology.orchestra.run.smithi050.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-08T18:02:43.990 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T18:02:43.990 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T18:02:43.990 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T18:02:43.990 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T18:02:43.990 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T18:02:43.990 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T18:02:43.990 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T18:02:43.990 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T18:02:43.990 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T18:02:43.990 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T18:02:43.990 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T18:02:43.991 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T18:02:43.991 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T18:02:43.991 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T18:02:43.991 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T18:02:43.991 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T18:02:43.991 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T18:02:43.991 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T18:02:43.991 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T18:02:43.991 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:43.991 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T18:02:43.991 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:02:43.991 INFO:teuthology.orchestra.run.smithi050.stdout:Install 36 Packages 2024-05-08T18:02:43.991 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:43.992 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 30 M 2024-05-08T18:02:43.992 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 108 M 2024-05-08T18:02:43.992 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T18:02:44.020 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:53 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:02:44.149 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:02:44.152 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:02:44.152 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-05-08T18:02:44.152 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:02:44.152 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T18:02:44.152 INFO:teuthology.orchestra.run.smithi046.stdout: ceph x86_64 2:19.0.0-3525.g27d37168.el9 ceph 6.6 k 2024-05-08T18:02:44.152 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-05-08T18:02:44.152 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds x86_64 2:19.0.0-3525.g27d37168.el9 ceph 2.4 M 2024-05-08T18:02:44.152 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr x86_64 2:19.0.0-3525.g27d37168.el9 ceph 1.5 M 2024-05-08T18:02:44.153 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 274 k 2024-05-08T18:02:44.153 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon x86_64 2:19.0.0-3525.g27d37168.el9 ceph 4.7 M 2024-05-08T18:02:44.153 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd x86_64 2:19.0.0-3525.g27d37168.el9 ceph 17 M 2024-05-08T18:02:44.153 INFO:teuthology.orchestra.run.smithi046.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T18:02:44.153 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T18:02:44.153 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T18:02:44.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T18:02:44.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T18:02:44.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T18:02:44.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T18:02:44.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T18:02:44.153 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T18:02:44.154 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T18:02:44.154 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections 2024-05-08T18:02:44.154 INFO:teuthology.orchestra.run.smithi046.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-08T18:02:44.154 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T18:02:44.154 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T18:02:44.154 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T18:02:44.154 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T18:02:44.154 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T18:02:44.154 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T18:02:44.155 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T18:02:44.155 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T18:02:44.155 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T18:02:44.155 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T18:02:44.155 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T18:02:44.155 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T18:02:44.155 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T18:02:44.155 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T18:02:44.155 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T18:02:44.155 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T18:02:44.155 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T18:02:44.155 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T18:02:44.155 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T18:02:44.155 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:44.156 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T18:02:44.156 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:02:44.156 INFO:teuthology.orchestra.run.smithi046.stdout:Install 36 Packages 2024-05-08T18:02:44.156 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:44.156 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 30 M 2024-05-08T18:02:44.157 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 108 M 2024-05-08T18:02:44.157 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T18:02:44.815 INFO:teuthology.orchestra.run.smithi050.stdout:(1/36): ceph-19.0.0-3525.g27d37168.el9.x86_64.r 22 kB/s | 6.6 kB 00:00 2024-05-08T18:02:44.916 INFO:teuthology.orchestra.run.smithi050.stdout:(2/36): ceph-mds-19.0.0-3525.g27d37168.el9.x86_ 5.9 MB/s | 2.4 MB 00:00 2024-05-08T18:02:44.957 INFO:teuthology.orchestra.run.smithi050.stdout:(3/36): ceph-mgr-19.0.0-3525.g27d37168.el9.x86_ 3.4 MB/s | 1.5 MB 00:00 2024-05-08T18:02:45.049 INFO:teuthology.orchestra.run.smithi050.stdout:(4/36): ceph-mgr-modules-core-19.0.0-3525.g27d3 2.9 MB/s | 274 kB 00:00 2024-05-08T18:02:45.139 INFO:teuthology.orchestra.run.smithi046.stdout:(1/36): ceph-19.0.0-3525.g27d37168.el9.x86_64.r 28 kB/s | 6.6 kB 00:00 2024-05-08T18:02:45.449 INFO:teuthology.orchestra.run.smithi050.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 633 kB/s | 253 kB 00:00 2024-05-08T18:02:45.725 INFO:teuthology.orchestra.run.smithi050.stdout:(6/36): ceph-osd-19.0.0-3525.g27d37168.el9.x86_ 21 MB/s | 17 MB 00:00 2024-05-08T18:02:45.950 INFO:teuthology.orchestra.run.smithi050.stdout:(7/36): python3-cryptography-36.0.1-4.el9.x86_6 2.5 MB/s | 1.2 MB 00:00 2024-05-08T18:02:46.026 INFO:teuthology.orchestra.run.smithi050.stdout:(8/36): python3-ply-3.11-14.el9.noarch.rpm 355 kB/s | 106 kB 00:00 2024-05-08T18:02:46.051 INFO:teuthology.orchestra.run.smithi050.stdout:(9/36): python3-pycparser-2.20-6.el9.noarch.rpm 1.3 MB/s | 135 kB 00:00 2024-05-08T18:02:46.056 INFO:teuthology.orchestra.run.smithi046.stdout:(2/36): ceph-mds-19.0.0-3525.g27d37168.el9.x86_ 2.1 MB/s | 2.4 MB 00:01 2024-05-08T18:02:46.176 INFO:teuthology.orchestra.run.smithi050.stdout:(10/36): python3-requests-2.25.1-8.el9.noarch.r 832 kB/s | 125 kB 00:00 2024-05-08T18:02:46.251 INFO:teuthology.orchestra.run.smithi050.stdout:(11/36): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-05-08T18:02:46.314 INFO:teuthology.orchestra.run.smithi046.stdout:(3/36): ceph-mon-19.0.0-3525.g27d37168.el9.x86_ 4.0 MB/s | 4.7 MB 00:01 2024-05-08T18:02:46.326 INFO:teuthology.orchestra.run.smithi050.stdout:(12/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-05-08T18:02:46.373 INFO:teuthology.orchestra.run.smithi046.stdout:(4/36): ceph-mgr-modules-core-19.0.0-3525.g27d3 4.6 MB/s | 274 kB 00:00 2024-05-08T18:02:46.377 INFO:teuthology.orchestra.run.smithi050.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 697 kB/s | 35 kB 00:00 2024-05-08T18:02:46.436 INFO:teuthology.orchestra.run.smithi050.stdout:(14/36): python3-pytz-2021.1-5.el9.noarch.rpm 874 kB/s | 51 kB 00:00 2024-05-08T18:02:46.511 INFO:teuthology.orchestra.run.smithi050.stdout:(15/36): python3-urllib3-1.26.5-5.el9.noarch.rp 467 kB/s | 215 kB 00:00 2024-05-08T18:02:46.536 INFO:teuthology.orchestra.run.smithi050.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 416 kB/s | 42 kB 00:00 2024-05-08T18:02:46.657 INFO:teuthology.orchestra.run.smithi046.stdout:(5/36): ceph-mgr-19.0.0-3525.g27d37168.el9.x86_ 884 kB/s | 1.5 MB 00:01 2024-05-08T18:02:46.803 INFO:teuthology.orchestra.run.smithi050.stdout:(17/36): luarocks-3.9.2-1.el9.noarch.rpm 566 kB/s | 151 kB 00:00 2024-05-08T18:02:46.828 INFO:teuthology.orchestra.run.smithi050.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 70 kB/s | 22 kB 00:00 2024-05-08T18:02:46.887 INFO:teuthology.orchestra.run.smithi050.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 519 kB/s | 43 kB 00:00 2024-05-08T18:02:47.071 INFO:teuthology.orchestra.run.smithi050.stdout:(20/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.9 MB/s | 358 kB 00:00 2024-05-08T18:02:47.096 INFO:teuthology.orchestra.run.smithi050.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 642 kB/s | 172 kB 00:00 2024-05-08T18:02:47.121 INFO:teuthology.orchestra.run.smithi050.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 213 kB/s | 11 kB 00:00 2024-05-08T18:02:47.196 INFO:teuthology.orchestra.run.smithi050.stdout:(23/36): ceph-mon-19.0.0-3525.g27d37168.el9.x86 2.0 MB/s | 4.7 MB 00:02 2024-05-08T18:02:47.222 INFO:teuthology.orchestra.run.smithi050.stdout:(24/36): python3-jaraco-classes-3.2.1-5.el9.noa 141 kB/s | 18 kB 00:00 2024-05-08T18:02:47.223 INFO:teuthology.orchestra.run.smithi046.stdout:(6/36): ceph-osd-19.0.0-3525.g27d37168.el9.x86_ 14 MB/s | 17 MB 00:01 2024-05-08T18:02:47.247 INFO:teuthology.orchestra.run.smithi050.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 185 kB/s | 23 kB 00:00 2024-05-08T18:02:47.289 INFO:teuthology.orchestra.run.smithi050.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 474 kB/s | 20 kB 00:00 2024-05-08T18:02:47.314 INFO:teuthology.orchestra.run.smithi050.stdout:(27/36): python3-jaraco-functools-3.5.0-2.el9.n 166 kB/s | 19 kB 00:00 2024-05-08T18:02:47.431 INFO:teuthology.orchestra.run.smithi050.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 556 kB/s | 79 kB 00:00 2024-05-08T18:02:47.465 INFO:teuthology.orchestra.run.smithi050.stdout:(29/36): python3-logutils-0.3.5-21.el9.noarch.r 213 kB/s | 46 kB 00:00 2024-05-08T18:02:47.466 INFO:teuthology.orchestra.run.smithi046.stdout:(7/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 232 kB/s | 253 kB 00:01 2024-05-08T18:02:47.503 INFO:teuthology.orchestra.run.smithi050.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-05-08T18:02:47.532 INFO:teuthology.orchestra.run.smithi050.stdout:(31/36): python3-portend-3.1.0-2.el9.noarch.rpm 163 kB/s | 16 kB 00:00 2024-05-08T18:02:47.582 INFO:teuthology.orchestra.run.smithi050.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 768 kB/s | 90 kB 00:00 2024-05-08T18:02:47.608 INFO:teuthology.orchestra.run.smithi050.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 345 kB/s | 36 kB 00:00 2024-05-08T18:02:47.649 INFO:teuthology.orchestra.run.smithi050.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-05-08T18:02:47.691 INFO:teuthology.orchestra.run.smithi050.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 239 kB/s | 20 kB 00:00 2024-05-08T18:02:48.034 INFO:teuthology.orchestra.run.smithi046.stdout:(8/36): python3-cryptography-36.0.1-4.el9.x86_6 913 kB/s | 1.2 MB 00:01 2024-05-08T18:02:48.092 INFO:teuthology.orchestra.run.smithi050.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 839 kB/s | 427 kB 00:00 2024-05-08T18:02:48.093 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:02:48.093 INFO:teuthology.orchestra.run.smithi050.stdout:Total 7.4 MB/s | 30 MB 00:04 2024-05-08T18:02:48.133 INFO:teuthology.orchestra.run.smithi046.stdout:(9/36): python3-ply-3.11-14.el9.noarch.rpm 117 kB/s | 106 kB 00:00 2024-05-08T18:02:48.244 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T18:02:48.286 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T18:02:48.286 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T18:02:48.483 INFO:teuthology.orchestra.run.smithi046.stdout:(10/36): python3-pycparser-2.20-6.el9.noarch.rp 137 kB/s | 135 kB 00:00 2024-05-08T18:02:48.633 INFO:teuthology.orchestra.run.smithi046.stdout:(11/36): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-05-08T18:02:48.700 INFO:teuthology.orchestra.run.smithi046.stdout:(12/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-05-08T18:02:48.742 INFO:teuthology.orchestra.run.smithi046.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 838 kB/s | 35 kB 00:00 2024-05-08T18:02:48.754 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T18:02:48.755 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T18:02:48.783 INFO:teuthology.orchestra.run.smithi046.stdout:(14/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-05-08T18:02:48.825 INFO:teuthology.orchestra.run.smithi046.stdout:(15/36): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-05-08T18:02:48.884 INFO:teuthology.orchestra.run.smithi046.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 383 kB/s | 22 kB 00:00 2024-05-08T18:02:48.951 INFO:teuthology.orchestra.run.smithi046.stdout:(17/36): python3-requests-2.25.1-8.el9.noarch.r 136 kB/s | 125 kB 00:00 2024-05-08T18:02:49.085 INFO:teuthology.orchestra.run.smithi046.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 331 kB/s | 43 kB 00:00 2024-05-08T18:02:49.110 INFO:teuthology.orchestra.run.smithi046.stdout:(19/36): python3-urllib3-1.26.5-5.el9.noarch.rp 220 kB/s | 215 kB 00:00 2024-05-08T18:02:49.135 INFO:teuthology.orchestra.run.smithi046.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 603 kB/s | 151 kB 00:00 2024-05-08T18:02:49.252 INFO:teuthology.orchestra.run.smithi046.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-05-08T18:02:49.277 INFO:teuthology.orchestra.run.smithi046.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 75 kB/s | 11 kB 00:00 2024-05-08T18:02:49.319 INFO:teuthology.orchestra.run.smithi046.stdout:(23/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-05-08T18:02:49.345 INFO:teuthology.orchestra.run.smithi046.stdout:(24/36): python3-jaraco-classes-3.2.1-5.el9.noa 192 kB/s | 18 kB 00:00 2024-05-08T18:02:49.370 INFO:teuthology.orchestra.run.smithi046.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 253 kB/s | 23 kB 00:00 2024-05-08T18:02:49.428 INFO:teuthology.orchestra.run.smithi046.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 179 kB/s | 19 kB 00:00 2024-05-08T18:02:49.512 INFO:teuthology.orchestra.run.smithi046.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 118 kB/s | 20 kB 00:00 2024-05-08T18:02:49.547 INFO:teuthology.orchestra.run.smithi046.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 788 kB/s | 79 kB 00:00 2024-05-08T18:02:49.554 INFO:teuthology.orchestra.run.smithi046.stdout:(29/36): python3-logutils-0.3.5-21.el9.noarch.r 251 kB/s | 46 kB 00:00 2024-05-08T18:02:49.630 INFO:teuthology.orchestra.run.smithi046.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-05-08T18:02:49.655 INFO:teuthology.orchestra.run.smithi046.stdout:(31/36): python3-portend-3.1.0-2.el9.noarch.rpm 131 kB/s | 16 kB 00:00 2024-05-08T18:02:49.680 INFO:teuthology.orchestra.run.smithi046.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 718 kB/s | 90 kB 00:00 2024-05-08T18:02:49.705 INFO:teuthology.orchestra.run.smithi046.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 477 kB/s | 36 kB 00:00 2024-05-08T18:02:49.964 INFO:teuthology.orchestra.run.smithi046.stdout:(34/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.5 MB/s | 427 kB 00:00 2024-05-08T18:02:49.997 INFO:teuthology.orchestra.run.smithi046.stdout:(35/36): python3-webob-1.8.7-6.el9.noarch.rpm 725 kB/s | 230 kB 00:00 2024-05-08T18:02:50.014 INFO:teuthology.orchestra.run.smithi046.stdout:(36/36): python3-zc-lockfile-2.0-10.el9.noarch. 65 kB/s | 20 kB 00:00 2024-05-08T18:02:50.019 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:02:50.020 INFO:teuthology.orchestra.run.smithi046.stdout:Total 5.1 MB/s | 30 MB 00:05 2024-05-08T18:02:50.131 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T18:02:50.168 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T18:02:50.186 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-08T18:02:50.212 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T18:02:50.213 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T18:02:50.219 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-08T18:02:50.247 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-08T18:02:50.321 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-08T18:02:50.363 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-05-08T18:02:50.395 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-08T18:02:50.492 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-08T18:02:50.668 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T18:02:50.668 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T18:02:50.701 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-08T18:02:50.748 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-08T18:02:50.783 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-08T18:02:50.813 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-08T18:02:50.847 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-08T18:02:50.881 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-08T18:02:50.922 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-08T18:02:50.966 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-08T18:02:50.999 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-08T18:02:51.029 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-08T18:02:51.106 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-08T18:02:51.257 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-08T18:02:51.349 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-08T18:02:51.408 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-08T18:02:51.446 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-08T18:02:51.504 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-08T18:02:51.569 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-08T18:02:51.764 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-08T18:02:51.808 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-08T18:02:51.890 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-08T18:02:51.938 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T18:02:51.979 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-08T18:02:52.000 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-08T18:02:52.031 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-08T18:02:52.033 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-08T18:02:52.069 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-08T18:02:52.147 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-08T18:02:52.163 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-08T18:02:52.194 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-05-08T18:02:52.227 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-08T18:02:52.324 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-08T18:02:52.327 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3525.g27d37168.el9. 31/36 2024-05-08T18:02:52.359 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-2:19.0.0-3525.g27d37168.el9.x86_64 32/36 2024-05-08T18:02:52.387 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3525.g27d37168.el9.x86_64 32/36 2024-05-08T18:02:52.387 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T18:02:52.387 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T18:02:52.387 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T18:02:52.387 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T18:02:52.387 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:52.399 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-08T18:02:52.447 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-08T18:02:52.481 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-08T18:02:52.511 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-08T18:02:52.545 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-08T18:02:52.579 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-08T18:02:52.620 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-08T18:02:52.663 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-08T18:02:52.697 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-08T18:02:52.735 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-08T18:02:52.813 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-08T18:02:52.966 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-08T18:02:53.047 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-08T18:02:53.097 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-08T18:02:53.143 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-08T18:02:53.201 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-08T18:02:53.275 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-08T18:02:53.479 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-08T18:02:53.530 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-08T18:02:53.612 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-08T18:02:53.709 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-08T18:02:53.749 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-08T18:02:53.888 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-08T18:02:53.910 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-osd-2:19.0.0-3525.g27d37168.el9.x86_64 33/36 2024-05-08T18:02:53.938 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-osd-2:19.0.0-3525.g27d37168.el9.x86_64 33/36 2024-05-08T18:02:53.939 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T18:02:53.939 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T18:02:53.939 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T18:02:53.939 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T18:02:53.939 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:54.058 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3525.g27d37168.el9. 31/36 2024-05-08T18:02:54.080 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-2:19.0.0-3525.g27d37168.el9.x86_64 32/36 2024-05-08T18:02:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3525.g27d37168.el9.x86_64 32/36 2024-05-08T18:02:54.109 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T18:02:54.109 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T18:02:54.109 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T18:02:54.109 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T18:02:54.109 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:54.425 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mon-2:19.0.0-3525.g27d37168.el9.x86_64 34/36 2024-05-08T18:02:54.450 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mon-2:19.0.0-3525.g27d37168.el9.x86_64 34/36 2024-05-08T18:02:54.450 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T18:02:54.451 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T18:02:54.451 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T18:02:54.451 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T18:02:54.451 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:54.695 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mds-2:19.0.0-3525.g27d37168.el9.x86_64 35/36 2024-05-08T18:02:54.720 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mds-2:19.0.0-3525.g27d37168.el9.x86_64 35/36 2024-05-08T18:02:54.720 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T18:02:54.720 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T18:02:54.720 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T18:02:54.720 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T18:02:54.720 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:54.786 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-2:19.0.0-3525.g27d37168.el9.x86_64 36/36 2024-05-08T18:02:55.632 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-osd-2:19.0.0-3525.g27d37168.el9.x86_64 33/36 2024-05-08T18:02:55.661 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-osd-2:19.0.0-3525.g27d37168.el9.x86_64 33/36 2024-05-08T18:02:55.661 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T18:02:55.661 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T18:02:55.661 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T18:02:55.661 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T18:02:55.661 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:56.124 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mon-2:19.0.0-3525.g27d37168.el9.x86_64 34/36 2024-05-08T18:02:56.150 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mon-2:19.0.0-3525.g27d37168.el9.x86_64 34/36 2024-05-08T18:02:56.150 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T18:02:56.150 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T18:02:56.150 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T18:02:56.150 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T18:02:56.150 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:56.391 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mds-2:19.0.0-3525.g27d37168.el9.x86_64 35/36 2024-05-08T18:02:56.417 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mds-2:19.0.0-3525.g27d37168.el9.x86_64 35/36 2024-05-08T18:02:56.417 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T18:02:56.417 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T18:02:56.417 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T18:02:56.417 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T18:02:56.417 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:56.484 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-2:19.0.0-3525.g27d37168.el9.x86_64 36/36 2024-05-08T18:02:56.493 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-2:19.0.0-3525.g27d37168.el9.x86_64 36/36 2024-05-08T18:02:56.493 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-2:19.0.0-3525.g27d37168.el9.x86_64 1/36 2024-05-08T18:02:56.493 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mds-2:19.0.0-3525.g27d37168.el9.x86_64 2/36 2024-05-08T18:02:56.494 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-2:19.0.0-3525.g27d37168.el9.x86_64 3/36 2024-05-08T18:02:56.494 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mon-2:19.0.0-3525.g27d37168.el9.x86_64 4/36 2024-05-08T18:02:56.494 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-osd-2:19.0.0-3525.g27d37168.el9.x86_64 5/36 2024-05-08T18:02:56.494 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3525.g27d37168.el9. 6/36 2024-05-08T18:02:56.494 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-08T18:02:56.494 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-08T18:02:56.494 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-08T18:02:56.494 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-08T18:02:56.494 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-08T18:02:56.494 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-08T18:02:56.494 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-08T18:02:56.494 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-08T18:02:56.495 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-08T18:02:56.495 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-08T18:02:56.495 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-08T18:02:56.495 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-08T18:02:56.495 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-05-08T18:02:56.495 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-08T18:02:56.495 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-08T18:02:56.495 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-08T18:02:56.495 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-08T18:02:56.495 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-08T18:02:56.495 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-08T18:02:56.495 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-08T18:02:56.495 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-08T18:02:56.496 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-08T18:02:56.496 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-08T18:02:56.497 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-08T18:02:56.497 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-08T18:02:56.497 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-08T18:02:56.497 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-08T18:02:56.497 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-08T18:02:56.497 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-08T18:02:57.547 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-08T18:02:57.547 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:57.547 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T18:02:57.547 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:57.547 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:57.547 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:57.547 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:02:57.548 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:57.548 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:57.548 INFO:teuthology.orchestra.run.smithi050.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T18:02:57.548 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T18:02:57.548 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T18:02:57.548 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T18:02:57.548 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T18:02:57.548 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T18:02:57.548 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T18:02:57.548 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T18:02:57.548 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T18:02:57.548 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T18:02:57.548 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T18:02:57.548 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T18:02:57.548 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T18:02:57.549 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T18:02:57.549 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T18:02:57.549 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T18:02:57.549 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T18:02:57.549 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T18:02:57.549 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T18:02:57.549 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T18:02:57.549 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T18:02:57.549 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T18:02:57.549 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T18:02:57.549 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T18:02:57.549 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T18:02:57.549 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T18:02:57.549 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T18:02:57.550 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T18:02:57.550 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T18:02:57.550 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T18:02:57.550 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:57.550 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:02:57.916 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-base 2024-05-08T18:02:58.314 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-2:19.0.0-3525.g27d37168.el9.x86_64 36/36 2024-05-08T18:02:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-2:19.0.0-3525.g27d37168.el9.x86_64 1/36 2024-05-08T18:02:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mds-2:19.0.0-3525.g27d37168.el9.x86_64 2/36 2024-05-08T18:02:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-2:19.0.0-3525.g27d37168.el9.x86_64 3/36 2024-05-08T18:02:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mon-2:19.0.0-3525.g27d37168.el9.x86_64 4/36 2024-05-08T18:02:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-osd-2:19.0.0-3525.g27d37168.el9.x86_64 5/36 2024-05-08T18:02:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3525.g27d37168.el9. 6/36 2024-05-08T18:02:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-08T18:02:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-08T18:02:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-08T18:02:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-08T18:02:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-08T18:02:58.315 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-08T18:02:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-08T18:02:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-08T18:02:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-08T18:02:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-08T18:02:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-08T18:02:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-08T18:02:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-05-08T18:02:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-08T18:02:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-08T18:02:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-08T18:02:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-08T18:02:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-08T18:02:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-08T18:02:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-08T18:02:58.316 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-08T18:02:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-08T18:02:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-08T18:02:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-08T18:02:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-08T18:02:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-08T18:02:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-08T18:02:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-08T18:02:58.317 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-08T18:02:58.509 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:00:58 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:02:58.593 INFO:teuthology.orchestra.run.smithi050.stdout:Package ceph-base-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:02:58.636 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:02:58.637 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-05-08T18:02:58.638 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:02:58.721 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install cephadm 2024-05-08T18:02:59.309 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:00:59 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:02:59.312 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-08T18:02:59.312 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:59.312 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T18:02:59.312 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:59.312 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:59.312 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:59.312 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:02:59.312 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:59.312 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:02:59.312 INFO:teuthology.orchestra.run.smithi046.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T18:02:59.312 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T18:02:59.312 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T18:02:59.312 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T18:02:59.312 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T18:02:59.313 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T18:02:59.313 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T18:02:59.313 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T18:02:59.313 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T18:02:59.313 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T18:02:59.313 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T18:02:59.313 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T18:02:59.313 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T18:02:59.313 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T18:02:59.313 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T18:02:59.313 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T18:02:59.313 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T18:02:59.313 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T18:02:59.313 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T18:02:59.314 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T18:02:59.314 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T18:02:59.314 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T18:02:59.314 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T18:02:59.314 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T18:02:59.314 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T18:02:59.314 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T18:02:59.314 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T18:02:59.314 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T18:02:59.314 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T18:02:59.314 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T18:02:59.314 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:02:59.314 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:02:59.436 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:02:59.437 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:02:59.437 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-05-08T18:02:59.437 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:02:59.437 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T18:02:59.437 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 763 k 2024-05-08T18:02:59.437 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:59.437 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T18:02:59.437 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:02:59.438 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2024-05-08T18:02:59.438 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:02:59.438 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 763 k 2024-05-08T18:02:59.438 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 770 k 2024-05-08T18:02:59.438 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T18:02:59.673 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-base 2024-05-08T18:03:00.152 INFO:teuthology.orchestra.run.smithi050.stdout:cephadm-19.0.0-3525.g27d37168.el9.noarch.rpm 1.0 MB/s | 763 kB 00:00 2024-05-08T18:03:00.153 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:00.153 INFO:teuthology.orchestra.run.smithi050.stdout:Total 1.0 MB/s | 763 kB 00:00 2024-05-08T18:03:00.153 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T18:03:00.159 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T18:03:00.159 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T18:03:00.165 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T18:03:00.166 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T18:03:00.243 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T18:03:00.260 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:09 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:00.344 INFO:teuthology.orchestra.run.smithi046.stdout:Package ceph-base-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:00.389 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:00.390 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-05-08T18:03:00.391 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:00.466 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install cephadm 2024-05-08T18:03:00.671 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephadm-2:19.0.0-3525.g27d37168.el9.noarch 1/1 2024-05-08T18:03:00.751 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephadm-2:19.0.0-3525.g27d37168.el9.noarch 1/1 2024-05-08T18:03:01.054 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:10 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:01.151 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephadm-2:19.0.0-3525.g27d37168.el9.noarch 1/1 2024-05-08T18:03:01.180 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:01.181 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:01.181 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-05-08T18:03:01.182 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:01.182 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T18:03:01.182 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 763 k 2024-05-08T18:03:01.182 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:01.182 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T18:03:01.182 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:01.182 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-05-08T18:03:01.183 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:01.183 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 763 k 2024-05-08T18:03:01.183 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 770 k 2024-05-08T18:03:01.183 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T18:03:01.394 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephadm-2:19.0.0-3525.g27d37168.el9.noarch 1/1 2024-05-08T18:03:01.394 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:01.394 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T18:03:01.394 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:03:01.394 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:01.394 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:01.607 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-immutable-object-cache 2024-05-08T18:03:01.666 INFO:teuthology.orchestra.run.smithi046.stdout:cephadm-19.0.0-3525.g27d37168.el9.noarch.rpm 1.5 MB/s | 763 kB 00:00 2024-05-08T18:03:01.666 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:01.666 INFO:teuthology.orchestra.run.smithi046.stdout:Total 1.5 MB/s | 763 kB 00:00 2024-05-08T18:03:01.666 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T18:03:01.672 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T18:03:01.672 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T18:03:01.678 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T18:03:01.678 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T18:03:01.756 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T18:03:02.197 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:02 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:02.233 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:19.0.0-3525.g27d37168.el9.noarch 1/1 2024-05-08T18:03:02.327 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:02.328 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:02.328 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repo Size 2024-05-08T18:03:02.328 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:02.328 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T18:03:02.328 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3525.g27d37168.el9 ceph 145 k 2024-05-08T18:03:02.328 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:02.328 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T18:03:02.328 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:02.328 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2024-05-08T18:03:02.328 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:02.329 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 145 k 2024-05-08T18:03:02.329 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 435 k 2024-05-08T18:03:02.329 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T18:03:02.330 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephadm-2:19.0.0-3525.g27d37168.el9.noarch 1/1 2024-05-08T18:03:02.553 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-immutable-object-cache-19.0.0-3525.g27d371 648 kB/s | 145 kB 00:00 2024-05-08T18:03:02.554 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:02.554 INFO:teuthology.orchestra.run.smithi050.stdout:Total 643 kB/s | 145 kB 00:00 2024-05-08T18:03:02.554 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T18:03:02.562 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T18:03:02.562 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T18:03:02.622 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T18:03:02.622 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T18:03:02.718 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:19.0.0-3525.g27d37168.el9.noarch 1/1 2024-05-08T18:03:02.791 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T18:03:02.854 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3525.g27d37168. 1/1 2024-05-08T18:03:02.881 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3525.g27d37168. 1/1 2024-05-08T18:03:02.881 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T18:03:02.881 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T18:03:02.881 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:03.086 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephadm-2:19.0.0-3525.g27d37168.el9.noarch 1/1 2024-05-08T18:03:03.086 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:03.086 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T18:03:03.086 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:03:03.086 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:03.087 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:03.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-immutable-object-cache 2024-05-08T18:03:03.743 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3525.g27d37168. 1/1 2024-05-08T18:03:03.744 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:03.744 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T18:03:03.744 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:03:03.744 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:03.744 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:03.867 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:12 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:03.994 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:03.996 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:03.997 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repo Size 2024-05-08T18:03:03.997 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:03.997 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T18:03:03.997 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3525.g27d37168.el9 ceph 145 k 2024-05-08T18:03:03.997 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:03.997 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T18:03:03.997 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:03.997 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-05-08T18:03:03.997 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:03.998 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 145 k 2024-05-08T18:03:03.998 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 435 k 2024-05-08T18:03:03.999 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T18:03:04.000 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-mgr 2024-05-08T18:03:04.363 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-immutable-object-cache-19.0.0-3525.g27d371 399 kB/s | 145 kB 00:00 2024-05-08T18:03:04.364 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:04.364 INFO:teuthology.orchestra.run.smithi046.stdout:Total 397 kB/s | 145 kB 00:00 2024-05-08T18:03:04.364 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T18:03:04.373 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T18:03:04.373 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T18:03:04.433 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T18:03:04.433 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T18:03:04.591 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:04 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:04.593 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T18:03:04.665 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3525.g27d37168. 1/1 2024-05-08T18:03:04.675 INFO:teuthology.orchestra.run.smithi050.stdout:Package ceph-mgr-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3525.g27d37168. 1/1 2024-05-08T18:03:04.691 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T18:03:04.691 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T18:03:04.691 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:04.719 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:04.720 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-05-08T18:03:04.720 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:04.804 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-mgr-dashboard 2024-05-08T18:03:05.391 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:05 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:05.518 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:05.519 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:05.519 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-05-08T18:03:05.519 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:05.519 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T18:03:05.519 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 3.5 M 2024-05-08T18:03:05.519 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-05-08T18:03:05.519 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T18:03:05.520 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 34 k 2024-05-08T18:03:05.520 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 16 k 2024-05-08T18:03:05.520 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T18:03:05.520 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T18:03:05.520 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T18:03:05.520 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T18:03:05.520 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T18:03:05.520 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T18:03:05.520 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T18:03:05.520 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T18:03:05.520 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:05.520 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T18:03:05.520 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:05.520 INFO:teuthology.orchestra.run.smithi050.stdout:Install 12 Packages 2024-05-08T18:03:05.521 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:05.521 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 8.6 M 2024-05-08T18:03:05.521 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 104 M 2024-05-08T18:03:05.521 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T18:03:05.636 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3525.g27d37168. 1/1 2024-05-08T18:03:05.636 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:05.636 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T18:03:05.637 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:03:05.637 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:05.637 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:05.893 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr 2024-05-08T18:03:06.317 INFO:teuthology.orchestra.run.smithi050.stdout:(1/12): ceph-prometheus-alerts-19.0.0-3525.g27d 69 kB/s | 16 kB 00:00 2024-05-08T18:03:06.400 INFO:teuthology.orchestra.run.smithi050.stdout:(2/12): ceph-grafana-dashboards-19.0.0-3525.g27 106 kB/s | 34 kB 00:00 2024-05-08T18:03:06.485 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:15 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:06.568 INFO:teuthology.orchestra.run.smithi046.stdout:Package ceph-mgr-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:06.613 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:06.614 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-05-08T18:03:06.614 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:06.687 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr-dashboard 2024-05-08T18:03:06.734 INFO:teuthology.orchestra.run.smithi050.stdout:(3/12): ceph-mgr-dashboard-19.0.0-3525.g27d3716 5.4 MB/s | 3.5 MB 00:00 2024-05-08T18:03:07.093 INFO:teuthology.orchestra.run.smithi050.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 389 kB/s | 269 kB 00:00 2024-05-08T18:03:07.151 INFO:teuthology.orchestra.run.smithi050.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 2.0 MB/s | 863 kB 00:00 2024-05-08T18:03:07.243 INFO:teuthology.orchestra.run.smithi050.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 211 kB/s | 19 kB 00:00 2024-05-08T18:03:07.278 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:16 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:07.285 INFO:teuthology.orchestra.run.smithi050.stdout:(7/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:00 2024-05-08T18:03:07.406 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:07.407 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:07.407 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-05-08T18:03:07.407 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:07.407 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T18:03:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 3.5 M 2024-05-08T18:03:07.408 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-05-08T18:03:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T18:03:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 34 k 2024-05-08T18:03:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 16 k 2024-05-08T18:03:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T18:03:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T18:03:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T18:03:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T18:03:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T18:03:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T18:03:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T18:03:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T18:03:07.408 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:07.408 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T18:03:07.409 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:07.409 INFO:teuthology.orchestra.run.smithi046.stdout:Install 12 Packages 2024-05-08T18:03:07.409 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:07.409 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 8.6 M 2024-05-08T18:03:07.409 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 104 M 2024-05-08T18:03:07.409 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T18:03:07.436 INFO:teuthology.orchestra.run.smithi050.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.6 MB/s | 548 kB 00:00 2024-05-08T18:03:07.511 INFO:teuthology.orchestra.run.smithi050.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 410 kB/s | 31 kB 00:00 2024-05-08T18:03:07.611 INFO:teuthology.orchestra.run.smithi050.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 443 kB/s | 144 kB 00:00 2024-05-08T18:03:07.686 INFO:teuthology.orchestra.run.smithi050.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.1 MB/s | 188 kB 00:00 2024-05-08T18:03:08.045 INFO:teuthology.orchestra.run.smithi050.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 2.5 MB/s | 2.0 MB 00:00 2024-05-08T18:03:08.046 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:08.047 INFO:teuthology.orchestra.run.smithi050.stdout:Total 3.4 MB/s | 8.6 MB 00:02 2024-05-08T18:03:08.101 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T18:03:08.121 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T18:03:08.121 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T18:03:08.263 INFO:teuthology.orchestra.run.smithi046.stdout:(1/12): ceph-grafana-dashboards-19.0.0-3525.g27 77 kB/s | 34 kB 00:00 2024-05-08T18:03:08.292 INFO:teuthology.orchestra.run.smithi046.stdout:(2/12): ceph-prometheus-alerts-19.0.0-3525.g27d 35 kB/s | 16 kB 00:00 2024-05-08T18:03:08.323 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T18:03:08.323 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T18:03:08.442 INFO:teuthology.orchestra.run.smithi046.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.8 MB/s | 269 kB 00:00 2024-05-08T18:03:08.476 INFO:teuthology.orchestra.run.smithi046.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-05-08T18:03:08.885 INFO:teuthology.orchestra.run.smithi046.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.3 MB/s | 548 kB 00:00 2024-05-08T18:03:08.943 INFO:teuthology.orchestra.run.smithi046.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 334 kB/s | 19 kB 00:00 2024-05-08T18:03:08.969 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T18:03:08.994 INFO:teuthology.orchestra.run.smithi046.stdout:(7/12): protobuf-compiler-3.14.0-13.el9.x86_64. 1.5 MB/s | 863 kB 00:00 2024-05-08T18:03:09.047 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-08T18:03:09.082 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-08T18:03:09.127 INFO:teuthology.orchestra.run.smithi046.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 11 MB/s | 2.0 MB 00:00 2024-05-08T18:03:09.129 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-08T18:03:09.158 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-08T18:03:09.186 INFO:teuthology.orchestra.run.smithi046.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 530 kB/s | 31 kB 00:00 2024-05-08T18:03:09.219 INFO:teuthology.orchestra.run.smithi046.stdout:(10/12): python3-grpcio-tools-1.46.7-10.el9.x86 641 kB/s | 144 kB 00:00 2024-05-08T18:03:09.244 INFO:teuthology.orchestra.run.smithi046.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.2 MB/s | 188 kB 00:00 2024-05-08T18:03:09.287 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-08T18:03:09.424 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-08T18:03:09.532 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-08T18:03:09.582 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-08T18:03:09.609 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-08T18:03:09.641 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3525.g27d37168.el9 10/12 2024-05-08T18:03:09.953 INFO:teuthology.orchestra.run.smithi046.stdout:(12/12): ceph-mgr-dashboard-19.0.0-3525.g27d371 1.7 MB/s | 3.5 MB 00:02 2024-05-08T18:03:09.955 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:09.955 INFO:teuthology.orchestra.run.smithi046.stdout:Total 3.4 MB/s | 8.6 MB 00:02 2024-05-08T18:03:10.010 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T18:03:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T18:03:10.030 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T18:03:10.220 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T18:03:10.220 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T18:03:10.745 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3525.g27d37168.el 11/12 2024-05-08T18:03:10.786 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3525.g27d37168.el9.noa 12/12 2024-05-08T18:03:10.814 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T18:03:10.888 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-08T18:03:10.923 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-08T18:03:10.971 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-08T18:03:11.000 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-08T18:03:11.113 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-08T18:03:11.257 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-08T18:03:11.348 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-08T18:03:11.398 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-08T18:03:11.424 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-08T18:03:11.459 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3525.g27d37168.el9 10/12 2024-05-08T18:03:12.123 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3525.g27d37168.el9.noa 12/12 2024-05-08T18:03:12.123 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3525.g27d37168.el 1/12 2024-05-08T18:03:12.123 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3525.g27d37168.el9.noa 2/12 2024-05-08T18:03:12.124 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3525.g27d37168.el9 3/12 2024-05-08T18:03:12.124 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-08T18:03:12.124 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-08T18:03:12.124 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-08T18:03:12.124 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-08T18:03:12.124 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-08T18:03:12.124 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-08T18:03:12.124 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-08T18:03:12.124 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-08T18:03:12.435 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-08T18:03:12.435 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:12.435 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T18:03:12.435 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T18:03:12.435 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:03:12.435 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:03:12.435 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:03:12.435 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T18:03:12.436 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T18:03:12.436 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T18:03:12.436 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T18:03:12.436 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T18:03:12.436 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T18:03:12.436 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T18:03:12.436 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T18:03:12.436 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:12.436 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:12.561 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3525.g27d37168.el 11/12 2024-05-08T18:03:12.600 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3525.g27d37168.el9.noa 12/12 2024-05-08T18:03:12.653 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-08T18:03:13.254 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:13 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:13.382 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:13.383 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================= 2024-05-08T18:03:13.383 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-05-08T18:03:13.383 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================= 2024-05-08T18:03:13.383 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T18:03:13.383 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 7.4 M 2024-05-08T18:03:13.383 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-05-08T18:03:13.383 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T18:03:13.384 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T18:03:13.384 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T18:03:13.384 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T18:03:13.384 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T18:03:13.384 INFO:teuthology.orchestra.run.smithi050.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T18:03:13.384 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T18:03:13.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T18:03:13.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T18:03:13.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T18:03:13.384 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T18:03:13.384 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:13.384 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T18:03:13.384 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================= 2024-05-08T18:03:13.384 INFO:teuthology.orchestra.run.smithi050.stdout:Install 12 Packages 2024-05-08T18:03:13.385 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:13.385 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 38 M 2024-05-08T18:03:13.385 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 209 M 2024-05-08T18:03:13.385 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T18:03:13.907 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3525.g27d37168.el9.noa 12/12 2024-05-08T18:03:13.907 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3525.g27d37168.el 1/12 2024-05-08T18:03:13.907 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3525.g27d37168.el9.noa 2/12 2024-05-08T18:03:13.907 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3525.g27d37168.el9 3/12 2024-05-08T18:03:13.907 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-08T18:03:13.907 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-08T18:03:13.907 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-08T18:03:13.907 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-08T18:03:13.907 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-08T18:03:13.907 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-08T18:03:13.907 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-08T18:03:13.908 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-08T18:03:14.189 INFO:teuthology.orchestra.run.smithi050.stdout:(1/12): ceph-mgr-diskprediction-local-19.0.0-35 12 MB/s | 7.4 MB 00:00 2024-05-08T18:03:14.227 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-08T18:03:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:14.228 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T18:03:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T18:03:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:03:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:03:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:03:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T18:03:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T18:03:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T18:03:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T18:03:14.228 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T18:03:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T18:03:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T18:03:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T18:03:14.229 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:14.229 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:14.434 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-08T18:03:14.472 INFO:teuthology.orchestra.run.smithi050.stdout:(2/12): libquadmath-11.4.1-3.el9.x86_64.rpm 220 kB/s | 195 kB 00:00 2024-05-08T18:03:14.639 INFO:teuthology.orchestra.run.smithi050.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 762 kB/s | 803 kB 00:01 2024-05-08T18:03:14.806 INFO:teuthology.orchestra.run.smithi050.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 53 kB/s | 33 kB 00:00 2024-05-08T18:03:15.028 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:24 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:15.157 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:15.158 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-05-08T18:03:15.158 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-05-08T18:03:15.158 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-05-08T18:03:15.158 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T18:03:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 7.4 M 2024-05-08T18:03:15.159 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-05-08T18:03:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T18:03:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T18:03:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T18:03:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T18:03:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T18:03:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T18:03:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T18:03:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T18:03:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T18:03:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T18:03:15.159 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T18:03:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:15.160 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T18:03:15.160 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-05-08T18:03:15.160 INFO:teuthology.orchestra.run.smithi046.stdout:Install 12 Packages 2024-05-08T18:03:15.160 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:15.160 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 38 M 2024-05-08T18:03:15.160 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 209 M 2024-05-08T18:03:15.161 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T18:03:15.181 INFO:teuthology.orchestra.run.smithi050.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 34 kB/s | 18 kB 00:00 2024-05-08T18:03:15.406 INFO:teuthology.orchestra.run.smithi050.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 59 kB/s | 35 kB 00:00 2024-05-08T18:03:15.496 INFO:teuthology.orchestra.run.smithi046.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-05-08T18:03:15.563 INFO:teuthology.orchestra.run.smithi046.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 491 kB/s | 33 kB 00:00 2024-05-08T18:03:15.596 INFO:teuthology.orchestra.run.smithi046.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.2 MB/s | 803 kB 00:00 2024-05-08T18:03:15.631 INFO:teuthology.orchestra.run.smithi050.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.6 MB/s | 3.0 MB 00:01 2024-05-08T18:03:15.688 INFO:teuthology.orchestra.run.smithi046.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 198 kB/s | 18 kB 00:00 2024-05-08T18:03:15.722 INFO:teuthology.orchestra.run.smithi046.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 35 kB 00:00 2024-05-08T18:03:15.805 INFO:teuthology.orchestra.run.smithi046.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-05-08T18:03:15.847 INFO:teuthology.orchestra.run.smithi046.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 5.7 MB/s | 244 kB 00:00 2024-05-08T18:03:15.923 INFO:teuthology.orchestra.run.smithi046.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 24 MB/s | 4.7 MB 00:00 2024-05-08T18:03:16.023 INFO:teuthology.orchestra.run.smithi046.stdout:(9/12): ceph-mgr-diskprediction-local-19.0.0-35 11 MB/s | 7.4 MB 00:00 2024-05-08T18:03:16.048 INFO:teuthology.orchestra.run.smithi046.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.0 MB/s | 250 kB 00:00 2024-05-08T18:03:16.165 INFO:teuthology.orchestra.run.smithi046.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-05-08T18:03:16.332 INFO:teuthology.orchestra.run.smithi050.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 272 kB/s | 244 kB 00:00 2024-05-08T18:03:16.415 INFO:teuthology.orchestra.run.smithi050.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 3.8 MB/s | 4.7 MB 00:01 2024-05-08T18:03:16.499 INFO:teuthology.orchestra.run.smithi046.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-05-08T18:03:16.500 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:16.500 INFO:teuthology.orchestra.run.smithi046.stdout:Total 28 MB/s | 38 MB 00:01 2024-05-08T18:03:16.864 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T18:03:16.892 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T18:03:16.893 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T18:03:17.074 INFO:teuthology.orchestra.run.smithi050.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.5 MB/s | 5.1 MB 00:01 2024-05-08T18:03:17.339 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T18:03:17.339 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T18:03:17.417 INFO:teuthology.orchestra.run.smithi050.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 231 kB/s | 250 kB 00:01 2024-05-08T18:03:18.125 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T18:03:18.203 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-08T18:03:18.247 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-08T18:03:18.354 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-08T18:03:18.390 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-08T18:03:18.458 INFO:teuthology.orchestra.run.smithi050.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.0 MB/s | 16 MB 00:02 2024-05-08T18:03:18.460 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:18.460 INFO:teuthology.orchestra.run.smithi050.stdout:Total 7.5 MB/s | 38 MB 00:05 2024-05-08T18:03:18.574 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-08T18:03:18.614 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-08T18:03:18.701 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-08T18:03:18.834 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T18:03:18.863 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T18:03:18.863 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T18:03:19.322 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T18:03:19.322 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T18:03:19.404 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-08T18:03:19.532 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-08T18:03:20.103 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T18:03:20.184 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-08T18:03:20.223 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-08T18:03:20.329 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-08T18:03:20.383 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-08T18:03:20.559 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-08T18:03:20.590 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-08T18:03:20.669 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-08T18:03:20.855 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T18:03:21.356 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-08T18:03:21.485 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-08T18:03:21.935 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-08T18:03:21.963 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3525.g27d37 12/12 2024-05-08T18:03:22.820 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T18:03:23.899 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-08T18:03:23.924 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3525.g27d37 12/12 2024-05-08T18:03:24.474 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3525.g27d37 12/12 2024-05-08T18:03:24.474 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3525.g27d37 1/12 2024-05-08T18:03:24.474 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-08T18:03:24.474 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-08T18:03:24.474 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-08T18:03:24.474 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-08T18:03:24.474 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-08T18:03:24.474 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-08T18:03:24.475 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-08T18:03:24.475 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-08T18:03:24.475 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T18:03:24.475 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-08T18:03:24.862 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-08T18:03:24.862 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:24.862 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T18:03:24.862 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:03:24.862 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T18:03:24.862 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T18:03:24.862 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T18:03:24.862 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T18:03:24.862 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T18:03:24.862 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T18:03:24.863 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T18:03:24.863 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T18:03:24.863 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T18:03:24.863 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T18:03:24.863 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T18:03:24.863 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:24.863 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:25.177 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr-rook 2024-05-08T18:03:25.774 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:34 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:25.902 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:25.903 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-05-08T18:03:25.903 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T18:03:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 49 k 2024-05-08T18:03:25.904 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-05-08T18:03:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T18:03:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T18:03:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T18:03:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T18:03:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T18:03:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T18:03:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T18:03:25.904 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T18:03:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T18:03:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T18:03:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T18:03:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T18:03:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:25.905 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T18:03:25.905 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:25.905 INFO:teuthology.orchestra.run.smithi046.stdout:Install 13 Packages 2024-05-08T18:03:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 2.2 M 2024-05-08T18:03:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 26 M 2024-05-08T18:03:25.906 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T18:03:26.196 INFO:teuthology.orchestra.run.smithi046.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 177 kB/s | 26 kB 00:00 2024-05-08T18:03:26.221 INFO:teuthology.orchestra.run.smithi046.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 113 kB/s | 19 kB 00:00 2024-05-08T18:03:26.354 INFO:teuthology.orchestra.run.smithi046.stdout:(3/13): ceph-mgr-rook-19.0.0-3525.g27d37168.el9 162 kB/s | 49 kB 00:00 2024-05-08T18:03:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3525.g27d37 12/12 2024-05-08T18:03:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3525.g27d37 1/12 2024-05-08T18:03:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-08T18:03:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-08T18:03:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-08T18:03:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-08T18:03:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-08T18:03:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-08T18:03:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-08T18:03:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-08T18:03:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T18:03:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-08T18:03:26.471 INFO:teuthology.orchestra.run.smithi046.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 635 kB/s | 159 kB 00:00 2024-05-08T18:03:26.530 INFO:teuthology.orchestra.run.smithi046.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 720 kB/s | 222 kB 00:00 2024-05-08T18:03:26.580 INFO:teuthology.orchestra.run.smithi046.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 494 kB/s | 54 kB 00:00 2024-05-08T18:03:26.764 INFO:teuthology.orchestra.run.smithi046.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 681 kB/s | 279 kB 00:00 2024-05-08T18:03:26.798 INFO:teuthology.orchestra.run.smithi046.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 120 kB/s | 32 kB 00:00 2024-05-08T18:03:26.823 INFO:teuthology.orchestra.run.smithi046.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 62 kB/s | 15 kB 00:00 2024-05-08T18:03:26.876 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-08T18:03:26.876 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:26.876 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T18:03:26.876 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:03:26.876 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T18:03:26.876 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T18:03:26.876 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T18:03:26.876 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T18:03:26.876 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T18:03:26.876 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T18:03:26.876 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T18:03:26.877 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T18:03:26.877 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T18:03:26.877 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T18:03:26.877 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T18:03:26.877 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:26.877 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:27.056 INFO:teuthology.orchestra.run.smithi046.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 253 kB/s | 59 kB 00:00 2024-05-08T18:03:27.219 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-mgr-rook 2024-05-08T18:03:27.374 INFO:teuthology.orchestra.run.smithi046.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 283 kB/s | 90 kB 00:00 2024-05-08T18:03:27.415 INFO:teuthology.orchestra.run.smithi046.stdout:(12/13): python3-google-auth-2.29.0-1.el9.noarc 316 kB/s | 206 kB 00:00 2024-05-08T18:03:27.815 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:27 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:27.944 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:27.946 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:27.946 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-05-08T18:03:27.946 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:27.946 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T18:03:27.946 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 49 k 2024-05-08T18:03:27.946 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-05-08T18:03:27.946 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T18:03:27.946 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T18:03:27.946 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T18:03:27.946 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T18:03:27.946 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T18:03:27.946 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T18:03:27.947 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T18:03:27.947 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T18:03:27.947 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T18:03:27.947 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T18:03:27.947 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T18:03:27.947 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T18:03:27.947 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:27.947 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T18:03:27.947 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:27.947 INFO:teuthology.orchestra.run.smithi050.stdout:Install 13 Packages 2024-05-08T18:03:27.947 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:27.948 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 2.2 M 2024-05-08T18:03:27.948 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 26 M 2024-05-08T18:03:27.948 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T18:03:28.693 INFO:teuthology.orchestra.run.smithi050.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 189 kB/s | 26 kB 00:00 2024-05-08T18:03:28.718 INFO:teuthology.orchestra.run.smithi050.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 119 kB/s | 19 kB 00:00 2024-05-08T18:03:28.843 INFO:teuthology.orchestra.run.smithi050.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-05-08T18:03:28.869 INFO:teuthology.orchestra.run.smithi050.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-05-08T18:03:28.894 INFO:teuthology.orchestra.run.smithi050.stdout:(5/13): ceph-mgr-rook-19.0.0-3525.g27d37168.el9 146 kB/s | 49 kB 00:00 2024-05-08T18:03:28.919 INFO:teuthology.orchestra.run.smithi050.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-05-08T18:03:28.982 INFO:teuthology.orchestra.run.smithi046.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 479 kB/s | 1.0 MB 00:02 2024-05-08T18:03:28.982 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:28.983 INFO:teuthology.orchestra.run.smithi046.stdout:Total 732 kB/s | 2.2 MB 00:03 2024-05-08T18:03:29.023 INFO:teuthology.orchestra.run.smithi050.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 257 kB/s | 32 kB 00:00 2024-05-08T18:03:29.045 INFO:teuthology.orchestra.run.smithi050.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 119 kB/s | 15 kB 00:00 2024-05-08T18:03:29.058 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T18:03:29.070 INFO:teuthology.orchestra.run.smithi050.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.2 MB/s | 279 kB 00:00 2024-05-08T18:03:29.074 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T18:03:29.075 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T18:03:29.154 INFO:teuthology.orchestra.run.smithi050.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 709 kB/s | 59 kB 00:00 2024-05-08T18:03:29.171 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T18:03:29.171 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T18:03:29.221 INFO:teuthology.orchestra.run.smithi050.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-05-08T18:03:29.329 INFO:teuthology.orchestra.run.smithi050.stdout:(12/13): python3-google-auth-2.29.0-1.el9.noarc 665 kB/s | 206 kB 00:00 2024-05-08T18:03:29.488 INFO:teuthology.orchestra.run.smithi050.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.3 MB/s | 1.0 MB 00:00 2024-05-08T18:03:29.488 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:29.489 INFO:teuthology.orchestra.run.smithi050.stdout:Total 1.4 MB/s | 2.2 MB 00:01 2024-05-08T18:03:29.564 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T18:03:29.580 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T18:03:29.580 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T18:03:29.676 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T18:03:29.678 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T18:03:29.841 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T18:03:30.021 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-08T18:03:30.077 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-08T18:03:30.132 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-08T18:03:30.168 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-08T18:03:30.201 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T18:03:30.210 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-08T18:03:30.309 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-08T18:03:30.369 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-08T18:03:30.420 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-08T18:03:30.424 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-08T18:03:30.471 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-08T18:03:30.480 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-08T18:03:30.516 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-08T18:03:30.557 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-08T18:03:30.655 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-08T18:03:30.765 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-08T18:03:30.818 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-08T18:03:31.231 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-08T18:03:31.283 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-08T18:03:31.317 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-08T18:03:31.366 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-08T18:03:31.394 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-rook-2:19.0.0-3525.g27d37168.el9.noarch 13/13 2024-05-08T18:03:31.578 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-08T18:03:31.631 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-08T18:03:31.665 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-08T18:03:31.714 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-08T18:03:31.742 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-rook-2:19.0.0-3525.g27d37168.el9.noarch 13/13 2024-05-08T18:03:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3525.g27d37168.el9.noarch 13/13 2024-05-08T18:03:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3525.g27d37168.el9.noarch 1/13 2024-05-08T18:03:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-08T18:03:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-08T18:03:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-08T18:03:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-08T18:03:32.199 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-08T18:03:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-08T18:03:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-08T18:03:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-08T18:03:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-08T18:03:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-08T18:03:32.200 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-08T18:03:32.517 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3525.g27d37168.el9.noarch 13/13 2024-05-08T18:03:32.517 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3525.g27d37168.el9.noarch 1/13 2024-05-08T18:03:32.517 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-08T18:03:32.517 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-08T18:03:32.517 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-08T18:03:32.517 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-08T18:03:32.517 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-08T18:03:32.517 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-08T18:03:32.517 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-08T18:03:32.517 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-08T18:03:32.518 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-08T18:03:32.518 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-08T18:03:32.518 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-08T18:03:32.526 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-08T18:03:32.526 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:32.526 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T18:03:32.526 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:03:32.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T18:03:32.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T18:03:32.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T18:03:32.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T18:03:32.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T18:03:32.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T18:03:32.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T18:03:32.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T18:03:32.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T18:03:32.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T18:03:32.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T18:03:32.527 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T18:03:32.527 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:32.527 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:32.783 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr-cephadm 2024-05-08T18:03:32.842 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-08T18:03:32.843 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:32.843 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T18:03:32.843 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:03:32.843 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T18:03:32.843 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T18:03:32.843 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T18:03:32.843 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T18:03:32.843 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T18:03:32.843 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T18:03:32.843 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T18:03:32.843 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T18:03:32.843 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T18:03:32.843 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T18:03:32.843 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T18:03:32.844 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T18:03:32.844 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:32.844 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:33.080 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-mgr-cephadm 2024-05-08T18:03:33.376 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:42 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:33.505 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:33.505 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:33.506 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-05-08T18:03:33.506 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:33.506 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T18:03:33.506 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 142 k 2024-05-08T18:03:33.506 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-05-08T18:03:33.506 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T18:03:33.506 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T18:03:33.506 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T18:03:33.506 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T18:03:33.506 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T18:03:33.506 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:33.506 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T18:03:33.506 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:33.506 INFO:teuthology.orchestra.run.smithi046.stdout:Install 6 Packages 2024-05-08T18:03:33.507 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:33.507 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 7.0 M 2024-05-08T18:03:33.507 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 33 M 2024-05-08T18:03:33.507 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T18:03:33.674 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:33 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:33.801 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:33.802 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-05-08T18:03:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T18:03:33.802 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 142 k 2024-05-08T18:03:33.802 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-05-08T18:03:33.802 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T18:03:33.802 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T18:03:33.803 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T18:03:33.803 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T18:03:33.803 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T18:03:33.803 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T18:03:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:Install 6 Packages 2024-05-08T18:03:33.803 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:33.803 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 7.0 M 2024-05-08T18:03:33.804 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 33 M 2024-05-08T18:03:33.804 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T18:03:33.926 INFO:teuthology.orchestra.run.smithi046.stdout:(1/6): ceph-mgr-cephadm-19.0.0-3525.g27d37168.e 578 kB/s | 142 kB 00:00 2024-05-08T18:03:34.276 INFO:teuthology.orchestra.run.smithi046.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 416 kB/s | 248 kB 00:00 2024-05-08T18:03:34.290 INFO:teuthology.orchestra.run.smithi050.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 2.4 MB/s | 248 kB 00:00 2024-05-08T18:03:34.518 INFO:teuthology.orchestra.run.smithi046.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 927 kB/s | 548 kB 00:00 2024-05-08T18:03:34.550 INFO:teuthology.orchestra.run.smithi050.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-05-08T18:03:34.594 INFO:teuthology.orchestra.run.smithi046.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 182 kB/s | 58 kB 00:00 2024-05-08T18:03:34.618 INFO:teuthology.orchestra.run.smithi046.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 508 kB/s | 51 kB 00:00 2024-05-08T18:03:34.708 INFO:teuthology.orchestra.run.smithi050.stdout:(3/6): ceph-mgr-cephadm-19.0.0-3525.g27d37168.e 274 kB/s | 142 kB 00:00 2024-05-08T18:03:34.992 INFO:teuthology.orchestra.run.smithi050.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 131 kB/s | 58 kB 00:00 2024-05-08T18:03:35.125 INFO:teuthology.orchestra.run.smithi050.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 122 kB/s | 51 kB 00:00 2024-05-08T18:03:35.259 INFO:teuthology.orchestra.run.smithi050.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 582 kB/s | 548 kB 00:00 2024-05-08T18:03:35.260 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:35.260 INFO:teuthology.orchestra.run.smithi050.stdout:Total 4.8 MB/s | 7.0 MB 00:01 2024-05-08T18:03:35.338 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T18:03:35.350 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T18:03:35.350 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T18:03:35.454 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T18:03:35.455 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T18:03:35.782 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T18:03:35.955 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-08T18:03:36.012 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-08T18:03:36.559 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-08T18:03:36.661 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-08T18:03:36.729 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-08T18:03:36.765 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3525.g27d37168.el9.noarch 6/6 2024-05-08T18:03:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3525.g27d37168.el9.noarch 6/6 2024-05-08T18:03:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3525.g27d37168.el9.noarch 1/6 2024-05-08T18:03:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-08T18:03:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-08T18:03:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-08T18:03:37.560 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-08T18:03:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-08T18:03:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:37.823 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T18:03:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:03:37.823 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T18:03:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T18:03:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T18:03:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T18:03:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T18:03:37.824 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:37.824 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:38.074 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-fuse 2024-05-08T18:03:38.535 INFO:teuthology.orchestra.run.smithi046.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 1.2 MB/s | 6.0 MB 00:04 2024-05-08T18:03:38.538 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:38.538 INFO:teuthology.orchestra.run.smithi046.stdout:Total 1.4 MB/s | 7.0 MB 00:05 2024-05-08T18:03:38.615 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T18:03:38.628 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T18:03:38.628 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T18:03:38.676 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:38 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:38.733 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T18:03:38.733 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T18:03:38.804 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:38.805 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:38.805 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-05-08T18:03:38.805 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:38.805 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T18:03:38.805 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse x86_64 2:19.0.0-3525.g27d37168.el9 ceph 891 k 2024-05-08T18:03:38.805 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-05-08T18:03:38.805 INFO:teuthology.orchestra.run.smithi050.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T18:03:38.805 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:38.805 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T18:03:38.805 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:38.805 INFO:teuthology.orchestra.run.smithi050.stdout:Install 2 Packages 2024-05-08T18:03:38.805 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:38.806 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 971 k 2024-05-08T18:03:38.806 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 2.8 M 2024-05-08T18:03:38.806 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T18:03:39.049 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T18:03:39.235 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-08T18:03:39.288 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-08T18:03:39.337 INFO:teuthology.orchestra.run.smithi050.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 165 kB/s | 80 kB 00:00 2024-05-08T18:03:39.579 INFO:teuthology.orchestra.run.smithi050.stdout:(2/2): ceph-fuse-19.0.0-3525.g27d37168.el9.x86_ 1.2 MB/s | 891 kB 00:00 2024-05-08T18:03:39.579 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:39.579 INFO:teuthology.orchestra.run.smithi050.stdout:Total 1.2 MB/s | 971 kB 00:00 2024-05-08T18:03:39.592 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T18:03:39.616 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T18:03:39.616 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T18:03:39.699 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T18:03:39.700 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T18:03:39.848 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-08T18:03:39.898 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T18:03:39.944 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-08T18:03:40.022 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-08T18:03:40.049 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3525.g27d37168.el9.noarch 6/6 2024-05-08T18:03:40.051 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-08T18:03:40.123 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-fuse-2:19.0.0-3525.g27d37168.el9.x86_64 2/2 2024-05-08T18:03:40.740 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-fuse-2:19.0.0-3525.g27d37168.el9.x86_64 2/2 2024-05-08T18:03:40.740 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-fuse-2:19.0.0-3525.g27d37168.el9.x86_64 1/2 2024-05-08T18:03:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3525.g27d37168.el9.noarch 6/6 2024-05-08T18:03:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3525.g27d37168.el9.noarch 1/6 2024-05-08T18:03:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-08T18:03:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-08T18:03:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-08T18:03:40.831 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-08T18:03:40.982 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-08T18:03:40.982 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:40.982 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T18:03:40.982 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse-2:19.0.0-3525.g27d37168.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-08T18:03:40.982 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:40.982 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:41.217 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-08T18:03:41.217 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:41.217 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T18:03:41.217 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:03:41.217 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T18:03:41.217 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T18:03:41.217 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T18:03:41.217 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T18:03:41.217 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T18:03:41.217 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:41.218 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:41.278 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-volume 2024-05-08T18:03:41.486 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-fuse 2024-05-08T18:03:41.881 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:41 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:42.011 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:42.012 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:42.012 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-05-08T18:03:42.012 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:42.012 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T18:03:42.012 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 273 k 2024-05-08T18:03:42.012 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-05-08T18:03:42.012 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T18:03:42.013 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:42.013 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T18:03:42.013 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:42.013 INFO:teuthology.orchestra.run.smithi050.stdout:Install 2 Packages 2024-05-08T18:03:42.013 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:42.013 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 350 k 2024-05-08T18:03:42.013 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 1.5 M 2024-05-08T18:03:42.013 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T18:03:42.087 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:51 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:42.143 INFO:teuthology.orchestra.run.smithi050.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 1.2 MB/s | 77 kB 00:00 2024-05-08T18:03:42.215 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:42.216 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:42.216 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-05-08T18:03:42.216 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:42.217 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T18:03:42.217 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse x86_64 2:19.0.0-3525.g27d37168.el9 ceph 891 k 2024-05-08T18:03:42.217 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-05-08T18:03:42.217 INFO:teuthology.orchestra.run.smithi046.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T18:03:42.217 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:42.217 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T18:03:42.217 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:42.217 INFO:teuthology.orchestra.run.smithi046.stdout:Install 2 Packages 2024-05-08T18:03:42.217 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:42.217 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 971 k 2024-05-08T18:03:42.217 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 2.8 M 2024-05-08T18:03:42.218 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T18:03:42.480 INFO:teuthology.orchestra.run.smithi046.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 564 kB/s | 80 kB 00:00 2024-05-08T18:03:42.486 INFO:teuthology.orchestra.run.smithi050.stdout:(2/2): ceph-volume-19.0.0-3525.g27d37168.el9.no 673 kB/s | 273 kB 00:00 2024-05-08T18:03:42.486 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:42.487 INFO:teuthology.orchestra.run.smithi050.stdout:Total 738 kB/s | 350 kB 00:00 2024-05-08T18:03:42.499 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T18:03:42.510 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T18:03:42.510 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T18:03:42.545 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T18:03:42.545 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T18:03:42.694 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T18:03:42.892 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-08T18:03:42.922 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-volume-2:19.0.0-3525.g27d37168.el9.noarch 2/2 2024-05-08T18:03:42.939 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-volume-2:19.0.0-3525.g27d37168.el9.noarch 2/2 2024-05-08T18:03:42.941 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T18:03:42.941 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T18:03:42.941 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:43.055 INFO:teuthology.orchestra.run.smithi046.stdout:(2/2): ceph-fuse-19.0.0-3525.g27d37168.el9.x86_ 1.2 MB/s | 891 kB 00:00 2024-05-08T18:03:43.056 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:43.056 INFO:teuthology.orchestra.run.smithi046.stdout:Total 1.1 MB/s | 971 kB 00:00 2024-05-08T18:03:43.070 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T18:03:43.095 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T18:03:43.095 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T18:03:43.181 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T18:03:43.182 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T18:03:43.382 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T18:03:43.519 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-08T18:03:43.591 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-fuse-2:19.0.0-3525.g27d37168.el9.x86_64 2/2 2024-05-08T18:03:43.674 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-volume-2:19.0.0-3525.g27d37168.el9.noarch 1/2 2024-05-08T18:03:43.965 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-08T18:03:43.965 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:43.965 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T18:03:43.965 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:03:43.965 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T18:03:43.965 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:43.965 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:44.278 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install librados-devel 2024-05-08T18:03:44.388 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-fuse-2:19.0.0-3525.g27d37168.el9.x86_64 2/2 2024-05-08T18:03:44.389 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-fuse-2:19.0.0-3525.g27d37168.el9.x86_64 1/2 2024-05-08T18:03:44.657 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-08T18:03:44.657 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:44.657 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T18:03:44.657 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse-2:19.0.0-3525.g27d37168.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-08T18:03:44.657 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:44.658 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:44.875 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:44 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:44.937 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-volume 2024-05-08T18:03:45.003 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:45.004 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:45.004 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repo Size 2024-05-08T18:03:45.004 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:45.004 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T18:03:45.004 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel x86_64 2:19.0.0-3525.g27d37168.el9 ceph 127 k 2024-05-08T18:03:45.004 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:45.004 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T18:03:45.004 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:45.004 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2024-05-08T18:03:45.004 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:45.004 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 127 k 2024-05-08T18:03:45.004 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 456 k 2024-05-08T18:03:45.004 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T18:03:45.226 INFO:teuthology.orchestra.run.smithi050.stdout:librados-devel-19.0.0-3525.g27d37168.el9.x86_64 573 kB/s | 127 kB 00:00 2024-05-08T18:03:45.227 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:45.227 INFO:teuthology.orchestra.run.smithi050.stdout:Total 569 kB/s | 127 kB 00:00 2024-05-08T18:03:45.227 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T18:03:45.232 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T18:03:45.233 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T18:03:45.300 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T18:03:45.300 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T18:03:45.458 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T18:03:45.538 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:54 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:45.552 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librados-devel-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:45.666 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:45.667 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:45.667 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-05-08T18:03:45.667 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:45.667 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T18:03:45.667 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume noarch 2:19.0.0-3525.g27d37168.el9 ceph-noarch 273 k 2024-05-08T18:03:45.667 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-05-08T18:03:45.667 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T18:03:45.668 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:45.668 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T18:03:45.668 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:45.668 INFO:teuthology.orchestra.run.smithi046.stdout:Install 2 Packages 2024-05-08T18:03:45.668 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:45.668 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 350 k 2024-05-08T18:03:45.668 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 1.5 M 2024-05-08T18:03:45.668 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T18:03:45.985 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados-devel-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:46.101 INFO:teuthology.orchestra.run.smithi046.stdout:(1/2): ceph-volume-19.0.0-3525.g27d37168.el9.no 1.0 MB/s | 273 kB 00:00 2024-05-08T18:03:46.221 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados-devel-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:46.222 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:46.222 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T18:03:46.222 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:03:46.222 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:46.222 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:46.471 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install libcephfs2 2024-05-08T18:03:46.877 INFO:teuthology.orchestra.run.smithi046.stdout:(2/2): python3-packaging-20.9-5.el9.noarch.rpm 74 kB/s | 77 kB 00:01 2024-05-08T18:03:46.878 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:46.878 INFO:teuthology.orchestra.run.smithi046.stdout:Total 289 kB/s | 350 kB 00:01 2024-05-08T18:03:46.893 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T18:03:46.905 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T18:03:46.905 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T18:03:46.945 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T18:03:46.945 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T18:03:47.069 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:47 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:47.103 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T18:03:47.153 INFO:teuthology.orchestra.run.smithi050.stdout:Package libcephfs2-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:47.196 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:47.198 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-05-08T18:03:47.198 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:47.269 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install libcephfs-devel 2024-05-08T18:03:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-08T18:03:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-volume-2:19.0.0-3525.g27d37168.el9.noarch 2/2 2024-05-08T18:03:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-volume-2:19.0.0-3525.g27d37168.el9.noarch 2/2 2024-05-08T18:03:47.388 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T18:03:47.388 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T18:03:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:47.864 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:47 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:47.991 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:47.992 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:47.992 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repo Size 2024-05-08T18:03:47.992 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:47.992 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T18:03:47.992 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel x86_64 2:19.0.0-3525.g27d37168.el9 ceph 32 k 2024-05-08T18:03:47.992 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:47.993 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T18:03:47.993 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:47.993 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2024-05-08T18:03:47.993 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:47.993 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 32 k 2024-05-08T18:03:47.993 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 142 k 2024-05-08T18:03:47.993 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T18:03:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-volume-2:19.0.0-3525.g27d37168.el9.noarch 1/2 2024-05-08T18:03:48.239 INFO:teuthology.orchestra.run.smithi050.stdout:libcephfs-devel-19.0.0-3525.g27d37168.el9.x86_6 129 kB/s | 32 kB 00:00 2024-05-08T18:03:48.240 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:48.240 INFO:teuthology.orchestra.run.smithi050.stdout:Total 128 kB/s | 32 kB 00:00 2024-05-08T18:03:48.240 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T18:03:48.243 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T18:03:48.243 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T18:03:48.255 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T18:03:48.255 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T18:03:48.344 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T18:03:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-08T18:03:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:48.425 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T18:03:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume-2:19.0.0-3525.g27d37168.el9.noarch 2024-05-08T18:03:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T18:03:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:48.426 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:48.447 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs-devel-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:48.811 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:48.863 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install librados-devel 2024-05-08T18:03:49.030 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs-devel-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:49.030 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:49.030 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T18:03:49.030 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:03:49.030 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:49.030 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:49.213 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install librados2 2024-05-08T18:03:49.465 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:58 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:49.593 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:49.594 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repo Size 2024-05-08T18:03:49.594 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:49.594 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T18:03:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel x86_64 2:19.0.0-3525.g27d37168.el9 ceph 127 k 2024-05-08T18:03:49.594 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:49.595 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T18:03:49.595 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:49.595 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-05-08T18:03:49.595 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:49.595 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 127 k 2024-05-08T18:03:49.595 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 456 k 2024-05-08T18:03:49.595 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T18:03:49.814 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:49 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:49.898 INFO:teuthology.orchestra.run.smithi050.stdout:Package librados2-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:49.919 INFO:teuthology.orchestra.run.smithi046.stdout:librados-devel-19.0.0-3525.g27d37168.el9.x86_64 391 kB/s | 127 kB 00:00 2024-05-08T18:03:49.920 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:49.920 INFO:teuthology.orchestra.run.smithi046.stdout:Total 389 kB/s | 127 kB 00:00 2024-05-08T18:03:49.920 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T18:03:49.927 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T18:03:49.927 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T18:03:49.942 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:49.943 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-05-08T18:03:49.943 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:49.994 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T18:03:49.994 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T18:03:50.012 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install librbd1 2024-05-08T18:03:50.155 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T18:03:50.271 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librados-devel-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:50.605 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:50 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:50.688 INFO:teuthology.orchestra.run.smithi050.stdout:Package librbd1-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:50.721 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados-devel-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:50.733 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:50.734 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-05-08T18:03:50.735 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:50.813 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install python3-rados 2024-05-08T18:03:50.974 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados-devel-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:51.011 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:51.012 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T18:03:51.012 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:03:51.012 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:51.012 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:51.229 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install libcephfs2 2024-05-08T18:03:51.412 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:51 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:51.497 INFO:teuthology.orchestra.run.smithi050.stdout:Package python3-rados-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:51.541 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:51.542 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-05-08T18:03:51.542 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:51.613 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install python3-rgw 2024-05-08T18:03:51.834 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:00 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:51.919 INFO:teuthology.orchestra.run.smithi046.stdout:Package libcephfs2-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:51.963 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:51.964 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-05-08T18:03:51.964 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:52.034 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install libcephfs-devel 2024-05-08T18:03:52.209 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:52 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:52.294 INFO:teuthology.orchestra.run.smithi050.stdout:Package python3-rgw-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:52.338 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:52.339 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-05-08T18:03:52.339 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:52.409 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install python3-cephfs 2024-05-08T18:03:52.621 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:01 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:52.748 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:52.749 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:52.749 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repo Size 2024-05-08T18:03:52.749 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:52.749 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T18:03:52.749 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel x86_64 2:19.0.0-3525.g27d37168.el9 ceph 32 k 2024-05-08T18:03:52.749 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:52.749 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T18:03:52.749 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:52.749 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-05-08T18:03:52.750 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:52.750 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 32 k 2024-05-08T18:03:52.750 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 142 k 2024-05-08T18:03:52.750 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T18:03:52.915 INFO:teuthology.orchestra.run.smithi046.stdout:libcephfs-devel-19.0.0-3525.g27d37168.el9.x86_6 192 kB/s | 32 kB 00:00 2024-05-08T18:03:52.916 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:52.916 INFO:teuthology.orchestra.run.smithi046.stdout:Total 190 kB/s | 32 kB 00:00 2024-05-08T18:03:52.916 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T18:03:52.920 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T18:03:52.920 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T18:03:52.934 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T18:03:52.934 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T18:03:53.009 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:53 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:53.014 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T18:03:53.092 INFO:teuthology.orchestra.run.smithi050.stdout:Package python3-cephfs-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:53.136 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:53.137 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-05-08T18:03:53.137 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:53.140 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs-devel-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:53.213 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install python3-rbd 2024-05-08T18:03:53.673 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:53.813 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:53 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:53.896 INFO:teuthology.orchestra.run.smithi050.stdout:Package python3-rbd-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:53.906 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs-devel-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:53.906 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:53.906 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T18:03:53.906 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:03:53.906 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:53.906 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:53.941 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:53.942 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-05-08T18:03:53.942 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:54.019 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install rbd-fuse 2024-05-08T18:03:54.073 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install librados2 2024-05-08T18:03:54.613 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:54 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:54.670 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:03 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:54.743 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:54.744 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:54.744 INFO:teuthology.orchestra.run.smithi050.stdout: Package Architecture Version Repository Size 2024-05-08T18:03:54.744 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:54.744 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T18:03:54.744 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse x86_64 2:19.0.0-3525.g27d37168.el9 ceph 85 k 2024-05-08T18:03:54.744 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:54.744 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T18:03:54.744 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:54.744 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2024-05-08T18:03:54.744 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:54.744 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 85 k 2024-05-08T18:03:54.745 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 231 k 2024-05-08T18:03:54.745 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T18:03:54.754 INFO:teuthology.orchestra.run.smithi046.stdout:Package librados2-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:54.798 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:54.799 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-05-08T18:03:54.799 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:54.863 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install librbd1 2024-05-08T18:03:55.092 INFO:teuthology.orchestra.run.smithi050.stdout:rbd-fuse-19.0.0-3525.g27d37168.el9.x86_64.rpm 245 kB/s | 85 kB 00:00 2024-05-08T18:03:55.093 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:55.093 INFO:teuthology.orchestra.run.smithi050.stdout:Total 245 kB/s | 85 kB 00:00 2024-05-08T18:03:55.093 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T18:03:55.100 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T18:03:55.100 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T18:03:55.166 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T18:03:55.167 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T18:03:55.317 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T18:03:55.434 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-fuse-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:55.456 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:04 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:55.540 INFO:teuthology.orchestra.run.smithi046.stdout:Package librbd1-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:55.584 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:55.585 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-05-08T18:03:55.585 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:55.646 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install python3-rados 2024-05-08T18:03:55.898 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: rbd-fuse-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:56.154 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-fuse-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:56.154 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:56.154 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T18:03:56.155 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:03:56.155 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:56.155 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:56.246 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:05 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:56.330 INFO:teuthology.orchestra.run.smithi046.stdout:Package python3-rados-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:56.374 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:56.375 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-05-08T18:03:56.375 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:56.423 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install rbd-mirror 2024-05-08T18:03:56.446 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install python3-rgw 2024-05-08T18:03:57.028 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:57 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:03:57.035 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:06 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:57.118 INFO:teuthology.orchestra.run.smithi046.stdout:Package python3-rgw-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:57.159 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:03:57.160 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:57.160 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repo Size 2024-05-08T18:03:57.160 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:57.160 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T18:03:57.160 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror x86_64 2:19.0.0-3525.g27d37168.el9 ceph 3.1 M 2024-05-08T18:03:57.160 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:57.160 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T18:03:57.160 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:03:57.160 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2024-05-08T18:03:57.161 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:57.161 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 3.1 M 2024-05-08T18:03:57.161 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 13 M 2024-05-08T18:03:57.161 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T18:03:57.162 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:57.163 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-05-08T18:03:57.163 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:57.233 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install python3-cephfs 2024-05-08T18:03:57.822 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:06 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:57.879 INFO:teuthology.orchestra.run.smithi050.stdout:rbd-mirror-19.0.0-3525.g27d37168.el9.x86_64.rpm 4.3 MB/s | 3.1 MB 00:00 2024-05-08T18:03:57.879 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:57.880 INFO:teuthology.orchestra.run.smithi050.stdout:Total 4.3 MB/s | 3.1 MB 00:00 2024-05-08T18:03:57.880 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T18:03:57.890 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T18:03:57.891 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T18:03:57.904 INFO:teuthology.orchestra.run.smithi046.stdout:Package python3-cephfs-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:57.948 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:57.949 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-05-08T18:03:57.949 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:57.976 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T18:03:57.977 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T18:03:58.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install python3-rbd 2024-05-08T18:03:58.425 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T18:03:58.488 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-mirror-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:58.514 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:58.515 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T18:03:58.515 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T18:03:58.515 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T18:03:58.515 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T18:03:58.515 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:58.613 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:07 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:58.698 INFO:teuthology.orchestra.run.smithi046.stdout:Package python3-rbd-2:19.0.0-3525.g27d37168.el9.x86_64 is already installed. 2024-05-08T18:03:58.742 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:58.743 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-05-08T18:03:58.743 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:03:58.808 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install rbd-fuse 2024-05-08T18:03:59.418 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:08 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:03:59.553 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:03:59.554 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:59.554 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-05-08T18:03:59.554 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:59.554 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T18:03:59.554 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse x86_64 2:19.0.0-3525.g27d37168.el9 ceph 85 k 2024-05-08T18:03:59.554 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:59.554 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T18:03:59.555 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:03:59.555 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-05-08T18:03:59.555 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:03:59.555 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 85 k 2024-05-08T18:03:59.555 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 231 k 2024-05-08T18:03:59.555 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T18:03:59.590 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-mirror-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:03:59.590 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:59.590 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T18:03:59.590 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:03:59.591 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:03:59.591 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:03:59.864 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install rbd-nbd 2024-05-08T18:03:59.877 INFO:teuthology.orchestra.run.smithi046.stdout:rbd-fuse-19.0.0-3525.g27d37168.el9.x86_64.rpm 265 kB/s | 85 kB 00:00 2024-05-08T18:03:59.878 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:03:59.878 INFO:teuthology.orchestra.run.smithi046.stdout:Total 264 kB/s | 85 kB 00:00 2024-05-08T18:03:59.878 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T18:03:59.885 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T18:03:59.886 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T18:03:59.951 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T18:03:59.952 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T18:04:00.099 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T18:04:00.203 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-fuse-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:04:00.463 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:02:00 ago on Wed 08 May 2024 06:02:00 PM UTC. 2024-05-08T18:04:00.589 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-05-08T18:04:00.590 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:04:00.590 INFO:teuthology.orchestra.run.smithi050.stdout: Package Architecture Version Repository Size 2024-05-08T18:04:00.590 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:04:00.591 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-05-08T18:04:00.591 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd x86_64 2:19.0.0-3525.g27d37168.el9 ceph 170 k 2024-05-08T18:04:00.591 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:00.591 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-05-08T18:04:00.591 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-05-08T18:04:00.591 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2024-05-08T18:04:00.591 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:00.591 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 170 k 2024-05-08T18:04:00.591 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 494 k 2024-05-08T18:04:00.591 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-05-08T18:04:00.593 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-fuse-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:04:00.862 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-fuse-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:04:00.863 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:00.863 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T18:04:00.863 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:04:00.863 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:00.863 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:04:00.990 INFO:teuthology.orchestra.run.smithi050.stdout:rbd-nbd-19.0.0-3525.g27d37168.el9.x86_64.rpm 427 kB/s | 170 kB 00:00 2024-05-08T18:04:00.991 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:04:00.991 INFO:teuthology.orchestra.run.smithi050.stdout:Total 425 kB/s | 170 kB 00:00 2024-05-08T18:04:00.991 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-05-08T18:04:01.000 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-05-08T18:04:01.001 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-05-08T18:04:01.068 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-05-08T18:04:01.069 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-05-08T18:04:01.127 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install rbd-mirror 2024-05-08T18:04:01.227 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-05-08T18:04:01.333 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-nbd-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:04:01.723 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:10 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:04:01.758 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: rbd-nbd-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:04:01.852 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:04:01.853 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:04:01.853 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repo Size 2024-05-08T18:04:01.853 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:04:01.853 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T18:04:01.853 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror x86_64 2:19.0.0-3525.g27d37168.el9 ceph 3.1 M 2024-05-08T18:04:01.853 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:01.853 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T18:04:01.854 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:04:01.854 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-05-08T18:04:01.854 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:01.854 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 3.1 M 2024-05-08T18:04:01.854 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 13 M 2024-05-08T18:04:01.854 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T18:04:02.028 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-nbd-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:04:02.028 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:02.028 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-05-08T18:04:02.028 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:04:02.028 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:02.028 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-05-08T18:04:02.270 DEBUG:teuthology.parallel:result is None 2024-05-08T18:04:02.514 INFO:teuthology.orchestra.run.smithi046.stdout:rbd-mirror-19.0.0-3525.g27d37168.el9.x86_64.rpm 4.7 MB/s | 3.1 MB 00:00 2024-05-08T18:04:02.514 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:04:02.515 INFO:teuthology.orchestra.run.smithi046.stdout:Total 4.7 MB/s | 3.1 MB 00:00 2024-05-08T18:04:02.515 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T18:04:02.525 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T18:04:02.526 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T18:04:02.611 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T18:04:02.611 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T18:04:03.064 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T18:04:03.140 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-mirror-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:04:03.166 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:04:03.167 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T18:04:03.167 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T18:04:03.167 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T18:04:03.167 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T18:04:03.167 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-mirror-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:04:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:04.347 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T18:04:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:04:04.347 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:04.347 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:04:04.583 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install rbd-nbd 2024-05-08T18:04:05.188 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:02:14 ago on Wed 08 May 2024 06:01:51 PM UTC. 2024-05-08T18:04:05.316 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-08T18:04:05.317 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:04:05.317 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-05-08T18:04:05.317 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:04:05.317 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-08T18:04:05.317 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd x86_64 2:19.0.0-3525.g27d37168.el9 ceph 170 k 2024-05-08T18:04:05.317 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:05.318 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-08T18:04:05.318 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-08T18:04:05.318 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-05-08T18:04:05.318 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:05.318 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 170 k 2024-05-08T18:04:05.318 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 494 k 2024-05-08T18:04:05.318 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-08T18:04:05.751 INFO:teuthology.orchestra.run.smithi046.stdout:rbd-nbd-19.0.0-3525.g27d37168.el9.x86_64.rpm 394 kB/s | 170 kB 00:00 2024-05-08T18:04:05.752 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-08T18:04:05.752 INFO:teuthology.orchestra.run.smithi046.stdout:Total 392 kB/s | 170 kB 00:00 2024-05-08T18:04:05.752 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-08T18:04:05.761 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-08T18:04:05.761 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-08T18:04:05.828 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-08T18:04:05.829 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-08T18:04:05.990 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-08T18:04:06.085 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-nbd-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:04:06.490 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-nbd-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:04:06.763 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-nbd-2:19.0.0-3525.g27d37168.el9.x86_64 1/1 2024-05-08T18:04:06.763 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:06.763 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-08T18:04:06.763 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd-2:19.0.0-3525.g27d37168.el9.x86_64 2024-05-08T18:04:06.763 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:06.763 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-08T18:04:06.987 DEBUG:teuthology.parallel:result is None 2024-05-08T18:04:06.988 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=27d371685027569bcfa4efc661d3d3186f3a1776 2024-05-08T18:04:07.129 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T18:04:07.152 INFO:teuthology.orchestra.run.smithi046.stdout:19.0.0-3525.g27d37168.el9 2024-05-08T18:04:07.152 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3525.g27d37168.el9 2024-05-08T18:04:07.152 INFO:teuthology.task.install:The correct ceph version 19.0.0-3525.g27d37168 is installed. 2024-05-08T18:04:07.154 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=27d371685027569bcfa4efc661d3d3186f3a1776 2024-05-08T18:04:07.375 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T18:04:07.401 INFO:teuthology.orchestra.run.smithi050.stdout:19.0.0-3525.g27d37168.el9 2024-05-08T18:04:07.402 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3525.g27d37168.el9 2024-05-08T18:04:07.402 INFO:teuthology.task.install:The correct ceph version 19.0.0-3525.g27d37168 is installed. 2024-05-08T18:04:07.404 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T18:04:07.404 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T18:04:07.404 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T18:04:07.431 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T18:04:07.431 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T18:04:07.472 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T18:04:07.473 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T18:04:07.473 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T18:04:07.500 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T18:04:07.566 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T18:04:07.567 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T18:04:07.596 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T18:04:07.665 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T18:04:07.665 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T18:04:07.665 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T18:04:07.691 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T18:04:07.769 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T18:04:07.769 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T18:04:07.797 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T18:04:07.866 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T18:04:07.867 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T18:04:07.867 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T18:04:07.894 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T18:04:07.960 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T18:04:07.960 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T18:04:07.989 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T18:04:08.057 INFO:teuthology.run_tasks:Running task ceph... 2024-05-08T18:04:08.148 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-08T18:04:08.149 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /var/log/ceph 2024-05-08T18:04:08.151 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 777 /var/log/ceph 2024-05-08T18:04:08.178 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-08T18:04:08.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T18:04:08.218 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T18:04:08.245 INFO:tasks.ceph:Creating extra log directories... 2024-05-08T18:04:08.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T18:04:08.286 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T18:04:08.322 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-08T18:04:08.322 INFO:tasks.ceph:config {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-08T18:04:08.322 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/ivancich-2024-05-08_17:43:43-rgw-wip-eric-testing-1-distro-default-smithi/7697857', 'branch': 'wip-eric-testing-1', 'description': 'rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_latest}}', 'email': 'ivancich@redhat.com', 'first_in_suite': False, 'job_id': '7697857', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'ivancich-2024-05-08_17:43:43-rgw-wip-eric-testing-1-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-eric-testing-1'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': '27d371685027569bcfa4efc661d3d3186f3a1776'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '27d371685027569bcfa4efc661d3d3186f3a1776'}}, 'rgw': {'ec-data-pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 3, 'm': 1, 'name': 'testprofile'}, 'frontend': 'beast'}, 's3tests': {'slow_backend': True}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-eric-testing-1', 'sha1': '27d371685027569bcfa4efc661d3d3186f3a1776'}}, 'owner': 'scheduled_ivancich@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0'], ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3']], 'seed': 831, 'sha1': '27d371685027569bcfa4efc661d3d3186f3a1776', 'sleep_before_teardown': 0, 'subset': '1/50', 'suite': 'rgw', 'suite_branch': 'wip-eric-testing-1', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_27d371685027569bcfa4efc661d3d3186f3a1776/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '27d371685027569bcfa4efc661d3d3186f3a1776', 'targets': {'smithi046.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJr4zEpCGn//LIxU7aGC/o5MVrcwMvj4d93v+lvCqZIHfWrWTiAHriNdCD5BWjC8YWvxXSHcVyN9fyPIQXuI6Ag=', 'smithi050.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEGj53oQeZ4et4D00re4FlURSj4VUvY3hlLD8oeqLymTYvDOiX4DMqjF3HRHVKwY+g2DKmAoy+04IF8euge4zm0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': '27d371685027569bcfa4efc661d3d3186f3a1776', 'cluster': 'ceph'}}, {'rgw': {'client.0': None}}, {'radosgw-admin': None}, {'radosgw-admin-rest': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-08_17:43:43', 'tube': 'smithi', 'user': 'ivancich', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-08T18:04:08.323 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T18:04:08.353 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T18:04:08.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T18:04:08.410 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T18:04:08.436 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T18:04:08.437 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T18:04:08.492 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T18:04:08.493 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-05-08T18:04:08.549 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T18:04:08.550 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T18:04:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-08T18:04:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T18:04:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T18:04:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-08 18:04:03.425774062 +0000 2024-05-08T18:04:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-08 18:01:04.516737062 +0000 2024-05-08T18:04:08.550 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-08 18:01:04.516737062 +0000 2024-05-08T18:04:08.550 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-08 18:01:04.516737062 +0000 2024-05-08T18:04:08.551 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T18:04:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-05-08T18:04:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-05-08T18:04:08.615 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000198577 s, 2.6 MB/s 2024-05-08T18:04:08.617 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T18:04:08.674 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-05-08T18:04:08.731 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T18:04:08.732 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T18:04:08.732 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-08T18:04:08.732 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T18:04:08.732 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T18:04:08.732 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-08 18:04:03.426774051 +0000 2024-05-08T18:04:08.733 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-08 18:01:04.879733277 +0000 2024-05-08T18:04:08.733 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-08 18:01:04.879733277 +0000 2024-05-08T18:04:08.733 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-08 18:01:04.879733277 +0000 2024-05-08T18:04:08.733 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T18:04:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-05-08T18:04:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-05-08T18:04:08.797 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000127849 s, 4.0 MB/s 2024-05-08T18:04:08.798 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T18:04:08.856 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-05-08T18:04:08.912 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T18:04:08.912 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T18:04:08.912 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-08T18:04:08.912 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T18:04:08.912 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T18:04:08.912 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-08 18:04:03.426774051 +0000 2024-05-08T18:04:08.913 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-08 18:01:05.273729168 +0000 2024-05-08T18:04:08.913 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-08 18:01:05.273729168 +0000 2024-05-08T18:04:08.913 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-08 18:01:05.273729168 +0000 2024-05-08T18:04:08.913 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T18:04:08.977 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-05-08T18:04:08.977 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-05-08T18:04:08.977 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 9.7886e-05 s, 5.2 MB/s 2024-05-08T18:04:08.978 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T18:04:09.036 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-05-08T18:04:09.093 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T18:04:09.093 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T18:04:09.094 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-08T18:04:09.094 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T18:04:09.094 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T18:04:09.094 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-08 18:04:03.427774041 +0000 2024-05-08T18:04:09.094 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-08 18:01:05.637725373 +0000 2024-05-08T18:04:09.094 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-08 18:01:05.637725373 +0000 2024-05-08T18:04:09.094 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-08 18:01:05.637725373 +0000 2024-05-08T18:04:09.094 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T18:04:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-05-08T18:04:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-05-08T18:04:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000106203 s, 4.8 MB/s 2024-05-08T18:04:09.160 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T18:04:09.218 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-05-08T18:04:09.219 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T18:04:09.219 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T18:04:09.237 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T18:04:09.237 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_1 2024-05-08T18:04:09.296 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T18:04:09.296 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T18:04:09.296 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-05-08T18:04:09.296 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T18:04:09.296 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T18:04:09.296 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-08 18:03:58.778795761 +0000 2024-05-08T18:04:09.297 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-08 18:01:03.962214779 +0000 2024-05-08T18:04:09.297 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-08 18:01:03.962214779 +0000 2024-05-08T18:04:09.297 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-08 18:01:03.962214779 +0000 2024-05-08T18:04:09.297 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T18:04:09.364 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-05-08T18:04:09.364 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-05-08T18:04:09.364 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 9.4361e-05 s, 5.4 MB/s 2024-05-08T18:04:09.365 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T18:04:09.426 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_2 2024-05-08T18:04:09.484 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T18:04:09.484 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T18:04:09.484 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-08T18:04:09.484 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T18:04:09.484 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T18:04:09.484 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-08 18:03:58.779795736 +0000 2024-05-08T18:04:09.484 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-08 18:01:04.334205089 +0000 2024-05-08T18:04:09.484 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-08 18:01:04.334205089 +0000 2024-05-08T18:04:09.484 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-08 18:01:04.334205089 +0000 2024-05-08T18:04:09.484 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T18:04:09.549 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-05-08T18:04:09.549 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-05-08T18:04:09.549 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000107361 s, 4.8 MB/s 2024-05-08T18:04:09.551 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T18:04:09.608 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_3 2024-05-08T18:04:09.665 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T18:04:09.665 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T18:04:09.666 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-08T18:04:09.666 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T18:04:09.666 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T18:04:09.666 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-08 18:03:58.779795736 +0000 2024-05-08T18:04:09.666 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-08 18:01:04.752194203 +0000 2024-05-08T18:04:09.666 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-08 18:01:04.752194203 +0000 2024-05-08T18:04:09.666 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-08 18:01:04.752194203 +0000 2024-05-08T18:04:09.666 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T18:04:09.731 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-05-08T18:04:09.731 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-05-08T18:04:09.731 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000107294 s, 4.8 MB/s 2024-05-08T18:04:09.732 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T18:04:09.790 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_4 2024-05-08T18:04:09.846 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T18:04:09.846 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T18:04:09.846 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-08T18:04:09.846 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T18:04:09.846 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T18:04:09.847 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-05-08 18:03:58.779795736 +0000 2024-05-08T18:04:09.847 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-05-08 18:01:05.111184853 +0000 2024-05-08T18:04:09.847 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-05-08 18:01:05.111184853 +0000 2024-05-08T18:04:09.847 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-05-08 18:01:05.111184853 +0000 2024-05-08T18:04:09.847 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T18:04:09.911 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-05-08T18:04:09.912 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-05-08T18:04:09.912 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 9.3671e-05 s, 5.5 MB/s 2024-05-08T18:04:09.913 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T18:04:09.969 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-05-08T18:04:09.970 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi050.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-05-08T18:04:09.970 INFO:tasks.ceph:Generating config... 2024-05-08T18:04:09.971 INFO:tasks.ceph:[client] debug ms = 1 2024-05-08T18:04:09.971 INFO:tasks.ceph:[client] rgw gc obj min wait = 15 2024-05-08T18:04:09.971 INFO:tasks.ceph:[client] debug rgw = 20 2024-05-08T18:04:09.971 INFO:tasks.ceph:[client] setgroup = ceph 2024-05-08T18:04:09.972 INFO:tasks.ceph:[client] setuser = ceph 2024-05-08T18:04:09.972 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-08T18:04:09.972 INFO:tasks.ceph:[osd] debug objclass = 20 2024-05-08T18:04:09.972 INFO:tasks.ceph:[osd] bdev async discard = True 2024-05-08T18:04:09.972 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-05-08T18:04:09.972 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-05-08T18:04:09.972 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-08T18:04:09.972 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-08T18:04:09.972 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-08T18:04:09.972 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-08T18:04:09.972 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-08T18:04:09.972 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-08T18:04:09.972 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-08T18:04:09.972 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-08T18:04:09.972 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-08T18:04:09.972 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-08T18:04:09.973 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-08T18:04:09.973 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-08T18:04:09.973 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-08T18:04:09.973 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-08T18:04:09.973 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-08T18:04:09.973 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-08T18:04:09.973 INFO:tasks.ceph:Setting up mon.a... 2024-05-08T18:04:09.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-08T18:04:10.028 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.keyring 2024-05-08T18:04:10.036 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-08T18:04:10.085 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T18:04:10.112 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.46')] 2024-05-08T18:04:10.113 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.46'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}} 2024-05-08T18:04:10.114 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T18:04:10.114 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T18:04:10.170 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.46 --print /home/ubuntu/cephtest/ceph.monmap 2024-05-08T18:04:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:04:10.253 INFO:teuthology.orchestra.run.smithi046.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:04:10.254 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-08T18:04:10.254 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: generated fsid d8088d1f-616e-46b1-995a-85e572267d38 2024-05-08T18:04:10.254 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = quincy 2024-05-08T18:04:10.254 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 0 2024-05-08T18:04:10.254 INFO:teuthology.orchestra.run.smithi046.stdout:fsid d8088d1f-616e-46b1-995a-85e572267d38 2024-05-08T18:04:10.254 INFO:teuthology.orchestra.run.smithi046.stdout:last_changed 2024-05-08T18:04:10.256692+0000 2024-05-08T18:04:10.255 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-05-08T18:04:10.256692+0000 2024-05-08T18:04:10.255 INFO:teuthology.orchestra.run.smithi046.stdout:min_mon_release 17 (quincy) 2024-05-08T18:04:10.255 INFO:teuthology.orchestra.run.smithi046.stdout:election_strategy: 1 2024-05-08T18:04:10.255 INFO:teuthology.orchestra.run.smithi046.stdout:0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-08T18:04:10.255 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-05-08T18:04:10.256 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T18:04:10.313 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d8088d1f-616e-46b1-995a-85e572267d38... 2024-05-08T18:04:10.314 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T18:04:10.356 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T18:04:10.401 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-05-08T18:04:10.401 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2024-05-08T18:04:10.401 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T18:04:10.401 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2024-05-08T18:04:10.401 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.401 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2024-05-08T18:04:10.401 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.402 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2024-05-08T18:04:10.402 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.402 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2024-05-08T18:04:10.402 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2024-05-08T18:04:10.402 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.402 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2024-05-08T18:04:10.402 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2024-05-08T18:04:10.402 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.402 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T18:04:10.402 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2024-05-08T18:04:10.402 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.402 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T18:04:10.402 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T18:04:10.402 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.403 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2024-05-08T18:04:10.403 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.403 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2024-05-08T18:04:10.403 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2024-05-08T18:04:10.403 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2024-05-08T18:04:10.403 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2024-05-08T18:04:10.403 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2024-05-08T18:04:10.403 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2024-05-08T18:04:10.403 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2024-05-08T18:04:10.403 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T18:04:10.403 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T18:04:10.403 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2024-05-08T18:04:10.403 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.403 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T18:04:10.403 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.403 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2024-05-08T18:04:10.404 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.404 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2024-05-08T18:04:10.404 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.404 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2024-05-08T18:04:10.404 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2024-05-08T18:04:10.404 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2024-05-08T18:04:10.404 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = 172.21.15.46 2024-05-08T18:04:10.404 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = d8088d1f-616e-46b1-995a-85e572267d38 2024-05-08T18:04:10.404 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.404 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2024-05-08T18:04:10.404 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2024-05-08T18:04:10.404 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.404 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2024-05-08T18:04:10.404 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2024-05-08T18:04:10.405 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2024-05-08T18:04:10.405 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.405 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2024-05-08T18:04:10.405 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2024-05-08T18:04:10.405 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.405 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2024-05-08T18:04:10.405 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2024-05-08T18:04:10.405 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2024-05-08T18:04:10.405 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug trim objects = true 2024-05-08T18:04:10.405 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.405 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2024-05-08T18:04:10.405 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2024-05-08T18:04:10.405 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.405 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2024-05-08T18:04:10.405 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.405 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2024-05-08T18:04:10.406 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.406 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2024-05-08T18:04:10.406 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.406 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2024-05-08T18:04:10.406 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2024-05-08T18:04:10.406 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.406 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2024-05-08T18:04:10.406 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2024-05-08T18:04:10.406 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-05-08T18:04:10.406 INFO:teuthology.orchestra.run.smithi046.stdout: debug objclass = 20 2024-05-08T18:04:10.406 INFO:teuthology.orchestra.run.smithi046.stdout: bdev async discard = True 2024-05-08T18:04:10.406 INFO:teuthology.orchestra.run.smithi046.stdout: bdev enable discard = True 2024-05-08T18:04:10.406 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore allocator = bitmap 2024-05-08T18:04:10.406 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2024-05-08T18:04:10.406 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2024-05-08T18:04:10.407 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 1/20 2024-05-08T18:04:10.407 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 1/20 2024-05-08T18:04:10.407 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2024-05-08T18:04:10.407 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 4/10 2024-05-08T18:04:10.407 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T18:04:10.407 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2024-05-08T18:04:10.407 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2024-05-08T18:04:10.407 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2024-05-08T18:04:10.407 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2024-05-08T18:04:10.407 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.407 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2024-05-08T18:04:10.407 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-05-08T18:04:10.407 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2024-05-08T18:04:10.407 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-05-08T18:04:10.408 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-05-08T18:04:10.408 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-05-08T18:04:10.408 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-05-08T18:04:10.408 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2024-05-08T18:04:10.408 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.408 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2024-05-08T18:04:10.408 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-05-08T18:04:10.408 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2024-05-08T18:04:10.408 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2024-05-08T18:04:10.408 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2024-05-08T18:04:10.408 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2024-05-08T18:04:10.408 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2024-05-08T18:04:10.408 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2024-05-08T18:04:10.408 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2024-05-08T18:04:10.408 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2024-05-08T18:04:10.408 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2024-05-08T18:04:10.409 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2024-05-08T18:04:10.410 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2024-05-08T18:04:10.410 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T18:04:10.410 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T18:04:10.410 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2024-05-08T18:04:10.410 INFO:teuthology.orchestra.run.smithi046.stdout: rgw gc obj min wait = 15 2024-05-08T18:04:10.410 INFO:teuthology.orchestra.run.smithi046.stdout: debug rgw = 20 2024-05-08T18:04:10.410 INFO:teuthology.orchestra.run.smithi046.stdout: setgroup = ceph 2024-05-08T18:04:10.410 INFO:teuthology.orchestra.run.smithi046.stdout: setuser = ceph 2024-05-08T18:04:10.410 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2024-05-08T18:04:10.410 INFO:teuthology.orchestra.run.smithi050.stdout:[global] 2024-05-08T18:04:10.411 INFO:teuthology.orchestra.run.smithi050.stdout: chdir = "" 2024-05-08T18:04:10.411 INFO:teuthology.orchestra.run.smithi050.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T18:04:10.411 INFO:teuthology.orchestra.run.smithi050.stdout: auth supported = cephx 2024-05-08T18:04:10.411 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.411 INFO:teuthology.orchestra.run.smithi050.stdout: filestore xattr use omap = true 2024-05-08T18:04:10.411 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.411 INFO:teuthology.orchestra.run.smithi050.stdout: mon clock drift allowed = 1.000 2024-05-08T18:04:10.411 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.411 INFO:teuthology.orchestra.run.smithi050.stdout: osd crush chooseleaf type = 0 2024-05-08T18:04:10.411 INFO:teuthology.orchestra.run.smithi050.stdout: auth debug = true 2024-05-08T18:04:10.411 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.411 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on old message = true 2024-05-08T18:04:10.411 INFO:teuthology.orchestra.run.smithi050.stdout: ms die on bug = true 2024-05-08T18:04:10.411 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.412 INFO:teuthology.orchestra.run.smithi050.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T18:04:10.412 INFO:teuthology.orchestra.run.smithi050.stdout: mon pg warn max object skew = 0 2024-05-08T18:04:10.412 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.412 INFO:teuthology.orchestra.run.smithi050.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T18:04:10.412 INFO:teuthology.orchestra.run.smithi050.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T18:04:10.412 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.412 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default size = 2 2024-05-08T18:04:10.412 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.412 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow primary affinity = true 2024-05-08T18:04:10.412 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd allow pg remap = true 2024-05-08T18:04:10.412 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on legacy crush tunables = false 2024-05-08T18:04:10.412 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on crush straw calc version zero = false 2024-05-08T18:04:10.412 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on no sortbitwise = false 2024-05-08T18:04:10.412 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on osd down out interval zero = false 2024-05-08T18:04:10.412 INFO:teuthology.orchestra.run.smithi050.stdout: mon warn on too few osds = false 2024-05-08T18:04:10.412 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T18:04:10.413 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T18:04:10.413 INFO:teuthology.orchestra.run.smithi050.stdout: mon_allow_pool_size_one = true 2024-05-08T18:04:10.413 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.413 INFO:teuthology.orchestra.run.smithi050.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T18:04:10.413 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.413 INFO:teuthology.orchestra.run.smithi050.stdout: osd default data pool replay window = 5 2024-05-08T18:04:10.413 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.413 INFO:teuthology.orchestra.run.smithi050.stdout: mon allow pool delete = true 2024-05-08T18:04:10.413 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.413 INFO:teuthology.orchestra.run.smithi050.stdout: mon cluster log file level = debug 2024-05-08T18:04:10.413 INFO:teuthology.orchestra.run.smithi050.stdout: debug asserts on shutdown = true 2024-05-08T18:04:10.413 INFO:teuthology.orchestra.run.smithi050.stdout: mon health detail to clog = false 2024-05-08T18:04:10.413 INFO:teuthology.orchestra.run.smithi050.stdout: mon host = 172.21.15.46 2024-05-08T18:04:10.413 INFO:teuthology.orchestra.run.smithi050.stdout: fsid = d8088d1f-616e-46b1-995a-85e572267d38 2024-05-08T18:04:10.413 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.413 INFO:teuthology.orchestra.run.smithi050.stdout:[osd] 2024-05-08T18:04:10.414 INFO:teuthology.orchestra.run.smithi050.stdout: osd journal size = 100 2024-05-08T18:04:10.414 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.414 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub load threshold = 5.0 2024-05-08T18:04:10.414 INFO:teuthology.orchestra.run.smithi050.stdout: osd scrub max interval = 600 2024-05-08T18:04:10.414 INFO:teuthology.orchestra.run.smithi050.stdout: osd mclock profile = high_recovery_ops 2024-05-08T18:04:10.414 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.414 INFO:teuthology.orchestra.run.smithi050.stdout: osd recover clone overlap = true 2024-05-08T18:04:10.414 INFO:teuthology.orchestra.run.smithi050.stdout: osd recovery max chunk = 1048576 2024-05-08T18:04:10.414 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.414 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug shutdown = true 2024-05-08T18:04:10.414 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug op order = true 2024-05-08T18:04:10.414 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug verify stray on activate = true 2024-05-08T18:04:10.414 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug trim objects = true 2024-05-08T18:04:10.414 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.414 INFO:teuthology.orchestra.run.smithi050.stdout: osd open classes on start = true 2024-05-08T18:04:10.415 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug pg log writeout = true 2024-05-08T18:04:10.415 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.415 INFO:teuthology.orchestra.run.smithi050.stdout: osd deep scrub update digest min age = 30 2024-05-08T18:04:10.415 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.415 INFO:teuthology.orchestra.run.smithi050.stdout: osd map max advance = 10 2024-05-08T18:04:10.415 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.415 INFO:teuthology.orchestra.run.smithi050.stdout: journal zero on create = true 2024-05-08T18:04:10.415 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.415 INFO:teuthology.orchestra.run.smithi050.stdout: filestore ondisk finisher threads = 3 2024-05-08T18:04:10.415 INFO:teuthology.orchestra.run.smithi050.stdout: filestore apply finisher threads = 3 2024-05-08T18:04:10.415 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.415 INFO:teuthology.orchestra.run.smithi050.stdout: bdev debug aio = true 2024-05-08T18:04:10.415 INFO:teuthology.orchestra.run.smithi050.stdout: osd debug misdirected ops = true 2024-05-08T18:04:10.415 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2024-05-08T18:04:10.415 INFO:teuthology.orchestra.run.smithi050.stdout: debug objclass = 20 2024-05-08T18:04:10.416 INFO:teuthology.orchestra.run.smithi050.stdout: bdev async discard = True 2024-05-08T18:04:10.416 INFO:teuthology.orchestra.run.smithi050.stdout: bdev enable discard = True 2024-05-08T18:04:10.416 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore allocator = bitmap 2024-05-08T18:04:10.416 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore block size = 96636764160 2024-05-08T18:04:10.416 INFO:teuthology.orchestra.run.smithi050.stdout: bluestore fsck on mount = True 2024-05-08T18:04:10.416 INFO:teuthology.orchestra.run.smithi050.stdout: debug bluefs = 1/20 2024-05-08T18:04:10.416 INFO:teuthology.orchestra.run.smithi050.stdout: debug bluestore = 1/20 2024-05-08T18:04:10.416 INFO:teuthology.orchestra.run.smithi050.stdout: debug osd = 20 2024-05-08T18:04:10.416 INFO:teuthology.orchestra.run.smithi050.stdout: debug rocksdb = 4/10 2024-05-08T18:04:10.416 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T18:04:10.416 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd full ratio = 0.9 2024-05-08T18:04:10.416 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd nearfull ratio = 0.8 2024-05-08T18:04:10.416 INFO:teuthology.orchestra.run.smithi050.stdout: osd failsafe full ratio = 0.95 2024-05-08T18:04:10.416 INFO:teuthology.orchestra.run.smithi050.stdout: osd objectstore = bluestore 2024-05-08T18:04:10.416 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.417 INFO:teuthology.orchestra.run.smithi050.stdout:[mgr] 2024-05-08T18:04:10.417 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2024-05-08T18:04:10.417 INFO:teuthology.orchestra.run.smithi050.stdout: debug mgr = 20 2024-05-08T18:04:10.417 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2024-05-08T18:04:10.417 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2024-05-08T18:04:10.417 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2024-05-08T18:04:10.417 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2024-05-08T18:04:10.417 INFO:teuthology.orchestra.run.smithi050.stdout: mgr/telemetry/nag = false 2024-05-08T18:04:10.417 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.417 INFO:teuthology.orchestra.run.smithi050.stdout:[mon] 2024-05-08T18:04:10.417 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2024-05-08T18:04:10.417 INFO:teuthology.orchestra.run.smithi050.stdout: debug mon = 20 2024-05-08T18:04:10.417 INFO:teuthology.orchestra.run.smithi050.stdout: debug paxos = 20 2024-05-08T18:04:10.417 INFO:teuthology.orchestra.run.smithi050.stdout: debug auth = 20 2024-05-08T18:04:10.417 INFO:teuthology.orchestra.run.smithi050.stdout: mon data avail warn = 5 2024-05-08T18:04:10.417 INFO:teuthology.orchestra.run.smithi050.stdout: mon mgr mkfs grace = 240 2024-05-08T18:04:10.418 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min pgs per osd = 4 2024-05-08T18:04:10.418 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd reporter subtree level = osd 2024-05-08T18:04:10.418 INFO:teuthology.orchestra.run.smithi050.stdout: mon osd prime pg temp = true 2024-05-08T18:04:10.418 INFO:teuthology.orchestra.run.smithi050.stdout: mon reweight min bytes per osd = 10 2024-05-08T18:04:10.418 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.418 INFO:teuthology.orchestra.run.smithi050.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T18:04:10.418 INFO:teuthology.orchestra.run.smithi050.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T18:04:10.418 INFO:teuthology.orchestra.run.smithi050.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T18:04:10.418 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.418 INFO:teuthology.orchestra.run.smithi050.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T18:04:10.418 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T18:04:10.418 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T18:04:10.418 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.418 INFO:teuthology.orchestra.run.smithi050.stdout: # 1m isn't quite enough 2024-05-08T18:04:10.418 INFO:teuthology.orchestra.run.smithi050.stdout: mon_down_mkfs_grace = 2m 2024-05-08T18:04:10.419 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.419 INFO:teuthology.orchestra.run.smithi050.stdout: mon_warn_on_filestore_osds = false 2024-05-08T18:04:10.419 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-05-08T18:04:10.419 INFO:teuthology.orchestra.run.smithi050.stdout:[client] 2024-05-08T18:04:10.419 INFO:teuthology.orchestra.run.smithi050.stdout: rgw cache enabled = true 2024-05-08T18:04:10.419 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable ops log = true 2024-05-08T18:04:10.419 INFO:teuthology.orchestra.run.smithi050.stdout: rgw enable usage log = true 2024-05-08T18:04:10.419 INFO:teuthology.orchestra.run.smithi050.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T18:04:10.419 INFO:teuthology.orchestra.run.smithi050.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T18:04:10.419 INFO:teuthology.orchestra.run.smithi050.stdout: debug ms = 1 2024-05-08T18:04:10.419 INFO:teuthology.orchestra.run.smithi050.stdout: rgw gc obj min wait = 15 2024-05-08T18:04:10.419 INFO:teuthology.orchestra.run.smithi050.stdout: debug rgw = 20 2024-05-08T18:04:10.419 INFO:teuthology.orchestra.run.smithi050.stdout: setgroup = ceph 2024-05-08T18:04:10.419 INFO:teuthology.orchestra.run.smithi050.stdout: setuser = ceph 2024-05-08T18:04:10.419 INFO:teuthology.orchestra.run.smithi050.stdout:[mon.a] 2024-05-08T18:04:10.421 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-08T18:04:10.421 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-08T18:04:10.517 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-08T18:04:10.517 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T18:04:10.517 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-08T18:04:10.535 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T18:04:10.535 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-08T18:04:10.592 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T18:04:10.592 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T18:04:10.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T18:04:10.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T18:04:10.670 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T18:04:10.671 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T18:04:10.727 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi050.front.sepia.ceph.com 2024-05-08T18:04:10.727 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T18:04:10.727 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T18:04:10.727 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T18:04:10.764 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T18:04:10.764 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T18:04:10.821 INFO:tasks.ceph:Setting up mon nodes... 2024-05-08T18:04:10.821 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-08T18:04:10.821 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T18:04:10.920 INFO:teuthology.orchestra.run.smithi050.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T18:04:10.922 INFO:tasks.ceph:Setting up mds nodes... 2024-05-08T18:04:10.922 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-08T18:04:10.923 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T18:04:10.965 INFO:teuthology.orchestra.run.smithi050.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-08T18:04:10.978 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-08T18:04:10.978 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi050.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-05-08T18:04:10.978 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-08T18:04:11.004 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-05-08T18:04:11.004 INFO:tasks.ceph:role: osd.0 2024-05-08T18:04:11.004 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T18:04:11.004 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T18:04:11.069 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T18:04:11.069 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T18:04:11.069 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T18:04:11.069 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T18:04:11.069 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T18:04:11.069 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2024-05-08T18:04:11.069 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T18:04:11.070 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T18:04:11.070 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T18:04:11.070 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T18:04:11.101 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2024-05-08T18:04:11.103 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2024-05-08T18:04:11.103 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-08T18:04:11.173 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-08T18:04:11.246 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T18:04:11.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:11.342+0000 7f85707a5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-08T18:04:11.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:11.343+0000 7f85707a5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-08T18:04:11.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:11.343+0000 7f85707a5640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T18:04:11.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:11.343+0000 7f85707a5640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T18:04:11.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:11.343+0000 7f85707a5640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T18:04:11.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:11.343+0000 7f85707a5640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-08T18:04:14.195 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-08T18:04:14.226 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-08T18:04:14.259 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-05-08T18:04:14.259 INFO:tasks.ceph:role: osd.1 2024-05-08T18:04:14.259 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi050.front.sepia.ceph.com 2024-05-08T18:04:14.259 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T18:04:14.329 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T18:04:14.330 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T18:04:14.330 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T18:04:14.330 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T18:04:14.330 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T18:04:14.330 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2024-05-08T18:04:14.330 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T18:04:14.330 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T18:04:14.330 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T18:04:14.330 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T18:04:14.362 INFO:teuthology.orchestra.run.smithi050.stdout:Discarding blocks...Done. 2024-05-08T18:04:14.364 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2024-05-08T18:04:14.364 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-05-08T18:04:14.436 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-08T18:04:14.508 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-08T18:04:14.576 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-05-08T18:04:14.576 INFO:tasks.ceph:role: osd.2 2024-05-08T18:04:14.576 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi050.front.sepia.ceph.com 2024-05-08T18:04:14.576 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T18:04:14.643 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T18:04:14.643 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T18:04:14.643 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T18:04:14.643 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T18:04:14.643 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T18:04:14.643 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2024-05-08T18:04:14.643 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T18:04:14.643 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T18:04:14.643 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T18:04:14.643 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T18:04:14.675 INFO:teuthology.orchestra.run.smithi050.stdout:Discarding blocks...Done. 2024-05-08T18:04:14.678 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2024-05-08T18:04:14.678 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2024-05-08T18:04:14.747 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-08T18:04:14.819 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-08T18:04:14.886 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-05-08T18:04:14.886 INFO:tasks.ceph:role: osd.3 2024-05-08T18:04:14.886 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi050.front.sepia.ceph.com 2024-05-08T18:04:14.886 DEBUG:teuthology.orchestra.run.smithi050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T18:04:14.950 INFO:teuthology.orchestra.run.smithi050.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T18:04:14.950 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T18:04:14.951 INFO:teuthology.orchestra.run.smithi050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T18:04:14.951 INFO:teuthology.orchestra.run.smithi050.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T18:04:14.951 INFO:teuthology.orchestra.run.smithi050.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T18:04:14.951 INFO:teuthology.orchestra.run.smithi050.stdout: = sunit=0 swidth=0 blks 2024-05-08T18:04:14.951 INFO:teuthology.orchestra.run.smithi050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T18:04:14.951 INFO:teuthology.orchestra.run.smithi050.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T18:04:14.951 INFO:teuthology.orchestra.run.smithi050.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T18:04:14.951 INFO:teuthology.orchestra.run.smithi050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T18:04:14.982 INFO:teuthology.orchestra.run.smithi050.stdout:Discarding blocks...Done. 2024-05-08T18:04:14.984 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi050.front.sepia.ceph.com -o noatime 2024-05-08T18:04:14.984 DEBUG:teuthology.orchestra.run.smithi050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2024-05-08T18:04:15.054 DEBUG:teuthology.orchestra.run.smithi050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-08T18:04:15.127 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T18:04:15.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:15.253+0000 7ff75682c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-08T18:04:15.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:15.253+0000 7ff75682c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-08T18:04:15.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:15.253+0000 7ff75682c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T18:04:15.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:15.253+0000 7ff75682c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T18:04:15.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:15.253+0000 7ff75682c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T18:04:15.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:15.253+0000 7ff75682c640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-08T18:04:18.103 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-08T18:04:18.134 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T18:04:18.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:18.229+0000 7f52c8a33640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-08T18:04:18.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:18.229+0000 7f52c8a33640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-08T18:04:18.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:18.229+0000 7f52c8a33640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T18:04:18.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:18.229+0000 7f52c8a33640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T18:04:18.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:18.229+0000 7f52c8a33640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T18:04:18.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:18.230+0000 7f52c8a33640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-08T18:04:21.079 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-08T18:04:21.109 DEBUG:teuthology.orchestra.run.smithi050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T18:04:21.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:21.203+0000 7fc4739dc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-08T18:04:21.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:21.203+0000 7fc4739dc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-08T18:04:21.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:21.203+0000 7fc4739dc640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T18:04:21.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:21.203+0000 7fc4739dc640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T18:04:21.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:21.203+0000 7fc4739dc640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T18:04:21.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:21.204+0000 7fc4739dc640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-08T18:04:24.056 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-08T18:04:24.086 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-08T18:04:24.086 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T18:04:24.087 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-08T18:04:24.118 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T18:04:24.118 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-08T18:04:24.156 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T18:04:24.156 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-08T18:04:24.224 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T18:04:24.225 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-08T18:04:24.292 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T18:04:24.292 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-08T18:04:24.361 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T18:04:24.361 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-08T18:04:24.417 INFO:tasks.ceph:Adding keys to all mons... 2024-05-08T18:04:24.418 DEBUG:teuthology.orchestra.run.smithi046:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T18:04:24.446 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.0] 2024-05-08T18:04:24.447 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCbvjtm0yx+FBAAxt2Po6O2CBYk1Be9KGUtyA== 2024-05-08T18:04:24.447 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.x] 2024-05-08T18:04:24.447 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCavjtmuIIENxAAGqaXCUvt0+qdtEIwsfI4Aw== 2024-05-08T18:04:24.447 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.1] 2024-05-08T18:04:24.447 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCfvjtmW8AiDxAA8uomTsyXykWPjdvggW1JCA== 2024-05-08T18:04:24.447 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.2] 2024-05-08T18:04:24.447 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCivjtmvYy8DRAAEdh4OghigVaoYpaEYKhqXw== 2024-05-08T18:04:24.447 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.3] 2024-05-08T18:04:24.447 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQClvjtmcgorDBAADJ96fdotdyOyByoZTWMkhQ== 2024-05-08T18:04:24.447 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-05-08T18:04:24.447 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCavjtm70qyORAA/kofd8jdynfvj2i1yX4Dzg== 2024-05-08T18:04:24.448 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T18:04:24.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T18:04:24.602 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T18:04:24.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T18:04:24.707 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T18:04:24.762 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T18:04:24.817 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-08T18:04:24.817 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-08T18:04:24.843 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T18:04:24.940 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-08T18:04:24.965 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T18:04:25.020 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-08T18:04:25.021 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T18:04:25.021 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T18:04:25.063 INFO:tasks.ceph.mon.a:Started 2024-05-08T18:04:25.064 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-08T18:04:25.064 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-08T18:04:25.064 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-08T18:04:25.066 INFO:tasks.ceph.mgr.x:Started 2024-05-08T18:04:25.066 INFO:tasks.ceph:Setting crush tunables to default 2024-05-08T18:04:25.067 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd crush tunables default 2024-05-08T18:04:25.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.171+0000 7f9e5ead5640 1 Processor -- start 2024-05-08T18:04:25.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.171+0000 7f9e5ead5640 1 -- start start 2024-05-08T18:04:25.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.171+0000 7f9e5ead5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9e5812cbb0 0x7f9e58134960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:25.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.171+0000 7f9e5ead5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e58071510 con 0x7f9e5812c0c0 2024-05-08T18:04:25.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.171+0000 7f9e5ead5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9e5812d000 con 0x7f9e5812cbb0 2024-05-08T18:04:25.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.171+0000 7f9e5c871640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9e5812c0c0 0x7f9e5812c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59160/0 (socket says 172.21.15.46:59160) 2024-05-08T18:04:25.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.171+0000 7f9e5c871640 1 -- 172.21.15.46:0/3739375867 learned_addr learned my addr 172.21.15.46:0/3739375867 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:25.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.171+0000 7f9e57fff640 1 --2- 172.21.15.46:0/3739375867 >> v2:172.21.15.46:3300/0 conn(0x7f9e5812cbb0 0x7f9e58134960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:25.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.172+0000 7f9e577fe640 1 -- 172.21.15.46:0/3739375867 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3200775705 0 0) 0x7f9e58071510 con 0x7f9e5812c0c0 2024-05-08T18:04:25.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.172+0000 7f9e577fe640 1 -- 172.21.15.46:0/3739375867 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e48003520 con 0x7f9e5812c0c0 2024-05-08T18:04:25.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.172+0000 7f9e57fff640 1 -- 172.21.15.46:0/3739375867 >> v1:172.21.15.46:6789/0 conn(0x7f9e5812c0c0 legacy=0x7f9e5812c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:25.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.172+0000 7f9e57fff640 1 -- 172.21.15.46:0/3739375867 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e58070fe0 con 0x7f9e5812cbb0 2024-05-08T18:04:25.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.173+0000 7f9e57fff640 1 --2- 172.21.15.46:0/3739375867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e5812cbb0 0x7f9e58134960 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009720 tx=0x7f9e4402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b541d08712e28d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:25.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.173+0000 7f9e577fe640 1 -- 172.21.15.46:0/3739375867 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e440316d0 con 0x7f9e5812cbb0 2024-05-08T18:04:25.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.173+0000 7f9e577fe640 1 -- 172.21.15.46:0/3739375867 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e44037070 con 0x7f9e5812cbb0 2024-05-08T18:04:25.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.173+0000 7f9e577fe640 1 -- 172.21.15.46:0/3739375867 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e440319c0 con 0x7f9e5812cbb0 2024-05-08T18:04:25.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.173+0000 7f9e5ead5640 1 -- 172.21.15.46:0/3739375867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e5812cbb0 msgr2=0x7f9e58134960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:25.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.173+0000 7f9e5ead5640 1 --2- 172.21.15.46:0/3739375867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e5812cbb0 0x7f9e58134960 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009720 tx=0x7f9e4402b840 comp rx=0 tx=0).stop 2024-05-08T18:04:25.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.173+0000 7f9e5ead5640 1 -- 172.21.15.46:0/3739375867 shutdown_connections 2024-05-08T18:04:25.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.173+0000 7f9e5ead5640 1 --2- 172.21.15.46:0/3739375867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e5812cbb0 0x7f9e58134960 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:25.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.173+0000 7f9e5ead5640 1 -- 172.21.15.46:0/3739375867 >> 172.21.15.46:0/3739375867 conn(0x7f9e5809e930 msgr2=0x7f9e5809ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:25.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.173+0000 7f9e5ead5640 1 -- 172.21.15.46:0/3739375867 shutdown_connections 2024-05-08T18:04:25.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.173+0000 7f9e5ead5640 1 -- 172.21.15.46:0/3739375867 wait complete. 2024-05-08T18:04:25.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.174+0000 7f9e5ead5640 1 Processor -- start 2024-05-08T18:04:25.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.174+0000 7f9e5ead5640 1 -- start start 2024-05-08T18:04:25.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.175+0000 7f9e5ead5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e5812c0c0 0x7f9e58130710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:25.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.175+0000 7f9e5ead5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58134ea0 con 0x7f9e5812c0c0 2024-05-08T18:04:25.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.175+0000 7f9e5c871640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e5812c0c0 0x7f9e58130710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:25.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.175+0000 7f9e5c871640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e5812c0c0 0x7f9e58130710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41436/0 (socket says 172.21.15.46:41436) 2024-05-08T18:04:25.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.175+0000 7f9e5c871640 1 -- 172.21.15.46:0/2704186113 learned_addr learned my addr 172.21.15.46:0/2704186113 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:25.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.175+0000 7f9e5c871640 1 -- 172.21.15.46:0/2704186113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e58133c90 con 0x7f9e5812c0c0 2024-05-08T18:04:25.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.176+0000 7f9e5c871640 1 --2- 172.21.15.46:0/2704186113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e5812c0c0 0x7f9e58130710 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00c480 tx=0x7f9e4c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:25.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.176+0000 7f9e557fa640 1 -- 172.21.15.46:0/2704186113 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e4c017070 con 0x7f9e5812c0c0 2024-05-08T18:04:25.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.176+0000 7f9e557fa640 1 -- 172.21.15.46:0/2704186113 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e4c005cd0 con 0x7f9e5812c0c0 2024-05-08T18:04:25.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.176+0000 7f9e5ead5640 1 -- 172.21.15.46:0/2704186113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e58132140 con 0x7f9e5812c0c0 2024-05-08T18:04:25.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.176+0000 7f9e557fa640 1 -- 172.21.15.46:0/2704186113 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e4c007200 con 0x7f9e5812c0c0 2024-05-08T18:04:25.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.176+0000 7f9e5ead5640 1 -- 172.21.15.46:0/2704186113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5806ffa0 con 0x7f9e5812c0c0 2024-05-08T18:04:25.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.176+0000 7f9e557fa640 1 -- 172.21.15.46:0/2704186113 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9e4c01a070 con 0x7f9e5812c0c0 2024-05-08T18:04:25.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.176+0000 7f9e557fa640 1 -- 172.21.15.46:0/2704186113 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9e4c012400 con 0x7f9e5812c0c0 2024-05-08T18:04:25.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.177+0000 7f9e5ead5640 1 -- 172.21.15.46:0/2704186113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e580640a0 con 0x7f9e5812c0c0 2024-05-08T18:04:25.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.179+0000 7f9e557fa640 1 -- 172.21.15.46:0/2704186113 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f9e4c01c440 con 0x7f9e5812c0c0 2024-05-08T18:04:25.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.245+0000 7f9e5ead5640 1 -- 172.21.15.46:0/2704186113 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f9e58132910 con 0x7f9e5812c0c0 2024-05-08T18:04:25.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.246+0000 7f9e557fa640 1 -- 172.21.15.46:0/2704186113 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f9e4c0073a0 con 0x7f9e5812c0c0 2024-05-08T18:04:25.244 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2024-05-08T18:04:25.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.247+0000 7f9e5ead5640 1 -- 172.21.15.46:0/2704186113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e5812c0c0 msgr2=0x7f9e58130710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:25.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.247+0000 7f9e5ead5640 1 --2- 172.21.15.46:0/2704186113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e5812c0c0 0x7f9e58130710 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00c480 tx=0x7f9e4c00c940 comp rx=0 tx=0).stop 2024-05-08T18:04:25.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.247+0000 7f9e5ead5640 1 -- 172.21.15.46:0/2704186113 shutdown_connections 2024-05-08T18:04:25.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.247+0000 7f9e5ead5640 1 --2- 172.21.15.46:0/2704186113 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e5812c0c0 0x7f9e58130710 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:25.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.247+0000 7f9e5ead5640 1 -- 172.21.15.46:0/2704186113 >> 172.21.15.46:0/2704186113 conn(0x7f9e5809e930 msgr2=0x7f9e5812afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:25.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.247+0000 7f9e5ead5640 1 -- 172.21.15.46:0/2704186113 shutdown_connections 2024-05-08T18:04:25.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:25.247+0000 7f9e5ead5640 1 -- 172.21.15.46:0/2704186113 wait complete. 2024-05-08T18:04:25.256 INFO:tasks.ceph:check_enable_crimson: False 2024-05-08T18:04:25.256 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-08T18:04:25.256 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-08T18:04:25.256 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-08T18:04:25.272 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:25.274+0000 7f9ee3e0e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T18:04:25.281 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T18:04:25.281 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-08T18:04:25.307 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T18:04:25.307 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-08T18:04:25.374 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-05-08T18:04:25.374 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-08T18:04:25.440 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new f7304d61-4d9e-455b-a968-5641c682e642 0 2024-05-08T18:04:25.441 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:25.443+0000 7f9ee3e0e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T18:04:25.499 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:25.501+0000 7f9ee3e0e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T18:04:25.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.544+0000 7f40a1b2b640 1 Processor -- start 2024-05-08T18:04:25.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.544+0000 7f40a1b2b640 1 -- start start 2024-05-08T18:04:25.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.545+0000 7f40a1b2b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f409c110d40 0x7f409c111120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:25.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.545+0000 7f40a1b2b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f409c111750 con 0x7f409c1156f0 2024-05-08T18:04:25.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.545+0000 7f40a1b2b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f409c1162b0 con 0x7f409c110d40 2024-05-08T18:04:25.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.545+0000 7f409b7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f409c110d40 0x7f409c111120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:25.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.545+0000 7f409b7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f409c110d40 0x7f409c111120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:59070/0 (socket says 172.21.15.50:59070) 2024-05-08T18:04:25.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.545+0000 7f409affd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f409c1156f0 0x7f409c115ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:49850/0 (socket says 172.21.15.50:49850) 2024-05-08T18:04:25.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.545+0000 7f409b7fe640 1 -- 172.21.15.50:0/1114620566 learned_addr learned my addr 172.21.15.50:0/1114620566 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:04:25.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.545+0000 7f409b7fe640 1 -- 172.21.15.50:0/1114620566 >> v1:172.21.15.46:6789/0 conn(0x7f409c1156f0 legacy=0x7f409c115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:25.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.545+0000 7f409b7fe640 1 -- 172.21.15.50:0/1114620566 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c116460 con 0x7f409c110d40 2024-05-08T18:04:25.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.546+0000 7f409b7fe640 1 --2- 172.21.15.50:0/1114620566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c110d40 0x7f409c111120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f408c0097f0 tx=0x7f408c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16bee48e43f18 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:25.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.546+0000 7f409a7fc640 1 -- 172.21.15.50:0/1114620566 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f408c035020 con 0x7f409c110d40 2024-05-08T18:04:25.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.546+0000 7f409a7fc640 1 -- 172.21.15.50:0/1114620566 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f408c0317a0 con 0x7f409c110d40 2024-05-08T18:04:25.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.546+0000 7f409a7fc640 1 -- 172.21.15.50:0/1114620566 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f408c031aa0 con 0x7f409c110d40 2024-05-08T18:04:25.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.547+0000 7f40a1b2b640 1 -- 172.21.15.50:0/1114620566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c110d40 msgr2=0x7f409c111120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:25.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.547+0000 7f40a1b2b640 1 --2- 172.21.15.50:0/1114620566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c110d40 0x7f409c111120 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f408c0097f0 tx=0x7f408c02b910 comp rx=0 tx=0).stop 2024-05-08T18:04:25.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.547+0000 7f40a1b2b640 1 -- 172.21.15.50:0/1114620566 shutdown_connections 2024-05-08T18:04:25.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.547+0000 7f40a1b2b640 1 --2- 172.21.15.50:0/1114620566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c110d40 0x7f409c111120 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:25.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.547+0000 7f40a1b2b640 1 -- 172.21.15.50:0/1114620566 >> 172.21.15.50:0/1114620566 conn(0x7f409c072360 msgr2=0x7f409c10ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:25.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.547+0000 7f40a1b2b640 1 -- 172.21.15.50:0/1114620566 shutdown_connections 2024-05-08T18:04:25.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.547+0000 7f40a1b2b640 1 -- 172.21.15.50:0/1114620566 wait complete. 2024-05-08T18:04:25.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.548+0000 7f40a1b2b640 1 Processor -- start 2024-05-08T18:04:25.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.548+0000 7f40a1b2b640 1 -- start start 2024-05-08T18:04:25.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.548+0000 7f40a1b2b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c110d40 0x7f409c06ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:25.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.548+0000 7f40a1b2b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f409c11ce00 con 0x7f409c110d40 2024-05-08T18:04:25.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.548+0000 7f409b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c110d40 0x7f409c06ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:25.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.549+0000 7f409b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c110d40 0x7f409c06ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:59086/0 (socket says 172.21.15.50:59086) 2024-05-08T18:04:25.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.549+0000 7f409b7fe640 1 -- 172.21.15.50:0/781702022 learned_addr learned my addr 172.21.15.50:0/781702022 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:04:25.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.549+0000 7f409b7fe640 1 -- 172.21.15.50:0/781702022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f409c06e410 con 0x7f409c110d40 2024-05-08T18:04:25.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.549+0000 7f409b7fe640 1 --2- 172.21.15.50:0/781702022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c110d40 0x7f409c06ded0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f408c0090b0 tx=0x7f408c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:25.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.549+0000 7f4083fff640 1 -- 172.21.15.50:0/781702022 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f408c049070 con 0x7f409c110d40 2024-05-08T18:04:25.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.549+0000 7f4083fff640 1 -- 172.21.15.50:0/781702022 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f408c031a80 con 0x7f409c110d40 2024-05-08T18:04:25.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.549+0000 7f40a1b2b640 1 -- 172.21.15.50:0/781702022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f409c06e6a0 con 0x7f409c110d40 2024-05-08T18:04:25.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.549+0000 7f4083fff640 1 -- 172.21.15.50:0/781702022 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f408c044550 con 0x7f409c110d40 2024-05-08T18:04:25.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.549+0000 7f40a1b2b640 1 -- 172.21.15.50:0/781702022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f409c06ebf0 con 0x7f409c110d40 2024-05-08T18:04:25.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.549+0000 7f4083fff640 1 -- 172.21.15.50:0/781702022 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f408c04c070 con 0x7f409c110d40 2024-05-08T18:04:25.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.549+0000 7f4083fff640 1 -- 172.21.15.50:0/781702022 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f408c04e520 con 0x7f409c110d40 2024-05-08T18:04:25.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.551+0000 7f40a1b2b640 1 -- 172.21.15.50:0/781702022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f409c051020 con 0x7f409c110d40 2024-05-08T18:04:25.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.553+0000 7f4083fff640 1 -- 172.21.15.50:0/781702022 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f408c03f070 con 0x7f409c110d40 2024-05-08T18:04:25.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.626+0000 7f40a1b2b640 1 -- 172.21.15.50:0/781702022 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f7304d61-4d9e-455b-a968-5641c682e642", "id": 0} v 0) -- 0x7f409c1156f0 con 0x7f409c110d40 2024-05-08T18:04:25.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.628+0000 7f4083fff640 1 -- 172.21.15.50:0/781702022 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f7304d61-4d9e-455b-a968-5641c682e642", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f408c031710 con 0x7f409c110d40 2024-05-08T18:04:25.626 INFO:teuthology.orchestra.run.smithi050.stdout:0 2024-05-08T18:04:25.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.629+0000 7f40a1b2b640 1 -- 172.21.15.50:0/781702022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c110d40 msgr2=0x7f409c06ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:25.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.629+0000 7f40a1b2b640 1 --2- 172.21.15.50:0/781702022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c110d40 0x7f409c06ded0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f408c0090b0 tx=0x7f408c0046c0 comp rx=0 tx=0).stop 2024-05-08T18:04:25.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.629+0000 7f40a1b2b640 1 -- 172.21.15.50:0/781702022 shutdown_connections 2024-05-08T18:04:25.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.629+0000 7f40a1b2b640 1 --2- 172.21.15.50:0/781702022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f409c110d40 0x7f409c06ded0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:25.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.629+0000 7f40a1b2b640 1 -- 172.21.15.50:0/781702022 >> 172.21.15.50:0/781702022 conn(0x7f409c072360 msgr2=0x7f409c11ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:25.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.629+0000 7f40a1b2b640 1 -- 172.21.15.50:0/781702022 shutdown_connections 2024-05-08T18:04:25.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.629+0000 7f40a1b2b640 1 -- 172.21.15.50:0/781702022 wait complete. 2024-05-08T18:04:25.637 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new 9a62c75d-b151-4453-a538-14408e8f4b78 1 2024-05-08T18:04:25.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.740+0000 7f21ac245640 1 Processor -- start 2024-05-08T18:04:25.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.741+0000 7f21ac245640 1 -- start start 2024-05-08T18:04:25.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.741+0000 7f21ac245640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f21a411bb70 0x7f21a411df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:25.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.741+0000 7f21ac245640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21a411e530 con 0x7f21a4118ef0 2024-05-08T18:04:25.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.741+0000 7f21ac245640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f21a411f730 con 0x7f21a411bb70 2024-05-08T18:04:25.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.741+0000 7f21a97e0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f21a411bb70 0x7f21a411df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:25.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.741+0000 7f21a9fe1640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f21a4118ef0 0x7f21a41192d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:49862/0 (socket says 172.21.15.50:49862) 2024-05-08T18:04:25.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.741+0000 7f21a9fe1640 1 -- 172.21.15.50:0/2455178886 learned_addr learned my addr 172.21.15.50:0/2455178886 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:04:25.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.741+0000 7f21a97e0640 1 -- 172.21.15.50:0/2455178886 >> v1:172.21.15.46:6789/0 conn(0x7f21a4118ef0 legacy=0x7f21a41192d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:25.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.741+0000 7f21a97e0640 1 -- 172.21.15.50:0/2455178886 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a411f8e0 con 0x7f21a411bb70 2024-05-08T18:04:25.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.742+0000 7f21a97e0640 1 --2- 172.21.15.50:0/2455178886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21a411bb70 0x7f21a411df60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2194009720 tx=0x7f219402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1ba0c599454ce3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:25.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.742+0000 7f21a8fdf640 1 -- 172.21.15.50:0/2455178886 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21940316d0 con 0x7f21a411bb70 2024-05-08T18:04:25.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.743+0000 7f21a8fdf640 1 -- 172.21.15.50:0/2455178886 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2194037070 con 0x7f21a411bb70 2024-05-08T18:04:25.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.743+0000 7f21a8fdf640 1 -- 172.21.15.50:0/2455178886 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21940319c0 con 0x7f21a411bb70 2024-05-08T18:04:25.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.743+0000 7f21ac245640 1 -- 172.21.15.50:0/2455178886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21a411bb70 msgr2=0x7f21a411df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:25.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.743+0000 7f21ac245640 1 --2- 172.21.15.50:0/2455178886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21a411bb70 0x7f21a411df60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2194009720 tx=0x7f219402b840 comp rx=0 tx=0).stop 2024-05-08T18:04:25.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.743+0000 7f21ac245640 1 -- 172.21.15.50:0/2455178886 shutdown_connections 2024-05-08T18:04:25.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.743+0000 7f21ac245640 1 --2- 172.21.15.50:0/2455178886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21a411bb70 0x7f21a411df60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:25.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.743+0000 7f21ac245640 1 -- 172.21.15.50:0/2455178886 >> 172.21.15.50:0/2455178886 conn(0x7f21a410ed90 msgr2=0x7f21a4111180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:25.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.743+0000 7f21ac245640 1 -- 172.21.15.50:0/2455178886 shutdown_connections 2024-05-08T18:04:25.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.743+0000 7f21ac245640 1 -- 172.21.15.50:0/2455178886 wait complete. 2024-05-08T18:04:25.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.744+0000 7f21ac245640 1 Processor -- start 2024-05-08T18:04:25.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.744+0000 7f21ac245640 1 -- start start 2024-05-08T18:04:25.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.744+0000 7f21ac245640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21a4118ef0 0x7f21a41a79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:25.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.744+0000 7f21ac245640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f21a411ebe0 con 0x7f21a4118ef0 2024-05-08T18:04:25.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.745+0000 7f21a9fe1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21a4118ef0 0x7f21a41a79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:25.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.745+0000 7f21a9fe1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21a4118ef0 0x7f21a41a79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:59106/0 (socket says 172.21.15.50:59106) 2024-05-08T18:04:25.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.745+0000 7f21a9fe1640 1 -- 172.21.15.50:0/4108641956 learned_addr learned my addr 172.21.15.50:0/4108641956 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:04:25.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.745+0000 7f21a9fe1640 1 -- 172.21.15.50:0/4108641956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a41a7f10 con 0x7f21a4118ef0 2024-05-08T18:04:25.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.745+0000 7f21a9fe1640 1 --2- 172.21.15.50:0/4108641956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21a4118ef0 0x7f21a41a79d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f21a000c480 tx=0x7f21a000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:25.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.745+0000 7f219a7fc640 1 -- 172.21.15.50:0/4108641956 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21a0017070 con 0x7f21a4118ef0 2024-05-08T18:04:25.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.745+0000 7f219a7fc640 1 -- 172.21.15.50:0/4108641956 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21a0005c10 con 0x7f21a4118ef0 2024-05-08T18:04:25.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.745+0000 7f219a7fc640 1 -- 172.21.15.50:0/4108641956 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f21a0007070 con 0x7f21a4118ef0 2024-05-08T18:04:25.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.745+0000 7f21ac245640 1 -- 172.21.15.50:0/4108641956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a41a8200 con 0x7f21a4118ef0 2024-05-08T18:04:25.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.745+0000 7f21ac245640 1 -- 172.21.15.50:0/4108641956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a41ac330 con 0x7f21a4118ef0 2024-05-08T18:04:25.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.745+0000 7f219a7fc640 1 -- 172.21.15.50:0/4108641956 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f21a001a070 con 0x7f21a4118ef0 2024-05-08T18:04:25.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.745+0000 7f219a7fc640 1 -- 172.21.15.50:0/4108641956 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f21a0012400 con 0x7f21a4118ef0 2024-05-08T18:04:25.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.747+0000 7f21ac245640 1 -- 172.21.15.50:0/4108641956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a4051020 con 0x7f21a4118ef0 2024-05-08T18:04:25.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.749+0000 7f219a7fc640 1 -- 172.21.15.50:0/4108641956 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f21a001c440 con 0x7f21a4118ef0 2024-05-08T18:04:25.767 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:25.769+0000 7f9ee3e0e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T18:04:25.815 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:25.817+0000 7f9ee3e0e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T18:04:25.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.823+0000 7f21ac245640 1 -- 172.21.15.50:0/4108641956 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9a62c75d-b151-4453-a538-14408e8f4b78", "id": 1} v 0) -- 0x7f21a40507d0 con 0x7f21a4118ef0 2024-05-08T18:04:25.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.824+0000 7f219a7fc640 1 -- 172.21.15.50:0/4108641956 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9a62c75d-b151-4453-a538-14408e8f4b78", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f21a0007210 con 0x7f21a4118ef0 2024-05-08T18:04:25.822 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-05-08T18:04:25.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.825+0000 7f21ac245640 1 -- 172.21.15.50:0/4108641956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21a4118ef0 msgr2=0x7f21a41a79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:25.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.825+0000 7f21ac245640 1 --2- 172.21.15.50:0/4108641956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21a4118ef0 0x7f21a41a79d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f21a000c480 tx=0x7f21a000c940 comp rx=0 tx=0).stop 2024-05-08T18:04:25.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.825+0000 7f21ac245640 1 -- 172.21.15.50:0/4108641956 shutdown_connections 2024-05-08T18:04:25.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.825+0000 7f21ac245640 1 --2- 172.21.15.50:0/4108641956 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f21a4118ef0 0x7f21a41a79d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:25.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.825+0000 7f21ac245640 1 -- 172.21.15.50:0/4108641956 >> 172.21.15.50:0/4108641956 conn(0x7f21a410ed90 msgr2=0x7f21a411bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:25.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.825+0000 7f21ac245640 1 -- 172.21.15.50:0/4108641956 shutdown_connections 2024-05-08T18:04:25.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.825+0000 7f21ac245640 1 -- 172.21.15.50:0/4108641956 wait complete. 2024-05-08T18:04:25.834 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new c2d4e8c1-79bb-4061-ade1-f0fc079c52f9 2 2024-05-08T18:04:25.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.940+0000 7f8ededb3640 1 Processor -- start 2024-05-08T18:04:25.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.940+0000 7f8ededb3640 1 -- start start 2024-05-08T18:04:25.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.940+0000 7f8ededb3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8ed8118ef0 0x7f8ed81192d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:25.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.940+0000 7f8ededb3640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ed8119960 con 0x7f8ed8112c60 2024-05-08T18:04:25.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.940+0000 7f8ededb3640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f8ed8113820 con 0x7f8ed8118ef0 2024-05-08T18:04:25.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.940+0000 7f8edcb4f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8ed8118ef0 0x7f8ed81192d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:25.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.940+0000 7f8ed7fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8ed8112c60 0x7f8ed8113110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:49868/0 (socket says 172.21.15.50:49868) 2024-05-08T18:04:25.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.940+0000 7f8edcb4f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8ed8118ef0 0x7f8ed81192d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:59114/0 (socket says 172.21.15.50:59114) 2024-05-08T18:04:25.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.940+0000 7f8ed7fff640 1 -- 172.21.15.50:0/3203439286 learned_addr learned my addr 172.21.15.50:0/3203439286 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:04:25.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.940+0000 7f8edcb4f640 1 -- 172.21.15.50:0/3203439286 >> v1:172.21.15.46:6789/0 conn(0x7f8ed8112c60 legacy=0x7f8ed8113110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:25.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.940+0000 7f8edcb4f640 1 -- 172.21.15.50:0/3203439286 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed81139d0 con 0x7f8ed8118ef0 2024-05-08T18:04:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.941+0000 7f8edcb4f640 1 --2- 172.21.15.50:0/3203439286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8118ef0 0x7f8ed81192d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009720 tx=0x7f8ecc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4e2a10a220f7c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.941+0000 7f8ed77fe640 1 -- 172.21.15.50:0/3203439286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ecc0316d0 con 0x7f8ed8118ef0 2024-05-08T18:04:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.941+0000 7f8ed77fe640 1 -- 172.21.15.50:0/3203439286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ecc037070 con 0x7f8ed8118ef0 2024-05-08T18:04:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.941+0000 7f8ed77fe640 1 -- 172.21.15.50:0/3203439286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ecc0319c0 con 0x7f8ed8118ef0 2024-05-08T18:04:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.942+0000 7f8ededb3640 1 -- 172.21.15.50:0/3203439286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8118ef0 msgr2=0x7f8ed81192d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.942+0000 7f8ededb3640 1 --2- 172.21.15.50:0/3203439286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8118ef0 0x7f8ed81192d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc009720 tx=0x7f8ecc02b840 comp rx=0 tx=0).stop 2024-05-08T18:04:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.942+0000 7f8ededb3640 1 -- 172.21.15.50:0/3203439286 shutdown_connections 2024-05-08T18:04:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.942+0000 7f8ededb3640 1 --2- 172.21.15.50:0/3203439286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8118ef0 0x7f8ed81192d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.942+0000 7f8ededb3640 1 -- 172.21.15.50:0/3203439286 >> 172.21.15.50:0/3203439286 conn(0x7f8ed810ed90 msgr2=0x7f8ed8111180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.942+0000 7f8ededb3640 1 -- 172.21.15.50:0/3203439286 shutdown_connections 2024-05-08T18:04:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.942+0000 7f8ededb3640 1 -- 172.21.15.50:0/3203439286 wait complete. 2024-05-08T18:04:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.943+0000 7f8ededb3640 1 Processor -- start 2024-05-08T18:04:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.943+0000 7f8ededb3640 1 -- start start 2024-05-08T18:04:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.943+0000 7f8ededb3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8112c60 0x7f8ed81a7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.943+0000 7f8ededb3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed811f070 con 0x7f8ed8112c60 2024-05-08T18:04:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.943+0000 7f8edcb4f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8112c60 0x7f8ed81a7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.943+0000 7f8edcb4f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8112c60 0x7f8ed81a7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:59118/0 (socket says 172.21.15.50:59118) 2024-05-08T18:04:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.943+0000 7f8edcb4f640 1 -- 172.21.15.50:0/588848547 learned_addr learned my addr 172.21.15.50:0/588848547 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:04:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.944+0000 7f8edcb4f640 1 -- 172.21.15.50:0/588848547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed81a7ed0 con 0x7f8ed8112c60 2024-05-08T18:04:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.944+0000 7f8edcb4f640 1 --2- 172.21.15.50:0/588848547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8112c60 0x7f8ed81a7990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc02b880 tx=0x7f8ecc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.944+0000 7f8ed57fa640 1 -- 172.21.15.50:0/588848547 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ecc032770 con 0x7f8ed8112c60 2024-05-08T18:04:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.944+0000 7f8ed57fa640 1 -- 172.21.15.50:0/588848547 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ecc032910 con 0x7f8ed8112c60 2024-05-08T18:04:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.944+0000 7f8ed57fa640 1 -- 172.21.15.50:0/588848547 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ecc032bf0 con 0x7f8ed8112c60 2024-05-08T18:04:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.944+0000 7f8ededb3640 1 -- 172.21.15.50:0/588848547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed81a8160 con 0x7f8ed8112c60 2024-05-08T18:04:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.944+0000 7f8ededb3640 1 -- 172.21.15.50:0/588848547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed81ac110 con 0x7f8ed8112c60 2024-05-08T18:04:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.944+0000 7f8ed57fa640 1 -- 172.21.15.50:0/588848547 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8ecc04c060 con 0x7f8ed8112c60 2024-05-08T18:04:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.944+0000 7f8ed57fa640 1 -- 172.21.15.50:0/588848547 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f8ecc02f050 con 0x7f8ed8112c60 2024-05-08T18:04:25.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.945+0000 7f8ededb3640 1 -- 172.21.15.50:0/588848547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ed8051020 con 0x7f8ed8112c60 2024-05-08T18:04:25.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:25.947+0000 7f8ed57fa640 1 -- 172.21.15.50:0/588848547 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8ecc040070 con 0x7f8ed8112c60 2024-05-08T18:04:26.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.021+0000 7f8ededb3640 1 -- 172.21.15.50:0/588848547 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c2d4e8c1-79bb-4061-ade1-f0fc079c52f9", "id": 2} v 0) -- 0x7f8ed80507d0 con 0x7f8ed8112c60 2024-05-08T18:04:26.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.022+0000 7f8ed57fa640 1 -- 172.21.15.50:0/588848547 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c2d4e8c1-79bb-4061-ade1-f0fc079c52f9", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f8ecc0067f0 con 0x7f8ed8112c60 2024-05-08T18:04:26.020 INFO:teuthology.orchestra.run.smithi050.stdout:2 2024-05-08T18:04:26.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.023+0000 7f8ededb3640 1 -- 172.21.15.50:0/588848547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8112c60 msgr2=0x7f8ed81a7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:26.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.023+0000 7f8ededb3640 1 --2- 172.21.15.50:0/588848547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8112c60 0x7f8ed81a7990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc02b880 tx=0x7f8ecc0046c0 comp rx=0 tx=0).stop 2024-05-08T18:04:26.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.023+0000 7f8ededb3640 1 -- 172.21.15.50:0/588848547 shutdown_connections 2024-05-08T18:04:26.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.023+0000 7f8ededb3640 1 --2- 172.21.15.50:0/588848547 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ed8112c60 0x7f8ed81a7990 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:26.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.023+0000 7f8ededb3640 1 -- 172.21.15.50:0/588848547 >> 172.21.15.50:0/588848547 conn(0x7f8ed810ed90 msgr2=0x7f8ed8114ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:26.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.023+0000 7f8ededb3640 1 -- 172.21.15.50:0/588848547 shutdown_connections 2024-05-08T18:04:26.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.023+0000 7f8ededb3640 1 -- 172.21.15.50:0/588848547 wait complete. 2024-05-08T18:04:26.032 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph --cluster ceph osd new e5e292cd-9ed4-44c9-9e91-7ee3736272ba 3 2024-05-08T18:04:26.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.137+0000 7f3733f92640 1 Processor -- start 2024-05-08T18:04:26.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.137+0000 7f3733f92640 1 -- start start 2024-05-08T18:04:26.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.138+0000 7f3733f92640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f372c118ef0 0x7f372c1192d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:26.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.138+0000 7f3733f92640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f372c119960 con 0x7f372c112c60 2024-05-08T18:04:26.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.138+0000 7f3733f92640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f372c113820 con 0x7f372c118ef0 2024-05-08T18:04:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.138+0000 7f3731d2e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f372c118ef0 0x7f372c1192d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.138+0000 7f373152d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f372c112c60 0x7f372c113110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:49876/0 (socket says 172.21.15.50:49876) 2024-05-08T18:04:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.138+0000 7f373152d640 1 -- 172.21.15.50:0/1767573013 learned_addr learned my addr 172.21.15.50:0/1767573013 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:04:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.138+0000 7f3730d2c640 1 -- 172.21.15.50:0/1767573013 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1758276853 0 0) 0x7f3720001110 con 0x7f372c112c60 2024-05-08T18:04:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.138+0000 7f3730d2c640 1 -- 172.21.15.50:0/1767573013 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3718003520 con 0x7f372c112c60 2024-05-08T18:04:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.138+0000 7f3731d2e640 1 -- 172.21.15.50:0/1767573013 >> v1:172.21.15.46:6789/0 conn(0x7f372c112c60 legacy=0x7f372c113110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.138+0000 7f3731d2e640 1 -- 172.21.15.50:0/1767573013 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c1139d0 con 0x7f372c118ef0 2024-05-08T18:04:26.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.139+0000 7f3731d2e640 1 --2- 172.21.15.50:0/1767573013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c118ef0 0x7f372c1192d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3728009720 tx=0x7f372802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e246f34bba940d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:26.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.139+0000 7f3730d2c640 1 -- 172.21.15.50:0/1767573013 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37280316d0 con 0x7f372c118ef0 2024-05-08T18:04:26.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.139+0000 7f3730d2c640 1 -- 172.21.15.50:0/1767573013 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3728037070 con 0x7f372c118ef0 2024-05-08T18:04:26.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.139+0000 7f3730d2c640 1 -- 172.21.15.50:0/1767573013 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37280319a0 con 0x7f372c118ef0 2024-05-08T18:04:26.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.140+0000 7f3733f92640 1 -- 172.21.15.50:0/1767573013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c118ef0 msgr2=0x7f372c1192d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:26.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.140+0000 7f3733f92640 1 --2- 172.21.15.50:0/1767573013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c118ef0 0x7f372c1192d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3728009720 tx=0x7f372802b840 comp rx=0 tx=0).stop 2024-05-08T18:04:26.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.140+0000 7f3733f92640 1 -- 172.21.15.50:0/1767573013 shutdown_connections 2024-05-08T18:04:26.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.140+0000 7f3733f92640 1 --2- 172.21.15.50:0/1767573013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c118ef0 0x7f372c1192d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:26.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.140+0000 7f3733f92640 1 -- 172.21.15.50:0/1767573013 >> 172.21.15.50:0/1767573013 conn(0x7f372c10ed90 msgr2=0x7f372c111180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:26.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.140+0000 7f3733f92640 1 -- 172.21.15.50:0/1767573013 shutdown_connections 2024-05-08T18:04:26.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.140+0000 7f3733f92640 1 -- 172.21.15.50:0/1767573013 wait complete. 2024-05-08T18:04:26.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.141+0000 7f3733f92640 1 Processor -- start 2024-05-08T18:04:26.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.141+0000 7f3733f92640 1 -- start start 2024-05-08T18:04:26.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.141+0000 7f3733f92640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c112c60 0x7f372c1a7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:26.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.141+0000 7f3733f92640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f372c11f070 con 0x7f372c112c60 2024-05-08T18:04:26.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.141+0000 7f3731d2e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c112c60 0x7f372c1a7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.141+0000 7f3731d2e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c112c60 0x7f372c1a7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:59140/0 (socket says 172.21.15.50:59140) 2024-05-08T18:04:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.141+0000 7f3731d2e640 1 -- 172.21.15.50:0/3178061075 learned_addr learned my addr 172.21.15.50:0/3178061075 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:04:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.142+0000 7f3731d2e640 1 -- 172.21.15.50:0/3178061075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c1a7ed0 con 0x7f372c112c60 2024-05-08T18:04:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.142+0000 7f3731d2e640 1 --2- 172.21.15.50:0/3178061075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c112c60 0x7f372c1a7990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3728009720 tx=0x7f37280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.142+0000 7f371e7fc640 1 -- 172.21.15.50:0/3178061075 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3728004160 con 0x7f372c112c60 2024-05-08T18:04:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.142+0000 7f371e7fc640 1 -- 172.21.15.50:0/3178061075 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3728004320 con 0x7f372c112c60 2024-05-08T18:04:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.142+0000 7f3733f92640 1 -- 172.21.15.50:0/3178061075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372c1a8160 con 0x7f372c112c60 2024-05-08T18:04:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.142+0000 7f371e7fc640 1 -- 172.21.15.50:0/3178061075 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3728003610 con 0x7f372c112c60 2024-05-08T18:04:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.142+0000 7f3733f92640 1 -- 172.21.15.50:0/3178061075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372c1ac110 con 0x7f372c112c60 2024-05-08T18:04:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.143+0000 7f371e7fc640 1 -- 172.21.15.50:0/3178061075 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f372804f440 con 0x7f372c112c60 2024-05-08T18:04:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.143+0000 7f371e7fc640 1 -- 172.21.15.50:0/3178061075 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f372802f050 con 0x7f372c112c60 2024-05-08T18:04:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.143+0000 7f3733f92640 1 -- 172.21.15.50:0/3178061075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f372c051020 con 0x7f372c112c60 2024-05-08T18:04:26.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.145+0000 7f371e7fc640 1 -- 172.21.15.50:0/3178061075 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3728065ec0 con 0x7f372c112c60 2024-05-08T18:04:26.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.219+0000 7f3733f92640 1 -- 172.21.15.50:0/3178061075 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e5e292cd-9ed4-44c9-9e91-7ee3736272ba", "id": 3} v 0) -- 0x7f372c0507d0 con 0x7f372c112c60 2024-05-08T18:04:26.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.220+0000 7f371e7fc640 1 -- 172.21.15.50:0/3178061075 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e5e292cd-9ed4-44c9-9e91-7ee3736272ba", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f3728054070 con 0x7f372c112c60 2024-05-08T18:04:26.218 INFO:teuthology.orchestra.run.smithi050.stdout:3 2024-05-08T18:04:26.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.221+0000 7f3733f92640 1 -- 172.21.15.50:0/3178061075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c112c60 msgr2=0x7f372c1a7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:26.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.221+0000 7f3733f92640 1 --2- 172.21.15.50:0/3178061075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c112c60 0x7f372c1a7990 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3728009720 tx=0x7f37280046c0 comp rx=0 tx=0).stop 2024-05-08T18:04:26.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.222+0000 7f3733f92640 1 -- 172.21.15.50:0/3178061075 shutdown_connections 2024-05-08T18:04:26.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.222+0000 7f3733f92640 1 --2- 172.21.15.50:0/3178061075 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f372c112c60 0x7f372c1a7990 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:26.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.222+0000 7f3733f92640 1 -- 172.21.15.50:0/3178061075 >> 172.21.15.50:0/3178061075 conn(0x7f372c10ed90 msgr2=0x7f372c114ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:26.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.222+0000 7f3733f92640 1 -- 172.21.15.50:0/3178061075 shutdown_connections 2024-05-08T18:04:26.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:26.222+0000 7f3733f92640 1 -- 172.21.15.50:0/3178061075 wait complete. 2024-05-08T18:04:26.230 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-08T18:04:26.230 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-08T18:04:26.233 INFO:tasks.ceph.osd.0:Started 2024-05-08T18:04:26.233 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-08T18:04:26.234 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-08T18:04:26.236 INFO:tasks.ceph.osd.1:Started 2024-05-08T18:04:26.236 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-08T18:04:26.237 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-08T18:04:26.238 INFO:tasks.ceph.osd.2:Started 2024-05-08T18:04:26.239 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-08T18:04:26.239 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-08T18:04:26.241 INFO:tasks.ceph.osd.3:Started 2024-05-08T18:04:26.241 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T18:04:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.358+0000 7f3913827640 1 Processor -- start 2024-05-08T18:04:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.358+0000 7f3913827640 1 -- start start 2024-05-08T18:04:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.358+0000 7f3913827640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f390c09b020 0x7f390c096390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.359+0000 7f3913827640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f390c097410 con 0x7f390c0968d0 2024-05-08T18:04:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.359+0000 7f3913827640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f390c09b400 con 0x7f390c09b020 2024-05-08T18:04:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.359+0000 7f39115c3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f390c09b020 0x7f390c096390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:26.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.359+0000 7f3910dc2640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f390c0968d0 0x7f390c133fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59174/0 (socket says 172.21.15.46:59174) 2024-05-08T18:04:26.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.359+0000 7f3910dc2640 1 -- 172.21.15.46:0/1844864957 learned_addr learned my addr 172.21.15.46:0/1844864957 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:26.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.359+0000 7f39115c3640 1 -- 172.21.15.46:0/1844864957 >> v1:172.21.15.46:6789/0 conn(0x7f390c0968d0 legacy=0x7f390c133fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:26.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.359+0000 7f39115c3640 1 -- 172.21.15.46:0/1844864957 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c06d080 con 0x7f390c09b020 2024-05-08T18:04:26.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.360+0000 7f39115c3640 1 --2- 172.21.15.46:0/1844864957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f390c09b020 0x7f390c096390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3908009720 tx=0x7f390802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98b11e7cc60b72b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:26.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.360+0000 7f3903fff640 1 -- 172.21.15.46:0/1844864957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39080316d0 con 0x7f390c09b020 2024-05-08T18:04:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.360+0000 7f3903fff640 1 -- 172.21.15.46:0/1844864957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3908037070 con 0x7f390c09b020 2024-05-08T18:04:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.360+0000 7f3903fff640 1 -- 172.21.15.46:0/1844864957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39080319c0 con 0x7f390c09b020 2024-05-08T18:04:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.360+0000 7f3913827640 1 -- 172.21.15.46:0/1844864957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f390c09b020 msgr2=0x7f390c096390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.360+0000 7f3913827640 1 --2- 172.21.15.46:0/1844864957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f390c09b020 0x7f390c096390 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3908009720 tx=0x7f390802b840 comp rx=0 tx=0).stop 2024-05-08T18:04:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.361+0000 7f3913827640 1 -- 172.21.15.46:0/1844864957 shutdown_connections 2024-05-08T18:04:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.361+0000 7f3913827640 1 --2- 172.21.15.46:0/1844864957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f390c09b020 0x7f390c096390 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.361+0000 7f3913827640 1 -- 172.21.15.46:0/1844864957 >> 172.21.15.46:0/1844864957 conn(0x7f390c09e440 msgr2=0x7f390c09a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.361+0000 7f3913827640 1 -- 172.21.15.46:0/1844864957 shutdown_connections 2024-05-08T18:04:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.361+0000 7f3913827640 1 -- 172.21.15.46:0/1844864957 wait complete. 2024-05-08T18:04:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.361+0000 7f3913827640 1 Processor -- start 2024-05-08T18:04:26.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.362+0000 7f3913827640 1 -- start start 2024-05-08T18:04:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.362+0000 7f3913827640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f390c0968d0 0x7f390c1a8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.362+0000 7f3913827640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f390c097080 con 0x7f390c0968d0 2024-05-08T18:04:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.362+0000 7f39115c3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f390c0968d0 0x7f390c1a8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.362+0000 7f39115c3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f390c0968d0 0x7f390c1a8c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41466/0 (socket says 172.21.15.46:41466) 2024-05-08T18:04:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.362+0000 7f39115c3640 1 -- 172.21.15.46:0/1665438462 learned_addr learned my addr 172.21.15.46:0/1665438462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.362+0000 7f39115c3640 1 -- 172.21.15.46:0/1665438462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c093fa0 con 0x7f390c0968d0 2024-05-08T18:04:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.362+0000 7f39115c3640 1 --2- 172.21.15.46:0/1665438462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f390c0968d0 0x7f390c1a8c30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f390802b880 tx=0x7f39080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.363+0000 7f3901ffb640 1 -- 172.21.15.46:0/1665438462 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3908032770 con 0x7f390c0968d0 2024-05-08T18:04:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.363+0000 7f3901ffb640 1 -- 172.21.15.46:0/1665438462 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3908032910 con 0x7f390c0968d0 2024-05-08T18:04:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.363+0000 7f3901ffb640 1 -- 172.21.15.46:0/1665438462 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3908032c10 con 0x7f390c0968d0 2024-05-08T18:04:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.363+0000 7f3913827640 1 -- 172.21.15.46:0/1665438462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390c1abd30 con 0x7f390c0968d0 2024-05-08T18:04:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.363+0000 7f3913827640 1 -- 172.21.15.46:0/1665438462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390c1ab630 con 0x7f390c0968d0 2024-05-08T18:04:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.363+0000 7f3901ffb640 1 -- 172.21.15.46:0/1665438462 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f390804c060 con 0x7f390c0968d0 2024-05-08T18:04:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.363+0000 7f3901ffb640 1 -- 172.21.15.46:0/1665438462 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f390802f050 con 0x7f390c0968d0 2024-05-08T18:04:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.364+0000 7f3913827640 1 -- 172.21.15.46:0/1665438462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390c1aa490 con 0x7f390c0968d0 2024-05-08T18:04:26.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.366+0000 7f3901ffb640 1 -- 172.21.15.46:0/1665438462 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f3908040070 con 0x7f390c0968d0 2024-05-08T18:04:26.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.428+0000 7f3913827640 1 -- 172.21.15.46:0/1665438462 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f390c0640a0 con 0x7f390c0968d0 2024-05-08T18:04:26.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.429+0000 7f3901ffb640 1 -- 172.21.15.46:0/1665438462 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) ==== 74+0+4153 (secure 0 0 0) 0x7f3908032db0 con 0x7f390c0968d0 2024-05-08T18:04:26.427 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:26.427 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"fsid":"d8088d1f-616e-46b1-995a-85e572267d38","created":"2024-05-08T18:04:25.152571+0000","modified":"2024-05-08T18:04:26.221033+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T18:04:26.221033+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f7304d61-4d9e-455b-a968-5641c682e642","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9a62c75d-b151-4453-a538-14408e8f4b78","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c2d4e8c1-79bb-4061-ade1-f0fc079c52f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e5e292cd-9ed4-44c9-9e91-7ee3736272ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T18:04:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.429+0000 7f3913827640 1 -- 172.21.15.46:0/1665438462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f390c0968d0 msgr2=0x7f390c1a8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.429+0000 7f3913827640 1 --2- 172.21.15.46:0/1665438462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f390c0968d0 0x7f390c1a8c30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f390802b880 tx=0x7f39080046c0 comp rx=0 tx=0).stop 2024-05-08T18:04:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.429+0000 7f3913827640 1 -- 172.21.15.46:0/1665438462 shutdown_connections 2024-05-08T18:04:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.429+0000 7f3913827640 1 --2- 172.21.15.46:0/1665438462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f390c0968d0 0x7f390c1a8c30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.429+0000 7f3913827640 1 -- 172.21.15.46:0/1665438462 >> 172.21.15.46:0/1665438462 conn(0x7f390c09e440 msgr2=0x7f390c092990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.429+0000 7f3913827640 1 -- 172.21.15.46:0/1665438462 shutdown_connections 2024-05-08T18:04:26.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:26.429+0000 7f3913827640 1 -- 172.21.15.46:0/1665438462 wait complete. 2024-05-08T18:04:26.438 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-08T18:04:26.438 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-08T18:04:26.603 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:26.604+0000 7f9ee3e0e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T18:04:26.694 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:26.696+0000 7f9ee3e0e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T18:04:26.738 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:26.740+0000 7f9ee3e0e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T18:04:26.829 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-05-08T18:04:26.831+0000 7f5ac9d44640 -1 Falling back to public interface 2024-05-08T18:04:26.832 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-05-08T18:04:26.834+0000 7fb0770a4640 -1 Falling back to public interface 2024-05-08T18:04:26.835 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-05-08T18:04:26.838+0000 7f3934216640 -1 Falling back to public interface 2024-05-08T18:04:26.836 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-05-08T18:04:26.838+0000 7f909e9fc640 -1 Falling back to public interface 2024-05-08T18:04:26.841 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T18:04:26.916 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:26.918+0000 7f9ee3e0e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T18:04:26.939 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.941+0000 7fb8f8369640 1 Processor -- start 2024-05-08T18:04:26.939 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.941+0000 7fb8f8369640 1 -- start start 2024-05-08T18:04:26.940 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.942+0000 7fb8f8369640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb8f00700e0 0x7fb8f0134740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:26.940 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.942+0000 7fb8f8369640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8f0073f40 con 0x7fb8f006f5f0 2024-05-08T18:04:26.940 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.942+0000 7fb8f8369640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb8f00717e0 con 0x7fb8f00700e0 2024-05-08T18:04:26.940 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.942+0000 7fb8f5904640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb8f00700e0 0x7fb8f0134740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:26.941 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.942+0000 7fb8f6105640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb8f006f5f0 0x7fb8f006f9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59182/0 (socket says 172.21.15.46:59182) 2024-05-08T18:04:26.941 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.942+0000 7fb8f6105640 1 -- 172.21.15.46:0/2641138549 learned_addr learned my addr 172.21.15.46:0/2641138549 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:26.941 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.942+0000 7fb8f5904640 1 -- 172.21.15.46:0/2641138549 >> v1:172.21.15.46:6789/0 conn(0x7fb8f006f5f0 legacy=0x7fb8f006f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:26.941 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.942+0000 7fb8f5904640 1 -- 172.21.15.46:0/2641138549 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f0070c70 con 0x7fb8f00700e0 2024-05-08T18:04:26.942 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.943+0000 7fb8f5904640 1 --2- 172.21.15.46:0/2641138549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f00700e0 0x7fb8f0134740 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009720 tx=0x7fb8e002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0dfe66993a44fcc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:26.942 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.943+0000 7fb8f5103640 1 -- 172.21.15.46:0/2641138549 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8e00316d0 con 0x7fb8f00700e0 2024-05-08T18:04:26.942 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.943+0000 7fb8f5103640 1 -- 172.21.15.46:0/2641138549 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8e0037070 con 0x7fb8f00700e0 2024-05-08T18:04:26.942 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.943+0000 7fb8f5103640 1 -- 172.21.15.46:0/2641138549 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8e00319c0 con 0x7fb8f00700e0 2024-05-08T18:04:26.942 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.944+0000 7fb8f8369640 1 -- 172.21.15.46:0/2641138549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f00700e0 msgr2=0x7fb8f0134740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:26.942 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.944+0000 7fb8f8369640 1 --2- 172.21.15.46:0/2641138549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f00700e0 0x7fb8f0134740 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009720 tx=0x7fb8e002b840 comp rx=0 tx=0).stop 2024-05-08T18:04:26.942 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.944+0000 7fb8f8369640 1 -- 172.21.15.46:0/2641138549 shutdown_connections 2024-05-08T18:04:26.942 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.944+0000 7fb8f8369640 1 --2- 172.21.15.46:0/2641138549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f00700e0 0x7fb8f0134740 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:26.943 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.944+0000 7fb8f8369640 1 -- 172.21.15.46:0/2641138549 >> 172.21.15.46:0/2641138549 conn(0x7fb8f009ec00 msgr2=0x7fb8f009f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:26.943 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.944+0000 7fb8f8369640 1 -- 172.21.15.46:0/2641138549 shutdown_connections 2024-05-08T18:04:26.943 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.944+0000 7fb8f8369640 1 -- 172.21.15.46:0/2641138549 wait complete. 2024-05-08T18:04:26.944 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.945+0000 7fb8f8369640 1 Processor -- start 2024-05-08T18:04:26.945 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.945+0000 7fb8f8369640 1 -- start start 2024-05-08T18:04:26.945 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.945+0000 7fb8f8369640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f006f5f0 0x7fb8f009b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:26.945 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.945+0000 7fb8f8369640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb8f0134e00 con 0x7fb8f006f5f0 2024-05-08T18:04:26.945 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.945+0000 7fb8f6105640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f006f5f0 0x7fb8f009b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:26.945 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.945+0000 7fb8f6105640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f006f5f0 0x7fb8f009b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41486/0 (socket says 172.21.15.46:41486) 2024-05-08T18:04:26.945 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.945+0000 7fb8f6105640 1 -- 172.21.15.46:0/4224549192 learned_addr learned my addr 172.21.15.46:0/4224549192 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:26.945 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.945+0000 7fb8f6105640 1 -- 172.21.15.46:0/4224549192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8f0133ad0 con 0x7fb8f006f5f0 2024-05-08T18:04:26.945 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.946+0000 7fb8f6105640 1 --2- 172.21.15.46:0/4224549192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f006f5f0 0x7fb8f009b730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00c480 tx=0x7fb8ec00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:26.945 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.946+0000 7fb8e6ffd640 1 -- 172.21.15.46:0/4224549192 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec017070 con 0x7fb8f006f5f0 2024-05-08T18:04:26.945 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.946+0000 7fb8e6ffd640 1 -- 172.21.15.46:0/4224549192 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8ec005c10 con 0x7fb8f006f5f0 2024-05-08T18:04:26.945 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.946+0000 7fb8f8369640 1 -- 172.21.15.46:0/4224549192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8f0132750 con 0x7fb8f006f5f0 2024-05-08T18:04:26.945 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.946+0000 7fb8e6ffd640 1 -- 172.21.15.46:0/4224549192 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec007070 con 0x7fb8f006f5f0 2024-05-08T18:04:26.945 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.946+0000 7fb8f8369640 1 -- 172.21.15.46:0/4224549192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8f0132df0 con 0x7fb8f006f5f0 2024-05-08T18:04:26.945 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.946+0000 7fb8e6ffd640 1 -- 172.21.15.46:0/4224549192 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb8ec01a070 con 0x7fb8f006f5f0 2024-05-08T18:04:26.945 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.946+0000 7fb8e6ffd640 1 -- 172.21.15.46:0/4224549192 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fb8ec01cb60 con 0x7fb8f006f5f0 2024-05-08T18:04:26.946 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.947+0000 7fb8f8369640 1 -- 172.21.15.46:0/4224549192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8f00640a0 con 0x7fb8f006f5f0 2024-05-08T18:04:26.946 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:26.949+0000 7fb8e6ffd640 1 -- 172.21.15.46:0/4224549192 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb8f0132df0 con 0x7fb8f006f5f0 2024-05-08T18:04:26.966 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:26.968+0000 7f9ee3e0e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T18:04:27.008 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:27.010+0000 7fb8f8369640 1 -- 172.21.15.46:0/4224549192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8f00507d0 con 0x7fb8f006f5f0 2024-05-08T18:04:27.008 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:27.010+0000 7fb8e6ffd640 1 -- 172.21.15.46:0/4224549192 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) ==== 74+0+4153 (secure 0 0 0) 0x7fb8ec01c440 con 0x7fb8f006f5f0 2024-05-08T18:04:27.008 INFO:teuthology.misc.health.smithi046.stdout: 2024-05-08T18:04:27.009 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":6,"fsid":"d8088d1f-616e-46b1-995a-85e572267d38","created":"2024-05-08T18:04:25.152571+0000","modified":"2024-05-08T18:04:26.221033+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T18:04:26.221033+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f7304d61-4d9e-455b-a968-5641c682e642","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9a62c75d-b151-4453-a538-14408e8f4b78","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c2d4e8c1-79bb-4061-ade1-f0fc079c52f9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e5e292cd-9ed4-44c9-9e91-7ee3736272ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T18:04:27.009 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:27.011+0000 7fb8f8369640 1 -- 172.21.15.46:0/4224549192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f006f5f0 msgr2=0x7fb8f009b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:27.009 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:27.011+0000 7fb8f8369640 1 --2- 172.21.15.46:0/4224549192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f006f5f0 0x7fb8f009b730 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00c480 tx=0x7fb8ec00c940 comp rx=0 tx=0).stop 2024-05-08T18:04:27.010 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:27.011+0000 7fb8f8369640 1 -- 172.21.15.46:0/4224549192 shutdown_connections 2024-05-08T18:04:27.010 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:27.011+0000 7fb8f8369640 1 --2- 172.21.15.46:0/4224549192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb8f006f5f0 0x7fb8f009b730 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:27.010 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:27.011+0000 7fb8f8369640 1 -- 172.21.15.46:0/4224549192 >> 172.21.15.46:0/4224549192 conn(0x7fb8f009ec00 msgr2=0x7fb8f009cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:27.010 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:27.011+0000 7fb8f8369640 1 -- 172.21.15.46:0/4224549192 shutdown_connections 2024-05-08T18:04:27.010 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:27.011+0000 7fb8f8369640 1 -- 172.21.15.46:0/4224549192 wait complete. 2024-05-08T18:04:27.018 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-05-08T18:04:27.021 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:27.023+0000 7f9ee3e0e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T18:04:27.140 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:27.142+0000 7f9ee3e0e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T18:04:27.541 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:27.543+0000 7f9ee3e0e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T18:04:27.688 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:27.690+0000 7f9ee3e0e040 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T18:04:27.962 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:27.964+0000 7f9ee3e0e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T18:04:28.017 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:28.019+0000 7f9ee3e0e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T18:04:28.139 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:28.141+0000 7f9ee3e0e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T18:04:28.445 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:28.447+0000 7f9ee3e0e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T18:04:28.496 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:28.498+0000 7f9ee3e0e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T18:04:28.660 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:28.662+0000 7f9ee3e0e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T18:04:28.805 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:28.807+0000 7f9ee3e0e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T18:04:28.857 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:28.860+0000 7f9ee3e0e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T18:04:28.931 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:28.933+0000 7f9ee3e0e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T18:04:28.978 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:28.981+0000 7f9ee3e0e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T18:04:29.491 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:29.493+0000 7f9ee3e0e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T18:04:29.654 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:29.656+0000 7f9ee3e0e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T18:04:30.259 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-05-08T18:04:30.261+0000 7fb0770a4640 -1 osd.0 0 log_to_monitors true 2024-05-08T18:04:30.300 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-05-08T18:04:30.301+0000 7f5ac9d44640 -1 osd.3 0 log_to_monitors true 2024-05-08T18:04:30.343 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-05-08T18:04:30.344+0000 7f909e9fc640 -1 osd.2 0 log_to_monitors true 2024-05-08T18:04:30.363 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-05-08T18:04:30.364+0000 7f3934216640 -1 osd.1 0 log_to_monitors true 2024-05-08T18:04:30.664 INFO:tasks.ceph.mgr.x.smithi050.stderr:2024-05-08T18:04:30.665+0000 7f9e83695640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-05-08T18:04:32.538 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-05-08T18:04:32.540+0000 7fb07303a640 -1 osd.0 0 waiting for initial osdmap 2024-05-08T18:04:32.549 INFO:tasks.ceph.osd.0.smithi046.stderr:2024-05-08T18:04:32.551+0000 7fb06ae5a640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T18:04:32.937 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-05-08T18:04:32.938+0000 7f5ac5cda640 -1 osd.3 0 waiting for initial osdmap 2024-05-08T18:04:32.943 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-05-08T18:04:32.944+0000 7f909a992640 -1 osd.2 0 waiting for initial osdmap 2024-05-08T18:04:32.950 INFO:tasks.ceph.osd.3.smithi050.stderr:2024-05-08T18:04:32.952+0000 7f5abdafa640 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T18:04:32.954 INFO:tasks.ceph.osd.2.smithi050.stderr:2024-05-08T18:04:32.956+0000 7f90927b2640 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T18:04:32.955 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-05-08T18:04:32.957+0000 7f392fb80640 -1 osd.1 0 waiting for initial osdmap 2024-05-08T18:04:32.969 INFO:tasks.ceph.osd.1.smithi050.stderr:2024-05-08T18:04:32.970+0000 7f39279a0640 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T18:04:33.422 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T18:04:33.527 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.529+0000 7f6ef7442640 1 Processor -- start 2024-05-08T18:04:33.527 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.529+0000 7f6ef7442640 1 -- start start 2024-05-08T18:04:33.529 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.529+0000 7f6ef7442640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ef008bae0 0x7f6ef0138ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:33.529 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.529+0000 7f6ef7442640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ef0097e80 con 0x7f6ef008aff0 2024-05-08T18:04:33.529 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.529+0000 7f6ef7442640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6ef008cfd0 con 0x7f6ef008bae0 2024-05-08T18:04:33.529 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.530+0000 7f6ef49dd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6ef008bae0 0x7f6ef0138ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:33.529 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.530+0000 7f6ef51de640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6ef008aff0 0x7f6ef008b3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39968/0 (socket says 172.21.15.46:39968) 2024-05-08T18:04:33.529 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.530+0000 7f6ef51de640 1 -- 172.21.15.46:0/586979854 learned_addr learned my addr 172.21.15.46:0/586979854 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:33.529 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.530+0000 7f6ef49dd640 1 -- 172.21.15.46:0/586979854 >> v1:172.21.15.46:6789/0 conn(0x7f6ef008aff0 legacy=0x7f6ef008b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:33.529 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.530+0000 7f6ef49dd640 1 -- 172.21.15.46:0/586979854 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef008e460 con 0x7f6ef008bae0 2024-05-08T18:04:33.529 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.531+0000 7f6ef49dd640 1 --2- 172.21.15.46:0/586979854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ef008bae0 0x7f6ef0138ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009720 tx=0x7f6ee002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b438ab5118166ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:33.529 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.531+0000 7f6ee7fff640 1 -- 172.21.15.46:0/586979854 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ee00316d0 con 0x7f6ef008bae0 2024-05-08T18:04:33.529 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.531+0000 7f6ee7fff640 1 -- 172.21.15.46:0/586979854 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ee0037070 con 0x7f6ef008bae0 2024-05-08T18:04:33.530 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.531+0000 7f6ee7fff640 1 -- 172.21.15.46:0/586979854 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ee00319c0 con 0x7f6ef008bae0 2024-05-08T18:04:33.532 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.531+0000 7f6ef7442640 1 -- 172.21.15.46:0/586979854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ef008bae0 msgr2=0x7f6ef0138ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:33.532 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.531+0000 7f6ef7442640 1 --2- 172.21.15.46:0/586979854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ef008bae0 0x7f6ef0138ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009720 tx=0x7f6ee002b840 comp rx=0 tx=0).stop 2024-05-08T18:04:33.532 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.531+0000 7f6ef7442640 1 -- 172.21.15.46:0/586979854 shutdown_connections 2024-05-08T18:04:33.532 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.531+0000 7f6ef7442640 1 --2- 172.21.15.46:0/586979854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ef008bae0 0x7f6ef0138ef0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:33.532 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.531+0000 7f6ef7442640 1 -- 172.21.15.46:0/586979854 >> 172.21.15.46:0/586979854 conn(0x7f6ef009d8d0 msgr2=0x7f6ef009dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:33.532 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.532+0000 7f6ef7442640 1 -- 172.21.15.46:0/586979854 shutdown_connections 2024-05-08T18:04:33.532 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.532+0000 7f6ef7442640 1 -- 172.21.15.46:0/586979854 wait complete. 2024-05-08T18:04:33.532 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.532+0000 7f6ef7442640 1 Processor -- start 2024-05-08T18:04:33.532 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.532+0000 7f6ef7442640 1 -- start start 2024-05-08T18:04:33.532 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.533+0000 7f6ef7442640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ef008aff0 0x7f6ef0125b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:33.532 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.533+0000 7f6ef7442640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6ef008ca10 con 0x7f6ef008aff0 2024-05-08T18:04:33.532 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.533+0000 7f6ef51de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ef008aff0 0x7f6ef0125b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:33.533 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.533+0000 7f6ef51de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ef008aff0 0x7f6ef0125b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39954/0 (socket says 172.21.15.46:39954) 2024-05-08T18:04:33.533 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.533+0000 7f6ef51de640 1 -- 172.21.15.46:0/2512623803 learned_addr learned my addr 172.21.15.46:0/2512623803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:33.533 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.533+0000 7f6ef51de640 1 -- 172.21.15.46:0/2512623803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ef0128140 con 0x7f6ef008aff0 2024-05-08T18:04:33.533 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.533+0000 7f6ef51de640 1 --2- 172.21.15.46:0/2512623803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ef008aff0 0x7f6ef0125b80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00c380 tx=0x7f6eec00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:33.533 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.534+0000 7f6ee5ffb640 1 -- 172.21.15.46:0/2512623803 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6eec017070 con 0x7f6ef008aff0 2024-05-08T18:04:33.533 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.534+0000 7f6ee5ffb640 1 -- 172.21.15.46:0/2512623803 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6eec005c10 con 0x7f6ef008aff0 2024-05-08T18:04:33.533 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.534+0000 7f6ef7442640 1 -- 172.21.15.46:0/2512623803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ef0128d70 con 0x7f6ef008aff0 2024-05-08T18:04:33.533 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.534+0000 7f6ee5ffb640 1 -- 172.21.15.46:0/2512623803 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6eec0070d0 con 0x7f6ef008aff0 2024-05-08T18:04:33.533 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.534+0000 7f6ef7442640 1 -- 172.21.15.46:0/2512623803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ef0128900 con 0x7f6ef008aff0 2024-05-08T18:04:33.534 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.535+0000 7f6ee5ffb640 1 -- 172.21.15.46:0/2512623803 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 49355+0+0 (secure 0 0 0) 0x7f6eec01a070 con 0x7f6ef008aff0 2024-05-08T18:04:33.534 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.535+0000 7f6ef7442640 1 -- 172.21.15.46:0/2512623803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ef006f5d0 con 0x7f6ef008aff0 2024-05-08T18:04:33.534 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.535+0000 7f6ee5ffb640 1 --2- 172.21.15.46:0/2512623803 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ed403d220 0x7f6ed403f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:33.534 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.535+0000 7f6ee5ffb640 1 -- 172.21.15.46:0/2512623803 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1623+0+0 (secure 0 0 0) 0x7f6eec04fd80 con 0x7f6ef008aff0 2024-05-08T18:04:33.534 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.535+0000 7f6ef49dd640 1 --2- 172.21.15.46:0/2512623803 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ed403d220 0x7f6ed403f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:33.534 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.536+0000 7f6ef49dd640 1 --2- 172.21.15.46:0/2512623803 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ed403d220 0x7f6ed403f6e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009720 tx=0x7f6ee0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:33.538 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.540+0000 7f6ee5ffb640 1 -- 172.21.15.46:0/2512623803 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6ef006f5d0 con 0x7f6ef008aff0 2024-05-08T18:04:33.734 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.735+0000 7f6ef7442640 1 -- 172.21.15.46:0/2512623803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ef00640a0 con 0x7f6ef008aff0 2024-05-08T18:04:33.734 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.735+0000 7f6ee5ffb640 1 -- 172.21.15.46:0/2512623803 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v9) ==== 74+0+6056 (secure 0 0 0) 0x7f6eec019cd0 con 0x7f6ef008aff0 2024-05-08T18:04:33.734 INFO:teuthology.misc.health.smithi046.stdout: 2024-05-08T18:04:33.734 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":9,"fsid":"d8088d1f-616e-46b1-995a-85e572267d38","created":"2024-05-08T18:04:25.152571+0000","modified":"2024-05-08T18:04:33.542468+0000","last_up_change":"2024-05-08T18:04:33.542468+0000","last_in_change":"2024-05-08T18:04:26.221033+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f7304d61-4d9e-455b-a968-5641c682e642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":337143975},{"type":"v1","addr":"172.21.15.46:6801","nonce":337143975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":337143975},{"type":"v1","addr":"172.21.15.46:6803","nonce":337143975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":337143975},{"type":"v1","addr":"172.21.15.46:6807","nonce":337143975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":337143975},{"type":"v1","addr":"172.21.15.46:6805","nonce":337143975}]},"public_addr":"172.21.15.46:6801/337143975","cluster_addr":"172.21.15.46:6803/337143975","heartbeat_back_addr":"172.21.15.46:6807/337143975","heartbeat_front_addr":"172.21.15.46:6805/337143975","state":["exists","up"]},{"osd":1,"uuid":"9a62c75d-b151-4453-a538-14408e8f4b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2141923372},{"type":"v1","addr":"172.21.15.50:6809","nonce":2141923372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2141923372},{"type":"v1","addr":"172.21.15.50:6812","nonce":2141923372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2141923372},{"type":"v1","addr":"172.21.15.50:6820","nonce":2141923372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2141923372},{"type":"v1","addr":"172.21.15.50:6816","nonce":2141923372}]},"public_addr":"172.21.15.50:6809/2141923372","cluster_addr":"172.21.15.50:6812/2141923372","heartbeat_back_addr":"172.21.15.50:6820/2141923372","heartbeat_front_addr":"172.21.15.50:6816/2141923372","state":["exists","up"]},{"osd":2,"uuid":"c2d4e8c1-79bb-4061-ade1-f0fc079c52f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":3039646617},{"type":"v1","addr":"172.21.15.50:6813","nonce":3039646617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6815","nonce":3039646617},{"type":"v1","addr":"172.21.15.50:6817","nonce":3039646617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3039646617},{"type":"v1","addr":"172.21.15.50:6823","nonce":3039646617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":3039646617},{"type":"v1","addr":"172.21.15.50:6821","nonce":3039646617}]},"public_addr":"172.21.15.50:6813/3039646617","cluster_addr":"172.21.15.50:6817/3039646617","heartbeat_back_addr":"172.21.15.50:6823/3039646617","heartbeat_front_addr":"172.21.15.50:6821/3039646617","state":["exists","up"]},{"osd":3,"uuid":"e5e292cd-9ed4-44c9-9e91-7ee3736272ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3130492744},{"type":"v1","addr":"172.21.15.50:6801","nonce":3130492744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3130492744},{"type":"v1","addr":"172.21.15.50:6803","nonce":3130492744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3130492744},{"type":"v1","addr":"172.21.15.50:6807","nonce":3130492744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3130492744},{"type":"v1","addr":"172.21.15.50:6805","nonce":3130492744}]},"public_addr":"172.21.15.50:6801/3130492744","cluster_addr":"172.21.15.50:6803/3130492744","heartbeat_back_addr":"172.21.15.50:6807/3130492744","heartbeat_front_addr":"172.21.15.50:6805/3130492744","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T18:04:33.736 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.737+0000 7f6ef7442640 1 -- 172.21.15.46:0/2512623803 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ed403d220 msgr2=0x7f6ed403f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:33.736 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.737+0000 7f6ef7442640 1 --2- 172.21.15.46:0/2512623803 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ed403d220 0x7f6ed403f6e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009720 tx=0x7f6ee0008030 comp rx=0 tx=0).stop 2024-05-08T18:04:33.736 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.737+0000 7f6ef7442640 1 -- 172.21.15.46:0/2512623803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ef008aff0 msgr2=0x7f6ef0125b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:33.736 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.737+0000 7f6ef7442640 1 --2- 172.21.15.46:0/2512623803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ef008aff0 0x7f6ef0125b80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00c380 tx=0x7f6eec00c840 comp rx=0 tx=0).stop 2024-05-08T18:04:33.736 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.737+0000 7f6ef7442640 1 -- 172.21.15.46:0/2512623803 shutdown_connections 2024-05-08T18:04:33.736 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.737+0000 7f6ef7442640 1 --2- 172.21.15.46:0/2512623803 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ed403d220 0x7f6ed403f6e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:33.736 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.737+0000 7f6ef7442640 1 --2- 172.21.15.46:0/2512623803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ef008aff0 0x7f6ef0125b80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:33.736 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.737+0000 7f6ef7442640 1 -- 172.21.15.46:0/2512623803 >> 172.21.15.46:0/2512623803 conn(0x7f6ef009d8d0 msgr2=0x7f6ef006fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:33.736 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.737+0000 7f6ef7442640 1 -- 172.21.15.46:0/2512623803 shutdown_connections 2024-05-08T18:04:33.736 INFO:teuthology.misc.health.smithi046.stderr:2024-05-08T18:04:33.737+0000 7f6ef7442640 1 -- 172.21.15.46:0/2512623803 wait complete. 2024-05-08T18:04:33.745 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-05-08T18:04:33.745 INFO:tasks.ceph:Creating RBD pool 2024-05-08T18:04:33.745 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd pool create rbd 8 2024-05-08T18:04:33.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.850+0000 7f0c24191640 1 Processor -- start 2024-05-08T18:04:33.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.850+0000 7f0c24191640 1 -- start start 2024-05-08T18:04:33.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.850+0000 7f0c24191640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0c1c0705f0 0x7f0c1c134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:33.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.850+0000 7f0c24191640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c1c09a2d0 con 0x7f0c1c12f580 2024-05-08T18:04:33.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.850+0000 7f0c24191640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0c1c070f00 con 0x7f0c1c0705f0 2024-05-08T18:04:33.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.850+0000 7f0c2172c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0c1c0705f0 0x7f0c1c134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:33.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.850+0000 7f0c2172c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0c1c0705f0 0x7f0c1c134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39958/0 (socket says 172.21.15.46:39958) 2024-05-08T18:04:33.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.850+0000 7f0c2172c640 1 -- 172.21.15.46:0/3194931309 learned_addr learned my addr 172.21.15.46:0/3194931309 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:33.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.850+0000 7f0c21f2d640 1 --1- 172.21.15.46:0/3194931309 >> v1:172.21.15.46:6789/0 conn(0x7f0c1c12f580 0x7f0c1c12f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39982/0 (socket says 172.21.15.46:39982) 2024-05-08T18:04:33.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.851+0000 7f0c2172c640 1 -- 172.21.15.46:0/3194931309 >> v1:172.21.15.46:6789/0 conn(0x7f0c1c12f580 legacy=0x7f0c1c12f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:33.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.851+0000 7f0c2172c640 1 -- 172.21.15.46:0/3194931309 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1c073fe0 con 0x7f0c1c0705f0 2024-05-08T18:04:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.851+0000 7f0c2172c640 1 --2- 172.21.15.46:0/3194931309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c0705f0 0x7f0c1c134ab0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0097f0 tx=0x7f0c0c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33115e7aa65eb697 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.851+0000 7f0c20f2b640 1 -- 172.21.15.46:0/3194931309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c0c034070 con 0x7f0c1c0705f0 2024-05-08T18:04:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.852+0000 7f0c20f2b640 1 -- 172.21.15.46:0/3194931309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c0c030790 con 0x7f0c1c0705f0 2024-05-08T18:04:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.852+0000 7f0c20f2b640 1 -- 172.21.15.46:0/3194931309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c0c030a90 con 0x7f0c1c0705f0 2024-05-08T18:04:33.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.852+0000 7f0c24191640 1 -- 172.21.15.46:0/3194931309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c0705f0 msgr2=0x7f0c1c134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.852+0000 7f0c24191640 1 --2- 172.21.15.46:0/3194931309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c0705f0 0x7f0c1c134ab0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0097f0 tx=0x7f0c0c02b9a0 comp rx=0 tx=0).stop 2024-05-08T18:04:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.852+0000 7f0c24191640 1 -- 172.21.15.46:0/3194931309 shutdown_connections 2024-05-08T18:04:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.852+0000 7f0c24191640 1 --2- 172.21.15.46:0/3194931309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c0705f0 0x7f0c1c134ab0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.852+0000 7f0c24191640 1 -- 172.21.15.46:0/3194931309 >> 172.21.15.46:0/3194931309 conn(0x7f0c1c09d8d0 msgr2=0x7f0c1c073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.852+0000 7f0c24191640 1 -- 172.21.15.46:0/3194931309 shutdown_connections 2024-05-08T18:04:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.852+0000 7f0c24191640 1 -- 172.21.15.46:0/3194931309 wait complete. 2024-05-08T18:04:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.853+0000 7f0c24191640 1 Processor -- start 2024-05-08T18:04:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.853+0000 7f0c24191640 1 -- start start 2024-05-08T18:04:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.853+0000 7f0c24191640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c0705f0 0x7f0c1c11e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.853+0000 7f0c24191640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c1301f0 con 0x7f0c1c0705f0 2024-05-08T18:04:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.853+0000 7f0c21f2d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c0705f0 0x7f0c1c11e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.853+0000 7f0c21f2d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c0705f0 0x7f0c1c11e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39968/0 (socket says 172.21.15.46:39968) 2024-05-08T18:04:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.853+0000 7f0c21f2d640 1 -- 172.21.15.46:0/3996742582 learned_addr learned my addr 172.21.15.46:0/3996742582 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:33.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.854+0000 7f0c21f2d640 1 -- 172.21.15.46:0/3996742582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1c134a60 con 0x7f0c1c0705f0 2024-05-08T18:04:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.854+0000 7f0c21f2d640 1 --2- 172.21.15.46:0/3996742582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c0705f0 0x7f0c1c11e450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800c450 tx=0x7f0c1800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.854+0000 7f0c127fc640 1 -- 172.21.15.46:0/3996742582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c18007410 con 0x7f0c1c0705f0 2024-05-08T18:04:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.854+0000 7f0c127fc640 1 -- 172.21.15.46:0/3996742582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c180075b0 con 0x7f0c1c0705f0 2024-05-08T18:04:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.854+0000 7f0c24191640 1 -- 172.21.15.46:0/3996742582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1c121160 con 0x7f0c1c0705f0 2024-05-08T18:04:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.854+0000 7f0c127fc640 1 -- 172.21.15.46:0/3996742582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c18004950 con 0x7f0c1c0705f0 2024-05-08T18:04:33.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.854+0000 7f0c24191640 1 -- 172.21.15.46:0/3996742582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1c11e990 con 0x7f0c1c0705f0 2024-05-08T18:04:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.855+0000 7f0c127fc640 1 -- 172.21.15.46:0/3996742582 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 49355+0+0 (secure 0 0 0) 0x7f0c18004af0 con 0x7f0c1c0705f0 2024-05-08T18:04:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.855+0000 7f0c24191640 1 -- 172.21.15.46:0/3996742582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c1c0640a0 con 0x7f0c1c0705f0 2024-05-08T18:04:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.855+0000 7f0c127fc640 1 --2- 172.21.15.46:0/3996742582 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0c0003d220 0x7f0c0003f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.855+0000 7f0c127fc640 1 -- 172.21.15.46:0/3996742582 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 2743+0+0 (secure 0 0 0) 0x7f0c180504c0 con 0x7f0c1c0705f0 2024-05-08T18:04:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.856+0000 7f0c2172c640 1 --2- 172.21.15.46:0/3996742582 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0c0003d220 0x7f0c0003f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:33.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.856+0000 7f0c2172c640 1 --2- 172.21.15.46:0/3996742582 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0c0003d220 0x7f0c0003f6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c036000 tx=0x7f0c0c03b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:33.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:33.860+0000 7f0c127fc640 1 -- 172.21.15.46:0/3996742582 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0c18019310 con 0x7f0c1c0705f0 2024-05-08T18:04:34.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.050+0000 7f0c24191640 1 -- 172.21.15.46:0/3996742582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f0c1c11f640 con 0x7f0c1c0705f0 2024-05-08T18:04:34.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.543+0000 7f0c127fc640 1 -- 172.21.15.46:0/3996742582 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v10) ==== 109+0+0 (secure 0 0 0) 0x7f0c18011dc0 con 0x7f0c1c0705f0 2024-05-08T18:04:34.541 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'rbd' created 2024-05-08T18:04:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.546+0000 7f0c24191640 1 -- 172.21.15.46:0/3996742582 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0c0003d220 msgr2=0x7f0c0003f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.546+0000 7f0c24191640 1 --2- 172.21.15.46:0/3996742582 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0c0003d220 0x7f0c0003f6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c036000 tx=0x7f0c0c03b040 comp rx=0 tx=0).stop 2024-05-08T18:04:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.546+0000 7f0c24191640 1 -- 172.21.15.46:0/3996742582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c0705f0 msgr2=0x7f0c1c11e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.546+0000 7f0c24191640 1 --2- 172.21.15.46:0/3996742582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c0705f0 0x7f0c1c11e450 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800c450 tx=0x7f0c1800c910 comp rx=0 tx=0).stop 2024-05-08T18:04:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.546+0000 7f0c24191640 1 -- 172.21.15.46:0/3996742582 shutdown_connections 2024-05-08T18:04:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.546+0000 7f0c24191640 1 --2- 172.21.15.46:0/3996742582 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0c0003d220 0x7f0c0003f6e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.546+0000 7f0c24191640 1 --2- 172.21.15.46:0/3996742582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c1c0705f0 0x7f0c1c11e450 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.546+0000 7f0c24191640 1 -- 172.21.15.46:0/3996742582 >> 172.21.15.46:0/3996742582 conn(0x7f0c1c09d8d0 msgr2=0x7f0c1c09a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:34.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.546+0000 7f0c24191640 1 -- 172.21.15.46:0/3996742582 shutdown_connections 2024-05-08T18:04:34.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.546+0000 7f0c24191640 1 -- 172.21.15.46:0/3996742582 wait complete. 2024-05-08T18:04:34.556 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-05-08T18:04:34.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.664+0000 7fbc3fe59640 1 Processor -- start 2024-05-08T18:04:34.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.664+0000 7fbc3fe59640 1 -- start start 2024-05-08T18:04:34.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.665+0000 7fbc3fe59640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc3806fed0 0x7fbc3812c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:34.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.665+0000 7fbc3fe59640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc380737e0 con 0x7fbc3812c9c0 2024-05-08T18:04:34.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.665+0000 7fbc3fe59640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fbc38073250 con 0x7fbc3806fed0 2024-05-08T18:04:34.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.665+0000 7fbc3dbf5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc3806fed0 0x7fbc3812c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:34.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.665+0000 7fbc3d3f4640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbc3812c9c0 0x7fbc38071870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:39994/0 (socket says 172.21.15.46:39994) 2024-05-08T18:04:34.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.665+0000 7fbc3d3f4640 1 -- 172.21.15.46:0/4247374171 learned_addr learned my addr 172.21.15.46:0/4247374171 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:34.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.665+0000 7fbc3dbf5640 1 -- 172.21.15.46:0/4247374171 >> v1:172.21.15.46:6789/0 conn(0x7fbc3812c9c0 legacy=0x7fbc38071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:34.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.665+0000 7fbc3dbf5640 1 -- 172.21.15.46:0/4247374171 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3812c090 con 0x7fbc3806fed0 2024-05-08T18:04:34.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.666+0000 7fbc3dbf5640 1 --2- 172.21.15.46:0/4247374171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3806fed0 0x7fbc3812c480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009720 tx=0x7fbc3402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad867979f90b4a73 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:34.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.666+0000 7fbc3cbf3640 1 -- 172.21.15.46:0/4247374171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc340316d0 con 0x7fbc3806fed0 2024-05-08T18:04:34.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.666+0000 7fbc3cbf3640 1 -- 172.21.15.46:0/4247374171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc34037070 con 0x7fbc3806fed0 2024-05-08T18:04:34.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.666+0000 7fbc3cbf3640 1 -- 172.21.15.46:0/4247374171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc340319c0 con 0x7fbc3806fed0 2024-05-08T18:04:34.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.667+0000 7fbc3fe59640 1 -- 172.21.15.46:0/4247374171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3806fed0 msgr2=0x7fbc3812c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:34.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.667+0000 7fbc3fe59640 1 --2- 172.21.15.46:0/4247374171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3806fed0 0x7fbc3812c480 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009720 tx=0x7fbc3402b840 comp rx=0 tx=0).stop 2024-05-08T18:04:34.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.667+0000 7fbc3fe59640 1 -- 172.21.15.46:0/4247374171 shutdown_connections 2024-05-08T18:04:34.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.667+0000 7fbc3fe59640 1 --2- 172.21.15.46:0/4247374171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3806fed0 0x7fbc3812c480 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:34.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.667+0000 7fbc3fe59640 1 -- 172.21.15.46:0/4247374171 >> 172.21.15.46:0/4247374171 conn(0x7fbc3809ab60 msgr2=0x7fbc3806e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:34.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.667+0000 7fbc3fe59640 1 -- 172.21.15.46:0/4247374171 shutdown_connections 2024-05-08T18:04:34.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.667+0000 7fbc3fe59640 1 -- 172.21.15.46:0/4247374171 wait complete. 2024-05-08T18:04:34.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.668+0000 7fbc3fe59640 1 Processor -- start 2024-05-08T18:04:34.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.668+0000 7fbc3fe59640 1 -- start start 2024-05-08T18:04:34.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.668+0000 7fbc3fe59640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3806fed0 0x7fbc381c2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:34.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.668+0000 7fbc3fe59640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbc38071f80 con 0x7fbc3806fed0 2024-05-08T18:04:34.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.668+0000 7fbc3dbf5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3806fed0 0x7fbc381c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:34.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.668+0000 7fbc3dbf5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3806fed0 0x7fbc381c2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39980/0 (socket says 172.21.15.46:39980) 2024-05-08T18:04:34.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.668+0000 7fbc3dbf5640 1 -- 172.21.15.46:0/3542614898 learned_addr learned my addr 172.21.15.46:0/3542614898 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.668+0000 7fbc3dbf5640 1 -- 172.21.15.46:0/3542614898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc381186c0 con 0x7fbc3806fed0 2024-05-08T18:04:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.669+0000 7fbc3dbf5640 1 --2- 172.21.15.46:0/3542614898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3806fed0 0x7fbc381c2260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc34034040 tx=0x7fbc340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.669+0000 7fbc2e7fc640 1 -- 172.21.15.46:0/3542614898 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc34032640 con 0x7fbc3806fed0 2024-05-08T18:04:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.669+0000 7fbc2e7fc640 1 -- 172.21.15.46:0/3542614898 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc340327e0 con 0x7fbc3806fed0 2024-05-08T18:04:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.669+0000 7fbc3fe59640 1 -- 172.21.15.46:0/3542614898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc38127ef0 con 0x7fbc3806fed0 2024-05-08T18:04:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.669+0000 7fbc2e7fc640 1 -- 172.21.15.46:0/3542614898 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc34032ac0 con 0x7fbc3806fed0 2024-05-08T18:04:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.669+0000 7fbc3fe59640 1 -- 172.21.15.46:0/3542614898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc38119350 con 0x7fbc3806fed0 2024-05-08T18:04:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.670+0000 7fbc3fe59640 1 -- 172.21.15.46:0/3542614898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc38119630 con 0x7fbc3806fed0 2024-05-08T18:04:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.670+0000 7fbc2e7fc640 1 -- 172.21.15.46:0/3542614898 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 49355+0+0 (secure 0 0 0) 0x7fbc3404c060 con 0x7fbc3806fed0 2024-05-08T18:04:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.670+0000 7fbc2e7fc640 1 --2- 172.21.15.46:0/3542614898 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbc1c03d220 0x7fbc1c03f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.670+0000 7fbc2e7fc640 1 -- 172.21.15.46:0/3542614898 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 3460+0+0 (secure 0 0 0) 0x7fbc3402f050 con 0x7fbc3806fed0 2024-05-08T18:04:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.670+0000 7fbc3d3f4640 1 --2- 172.21.15.46:0/3542614898 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbc1c03d220 0x7fbc1c03f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:34.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.671+0000 7fbc3d3f4640 1 --2- 172.21.15.46:0/3542614898 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbc1c03d220 0x7fbc1c03f6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbc28007fd0 tx=0x7fbc28007aa0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:34.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.675+0000 7fbc2e7fc640 1 -- 172.21.15.46:0/3542614898 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fbc34040070 con 0x7fbc3806fed0 2024-05-08T18:04:34.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:34.865+0000 7fbc3fe59640 1 -- 172.21.15.46:0/3542614898 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) -- 0x7fbc380640a0 con 0x7fbc3806fed0 2024-05-08T18:04:35.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.544+0000 7fbc2e7fc640 1 -- 172.21.15.46:0/3542614898 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v11) ==== 173+0+0 (secure 0 0 0) 0x7fbc3403a380 con 0x7fbc3806fed0 2024-05-08T18:04:35.542 INFO:teuthology.orchestra.run.smithi046.stderr:enabled application 'rbd' on pool 'rbd' 2024-05-08T18:04:35.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.547+0000 7fbc3fe59640 1 -- 172.21.15.46:0/3542614898 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbc1c03d220 msgr2=0x7fbc1c03f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:35.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.547+0000 7fbc3fe59640 1 --2- 172.21.15.46:0/3542614898 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbc1c03d220 0x7fbc1c03f6e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fbc28007fd0 tx=0x7fbc28007aa0 comp rx=0 tx=0).stop 2024-05-08T18:04:35.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.547+0000 7fbc3fe59640 1 -- 172.21.15.46:0/3542614898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3806fed0 msgr2=0x7fbc381c2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:35.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.547+0000 7fbc3fe59640 1 --2- 172.21.15.46:0/3542614898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3806fed0 0x7fbc381c2260 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc34034040 tx=0x7fbc340046c0 comp rx=0 tx=0).stop 2024-05-08T18:04:35.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.547+0000 7fbc3fe59640 1 -- 172.21.15.46:0/3542614898 shutdown_connections 2024-05-08T18:04:35.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.547+0000 7fbc3fe59640 1 --2- 172.21.15.46:0/3542614898 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbc1c03d220 0x7fbc1c03f6e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:35.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.547+0000 7fbc3fe59640 1 --2- 172.21.15.46:0/3542614898 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc3806fed0 0x7fbc381c2260 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:35.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.547+0000 7fbc3fe59640 1 -- 172.21.15.46:0/3542614898 >> 172.21.15.46:0/3542614898 conn(0x7fbc3809ab60 msgr2=0x7fbc38122780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:35.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.548+0000 7fbc3fe59640 1 -- 172.21.15.46:0/3542614898 shutdown_connections 2024-05-08T18:04:35.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.548+0000 7fbc3fe59640 1 -- 172.21.15.46:0/3542614898 wait complete. 2024-05-08T18:04:35.560 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-08T18:04:35.561 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T18:04:35.561 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-08T18:04:35.561 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-08T18:04:35.561 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-08T18:04:35.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.675+0000 7f70daeae640 1 Processor -- start 2024-05-08T18:04:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.675+0000 7f70daeae640 1 -- start start 2024-05-08T18:04:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.675+0000 7f70daeae640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f70d4070f20 0x7f70d4071300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.675+0000 7f70daeae640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70d4072da0 con 0x7f70d4071840 2024-05-08T18:04:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.675+0000 7f70daeae640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f70d4073040 con 0x7f70d4070f20 2024-05-08T18:04:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.675+0000 7f70d8c4a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f70d4070f20 0x7f70d4071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.675+0000 7f70d3fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f70d4071840 0x7f70d412c160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40008/0 (socket says 172.21.15.46:40008) 2024-05-08T18:04:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.675+0000 7f70d3fff640 1 -- 172.21.15.46:0/1648718373 learned_addr learned my addr 172.21.15.46:0/1648718373 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.676+0000 7f70d8c4a640 1 -- 172.21.15.46:0/1648718373 >> v1:172.21.15.46:6789/0 conn(0x7f70d4071840 legacy=0x7f70d412c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:35.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.676+0000 7f70d8c4a640 1 -- 172.21.15.46:0/1648718373 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d4071c50 con 0x7f70d4070f20 2024-05-08T18:04:35.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.676+0000 7f70d8c4a640 1 --2- 172.21.15.46:0/1648718373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d4070f20 0x7f70d4071300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009720 tx=0x7f70c802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922ec8fa326067c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:35.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.677+0000 7f70d37fe640 1 -- 172.21.15.46:0/1648718373 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70c80316d0 con 0x7f70d4070f20 2024-05-08T18:04:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.677+0000 7f70d37fe640 1 -- 172.21.15.46:0/1648718373 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c8037070 con 0x7f70d4070f20 2024-05-08T18:04:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.677+0000 7f70d37fe640 1 -- 172.21.15.46:0/1648718373 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70c80319c0 con 0x7f70d4070f20 2024-05-08T18:04:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.677+0000 7f70daeae640 1 -- 172.21.15.46:0/1648718373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d4070f20 msgr2=0x7f70d4071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.677+0000 7f70daeae640 1 --2- 172.21.15.46:0/1648718373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d4070f20 0x7f70d4071300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009720 tx=0x7f70c802b840 comp rx=0 tx=0).stop 2024-05-08T18:04:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.677+0000 7f70daeae640 1 -- 172.21.15.46:0/1648718373 shutdown_connections 2024-05-08T18:04:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.677+0000 7f70daeae640 1 --2- 172.21.15.46:0/1648718373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d4070f20 0x7f70d4071300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.677+0000 7f70daeae640 1 -- 172.21.15.46:0/1648718373 >> 172.21.15.46:0/1648718373 conn(0x7f70d409de50 msgr2=0x7f70d409e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.677+0000 7f70daeae640 1 -- 172.21.15.46:0/1648718373 shutdown_connections 2024-05-08T18:04:35.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.677+0000 7f70daeae640 1 -- 172.21.15.46:0/1648718373 wait complete. 2024-05-08T18:04:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.678+0000 7f70daeae640 1 Processor -- start 2024-05-08T18:04:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.678+0000 7f70daeae640 1 -- start start 2024-05-08T18:04:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.679+0000 7f70daeae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d4070f20 0x7f70d411a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.679+0000 7f70daeae640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f70d412c9f0 con 0x7f70d4070f20 2024-05-08T18:04:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.679+0000 7f70d8c4a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d4070f20 0x7f70d411a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.679+0000 7f70d8c4a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d4070f20 0x7f70d411a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39990/0 (socket says 172.21.15.46:39990) 2024-05-08T18:04:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.679+0000 7f70d8c4a640 1 -- 172.21.15.46:0/4141047112 learned_addr learned my addr 172.21.15.46:0/4141047112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.679+0000 7f70d8c4a640 1 -- 172.21.15.46:0/4141047112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d411d430 con 0x7f70d4070f20 2024-05-08T18:04:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.679+0000 7f70d8c4a640 1 --2- 172.21.15.46:0/4141047112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d4070f20 0x7f70d411a820 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f70c8034040 tx=0x7f70c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.679+0000 7f70d17fa640 1 -- 172.21.15.46:0/4141047112 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70c8032770 con 0x7f70d4070f20 2024-05-08T18:04:35.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.679+0000 7f70d17fa640 1 -- 172.21.15.46:0/4141047112 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70c8032930 con 0x7f70d4070f20 2024-05-08T18:04:35.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.679+0000 7f70d17fa640 1 -- 172.21.15.46:0/4141047112 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70c8032c50 con 0x7f70d4070f20 2024-05-08T18:04:35.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.679+0000 7f70daeae640 1 -- 172.21.15.46:0/4141047112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d411bfb0 con 0x7f70d4070f20 2024-05-08T18:04:35.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.680+0000 7f70daeae640 1 -- 172.21.15.46:0/4141047112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d412a1f0 con 0x7f70d4070f20 2024-05-08T18:04:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.680+0000 7f70d17fa640 1 -- 172.21.15.46:0/4141047112 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 49355+0+0 (secure 0 0 0) 0x7f70c804c060 con 0x7f70d4070f20 2024-05-08T18:04:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.681+0000 7f70daeae640 1 -- 172.21.15.46:0/4141047112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d412c7f0 con 0x7f70d4070f20 2024-05-08T18:04:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.681+0000 7f70d17fa640 1 --2- 172.21.15.46:0/4141047112 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f70b803d220 0x7f70b803f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.681+0000 7f70d17fa640 1 -- 172.21.15.46:0/4141047112 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 3482+0+0 (secure 0 0 0) 0x7f70c802f050 con 0x7f70d4070f20 2024-05-08T18:04:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.681+0000 7f70d3fff640 1 --2- 172.21.15.46:0/4141047112 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f70b803d220 0x7f70b803f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:35.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.682+0000 7f70d3fff640 1 --2- 172.21.15.46:0/4141047112 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f70b803d220 0x7f70b803f6e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f70c0007fd0 tx=0x7f70c0007aa0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:35.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.686+0000 7f70d17fa640 1 -- 172.21.15.46:0/4141047112 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f70c8040070 con 0x7f70d4070f20 2024-05-08T18:04:35.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.935+0000 7f70daeae640 1 -- 172.21.15.46:0/4141047112 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f70d40640a0 con 0x7f70d4070f20 2024-05-08T18:04:35.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.937+0000 7f70d17fa640 1 -- 172.21.15.46:0/4141047112 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) ==== 74+0+95398 (secure 0 0 0) 0x7f70c804dc40 con 0x7f70d4070f20 2024-05-08T18:04:35.936 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:35.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.939+0000 7f70daeae640 1 -- 172.21.15.46:0/4141047112 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f70b803d220 msgr2=0x7f70b803f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:35.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.939+0000 7f70daeae640 1 --2- 172.21.15.46:0/4141047112 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f70b803d220 0x7f70b803f6e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f70c0007fd0 tx=0x7f70c0007aa0 comp rx=0 tx=0).stop 2024-05-08T18:04:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.939+0000 7f70daeae640 1 -- 172.21.15.46:0/4141047112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d4070f20 msgr2=0x7f70d411a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.939+0000 7f70daeae640 1 --2- 172.21.15.46:0/4141047112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d4070f20 0x7f70d411a820 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f70c8034040 tx=0x7f70c80046c0 comp rx=0 tx=0).stop 2024-05-08T18:04:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.939+0000 7f70daeae640 1 -- 172.21.15.46:0/4141047112 shutdown_connections 2024-05-08T18:04:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.939+0000 7f70daeae640 1 --2- 172.21.15.46:0/4141047112 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f70b803d220 0x7f70b803f6e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.939+0000 7f70daeae640 1 --2- 172.21.15.46:0/4141047112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f70d4070f20 0x7f70d411a820 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.939+0000 7f70daeae640 1 -- 172.21.15.46:0/4141047112 >> 172.21.15.46:0/4141047112 conn(0x7f70d409de50 msgr2=0x7f70d41b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.939+0000 7f70daeae640 1 -- 172.21.15.46:0/4141047112 shutdown_connections 2024-05-08T18:04:35.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:35.939+0000 7f70daeae640 1 -- 172.21.15.46:0/4141047112 wait complete. 2024-05-08T18:04:35.948 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":4,"flags":0,"active_gid":4098,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":2602185226},{"type":"v1","addr":"172.21.15.50:6825","nonce":2602185226}]},"active_addr":"172.21.15.50:6825/2602185226","active_change":"2024-05-08T18:04:29.659011+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":2902081163}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":3917375992}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.50:0","nonce":2594843589}]}]} 2024-05-08T18:04:35.950 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-08T18:04:35.951 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T18:04:35.951 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T18:04:36.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.054+0000 7f1b7ac63640 1 Processor -- start 2024-05-08T18:04:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.055+0000 7f1b7ac63640 1 -- start start 2024-05-08T18:04:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.055+0000 7f1b7ac63640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b74070650 0x7f1b74070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.055+0000 7f1b7ac63640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b74073480 con 0x7f1b74072540 2024-05-08T18:04:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.055+0000 7f1b7ac63640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1b74071a80 con 0x7f1b74070650 2024-05-08T18:04:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.055+0000 7f1b789ff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1b74070650 0x7f1b74070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.055+0000 7f1b73fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1b74072540 0x7f1b7412f580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40016/0 (socket says 172.21.15.46:40016) 2024-05-08T18:04:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.055+0000 7f1b73fff640 1 -- 172.21.15.46:0/122017138 learned_addr learned my addr 172.21.15.46:0/122017138 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.056+0000 7f1b789ff640 1 -- 172.21.15.46:0/122017138 >> v1:172.21.15.46:6789/0 conn(0x7f1b74072540 legacy=0x7f1b7412f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.056+0000 7f1b789ff640 1 -- 172.21.15.46:0/122017138 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b74072c20 con 0x7f1b74070650 2024-05-08T18:04:36.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.056+0000 7f1b789ff640 1 --2- 172.21.15.46:0/122017138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74070650 0x7f1b74070a30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009720 tx=0x7f1b6402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beaf482bd09c3181 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.056+0000 7f1b737fe640 1 -- 172.21.15.46:0/122017138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b640316d0 con 0x7f1b74070650 2024-05-08T18:04:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.057+0000 7f1b737fe640 1 -- 172.21.15.46:0/122017138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b64037070 con 0x7f1b74070650 2024-05-08T18:04:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.057+0000 7f1b737fe640 1 -- 172.21.15.46:0/122017138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b640319c0 con 0x7f1b74070650 2024-05-08T18:04:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.057+0000 7f1b7ac63640 1 -- 172.21.15.46:0/122017138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74070650 msgr2=0x7f1b74070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.057+0000 7f1b7ac63640 1 --2- 172.21.15.46:0/122017138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74070650 0x7f1b74070a30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1b64009720 tx=0x7f1b6402b840 comp rx=0 tx=0).stop 2024-05-08T18:04:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.057+0000 7f1b7ac63640 1 -- 172.21.15.46:0/122017138 shutdown_connections 2024-05-08T18:04:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.057+0000 7f1b7ac63640 1 --2- 172.21.15.46:0/122017138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74070650 0x7f1b74070a30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.057+0000 7f1b7ac63640 1 -- 172.21.15.46:0/122017138 >> 172.21.15.46:0/122017138 conn(0x7f1b740a1e60 msgr2=0x7f1b740a2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.057+0000 7f1b7ac63640 1 -- 172.21.15.46:0/122017138 shutdown_connections 2024-05-08T18:04:36.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.057+0000 7f1b7ac63640 1 -- 172.21.15.46:0/122017138 wait complete. 2024-05-08T18:04:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.058+0000 7f1b7ac63640 1 Processor -- start 2024-05-08T18:04:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.058+0000 7f1b7ac63640 1 -- start start 2024-05-08T18:04:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.059+0000 7f1b7ac63640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74070650 0x7f1b741c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.059+0000 7f1b7ac63640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7412fc90 con 0x7f1b74070650 2024-05-08T18:04:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.059+0000 7f1b789ff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74070650 0x7f1b741c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.059+0000 7f1b789ff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74070650 0x7f1b741c4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40016/0 (socket says 172.21.15.46:40016) 2024-05-08T18:04:36.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.059+0000 7f1b789ff640 1 -- 172.21.15.46:0/774372275 learned_addr learned my addr 172.21.15.46:0/774372275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.059+0000 7f1b789ff640 1 -- 172.21.15.46:0/774372275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7412c5d0 con 0x7f1b74070650 2024-05-08T18:04:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.059+0000 7f1b789ff640 1 --2- 172.21.15.46:0/774372275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74070650 0x7f1b741c4450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b64034040 tx=0x7f1b640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.059+0000 7f1b717fa640 1 -- 172.21.15.46:0/774372275 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b64032770 con 0x7f1b74070650 2024-05-08T18:04:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.059+0000 7f1b717fa640 1 -- 172.21.15.46:0/774372275 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b64032910 con 0x7f1b74070650 2024-05-08T18:04:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.059+0000 7f1b7ac63640 1 -- 172.21.15.46:0/774372275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7411e9b0 con 0x7f1b74070650 2024-05-08T18:04:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.059+0000 7f1b717fa640 1 -- 172.21.15.46:0/774372275 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b64032c30 con 0x7f1b74070650 2024-05-08T18:04:36.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.059+0000 7f1b7ac63640 1 -- 172.21.15.46:0/774372275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7411f640 con 0x7f1b74070650 2024-05-08T18:04:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.060+0000 7f1b717fa640 1 -- 172.21.15.46:0/774372275 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 49355+0+0 (secure 0 0 0) 0x7f1b6404c060 con 0x7f1b74070650 2024-05-08T18:04:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.060+0000 7f1b7ac63640 1 -- 172.21.15.46:0/774372275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b74120150 con 0x7f1b74070650 2024-05-08T18:04:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.061+0000 7f1b717fa640 1 --2- 172.21.15.46:0/774372275 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1b6003d220 0x7f1b6003f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.061+0000 7f1b717fa640 1 -- 172.21.15.46:0/774372275 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 3482+0+0 (secure 0 0 0) 0x7f1b6402f050 con 0x7f1b74070650 2024-05-08T18:04:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.061+0000 7f1b73fff640 1 --2- 172.21.15.46:0/774372275 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1b6003d220 0x7f1b6003f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.062+0000 7f1b73fff640 1 --2- 172.21.15.46:0/774372275 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1b6003d220 0x7f1b6003f6e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1b68007fd0 tx=0x7f1b68007aa0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.066+0000 7f1b717fa640 1 -- 172.21.15.46:0/774372275 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f1b7412c5d0 con 0x7f1b74070650 2024-05-08T18:04:36.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.260+0000 7f1b7ac63640 1 -- 172.21.15.46:0/774372275 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1b740640a0 con 0x7f1b74070650 2024-05-08T18:04:36.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.261+0000 7f1b717fa640 1 -- 172.21.15.46:0/774372275 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) ==== 74+0+9733 (secure 0 0 0) 0x7f1b64040070 con 0x7f1b74070650 2024-05-08T18:04:36.259 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:36.259 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":11,"fsid":"d8088d1f-616e-46b1-995a-85e572267d38","created":"2024-05-08T18:04:25.152571+0000","modified":"2024-05-08T18:04:35.544175+0000","last_up_change":"2024-05-08T18:04:33.542468+0000","last_in_change":"2024-05-08T18:04:26.221033+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T18:04:33.679407+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T18:04:34.053953+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7304d61-4d9e-455b-a968-5641c682e642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":337143975},{"type":"v1","addr":"172.21.15.46:6801","nonce":337143975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":337143975},{"type":"v1","addr":"172.21.15.46:6803","nonce":337143975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":337143975},{"type":"v1","addr":"172.21.15.46:6807","nonce":337143975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":337143975},{"type":"v1","addr":"172.21.15.46:6805","nonce":337143975}]},"public_addr":"172.21.15.46:6801/337143975","cluster_addr":"172.21.15.46:6803/337143975","heartbeat_back_addr":"172.21.15.46:6807/337143975","heartbeat_front_addr":"172.21.15.46:6805/337143975","state":["exists","up"]},{"osd":1,"uuid":"9a62c75d-b151-4453-a538-14408e8f4b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2141923372},{"type":"v1","addr":"172.21.15.50:6809","nonce":2141923372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2141923372},{"type":"v1","addr":"172.21.15.50:6812","nonce":2141923372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2141923372},{"type":"v1","addr":"172.21.15.50:6820","nonce":2141923372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2141923372},{"type":"v1","addr":"172.21.15.50:6816","nonce":2141923372}]},"public_addr":"172.21.15.50:6809/2141923372","cluster_addr":"172.21.15.50:6812/2141923372","heartbeat_back_addr":"172.21.15.50:6820/2141923372","heartbeat_front_addr":"172.21.15.50:6816/2141923372","state":["exists","up"]},{"osd":2,"uuid":"c2d4e8c1-79bb-4061-ade1-f0fc079c52f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":3039646617},{"type":"v1","addr":"172.21.15.50:6813","nonce":3039646617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6815","nonce":3039646617},{"type":"v1","addr":"172.21.15.50:6817","nonce":3039646617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3039646617},{"type":"v1","addr":"172.21.15.50:6823","nonce":3039646617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":3039646617},{"type":"v1","addr":"172.21.15.50:6821","nonce":3039646617}]},"public_addr":"172.21.15.50:6813/3039646617","cluster_addr":"172.21.15.50:6817/3039646617","heartbeat_back_addr":"172.21.15.50:6823/3039646617","heartbeat_front_addr":"172.21.15.50:6821/3039646617","state":["exists","up"]},{"osd":3,"uuid":"e5e292cd-9ed4-44c9-9e91-7ee3736272ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3130492744},{"type":"v1","addr":"172.21.15.50:6801","nonce":3130492744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3130492744},{"type":"v1","addr":"172.21.15.50:6803","nonce":3130492744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3130492744},{"type":"v1","addr":"172.21.15.50:6807","nonce":3130492744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3130492744},{"type":"v1","addr":"172.21.15.50:6805","nonce":3130492744}]},"public_addr":"172.21.15.50:6801/3130492744","cluster_addr":"172.21.15.50:6803/3130492744","heartbeat_back_addr":"172.21.15.50:6807/3130492744","heartbeat_front_addr":"172.21.15.50:6805/3130492744","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T18:04:31.282728+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T18:04:31.387361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T18:04:31.386518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T18:04:31.319396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T18:04:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.263+0000 7f1b7ac63640 1 -- 172.21.15.46:0/774372275 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1b6003d220 msgr2=0x7f1b6003f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.263+0000 7f1b7ac63640 1 --2- 172.21.15.46:0/774372275 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1b6003d220 0x7f1b6003f6e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1b68007fd0 tx=0x7f1b68007aa0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.263+0000 7f1b7ac63640 1 -- 172.21.15.46:0/774372275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74070650 msgr2=0x7f1b741c4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.263+0000 7f1b7ac63640 1 --2- 172.21.15.46:0/774372275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74070650 0x7f1b741c4450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1b64034040 tx=0x7f1b640046c0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.263+0000 7f1b7ac63640 1 -- 172.21.15.46:0/774372275 shutdown_connections 2024-05-08T18:04:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.263+0000 7f1b7ac63640 1 --2- 172.21.15.46:0/774372275 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1b6003d220 0x7f1b6003f6e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.263+0000 7f1b7ac63640 1 --2- 172.21.15.46:0/774372275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1b74070650 0x7f1b741c4450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.263+0000 7f1b7ac63640 1 -- 172.21.15.46:0/774372275 >> 172.21.15.46:0/774372275 conn(0x7f1b740a1e60 msgr2=0x7f1b7412bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.263+0000 7f1b7ac63640 1 -- 172.21.15.46:0/774372275 shutdown_connections 2024-05-08T18:04:36.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.263+0000 7f1b7ac63640 1 -- 172.21.15.46:0/774372275 wait complete. 2024-05-08T18:04:36.273 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T18:04:36.273 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T18:04:36.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.377+0000 7fca89eec640 1 Processor -- start 2024-05-08T18:04:36.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.377+0000 7fca89eec640 1 -- start start 2024-05-08T18:04:36.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.377+0000 7fca89eec640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca8412f420 0x7fca84131810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.377+0000 7fca89eec640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca8412c230 con 0x7fca8412c530 2024-05-08T18:04:36.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.377+0000 7fca89eec640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fca8409ad20 con 0x7fca8412f420 2024-05-08T18:04:36.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.378+0000 7fca82ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fca8412f420 0x7fca84131810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.378+0000 7fca837fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fca8412c530 0x7fca8412c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40030/0 (socket says 172.21.15.46:40030) 2024-05-08T18:04:36.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.378+0000 7fca837fe640 1 -- 172.21.15.46:0/677200810 learned_addr learned my addr 172.21.15.46:0/677200810 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:36.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.378+0000 7fca82ffd640 1 -- 172.21.15.46:0/677200810 >> v1:172.21.15.46:6789/0 conn(0x7fca8412c530 legacy=0x7fca8412c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.378+0000 7fca82ffd640 1 -- 172.21.15.46:0/677200810 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca84073510 con 0x7fca8412f420 2024-05-08T18:04:36.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.378+0000 7fca827fc640 1 -- 172.21.15.46:0/677200810 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 20823182 0 0) 0x7fca8412c230 con 0x7fca8412c530 2024-05-08T18:04:36.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.379+0000 7fca82ffd640 1 --2- 172.21.15.46:0/677200810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8412f420 0x7fca84131810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fca78009720 tx=0x7fca7802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23605e0489f96ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.379+0000 7fca827fc640 1 -- 172.21.15.46:0/677200810 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca780316d0 con 0x7fca8412f420 2024-05-08T18:04:36.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.379+0000 7fca827fc640 1 -- 172.21.15.46:0/677200810 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca78037070 con 0x7fca8412f420 2024-05-08T18:04:36.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.379+0000 7fca827fc640 1 -- 172.21.15.46:0/677200810 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca780319c0 con 0x7fca8412f420 2024-05-08T18:04:36.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.379+0000 7fca89eec640 1 -- 172.21.15.46:0/677200810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8412f420 msgr2=0x7fca84131810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.379+0000 7fca89eec640 1 --2- 172.21.15.46:0/677200810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8412f420 0x7fca84131810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fca78009720 tx=0x7fca7802b840 comp rx=0 tx=0).stop 2024-05-08T18:04:36.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.379+0000 7fca89eec640 1 -- 172.21.15.46:0/677200810 shutdown_connections 2024-05-08T18:04:36.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.379+0000 7fca89eec640 1 --2- 172.21.15.46:0/677200810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8412f420 0x7fca84131810 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.379+0000 7fca89eec640 1 -- 172.21.15.46:0/677200810 >> 172.21.15.46:0/677200810 conn(0x7fca8409d7a0 msgr2=0x7fca8409dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:36.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.379+0000 7fca89eec640 1 -- 172.21.15.46:0/677200810 shutdown_connections 2024-05-08T18:04:36.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.379+0000 7fca89eec640 1 -- 172.21.15.46:0/677200810 wait complete. 2024-05-08T18:04:36.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.380+0000 7fca89eec640 1 Processor -- start 2024-05-08T18:04:36.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.380+0000 7fca89eec640 1 -- start start 2024-05-08T18:04:36.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.381+0000 7fca89eec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8412c530 0x7fca841273c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.381+0000 7fca89eec640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fca84131ed0 con 0x7fca8412c530 2024-05-08T18:04:36.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.381+0000 7fca837fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8412c530 0x7fca841273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.381+0000 7fca837fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8412c530 0x7fca841273c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40028/0 (socket says 172.21.15.46:40028) 2024-05-08T18:04:36.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.381+0000 7fca837fe640 1 -- 172.21.15.46:0/4127631431 learned_addr learned my addr 172.21.15.46:0/4127631431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:36.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.381+0000 7fca837fe640 1 -- 172.21.15.46:0/4127631431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca84070ea0 con 0x7fca8412c530 2024-05-08T18:04:36.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.381+0000 7fca837fe640 1 --2- 172.21.15.46:0/4127631431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8412c530 0x7fca841273c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fca7400c3d0 tx=0x7fca7400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.382+0000 7fca6bfff640 1 -- 172.21.15.46:0/4127631431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca740074d0 con 0x7fca8412c530 2024-05-08T18:04:36.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.382+0000 7fca6bfff640 1 -- 172.21.15.46:0/4127631431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca740047e0 con 0x7fca8412c530 2024-05-08T18:04:36.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.382+0000 7fca89eec640 1 -- 172.21.15.46:0/4127631431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca84127900 con 0x7fca8412c530 2024-05-08T18:04:36.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.382+0000 7fca6bfff640 1 -- 172.21.15.46:0/4127631431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca74007670 con 0x7fca8412c530 2024-05-08T18:04:36.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.382+0000 7fca89eec640 1 -- 172.21.15.46:0/4127631431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca84127dd0 con 0x7fca8412c530 2024-05-08T18:04:36.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.382+0000 7fca6bfff640 1 -- 172.21.15.46:0/4127631431 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 49355+0+0 (secure 0 0 0) 0x7fca74011070 con 0x7fca8412c530 2024-05-08T18:04:36.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.383+0000 7fca6bfff640 1 --2- 172.21.15.46:0/4127631431 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca7003d220 0x7fca7003f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.383+0000 7fca89eec640 1 -- 172.21.15.46:0/4127631431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca840640a0 con 0x7fca8412c530 2024-05-08T18:04:36.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.383+0000 7fca6bfff640 1 -- 172.21.15.46:0/4127631431 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 3482+0+0 (secure 0 0 0) 0x7fca74050de0 con 0x7fca8412c530 2024-05-08T18:04:36.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.383+0000 7fca82ffd640 1 --2- 172.21.15.46:0/4127631431 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca7003d220 0x7fca7003f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.384+0000 7fca82ffd640 1 --2- 172.21.15.46:0/4127631431 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca7003d220 0x7fca7003f6e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fca78036000 tx=0x7fca78008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.388+0000 7fca6bfff640 1 -- 172.21.15.46:0/4127631431 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fca74011360 con 0x7fca8412c530 2024-05-08T18:04:36.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.549+0000 7fca6bfff640 1 -- 172.21.15.46:0/4127631431 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fca740133c0 con 0x7fca8412c530 2024-05-08T18:04:36.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.579+0000 7fca89eec640 1 -- 172.21.15.46:0/4127631431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca84131d00 con 0x7fca8412c530 2024-05-08T18:04:36.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.580+0000 7fca6bfff640 1 -- 172.21.15.46:0/4127631431 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+9707 (secure 0 0 0) 0x7fca7401f0c0 con 0x7fca8412c530 2024-05-08T18:04:36.578 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:36.578 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":12,"fsid":"d8088d1f-616e-46b1-995a-85e572267d38","created":"2024-05-08T18:04:25.152571+0000","modified":"2024-05-08T18:04:36.545197+0000","last_up_change":"2024-05-08T18:04:33.542468+0000","last_in_change":"2024-05-08T18:04:26.221033+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T18:04:33.679407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T18:04:34.053953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7304d61-4d9e-455b-a968-5641c682e642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":337143975},{"type":"v1","addr":"172.21.15.46:6801","nonce":337143975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":337143975},{"type":"v1","addr":"172.21.15.46:6803","nonce":337143975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":337143975},{"type":"v1","addr":"172.21.15.46:6807","nonce":337143975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":337143975},{"type":"v1","addr":"172.21.15.46:6805","nonce":337143975}]},"public_addr":"172.21.15.46:6801/337143975","cluster_addr":"172.21.15.46:6803/337143975","heartbeat_back_addr":"172.21.15.46:6807/337143975","heartbeat_front_addr":"172.21.15.46:6805/337143975","state":["exists","up"]},{"osd":1,"uuid":"9a62c75d-b151-4453-a538-14408e8f4b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2141923372},{"type":"v1","addr":"172.21.15.50:6809","nonce":2141923372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2141923372},{"type":"v1","addr":"172.21.15.50:6812","nonce":2141923372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2141923372},{"type":"v1","addr":"172.21.15.50:6820","nonce":2141923372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2141923372},{"type":"v1","addr":"172.21.15.50:6816","nonce":2141923372}]},"public_addr":"172.21.15.50:6809/2141923372","cluster_addr":"172.21.15.50:6812/2141923372","heartbeat_back_addr":"172.21.15.50:6820/2141923372","heartbeat_front_addr":"172.21.15.50:6816/2141923372","state":["exists","up"]},{"osd":2,"uuid":"c2d4e8c1-79bb-4061-ade1-f0fc079c52f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":3039646617},{"type":"v1","addr":"172.21.15.50:6813","nonce":3039646617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6815","nonce":3039646617},{"type":"v1","addr":"172.21.15.50:6817","nonce":3039646617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3039646617},{"type":"v1","addr":"172.21.15.50:6823","nonce":3039646617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":3039646617},{"type":"v1","addr":"172.21.15.50:6821","nonce":3039646617}]},"public_addr":"172.21.15.50:6813/3039646617","cluster_addr":"172.21.15.50:6817/3039646617","heartbeat_back_addr":"172.21.15.50:6823/3039646617","heartbeat_front_addr":"172.21.15.50:6821/3039646617","state":["exists","up"]},{"osd":3,"uuid":"e5e292cd-9ed4-44c9-9e91-7ee3736272ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3130492744},{"type":"v1","addr":"172.21.15.50:6801","nonce":3130492744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3130492744},{"type":"v1","addr":"172.21.15.50:6803","nonce":3130492744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3130492744},{"type":"v1","addr":"172.21.15.50:6807","nonce":3130492744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3130492744},{"type":"v1","addr":"172.21.15.50:6805","nonce":3130492744}]},"public_addr":"172.21.15.50:6801/3130492744","cluster_addr":"172.21.15.50:6803/3130492744","heartbeat_back_addr":"172.21.15.50:6807/3130492744","heartbeat_front_addr":"172.21.15.50:6805/3130492744","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T18:04:31.282728+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T18:04:31.387361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T18:04:31.386518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T18:04:31.319396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T18:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.582+0000 7fca89eec640 1 -- 172.21.15.46:0/4127631431 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca7003d220 msgr2=0x7fca7003f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.582+0000 7fca89eec640 1 --2- 172.21.15.46:0/4127631431 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca7003d220 0x7fca7003f6e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fca78036000 tx=0x7fca78008030 comp rx=0 tx=0).stop 2024-05-08T18:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.582+0000 7fca89eec640 1 -- 172.21.15.46:0/4127631431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8412c530 msgr2=0x7fca841273c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.582+0000 7fca89eec640 1 --2- 172.21.15.46:0/4127631431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8412c530 0x7fca841273c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fca7400c3d0 tx=0x7fca7400c890 comp rx=0 tx=0).stop 2024-05-08T18:04:36.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.582+0000 7fca89eec640 1 -- 172.21.15.46:0/4127631431 shutdown_connections 2024-05-08T18:04:36.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.582+0000 7fca89eec640 1 --2- 172.21.15.46:0/4127631431 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca7003d220 0x7fca7003f6e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.582+0000 7fca89eec640 1 --2- 172.21.15.46:0/4127631431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca8412c530 0x7fca841273c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.582+0000 7fca89eec640 1 -- 172.21.15.46:0/4127631431 >> 172.21.15.46:0/4127631431 conn(0x7fca8409d7a0 msgr2=0x7fca8406ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:36.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.582+0000 7fca89eec640 1 -- 172.21.15.46:0/4127631431 shutdown_connections 2024-05-08T18:04:36.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.582+0000 7fca89eec640 1 -- 172.21.15.46:0/4127631431 wait complete. 2024-05-08T18:04:36.591 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T18:04:36.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T18:04:36.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T18:04:36.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T18:04:36.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.702+0000 7f9333e33640 1 Processor -- start 2024-05-08T18:04:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.702+0000 7f15200d4640 1 Processor -- start 2024-05-08T18:04:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.702+0000 7fa3b47e8640 1 Processor -- start 2024-05-08T18:04:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.702+0000 7fa3b47e8640 1 -- start start 2024-05-08T18:04:36.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.702+0000 7f4adadc7640 1 Processor -- start 2024-05-08T18:04:36.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.702+0000 7f4adadc7640 1 -- start start 2024-05-08T18:04:36.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7f9333e33640 1 -- start start 2024-05-08T18:04:36.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7f15200d4640 1 -- start start 2024-05-08T18:04:36.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7fa3b47e8640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa3ac12f420 0x7fa3ac131810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7f4adadc7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4ad412f420 0x7f4ad4131810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7fa3b47e8640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3ac12c230 con 0x7fa3ac12c530 2024-05-08T18:04:36.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7fa3b47e8640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa3ac09ad20 con 0x7fa3ac12f420 2024-05-08T18:04:36.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7f4adadc7640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ad412c230 con 0x7f4ad412c530 2024-05-08T18:04:36.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7f4adadc7640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f4ad409ad20 con 0x7f4ad412f420 2024-05-08T18:04:36.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7f4ad8b63640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4ad412c530 0x7f4ad412c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40038/0 (socket says 172.21.15.46:40038) 2024-05-08T18:04:36.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7f4ad8b63640 1 -- 172.21.15.46:0/3109422767 learned_addr learned my addr 172.21.15.46:0/3109422767 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:36.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7fa3b2584640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa3ac12c530 0x7fa3ac12c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40036/0 (socket says 172.21.15.46:40036) 2024-05-08T18:04:36.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7fa3b2584640 1 -- 172.21.15.46:0/926214849 learned_addr learned my addr 172.21.15.46:0/926214849 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:36.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7f15200d4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1518072540 0x7f15181348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7f9333e33640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f932c12c530 0x7f932c12c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7f9333e33640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f932c12c230 con 0x7f932c071d40 2024-05-08T18:04:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7f9333e33640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f932c09ad20 con 0x7f932c12c530 2024-05-08T18:04:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7f15200d4640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1518073480 con 0x7f1518070650 2024-05-08T18:04:36.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7f15200d4640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1518071a80 con 0x7f1518072540 2024-05-08T18:04:36.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7f4ad3fff640 1 --2- 172.21.15.46:0/3109422767 >> v2:172.21.15.46:3300/0 conn(0x7f4ad412f420 0x7f4ad4131810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.703+0000 7fa3b1d83640 1 --2- 172.21.15.46:0/926214849 >> v2:172.21.15.46:3300/0 conn(0x7fa3ac12f420 0x7fa3ac131810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.704+0000 7f9331bcf640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f932c12c530 0x7f932c12c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.704+0000 7f93313ce640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f932c071d40 0x7f932c072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40068/0 (socket says 172.21.15.46:40068) 2024-05-08T18:04:36.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.704+0000 7f93313ce640 1 -- 172.21.15.46:0/1280895935 learned_addr learned my addr 172.21.15.46:0/1280895935 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:36.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.704+0000 7f151d66f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1518072540 0x7f15181348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.704+0000 7f151d66f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1518072540 0x7f15181348a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40068/0 (socket says 172.21.15.46:40068) 2024-05-08T18:04:36.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.704+0000 7f151d66f640 1 -- 172.21.15.46:0/648344648 learned_addr learned my addr 172.21.15.46:0/648344648 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:36.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.704+0000 7f4ad37fe640 1 -- 172.21.15.46:0/3109422767 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1782334755 0 0) 0x7f4ad412c230 con 0x7f4ad412c530 2024-05-08T18:04:36.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.704+0000 7f4ad37fe640 1 -- 172.21.15.46:0/3109422767 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ac4003520 con 0x7f4ad412c530 2024-05-08T18:04:36.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.704+0000 7fa3b1d83640 1 -- 172.21.15.46:0/926214849 >> v1:172.21.15.46:6789/0 conn(0x7fa3ac12c530 legacy=0x7fa3ac12c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.704+0000 7fa3b1d83640 1 -- 172.21.15.46:0/926214849 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac073510 con 0x7fa3ac12f420 2024-05-08T18:04:36.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.704+0000 7f4ad3fff640 1 -- 172.21.15.46:0/3109422767 >> v1:172.21.15.46:6789/0 conn(0x7f4ad412c530 legacy=0x7f4ad412c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.704+0000 7f4ad3fff640 1 -- 172.21.15.46:0/3109422767 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad4073510 con 0x7f4ad412f420 2024-05-08T18:04:36.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.704+0000 7f151d66f640 1 -- 172.21.15.46:0/648344648 >> v1:172.21.15.46:6789/0 conn(0x7f1518070650 legacy=0x7f1518070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.704+0000 7f151d66f640 1 -- 172.21.15.46:0/648344648 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1518071140 con 0x7f1518072540 2024-05-08T18:04:36.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.704+0000 7f9331bcf640 1 -- 172.21.15.46:0/1280895935 >> v1:172.21.15.46:6789/0 conn(0x7f932c071d40 legacy=0x7f932c072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.704+0000 7f9331bcf640 1 -- 172.21.15.46:0/1280895935 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c073510 con 0x7f932c12c530 2024-05-08T18:04:36.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.705+0000 7fa3b1d83640 1 --2- 172.21.15.46:0/926214849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac12f420 0x7fa3ac131810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009720 tx=0x7fa39c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4ba367427c45a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.705+0000 7fa3b1582640 1 -- 172.21.15.46:0/926214849 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa39c0316d0 con 0x7fa3ac12f420 2024-05-08T18:04:36.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.705+0000 7f9331bcf640 1 --2- 172.21.15.46:0/1280895935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c12c530 0x7f932c12c910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f932800b9c0 tx=0x7f932802e010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fc8b476248f10b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.705+0000 7fa3b1582640 1 -- 172.21.15.46:0/926214849 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa39c037070 con 0x7fa3ac12f420 2024-05-08T18:04:36.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.705+0000 7fa3b1582640 1 -- 172.21.15.46:0/926214849 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa39c0319c0 con 0x7fa3ac12f420 2024-05-08T18:04:36.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.705+0000 7f4ad3fff640 1 --2- 172.21.15.46:0/3109422767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ad412f420 0x7f4ad4131810 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009720 tx=0x7f4ac002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a39f103bb161a4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.705+0000 7f9330bcd640 1 -- 172.21.15.46:0/1280895935 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f932803b070 con 0x7f932c12c530 2024-05-08T18:04:36.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.705+0000 7f9330bcd640 1 -- 172.21.15.46:0/1280895935 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f932802ecf0 con 0x7f932c12c530 2024-05-08T18:04:36.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.705+0000 7f9330bcd640 1 -- 172.21.15.46:0/1280895935 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9328036440 con 0x7f932c12c530 2024-05-08T18:04:36.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.705+0000 7f4ad37fe640 1 -- 172.21.15.46:0/3109422767 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ac00316d0 con 0x7f4ad412f420 2024-05-08T18:04:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.705+0000 7f151d66f640 1 --2- 172.21.15.46:0/648344648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518072540 0x7f15181348a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f15080097f0 tx=0x7f150802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1bd18346414a6f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7f4ad37fe640 1 -- 172.21.15.46:0/3109422767 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac0037070 con 0x7f4ad412f420 2024-05-08T18:04:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7f4ad37fe640 1 -- 172.21.15.46:0/3109422767 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ac00319a0 con 0x7f4ad412f420 2024-05-08T18:04:36.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7f151ce6e640 1 -- 172.21.15.46:0/648344648 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1508035020 con 0x7f1518072540 2024-05-08T18:04:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7fa3b47e8640 1 -- 172.21.15.46:0/926214849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac12f420 msgr2=0x7fa3ac131810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7fa3b47e8640 1 --2- 172.21.15.46:0/926214849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac12f420 0x7fa3ac131810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009720 tx=0x7fa39c02b840 comp rx=0 tx=0).stop 2024-05-08T18:04:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7f151ce6e640 1 -- 172.21.15.46:0/648344648 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15080317a0 con 0x7f1518072540 2024-05-08T18:04:36.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7fa3b47e8640 1 -- 172.21.15.46:0/926214849 shutdown_connections 2024-05-08T18:04:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7f151ce6e640 1 -- 172.21.15.46:0/648344648 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1508031aa0 con 0x7f1518072540 2024-05-08T18:04:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7fa3b47e8640 1 --2- 172.21.15.46:0/926214849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac12f420 0x7fa3ac131810 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7fa3b47e8640 1 -- 172.21.15.46:0/926214849 >> 172.21.15.46:0/926214849 conn(0x7fa3ac09d7a0 msgr2=0x7fa3ac09dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:36.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7fa3b47e8640 1 -- 172.21.15.46:0/926214849 shutdown_connections 2024-05-08T18:04:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7f9333e33640 1 -- 172.21.15.46:0/1280895935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c12c530 msgr2=0x7f932c12c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7f9333e33640 1 --2- 172.21.15.46:0/1280895935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c12c530 0x7f932c12c910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f932800b9c0 tx=0x7f932802e010 comp rx=0 tx=0).stop 2024-05-08T18:04:36.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7fa3b47e8640 1 -- 172.21.15.46:0/926214849 wait complete. 2024-05-08T18:04:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7f9333e33640 1 -- 172.21.15.46:0/1280895935 shutdown_connections 2024-05-08T18:04:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7f9333e33640 1 --2- 172.21.15.46:0/1280895935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c12c530 0x7f932c12c910 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.706+0000 7f9333e33640 1 -- 172.21.15.46:0/1280895935 >> 172.21.15.46:0/1280895935 conn(0x7f932c09d7a0 msgr2=0x7f932c09dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7f4adadc7640 1 -- 172.21.15.46:0/3109422767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ad412f420 msgr2=0x7f4ad4131810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7f9333e33640 1 -- 172.21.15.46:0/1280895935 shutdown_connections 2024-05-08T18:04:36.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7f9333e33640 1 -- 172.21.15.46:0/1280895935 wait complete. 2024-05-08T18:04:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7f4adadc7640 1 --2- 172.21.15.46:0/3109422767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ad412f420 0x7f4ad4131810 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009720 tx=0x7f4ac002b840 comp rx=0 tx=0).stop 2024-05-08T18:04:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7f15200d4640 1 -- 172.21.15.46:0/648344648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518072540 msgr2=0x7f15181348a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7f15200d4640 1 --2- 172.21.15.46:0/648344648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518072540 0x7f15181348a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f15080097f0 tx=0x7f150802b910 comp rx=0 tx=0).stop 2024-05-08T18:04:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7f4adadc7640 1 -- 172.21.15.46:0/3109422767 shutdown_connections 2024-05-08T18:04:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7f4adadc7640 1 --2- 172.21.15.46:0/3109422767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ad412f420 0x7f4ad4131810 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7f15200d4640 1 -- 172.21.15.46:0/648344648 shutdown_connections 2024-05-08T18:04:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7f15200d4640 1 --2- 172.21.15.46:0/648344648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518072540 0x7f15181348a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7f15200d4640 1 -- 172.21.15.46:0/648344648 >> 172.21.15.46:0/648344648 conn(0x7f15180a1e60 msgr2=0x7f15180a2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7f4adadc7640 1 -- 172.21.15.46:0/3109422767 >> 172.21.15.46:0/3109422767 conn(0x7f4ad409d7a0 msgr2=0x7f4ad409dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7f4adadc7640 1 -- 172.21.15.46:0/3109422767 shutdown_connections 2024-05-08T18:04:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7f4adadc7640 1 -- 172.21.15.46:0/3109422767 wait complete. 2024-05-08T18:04:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7f15200d4640 1 -- 172.21.15.46:0/648344648 shutdown_connections 2024-05-08T18:04:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7f15200d4640 1 -- 172.21.15.46:0/648344648 wait complete. 2024-05-08T18:04:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.707+0000 7fa3b47e8640 1 Processor -- start 2024-05-08T18:04:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.708+0000 7f9333e33640 1 Processor -- start 2024-05-08T18:04:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.708+0000 7fa3b47e8640 1 -- start start 2024-05-08T18:04:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.708+0000 7f9333e33640 1 -- start start 2024-05-08T18:04:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.708+0000 7f4adadc7640 1 Processor -- start 2024-05-08T18:04:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.708+0000 7f9333e33640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c071d40 0x7f932c1c0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.708+0000 7f9333e33640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f932c12cfd0 con 0x7f932c071d40 2024-05-08T18:04:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.708+0000 7f4adadc7640 1 -- start start 2024-05-08T18:04:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.708+0000 7f9331bcf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c071d40 0x7f932c1c0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.708+0000 7f9331bcf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c071d40 0x7f932c1c0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40072/0 (socket says 172.21.15.46:40072) 2024-05-08T18:04:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.708+0000 7f9331bcf640 1 -- 172.21.15.46:0/279927519 learned_addr learned my addr 172.21.15.46:0/279927519 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.708+0000 7f15200d4640 1 Processor -- start 2024-05-08T18:04:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.708+0000 7f15200d4640 1 -- start start 2024-05-08T18:04:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.708+0000 7fa3b47e8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac12c530 0x7fa3ac1214b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7fa3b47e8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ac131ed0 con 0x7fa3ac12c530 2024-05-08T18:04:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7f4adadc7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ad412c530 0x7f4ad4126800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7f4adadc7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad4131ed0 con 0x7f4ad412c530 2024-05-08T18:04:36.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7f9331bcf640 1 -- 172.21.15.46:0/279927519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f932c117630 con 0x7f932c071d40 2024-05-08T18:04:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7fa3b2584640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac12c530 0x7fa3ac1214b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7f4ad8b63640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ad412c530 0x7f4ad4126800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7f4ad8b63640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ad412c530 0x7f4ad4126800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40086/0 (socket says 172.21.15.46:40086) 2024-05-08T18:04:36.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7f15200d4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518070650 0x7f151812ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7fa3b2584640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac12c530 0x7fa3ac1214b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40080/0 (socket says 172.21.15.46:40080) 2024-05-08T18:04:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7fa3b2584640 1 -- 172.21.15.46:0/4057151532 learned_addr learned my addr 172.21.15.46:0/4057151532 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7f4ad8b63640 1 -- 172.21.15.46:0/2703585797 learned_addr learned my addr 172.21.15.46:0/2703585797 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7f15200d4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1518134f60 con 0x7f1518070650 2024-05-08T18:04:36.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7f9331bcf640 1 --2- 172.21.15.46:0/279927519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c071d40 0x7f932c1c0220 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f932800bae0 tx=0x7f93280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7f151de70640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518070650 0x7f151812ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7f4ad8b63640 1 -- 172.21.15.46:0/2703585797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad4127ba0 con 0x7f4ad412c530 2024-05-08T18:04:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7f151de70640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518070650 0x7f151812ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40090/0 (socket says 172.21.15.46:40090) 2024-05-08T18:04:36.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7f151de70640 1 -- 172.21.15.46:0/143861864 learned_addr learned my addr 172.21.15.46:0/143861864 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:36.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7f93227fc640 1 -- 172.21.15.46:0/279927519 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9328046070 con 0x7f932c071d40 2024-05-08T18:04:36.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.709+0000 7fa3b2584640 1 -- 172.21.15.46:0/4057151532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ac113d90 con 0x7fa3ac12c530 2024-05-08T18:04:36.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7f9333e33640 1 -- 172.21.15.46:0/279927519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f932c117190 con 0x7f932c071d40 2024-05-08T18:04:36.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7f4ad8b63640 1 --2- 172.21.15.46:0/2703585797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ad412c530 0x7f4ad4126800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800c480 tx=0x7f4ac800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7f93227fc640 1 -- 172.21.15.46:0/279927519 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9328041560 con 0x7f932c071d40 2024-05-08T18:04:36.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7fa3b2584640 1 --2- 172.21.15.46:0/4057151532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac12c530 0x7fa3ac1214b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8001ea0 tx=0x7fa3a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7f9333e33640 1 -- 172.21.15.46:0/279927519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f932c1c0760 con 0x7f932c071d40 2024-05-08T18:04:36.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7f4ad17fa640 1 -- 172.21.15.46:0/2703585797 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ac8017070 con 0x7f4ad412c530 2024-05-08T18:04:36.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7f4ad17fa640 1 -- 172.21.15.46:0/2703585797 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ac8005cd0 con 0x7f4ad412c530 2024-05-08T18:04:36.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7f151de70640 1 -- 172.21.15.46:0/143861864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1518134b40 con 0x7f1518070650 2024-05-08T18:04:36.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7fa3a2ffd640 1 -- 172.21.15.46:0/4057151532 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3a8005c10 con 0x7fa3ac12c530 2024-05-08T18:04:36.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7fa3b47e8640 1 -- 172.21.15.46:0/4057151532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ac131820 con 0x7fa3ac12c530 2024-05-08T18:04:36.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7f151de70640 1 --2- 172.21.15.46:0/143861864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518070650 0x7f151812ba70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f151400c480 tx=0x7f151400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7fa3b47e8640 1 -- 172.21.15.46:0/4057151532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ac129840 con 0x7fa3ac12c530 2024-05-08T18:04:36.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7fa3a2ffd640 1 -- 172.21.15.46:0/4057151532 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3a8016070 con 0x7fa3ac12c530 2024-05-08T18:04:36.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7fa3a2ffd640 1 -- 172.21.15.46:0/4057151532 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3a80070d0 con 0x7fa3ac12c530 2024-05-08T18:04:36.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7f4adadc7640 1 -- 172.21.15.46:0/2703585797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad41260d0 con 0x7f4ad412c530 2024-05-08T18:04:36.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7f93227fc640 1 -- 172.21.15.46:0/279927519 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f932803b040 con 0x7f932c071d40 2024-05-08T18:04:36.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.711+0000 7f150e7fc640 1 -- 172.21.15.46:0/143861864 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1514017070 con 0x7f1518070650 2024-05-08T18:04:36.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.711+0000 7f9333e33640 1 -- 172.21.15.46:0/279927519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f932c0640a0 con 0x7f932c071d40 2024-05-08T18:04:36.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.711+0000 7fa3b47e8640 1 -- 172.21.15.46:0/4057151532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa394000f80 con 0x7fa3ac12c530 2024-05-08T18:04:36.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7f4ad17fa640 1 -- 172.21.15.46:0/2703585797 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ac8007200 con 0x7f4ad412c530 2024-05-08T18:04:36.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.710+0000 7f4adadc7640 1 -- 172.21.15.46:0/2703585797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad4128fa0 con 0x7f4ad412c530 2024-05-08T18:04:36.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.711+0000 7f150e7fc640 1 -- 172.21.15.46:0/143861864 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1514005200 con 0x7f1518070650 2024-05-08T18:04:36.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.711+0000 7f15200d4640 1 -- 172.21.15.46:0/143861864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151811dc60 con 0x7f1518070650 2024-05-08T18:04:36.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.711+0000 7f15200d4640 1 -- 172.21.15.46:0/143861864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151811e8d0 con 0x7f1518070650 2024-05-08T18:04:36.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7f93227fc640 1 -- 172.21.15.46:0/279927519 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9328049070 con 0x7f932c071d40 2024-05-08T18:04:36.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.711+0000 7f4adadc7640 1 -- 172.21.15.46:0/2703585797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ad40640a0 con 0x7f4ad412c530 2024-05-08T18:04:36.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7f4ad17fa640 1 -- 172.21.15.46:0/2703585797 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f4ac801a070 con 0x7f4ad412c530 2024-05-08T18:04:36.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7f150e7fc640 1 -- 172.21.15.46:0/143861864 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15140035f0 con 0x7f1518070650 2024-05-08T18:04:36.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7f4ad17fa640 1 --2- 172.21.15.46:0/2703585797 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4ab803d300 0x7f4ab803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7f4ad17fa640 1 -- 172.21.15.46:0/2703585797 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f4ac80505c0 con 0x7f4ad412c530 2024-05-08T18:04:36.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7f93227fc640 1 --2- 172.21.15.46:0/279927519 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f931003d300 0x7f931003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7f93227fc640 1 -- 172.21.15.46:0/279927519 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f93280731e0 con 0x7f932c071d40 2024-05-08T18:04:36.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7f15200d4640 1 -- 172.21.15.46:0/143861864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1500000f80 con 0x7f1518070650 2024-05-08T18:04:36.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7fa3a2ffd640 1 -- 172.21.15.46:0/4057151532 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa3a8007270 con 0x7fa3ac12c530 2024-05-08T18:04:36.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7f93227fc640 1 --2- 172.21.15.46:0/279927519 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9310042a80 0x7f9310044ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7f4ad17fa640 1 --2- 172.21.15.46:0/2703585797 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x7f4ab8042a80 0x7f4ab8044ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7f4ad17fa640 1 -- 172.21.15.46:0/2703585797 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ac8020e50 con 0x7f4ab8042a80 2024-05-08T18:04:36.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7f4ad3fff640 1 --2- 172.21.15.46:0/2703585797 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4ab803d300 0x7f4ab803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7f93227fc640 1 -- 172.21.15.46:0/279927519 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f932802ecf0 con 0x7f9310042a80 2024-05-08T18:04:36.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7f93313ce640 1 --2- 172.21.15.46:0/279927519 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f931003d300 0x7f931003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7fa3a2ffd640 1 --2- 172.21.15.46:0/4057151532 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa39003d2b0 0x7fa39003f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7f4ad9364640 1 --2- 172.21.15.46:0/2703585797 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x7f4ab8042a80 0x7f4ab8044ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.712+0000 7f93227fc640 1 -- 172.21.15.46:0/279927519 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=12) ==== 24+0+0 (secure 0 0 0) 0x7f932807a050 con 0x7f932c071d40 2024-05-08T18:04:36.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7f4ad17fa640 1 -- 172.21.15.46:0/2703585797 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=12) ==== 24+0+0 (secure 0 0 0) 0x7f4ac8057050 con 0x7f4ad412c530 2024-05-08T18:04:36.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7fa3a2ffd640 1 -- 172.21.15.46:0/4057151532 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7fa3a80537c0 con 0x7fa3ac12c530 2024-05-08T18:04:36.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7f93323d0640 1 --2- 172.21.15.46:0/279927519 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9310042a80 0x7f9310044ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7fa3a2ffd640 1 --2- 172.21.15.46:0/4057151532 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x7fa390042a30 0x7fa390044e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7fa3a2ffd640 1 -- 172.21.15.46:0/4057151532 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3a8012ec0 con 0x7fa390042a30 2024-05-08T18:04:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7fa3a2ffd640 1 -- 172.21.15.46:0/4057151532 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=12) ==== 24+0+0 (secure 0 0 0) 0x7fa3a8053c00 con 0x7fa3ac12c530 2024-05-08T18:04:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7fa3b1d83640 1 --2- 172.21.15.46:0/4057151532 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa39003d2b0 0x7fa39003f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7fa3b2d85640 1 --2- 172.21.15.46:0/4057151532 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x7fa390042a30 0x7fa390044e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7f150e7fc640 1 -- 172.21.15.46:0/143861864 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f151401a070 con 0x7f1518070650 2024-05-08T18:04:36.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7f93313ce640 1 --2- 172.21.15.46:0/279927519 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f931003d300 0x7f931003f7c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f931c007fd0 tx=0x7f931c007aa0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7f4ad3fff640 1 --2- 172.21.15.46:0/2703585797 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4ab803d300 0x7f4ab803f7c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0036000 tx=0x7f4ac0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7f4ad9364640 1 --2- 172.21.15.46:0/2703585797 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x7f4ab8042a80 0x7f4ab8044ec0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7f150e7fc640 1 --2- 172.21.15.46:0/143861864 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f14fc03d300 0x7f14fc03f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7f93323d0640 1 --2- 172.21.15.46:0/279927519 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9310042a80 0x7f9310044ec0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7f150e7fc640 1 -- 172.21.15.46:0/143861864 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f1514050d70 con 0x7f1518070650 2024-05-08T18:04:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7fa3b2d85640 1 --2- 172.21.15.46:0/4057151532 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x7fa390042a30 0x7fa390044e70 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7f150e7fc640 1 --2- 172.21.15.46:0/143861864 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x7f14fc042a80 0x7f14fc044ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7f150e7fc640 1 -- 172.21.15.46:0/143861864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1514014ec0 con 0x7f14fc042a80 2024-05-08T18:04:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7f150e7fc640 1 -- 172.21.15.46:0/143861864 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=12) ==== 24+0+0 (secure 0 0 0) 0x7f1514051130 con 0x7f1518070650 2024-05-08T18:04:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.713+0000 7fa3b1d83640 1 --2- 172.21.15.46:0/4057151532 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa39003d2b0 0x7fa39003f770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa39c036000 tx=0x7fa39c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.714+0000 7f151d66f640 1 --2- 172.21.15.46:0/143861864 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f14fc03d300 0x7f14fc03f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.714+0000 7f151e671640 1 --2- 172.21.15.46:0/143861864 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x7f14fc042a80 0x7f14fc044ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.714+0000 7f4ad17fa640 1 -- 172.21.15.46:0/2703585797 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ac8020e50 con 0x7f4ab8042a80 2024-05-08T18:04:36.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.714+0000 7f151d66f640 1 --2- 172.21.15.46:0/143861864 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f14fc03d300 0x7f14fc03f7c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1508036000 tx=0x7f1508008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.714+0000 7f151e671640 1 --2- 172.21.15.46:0/143861864 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x7f14fc042a80 0x7f14fc044ec0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.714+0000 7f93227fc640 1 -- 172.21.15.46:0/279927519 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f932802ecf0 con 0x7f9310042a80 2024-05-08T18:04:36.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.714+0000 7fa3a2ffd640 1 -- 172.21.15.46:0/4057151532 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa3a8012ec0 con 0x7fa390042a30 2024-05-08T18:04:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.715+0000 7f150e7fc640 1 -- 172.21.15.46:0/143861864 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1514014ec0 con 0x7f14fc042a80 2024-05-08T18:04:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.729+0000 7fa3b47e8640 1 -- 172.21.15.46:0/4057151532 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa394002c70 con 0x7fa390042a30 2024-05-08T18:04:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.729+0000 7f9333e33640 1 -- 172.21.15.46:0/279927519 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f932c1184a0 con 0x7f9310042a80 2024-05-08T18:04:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.729+0000 7fa3a2ffd640 1 -- 172.21.15.46:0/4057151532 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa394002c70 con 0x7fa390042a30 2024-05-08T18:04:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.729+0000 7fa3b47e8640 1 -- 172.21.15.46:0/4057151532 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x7fa390042a30 msgr2=0x7fa390044e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.729+0000 7fa3b47e8640 1 --2- 172.21.15.46:0/4057151532 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x7fa390042a30 0x7fa390044e70 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7fa3b47e8640 1 -- 172.21.15.46:0/4057151532 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa39003d2b0 msgr2=0x7fa39003f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7fa3b47e8640 1 --2- 172.21.15.46:0/4057151532 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa39003d2b0 0x7fa39003f770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa39c036000 tx=0x7fa39c008030 comp rx=0 tx=0).stop 2024-05-08T18:04:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7fa3b47e8640 1 -- 172.21.15.46:0/4057151532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac12c530 msgr2=0x7fa3ac1214b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7fa3b47e8640 1 --2- 172.21.15.46:0/4057151532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac12c530 0x7fa3ac1214b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8001ea0 tx=0x7fa3a800c910 comp rx=0 tx=0).stop 2024-05-08T18:04:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7f93227fc640 1 -- 172.21.15.46:0/279927519 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f932c1184a0 con 0x7f9310042a80 2024-05-08T18:04:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7fa3b47e8640 1 -- 172.21.15.46:0/4057151532 shutdown_connections 2024-05-08T18:04:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7fa3b47e8640 1 --2- 172.21.15.46:0/4057151532 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x7fa390042a30 0x7fa390044e70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7fa3b47e8640 1 --2- 172.21.15.46:0/4057151532 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa39003d2b0 0x7fa39003f770 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7fa3b47e8640 1 --2- 172.21.15.46:0/4057151532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ac12c530 0x7fa3ac1214b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7fa3b47e8640 1 -- 172.21.15.46:0/4057151532 >> 172.21.15.46:0/4057151532 conn(0x7fa3ac09d7a0 msgr2=0x7fa3ac12a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7fa3b47e8640 1 -- 172.21.15.46:0/4057151532 shutdown_connections 2024-05-08T18:04:36.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7fa3b47e8640 1 -- 172.21.15.46:0/4057151532 wait complete. 2024-05-08T18:04:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7f9333e33640 1 -- 172.21.15.46:0/279927519 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9310042a80 msgr2=0x7f9310044ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7f9333e33640 1 --2- 172.21.15.46:0/279927519 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9310042a80 0x7f9310044ec0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7f9333e33640 1 -- 172.21.15.46:0/279927519 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f931003d300 msgr2=0x7f931003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7f9333e33640 1 --2- 172.21.15.46:0/279927519 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f931003d300 0x7f931003f7c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f931c007fd0 tx=0x7f931c007aa0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7f9333e33640 1 -- 172.21.15.46:0/279927519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c071d40 msgr2=0x7f932c1c0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7f9333e33640 1 --2- 172.21.15.46:0/279927519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c071d40 0x7f932c1c0220 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f932800bae0 tx=0x7f93280046c0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7f9333e33640 1 -- 172.21.15.46:0/279927519 shutdown_connections 2024-05-08T18:04:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7f9333e33640 1 --2- 172.21.15.46:0/279927519 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9310042a80 0x7f9310044ec0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7f9333e33640 1 --2- 172.21.15.46:0/279927519 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f931003d300 0x7f931003f7c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7f9333e33640 1 --2- 172.21.15.46:0/279927519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f932c071d40 0x7f932c1c0220 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7f9333e33640 1 -- 172.21.15.46:0/279927519 >> 172.21.15.46:0/279927519 conn(0x7f932c09d7a0 msgr2=0x7f932c12a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7f9333e33640 1 -- 172.21.15.46:0/279927519 shutdown_connections 2024-05-08T18:04:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.730+0000 7f9333e33640 1 -- 172.21.15.46:0/279927519 wait complete. 2024-05-08T18:04:36.750 INFO:teuthology.orchestra.run.smithi046.stdout:38654705667 2024-05-08T18:04:36.750 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T18:04:36.751 INFO:teuthology.orchestra.run.smithi046.stdout:38654705667 2024-05-08T18:04:36.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T18:04:36.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.747+0000 7f4adadc7640 1 -- 172.21.15.46:0/2703585797 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ad4128350 con 0x7f4ab8042a80 2024-05-08T18:04:36.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.748+0000 7f4ad17fa640 1 -- 172.21.15.46:0/2703585797 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4ad4128350 con 0x7f4ab8042a80 2024-05-08T18:04:36.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.748+0000 7f4adadc7640 1 -- 172.21.15.46:0/2703585797 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x7f4ab8042a80 msgr2=0x7f4ab8044ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.748+0000 7f4adadc7640 1 --2- 172.21.15.46:0/2703585797 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x7f4ab8042a80 0x7f4ab8044ec0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.748+0000 7f4adadc7640 1 -- 172.21.15.46:0/2703585797 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4ab803d300 msgr2=0x7f4ab803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.748+0000 7f4adadc7640 1 --2- 172.21.15.46:0/2703585797 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4ab803d300 0x7f4ab803f7c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0036000 tx=0x7f4ac0008030 comp rx=0 tx=0).stop 2024-05-08T18:04:36.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.748+0000 7f4adadc7640 1 -- 172.21.15.46:0/2703585797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ad412c530 msgr2=0x7f4ad4126800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.748+0000 7f4adadc7640 1 --2- 172.21.15.46:0/2703585797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ad412c530 0x7f4ad4126800 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800c480 tx=0x7f4ac800c940 comp rx=0 tx=0).stop 2024-05-08T18:04:36.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.748+0000 7f4adadc7640 1 -- 172.21.15.46:0/2703585797 shutdown_connections 2024-05-08T18:04:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.748+0000 7f4adadc7640 1 --2- 172.21.15.46:0/2703585797 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x7f4ab8042a80 0x7f4ab8044ec0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.748+0000 7f4adadc7640 1 --2- 172.21.15.46:0/2703585797 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4ab803d300 0x7f4ab803f7c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.748+0000 7f4adadc7640 1 --2- 172.21.15.46:0/2703585797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ad412c530 0x7f4ad4126800 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.748+0000 7f4adadc7640 1 -- 172.21.15.46:0/2703585797 >> 172.21.15.46:0/2703585797 conn(0x7f4ad409d7a0 msgr2=0x7f4ad412ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.748+0000 7f15200d4640 1 -- 172.21.15.46:0/143861864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1500002c70 con 0x7f14fc042a80 2024-05-08T18:04:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.748+0000 7f4adadc7640 1 -- 172.21.15.46:0/2703585797 shutdown_connections 2024-05-08T18:04:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.748+0000 7f4adadc7640 1 -- 172.21.15.46:0/2703585797 wait complete. 2024-05-08T18:04:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.749+0000 7f150e7fc640 1 -- 172.21.15.46:0/143861864 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1500002c70 con 0x7f14fc042a80 2024-05-08T18:04:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.749+0000 7f15200d4640 1 -- 172.21.15.46:0/143861864 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x7f14fc042a80 msgr2=0x7f14fc044ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.749+0000 7f15200d4640 1 --2- 172.21.15.46:0/143861864 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x7f14fc042a80 0x7f14fc044ec0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.749+0000 7f15200d4640 1 -- 172.21.15.46:0/143861864 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f14fc03d300 msgr2=0x7f14fc03f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.749+0000 7f15200d4640 1 --2- 172.21.15.46:0/143861864 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f14fc03d300 0x7f14fc03f7c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1508036000 tx=0x7f1508008030 comp rx=0 tx=0).stop 2024-05-08T18:04:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.749+0000 7f15200d4640 1 -- 172.21.15.46:0/143861864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518070650 msgr2=0x7f151812ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.749+0000 7f15200d4640 1 --2- 172.21.15.46:0/143861864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518070650 0x7f151812ba70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f151400c480 tx=0x7f151400c940 comp rx=0 tx=0).stop 2024-05-08T18:04:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.749+0000 7f15200d4640 1 -- 172.21.15.46:0/143861864 shutdown_connections 2024-05-08T18:04:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.749+0000 7f15200d4640 1 --2- 172.21.15.46:0/143861864 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x7f14fc042a80 0x7f14fc044ec0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.749+0000 7f15200d4640 1 --2- 172.21.15.46:0/143861864 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f14fc03d300 0x7f14fc03f7c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.749+0000 7f15200d4640 1 --2- 172.21.15.46:0/143861864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1518070650 0x7f151812ba70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.749+0000 7f15200d4640 1 -- 172.21.15.46:0/143861864 >> 172.21.15.46:0/143861864 conn(0x7f15180a1e60 msgr2=0x7f151812ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.749+0000 7f15200d4640 1 -- 172.21.15.46:0/143861864 shutdown_connections 2024-05-08T18:04:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.749+0000 7f15200d4640 1 -- 172.21.15.46:0/143861864 wait complete. 2024-05-08T18:04:36.758 INFO:teuthology.orchestra.run.smithi046.stdout:38654705667 2024-05-08T18:04:36.758 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T18:04:36.759 INFO:teuthology.orchestra.run.smithi046.stdout:38654705666 2024-05-08T18:04:36.760 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T18:04:36.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.866+0000 7f9d1df63640 1 Processor -- start 2024-05-08T18:04:36.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.866+0000 7f9d1df63640 1 -- start start 2024-05-08T18:04:36.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.866+0000 7f9d1df63640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9d18072540 0x7f9d181348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.866+0000 7f9d1df63640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d18073480 con 0x7f9d18070650 2024-05-08T18:04:36.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.866+0000 7f9d1df63640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9d18071a80 con 0x7f9d18072540 2024-05-08T18:04:36.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.866+0000 7f9d16ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9d18072540 0x7f9d181348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.866+0000 7f9d177fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9d18070650 0x7f9d18070a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40084/0 (socket says 172.21.15.46:40084) 2024-05-08T18:04:36.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.866+0000 7f9d177fe640 1 -- 172.21.15.46:0/406484125 learned_addr learned my addr 172.21.15.46:0/406484125 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:36.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.867+0000 7f9d16ffd640 1 -- 172.21.15.46:0/406484125 >> v1:172.21.15.46:6789/0 conn(0x7f9d18070650 legacy=0x7f9d18070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.867+0000 7f9d16ffd640 1 -- 172.21.15.46:0/406484125 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d18071140 con 0x7f9d18072540 2024-05-08T18:04:36.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.868+0000 7f9d16ffd640 1 --2- 172.21.15.46:0/406484125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d18072540 0x7f9d181348a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009720 tx=0x7f9d0c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c69ab10c9168b4e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.868+0000 7f9d167fc640 1 -- 172.21.15.46:0/406484125 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d0c0316d0 con 0x7f9d18072540 2024-05-08T18:04:36.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.868+0000 7f9d167fc640 1 -- 172.21.15.46:0/406484125 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d0c037070 con 0x7f9d18072540 2024-05-08T18:04:36.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.868+0000 7f9d167fc640 1 -- 172.21.15.46:0/406484125 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d0c0319c0 con 0x7f9d18072540 2024-05-08T18:04:36.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.869+0000 7f9d1df63640 1 -- 172.21.15.46:0/406484125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d18072540 msgr2=0x7f9d181348a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.869+0000 7f9d1df63640 1 --2- 172.21.15.46:0/406484125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d18072540 0x7f9d181348a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009720 tx=0x7f9d0c02b840 comp rx=0 tx=0).stop 2024-05-08T18:04:36.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.869+0000 7f9d1df63640 1 -- 172.21.15.46:0/406484125 shutdown_connections 2024-05-08T18:04:36.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.869+0000 7f9d1df63640 1 --2- 172.21.15.46:0/406484125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d18072540 0x7f9d181348a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.869+0000 7f9d1df63640 1 -- 172.21.15.46:0/406484125 >> 172.21.15.46:0/406484125 conn(0x7f9d180a1e60 msgr2=0x7f9d180a2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:36.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.869+0000 7f9d1df63640 1 -- 172.21.15.46:0/406484125 shutdown_connections 2024-05-08T18:04:36.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.869+0000 7f9d1df63640 1 -- 172.21.15.46:0/406484125 wait complete. 2024-05-08T18:04:36.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.871+0000 7f9d1df63640 1 Processor -- start 2024-05-08T18:04:36.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.871+0000 7f9d1df63640 1 -- start start 2024-05-08T18:04:36.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.871+0000 7f9d1df63640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d18070650 0x7f9d1811be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.871+0000 7f9d1df63640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9d18134f60 con 0x7f9d18070650 2024-05-08T18:04:36.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.872+0000 7f9d177fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d18070650 0x7f9d1811be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.872+0000 7f9d177fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d18070650 0x7f9d1811be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40114/0 (socket says 172.21.15.46:40114) 2024-05-08T18:04:36.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.872+0000 7f9d177fe640 1 -- 172.21.15.46:0/1512170427 learned_addr learned my addr 172.21.15.46:0/1512170427 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:36.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.872+0000 7f9d177fe640 1 -- 172.21.15.46:0/1512170427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1811f000 con 0x7f9d18070650 2024-05-08T18:04:36.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.872+0000 7f9d177fe640 1 --2- 172.21.15.46:0/1512170427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d18070650 0x7f9d1811be00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800c480 tx=0x7f9d0800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.872+0000 7f9cfffff640 1 -- 172.21.15.46:0/1512170427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d08017070 con 0x7f9d18070650 2024-05-08T18:04:36.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.872+0000 7f9d1df63640 1 -- 172.21.15.46:0/1512170427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d18130380 con 0x7f9d18070650 2024-05-08T18:04:36.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.872+0000 7f9cfffff640 1 -- 172.21.15.46:0/1512170427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d08005c10 con 0x7f9d18070650 2024-05-08T18:04:36.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.872+0000 7f9cfffff640 1 -- 172.21.15.46:0/1512170427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d08007070 con 0x7f9d18070650 2024-05-08T18:04:36.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.872+0000 7f9d1df63640 1 -- 172.21.15.46:0/1512170427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1811e970 con 0x7f9d18070650 2024-05-08T18:04:36.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.874+0000 7f9d1df63640 1 -- 172.21.15.46:0/1512170427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1811d760 con 0x7f9d18070650 2024-05-08T18:04:36.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.874+0000 7f9cfffff640 1 -- 172.21.15.46:0/1512170427 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9d0801a070 con 0x7f9d18070650 2024-05-08T18:04:36.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.874+0000 7f9cfffff640 1 --2- 172.21.15.46:0/1512170427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9d0403d260 0x7f9d0403f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.874+0000 7f9cfffff640 1 -- 172.21.15.46:0/1512170427 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f9d080505d0 con 0x7f9d18070650 2024-05-08T18:04:36.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.874+0000 7f9d16ffd640 1 --2- 172.21.15.46:0/1512170427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9d0403d260 0x7f9d0403f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.875+0000 7f9d16ffd640 1 --2- 172.21.15.46:0/1512170427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9d0403d260 0x7f9d0403f720 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c036000 tx=0x7f9d0c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.883+0000 7f9cfffff640 1 -- 172.21.15.46:0/1512170427 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9d1811f000 con 0x7f9d18070650 2024-05-08T18:04:36.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.910+0000 7fc2a3fce640 1 Processor -- start 2024-05-08T18:04:36.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.910+0000 7fdb47169640 1 Processor -- start 2024-05-08T18:04:36.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.911+0000 7fc2a3fce640 1 -- start start 2024-05-08T18:04:36.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.911+0000 7fdb47169640 1 -- start start 2024-05-08T18:04:36.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.911+0000 7fd4dacd4640 1 Processor -- start 2024-05-08T18:04:36.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.911+0000 7fd4dacd4640 1 -- start start 2024-05-08T18:04:36.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.911+0000 7fc2a3fce640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc29c070c30 0x7fc29c071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.911+0000 7fdb47169640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdb40070650 0x7fdb40070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.911+0000 7fdb47169640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb40073480 con 0x7fdb40072540 2024-05-08T18:04:36.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.911+0000 7fc2a3fce640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc29c09acc0 con 0x7fc29c071550 2024-05-08T18:04:36.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.911+0000 7fdb47169640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fdb40071a80 con 0x7fdb40070650 2024-05-08T18:04:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.911+0000 7fc2a3fce640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fc29c073920 con 0x7fc29c070c30 2024-05-08T18:04:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.911+0000 7fc2a1d6a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc29c070c30 0x7fc29c071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.911+0000 7fc2a1d6a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc29c070c30 0x7fc29c071010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40130/0 (socket says 172.21.15.46:40130) 2024-05-08T18:04:36.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.911+0000 7fc2a1d6a640 1 -- 172.21.15.46:0/3503603565 learned_addr learned my addr 172.21.15.46:0/3503603565 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.911+0000 7fdb44f05640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdb40070650 0x7fdb40070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.911+0000 7fdb44f05640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdb40070650 0x7fdb40070a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40132/0 (socket says 172.21.15.46:40132) 2024-05-08T18:04:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.911+0000 7fdb44f05640 1 -- 172.21.15.46:0/1002784045 learned_addr learned my addr 172.21.15.46:0/1002784045 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.912+0000 7fc2a1d6a640 1 -- 172.21.15.46:0/3503603565 >> v1:172.21.15.46:6789/0 conn(0x7fc29c071550 legacy=0x7fc29c124e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T18:04:36.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.912+0000 7fc2a1d6a640 1 -- 172.21.15.46:0/3503603565 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c071970 con 0x7fc29c070c30 2024-05-08T18:04:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.912+0000 7fd4dacd4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd4d4071d50 0x7fd4d4132b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.912+0000 7fd4dacd4640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4d40a0780 con 0x7fd4d4071260 2024-05-08T18:04:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.912+0000 7fd4dacd4640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fd4d4073b30 con 0x7fd4d4071d50 2024-05-08T18:04:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.912+0000 7fdb44f05640 1 -- 172.21.15.46:0/1002784045 >> v1:172.21.15.46:6789/0 conn(0x7fdb40072540 legacy=0x7fdb4012f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T18:04:36.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.912+0000 7fdb44f05640 1 -- 172.21.15.46:0/1002784045 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb40072c20 con 0x7fdb40070650 2024-05-08T18:04:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.912+0000 7fd4d8a70640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd4d4071260 0x7fd4d4071640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40108/0 (socket says 172.21.15.46:40108) 2024-05-08T18:04:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.912+0000 7fd4d8a70640 1 -- 172.21.15.46:0/2049791542 learned_addr learned my addr 172.21.15.46:0/2049791542 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.912+0000 7fd4d3fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd4d4071d50 0x7fd4d4132b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.912+0000 7fd4d3fff640 1 -- 172.21.15.46:0/2049791542 >> v1:172.21.15.46:6789/0 conn(0x7fd4d4071260 legacy=0x7fd4d4071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.912+0000 7fd4d3fff640 1 -- 172.21.15.46:0/2049791542 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d40a03c0 con 0x7fd4d4071d50 2024-05-08T18:04:36.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.913+0000 7fc2a1d6a640 1 --2- 172.21.15.46:0/3503603565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc29c070c30 0x7fc29c071010 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc29800ba80 tx=0x7fc29802de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a70aeb2adfa30a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.913+0000 7fdb44f05640 1 --2- 172.21.15.46:0/1002784045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb40070650 0x7fdb40070a30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb340097f0 tx=0x7fdb3402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8af6472e31d50ff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.913+0000 7fc2a0d68640 1 -- 172.21.15.46:0/3503603565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc298034070 con 0x7fc29c070c30 2024-05-08T18:04:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.913+0000 7fc2a0d68640 1 -- 172.21.15.46:0/3503603565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc29802eaf0 con 0x7fc29c070c30 2024-05-08T18:04:36.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.913+0000 7fc2a0d68640 1 -- 172.21.15.46:0/3503603565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc29802edf0 con 0x7fc29c070c30 2024-05-08T18:04:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.913+0000 7fdb3f7fe640 1 -- 172.21.15.46:0/1002784045 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb3402e070 con 0x7fdb40070650 2024-05-08T18:04:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.913+0000 7fdb3f7fe640 1 -- 172.21.15.46:0/1002784045 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb34037070 con 0x7fdb40070650 2024-05-08T18:04:36.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.913+0000 7fdb3f7fe640 1 -- 172.21.15.46:0/1002784045 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb34031730 con 0x7fdb40070650 2024-05-08T18:04:36.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.913+0000 7fd4d3fff640 1 --2- 172.21.15.46:0/2049791542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d4071d50 0x7fd4d4132b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009720 tx=0x7fd4c002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a5617c3ab23dd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fd4d37fe640 1 -- 172.21.15.46:0/2049791542 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4c00316d0 con 0x7fd4d4071d50 2024-05-08T18:04:36.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fd4d37fe640 1 -- 172.21.15.46:0/2049791542 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c0037070 con 0x7fd4d4071d50 2024-05-08T18:04:36.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fd4d37fe640 1 -- 172.21.15.46:0/2049791542 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4c00319c0 con 0x7fd4d4071d50 2024-05-08T18:04:36.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fc2a3fce640 1 -- 172.21.15.46:0/3503603565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc29c070c30 msgr2=0x7fc29c071010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fc2a3fce640 1 --2- 172.21.15.46:0/3503603565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc29c070c30 0x7fc29c071010 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc29800ba80 tx=0x7fc29802de10 comp rx=0 tx=0).stop 2024-05-08T18:04:36.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fc2a3fce640 1 -- 172.21.15.46:0/3503603565 shutdown_connections 2024-05-08T18:04:36.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fc2a3fce640 1 --2- 172.21.15.46:0/3503603565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc29c070c30 0x7fc29c071010 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fc2a3fce640 1 -- 172.21.15.46:0/3503603565 >> 172.21.15.46:0/3503603565 conn(0x7fc29c09d7a0 msgr2=0x7fc29c09dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:36.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fdb47169640 1 -- 172.21.15.46:0/1002784045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb40070650 msgr2=0x7fdb40070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fc2a3fce640 1 -- 172.21.15.46:0/3503603565 shutdown_connections 2024-05-08T18:04:36.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fc2a3fce640 1 -- 172.21.15.46:0/3503603565 wait complete. 2024-05-08T18:04:36.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fdb47169640 1 --2- 172.21.15.46:0/1002784045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb40070650 0x7fdb40070a30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdb340097f0 tx=0x7fdb3402b880 comp rx=0 tx=0).stop 2024-05-08T18:04:36.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fdb47169640 1 -- 172.21.15.46:0/1002784045 shutdown_connections 2024-05-08T18:04:36.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fdb47169640 1 --2- 172.21.15.46:0/1002784045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb40070650 0x7fdb40070a30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fdb47169640 1 -- 172.21.15.46:0/1002784045 >> 172.21.15.46:0/1002784045 conn(0x7fdb400a1e60 msgr2=0x7fdb400a2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fdb47169640 1 -- 172.21.15.46:0/1002784045 shutdown_connections 2024-05-08T18:04:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fdb47169640 1 -- 172.21.15.46:0/1002784045 wait complete. 2024-05-08T18:04:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fd4dacd4640 1 -- 172.21.15.46:0/2049791542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d4071d50 msgr2=0x7fd4d4132b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fd4dacd4640 1 --2- 172.21.15.46:0/2049791542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d4071d50 0x7fd4d4132b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009720 tx=0x7fd4c002b840 comp rx=0 tx=0).stop 2024-05-08T18:04:36.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fd4dacd4640 1 -- 172.21.15.46:0/2049791542 shutdown_connections 2024-05-08T18:04:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fd4dacd4640 1 --2- 172.21.15.46:0/2049791542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d4071d50 0x7fd4d4132b90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fd4dacd4640 1 -- 172.21.15.46:0/2049791542 >> 172.21.15.46:0/2049791542 conn(0x7fd4d409df70 msgr2=0x7fd4d409fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.914+0000 7fd4dacd4640 1 -- 172.21.15.46:0/2049791542 shutdown_connections 2024-05-08T18:04:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.915+0000 7fd4dacd4640 1 -- 172.21.15.46:0/2049791542 wait complete. 2024-05-08T18:04:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.915+0000 7fc2a3fce640 1 Processor -- start 2024-05-08T18:04:36.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.915+0000 7fc2a3fce640 1 -- start start 2024-05-08T18:04:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.915+0000 7fd4dacd4640 1 Processor -- start 2024-05-08T18:04:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.915+0000 7fd4dacd4640 1 -- start start 2024-05-08T18:04:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.915+0000 7fc2a3fce640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc29c070c30 0x7fc29c11c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.916+0000 7fdb47169640 1 Processor -- start 2024-05-08T18:04:36.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.916+0000 7fc2a3fce640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c0722c0 con 0x7fc29c070c30 2024-05-08T18:04:36.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.916+0000 7fc2a1d6a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc29c070c30 0x7fc29c11c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.916+0000 7fc2a1d6a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc29c070c30 0x7fc29c11c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40152/0 (socket says 172.21.15.46:40152) 2024-05-08T18:04:36.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.916+0000 7fc2a1d6a640 1 -- 172.21.15.46:0/2665912420 learned_addr learned my addr 172.21.15.46:0/2665912420 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:36.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.916+0000 7fdb47169640 1 -- start start 2024-05-08T18:04:36.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.916+0000 7fc2a1d6a640 1 -- 172.21.15.46:0/2665912420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c11e600 con 0x7fc29c070c30 2024-05-08T18:04:36.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.916+0000 7fd4dacd4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d4071260 0x7fd4d411a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.916+0000 7fd4dacd4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d4074170 con 0x7fd4d4071260 2024-05-08T18:04:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.916+0000 7fc2a1d6a640 1 --2- 172.21.15.46:0/2665912420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc29c070c30 0x7fc29c11c970 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc29800ba80 tx=0x7fc2980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.916+0000 7fd4d8a70640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d4071260 0x7fd4d411a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.916+0000 7fd4d8a70640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d4071260 0x7fd4d411a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40156/0 (socket says 172.21.15.46:40156) 2024-05-08T18:04:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.916+0000 7fd4d8a70640 1 -- 172.21.15.46:0/2606842083 learned_addr learned my addr 172.21.15.46:0/2606842083 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:36.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.916+0000 7fc2927fc640 1 -- 172.21.15.46:0/2665912420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2980376f0 con 0x7fc29c070c30 2024-05-08T18:04:36.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.916+0000 7fc2927fc640 1 -- 172.21.15.46:0/2665912420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2980378b0 con 0x7fc29c070c30 2024-05-08T18:04:36.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.916+0000 7fc2a3fce640 1 -- 172.21.15.46:0/2665912420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29c11ed60 con 0x7fc29c070c30 2024-05-08T18:04:36.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.917+0000 7fd4d8a70640 1 -- 172.21.15.46:0/2606842083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d411d8d0 con 0x7fd4d4071260 2024-05-08T18:04:36.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.917+0000 7fc2927fc640 1 -- 172.21.15.46:0/2665912420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc298037bd0 con 0x7fc29c070c30 2024-05-08T18:04:36.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.917+0000 7fc2a3fce640 1 -- 172.21.15.46:0/2665912420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29c11ceb0 con 0x7fc29c070c30 2024-05-08T18:04:36.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.917+0000 7fdb47169640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb40070650 0x7fdb401c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.917+0000 7fd4d8a70640 1 --2- 172.21.15.46:0/2606842083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d4071260 0x7fd4d411a2e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800c380 tx=0x7fd4c800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.917+0000 7fdb47169640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4012fc90 con 0x7fdb40070650 2024-05-08T18:04:36.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.917+0000 7fd4d17fa640 1 -- 172.21.15.46:0/2606842083 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4c8017070 con 0x7fd4d4071260 2024-05-08T18:04:36.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.917+0000 7fdb44f05640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb40070650 0x7fdb401c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.917+0000 7fd4d17fa640 1 -- 172.21.15.46:0/2606842083 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4c8005c10 con 0x7fd4d4071260 2024-05-08T18:04:36.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.917+0000 7fdb44f05640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb40070650 0x7fdb401c4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40166/0 (socket says 172.21.15.46:40166) 2024-05-08T18:04:36.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.917+0000 7fdb44f05640 1 -- 172.21.15.46:0/3561941742 learned_addr learned my addr 172.21.15.46:0/3561941742 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:36.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.917+0000 7fd4dacd4640 1 -- 172.21.15.46:0/2606842083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d411cf40 con 0x7fd4d4071260 2024-05-08T18:04:36.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.917+0000 7fd4d17fa640 1 -- 172.21.15.46:0/2606842083 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4c80070d0 con 0x7fd4d4071260 2024-05-08T18:04:36.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.918+0000 7fd4dacd4640 1 -- 172.21.15.46:0/2606842083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d411a820 con 0x7fd4d4071260 2024-05-08T18:04:36.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.918+0000 7fdb44f05640 1 -- 172.21.15.46:0/3561941742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4012c0c0 con 0x7fdb40070650 2024-05-08T18:04:36.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.918+0000 7fc2927fc640 1 -- 172.21.15.46:0/2665912420 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fc29804b400 con 0x7fc29c070c30 2024-05-08T18:04:36.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.918+0000 7fc2a3fce640 1 -- 172.21.15.46:0/2665912420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc29c124ed0 con 0x7fc29c070c30 2024-05-08T18:04:36.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.918+0000 7fdb44f05640 1 --2- 172.21.15.46:0/3561941742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb40070650 0x7fdb401c4490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdb34000c00 tx=0x7fdb340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.918+0000 7fc2927fc640 1 --2- 172.21.15.46:0/2665912420 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc28003d300 0x7fc28003f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.918+0000 7fdb3d7fa640 1 -- 172.21.15.46:0/3561941742 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb340326e0 con 0x7fdb40070650 2024-05-08T18:04:36.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.918+0000 7fc2927fc640 1 -- 172.21.15.46:0/2665912420 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7fc29803f050 con 0x7fc29c070c30 2024-05-08T18:04:36.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.919+0000 7fdb47169640 1 -- 172.21.15.46:0/3561941742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb40122190 con 0x7fdb40070650 2024-05-08T18:04:36.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.919+0000 7fdb47169640 1 -- 172.21.15.46:0/3561941742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb401c49d0 con 0x7fdb40070650 2024-05-08T18:04:36.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.919+0000 7fc2a1569640 1 --2- 172.21.15.46:0/2665912420 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc28003d300 0x7fc28003f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.919+0000 7fd4dacd4640 1 -- 172.21.15.46:0/2606842083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d40640a0 con 0x7fd4d4071260 2024-05-08T18:04:36.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.919+0000 7fd4d17fa640 1 -- 172.21.15.46:0/2606842083 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fd4c801a070 con 0x7fd4d4071260 2024-05-08T18:04:36.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.919+0000 7fc2a1569640 1 --2- 172.21.15.46:0/2665912420 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc28003d300 0x7fc28003f7c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc28c003cd0 tx=0x7fc28c007a30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.919+0000 7fd4d17fa640 1 --2- 172.21.15.46:0/2606842083 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd4b803d260 0x7fd4b803f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.919+0000 7fd4d17fa640 1 -- 172.21.15.46:0/2606842083 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7fd4c80505e0 con 0x7fd4d4071260 2024-05-08T18:04:36.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.920+0000 7fd4d3fff640 1 --2- 172.21.15.46:0/2606842083 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd4b803d260 0x7fd4b803f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.920+0000 7fdb47169640 1 -- 172.21.15.46:0/3561941742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb400640a0 con 0x7fdb40070650 2024-05-08T18:04:36.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.920+0000 7fd4d3fff640 1 --2- 172.21.15.46:0/2606842083 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd4b803d260 0x7fd4b803f720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0036000 tx=0x7fd4c0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.923+0000 7fc2927fc640 1 -- 172.21.15.46:0/2665912420 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fc298037d70 con 0x7fc29c070c30 2024-05-08T18:04:36.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.924+0000 7fdb3d7fa640 1 -- 172.21.15.46:0/3561941742 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb34037070 con 0x7fdb40070650 2024-05-08T18:04:36.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.924+0000 7fdb3d7fa640 1 -- 172.21.15.46:0/3561941742 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb340326e0 con 0x7fdb40070650 2024-05-08T18:04:36.937 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.929+0000 7fd4d17fa640 1 -- 172.21.15.46:0/2606842083 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fd4d40a0780 con 0x7fd4d4071260 2024-05-08T18:04:36.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.929+0000 7fdb3d7fa640 1 -- 172.21.15.46:0/3561941742 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fdb34032880 con 0x7fdb40070650 2024-05-08T18:04:36.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.930+0000 7fdb3d7fa640 1 --2- 172.21.15.46:0/3561941742 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdb2403d440 0x7fdb2403f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:36.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.930+0000 7fdb3d7fa640 1 -- 172.21.15.46:0/3561941742 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7fdb34072f80 con 0x7fdb40070650 2024-05-08T18:04:36.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.930+0000 7fdb3ffff640 1 --2- 172.21.15.46:0/3561941742 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdb2403d440 0x7fdb2403f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:36.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.931+0000 7fdb3ffff640 1 --2- 172.21.15.46:0/3561941742 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdb2403d440 0x7fdb2403f900 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdb40075ab0 tx=0x7fdb2c007a30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:36.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:36.934+0000 7fdb3d7fa640 1 -- 172.21.15.46:0/3561941742 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fdb34040070 con 0x7fdb40070650 2024-05-08T18:04:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.081+0000 7f9d1df63640 1 -- 172.21.15.46:0/1512170427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9d18134d80 con 0x7f9d18070650 2024-05-08T18:04:37.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.081+0000 7f9cfffff640 1 -- 172.21.15.46:0/1512170427 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9d18134d80 con 0x7f9d18070650 2024-05-08T18:04:37.079 INFO:teuthology.orchestra.run.smithi046.stdout:38654705666 2024-05-08T18:04:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.083+0000 7f9d1df63640 1 -- 172.21.15.46:0/1512170427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9d0403d260 msgr2=0x7f9d0403f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.083+0000 7f9d1df63640 1 --2- 172.21.15.46:0/1512170427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9d0403d260 0x7f9d0403f720 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c036000 tx=0x7f9d0c008030 comp rx=0 tx=0).stop 2024-05-08T18:04:37.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.083+0000 7f9d1df63640 1 -- 172.21.15.46:0/1512170427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d18070650 msgr2=0x7f9d1811be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.083+0000 7f9d1df63640 1 --2- 172.21.15.46:0/1512170427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d18070650 0x7f9d1811be00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800c480 tx=0x7f9d0800c940 comp rx=0 tx=0).stop 2024-05-08T18:04:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.083+0000 7f9d1df63640 1 -- 172.21.15.46:0/1512170427 shutdown_connections 2024-05-08T18:04:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.083+0000 7f9d1df63640 1 --2- 172.21.15.46:0/1512170427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9d0403d260 0x7f9d0403f720 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.083+0000 7f9d1df63640 1 --2- 172.21.15.46:0/1512170427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9d18070650 0x7f9d1811be00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.083+0000 7f9d1df63640 1 -- 172.21.15.46:0/1512170427 >> 172.21.15.46:0/1512170427 conn(0x7f9d180a1e60 msgr2=0x7f9d1806e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.083+0000 7f9d1df63640 1 -- 172.21.15.46:0/1512170427 shutdown_connections 2024-05-08T18:04:37.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.083+0000 7f9d1df63640 1 -- 172.21.15.46:0/1512170427 wait complete. 2024-05-08T18:04:37.096 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.0 2024-05-08T18:04:37.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.126+0000 7fdb47169640 1 -- 172.21.15.46:0/3561941742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fdb40069d40 con 0x7fdb40070650 2024-05-08T18:04:37.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.127+0000 7fdb3d7fa640 1 -- 172.21.15.46:0/3561941742 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdb340443a0 con 0x7fdb40070650 2024-05-08T18:04:37.125 INFO:teuthology.orchestra.run.smithi046.stdout:38654705665 2024-05-08T18:04:37.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.128+0000 7fdb47169640 1 -- 172.21.15.46:0/3561941742 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdb2403d440 msgr2=0x7fdb2403f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:37.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.128+0000 7fdb47169640 1 --2- 172.21.15.46:0/3561941742 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdb2403d440 0x7fdb2403f900 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdb40075ab0 tx=0x7fdb2c007a30 comp rx=0 tx=0).stop 2024-05-08T18:04:37.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.128+0000 7fdb47169640 1 -- 172.21.15.46:0/3561941742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb40070650 msgr2=0x7fdb401c4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:37.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.128+0000 7fdb47169640 1 --2- 172.21.15.46:0/3561941742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb40070650 0x7fdb401c4490 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fdb34000c00 tx=0x7fdb340046c0 comp rx=0 tx=0).stop 2024-05-08T18:04:37.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.129+0000 7fdb47169640 1 -- 172.21.15.46:0/3561941742 shutdown_connections 2024-05-08T18:04:37.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.129+0000 7fdb47169640 1 --2- 172.21.15.46:0/3561941742 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdb2403d440 0x7fdb2403f900 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:37.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.129+0000 7fdb47169640 1 --2- 172.21.15.46:0/3561941742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb40070650 0x7fdb401c4490 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:37.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.129+0000 7fdb47169640 1 -- 172.21.15.46:0/3561941742 >> 172.21.15.46:0/3561941742 conn(0x7fdb400a1e60 msgr2=0x7fdb4012c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:37.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.129+0000 7fdb47169640 1 -- 172.21.15.46:0/3561941742 shutdown_connections 2024-05-08T18:04:37.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.129+0000 7fdb47169640 1 -- 172.21.15.46:0/3561941742 wait complete. 2024-05-08T18:04:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.139+0000 7fc2a3fce640 1 -- 172.21.15.46:0/2665912420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fc29c0640a0 con 0x7fc29c070c30 2024-05-08T18:04:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.139+0000 7fc2927fc640 1 -- 172.21.15.46:0/2665912420 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc29804bc10 con 0x7fc29c070c30 2024-05-08T18:04:37.138 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705666 got 38654705665 for osd.3 2024-05-08T18:04:37.138 INFO:teuthology.orchestra.run.smithi046.stdout:38654705666 2024-05-08T18:04:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.141+0000 7fc2a3fce640 1 -- 172.21.15.46:0/2665912420 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc28003d300 msgr2=0x7fc28003f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.141+0000 7fc2a3fce640 1 --2- 172.21.15.46:0/2665912420 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc28003d300 0x7fc28003f7c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc28c003cd0 tx=0x7fc28c007a30 comp rx=0 tx=0).stop 2024-05-08T18:04:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.141+0000 7fc2a3fce640 1 -- 172.21.15.46:0/2665912420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc29c070c30 msgr2=0x7fc29c11c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.141+0000 7fc2a3fce640 1 --2- 172.21.15.46:0/2665912420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc29c070c30 0x7fc29c11c970 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc29800ba80 tx=0x7fc2980046c0 comp rx=0 tx=0).stop 2024-05-08T18:04:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.141+0000 7fc2a3fce640 1 -- 172.21.15.46:0/2665912420 shutdown_connections 2024-05-08T18:04:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.141+0000 7fc2a3fce640 1 --2- 172.21.15.46:0/2665912420 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc28003d300 0x7fc28003f7c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.141+0000 7fc2a3fce640 1 --2- 172.21.15.46:0/2665912420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc29c070c30 0x7fc29c11c970 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.141+0000 7fc2a3fce640 1 -- 172.21.15.46:0/2665912420 >> 172.21.15.46:0/2665912420 conn(0x7fc29c09d7a0 msgr2=0x7fc29c125300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.141+0000 7fc2a3fce640 1 -- 172.21.15.46:0/2665912420 shutdown_connections 2024-05-08T18:04:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.141+0000 7fc2a3fce640 1 -- 172.21.15.46:0/2665912420 wait complete. 2024-05-08T18:04:37.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.151+0000 7fd4dacd4640 1 -- 172.21.15.46:0/2606842083 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fd4d412c490 con 0x7fd4d4071260 2024-05-08T18:04:37.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.151+0000 7fd4d17fa640 1 -- 172.21.15.46:0/2606842083 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd4c801ac30 con 0x7fd4d4071260 2024-05-08T18:04:37.150 INFO:teuthology.orchestra.run.smithi046.stdout:38654705666 2024-05-08T18:04:37.150 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.2 2024-05-08T18:04:37.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.153+0000 7fd4dacd4640 1 -- 172.21.15.46:0/2606842083 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd4b803d260 msgr2=0x7fd4b803f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:37.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.153+0000 7fd4dacd4640 1 --2- 172.21.15.46:0/2606842083 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd4b803d260 0x7fd4b803f720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0036000 tx=0x7fd4c0008030 comp rx=0 tx=0).stop 2024-05-08T18:04:37.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.153+0000 7fd4dacd4640 1 -- 172.21.15.46:0/2606842083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d4071260 msgr2=0x7fd4d411a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:37.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.153+0000 7fd4dacd4640 1 --2- 172.21.15.46:0/2606842083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d4071260 0x7fd4d411a2e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd4c800c380 tx=0x7fd4c800c840 comp rx=0 tx=0).stop 2024-05-08T18:04:37.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.154+0000 7fd4dacd4640 1 -- 172.21.15.46:0/2606842083 shutdown_connections 2024-05-08T18:04:37.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.154+0000 7fd4dacd4640 1 --2- 172.21.15.46:0/2606842083 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd4b803d260 0x7fd4b803f720 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:37.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.154+0000 7fd4dacd4640 1 --2- 172.21.15.46:0/2606842083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd4d4071260 0x7fd4d411a2e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:37.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.154+0000 7fd4dacd4640 1 -- 172.21.15.46:0/2606842083 >> 172.21.15.46:0/2606842083 conn(0x7fd4d409df70 msgr2=0x7fd4d412d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:37.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.154+0000 7fd4dacd4640 1 -- 172.21.15.46:0/2606842083 shutdown_connections 2024-05-08T18:04:37.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:37.154+0000 7fd4dacd4640 1 -- 172.21.15.46:0/2606842083 wait complete. 2024-05-08T18:04:37.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.1 2024-05-08T18:04:38.097 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T18:04:38.139 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T18:04:38.152 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T18:04:38.163 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T18:04:38.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.219+0000 7f7f6c88e640 1 Processor -- start 2024-05-08T18:04:38.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.219+0000 7f7f6c88e640 1 -- start start 2024-05-08T18:04:38.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.219+0000 7f7f6c88e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7f68072540 0x7f7f681348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:38.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.219+0000 7f7f6c88e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f68073480 con 0x7f7f68070650 2024-05-08T18:04:38.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.219+0000 7f7f6c88e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7f68071a80 con 0x7f7f68072540 2024-05-08T18:04:38.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.220+0000 7f7f65d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7f68072540 0x7f7f681348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:38.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.220+0000 7f7f6659c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7f68070650 0x7f7f68070a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40112/0 (socket says 172.21.15.46:40112) 2024-05-08T18:04:38.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.220+0000 7f7f6659c640 1 -- 172.21.15.46:0/2910946483 learned_addr learned my addr 172.21.15.46:0/2910946483 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:38.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.220+0000 7f7f65d9b640 1 -- 172.21.15.46:0/2910946483 >> v1:172.21.15.46:6789/0 conn(0x7f7f68070650 legacy=0x7f7f68070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.220+0000 7f7f65d9b640 1 -- 172.21.15.46:0/2910946483 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f68071140 con 0x7f7f68072540 2024-05-08T18:04:38.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.221+0000 7f7f65d9b640 1 --2- 172.21.15.46:0/2910946483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f68072540 0x7f7f681348a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009720 tx=0x7f7f5c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11d14d9b4176ef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:38.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.221+0000 7f7f6559a640 1 -- 172.21.15.46:0/2910946483 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f5c0316d0 con 0x7f7f68072540 2024-05-08T18:04:38.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.221+0000 7f7f6559a640 1 -- 172.21.15.46:0/2910946483 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f5c037070 con 0x7f7f68072540 2024-05-08T18:04:38.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.221+0000 7f7f6559a640 1 -- 172.21.15.46:0/2910946483 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f5c0319c0 con 0x7f7f68072540 2024-05-08T18:04:38.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.222+0000 7f7f6c88e640 1 -- 172.21.15.46:0/2910946483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f68072540 msgr2=0x7f7f681348a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.222+0000 7f7f6c88e640 1 --2- 172.21.15.46:0/2910946483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f68072540 0x7f7f681348a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009720 tx=0x7f7f5c02b840 comp rx=0 tx=0).stop 2024-05-08T18:04:38.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.222+0000 7f7f6c88e640 1 -- 172.21.15.46:0/2910946483 shutdown_connections 2024-05-08T18:04:38.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.222+0000 7f7f6c88e640 1 --2- 172.21.15.46:0/2910946483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f68072540 0x7f7f681348a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.222+0000 7f7f6c88e640 1 -- 172.21.15.46:0/2910946483 >> 172.21.15.46:0/2910946483 conn(0x7f7f680a1e60 msgr2=0x7f7f680a2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:38.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.222+0000 7f7f6c88e640 1 -- 172.21.15.46:0/2910946483 shutdown_connections 2024-05-08T18:04:38.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.222+0000 7f7f6c88e640 1 -- 172.21.15.46:0/2910946483 wait complete. 2024-05-08T18:04:38.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.223+0000 7f7f6c88e640 1 Processor -- start 2024-05-08T18:04:38.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.223+0000 7f7f6c88e640 1 -- start start 2024-05-08T18:04:38.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.224+0000 7f7f6c88e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f68070650 0x7f7f6811be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:38.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.224+0000 7f7f6c88e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7f68134f60 con 0x7f7f68070650 2024-05-08T18:04:38.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.224+0000 7f7f6659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f68070650 0x7f7f6811be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:38.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.224+0000 7f7f6659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f68070650 0x7f7f6811be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40180/0 (socket says 172.21.15.46:40180) 2024-05-08T18:04:38.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.224+0000 7f7f6659c640 1 -- 172.21.15.46:0/2088447858 learned_addr learned my addr 172.21.15.46:0/2088447858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:38.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.224+0000 7f7f6659c640 1 -- 172.21.15.46:0/2088447858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6811f000 con 0x7f7f68070650 2024-05-08T18:04:38.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.225+0000 7f7f6659c640 1 --2- 172.21.15.46:0/2088447858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f68070650 0x7f7f6811be00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800c480 tx=0x7f7f5800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:38.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.225+0000 7f7f56ffd640 1 -- 172.21.15.46:0/2088447858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f58017070 con 0x7f7f68070650 2024-05-08T18:04:38.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.225+0000 7f7f56ffd640 1 -- 172.21.15.46:0/2088447858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f58005c10 con 0x7f7f68070650 2024-05-08T18:04:38.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.225+0000 7f7f6c88e640 1 -- 172.21.15.46:0/2088447858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f68130380 con 0x7f7f68070650 2024-05-08T18:04:38.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.225+0000 7f7f56ffd640 1 -- 172.21.15.46:0/2088447858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f58007070 con 0x7f7f68070650 2024-05-08T18:04:38.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.225+0000 7f7f6c88e640 1 -- 172.21.15.46:0/2088447858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6811e970 con 0x7f7f68070650 2024-05-08T18:04:38.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.226+0000 7f7f6c88e640 1 -- 172.21.15.46:0/2088447858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f6811d760 con 0x7f7f68070650 2024-05-08T18:04:38.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.226+0000 7f7f56ffd640 1 -- 172.21.15.46:0/2088447858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7f5801a070 con 0x7f7f68070650 2024-05-08T18:04:38.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.227+0000 7f7f56ffd640 1 --2- 172.21.15.46:0/2088447858 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7f4803d2b0 0x7f7f4803f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:38.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.227+0000 7f7f56ffd640 1 -- 172.21.15.46:0/2088447858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f7f58050580 con 0x7f7f68070650 2024-05-08T18:04:38.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.227+0000 7f7f65d9b640 1 --2- 172.21.15.46:0/2088447858 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7f4803d2b0 0x7f7f4803f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:38.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.227+0000 7f7f65d9b640 1 --2- 172.21.15.46:0/2088447858 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7f4803d2b0 0x7f7f4803f770 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c036000 tx=0x7f7f5c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:38.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.236+0000 7f7f56ffd640 1 -- 172.21.15.46:0/2088447858 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7f6811f000 con 0x7f7f68070650 2024-05-08T18:04:38.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.260+0000 7f35a3ad2640 1 Processor -- start 2024-05-08T18:04:38.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.261+0000 7f35a3ad2640 1 -- start start 2024-05-08T18:04:38.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.261+0000 7f35a3ad2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f359c072540 0x7f359c1348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:38.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.261+0000 7f35a3ad2640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f359c073480 con 0x7f359c070650 2024-05-08T18:04:38.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.261+0000 7f35a3ad2640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f359c071a80 con 0x7f359c072540 2024-05-08T18:04:38.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.261+0000 7f35a106d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f359c072540 0x7f359c1348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:38.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.261+0000 7f35a106d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f359c072540 0x7f359c1348a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40184/0 (socket says 172.21.15.46:40184) 2024-05-08T18:04:38.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.261+0000 7f35a106d640 1 -- 172.21.15.46:0/3591080211 learned_addr learned my addr 172.21.15.46:0/3591080211 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:38.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.261+0000 7f35a186e640 1 --1- 172.21.15.46:0/3591080211 >> v1:172.21.15.46:6789/0 conn(0x7f359c070650 0x7f359c070a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40124/0 (socket says 172.21.15.46:40124) 2024-05-08T18:04:38.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.262+0000 7f35a106d640 1 -- 172.21.15.46:0/3591080211 >> v1:172.21.15.46:6789/0 conn(0x7f359c070650 legacy=0x7f359c070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.262+0000 7f35a106d640 1 -- 172.21.15.46:0/3591080211 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c071140 con 0x7f359c072540 2024-05-08T18:04:38.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.262+0000 7f35a086c640 1 -- 172.21.15.46:0/3591080211 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2080672717 0 0) 0x7f359c073480 con 0x7f359c070650 2024-05-08T18:04:38.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.262+0000 7f35a106d640 1 --2- 172.21.15.46:0/3591080211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f359c072540 0x7f359c1348a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f358c0097f0 tx=0x7f358c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d26369a6afd025ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:38.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.263+0000 7f35a086c640 1 -- 172.21.15.46:0/3591080211 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f358c035020 con 0x7f359c072540 2024-05-08T18:04:38.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.263+0000 7f35a086c640 1 -- 172.21.15.46:0/3591080211 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f358c0317a0 con 0x7f359c072540 2024-05-08T18:04:38.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.263+0000 7f35a086c640 1 -- 172.21.15.46:0/3591080211 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f358c031aa0 con 0x7f359c072540 2024-05-08T18:04:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.263+0000 7f35a3ad2640 1 -- 172.21.15.46:0/3591080211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f359c072540 msgr2=0x7f359c1348a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.263+0000 7f35a3ad2640 1 --2- 172.21.15.46:0/3591080211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f359c072540 0x7f359c1348a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f358c0097f0 tx=0x7f358c02b910 comp rx=0 tx=0).stop 2024-05-08T18:04:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.264+0000 7f35a3ad2640 1 -- 172.21.15.46:0/3591080211 shutdown_connections 2024-05-08T18:04:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.264+0000 7f35a3ad2640 1 --2- 172.21.15.46:0/3591080211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f359c072540 0x7f359c1348a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.264+0000 7f35a3ad2640 1 -- 172.21.15.46:0/3591080211 >> 172.21.15.46:0/3591080211 conn(0x7f359c0a1e60 msgr2=0x7f359c0a2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.264+0000 7f35a3ad2640 1 -- 172.21.15.46:0/3591080211 shutdown_connections 2024-05-08T18:04:38.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.264+0000 7f35a3ad2640 1 -- 172.21.15.46:0/3591080211 wait complete. 2024-05-08T18:04:38.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.265+0000 7f35a3ad2640 1 Processor -- start 2024-05-08T18:04:38.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.265+0000 7f35a3ad2640 1 -- start start 2024-05-08T18:04:38.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.266+0000 7f35a3ad2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f359c070650 0x7f359c12b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:38.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.266+0000 7f35a3ad2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f359c134f60 con 0x7f359c070650 2024-05-08T18:04:38.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.266+0000 7f35a186e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f359c070650 0x7f359c12b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:38.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.266+0000 7f35a186e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f359c070650 0x7f359c12b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40194/0 (socket says 172.21.15.46:40194) 2024-05-08T18:04:38.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.266+0000 7f35a186e640 1 -- 172.21.15.46:0/3948023313 learned_addr learned my addr 172.21.15.46:0/3948023313 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:38.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.266+0000 7f35a186e640 1 -- 172.21.15.46:0/3948023313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359c11da00 con 0x7f359c070650 2024-05-08T18:04:38.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.267+0000 7f35a186e640 1 --2- 172.21.15.46:0/3948023313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f359c070650 0x7f359c12b910 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f359800c3d0 tx=0x7f359800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:38.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.267+0000 7f35927fc640 1 -- 172.21.15.46:0/3948023313 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35980074d0 con 0x7f359c070650 2024-05-08T18:04:38.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.267+0000 7f35a3ad2640 1 -- 172.21.15.46:0/3948023313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359c134a60 con 0x7f359c070650 2024-05-08T18:04:38.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.267+0000 7f35927fc640 1 -- 172.21.15.46:0/3948023313 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35980047e0 con 0x7f359c070650 2024-05-08T18:04:38.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.267+0000 7f35a3ad2640 1 -- 172.21.15.46:0/3948023313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359c11e870 con 0x7f359c070650 2024-05-08T18:04:38.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.267+0000 7f35927fc640 1 -- 172.21.15.46:0/3948023313 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35980074d0 con 0x7f359c070650 2024-05-08T18:04:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.268+0000 7f35927fc640 1 -- 172.21.15.46:0/3948023313 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3598011070 con 0x7f359c070650 2024-05-08T18:04:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.269+0000 7f35a3ad2640 1 -- 172.21.15.46:0/3948023313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359c0640a0 con 0x7f359c070650 2024-05-08T18:04:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.269+0000 7f35927fc640 1 --2- 172.21.15.46:0/3948023313 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f358003d640 0x7f358003fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:38.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.269+0000 7f35927fc640 1 -- 172.21.15.46:0/3948023313 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f359801bd50 con 0x7f359c070650 2024-05-08T18:04:38.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.269+0000 7f35a106d640 1 --2- 172.21.15.46:0/3948023313 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f358003d640 0x7f358003fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:38.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.270+0000 7f35a106d640 1 --2- 172.21.15.46:0/3948023313 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f358003d640 0x7f358003fb00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f358c036000 tx=0x7f358c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:38.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.274+0000 7f35927fc640 1 -- 172.21.15.46:0/3948023313 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f359801bb20 con 0x7f359c070650 2024-05-08T18:04:38.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.299+0000 7fa71c40d640 1 Processor -- start 2024-05-08T18:04:38.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.299+0000 7fa71c40d640 1 -- start start 2024-05-08T18:04:38.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.299+0000 7f817bd39640 1 Processor -- start 2024-05-08T18:04:38.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.300+0000 7f817bd39640 1 -- start start 2024-05-08T18:04:38.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.300+0000 7fa71c40d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa714072540 0x7fa7141348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:38.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.300+0000 7fa71c40d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa714073480 con 0x7fa714070650 2024-05-08T18:04:38.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.300+0000 7fa71c40d640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fa714071a80 con 0x7fa714072540 2024-05-08T18:04:38.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.300+0000 7fa7199a8640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa714072540 0x7fa7141348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:38.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.300+0000 7fa71a1a9640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa714070650 0x7fa714070a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40128/0 (socket says 172.21.15.46:40128) 2024-05-08T18:04:38.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.300+0000 7fa71a1a9640 1 -- 172.21.15.46:0/3749889265 learned_addr learned my addr 172.21.15.46:0/3749889265 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:38.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.300+0000 7f817bd39640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f817412a440 0x7f817412a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:38.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.300+0000 7f817bd39640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8174073140 con 0x7f817412d410 2024-05-08T18:04:38.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.300+0000 7f817bd39640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f8174073c20 con 0x7f817412a440 2024-05-08T18:04:38.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.300+0000 7fa7199a8640 1 -- 172.21.15.46:0/3749889265 >> v1:172.21.15.46:6789/0 conn(0x7fa714070650 legacy=0x7fa714070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.300+0000 7fa7199a8640 1 -- 172.21.15.46:0/3749889265 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa714071140 con 0x7fa714072540 2024-05-08T18:04:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.300+0000 7f8179ad5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f817412a440 0x7f817412a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.300+0000 7f8179ad5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f817412a440 0x7f817412a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40212/0 (socket says 172.21.15.46:40212) 2024-05-08T18:04:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.300+0000 7f8179ad5640 1 -- 172.21.15.46:0/2651505381 learned_addr learned my addr 172.21.15.46:0/2651505381 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.301+0000 7f8179ad5640 1 -- 172.21.15.46:0/2651505381 >> v1:172.21.15.46:6789/0 conn(0x7f817412d410 legacy=0x7f817412d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.301+0000 7f8179ad5640 1 -- 172.21.15.46:0/2651505381 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81740729f0 con 0x7f817412a440 2024-05-08T18:04:38.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.301+0000 7fa7199a8640 1 --2- 172.21.15.46:0/3749889265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714072540 0x7fa7141348a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa704009720 tx=0x7fa70402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d58e48958bee3c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:38.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.301+0000 7fa7191a7640 1 -- 172.21.15.46:0/3749889265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7040316d0 con 0x7fa714072540 2024-05-08T18:04:38.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.301+0000 7fa7191a7640 1 -- 172.21.15.46:0/3749889265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa704037070 con 0x7fa714072540 2024-05-08T18:04:38.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.301+0000 7fa7191a7640 1 -- 172.21.15.46:0/3749889265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7040319c0 con 0x7fa714072540 2024-05-08T18:04:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.302+0000 7f8179ad5640 1 --2- 172.21.15.46:0/2651505381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f817412a440 0x7f817412a820 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f81700097f0 tx=0x7f817002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdf0344ff23772db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.302+0000 7f8178ad3640 1 -- 172.21.15.46:0/2651505381 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8170035020 con 0x7f817412a440 2024-05-08T18:04:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.302+0000 7f8178ad3640 1 -- 172.21.15.46:0/2651505381 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81700317a0 con 0x7f817412a440 2024-05-08T18:04:38.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.302+0000 7f8178ad3640 1 -- 172.21.15.46:0/2651505381 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8170031aa0 con 0x7f817412a440 2024-05-08T18:04:38.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.302+0000 7fa71c40d640 1 -- 172.21.15.46:0/3749889265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714072540 msgr2=0x7fa7141348a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.302+0000 7fa71c40d640 1 --2- 172.21.15.46:0/3749889265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714072540 0x7fa7141348a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa704009720 tx=0x7fa70402b840 comp rx=0 tx=0).stop 2024-05-08T18:04:38.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.302+0000 7fa71c40d640 1 -- 172.21.15.46:0/3749889265 shutdown_connections 2024-05-08T18:04:38.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.302+0000 7fa71c40d640 1 --2- 172.21.15.46:0/3749889265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714072540 0x7fa7141348a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.302+0000 7fa71c40d640 1 -- 172.21.15.46:0/3749889265 >> 172.21.15.46:0/3749889265 conn(0x7fa7140a1e60 msgr2=0x7fa7140a2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:38.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.302+0000 7fa71c40d640 1 -- 172.21.15.46:0/3749889265 shutdown_connections 2024-05-08T18:04:38.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.302+0000 7fa71c40d640 1 -- 172.21.15.46:0/3749889265 wait complete. 2024-05-08T18:04:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.303+0000 7f817bd39640 1 -- 172.21.15.46:0/2651505381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f817412a440 msgr2=0x7f817412a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.303+0000 7f817bd39640 1 --2- 172.21.15.46:0/2651505381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f817412a440 0x7f817412a820 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f81700097f0 tx=0x7f817002b910 comp rx=0 tx=0).stop 2024-05-08T18:04:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.303+0000 7f817bd39640 1 -- 172.21.15.46:0/2651505381 shutdown_connections 2024-05-08T18:04:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.303+0000 7f817bd39640 1 --2- 172.21.15.46:0/2651505381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f817412a440 0x7f817412a820 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.303+0000 7f817bd39640 1 -- 172.21.15.46:0/2651505381 >> 172.21.15.46:0/2651505381 conn(0x7f817409c880 msgr2=0x7f817409cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.303+0000 7f817bd39640 1 -- 172.21.15.46:0/2651505381 shutdown_connections 2024-05-08T18:04:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.303+0000 7f817bd39640 1 -- 172.21.15.46:0/2651505381 wait complete. 2024-05-08T18:04:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.303+0000 7fa71c40d640 1 Processor -- start 2024-05-08T18:04:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.304+0000 7fa71c40d640 1 -- start start 2024-05-08T18:04:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.304+0000 7fa71c40d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714070650 0x7fa71411bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.304+0000 7fa71c40d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa714134f60 con 0x7fa714070650 2024-05-08T18:04:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.304+0000 7f817bd39640 1 Processor -- start 2024-05-08T18:04:38.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.304+0000 7fa71a1a9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714070650 0x7fa71411bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:38.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.304+0000 7fa71a1a9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714070650 0x7fa71411bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40218/0 (socket says 172.21.15.46:40218) 2024-05-08T18:04:38.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.304+0000 7fa71a1a9640 1 -- 172.21.15.46:0/1642418466 learned_addr learned my addr 172.21.15.46:0/1642418466 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:38.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.304+0000 7f817bd39640 1 -- start start 2024-05-08T18:04:38.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.305+0000 7fa71a1a9640 1 -- 172.21.15.46:0/1642418466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71411e9b0 con 0x7fa714070650 2024-05-08T18:04:38.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.305+0000 7fa71a1a9640 1 --2- 172.21.15.46:0/1642418466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714070650 0x7fa71411bf20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa71000c450 tx=0x7fa71000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:38.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.305+0000 7f817bd39640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f817412a440 0x7f817411a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:38.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.305+0000 7f817bd39640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8174073760 con 0x7f817412a440 2024-05-08T18:04:38.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.305+0000 7fa70affd640 1 -- 172.21.15.46:0/1642418466 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa710017070 con 0x7fa714070650 2024-05-08T18:04:38.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.305+0000 7f8179ad5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f817412a440 0x7f817411a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:38.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.305+0000 7fa70affd640 1 -- 172.21.15.46:0/1642418466 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa710005c10 con 0x7fa714070650 2024-05-08T18:04:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.305+0000 7f8179ad5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f817412a440 0x7f817411a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40232/0 (socket says 172.21.15.46:40232) 2024-05-08T18:04:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.305+0000 7fa70affd640 1 -- 172.21.15.46:0/1642418466 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa7100070d0 con 0x7fa714070650 2024-05-08T18:04:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.305+0000 7f8179ad5640 1 -- 172.21.15.46:0/2579468648 learned_addr learned my addr 172.21.15.46:0/2579468648 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.305+0000 7fa71c40d640 1 -- 172.21.15.46:0/1642418466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71411c460 con 0x7fa714070650 2024-05-08T18:04:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.305+0000 7f8179ad5640 1 -- 172.21.15.46:0/2579468648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817411d560 con 0x7f817412a440 2024-05-08T18:04:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.306+0000 7f8179ad5640 1 --2- 172.21.15.46:0/2579468648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f817412a440 0x7f817411a210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f81700097f0 tx=0x7f81700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.306+0000 7fa71c40d640 1 -- 172.21.15.46:0/1642418466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71411c930 con 0x7fa714070650 2024-05-08T18:04:38.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.306+0000 7f816a7fc640 1 -- 172.21.15.46:0/2579468648 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8170040070 con 0x7f817412a440 2024-05-08T18:04:38.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.306+0000 7f816a7fc640 1 -- 172.21.15.46:0/2579468648 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8170030c50 con 0x7f817412a440 2024-05-08T18:04:38.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.306+0000 7f816a7fc640 1 -- 172.21.15.46:0/2579468648 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8170035040 con 0x7f817412a440 2024-05-08T18:04:38.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.306+0000 7fa70affd640 1 -- 172.21.15.46:0/1642418466 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa71001a070 con 0x7fa714070650 2024-05-08T18:04:38.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.307+0000 7f817bd39640 1 -- 172.21.15.46:0/2579468648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f817411d800 con 0x7f817412a440 2024-05-08T18:04:38.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.307+0000 7f817bd39640 1 -- 172.21.15.46:0/2579468648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f817411a750 con 0x7f817412a440 2024-05-08T18:04:38.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.307+0000 7fa70affd640 1 --2- 172.21.15.46:0/1642418466 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa6f803d2b0 0x7fa6f803f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:38.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.307+0000 7fa70affd640 1 -- 172.21.15.46:0/1642418466 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7fa710054120 con 0x7fa714070650 2024-05-08T18:04:38.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.307+0000 7fa71c40d640 1 -- 172.21.15.46:0/1642418466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa71412f560 con 0x7fa714070650 2024-05-08T18:04:38.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.307+0000 7fa7199a8640 1 --2- 172.21.15.46:0/1642418466 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa6f803d2b0 0x7fa6f803f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:38.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.308+0000 7f817bd39640 1 -- 172.21.15.46:0/2579468648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f817411b3b0 con 0x7f817412a440 2024-05-08T18:04:38.315 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.308+0000 7fa7199a8640 1 --2- 172.21.15.46:0/1642418466 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa6f803d2b0 0x7fa6f803f770 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa704036000 tx=0x7fa704008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:38.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.308+0000 7f816a7fc640 1 -- 172.21.15.46:0/2579468648 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f817004b060 con 0x7f817412a440 2024-05-08T18:04:38.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.309+0000 7f816a7fc640 1 --2- 172.21.15.46:0/2579468648 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f815803d300 0x7f815803f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:38.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.309+0000 7f816a7fc640 1 -- 172.21.15.46:0/2579468648 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f81700733a0 con 0x7f817412a440 2024-05-08T18:04:38.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.309+0000 7f81792d4640 1 --2- 172.21.15.46:0/2579468648 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f815803d300 0x7f815803f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:38.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.309+0000 7f81792d4640 1 --2- 172.21.15.46:0/2579468648 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f815803d300 0x7f815803f7c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8164001600 tx=0x7f81640079f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:38.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.312+0000 7fa70affd640 1 -- 172.21.15.46:0/1642418466 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fa71412f560 con 0x7fa714070650 2024-05-08T18:04:38.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.317+0000 7f816a7fc640 1 -- 172.21.15.46:0/2579468648 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f817411d560 con 0x7f817412a440 2024-05-08T18:04:38.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.460+0000 7f7f6c88e640 1 -- 172.21.15.46:0/2088447858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7f68134d80 con 0x7f7f68070650 2024-05-08T18:04:38.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.460+0000 7f7f56ffd640 1 -- 172.21.15.46:0/2088447858 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7f68134d80 con 0x7f7f68070650 2024-05-08T18:04:38.459 INFO:teuthology.orchestra.run.smithi046.stdout:38654705667 2024-05-08T18:04:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.462+0000 7f7f6c88e640 1 -- 172.21.15.46:0/2088447858 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7f4803d2b0 msgr2=0x7f7f4803f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.462+0000 7f7f6c88e640 1 --2- 172.21.15.46:0/2088447858 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7f4803d2b0 0x7f7f4803f770 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c036000 tx=0x7f7f5c008030 comp rx=0 tx=0).stop 2024-05-08T18:04:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.462+0000 7f7f6c88e640 1 -- 172.21.15.46:0/2088447858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f68070650 msgr2=0x7f7f6811be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.462+0000 7f7f6c88e640 1 --2- 172.21.15.46:0/2088447858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f68070650 0x7f7f6811be00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7f5800c480 tx=0x7f7f5800c940 comp rx=0 tx=0).stop 2024-05-08T18:04:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.462+0000 7f7f6c88e640 1 -- 172.21.15.46:0/2088447858 shutdown_connections 2024-05-08T18:04:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.462+0000 7f7f6c88e640 1 --2- 172.21.15.46:0/2088447858 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7f4803d2b0 0x7f7f4803f770 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.462+0000 7f7f6c88e640 1 --2- 172.21.15.46:0/2088447858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f68070650 0x7f7f6811be00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.462+0000 7f7f6c88e640 1 -- 172.21.15.46:0/2088447858 >> 172.21.15.46:0/2088447858 conn(0x7f7f680a1e60 msgr2=0x7f7f6806e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.462+0000 7f7f6c88e640 1 -- 172.21.15.46:0/2088447858 shutdown_connections 2024-05-08T18:04:38.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.462+0000 7f7f6c88e640 1 -- 172.21.15.46:0/2088447858 wait complete. 2024-05-08T18:04:38.472 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705667 for osd.0 2024-05-08T18:04:38.472 DEBUG:teuthology.parallel:result is None 2024-05-08T18:04:38.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.474+0000 7f35a3ad2640 1 -- 172.21.15.46:0/3948023313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f359c11e1d0 con 0x7f359c070650 2024-05-08T18:04:38.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.475+0000 7f35927fc640 1 -- 172.21.15.46:0/3948023313 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3598004aa0 con 0x7f359c070650 2024-05-08T18:04:38.473 INFO:teuthology.orchestra.run.smithi046.stdout:38654705666 2024-05-08T18:04:38.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.477+0000 7f35a3ad2640 1 -- 172.21.15.46:0/3948023313 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f358003d640 msgr2=0x7f358003fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.477+0000 7f35a3ad2640 1 --2- 172.21.15.46:0/3948023313 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f358003d640 0x7f358003fb00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f358c036000 tx=0x7f358c008030 comp rx=0 tx=0).stop 2024-05-08T18:04:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.477+0000 7f35a3ad2640 1 -- 172.21.15.46:0/3948023313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f359c070650 msgr2=0x7f359c12b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.477+0000 7f35a3ad2640 1 --2- 172.21.15.46:0/3948023313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f359c070650 0x7f359c12b910 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f359800c3d0 tx=0x7f359800c890 comp rx=0 tx=0).stop 2024-05-08T18:04:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.477+0000 7f35a3ad2640 1 -- 172.21.15.46:0/3948023313 shutdown_connections 2024-05-08T18:04:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.477+0000 7f35a3ad2640 1 --2- 172.21.15.46:0/3948023313 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f358003d640 0x7f358003fb00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.477+0000 7f35a3ad2640 1 --2- 172.21.15.46:0/3948023313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f359c070650 0x7f359c12b910 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.477+0000 7f35a3ad2640 1 -- 172.21.15.46:0/3948023313 >> 172.21.15.46:0/3948023313 conn(0x7f359c0a1e60 msgr2=0x7f359c12ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.477+0000 7f35a3ad2640 1 -- 172.21.15.46:0/3948023313 shutdown_connections 2024-05-08T18:04:38.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.477+0000 7f35a3ad2640 1 -- 172.21.15.46:0/3948023313 wait complete. 2024-05-08T18:04:38.486 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705666 got 38654705666 for osd.3 2024-05-08T18:04:38.486 DEBUG:teuthology.parallel:result is None 2024-05-08T18:04:38.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.511+0000 7f817bd39640 1 -- 172.21.15.46:0/2579468648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f81740640a0 con 0x7f817412a440 2024-05-08T18:04:38.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.511+0000 7f816a7fc640 1 -- 172.21.15.46:0/2579468648 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f817004dda0 con 0x7f817412a440 2024-05-08T18:04:38.509 INFO:teuthology.orchestra.run.smithi046.stdout:38654705667 2024-05-08T18:04:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.514+0000 7f817bd39640 1 -- 172.21.15.46:0/2579468648 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f815803d300 msgr2=0x7f815803f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.514+0000 7f817bd39640 1 --2- 172.21.15.46:0/2579468648 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f815803d300 0x7f815803f7c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8164001600 tx=0x7f81640079f0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.514+0000 7f817bd39640 1 -- 172.21.15.46:0/2579468648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f817412a440 msgr2=0x7f817411a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.514+0000 7f817bd39640 1 --2- 172.21.15.46:0/2579468648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f817412a440 0x7f817411a210 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f81700097f0 tx=0x7f81700046c0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.514+0000 7f817bd39640 1 -- 172.21.15.46:0/2579468648 shutdown_connections 2024-05-08T18:04:38.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.514+0000 7f817bd39640 1 --2- 172.21.15.46:0/2579468648 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f815803d300 0x7f815803f7c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.514+0000 7f817bd39640 1 --2- 172.21.15.46:0/2579468648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f817412a440 0x7f817411a210 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.514+0000 7f817bd39640 1 -- 172.21.15.46:0/2579468648 >> 172.21.15.46:0/2579468648 conn(0x7f817409c880 msgr2=0x7f81741b5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.514+0000 7f817bd39640 1 -- 172.21.15.46:0/2579468648 shutdown_connections 2024-05-08T18:04:38.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.514+0000 7f817bd39640 1 -- 172.21.15.46:0/2579468648 wait complete. 2024-05-08T18:04:38.523 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705667 for osd.1 2024-05-08T18:04:38.523 DEBUG:teuthology.parallel:result is None 2024-05-08T18:04:38.527 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.529+0000 7fa71c40d640 1 -- 172.21.15.46:0/1642418466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa7140640a0 con 0x7fa714070650 2024-05-08T18:04:38.528 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.530+0000 7fa70affd640 1 -- 172.21.15.46:0/1642418466 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa71004f330 con 0x7fa714070650 2024-05-08T18:04:38.528 INFO:teuthology.orchestra.run.smithi046.stdout:38654705667 2024-05-08T18:04:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.532+0000 7fa71c40d640 1 -- 172.21.15.46:0/1642418466 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa6f803d2b0 msgr2=0x7fa6f803f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.532+0000 7fa71c40d640 1 --2- 172.21.15.46:0/1642418466 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa6f803d2b0 0x7fa6f803f770 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa704036000 tx=0x7fa704008030 comp rx=0 tx=0).stop 2024-05-08T18:04:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.532+0000 7fa71c40d640 1 -- 172.21.15.46:0/1642418466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714070650 msgr2=0x7fa71411bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.532+0000 7fa71c40d640 1 --2- 172.21.15.46:0/1642418466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714070650 0x7fa71411bf20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa71000c450 tx=0x7fa71000c910 comp rx=0 tx=0).stop 2024-05-08T18:04:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.532+0000 7fa71c40d640 1 -- 172.21.15.46:0/1642418466 shutdown_connections 2024-05-08T18:04:38.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.532+0000 7fa71c40d640 1 --2- 172.21.15.46:0/1642418466 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa6f803d2b0 0x7fa6f803f770 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.532+0000 7fa71c40d640 1 --2- 172.21.15.46:0/1642418466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa714070650 0x7fa71411bf20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.532+0000 7fa71c40d640 1 -- 172.21.15.46:0/1642418466 >> 172.21.15.46:0/1642418466 conn(0x7fa7140a1e60 msgr2=0x7fa71406e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:38.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.532+0000 7fa71c40d640 1 -- 172.21.15.46:0/1642418466 shutdown_connections 2024-05-08T18:04:38.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.532+0000 7fa71c40d640 1 -- 172.21.15.46:0/1642418466 wait complete. 2024-05-08T18:04:38.540 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705667 for osd.2 2024-05-08T18:04:38.541 DEBUG:teuthology.parallel:result is None 2024-05-08T18:04:38.541 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T18:04:38.541 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T18:04:38.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.685+0000 7ff57f379640 1 Processor -- start 2024-05-08T18:04:38.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.685+0000 7ff57f379640 1 -- start start 2024-05-08T18:04:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.686+0000 7ff57f379640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff578072540 0x7ff5781348a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.686+0000 7ff57f379640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff578073480 con 0x7ff578070650 2024-05-08T18:04:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.686+0000 7ff57f379640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff578071a80 con 0x7ff578072540 2024-05-08T18:04:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.686+0000 7ff57c914640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff578072540 0x7ff5781348a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.686+0000 7ff57d115640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff578070650 0x7ff578070a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40136/0 (socket says 172.21.15.46:40136) 2024-05-08T18:04:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.686+0000 7ff57c914640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff578072540 0x7ff5781348a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40246/0 (socket says 172.21.15.46:40246) 2024-05-08T18:04:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.686+0000 7ff57d115640 1 -- 172.21.15.46:0/1263541125 learned_addr learned my addr 172.21.15.46:0/1263541125 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.686+0000 7ff57c914640 1 -- 172.21.15.46:0/1263541125 >> v1:172.21.15.46:6789/0 conn(0x7ff578070650 legacy=0x7ff578070a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.686+0000 7ff57c914640 1 -- 172.21.15.46:0/1263541125 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff578071140 con 0x7ff578072540 2024-05-08T18:04:38.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.687+0000 7ff57c914640 1 --2- 172.21.15.46:0/1263541125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff578072540 0x7ff5781348a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff568009720 tx=0x7ff56802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0b17d3fc691eff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:38.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.687+0000 7ff56ffff640 1 -- 172.21.15.46:0/1263541125 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5680316d0 con 0x7ff578072540 2024-05-08T18:04:38.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.687+0000 7ff56ffff640 1 -- 172.21.15.46:0/1263541125 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff568037070 con 0x7ff578072540 2024-05-08T18:04:38.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.687+0000 7ff56ffff640 1 -- 172.21.15.46:0/1263541125 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5680319c0 con 0x7ff578072540 2024-05-08T18:04:38.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.688+0000 7ff57f379640 1 -- 172.21.15.46:0/1263541125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff578072540 msgr2=0x7ff5781348a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.688+0000 7ff57f379640 1 --2- 172.21.15.46:0/1263541125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff578072540 0x7ff5781348a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff568009720 tx=0x7ff56802b840 comp rx=0 tx=0).stop 2024-05-08T18:04:38.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.688+0000 7ff57f379640 1 -- 172.21.15.46:0/1263541125 shutdown_connections 2024-05-08T18:04:38.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.688+0000 7ff57f379640 1 --2- 172.21.15.46:0/1263541125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff578072540 0x7ff5781348a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.688+0000 7ff57f379640 1 -- 172.21.15.46:0/1263541125 >> 172.21.15.46:0/1263541125 conn(0x7ff5780a1e60 msgr2=0x7ff5780a2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:38.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.688+0000 7ff57f379640 1 -- 172.21.15.46:0/1263541125 shutdown_connections 2024-05-08T18:04:38.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.688+0000 7ff57f379640 1 -- 172.21.15.46:0/1263541125 wait complete. 2024-05-08T18:04:38.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.689+0000 7ff57f379640 1 Processor -- start 2024-05-08T18:04:38.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.689+0000 7ff57f379640 1 -- start start 2024-05-08T18:04:38.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.689+0000 7ff57f379640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff578070650 0x7ff57811bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:38.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.689+0000 7ff57f379640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff578134f60 con 0x7ff578070650 2024-05-08T18:04:38.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.689+0000 7ff57d115640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff578070650 0x7ff57811bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:38.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.689+0000 7ff57d115640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff578070650 0x7ff57811bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40248/0 (socket says 172.21.15.46:40248) 2024-05-08T18:04:38.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.689+0000 7ff57d115640 1 -- 172.21.15.46:0/132825854 learned_addr learned my addr 172.21.15.46:0/132825854 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:38.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.689+0000 7ff57d115640 1 -- 172.21.15.46:0/132825854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff57811eb30 con 0x7ff578070650 2024-05-08T18:04:38.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.689+0000 7ff57d115640 1 --2- 172.21.15.46:0/132825854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff578070650 0x7ff57811bd80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff57400cbd0 tx=0x7ff57400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:38.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.690+0000 7ff56dffb640 1 -- 172.21.15.46:0/132825854 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff574017070 con 0x7ff578070650 2024-05-08T18:04:38.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.690+0000 7ff56dffb640 1 -- 172.21.15.46:0/132825854 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff57400abc0 con 0x7ff578070650 2024-05-08T18:04:38.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.690+0000 7ff56dffb640 1 -- 172.21.15.46:0/132825854 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff574012400 con 0x7ff578070650 2024-05-08T18:04:38.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.690+0000 7ff57f379640 1 -- 172.21.15.46:0/132825854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff57811e390 con 0x7ff578070650 2024-05-08T18:04:38.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.690+0000 7ff57f379640 1 -- 172.21.15.46:0/132825854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff57811c2c0 con 0x7ff578070650 2024-05-08T18:04:38.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.691+0000 7ff56dffb640 1 -- 172.21.15.46:0/132825854 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff57401a070 con 0x7ff578070650 2024-05-08T18:04:38.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.691+0000 7ff56dffb640 1 --2- 172.21.15.46:0/132825854 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff55c03d2b0 0x7ff55c03f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:38.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.691+0000 7ff57f379640 1 -- 172.21.15.46:0/132825854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff57811d720 con 0x7ff578070650 2024-05-08T18:04:38.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.691+0000 7ff56dffb640 1 -- 172.21.15.46:0/132825854 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7ff574051290 con 0x7ff578070650 2024-05-08T18:04:38.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.691+0000 7ff57c914640 1 --2- 172.21.15.46:0/132825854 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff55c03d2b0 0x7ff55c03f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:38.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.692+0000 7ff57c914640 1 --2- 172.21.15.46:0/132825854 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff55c03d2b0 0x7ff55c03f770 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff568036000 tx=0x7ff568008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:38.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.696+0000 7ff56dffb640 1 -- 172.21.15.46:0/132825854 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff5740078c0 con 0x7ff578070650 2024-05-08T18:04:38.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.883+0000 7ff57f379640 1 -- 172.21.15.46:0/132825854 --> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff5780640a0 con 0x7ff55c03d2b0 2024-05-08T18:04:38.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.885+0000 7ff56dffb640 1 -- 172.21.15.46:0/132825854 <== mgr.4098 v2:172.21.15.50:6824/2602185226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+31799 (secure 0 0 0) 0x7ff5780640a0 con 0x7ff55c03d2b0 2024-05-08T18:04:38.883 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:38.883 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T18:04:38.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.886+0000 7ff57f379640 1 -- 172.21.15.46:0/132825854 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff55c03d2b0 msgr2=0x7ff55c03f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.886+0000 7ff57f379640 1 --2- 172.21.15.46:0/132825854 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff55c03d2b0 0x7ff55c03f770 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff568036000 tx=0x7ff568008030 comp rx=0 tx=0).stop 2024-05-08T18:04:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.886+0000 7ff57f379640 1 -- 172.21.15.46:0/132825854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff578070650 msgr2=0x7ff57811bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.886+0000 7ff57f379640 1 --2- 172.21.15.46:0/132825854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff578070650 0x7ff57811bd80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff57400cbd0 tx=0x7ff57400a430 comp rx=0 tx=0).stop 2024-05-08T18:04:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.886+0000 7ff57f379640 1 -- 172.21.15.46:0/132825854 shutdown_connections 2024-05-08T18:04:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.886+0000 7ff57f379640 1 --2- 172.21.15.46:0/132825854 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff55c03d2b0 0x7ff55c03f770 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.886+0000 7ff57f379640 1 --2- 172.21.15.46:0/132825854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff578070650 0x7ff57811bd80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.886+0000 7ff57f379640 1 -- 172.21.15.46:0/132825854 >> 172.21.15.46:0/132825854 conn(0x7ff5780a1e60 msgr2=0x7ff57806e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.886+0000 7ff57f379640 1 -- 172.21.15.46:0/132825854 shutdown_connections 2024-05-08T18:04:38.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:38.886+0000 7ff57f379640 1 -- 172.21.15.46:0/132825854 wait complete. 2024-05-08T18:04:38.895 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-05-08T18:04:37.665013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":130916,"kb_used_data":1544,"kb_used_omap":6,"kb_used_meta":128825,"kb_avail":377356444,"statfs":{"total":386547056640,"available":386412998656,"internally_reserved":0,"allocated":1581056,"data_stored":1287380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":131917590},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-08T18:04:36.550349+0000","last_change":"2024-05-08T18:04:35.551765+0000","last_active":"2024-05-08T18:04:36.550349+0000","last_peered":"2024-05-08T18:04:36.550349+0000","last_clean":"2024-05-08T18:04:36.550349+0000","last_became_active":"2024-05-08T18:04:35.551412+0000","last_became_peered":"2024-05-08T18:04:35.551412+0000","last_unstale":"2024-05-08T18:04:36.550349+0000","last_undegraded":"2024-05-08T18:04:36.550349+0000","last_fullsized":"2024-05-08T18:04:36.550349+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:37:38.776558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-05-08T18:04:35.552336+0000","last_change":"2024-05-08T18:04:35.551595+0000","last_active":"2024-05-08T18:04:35.552336+0000","last_peered":"2024-05-08T18:04:35.552336+0000","last_clean":"2024-05-08T18:04:35.552336+0000","last_became_active":"2024-05-08T18:04:35.551366+0000","last_became_peered":"2024-05-08T18:04:35.551366+0000","last_unstale":"2024-05-08T18:04:35.552336+0000","last_undegraded":"2024-05-08T18:04:35.552336+0000","last_fullsized":"2024-05-08T18:04:35.552336+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:45.895345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-08T18:04:36.550322+0000","last_change":"2024-05-08T18:04:35.551253+0000","last_active":"2024-05-08T18:04:36.550322+0000","last_peered":"2024-05-08T18:04:36.550322+0000","last_clean":"2024-05-08T18:04:36.550322+0000","last_became_active":"2024-05-08T18:04:35.550912+0000","last_became_peered":"2024-05-08T18:04:35.550912+0000","last_unstale":"2024-05-08T18:04:36.550322+0000","last_undegraded":"2024-05-08T18:04:36.550322+0000","last_fullsized":"2024-05-08T18:04:36.550322+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:49.303072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-05-08T18:04:35.553340+0000","last_change":"2024-05-08T18:04:35.552512+0000","last_active":"2024-05-08T18:04:35.553340+0000","last_peered":"2024-05-08T18:04:35.553340+0000","last_clean":"2024-05-08T18:04:35.553340+0000","last_became_active":"2024-05-08T18:04:35.552173+0000","last_became_peered":"2024-05-08T18:04:35.552173+0000","last_unstale":"2024-05-08T18:04:35.553340+0000","last_undegraded":"2024-05-08T18:04:35.553340+0000","last_fullsized":"2024-05-08T18:04:35.553340+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:58.749794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-08T18:04:36.550354+0000","last_change":"2024-05-08T18:04:35.551456+0000","last_active":"2024-05-08T18:04:36.550354+0000","last_peered":"2024-05-08T18:04:36.550354+0000","last_clean":"2024-05-08T18:04:36.550354+0000","last_became_active":"2024-05-08T18:04:35.551120+0000","last_became_peered":"2024-05-08T18:04:35.551120+0000","last_unstale":"2024-05-08T18:04:36.550354+0000","last_undegraded":"2024-05-08T18:04:36.550354+0000","last_fullsized":"2024-05-08T18:04:36.550354+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:14.181756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-05-08T18:04:35.553463+0000","last_change":"2024-05-08T18:04:35.551648+0000","last_active":"2024-05-08T18:04:35.553463+0000","last_peered":"2024-05-08T18:04:35.553463+0000","last_clean":"2024-05-08T18:04:35.553463+0000","last_became_active":"2024-05-08T18:04:35.551313+0000","last_became_peered":"2024-05-08T18:04:35.551313+0000","last_unstale":"2024-05-08T18:04:35.553463+0000","last_undegraded":"2024-05-08T18:04:35.553463+0000","last_fullsized":"2024-05-08T18:04:35.553463+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:45.895345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-08T18:04:36.550339+0000","last_change":"2024-05-08T18:04:35.552553+0000","last_active":"2024-05-08T18:04:36.550339+0000","last_peered":"2024-05-08T18:04:36.550339+0000","last_clean":"2024-05-08T18:04:36.550339+0000","last_became_active":"2024-05-08T18:04:35.552214+0000","last_became_peered":"2024-05-08T18:04:35.552214+0000","last_unstale":"2024-05-08T18:04:36.550339+0000","last_undegraded":"2024-05-08T18:04:36.550339+0000","last_fullsized":"2024-05-08T18:04:36.550339+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:12:16.476925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-08T18:04:36.550416+0000","last_change":"2024-05-08T18:04:35.552341+0000","last_active":"2024-05-08T18:04:36.550416+0000","last_peered":"2024-05-08T18:04:36.550416+0000","last_clean":"2024-05-08T18:04:36.550416+0000","last_became_active":"2024-05-08T18:04:35.552017+0000","last_became_peered":"2024-05-08T18:04:35.552017+0000","last_unstale":"2024-05-08T18:04:36.550416+0000","last_undegraded":"2024-05-08T18:04:36.550416+0000","last_fullsized":"2024-05-08T18:04:36.550416+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:30.242269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":99,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-05-08T18:04:35.911730+0000","last_change":"2024-05-08T18:04:35.552002+0000","last_active":"2024-05-08T18:04:35.911730+0000","last_peered":"2024-05-08T18:04:35.911730+0000","last_clean":"2024-05-08T18:04:35.911730+0000","last_became_active":"2024-05-08T18:04:35.551437+0000","last_became_peered":"2024-05-08T18:04:35.551437+0000","last_unstale":"2024-05-08T18:04:35.911730+0000","last_undegraded":"2024-05-08T18:04:35.911730+0000","last_fullsized":"2024-05-08T18:04:35.911730+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:45.895345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705666,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":98304,"data_stored":26661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705667,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":692224,"data_stored":617029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":692224,"data_stored":617029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35436,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336404,"statfs":{"total":96636764160,"available":96600477696,"internally_reserved":0,"allocated":98304,"data_stored":26661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T18:04:38.896 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T18:04:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.000+0000 7f3f31b06640 1 Processor -- start 2024-05-08T18:04:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.000+0000 7f3f31b06640 1 -- start start 2024-05-08T18:04:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.000+0000 7f3f31b06640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3f2c070650 0x7f3f2c070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.000+0000 7f3f31b06640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f2c073480 con 0x7f3f2c072540 2024-05-08T18:04:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.000+0000 7f3f31b06640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f3f2c071a80 con 0x7f3f2c070650 2024-05-08T18:04:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.000+0000 7f3f2b7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3f2c070650 0x7f3f2c070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.000+0000 7f3f2affd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3f2c072540 0x7f3f2c12f580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40140/0 (socket says 172.21.15.46:40140) 2024-05-08T18:04:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.000+0000 7f3f2affd640 1 -- 172.21.15.46:0/2513208994 learned_addr learned my addr 172.21.15.46:0/2513208994 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.001+0000 7f3f2b7fe640 1 -- 172.21.15.46:0/2513208994 >> v1:172.21.15.46:6789/0 conn(0x7f3f2c072540 legacy=0x7f3f2c12f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:38.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.001+0000 7f3f2b7fe640 1 -- 172.21.15.46:0/2513208994 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c072c20 con 0x7f3f2c070650 2024-05-08T18:04:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.001+0000 7f3f2b7fe640 1 --2- 172.21.15.46:0/2513208994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c070650 0x7f3f2c070a30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c009720 tx=0x7f3f1c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dfd19142f00246f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.002+0000 7f3f2a7fc640 1 -- 172.21.15.46:0/2513208994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f1c0316d0 con 0x7f3f2c070650 2024-05-08T18:04:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.002+0000 7f3f2a7fc640 1 -- 172.21.15.46:0/2513208994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f1c037070 con 0x7f3f2c070650 2024-05-08T18:04:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.002+0000 7f3f2a7fc640 1 -- 172.21.15.46:0/2513208994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f1c0319c0 con 0x7f3f2c070650 2024-05-08T18:04:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.002+0000 7f3f31b06640 1 -- 172.21.15.46:0/2513208994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c070650 msgr2=0x7f3f2c070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.002+0000 7f3f31b06640 1 --2- 172.21.15.46:0/2513208994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c070650 0x7f3f2c070a30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c009720 tx=0x7f3f1c02b840 comp rx=0 tx=0).stop 2024-05-08T18:04:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.002+0000 7f3f31b06640 1 -- 172.21.15.46:0/2513208994 shutdown_connections 2024-05-08T18:04:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.002+0000 7f3f31b06640 1 --2- 172.21.15.46:0/2513208994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c070650 0x7f3f2c070a30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:39.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.002+0000 7f3f31b06640 1 -- 172.21.15.46:0/2513208994 >> 172.21.15.46:0/2513208994 conn(0x7f3f2c0a1e60 msgr2=0x7f3f2c0a2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:39.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.002+0000 7f3f31b06640 1 -- 172.21.15.46:0/2513208994 shutdown_connections 2024-05-08T18:04:39.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.002+0000 7f3f31b06640 1 -- 172.21.15.46:0/2513208994 wait complete. 2024-05-08T18:04:39.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.003+0000 7f3f31b06640 1 Processor -- start 2024-05-08T18:04:39.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.003+0000 7f3f31b06640 1 -- start start 2024-05-08T18:04:39.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.003+0000 7f3f31b06640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c070650 0x7f3f2c1c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.003+0000 7f3f31b06640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2c12fc90 con 0x7f3f2c070650 2024-05-08T18:04:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.004+0000 7f3f2b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c070650 0x7f3f2c1c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.004+0000 7f3f2b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c070650 0x7f3f2c1c4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40264/0 (socket says 172.21.15.46:40264) 2024-05-08T18:04:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.004+0000 7f3f2b7fe640 1 -- 172.21.15.46:0/2980727414 learned_addr learned my addr 172.21.15.46:0/2980727414 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.004+0000 7f3f2b7fe640 1 -- 172.21.15.46:0/2980727414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f2c12c5d0 con 0x7f3f2c070650 2024-05-08T18:04:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.004+0000 7f3f2b7fe640 1 --2- 172.21.15.46:0/2980727414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c070650 0x7f3f2c1c4450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c034040 tx=0x7f3f1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.004+0000 7f3f13fff640 1 -- 172.21.15.46:0/2980727414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f1c032770 con 0x7f3f2c070650 2024-05-08T18:04:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.004+0000 7f3f13fff640 1 -- 172.21.15.46:0/2980727414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f1c032910 con 0x7f3f2c070650 2024-05-08T18:04:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.004+0000 7f3f31b06640 1 -- 172.21.15.46:0/2980727414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f2c11e9b0 con 0x7f3f2c070650 2024-05-08T18:04:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.004+0000 7f3f13fff640 1 -- 172.21.15.46:0/2980727414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f1c032c30 con 0x7f3f2c070650 2024-05-08T18:04:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.004+0000 7f3f31b06640 1 -- 172.21.15.46:0/2980727414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f2c11f640 con 0x7f3f2c070650 2024-05-08T18:04:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.005+0000 7f3f13fff640 1 -- 172.21.15.46:0/2980727414 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3f1c04c060 con 0x7f3f2c070650 2024-05-08T18:04:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.005+0000 7f3f13fff640 1 --2- 172.21.15.46:0/2980727414 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3f1803d2b0 0x7f3f1803f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.005+0000 7f3f13fff640 1 -- 172.21.15.46:0/2980727414 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f3f1c02f050 con 0x7f3f2c070650 2024-05-08T18:04:39.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.005+0000 7f3f31b06640 1 -- 172.21.15.46:0/2980727414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f2c0640a0 con 0x7f3f2c070650 2024-05-08T18:04:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.006+0000 7f3f2affd640 1 --2- 172.21.15.46:0/2980727414 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3f1803d2b0 0x7f3f1803f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:39.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.006+0000 7f3f2affd640 1 --2- 172.21.15.46:0/2980727414 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3f1803d2b0 0x7f3f1803f770 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3f20007fd0 tx=0x7f3f20007aa0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:39.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.010+0000 7f3f13fff640 1 -- 172.21.15.46:0/2980727414 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3f2c12c5d0 con 0x7f3f2c070650 2024-05-08T18:04:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.196+0000 7f3f31b06640 1 -- 172.21.15.46:0/2980727414 --> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3f2c120100 con 0x7f3f1803d2b0 2024-05-08T18:04:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.198+0000 7f3f13fff640 1 -- 172.21.15.46:0/2980727414 <== mgr.4098 v2:172.21.15.50:6824/2602185226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+31799 (secure 0 0 0) 0x7f3f2c120100 con 0x7f3f1803d2b0 2024-05-08T18:04:39.196 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T18:04:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.200+0000 7f3f31b06640 1 -- 172.21.15.46:0/2980727414 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3f1803d2b0 msgr2=0x7f3f1803f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.200+0000 7f3f31b06640 1 --2- 172.21.15.46:0/2980727414 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3f1803d2b0 0x7f3f1803f770 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3f20007fd0 tx=0x7f3f20007aa0 comp rx=0 tx=0).stop 2024-05-08T18:04:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.200+0000 7f3f31b06640 1 -- 172.21.15.46:0/2980727414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c070650 msgr2=0x7f3f2c1c4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.200+0000 7f3f31b06640 1 --2- 172.21.15.46:0/2980727414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c070650 0x7f3f2c1c4450 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c034040 tx=0x7f3f1c0046c0 comp rx=0 tx=0).stop 2024-05-08T18:04:39.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.200+0000 7f3f31b06640 1 -- 172.21.15.46:0/2980727414 shutdown_connections 2024-05-08T18:04:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.200+0000 7f3f31b06640 1 --2- 172.21.15.46:0/2980727414 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3f1803d2b0 0x7f3f1803f770 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.200+0000 7f3f31b06640 1 --2- 172.21.15.46:0/2980727414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f2c070650 0x7f3f2c1c4450 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.200+0000 7f3f31b06640 1 -- 172.21.15.46:0/2980727414 >> 172.21.15.46:0/2980727414 conn(0x7f3f2c0a1e60 msgr2=0x7f3f2c12bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.200+0000 7f3f31b06640 1 -- 172.21.15.46:0/2980727414 shutdown_connections 2024-05-08T18:04:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.200+0000 7f3f31b06640 1 -- 172.21.15.46:0/2980727414 wait complete. 2024-05-08T18:04:39.209 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-05-08T18:04:37.665013+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":130916,"kb_used_data":1544,"kb_used_omap":6,"kb_used_meta":128825,"kb_avail":377356444,"statfs":{"total":386547056640,"available":386412998656,"internally_reserved":0,"allocated":1581056,"data_stored":1287380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":131917590},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-08T18:04:36.550349+0000","last_change":"2024-05-08T18:04:35.551765+0000","last_active":"2024-05-08T18:04:36.550349+0000","last_peered":"2024-05-08T18:04:36.550349+0000","last_clean":"2024-05-08T18:04:36.550349+0000","last_became_active":"2024-05-08T18:04:35.551412+0000","last_became_peered":"2024-05-08T18:04:35.551412+0000","last_unstale":"2024-05-08T18:04:36.550349+0000","last_undegraded":"2024-05-08T18:04:36.550349+0000","last_fullsized":"2024-05-08T18:04:36.550349+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:37:38.776558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-05-08T18:04:35.552336+0000","last_change":"2024-05-08T18:04:35.551595+0000","last_active":"2024-05-08T18:04:35.552336+0000","last_peered":"2024-05-08T18:04:35.552336+0000","last_clean":"2024-05-08T18:04:35.552336+0000","last_became_active":"2024-05-08T18:04:35.551366+0000","last_became_peered":"2024-05-08T18:04:35.551366+0000","last_unstale":"2024-05-08T18:04:35.552336+0000","last_undegraded":"2024-05-08T18:04:35.552336+0000","last_fullsized":"2024-05-08T18:04:35.552336+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:45.895345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-08T18:04:36.550322+0000","last_change":"2024-05-08T18:04:35.551253+0000","last_active":"2024-05-08T18:04:36.550322+0000","last_peered":"2024-05-08T18:04:36.550322+0000","last_clean":"2024-05-08T18:04:36.550322+0000","last_became_active":"2024-05-08T18:04:35.550912+0000","last_became_peered":"2024-05-08T18:04:35.550912+0000","last_unstale":"2024-05-08T18:04:36.550322+0000","last_undegraded":"2024-05-08T18:04:36.550322+0000","last_fullsized":"2024-05-08T18:04:36.550322+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:49.303072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-05-08T18:04:35.553340+0000","last_change":"2024-05-08T18:04:35.552512+0000","last_active":"2024-05-08T18:04:35.553340+0000","last_peered":"2024-05-08T18:04:35.553340+0000","last_clean":"2024-05-08T18:04:35.553340+0000","last_became_active":"2024-05-08T18:04:35.552173+0000","last_became_peered":"2024-05-08T18:04:35.552173+0000","last_unstale":"2024-05-08T18:04:35.553340+0000","last_undegraded":"2024-05-08T18:04:35.553340+0000","last_fullsized":"2024-05-08T18:04:35.553340+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:58.749794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-08T18:04:36.550354+0000","last_change":"2024-05-08T18:04:35.551456+0000","last_active":"2024-05-08T18:04:36.550354+0000","last_peered":"2024-05-08T18:04:36.550354+0000","last_clean":"2024-05-08T18:04:36.550354+0000","last_became_active":"2024-05-08T18:04:35.551120+0000","last_became_peered":"2024-05-08T18:04:35.551120+0000","last_unstale":"2024-05-08T18:04:36.550354+0000","last_undegraded":"2024-05-08T18:04:36.550354+0000","last_fullsized":"2024-05-08T18:04:36.550354+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:14.181756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-05-08T18:04:35.553463+0000","last_change":"2024-05-08T18:04:35.551648+0000","last_active":"2024-05-08T18:04:35.553463+0000","last_peered":"2024-05-08T18:04:35.553463+0000","last_clean":"2024-05-08T18:04:35.553463+0000","last_became_active":"2024-05-08T18:04:35.551313+0000","last_became_peered":"2024-05-08T18:04:35.551313+0000","last_unstale":"2024-05-08T18:04:35.553463+0000","last_undegraded":"2024-05-08T18:04:35.553463+0000","last_fullsized":"2024-05-08T18:04:35.553463+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:45.895345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-08T18:04:36.550339+0000","last_change":"2024-05-08T18:04:35.552553+0000","last_active":"2024-05-08T18:04:36.550339+0000","last_peered":"2024-05-08T18:04:36.550339+0000","last_clean":"2024-05-08T18:04:36.550339+0000","last_became_active":"2024-05-08T18:04:35.552214+0000","last_became_peered":"2024-05-08T18:04:35.552214+0000","last_unstale":"2024-05-08T18:04:36.550339+0000","last_undegraded":"2024-05-08T18:04:36.550339+0000","last_fullsized":"2024-05-08T18:04:36.550339+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:12:16.476925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-05-08T18:04:36.550416+0000","last_change":"2024-05-08T18:04:35.552341+0000","last_active":"2024-05-08T18:04:36.550416+0000","last_peered":"2024-05-08T18:04:36.550416+0000","last_clean":"2024-05-08T18:04:36.550416+0000","last_became_active":"2024-05-08T18:04:35.552017+0000","last_became_peered":"2024-05-08T18:04:35.552017+0000","last_unstale":"2024-05-08T18:04:36.550416+0000","last_undegraded":"2024-05-08T18:04:36.550416+0000","last_fullsized":"2024-05-08T18:04:36.550416+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:30.242269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":99,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-05-08T18:04:35.911730+0000","last_change":"2024-05-08T18:04:35.552002+0000","last_active":"2024-05-08T18:04:35.911730+0000","last_peered":"2024-05-08T18:04:35.911730+0000","last_clean":"2024-05-08T18:04:35.911730+0000","last_became_active":"2024-05-08T18:04:35.551437+0000","last_became_peered":"2024-05-08T18:04:35.551437+0000","last_unstale":"2024-05-08T18:04:35.911730+0000","last_undegraded":"2024-05-08T18:04:35.911730+0000","last_fullsized":"2024-05-08T18:04:35.911730+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:02:45.895345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705666,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":98304,"data_stored":26661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705667,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":692224,"data_stored":617029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":692224,"data_stored":617029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35436,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336404,"statfs":{"total":96636764160,"available":96600477696,"internally_reserved":0,"allocated":98304,"data_stored":26661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T18:04:39.210 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T18:04:39.210 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T18:04:39.210 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-08T18:04:39.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T18:04:39.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.313+0000 7f92950dd640 1 Processor -- start 2024-05-08T18:04:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.313+0000 7f92950dd640 1 -- start start 2024-05-08T18:04:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.313+0000 7f92950dd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f929012c530 0x7f929012c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.313+0000 7f92950dd640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f929012c230 con 0x7f9290071d40 2024-05-08T18:04:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.313+0000 7f92950dd640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f929009ad20 con 0x7f929012c530 2024-05-08T18:04:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.314+0000 7f928ed9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f929012c530 0x7f929012c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.314+0000 7f928e59c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9290071d40 0x7f9290072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:40150/0 (socket says 172.21.15.46:40150) 2024-05-08T18:04:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.314+0000 7f928e59c640 1 -- 172.21.15.46:0/1919601468 learned_addr learned my addr 172.21.15.46:0/1919601468 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:04:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.314+0000 7f928ed9d640 1 -- 172.21.15.46:0/1919601468 >> v1:172.21.15.46:6789/0 conn(0x7f9290071d40 legacy=0x7f9290072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.314+0000 7f928dd9b640 1 -- 172.21.15.46:0/1919601468 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1480489545 0 0) 0x7f929012c230 con 0x7f9290071d40 2024-05-08T18:04:39.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.314+0000 7f928ed9d640 1 -- 172.21.15.46:0/1919601468 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9290073510 con 0x7f929012c530 2024-05-08T18:04:39.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.315+0000 7f928ed9d640 1 --2- 172.21.15.46:0/1919601468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929012c530 0x7f929012c910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f928800b9c0 tx=0x7f928802e010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8918e3135d66f64f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:39.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.315+0000 7f928dd9b640 1 -- 172.21.15.46:0/1919601468 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f928803b070 con 0x7f929012c530 2024-05-08T18:04:39.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.315+0000 7f928dd9b640 1 -- 172.21.15.46:0/1919601468 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928802ecf0 con 0x7f929012c530 2024-05-08T18:04:39.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.315+0000 7f928dd9b640 1 -- 172.21.15.46:0/1919601468 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9288036440 con 0x7f929012c530 2024-05-08T18:04:39.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.315+0000 7f92950dd640 1 -- 172.21.15.46:0/1919601468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929012c530 msgr2=0x7f929012c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:39.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.315+0000 7f92950dd640 1 --2- 172.21.15.46:0/1919601468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929012c530 0x7f929012c910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f928800b9c0 tx=0x7f928802e010 comp rx=0 tx=0).stop 2024-05-08T18:04:39.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.315+0000 7f92950dd640 1 -- 172.21.15.46:0/1919601468 shutdown_connections 2024-05-08T18:04:39.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.315+0000 7f92950dd640 1 --2- 172.21.15.46:0/1919601468 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f929012c530 0x7f929012c910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.315+0000 7f92950dd640 1 -- 172.21.15.46:0/1919601468 >> 172.21.15.46:0/1919601468 conn(0x7f929009d7a0 msgr2=0x7f929009dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.315+0000 7f92950dd640 1 -- 172.21.15.46:0/1919601468 shutdown_connections 2024-05-08T18:04:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.315+0000 7f92950dd640 1 -- 172.21.15.46:0/1919601468 wait complete. 2024-05-08T18:04:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.316+0000 7f92950dd640 1 Processor -- start 2024-05-08T18:04:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.317+0000 7f92950dd640 1 -- start start 2024-05-08T18:04:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.317+0000 7f92950dd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9290071d40 0x7f92901c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.317+0000 7f92950dd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f929012cfd0 con 0x7f9290071d40 2024-05-08T18:04:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.317+0000 7f928ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9290071d40 0x7f92901c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.317+0000 7f928ed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9290071d40 0x7f92901c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40280/0 (socket says 172.21.15.46:40280) 2024-05-08T18:04:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.317+0000 7f928ed9d640 1 -- 172.21.15.46:0/2583397324 learned_addr learned my addr 172.21.15.46:0/2583397324 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:04:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.317+0000 7f928ed9d640 1 -- 172.21.15.46:0/2583397324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9290118710 con 0x7f9290071d40 2024-05-08T18:04:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.317+0000 7f928ed9d640 1 --2- 172.21.15.46:0/2583397324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9290071d40 0x7f92901c4380 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f928800b9c0 tx=0x7f9288004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.318+0000 7f92877fe640 1 -- 172.21.15.46:0/2583397324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9288046070 con 0x7f9290071d40 2024-05-08T18:04:39.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.318+0000 7f92877fe640 1 -- 172.21.15.46:0/2583397324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9288004160 con 0x7f9290071d40 2024-05-08T18:04:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.318+0000 7f92950dd640 1 -- 172.21.15.46:0/2583397324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9290118ee0 con 0x7f9290071d40 2024-05-08T18:04:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.318+0000 7f92877fe640 1 -- 172.21.15.46:0/2583397324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f928803b040 con 0x7f9290071d40 2024-05-08T18:04:39.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.318+0000 7f92950dd640 1 -- 172.21.15.46:0/2583397324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9290119580 con 0x7f9290071d40 2024-05-08T18:04:39.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.318+0000 7f92950dd640 1 -- 172.21.15.46:0/2583397324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f929006f100 con 0x7f9290071d40 2024-05-08T18:04:39.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.318+0000 7f92877fe640 1 -- 172.21.15.46:0/2583397324 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9288049070 con 0x7f9290071d40 2024-05-08T18:04:39.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.319+0000 7f92877fe640 1 --2- 172.21.15.46:0/2583397324 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f927003d2b0 0x7f927003f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:39.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.319+0000 7f92877fe640 1 -- 172.21.15.46:0/2583397324 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f9288073280 con 0x7f9290071d40 2024-05-08T18:04:39.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.319+0000 7f928e59c640 1 --2- 172.21.15.46:0/2583397324 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f927003d2b0 0x7f927003f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:39.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.320+0000 7f928e59c640 1 --2- 172.21.15.46:0/2583397324 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f927003d2b0 0x7f927003f770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9278000c00 tx=0x7f92780078d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:39.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.323+0000 7f92877fe640 1 -- 172.21.15.46:0/2583397324 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f929006f100 con 0x7f9290071d40 2024-05-08T18:04:39.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.552+0000 7f92950dd640 1 -- 172.21.15.46:0/2583397324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f92900640a0 con 0x7f9290071d40 2024-05-08T18:04:39.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.552+0000 7f92877fe640 1 -- 172.21.15.46:0/2583397324 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f92880403a0 con 0x7f9290071d40 2024-05-08T18:04:39.551 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:04:39.551 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T18:04:39.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.554+0000 7f92950dd640 1 -- 172.21.15.46:0/2583397324 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f927003d2b0 msgr2=0x7f927003f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:39.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.554+0000 7f92950dd640 1 --2- 172.21.15.46:0/2583397324 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f927003d2b0 0x7f927003f770 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9278000c00 tx=0x7f92780078d0 comp rx=0 tx=0).stop 2024-05-08T18:04:39.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.554+0000 7f92950dd640 1 -- 172.21.15.46:0/2583397324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9290071d40 msgr2=0x7f92901c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:39.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.554+0000 7f92950dd640 1 --2- 172.21.15.46:0/2583397324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9290071d40 0x7f92901c4380 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f928800b9c0 tx=0x7f9288004730 comp rx=0 tx=0).stop 2024-05-08T18:04:39.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.554+0000 7f92950dd640 1 -- 172.21.15.46:0/2583397324 shutdown_connections 2024-05-08T18:04:39.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.554+0000 7f92950dd640 1 --2- 172.21.15.46:0/2583397324 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f927003d2b0 0x7f927003f770 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:39.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.554+0000 7f92950dd640 1 --2- 172.21.15.46:0/2583397324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9290071d40 0x7f92901c4380 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:39.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.554+0000 7f92950dd640 1 -- 172.21.15.46:0/2583397324 >> 172.21.15.46:0/2583397324 conn(0x7f929009d7a0 msgr2=0x7f92901225d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:39.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.555+0000 7f92950dd640 1 -- 172.21.15.46:0/2583397324 shutdown_connections 2024-05-08T18:04:39.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:04:39.555+0000 7f92950dd640 1 -- 172.21.15.46:0/2583397324 wait complete. 2024-05-08T18:04:39.564 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-08T18:04:39.564 INFO:teuthology.run_tasks:Running task rgw... 2024-05-08T18:04:39.579 DEBUG:tasks.rgw:config is {'client.0': None} 2024-05-08T18:04:39.579 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2024-05-08T18:04:39.580 INFO:tasks.rgw:Creating data pools 2024-05-08T18:04:39.580 DEBUG:tasks.rgw:Obtaining remote for client client.0 2024-05-08T18:04:39.580 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph osd erasure-code-profile set client.0 crush-failure-domain=osd k=3 m=1 name=testprofile --cluster ceph 2024-05-08T18:04:39.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.699+0000 7fe3445e5640 1 Processor -- start 2024-05-08T18:04:39.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.699+0000 7fe3445e5640 1 -- start start 2024-05-08T18:04:39.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.700+0000 7fe3445e5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe33c11bb70 0x7fe33c11df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:39.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.700+0000 7fe3445e5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe33c11e530 con 0x7fe33c118ef0 2024-05-08T18:04:39.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.700+0000 7fe3445e5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe33c11f730 con 0x7fe33c11bb70 2024-05-08T18:04:39.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.700+0000 7fe341b80640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe33c11bb70 0x7fe33c11df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:39.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.700+0000 7fe342381640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe33c118ef0 0x7fe33c1192d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:45798/0 (socket says 172.21.15.50:45798) 2024-05-08T18:04:39.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.700+0000 7fe342381640 1 -- 172.21.15.50:0/1729086116 learned_addr learned my addr 172.21.15.50:0/1729086116 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:04:39.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.700+0000 7fe341b80640 1 -- 172.21.15.50:0/1729086116 >> v1:172.21.15.46:6789/0 conn(0x7fe33c118ef0 legacy=0x7fe33c1192d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:39.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.700+0000 7fe341b80640 1 -- 172.21.15.50:0/1729086116 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33c11f8e0 con 0x7fe33c11bb70 2024-05-08T18:04:39.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.701+0000 7fe341b80640 1 --2- 172.21.15.50:0/1729086116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c11bb70 0x7fe33c11df60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009720 tx=0x7fe32c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a62ec977fb5b06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:39.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.701+0000 7fe34137f640 1 -- 172.21.15.50:0/1729086116 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe32c0316d0 con 0x7fe33c11bb70 2024-05-08T18:04:39.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.701+0000 7fe34137f640 1 -- 172.21.15.50:0/1729086116 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe32c037070 con 0x7fe33c11bb70 2024-05-08T18:04:39.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.701+0000 7fe34137f640 1 -- 172.21.15.50:0/1729086116 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe32c0319c0 con 0x7fe33c11bb70 2024-05-08T18:04:39.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.702+0000 7fe3445e5640 1 -- 172.21.15.50:0/1729086116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c11bb70 msgr2=0x7fe33c11df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:39.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.702+0000 7fe3445e5640 1 --2- 172.21.15.50:0/1729086116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c11bb70 0x7fe33c11df60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe32c009720 tx=0x7fe32c02b840 comp rx=0 tx=0).stop 2024-05-08T18:04:39.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.702+0000 7fe3445e5640 1 -- 172.21.15.50:0/1729086116 shutdown_connections 2024-05-08T18:04:39.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.702+0000 7fe3445e5640 1 --2- 172.21.15.50:0/1729086116 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c11bb70 0x7fe33c11df60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:39.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.702+0000 7fe3445e5640 1 -- 172.21.15.50:0/1729086116 >> 172.21.15.50:0/1729086116 conn(0x7fe33c10ed90 msgr2=0x7fe33c111180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:39.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.702+0000 7fe3445e5640 1 -- 172.21.15.50:0/1729086116 shutdown_connections 2024-05-08T18:04:39.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.702+0000 7fe3445e5640 1 -- 172.21.15.50:0/1729086116 wait complete. 2024-05-08T18:04:39.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.703+0000 7fe3445e5640 1 Processor -- start 2024-05-08T18:04:39.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.703+0000 7fe3445e5640 1 -- start start 2024-05-08T18:04:39.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.703+0000 7fe3445e5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c118ef0 0x7fe33c1a7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:39.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.703+0000 7fe3445e5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c11ebe0 con 0x7fe33c118ef0 2024-05-08T18:04:39.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.703+0000 7fe342381640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c118ef0 0x7fe33c1a7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:39.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.703+0000 7fe342381640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c118ef0 0x7fe33c1a7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:56290/0 (socket says 172.21.15.50:56290) 2024-05-08T18:04:39.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.703+0000 7fe342381640 1 -- 172.21.15.50:0/3159979355 learned_addr learned my addr 172.21.15.50:0/3159979355 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:04:39.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.703+0000 7fe342381640 1 -- 172.21.15.50:0/3159979355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33c1a7fd0 con 0x7fe33c118ef0 2024-05-08T18:04:39.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.704+0000 7fe342381640 1 --2- 172.21.15.50:0/3159979355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c118ef0 0x7fe33c1a7a90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe33800c450 tx=0x7fe33800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:39.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.704+0000 7fe332ffd640 1 -- 172.21.15.50:0/3159979355 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe338017070 con 0x7fe33c118ef0 2024-05-08T18:04:39.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.704+0000 7fe332ffd640 1 -- 172.21.15.50:0/3159979355 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe338005c10 con 0x7fe33c118ef0 2024-05-08T18:04:39.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.704+0000 7fe3445e5640 1 -- 172.21.15.50:0/3159979355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33c1a82c0 con 0x7fe33c118ef0 2024-05-08T18:04:39.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.704+0000 7fe332ffd640 1 -- 172.21.15.50:0/3159979355 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3380070d0 con 0x7fe33c118ef0 2024-05-08T18:04:39.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.704+0000 7fe3445e5640 1 -- 172.21.15.50:0/3159979355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33c1ac370 con 0x7fe33c118ef0 2024-05-08T18:04:39.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.705+0000 7fe332ffd640 1 -- 172.21.15.50:0/3159979355 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fe33801a070 con 0x7fe33c118ef0 2024-05-08T18:04:39.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.705+0000 7fe3445e5640 1 -- 172.21.15.50:0/3159979355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe33c051020 con 0x7fe33c118ef0 2024-05-08T18:04:39.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.705+0000 7fe332ffd640 1 --2- 172.21.15.50:0/3159979355 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe32003d260 0x7fe32003f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:39.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.705+0000 7fe332ffd640 1 -- 172.21.15.50:0/3159979355 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7fe338050720 con 0x7fe33c118ef0 2024-05-08T18:04:39.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.705+0000 7fe341b80640 1 --2- 172.21.15.50:0/3159979355 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe32003d260 0x7fe32003f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:39.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.706+0000 7fe341b80640 1 --2- 172.21.15.50:0/3159979355 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe32003d260 0x7fe32003f720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe32c036000 tx=0x7fe32c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:39.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.710+0000 7fe332ffd640 1 -- 172.21.15.50:0/3159979355 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe33801ad80 con 0x7fe33c118ef0 2024-05-08T18:04:39.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:39.908+0000 7fe3445e5640 1 -- 172.21.15.50:0/3159979355 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "client.0", "profile": ["crush-failure-domain=osd", "k=3", "m=1", "name=testprofile"]} v 0) -- 0x7fe33c0507d0 con 0x7fe33c118ef0 2024-05-08T18:04:40.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.778+0000 7fe332ffd640 1 -- 172.21.15.50:0/3159979355 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "client.0", "profile": ["crush-failure-domain=osd", "k=3", "m=1", "name=testprofile"]}]=0 v13) ==== 171+0+0 (secure 0 0 0) 0x7fe33801b3b0 con 0x7fe33c118ef0 2024-05-08T18:04:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.781+0000 7fe3445e5640 1 -- 172.21.15.50:0/3159979355 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe32003d260 msgr2=0x7fe32003f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.781+0000 7fe3445e5640 1 --2- 172.21.15.50:0/3159979355 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe32003d260 0x7fe32003f720 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe32c036000 tx=0x7fe32c008030 comp rx=0 tx=0).stop 2024-05-08T18:04:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.781+0000 7fe3445e5640 1 -- 172.21.15.50:0/3159979355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c118ef0 msgr2=0x7fe33c1a7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.781+0000 7fe3445e5640 1 --2- 172.21.15.50:0/3159979355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c118ef0 0x7fe33c1a7a90 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe33800c450 tx=0x7fe33800c910 comp rx=0 tx=0).stop 2024-05-08T18:04:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.781+0000 7fe3445e5640 1 -- 172.21.15.50:0/3159979355 shutdown_connections 2024-05-08T18:04:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.781+0000 7fe3445e5640 1 --2- 172.21.15.50:0/3159979355 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe32003d260 0x7fe32003f720 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.781+0000 7fe3445e5640 1 --2- 172.21.15.50:0/3159979355 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe33c118ef0 0x7fe33c1a7a90 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.781+0000 7fe3445e5640 1 -- 172.21.15.50:0/3159979355 >> 172.21.15.50:0/3159979355 conn(0x7fe33c10ed90 msgr2=0x7fe33c11bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:40.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.781+0000 7fe3445e5640 1 -- 172.21.15.50:0/3159979355 shutdown_connections 2024-05-08T18:04:40.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.781+0000 7fe3445e5640 1 -- 172.21.15.50:0/3159979355 wait complete. 2024-05-08T18:04:40.794 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2024-05-08T18:04:40.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.912+0000 7f6150d9f640 1 Processor -- start 2024-05-08T18:04:40.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.912+0000 7f6150d9f640 1 -- start start 2024-05-08T18:04:40.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.912+0000 7f6150d9f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f614c06e1c0 0x7f614c06e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:40.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.912+0000 7f6150d9f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f614c06ece0 con 0x7f614c11aac0 2024-05-08T18:04:40.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.912+0000 7f6150d9f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f614c11f5e0 con 0x7f614c06e1c0 2024-05-08T18:04:40.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.912+0000 7f614b7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f614c06e1c0 0x7f614c06e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:40.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.912+0000 7f614affd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f614c11aac0 0x7f614c11ceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:45806/0 (socket says 172.21.15.50:45806) 2024-05-08T18:04:40.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.912+0000 7f614affd640 1 -- 172.21.15.50:0/2302514276 learned_addr learned my addr 172.21.15.50:0/2302514276 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:04:40.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.912+0000 7f614b7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f614c06e1c0 0x7f614c06e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:56298/0 (socket says 172.21.15.50:56298) 2024-05-08T18:04:40.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.913+0000 7f614b7fe640 1 -- 172.21.15.50:0/2302514276 >> v1:172.21.15.46:6789/0 conn(0x7f614c11aac0 legacy=0x7f614c11ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:40.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.913+0000 7f614b7fe640 1 -- 172.21.15.50:0/2302514276 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c11f760 con 0x7f614c06e1c0 2024-05-08T18:04:40.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.914+0000 7f614b7fe640 1 --2- 172.21.15.50:0/2302514276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614c06e1c0 0x7f614c06e5a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f613c009720 tx=0x7f613c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c391c0d61dd298f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:40.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.914+0000 7f614a7fc640 1 -- 172.21.15.50:0/2302514276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f613c0316d0 con 0x7f614c06e1c0 2024-05-08T18:04:40.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.914+0000 7f614a7fc640 1 -- 172.21.15.50:0/2302514276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f613c037070 con 0x7f614c06e1c0 2024-05-08T18:04:40.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.914+0000 7f614a7fc640 1 -- 172.21.15.50:0/2302514276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f613c0319c0 con 0x7f614c06e1c0 2024-05-08T18:04:40.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.914+0000 7f6150d9f640 1 -- 172.21.15.50:0/2302514276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614c06e1c0 msgr2=0x7f614c06e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:40.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.914+0000 7f6150d9f640 1 --2- 172.21.15.50:0/2302514276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614c06e1c0 0x7f614c06e5a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f613c009720 tx=0x7f613c02b840 comp rx=0 tx=0).stop 2024-05-08T18:04:40.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.914+0000 7f6150d9f640 1 -- 172.21.15.50:0/2302514276 shutdown_connections 2024-05-08T18:04:40.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.914+0000 7f6150d9f640 1 --2- 172.21.15.50:0/2302514276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614c06e1c0 0x7f614c06e5a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:40.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.914+0000 7f6150d9f640 1 -- 172.21.15.50:0/2302514276 >> 172.21.15.50:0/2302514276 conn(0x7f614c072100 msgr2=0x7f614c074520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:40.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.915+0000 7f6150d9f640 1 -- 172.21.15.50:0/2302514276 shutdown_connections 2024-05-08T18:04:40.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.915+0000 7f6150d9f640 1 -- 172.21.15.50:0/2302514276 wait complete. 2024-05-08T18:04:40.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.915+0000 7f6150d9f640 1 Processor -- start 2024-05-08T18:04:40.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.915+0000 7f6150d9f640 1 -- start start 2024-05-08T18:04:40.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.916+0000 7f6150d9f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614c06e1c0 0x7f614c1ad490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:40.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.916+0000 7f6150d9f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f614c11eab0 con 0x7f614c06e1c0 2024-05-08T18:04:40.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.916+0000 7f614b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614c06e1c0 0x7f614c1ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:40.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.916+0000 7f614b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614c06e1c0 0x7f614c1ad490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:56306/0 (socket says 172.21.15.50:56306) 2024-05-08T18:04:40.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.916+0000 7f614b7fe640 1 -- 172.21.15.50:0/4003839423 learned_addr learned my addr 172.21.15.50:0/4003839423 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:04:40.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.916+0000 7f614b7fe640 1 -- 172.21.15.50:0/4003839423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c1ada60 con 0x7f614c06e1c0 2024-05-08T18:04:40.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.916+0000 7f614b7fe640 1 --2- 172.21.15.50:0/4003839423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614c06e1c0 0x7f614c1ad490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f613c034040 tx=0x7f613c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:40.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.917+0000 7f6133fff640 1 -- 172.21.15.50:0/4003839423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f613c032770 con 0x7f614c06e1c0 2024-05-08T18:04:40.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.917+0000 7f6150d9f640 1 -- 172.21.15.50:0/4003839423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614c1adcf0 con 0x7f614c06e1c0 2024-05-08T18:04:40.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.917+0000 7f6133fff640 1 -- 172.21.15.50:0/4003839423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f613c032910 con 0x7f614c06e1c0 2024-05-08T18:04:40.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.917+0000 7f6133fff640 1 -- 172.21.15.50:0/4003839423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f613c032c10 con 0x7f614c06e1c0 2024-05-08T18:04:40.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.917+0000 7f6150d9f640 1 -- 172.21.15.50:0/4003839423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f614c1a7580 con 0x7f614c06e1c0 2024-05-08T18:04:40.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.918+0000 7f6150d9f640 1 -- 172.21.15.50:0/4003839423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614c051020 con 0x7f614c06e1c0 2024-05-08T18:04:40.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.918+0000 7f6133fff640 1 -- 172.21.15.50:0/4003839423 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f613c04c060 con 0x7f614c06e1c0 2024-05-08T18:04:40.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.918+0000 7f6133fff640 1 --2- 172.21.15.50:0/4003839423 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f613803d260 0x7f613803f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:40.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.918+0000 7f6133fff640 1 -- 172.21.15.50:0/4003839423 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 3797+0+0 (secure 0 0 0) 0x7f613c02f050 con 0x7f614c06e1c0 2024-05-08T18:04:40.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.918+0000 7f614affd640 1 --2- 172.21.15.50:0/4003839423 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f613803d260 0x7f613803f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:40.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.919+0000 7f614affd640 1 --2- 172.21.15.50:0/4003839423 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f613803d260 0x7f613803f720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6140000f60 tx=0x7f6140007ae0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:40.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:40.924+0000 7f6133fff640 1 -- 172.21.15.50:0/4003839423 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f613c040070 con 0x7f614c06e1c0 2024-05-08T18:04:41.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:41.115+0000 7f6150d9f640 1 -- 172.21.15.50:0/4003839423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64, "pool_type": "erasure", "erasure_code_profile": "client.0"} v 0) -- 0x7f614c1a8be0 con 0x7f614c06e1c0 2024-05-08T18:04:42.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.785+0000 7f6133fff640 1 -- 172.21.15.50:0/4003839423 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64, "pool_type": "erasure", "erasure_code_profile": "client.0"}]=0 pool 'default.rgw.buckets.data' created v15) ==== 227+0+0 (secure 0 0 0) 0x7f613c03aca0 con 0x7f614c06e1c0 2024-05-08T18:04:42.784 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'default.rgw.buckets.data' created 2024-05-08T18:04:42.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.789+0000 7f6150d9f640 1 -- 172.21.15.50:0/4003839423 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f613803d260 msgr2=0x7f613803f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:42.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.789+0000 7f6150d9f640 1 --2- 172.21.15.50:0/4003839423 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f613803d260 0x7f613803f720 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6140000f60 tx=0x7f6140007ae0 comp rx=0 tx=0).stop 2024-05-08T18:04:42.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.789+0000 7f6150d9f640 1 -- 172.21.15.50:0/4003839423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614c06e1c0 msgr2=0x7f614c1ad490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:42.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.789+0000 7f6150d9f640 1 --2- 172.21.15.50:0/4003839423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614c06e1c0 0x7f614c1ad490 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f613c034040 tx=0x7f613c0046c0 comp rx=0 tx=0).stop 2024-05-08T18:04:42.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.789+0000 7f6150d9f640 1 -- 172.21.15.50:0/4003839423 shutdown_connections 2024-05-08T18:04:42.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.789+0000 7f6150d9f640 1 --2- 172.21.15.50:0/4003839423 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f613803d260 0x7f613803f720 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6140000f60 tx=0x7f6140007ae0 comp rx=0 tx=0).stop 2024-05-08T18:04:42.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.789+0000 7f6150d9f640 1 --2- 172.21.15.50:0/4003839423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f614c06e1c0 0x7f614c1ad490 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:42.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.789+0000 7f6150d9f640 1 -- 172.21.15.50:0/4003839423 >> 172.21.15.50:0/4003839423 conn(0x7f614c072100 msgr2=0x7f614c11aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:42.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.790+0000 7f6150d9f640 1 -- 172.21.15.50:0/4003839423 shutdown_connections 2024-05-08T18:04:42.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.790+0000 7f6150d9f640 1 -- 172.21.15.50:0/4003839423 wait complete. 2024-05-08T18:04:42.811 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2024-05-08T18:04:42.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.922+0000 7fb206544640 1 Processor -- start 2024-05-08T18:04:42.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.922+0000 7fb206544640 1 -- start start 2024-05-08T18:04:42.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.922+0000 7fb206544640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2001188a0 0x7fb20011ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:42.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.922+0000 7fb206544640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb20011b1d0 con 0x7fb200115bc0 2024-05-08T18:04:42.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.922+0000 7fb206544640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb20011c3d0 con 0x7fb2001188a0 2024-05-08T18:04:42.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.922+0000 7fb1ff7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2001188a0 0x7fb20011ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:42.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.922+0000 7fb1ff7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb2001188a0 0x7fb20011ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:56320/0 (socket says 172.21.15.50:56320) 2024-05-08T18:04:42.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.922+0000 7fb1ff7fe640 1 -- 172.21.15.50:0/3106100525 learned_addr learned my addr 172.21.15.50:0/3106100525 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:04:42.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.923+0000 7fb1ff7fe640 1 -- 172.21.15.50:0/3106100525 >> v1:172.21.15.46:6789/0 conn(0x7fb200115bc0 legacy=0x7fb200115fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:42.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.923+0000 7fb1ff7fe640 1 -- 172.21.15.50:0/3106100525 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20011c580 con 0x7fb2001188a0 2024-05-08T18:04:42.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.924+0000 7fb1ff7fe640 1 --2- 172.21.15.50:0/3106100525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2001188a0 0x7fb20011ac90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400ba90 tx=0x7fb1f402e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722f02468fc2793f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:42.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.924+0000 7fb1feffd640 1 -- 172.21.15.50:0/3106100525 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1f403b070 con 0x7fb2001188a0 2024-05-08T18:04:42.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.924+0000 7fb1feffd640 1 -- 172.21.15.50:0/3106100525 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f402ee50 con 0x7fb2001188a0 2024-05-08T18:04:42.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.924+0000 7fb1feffd640 1 -- 172.21.15.50:0/3106100525 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1f4036590 con 0x7fb2001188a0 2024-05-08T18:04:42.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.924+0000 7fb206544640 1 -- 172.21.15.50:0/3106100525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2001188a0 msgr2=0x7fb20011ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:42.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.924+0000 7fb206544640 1 --2- 172.21.15.50:0/3106100525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2001188a0 0x7fb20011ac90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400ba90 tx=0x7fb1f402e170 comp rx=0 tx=0).stop 2024-05-08T18:04:42.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.924+0000 7fb206544640 1 -- 172.21.15.50:0/3106100525 shutdown_connections 2024-05-08T18:04:42.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.924+0000 7fb206544640 1 --2- 172.21.15.50:0/3106100525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2001188a0 0x7fb20011ac90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:42.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.924+0000 7fb206544640 1 -- 172.21.15.50:0/3106100525 >> 172.21.15.50:0/3106100525 conn(0x7fb200070840 msgr2=0x7fb200072c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:42.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.925+0000 7fb206544640 1 -- 172.21.15.50:0/3106100525 shutdown_connections 2024-05-08T18:04:42.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.925+0000 7fb206544640 1 -- 172.21.15.50:0/3106100525 wait complete. 2024-05-08T18:04:42.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.925+0000 7fb206544640 1 Processor -- start 2024-05-08T18:04:42.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.925+0000 7fb206544640 1 -- start start 2024-05-08T18:04:42.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.926+0000 7fb206544640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb200115bc0 0x7fb2001a91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:42.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.926+0000 7fb206544640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb20011b880 con 0x7fb200115bc0 2024-05-08T18:04:42.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.926+0000 7fb1fffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb200115bc0 0x7fb2001a91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:42.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.926+0000 7fb1fffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb200115bc0 0x7fb2001a91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:56322/0 (socket says 172.21.15.50:56322) 2024-05-08T18:04:42.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.926+0000 7fb1fffff640 1 -- 172.21.15.50:0/3593920419 learned_addr learned my addr 172.21.15.50:0/3593920419 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:04:42.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.926+0000 7fb1fffff640 1 -- 172.21.15.50:0/3593920419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2001a97c0 con 0x7fb200115bc0 2024-05-08T18:04:42.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.926+0000 7fb1fffff640 1 --2- 172.21.15.50:0/3593920419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb200115bc0 0x7fb2001a91f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000c3a0 tx=0x7fb1f000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:42.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.926+0000 7fb1fcff9640 1 -- 172.21.15.50:0/3593920419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1f0017070 con 0x7fb200115bc0 2024-05-08T18:04:42.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.927+0000 7fb1fcff9640 1 -- 172.21.15.50:0/3593920419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f0005200 con 0x7fb200115bc0 2024-05-08T18:04:42.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.927+0000 7fb206544640 1 -- 172.21.15.50:0/3593920419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2001a9a50 con 0x7fb200115bc0 2024-05-08T18:04:42.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.927+0000 7fb1fcff9640 1 -- 172.21.15.50:0/3593920419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1f00035f0 con 0x7fb200115bc0 2024-05-08T18:04:42.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.927+0000 7fb206544640 1 -- 172.21.15.50:0/3593920419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2001a34e0 con 0x7fb200115bc0 2024-05-08T18:04:42.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.927+0000 7fb206544640 1 -- 172.21.15.50:0/3593920419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb200051020 con 0x7fb200115bc0 2024-05-08T18:04:42.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.928+0000 7fb1fcff9640 1 -- 172.21.15.50:0/3593920419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fb1f001a070 con 0x7fb200115bc0 2024-05-08T18:04:42.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.928+0000 7fb1fcff9640 1 --2- 172.21.15.50:0/3593920419 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb1ec03d2b0 0x7fb1ec03f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:42.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.928+0000 7fb1fcff9640 1 -- 172.21.15.50:0/3593920419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4272+0+0 (secure 0 0 0) 0x7fb1f0050980 con 0x7fb200115bc0 2024-05-08T18:04:42.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.928+0000 7fb1ff7fe640 1 --2- 172.21.15.50:0/3593920419 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb1ec03d2b0 0x7fb1ec03f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:42.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.928+0000 7fb1ff7fe640 1 --2- 172.21.15.50:0/3593920419 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb1ec03d2b0 0x7fb1ec03f770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4038000 tx=0x7fb1f4032000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:42.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:42.933+0000 7fb1fcff9640 1 -- 172.21.15.50:0/3593920419 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb1f001be00 con 0x7fb200115bc0 2024-05-08T18:04:43.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.127+0000 7fb206544640 1 -- 172.21.15.50:0/3593920419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"} v 0) -- 0x7fb2001188a0 con 0x7fb200115bc0 2024-05-08T18:04:43.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.789+0000 7fb1fcff9640 1 -- 172.21.15.50:0/3593920419 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.data' v16) ==== 185+0+0 (secure 0 0 0) 0x7fb1f00194a0 con 0x7fb200115bc0 2024-05-08T18:04:43.787 INFO:teuthology.orchestra.run.smithi050.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-05-08T18:04:43.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.792+0000 7fb206544640 1 -- 172.21.15.50:0/3593920419 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb1ec03d2b0 msgr2=0x7fb1ec03f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:43.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.792+0000 7fb206544640 1 --2- 172.21.15.50:0/3593920419 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb1ec03d2b0 0x7fb1ec03f770 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4038000 tx=0x7fb1f4032000 comp rx=0 tx=0).stop 2024-05-08T18:04:43.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.792+0000 7fb206544640 1 -- 172.21.15.50:0/3593920419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb200115bc0 msgr2=0x7fb2001a91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:43.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.792+0000 7fb206544640 1 --2- 172.21.15.50:0/3593920419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb200115bc0 0x7fb2001a91f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb1f000c3a0 tx=0x7fb1f000c860 comp rx=0 tx=0).stop 2024-05-08T18:04:43.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.792+0000 7fb206544640 1 -- 172.21.15.50:0/3593920419 shutdown_connections 2024-05-08T18:04:43.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.792+0000 7fb206544640 1 --2- 172.21.15.50:0/3593920419 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb1ec03d2b0 0x7fb1ec03f770 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:43.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.792+0000 7fb206544640 1 --2- 172.21.15.50:0/3593920419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb200115bc0 0x7fb2001a91f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:43.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.792+0000 7fb206544640 1 -- 172.21.15.50:0/3593920419 >> 172.21.15.50:0/3593920419 conn(0x7fb200070840 msgr2=0x7fb200110310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:43.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.793+0000 7fb206544640 1 -- 172.21.15.50:0/3593920419 shutdown_connections 2024-05-08T18:04:43.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.793+0000 7fb206544640 1 -- 172.21.15.50:0/3593920419 wait complete. 2024-05-08T18:04:43.816 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2024-05-08T18:04:43.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.943+0000 7f3f3eeff640 1 Processor -- start 2024-05-08T18:04:43.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.943+0000 7f3f3eeff640 1 -- start start 2024-05-08T18:04:43.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.943+0000 7f3f3eeff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3f38118ef0 0x7f3f381192d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:43.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.943+0000 7f3f3eeff640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f38119960 con 0x7f3f38112c60 2024-05-08T18:04:43.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.943+0000 7f3f3eeff640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f3f38113820 con 0x7f3f38118ef0 2024-05-08T18:04:43.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.944+0000 7f3f3cc9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3f38118ef0 0x7f3f381192d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:43.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.944+0000 7f3f37fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3f38112c60 0x7f3f38113110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:45816/0 (socket says 172.21.15.50:45816) 2024-05-08T18:04:43.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.944+0000 7f3f3cc9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3f38118ef0 0x7f3f381192d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:56338/0 (socket says 172.21.15.50:56338) 2024-05-08T18:04:43.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.944+0000 7f3f37fff640 1 -- 172.21.15.50:0/3540558107 learned_addr learned my addr 172.21.15.50:0/3540558107 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:04:43.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.944+0000 7f3f3cc9b640 1 -- 172.21.15.50:0/3540558107 >> v1:172.21.15.46:6789/0 conn(0x7f3f38112c60 legacy=0x7f3f38113110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:43.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.944+0000 7f3f3cc9b640 1 -- 172.21.15.50:0/3540558107 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f381139d0 con 0x7f3f38118ef0 2024-05-08T18:04:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.945+0000 7f3f3cc9b640 1 --2- 172.21.15.50:0/3540558107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f38118ef0 0x7f3f381192d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009720 tx=0x7f3f3002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6af903ce5925556 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.945+0000 7f3f377fe640 1 -- 172.21.15.50:0/3540558107 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f300316d0 con 0x7f3f38118ef0 2024-05-08T18:04:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.945+0000 7f3f377fe640 1 -- 172.21.15.50:0/3540558107 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f30037070 con 0x7f3f38118ef0 2024-05-08T18:04:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.945+0000 7f3f377fe640 1 -- 172.21.15.50:0/3540558107 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f300319c0 con 0x7f3f38118ef0 2024-05-08T18:04:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.946+0000 7f3f3eeff640 1 -- 172.21.15.50:0/3540558107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f38118ef0 msgr2=0x7f3f381192d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.946+0000 7f3f3eeff640 1 --2- 172.21.15.50:0/3540558107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f38118ef0 0x7f3f381192d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009720 tx=0x7f3f3002b840 comp rx=0 tx=0).stop 2024-05-08T18:04:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.946+0000 7f3f3eeff640 1 -- 172.21.15.50:0/3540558107 shutdown_connections 2024-05-08T18:04:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.946+0000 7f3f3eeff640 1 --2- 172.21.15.50:0/3540558107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f38118ef0 0x7f3f381192d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.946+0000 7f3f3eeff640 1 -- 172.21.15.50:0/3540558107 >> 172.21.15.50:0/3540558107 conn(0x7f3f3810ed90 msgr2=0x7f3f38111180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.946+0000 7f3f3eeff640 1 -- 172.21.15.50:0/3540558107 shutdown_connections 2024-05-08T18:04:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.946+0000 7f3f3eeff640 1 -- 172.21.15.50:0/3540558107 wait complete. 2024-05-08T18:04:43.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.946+0000 7f3f3eeff640 1 Processor -- start 2024-05-08T18:04:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.946+0000 7f3f3eeff640 1 -- start start 2024-05-08T18:04:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.947+0000 7f3f3eeff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f38112c60 0x7f3f381a79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.947+0000 7f3f3eeff640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3811f070 con 0x7f3f38112c60 2024-05-08T18:04:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.947+0000 7f3f3cc9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f38112c60 0x7f3f381a79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.947+0000 7f3f3cc9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f38112c60 0x7f3f381a79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:56346/0 (socket says 172.21.15.50:56346) 2024-05-08T18:04:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.947+0000 7f3f3cc9b640 1 -- 172.21.15.50:0/3985497932 learned_addr learned my addr 172.21.15.50:0/3985497932 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:04:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.947+0000 7f3f3cc9b640 1 -- 172.21.15.50:0/3985497932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f381a7f10 con 0x7f3f38112c60 2024-05-08T18:04:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.948+0000 7f3f3cc9b640 1 --2- 172.21.15.50:0/3985497932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f38112c60 0x7f3f381a79d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3f3002b880 tx=0x7f3f300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.948+0000 7f3f357fa640 1 -- 172.21.15.50:0/3985497932 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f30032770 con 0x7f3f38112c60 2024-05-08T18:04:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.948+0000 7f3f357fa640 1 -- 172.21.15.50:0/3985497932 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f30032910 con 0x7f3f38112c60 2024-05-08T18:04:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.948+0000 7f3f3eeff640 1 -- 172.21.15.50:0/3985497932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f381a81a0 con 0x7f3f38112c60 2024-05-08T18:04:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.948+0000 7f3f357fa640 1 -- 172.21.15.50:0/3985497932 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f30032bf0 con 0x7f3f38112c60 2024-05-08T18:04:43.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.948+0000 7f3f3eeff640 1 -- 172.21.15.50:0/3985497932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f381ac150 con 0x7f3f38112c60 2024-05-08T18:04:43.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.949+0000 7f3f3eeff640 1 -- 172.21.15.50:0/3985497932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f38051020 con 0x7f3f38112c60 2024-05-08T18:04:43.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.949+0000 7f3f357fa640 1 -- 172.21.15.50:0/3985497932 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3f3004c060 con 0x7f3f38112c60 2024-05-08T18:04:43.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.949+0000 7f3f357fa640 1 --2- 172.21.15.50:0/3985497932 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3f1803d260 0x7f3f1803f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:43.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.949+0000 7f3f357fa640 1 -- 172.21.15.50:0/3985497932 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4283+0+0 (secure 0 0 0) 0x7f3f3002f050 con 0x7f3f38112c60 2024-05-08T18:04:43.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.949+0000 7f3f37fff640 1 --2- 172.21.15.50:0/3985497932 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3f1803d260 0x7f3f1803f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:43.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.949+0000 7f3f37fff640 1 --2- 172.21.15.50:0/3985497932 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3f1803d260 0x7f3f1803f720 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3f28002060 tx=0x7f3f28001650 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:43.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:43.954+0000 7f3f357fa640 1 -- 172.21.15.50:0/3985497932 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3f30040070 con 0x7f3f38112c60 2024-05-08T18:04:44.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.143+0000 7f3f3eeff640 1 -- 172.21.15.50:0/3985497932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64} v 0) -- 0x7f3f380507d0 con 0x7f3f38112c60 2024-05-08T18:04:44.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.790+0000 7f3f357fa640 1 -- 172.21.15.50:0/3985497932 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64}]=0 pool 'default.rgw.buckets.index' created v17) ==== 169+0+0 (secure 0 0 0) 0x7f3f30032d90 con 0x7f3f38112c60 2024-05-08T18:04:44.788 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'default.rgw.buckets.index' created 2024-05-08T18:04:44.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.794+0000 7f3f3eeff640 1 -- 172.21.15.50:0/3985497932 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3f1803d260 msgr2=0x7f3f1803f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:44.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.794+0000 7f3f3eeff640 1 --2- 172.21.15.50:0/3985497932 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3f1803d260 0x7f3f1803f720 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3f28002060 tx=0x7f3f28001650 comp rx=0 tx=0).stop 2024-05-08T18:04:44.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.794+0000 7f3f3eeff640 1 -- 172.21.15.50:0/3985497932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f38112c60 msgr2=0x7f3f381a79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:44.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.794+0000 7f3f3eeff640 1 --2- 172.21.15.50:0/3985497932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f38112c60 0x7f3f381a79d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3f3002b880 tx=0x7f3f300046c0 comp rx=0 tx=0).stop 2024-05-08T18:04:44.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.795+0000 7f3f3eeff640 1 -- 172.21.15.50:0/3985497932 shutdown_connections 2024-05-08T18:04:44.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.795+0000 7f3f3eeff640 1 --2- 172.21.15.50:0/3985497932 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3f1803d260 0x7f3f1803f720 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:44.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.795+0000 7f3f3eeff640 1 --2- 172.21.15.50:0/3985497932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f38112c60 0x7f3f381a79d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:44.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.795+0000 7f3f3eeff640 1 -- 172.21.15.50:0/3985497932 >> 172.21.15.50:0/3985497932 conn(0x7f3f3810ed90 msgr2=0x7f3f38114ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:44.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.795+0000 7f3f3eeff640 1 -- 172.21.15.50:0/3985497932 shutdown_connections 2024-05-08T18:04:44.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.795+0000 7f3f3eeff640 1 -- 172.21.15.50:0/3985497932 wait complete. 2024-05-08T18:04:44.815 DEBUG:teuthology.orchestra.run.smithi050:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2024-05-08T18:04:44.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.925+0000 7f90ada4c640 1 Processor -- start 2024-05-08T18:04:44.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.926+0000 7f90ada4c640 1 -- start start 2024-05-08T18:04:44.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.926+0000 7f90ada4c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f90a81188a0 0x7f90a811ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:44.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.926+0000 7f90ada4c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90a811b1d0 con 0x7f90a8115bc0 2024-05-08T18:04:44.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.926+0000 7f90ada4c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f90a811c3d0 con 0x7f90a81188a0 2024-05-08T18:04:44.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.926+0000 7f90a67fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f90a81188a0 0x7f90a811ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:44.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.926+0000 7f90a6ffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f90a8115bc0 0x7f90a8115fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:44870/0 (socket says 172.21.15.50:44870) 2024-05-08T18:04:44.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.926+0000 7f90a67fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f90a81188a0 0x7f90a811ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:52536/0 (socket says 172.21.15.50:52536) 2024-05-08T18:04:44.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.926+0000 7f90a6ffd640 1 -- 172.21.15.50:0/1068332865 learned_addr learned my addr 172.21.15.50:0/1068332865 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:04:44.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.927+0000 7f90a67fc640 1 -- 172.21.15.50:0/1068332865 >> v1:172.21.15.46:6789/0 conn(0x7f90a8115bc0 legacy=0x7f90a8115fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:44.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.927+0000 7f90a67fc640 1 -- 172.21.15.50:0/1068332865 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a811c580 con 0x7f90a81188a0 2024-05-08T18:04:44.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.928+0000 7f90a67fc640 1 --2- 172.21.15.50:0/1068332865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90a81188a0 0x7f90a811ac90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f909c009720 tx=0x7f909c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f4a1c2cf810bd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:44.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.928+0000 7f90a5ffb640 1 -- 172.21.15.50:0/1068332865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f909c035020 con 0x7f90a81188a0 2024-05-08T18:04:44.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.928+0000 7f90a5ffb640 1 -- 172.21.15.50:0/1068332865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f909c031760 con 0x7f90a81188a0 2024-05-08T18:04:44.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.928+0000 7f90a5ffb640 1 -- 172.21.15.50:0/1068332865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f909c031a60 con 0x7f90a81188a0 2024-05-08T18:04:44.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.928+0000 7f90ada4c640 1 -- 172.21.15.50:0/1068332865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90a81188a0 msgr2=0x7f90a811ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:44.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.928+0000 7f90ada4c640 1 --2- 172.21.15.50:0/1068332865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90a81188a0 0x7f90a811ac90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f909c009720 tx=0x7f909c02b8d0 comp rx=0 tx=0).stop 2024-05-08T18:04:44.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.928+0000 7f90ada4c640 1 -- 172.21.15.50:0/1068332865 shutdown_connections 2024-05-08T18:04:44.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.928+0000 7f90ada4c640 1 --2- 172.21.15.50:0/1068332865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90a81188a0 0x7f90a811ac90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:44.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.928+0000 7f90ada4c640 1 -- 172.21.15.50:0/1068332865 >> 172.21.15.50:0/1068332865 conn(0x7f90a8070840 msgr2=0x7f90a8072c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:44.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.928+0000 7f90ada4c640 1 -- 172.21.15.50:0/1068332865 shutdown_connections 2024-05-08T18:04:44.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.928+0000 7f90ada4c640 1 -- 172.21.15.50:0/1068332865 wait complete. 2024-05-08T18:04:44.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.929+0000 7f90ada4c640 1 Processor -- start 2024-05-08T18:04:44.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.929+0000 7f90ada4c640 1 -- start start 2024-05-08T18:04:44.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.929+0000 7f90ada4c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90a8115bc0 0x7f90a81a7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:44.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.929+0000 7f90ada4c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f90a811b880 con 0x7f90a8115bc0 2024-05-08T18:04:44.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.930+0000 7f90a6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90a8115bc0 0x7f90a81a7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:44.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.930+0000 7f90a6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90a8115bc0 0x7f90a81a7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:52544/0 (socket says 172.21.15.50:52544) 2024-05-08T18:04:44.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.930+0000 7f90a6ffd640 1 -- 172.21.15.50:0/985355152 learned_addr learned my addr 172.21.15.50:0/985355152 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:04:44.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.930+0000 7f90a6ffd640 1 -- 172.21.15.50:0/985355152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a81a7f90 con 0x7f90a8115bc0 2024-05-08T18:04:44.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.930+0000 7f90a6ffd640 1 --2- 172.21.15.50:0/985355152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90a8115bc0 0x7f90a81a7a50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f909800c450 tx=0x7f909800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:44.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.930+0000 7f908ffff640 1 -- 172.21.15.50:0/985355152 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9098017070 con 0x7f90a8115bc0 2024-05-08T18:04:44.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.930+0000 7f908ffff640 1 -- 172.21.15.50:0/985355152 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9098005c10 con 0x7f90a8115bc0 2024-05-08T18:04:44.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.930+0000 7f90ada4c640 1 -- 172.21.15.50:0/985355152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a81a8280 con 0x7f90a8115bc0 2024-05-08T18:04:44.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.930+0000 7f908ffff640 1 -- 172.21.15.50:0/985355152 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90980070d0 con 0x7f90a8115bc0 2024-05-08T18:04:44.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.930+0000 7f90ada4c640 1 -- 172.21.15.50:0/985355152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a81ac330 con 0x7f90a8115bc0 2024-05-08T18:04:44.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.931+0000 7f90ada4c640 1 -- 172.21.15.50:0/985355152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a8051020 con 0x7f90a8115bc0 2024-05-08T18:04:44.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.932+0000 7f908ffff640 1 -- 172.21.15.50:0/985355152 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f909801a070 con 0x7f90a8115bc0 2024-05-08T18:04:44.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.932+0000 7f908ffff640 1 --2- 172.21.15.50:0/985355152 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f909403d290 0x7f909403f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:04:44.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.932+0000 7f908ffff640 1 -- 172.21.15.50:0/985355152 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4647+0+0 (secure 0 0 0) 0x7f9098050a70 con 0x7f90a8115bc0 2024-05-08T18:04:44.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.932+0000 7f90a67fc640 1 --2- 172.21.15.50:0/985355152 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f909403d290 0x7f909403f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:04:44.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.932+0000 7f90a67fc640 1 --2- 172.21.15.50:0/985355152 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f909403d290 0x7f909403f750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f909c036000 tx=0x7f909c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:04:44.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:44.938+0000 7f908ffff640 1 -- 172.21.15.50:0/985355152 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9098010db0 con 0x7f90a8115bc0 2024-05-08T18:04:45.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:45.132+0000 7f90ada4c640 1 -- 172.21.15.50:0/985355152 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"} v 0) -- 0x7f90a81a8450 con 0x7f90a8115bc0 2024-05-08T18:04:45.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:45.796+0000 7f908ffff640 1 -- 172.21.15.50:0/985355152 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.index' v18) ==== 187+0+0 (secure 0 0 0) 0x7f909801ac10 con 0x7f90a8115bc0 2024-05-08T18:04:45.794 INFO:teuthology.orchestra.run.smithi050.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-05-08T18:04:45.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:45.801+0000 7f90ada4c640 1 -- 172.21.15.50:0/985355152 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f909403d290 msgr2=0x7f909403f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:45.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:45.801+0000 7f90ada4c640 1 --2- 172.21.15.50:0/985355152 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f909403d290 0x7f909403f750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f909c036000 tx=0x7f909c008030 comp rx=0 tx=0).stop 2024-05-08T18:04:45.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:45.801+0000 7f90ada4c640 1 -- 172.21.15.50:0/985355152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90a8115bc0 msgr2=0x7f90a81a7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:04:45.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:45.801+0000 7f90ada4c640 1 --2- 172.21.15.50:0/985355152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90a8115bc0 0x7f90a81a7a50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f909800c450 tx=0x7f909800c910 comp rx=0 tx=0).stop 2024-05-08T18:04:45.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:45.802+0000 7f90ada4c640 1 -- 172.21.15.50:0/985355152 shutdown_connections 2024-05-08T18:04:45.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:45.802+0000 7f90ada4c640 1 --2- 172.21.15.50:0/985355152 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f909403d290 0x7f909403f750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:45.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:45.802+0000 7f90ada4c640 1 --2- 172.21.15.50:0/985355152 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f90a8115bc0 0x7f90a81a7a50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:04:45.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:45.802+0000 7f90ada4c640 1 -- 172.21.15.50:0/985355152 >> 172.21.15.50:0/985355152 conn(0x7f90a8070840 msgr2=0x7f90a810f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:04:45.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:45.804+0000 7f90ada4c640 1 -- 172.21.15.50:0/985355152 shutdown_connections 2024-05-08T18:04:45.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:04:45.806+0000 7f90ada4c640 1 -- 172.21.15.50:0/985355152 wait complete. 2024-05-08T18:04:45.829 DEBUG:tasks.rgw:Pools created 2024-05-08T18:04:45.829 INFO:tasks.rgw:Starting rgw... 2024-05-08T18:04:45.830 INFO:tasks.rgw:rgw client.0 config is {} 2024-05-08T18:04:45.830 INFO:tasks.rgw:Using beast as radosgw frontend 2024-05-08T18:04:45.830 DEBUG:teuthology.orchestra.run.smithi050:> sudo echo -n http://smithi050.front.sepia.ceph.com:80 | sudo tee /home/ubuntu/cephtest/url_file 2024-05-08T18:04:45.859 INFO:teuthology.orchestra.run.smithi050.stdout:http://smithi050.front.sepia.ceph.com:80 2024-05-08T18:04:45.860 DEBUG:teuthology.orchestra.run.smithi050:> sudo chown ceph /home/ubuntu/cephtest/url_file 2024-05-08T18:04:45.927 INFO:tasks.rgw.client.0:Restarting daemon 2024-05-08T18:04:45.927 DEBUG:teuthology.orchestra.run.smithi050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2024-05-08T18:04:45.970 INFO:tasks.rgw.client.0:Started 2024-05-08T18:04:45.970 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi050.front.sepia.ceph.com:80/ 2024-05-08T18:04:45.971 DEBUG:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:80/ 2024-05-08T18:04:46.002 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-08T18:04:46.003 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2024-05-08T18:04:46.003 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-05-08T18:04:46.004 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-05-08T18:04:46.004 INFO:teuthology.orchestra.run.smithi050.stderr:curl: (7) Failed to connect to smithi050.front.sepia.ceph.com port 80: Connection refused 2024-05-08T18:04:47.006 DEBUG:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:80/ 2024-05-08T18:04:47.030 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-08T18:04:47.031 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2024-05-08T18:04:47.033 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-05-08T18:04:47.033 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-05-08T18:04:47.033 INFO:teuthology.orchestra.run.smithi050.stderr:curl: (7) Failed to connect to smithi050.front.sepia.ceph.com port 80: Connection refused 2024-05-08T18:04:49.035 DEBUG:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:80/ 2024-05-08T18:04:49.059 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-08T18:04:49.059 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2024-05-08T18:04:49.061 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-05-08T18:04:49.061 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-05-08T18:04:49.062 INFO:teuthology.orchestra.run.smithi050.stderr:curl: (7) Failed to connect to smithi050.front.sepia.ceph.com port 80: Connection refused 2024-05-08T18:04:53.062 DEBUG:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:80/ 2024-05-08T18:04:53.088 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-08T18:04:53.088 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2024-05-08T18:04:53.089 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-05-08T18:04:53.090 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-05-08T18:04:53.090 INFO:teuthology.orchestra.run.smithi050.stderr:curl: (7) Failed to connect to smithi050.front.sepia.ceph.com port 80: Connection refused 2024-05-08T18:05:01.091 DEBUG:teuthology.orchestra.run.smithi050:> curl http://smithi050.front.sepia.ceph.com:80/ 2024-05-08T18:05:01.115 INFO:teuthology.orchestra.run.smithi050.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-05-08T18:05:01.116 INFO:teuthology.orchestra.run.smithi050.stderr: Dload Upload Total Spent Left Speed 2024-05-08T18:05:01.117 INFO:teuthology.orchestra.run.smithi050.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 93500 0 --:--:-- --:--:-- --:--:-- 93500 2024-05-08T18:05:01.118 INFO:teuthology.orchestra.run.smithi050.stdout:anonymous 2024-05-08T18:05:01.119 INFO:teuthology.run_tasks:Running task radosgw-admin... 2024-05-08T18:05:02.539 DEBUG:tasks.radosgw_admin:config is: {'client.0': None} 2024-05-08T18:05:02.540 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2024-05-08T18:05:02.540 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2024-05-08T18:05:02.540 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2024-05-08T18:05:02.591 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:02.591 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:02.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.594+0000 7f6258533800 1 Processor -- start 2024-05-08T18:05:02.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.594+0000 7f6258533800 1 -- start start 2024-05-08T18:05:02.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.594+0000 7f6258533800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x555621c22540 0x555621c22960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.594+0000 7f6258533800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x555621c22fe0 con 0x555621c237a0 2024-05-08T18:05:02.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.594+0000 7f6258533800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x555621ac3e20 con 0x555621c22540 2024-05-08T18:05:02.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.595+0000 7f62541ca640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x555621c22540 0x555621c22960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.595+0000 7f62541ca640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x555621c22540 0x555621c22960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57510/0 (socket says 172.21.15.50:57510) 2024-05-08T18:05:02.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.595+0000 7f62541ca640 1 -- 172.21.15.50:0/1362607706 learned_addr learned my addr 172.21.15.50:0/1362607706 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.595+0000 7f62539c9640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x555621c237a0 0x555621c23c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:52966/0 (socket says 172.21.15.50:52966) 2024-05-08T18:05:02.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.595+0000 7f62541ca640 1 -- 172.21.15.50:0/1362607706 >> v1:172.21.15.46:6789/0 conn(0x555621c237a0 legacy=0x555621c23c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.595+0000 7f62541ca640 1 -- 172.21.15.50:0/1362607706 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x555621c24360 con 0x555621c22540 2024-05-08T18:05:02.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.597+0000 7f62541ca640 1 --2- 172.21.15.50:0/1362607706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 0x555621c22960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f624400a6b0 tx=0x7f624401eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c0905ea98bc668 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.598+0000 7f62531c8640 1 -- 172.21.15.50:0/1362607706 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f624404ce60 con 0x555621c22540 2024-05-08T18:05:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.598+0000 7f62531c8640 1 -- 172.21.15.50:0/1362607706 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f624400a200 con 0x555621c22540 2024-05-08T18:05:02.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.598+0000 7f62531c8640 1 -- 172.21.15.50:0/1362607706 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6244008590 con 0x555621c22540 2024-05-08T18:05:02.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.598+0000 7f6258533800 1 -- 172.21.15.50:0/1362607706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 msgr2=0x555621c22960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.598+0000 7f6258533800 1 --2- 172.21.15.50:0/1362607706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 0x555621c22960 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f624400a6b0 tx=0x7f624401eed0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.598+0000 7f6258533800 1 -- 172.21.15.50:0/1362607706 shutdown_connections 2024-05-08T18:05:02.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.598+0000 7f6258533800 1 --2- 172.21.15.50:0/1362607706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 0x555621c22960 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.598+0000 7f6258533800 1 -- 172.21.15.50:0/1362607706 >> 172.21.15.50:0/1362607706 conn(0x555621c1db40 msgr2=0x555621c1ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:02.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.598+0000 7f6258533800 1 -- 172.21.15.50:0/1362607706 shutdown_connections 2024-05-08T18:05:02.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.598+0000 7f6258533800 1 -- 172.21.15.50:0/1362607706 wait complete. 2024-05-08T18:05:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.616+0000 7f6258533800 1 Processor -- start 2024-05-08T18:05:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.616+0000 7f6258533800 1 -- start start 2024-05-08T18:05:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.616+0000 7f6258533800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 0x555621d12a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.616+0000 7f6258533800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555621b6ea00 con 0x555621c22540 2024-05-08T18:05:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.617+0000 7f62541ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 0x555621d12a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.617+0000 7f62541ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 0x555621d12a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57518/0 (socket says 172.21.15.50:57518) 2024-05-08T18:05:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.617+0000 7f62541ca640 1 -- 172.21.15.50:0/967976417 learned_addr learned my addr 172.21.15.50:0/967976417 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.617+0000 7f62541ca640 1 -- 172.21.15.50:0/967976417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555621d13000 con 0x555621c22540 2024-05-08T18:05:02.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.617+0000 7f62541ca640 1 --2- 172.21.15.50:0/967976417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 0x555621d12a30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6244056040 tx=0x7f624404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b8ea93478062cb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.618+0000 7f614260c640 1 -- 172.21.15.50:0/967976417 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6244065070 con 0x555621c22540 2024-05-08T18:05:02.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.618+0000 7f614260c640 1 -- 172.21.15.50:0/967976417 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f624400a3b0 con 0x555621c22540 2024-05-08T18:05:02.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.618+0000 7f614260c640 1 -- 172.21.15.50:0/967976417 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6244008700 con 0x555621c22540 2024-05-08T18:05:02.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.618+0000 7f6258533800 1 -- 172.21.15.50:0/967976417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 msgr2=0x555621d12a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.618+0000 7f6258533800 1 --2- 172.21.15.50:0/967976417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 0x555621d12a30 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6244056040 tx=0x7f624404e000 comp rx=0 tx=0).stop 2024-05-08T18:05:02.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.618+0000 7f6258533800 1 -- 172.21.15.50:0/967976417 shutdown_connections 2024-05-08T18:05:02.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.618+0000 7f6258533800 1 --2- 172.21.15.50:0/967976417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 0x555621d12a30 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.618+0000 7f6258533800 1 -- 172.21.15.50:0/967976417 >> 172.21.15.50:0/967976417 conn(0x555621c1db40 msgr2=0x555621d0e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:02.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.618+0000 7f6258533800 1 -- 172.21.15.50:0/967976417 shutdown_connections 2024-05-08T18:05:02.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.619+0000 7f6258533800 1 -- 172.21.15.50:0/967976417 wait complete. 2024-05-08T18:05:02.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.619+0000 7f6258533800 1 Processor -- start 2024-05-08T18:05:02.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.619+0000 7f6258533800 1 -- start start 2024-05-08T18:05:02.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.619+0000 7f6258533800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 0x555621d1ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.619+0000 7f6258533800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555621b95c20 con 0x555621c22540 2024-05-08T18:05:02.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.619+0000 7f62541ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 0x555621d1ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.619+0000 7f62541ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 0x555621d1ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57532/0 (socket says 172.21.15.50:57532) 2024-05-08T18:05:02.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.619+0000 7f62541ca640 1 -- 172.21.15.50:0/4151661848 learned_addr learned my addr 172.21.15.50:0/4151661848 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.620+0000 7f62541ca640 1 -- 172.21.15.50:0/4151661848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555621d1ef90 con 0x555621c22540 2024-05-08T18:05:02.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.620+0000 7f62541ca640 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 0x555621d1ea50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f624400a580 tx=0x7f6244002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.620+0000 7f6140e09640 1 -- 172.21.15.50:0/4151661848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6244065070 con 0x555621c22540 2024-05-08T18:05:02.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.620+0000 7f6140e09640 1 -- 172.21.15.50:0/4151661848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6244060c10 con 0x555621c22540 2024-05-08T18:05:02.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.620+0000 7f6258533800 1 -- 172.21.15.50:0/4151661848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555621d1f220 con 0x555621c22540 2024-05-08T18:05:02.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.620+0000 7f6140e09640 1 -- 172.21.15.50:0/4151661848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f624405e420 con 0x555621c22540 2024-05-08T18:05:02.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.620+0000 7f6258533800 1 -- 172.21.15.50:0/4151661848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555621d21e80 con 0x555621c22540 2024-05-08T18:05:02.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.621+0000 7f6140e09640 1 -- 172.21.15.50:0/4151661848 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f624405e5c0 con 0x555621c22540 2024-05-08T18:05:02.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.622+0000 7f6140e09640 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f613c03bba0 0x7f613c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.622+0000 7f6140e09640 1 -- 172.21.15.50:0/4151661848 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f6244093a30 con 0x555621c22540 2024-05-08T18:05:02.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.622+0000 7f62539c9640 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f613c03bba0 0x7f613c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.622+0000 7f6258533800 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555621c237a0 0x555621d25c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.622+0000 7f6258533800 1 -- 172.21.15.50:0/4151661848 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d26260 con 0x555621c237a0 2024-05-08T18:05:02.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.622+0000 7f62549cb640 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555621c237a0 0x555621d25c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.622+0000 7f62539c9640 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f613c03bba0 0x7f613c03e060 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6248001e30 tx=0x7f6248001390 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.622+0000 7f62549cb640 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555621c237a0 0x555621d25c40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.623+0000 7f62549cb640 1 -- 172.21.15.50:0/4151661848 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621c237a0 2024-05-08T18:05:02.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.623+0000 7f6258533800 1 -- 172.21.15.50:0/4151661848 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d26ae0 con 0x555621c237a0 2024-05-08T18:05:02.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.623+0000 7f62549cb640 1 -- 172.21.15.50:0/4151661848 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621c237a0 2024-05-08T18:05:02.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.623+0000 7f6258533800 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555621d28430 0x555621d2a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.623+0000 7f6258533800 1 -- 172.21.15.50:0/4151661848 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d2b050 con 0x555621d28430 2024-05-08T18:05:02.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.623+0000 7f62541ca640 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555621d28430 0x555621d2a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.624+0000 7f62541ca640 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555621d28430 0x555621d2a8c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.624+0000 7f62541ca640 1 -- 172.21.15.50:0/4151661848 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f624406f020 con 0x555621d28430 2024-05-08T18:05:02.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.624+0000 7f6258533800 1 -- 172.21.15.50:0/4151661848 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d27330 con 0x555621c237a0 2024-05-08T18:05:02.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.625+0000 7f62549cb640 1 -- 172.21.15.50:0/4151661848 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621c237a0 2024-05-08T18:05:02.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.625+0000 7f6258533800 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555621d2c4b0 0x555621d2e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.625+0000 7f6258533800 1 -- 172.21.15.50:0/4151661848 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d2ef90 con 0x555621d2c4b0 2024-05-08T18:05:02.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.625+0000 7f62539c9640 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555621d2c4b0 0x555621d2e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.625+0000 7f62539c9640 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555621d2c4b0 0x555621d2e8a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.625+0000 7f62539c9640 1 -- 172.21.15.50:0/4151661848 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f624800bd50 con 0x555621d2c4b0 2024-05-08T18:05:02.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.626+0000 7f6258533800 1 Processor -- start 2024-05-08T18:05:02.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.626+0000 7f6258533800 1 -- start start 2024-05-08T18:05:02.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.626+0000 7f6258533800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d3d280 0x555621d3d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.626+0000 7f6258533800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555621b15330 con 0x555621d3d280 2024-05-08T18:05:02.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.626+0000 7f62541ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d3d280 0x555621d3d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.626+0000 7f62541ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d3d280 0x555621d3d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57534/0 (socket says 172.21.15.50:57534) 2024-05-08T18:05:02.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.626+0000 7f62541ca640 1 -- 172.21.15.50:0/2605288319 learned_addr learned my addr 172.21.15.50:0/2605288319 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.627+0000 7f62541ca640 1 -- 172.21.15.50:0/2605288319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555621d3dbe0 con 0x555621d3d280 2024-05-08T18:05:02.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.627+0000 7f62541ca640 1 --2- 172.21.15.50:0/2605288319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d3d280 0x555621d3d6a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6244091e80 tx=0x7f624405df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.627+0000 7f6139ffb640 1 -- 172.21.15.50:0/2605288319 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f624405ee10 con 0x555621d3d280 2024-05-08T18:05:02.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.627+0000 7f6139ffb640 1 -- 172.21.15.50:0/2605288319 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f624408acf0 con 0x555621d3d280 2024-05-08T18:05:02.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.627+0000 7f6258533800 1 -- 172.21.15.50:0/2605288319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555621d3e650 con 0x555621d3d280 2024-05-08T18:05:02.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.627+0000 7f6139ffb640 1 -- 172.21.15.50:0/2605288319 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f624408afd0 con 0x555621d3d280 2024-05-08T18:05:02.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.627+0000 7f6258533800 1 -- 172.21.15.50:0/2605288319 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555621d3e650 con 0x555621d3d280 2024-05-08T18:05:02.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.628+0000 7f6139ffb640 1 -- 172.21.15.50:0/2605288319 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6244068810 con 0x555621d3d280 2024-05-08T18:05:02.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.628+0000 7f6139ffb640 1 --2- 172.21.15.50:0/2605288319 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f612c03bba0 0x7f612c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.628+0000 7f6139ffb640 1 -- 172.21.15.50:0/2605288319 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f624407cc90 con 0x555621d3d280 2024-05-08T18:05:02.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.628+0000 7f62539c9640 1 --2- 172.21.15.50:0/2605288319 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f612c03bba0 0x7f612c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.629+0000 7f6258533800 1 Processor -- start 2024-05-08T18:05:02.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.629+0000 7f6258533800 1 -- start start 2024-05-08T18:05:02.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.629+0000 7f62539c9640 1 --2- 172.21.15.50:0/2605288319 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f612c03bba0 0x7f612c03e060 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x555621d23250 tx=0x7f624800f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.629+0000 7f6258533800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d4a260 0x555621d4a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.629+0000 7f6258533800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555621d13320 con 0x555621d4a260 2024-05-08T18:05:02.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.629+0000 7f62541ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d4a260 0x555621d4a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.629+0000 7f62541ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d4a260 0x555621d4a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57548/0 (socket says 172.21.15.50:57548) 2024-05-08T18:05:02.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.629+0000 7f62541ca640 1 -- 172.21.15.50:0/3530697046 learned_addr learned my addr 172.21.15.50:0/3530697046 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.630+0000 7f62541ca640 1 -- 172.21.15.50:0/3530697046 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555621d4ac50 con 0x555621d4a260 2024-05-08T18:05:02.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.630+0000 7f62541ca640 1 --2- 172.21.15.50:0/3530697046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d4a260 0x555621d4a680 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6244059fd0 tx=0x7f6244088a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce5112b1203ac26 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.630+0000 7f61337fe640 1 -- 172.21.15.50:0/3530697046 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6244088450 con 0x555621d4a260 2024-05-08T18:05:02.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.630+0000 7f61337fe640 1 -- 172.21.15.50:0/3530697046 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6244069410 con 0x555621d4a260 2024-05-08T18:05:02.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.630+0000 7f61337fe640 1 -- 172.21.15.50:0/3530697046 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f624406bec0 con 0x555621d4a260 2024-05-08T18:05:02.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.630+0000 7f6258533800 1 -- 172.21.15.50:0/3530697046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d4a260 msgr2=0x555621d4a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.630+0000 7f6258533800 1 --2- 172.21.15.50:0/3530697046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d4a260 0x555621d4a680 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6244059fd0 tx=0x7f6244088a00 comp rx=0 tx=0).stop 2024-05-08T18:05:02.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.630+0000 7f6258533800 1 -- 172.21.15.50:0/3530697046 shutdown_connections 2024-05-08T18:05:02.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.630+0000 7f6258533800 1 --2- 172.21.15.50:0/3530697046 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d4a260 0x555621d4a680 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.630+0000 7f6258533800 1 -- 172.21.15.50:0/3530697046 >> 172.21.15.50:0/3530697046 conn(0x555621d465c0 msgr2=0x555621d44620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:02.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.631+0000 7f6258533800 1 -- 172.21.15.50:0/3530697046 shutdown_connections 2024-05-08T18:05:02.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.631+0000 7f6258533800 1 -- 172.21.15.50:0/3530697046 wait complete. 2024-05-08T18:05:02.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.631+0000 7f6258533800 1 Processor -- start 2024-05-08T18:05:02.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.631+0000 7f6258533800 1 -- start start 2024-05-08T18:05:02.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.631+0000 7f6258533800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d4a260 0x555621d51650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.631+0000 7f6258533800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555621d4b300 con 0x555621d4a260 2024-05-08T18:05:02.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.631+0000 7f62541ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d4a260 0x555621d51650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.631+0000 7f62541ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d4a260 0x555621d51650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57560/0 (socket says 172.21.15.50:57560) 2024-05-08T18:05:02.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.631+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 learned_addr learned my addr 172.21.15.50:0/1560457818 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.632+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555621d51b90 con 0x555621d4a260 2024-05-08T18:05:02.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.632+0000 7f62541ca640 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d4a260 0x555621d51650 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f62440695c0 tx=0x7f624408abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.632+0000 7f6131ffb640 1 -- 172.21.15.50:0/1560457818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62440b0070 con 0x555621d4a260 2024-05-08T18:05:02.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.632+0000 7f6131ffb640 1 -- 172.21.15.50:0/1560457818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f624408a550 con 0x555621d4a260 2024-05-08T18:05:02.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.632+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555621d51b90 con 0x555621d4a260 2024-05-08T18:05:02.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.632+0000 7f6131ffb640 1 -- 172.21.15.50:0/1560457818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6244088450 con 0x555621d4a260 2024-05-08T18:05:02.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.632+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555621d51b90 con 0x555621d4a260 2024-05-08T18:05:02.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.633+0000 7f6131ffb640 1 -- 172.21.15.50:0/1560457818 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f62440b3070 con 0x555621d4a260 2024-05-08T18:05:02.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.633+0000 7f6131ffb640 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f612403bb50 0x7f612403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.633+0000 7f6131ffb640 1 -- 172.21.15.50:0/1560457818 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f62440b6ca0 con 0x555621d4a260 2024-05-08T18:05:02.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.633+0000 7f62539c9640 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f612403bb50 0x7f612403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.633+0000 7f6258533800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.634+0000 7f6258533800 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555621d5cbf0 0x555621d5f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.634+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d5f800 con 0x555621d5cbf0 2024-05-08T18:05:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.634+0000 7f62549cb640 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555621d5cbf0 0x555621d5f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.634+0000 7f62539c9640 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f612403bb50 0x7f612403e010 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x555621d2f390 tx=0x7f6248010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.634+0000 7f62549cb640 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555621d5cbf0 0x555621d5f0b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.634+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621d5cbf0 2024-05-08T18:05:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.634+0000 7f6258533800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.634+0000 7f6258533800 20 realm 2024-05-08T18:05:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.634+0000 7f6258533800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.634+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d5fc10 con 0x555621d5cbf0 2024-05-08T18:05:02.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.635+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621d5cbf0 2024-05-08T18:05:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.635+0000 7f6258533800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.635+0000 7f6258533800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.635+0000 7f6258533800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.635+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d604e0 con 0x555621d5cbf0 2024-05-08T18:05:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.635+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621d5cbf0 2024-05-08T18:05:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.635+0000 7f6258533800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.635+0000 7f6258533800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.635+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d5cbf0 2024-05-08T18:05:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.635+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621d5cbf0 2024-05-08T18:05:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.635+0000 7f6258533800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.635+0000 7f6258533800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.635+0000 7f6258533800 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555621d61370 0x555621d63760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.635+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d63ec0 con 0x555621d61370 2024-05-08T18:05:02.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.636+0000 7f62541ca640 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555621d61370 0x555621d63760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.636+0000 7f62541ca640 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555621d61370 0x555621d63760 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.636+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.636+0000 7f6258533800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:02.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.636+0000 7f6258533800 20 searching for the correct realm 2024-05-08T18:05:02.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.637+0000 7f6258533800 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555621d65f70 0x555621d68430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.637+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d68b00 con 0x555621d65f70 2024-05-08T18:05:02.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.637+0000 7f62539c9640 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555621d65f70 0x555621d68430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.637+0000 7f62539c9640 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555621d65f70 0x555621d68430 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.637+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.637+0000 7f62539c9640 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f624800f070 0x7f624802e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.637+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d68b00 con 0x7f624800f070 2024-05-08T18:05:02.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.637+0000 7f62549cb640 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f624800f070 0x7f624802e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.638+0000 7f62549cb640 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f624800f070 0x7f624802e910 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.638+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c00a4a0 con 0x7f624800f070 2024-05-08T18:05:02.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.638+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d61370 2024-05-08T18:05:02.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.638+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.638+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d5cbf0 2024-05-08T18:05:02.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.639+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621d5cbf0 2024-05-08T18:05:02.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.639+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d5cbf0 2024-05-08T18:05:02.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.639+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621d5cbf0 2024-05-08T18:05:02.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.639+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d61370 2024-05-08T18:05:02.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.639+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.639+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d61370 2024-05-08T18:05:02.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.640+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v20'1 uv1 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.640+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x7f624800f070 2024-05-08T18:05:02.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.640+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c00a4a0 con 0x7f624800f070 2024-05-08T18:05:02.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.640+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d65f70 2024-05-08T18:05:02.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.640+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.640+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d5cbf0 2024-05-08T18:05:02.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.640+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621d5cbf0 2024-05-08T18:05:02.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.640+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d61370 2024-05-08T18:05:02.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.641+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.641+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d5cbf0 2024-05-08T18:05:02.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.641+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621d5cbf0 2024-05-08T18:05:02.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.641+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x7f624800f070 2024-05-08T18:05:02.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.641+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f624c00a4a0 con 0x7f624800f070 2024-05-08T18:05:02.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.641+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d65f70 2024-05-08T18:05:02.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.642+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.642+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d65f70 2024-05-08T18:05:02.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.642+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.642+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d61370 2024-05-08T18:05:02.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.642+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.642+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d65f70 2024-05-08T18:05:02.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.643+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.643+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d65f70 2024-05-08T18:05:02.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.643+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.643+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d5cbf0 2024-05-08T18:05:02.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.643+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621d5cbf0 2024-05-08T18:05:02.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.643+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d5cbf0 2024-05-08T18:05:02.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.643+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621d5cbf0 2024-05-08T18:05:02.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.643+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d61370 2024-05-08T18:05:02.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.644+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.644+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d65f70 2024-05-08T18:05:02.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.644+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.644+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x7f624800f070 2024-05-08T18:05:02.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.644+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c00a4a0 con 0x7f624800f070 2024-05-08T18:05:02.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.644+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d5cbf0 2024-05-08T18:05:02.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.645+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621d5cbf0 2024-05-08T18:05:02.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.645+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d61370 2024-05-08T18:05:02.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.645+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.645+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d65f70 2024-05-08T18:05:02.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.645+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.645+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d61370 2024-05-08T18:05:02.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.646+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.646+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x7f624800f070 2024-05-08T18:05:02.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.646+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c00a4a0 con 0x7f624800f070 2024-05-08T18:05:02.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.646+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d65f70 2024-05-08T18:05:02.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.646+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.646+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d61370 2024-05-08T18:05:02.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.647+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.647+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d5cbf0 2024-05-08T18:05:02.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.647+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621d5cbf0 2024-05-08T18:05:02.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.647+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x7f624800f070 2024-05-08T18:05:02.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.647+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c00a4a0 con 0x7f624800f070 2024-05-08T18:05:02.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.647+0000 7f6258533800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:02.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.647+0000 7f6258533800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:02.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.647+0000 7f6258533800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:02.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.647+0000 7f6258533800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:02.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.647+0000 7f6258533800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:02.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.647+0000 7f6258533800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:02.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.647+0000 7f6258533800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.647+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d64870 con 0x555621d5cbf0 2024-05-08T18:05:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621d5cbf0 2024-05-08T18:05:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f6258533800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f6258533800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d64870 con 0x555621d5cbf0 2024-05-08T18:05:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f624c00a4a0 con 0x555621d5cbf0 2024-05-08T18:05:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f6258533800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f6258533800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d64840 con 0x555621d65f70 2024-05-08T18:05:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f6258533800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f6258533800 20 zone default found 2024-05-08T18:05:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f6258533800 4 Realm: () 2024-05-08T18:05:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f6258533800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:02.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f6258533800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f6258533800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f6258533800 20 zonegroup default 2024-05-08T18:05:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f6258533800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.648+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d65110 con 0x555621d61370 2024-05-08T18:05:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.649+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.649+0000 7f6258533800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.649+0000 7f6258533800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.649+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d6a960 con 0x555621d65f70 2024-05-08T18:05:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.649+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.649+0000 7f6258533800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.649+0000 7f6258533800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.649+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d6ad00 con 0x555621d65f70 2024-05-08T18:05:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.649+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.649+0000 7f6258533800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.649+0000 7f6258533800 20 started sync module instance, tier type = 2024-05-08T18:05:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.649+0000 7f6258533800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.649+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d69e00 con 0x7f624800f070 2024-05-08T18:05:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.651+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f624c00a4a0 con 0x7f624800f070 2024-05-08T18:05:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.651+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93828423263056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d69e00 con 0x7f624800f070 2024-05-08T18:05:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.651+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d6b720 con 0x555621d5cbf0 2024-05-08T18:05:02.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.652+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93828423263056] v27'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f624c040040 con 0x7f624800f070 2024-05-08T18:05:02.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.652+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.652+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93828423266832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d69e00 con 0x555621d5cbf0 2024-05-08T18:05:02.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.652+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d6c190 con 0x555621d65f70 2024-05-08T18:05:02.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.654+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.654+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93828423269504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d6ca60 con 0x555621d65f70 2024-05-08T18:05:02.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.654+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d6d510 con 0x7f624800f070 2024-05-08T18:05:02.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.654+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93828423266832] v27'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.655+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93828423269504] v27'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.655+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f624c040040 con 0x7f624800f070 2024-05-08T18:05:02.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.655+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93828423271856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d6d020 con 0x7f624800f070 2024-05-08T18:05:02.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.655+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d6de20 con 0x555621d5cbf0 2024-05-08T18:05:02.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.656+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93828423271856] v27'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f624c040040 con 0x7f624800f070 2024-05-08T18:05:02.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.656+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.656+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93828423260496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d6dd90 con 0x555621d5cbf0 2024-05-08T18:05:02.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.656+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d6e830 con 0x555621d61370 2024-05-08T18:05:02.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.657+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.657+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93828423279488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d6f080 con 0x555621d61370 2024-05-08T18:05:02.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.657+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d61370 2024-05-08T18:05:02.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.658+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93828423260496] v27'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.659+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.659+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 93828423279488] v27'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.659+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93828423281616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d6fec0 con 0x555621d61370 2024-05-08T18:05:02.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.659+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d70710 con 0x555621d61370 2024-05-08T18:05:02.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.660+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93828423281616] v27'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.660+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.660+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93828423286464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d70790 con 0x555621d61370 2024-05-08T18:05:02.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.660+0000 7f6258533800 20 add_watcher() i=0 2024-05-08T18:05:02.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.660+0000 7f6258533800 20 add_watcher() i=1 2024-05-08T18:05:02.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.660+0000 7f6258533800 20 add_watcher() i=2 2024-05-08T18:05:02.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.660+0000 7f6258533800 20 add_watcher() i=3 2024-05-08T18:05:02.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.660+0000 7f6258533800 20 add_watcher() i=4 2024-05-08T18:05:02.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.660+0000 7f6258533800 20 add_watcher() i=5 2024-05-08T18:05:02.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.660+0000 7f6258533800 20 add_watcher() i=6 2024-05-08T18:05:02.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.661+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93828423286464] v27'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.661+0000 7f6258533800 20 add_watcher() i=7 2024-05-08T18:05:02.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.661+0000 7f6258533800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:02.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.661+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555621d73ac0 con 0x555621d5cbf0 2024-05-08T18:05:02.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.661+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.661+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93828423287888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d70d20 con 0x555621d5cbf0 2024-05-08T18:05:02.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.662+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93828423287888] v27'24 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.663+0000 7f610a7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:02.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.663+0000 7f6258533800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:02.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.663+0000 7f6258533800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:02.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.663+0000 7f6258533800 5 note: GC not initialized 2024-05-08T18:05:02.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.663+0000 7f61097e2640 20 reqs_thread_entry: start 2024-05-08T18:05:02.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.663+0000 7f6258533800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:02.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.663+0000 7f6258533800 20 Filter name: none 2024-05-08T18:05:02.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.663+0000 7f607affd640 20 reqs_thread_entry: start 2024-05-08T18:05:02.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.663+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:62 8.0 8:00000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621da6820 con 0x555621d65f70 2024-05-08T18:05:02.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.664+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(62 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.664+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:63 8.10 8:08000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621da6820 con 0x555621d65f70 2024-05-08T18:05:02.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.664+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(63 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.664+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 8.8 8:10000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621da6820 con 0x555621d5cbf0 2024-05-08T18:05:02.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.664+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(64 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.664+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 8.18 8:18000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d5cbf0 2024-05-08T18:05:02.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.664+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.664+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 8.4 8:20000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x7f624800f070 2024-05-08T18:05:02.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.665+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c040040 con 0x7f624800f070 2024-05-08T18:05:02.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.665+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 8.14 8:28000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d65f70 2024-05-08T18:05:02.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.665+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(67 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.665+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 8.c 8:30000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d5cbf0 2024-05-08T18:05:02.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.665+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.665+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 8.1c 8:38000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d65f70 2024-05-08T18:05:02.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.665+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(69 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.665+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 8.2 8:40000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x7f624800f070 2024-05-08T18:05:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.666+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c040040 con 0x7f624800f070 2024-05-08T18:05:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.666+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 8.12 8:48000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d65f70 2024-05-08T18:05:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.666+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(71 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.666+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 8.a 8:50000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d5cbf0 2024-05-08T18:05:02.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.666+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(72 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.666+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 8.1a 8:58000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d61370 2024-05-08T18:05:02.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.667+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(73 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.667+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 8.6 8:60000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d61370 2024-05-08T18:05:02.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.667+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(74 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.667+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:75 8.16 8:68000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d65f70 2024-05-08T18:05:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.667+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(75 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.667+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 8.e 8:70000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d61370 2024-05-08T18:05:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.668+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(76 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.668+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 8.1e 8:78000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d61370 2024-05-08T18:05:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.668+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(77 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.668+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 8.1 8:80000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d61370 2024-05-08T18:05:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.668+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(78 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.668+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 8.11 8:88000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d65f70 2024-05-08T18:05:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.668+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(79 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.668+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:80 8.9 8:90000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x7f624800f070 2024-05-08T18:05:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.669+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(80 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c040040 con 0x7f624800f070 2024-05-08T18:05:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.669+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 8.19 8:98000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d5cbf0 2024-05-08T18:05:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.669+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(81 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.669+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:82 8.5 8:a0000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d5cbf0 2024-05-08T18:05:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.669+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(82 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.669+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:83 8.15 8:a8000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d65f70 2024-05-08T18:05:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.669+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(83 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.669+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:84 8.d 8:b0000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x7f624800f070 2024-05-08T18:05:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.670+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(84 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c040040 con 0x7f624800f070 2024-05-08T18:05:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.670+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:85 8.1d 8:b8000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d61370 2024-05-08T18:05:02.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.670+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(85 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.670+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:86 8.3 8:c0000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d5cbf0 2024-05-08T18:05:02.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.670+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(86 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.670+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:87 8.13 8:c8000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d61370 2024-05-08T18:05:02.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.671+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(87 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.671+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:88 8.b 8:d0000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d65f70 2024-05-08T18:05:02.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.671+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(88 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.671+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:89 8.1b 8:d8000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x7f624800f070 2024-05-08T18:05:02.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.671+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(89 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c040040 con 0x7f624800f070 2024-05-08T18:05:02.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.671+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 8.7 8:e0000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d61370 2024-05-08T18:05:02.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.672+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(90 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.672+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:91 8.17 8:e8000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d5cbf0 2024-05-08T18:05:02.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.672+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(91 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.672+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:92 8.f 8:f0000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x555621d65f70 2024-05-08T18:05:02.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.672+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(92 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.672+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 8.1f 8:f8000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555621d6faa0 con 0x7f624800f070 2024-05-08T18:05:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.672+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(93 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f624c040040 con 0x7f624800f070 2024-05-08T18:05:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.673+0000 7f610a7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.673+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93828423287888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d5cbf0 2024-05-08T18:05:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.675+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 93828423287888] v27'25 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.675+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93828423263056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x7f624800f070 2024-05-08T18:05:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.676+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 93828423263056] v27'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f624c040040 con 0x7f624800f070 2024-05-08T18:05:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.676+0000 7f6258533800 20 remove_watcher() i=0 2024-05-08T18:05:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.676+0000 7f6258533800 2 removed watcher, disabling cache 2024-05-08T18:05:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.676+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93828423266832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d5cbf0 2024-05-08T18:05:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.677+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 93828423266832] v27'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.677+0000 7f6258533800 20 remove_watcher() i=1 2024-05-08T18:05:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.677+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93828423269504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d60880 con 0x555621d65f70 2024-05-08T18:05:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.678+0000 7f62539c9640 1 -- 172.21.15.50:0/1560457818 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 93828423269504] v27'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6248037040 con 0x555621d65f70 2024-05-08T18:05:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.678+0000 7f6258533800 20 remove_watcher() i=2 2024-05-08T18:05:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.678+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93828423271856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d70450 con 0x7f624800f070 2024-05-08T18:05:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.679+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 93828423271856] v27'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f624c040040 con 0x7f624800f070 2024-05-08T18:05:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.679+0000 7f6258533800 20 remove_watcher() i=3 2024-05-08T18:05:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.679+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93828423260496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d70450 con 0x555621d5cbf0 2024-05-08T18:05:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.680+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 93828423260496] v27'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f624c040040 con 0x555621d5cbf0 2024-05-08T18:05:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.680+0000 7f6258533800 20 remove_watcher() i=4 2024-05-08T18:05:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.680+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93828423279488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d70450 con 0x555621d61370 2024-05-08T18:05:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.682+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 93828423279488] v27'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.682+0000 7f6258533800 20 remove_watcher() i=5 2024-05-08T18:05:02.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.682+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93828423281616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d6e380 con 0x555621d61370 2024-05-08T18:05:02.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.683+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 93828423281616] v27'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.683+0000 7f6258533800 20 remove_watcher() i=6 2024-05-08T18:05:02.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.683+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93828423286464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555621d6e380 con 0x555621d61370 2024-05-08T18:05:02.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f62541ca640 1 -- 172.21.15.50:0/1560457818 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 93828423286464] v27'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62440b2030 con 0x555621d61370 2024-05-08T18:05:02.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 20 remove_watcher() i=7 2024-05-08T18:05:02.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555621d61370 msgr2=0x555621d63760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555621d61370 0x555621d63760 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555621d5cbf0 msgr2=0x555621d5f0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555621d5cbf0 0x555621d5f0b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f624800f070 msgr2=0x7f624802e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f624800f070 0x7f624802e910 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555621d65f70 msgr2=0x555621d68430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555621d65f70 0x555621d68430 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f612403bb50 msgr2=0x7f612403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f612403bb50 0x7f612403e010 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x555621d2f390 tx=0x7f6248010000 comp rx=0 tx=0).stop 2024-05-08T18:05:02.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d4a260 msgr2=0x555621d51650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 1 --2- 172.21.15.50:0/1560457818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d4a260 0x555621d51650 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f62440695c0 tx=0x7f624408abd0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 reap_dead start 2024-05-08T18:05:02.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f62549cb640 1 -- 172.21.15.50:0/1560457818 reap_dead start 2024-05-08T18:05:02.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 shutdown_connections 2024-05-08T18:05:02.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 >> 172.21.15.50:0/1560457818 conn(0x555621d465c0 msgr2=0x555621d43c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:02.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 shutdown_connections 2024-05-08T18:05:02.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.684+0000 7f6258533800 1 -- 172.21.15.50:0/1560457818 wait complete. 2024-05-08T18:05:02.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 -- 172.21.15.50:0/2605288319 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f612c03bba0 msgr2=0x7f612c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 --2- 172.21.15.50:0/2605288319 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f612c03bba0 0x7f612c03e060 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x555621d23250 tx=0x7f624800f000 comp rx=0 tx=0).stop 2024-05-08T18:05:02.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 -- 172.21.15.50:0/2605288319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d3d280 msgr2=0x555621d3d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 --2- 172.21.15.50:0/2605288319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d3d280 0x555621d3d6a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6244091e80 tx=0x7f624405df70 comp rx=0 tx=0).stop 2024-05-08T18:05:02.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 -- 172.21.15.50:0/2605288319 shutdown_connections 2024-05-08T18:05:02.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 --2- 172.21.15.50:0/2605288319 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f612c03bba0 0x7f612c03e060 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 --2- 172.21.15.50:0/2605288319 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621d3d280 0x555621d3d6a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 -- 172.21.15.50:0/2605288319 >> 172.21.15.50:0/2605288319 conn(0x555621d31320 msgr2=0x555621d31720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:02.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 -- 172.21.15.50:0/2605288319 shutdown_connections 2024-05-08T18:05:02.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 -- 172.21.15.50:0/2605288319 wait complete. 2024-05-08T18:05:02.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 -- 172.21.15.50:0/4151661848 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555621d28430 msgr2=0x555621d2a8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555621d28430 0x555621d2a8c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 -- 172.21.15.50:0/4151661848 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555621c237a0 msgr2=0x555621d25c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555621c237a0 0x555621d25c40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 -- 172.21.15.50:0/4151661848 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555621d2c4b0 msgr2=0x555621d2e8a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555621d2c4b0 0x555621d2e8a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 -- 172.21.15.50:0/4151661848 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f613c03bba0 msgr2=0x7f613c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f613c03bba0 0x7f613c03e060 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6248001e30 tx=0x7f6248001390 comp rx=0 tx=0).stop 2024-05-08T18:05:02.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 -- 172.21.15.50:0/4151661848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 msgr2=0x555621d1ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 --2- 172.21.15.50:0/4151661848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555621c22540 0x555621d1ea50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f624400a580 tx=0x7f6244002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f62549cb640 1 -- 172.21.15.50:0/4151661848 reap_dead start 2024-05-08T18:05:02.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 -- 172.21.15.50:0/4151661848 shutdown_connections 2024-05-08T18:05:02.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 -- 172.21.15.50:0/4151661848 >> 172.21.15.50:0/4151661848 conn(0x555621c1db40 msgr2=0x555621d0cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:02.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 -- 172.21.15.50:0/4151661848 shutdown_connections 2024-05-08T18:05:02.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.685+0000 7f6258533800 1 -- 172.21.15.50:0/4151661848 wait complete. 2024-05-08T18:05:02.710 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-08T18:05:02.714 DEBUG:tasks.util.rgw: json result: [] 2024-05-08T18:05:02.714 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud'] 2024-05-08T18:05:02.714 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud'] 2024-05-08T18:05:02.714 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud 2024-05-08T18:05:02.760 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:02.761 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:02.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.763+0000 7f3d1e20a800 1 Processor -- start 2024-05-08T18:05:02.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.763+0000 7f3d1e20a800 1 -- start start 2024-05-08T18:05:02.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.764+0000 7f3d1e20a800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55a5ee763cc0 0x55a5ee7f8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.764+0000 7f3d1e20a800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a5ee764200 con 0x55a5ee765050 2024-05-08T18:05:02.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.764+0000 7f3d1e20a800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55a5ee68fe20 con 0x55a5ee763cc0 2024-05-08T18:05:02.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.764+0000 7f3d1b501640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55a5ee763cc0 0x55a5ee7f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.764+0000 7f3d1bd02640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55a5ee765050 0x55a5ee7635b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:52974/0 (socket says 172.21.15.50:52974) 2024-05-08T18:05:02.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.764+0000 7f3d1bd02640 1 -- 172.21.15.50:0/1989262826 learned_addr learned my addr 172.21.15.50:0/1989262826 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:02.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.764+0000 7f3d1b501640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55a5ee763cc0 0x55a5ee7f8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57574/0 (socket says 172.21.15.50:57574) 2024-05-08T18:05:02.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.764+0000 7f3d1b501640 1 -- 172.21.15.50:0/1989262826 >> v1:172.21.15.46:6789/0 conn(0x55a5ee765050 legacy=0x55a5ee7635b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.764+0000 7f3d1b501640 1 -- 172.21.15.50:0/1989262826 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ee7643d0 con 0x55a5ee763cc0 2024-05-08T18:05:02.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.766+0000 7f3d1b501640 1 --2- 172.21.15.50:0/1989262826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee763cc0 0x55a5ee7f8590 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400a470 tx=0x7f3d0401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de93c81bf62c662 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.766+0000 7f3d1ad00640 1 -- 172.21.15.50:0/1989262826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d04057070 con 0x55a5ee763cc0 2024-05-08T18:05:02.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.767+0000 7f3d1ad00640 1 -- 172.21.15.50:0/1989262826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d04052570 con 0x55a5ee763cc0 2024-05-08T18:05:02.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.767+0000 7f3d1ad00640 1 -- 172.21.15.50:0/1989262826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d04052870 con 0x55a5ee763cc0 2024-05-08T18:05:02.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.767+0000 7f3d1e20a800 1 -- 172.21.15.50:0/1989262826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee763cc0 msgr2=0x55a5ee7f8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.767+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/1989262826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee763cc0 0x55a5ee7f8590 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400a470 tx=0x7f3d0401ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:02.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.767+0000 7f3d1e20a800 1 -- 172.21.15.50:0/1989262826 shutdown_connections 2024-05-08T18:05:02.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.767+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/1989262826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee763cc0 0x55a5ee7f8590 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.767+0000 7f3d1e20a800 1 -- 172.21.15.50:0/1989262826 >> 172.21.15.50:0/1989262826 conn(0x55a5ee7eac70 msgr2=0x55a5ee7ed0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:02.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.767+0000 7f3d1e20a800 1 -- 172.21.15.50:0/1989262826 shutdown_connections 2024-05-08T18:05:02.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.767+0000 7f3d1e20a800 1 -- 172.21.15.50:0/1989262826 wait complete. 2024-05-08T18:05:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.782+0000 7f3d1e20a800 1 Processor -- start 2024-05-08T18:05:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.782+0000 7f3d1e20a800 1 -- start start 2024-05-08T18:05:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.782+0000 7f3d1e20a800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee763cc0 0x55a5ee8e2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.782+0000 7f3d1e20a800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a5ee73aa10 con 0x55a5ee763cc0 2024-05-08T18:05:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.783+0000 7f3d1bd02640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee763cc0 0x55a5ee8e2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.783+0000 7f3d1bd02640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee763cc0 0x55a5ee8e2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57580/0 (socket says 172.21.15.50:57580) 2024-05-08T18:05:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.783+0000 7f3d1bd02640 1 -- 172.21.15.50:0/2799280058 learned_addr learned my addr 172.21.15.50:0/2799280058 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.783+0000 7f3d1bd02640 1 -- 172.21.15.50:0/2799280058 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ee8e32f0 con 0x55a5ee763cc0 2024-05-08T18:05:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.783+0000 7f3d1bd02640 1 --2- 172.21.15.50:0/2799280058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee763cc0 0x55a5ee8e2d20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c001e40 tx=0x7f3d0c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ef4ebd1679dfff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.783+0000 7f3c09e0b640 1 -- 172.21.15.50:0/2799280058 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d0c012020 con 0x55a5ee763cc0 2024-05-08T18:05:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.784+0000 7f3c09e0b640 1 -- 172.21.15.50:0/2799280058 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d0c00e5a0 con 0x55a5ee763cc0 2024-05-08T18:05:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.784+0000 7f3c09e0b640 1 -- 172.21.15.50:0/2799280058 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d0c00e8c0 con 0x55a5ee763cc0 2024-05-08T18:05:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.784+0000 7f3d1e20a800 1 -- 172.21.15.50:0/2799280058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee763cc0 msgr2=0x55a5ee8e2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.784+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/2799280058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee763cc0 0x55a5ee8e2d20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c001e40 tx=0x7f3d0c001e70 comp rx=0 tx=0).stop 2024-05-08T18:05:02.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.784+0000 7f3d1e20a800 1 -- 172.21.15.50:0/2799280058 shutdown_connections 2024-05-08T18:05:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.784+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/2799280058 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee763cc0 0x55a5ee8e2d20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.784+0000 7f3d1e20a800 1 -- 172.21.15.50:0/2799280058 >> 172.21.15.50:0/2799280058 conn(0x55a5ee7eac70 msgr2=0x55a5ee8de640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.784+0000 7f3d1e20a800 1 -- 172.21.15.50:0/2799280058 shutdown_connections 2024-05-08T18:05:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.784+0000 7f3d1e20a800 1 -- 172.21.15.50:0/2799280058 wait complete. 2024-05-08T18:05:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.785+0000 7f3d1e20a800 1 Processor -- start 2024-05-08T18:05:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.785+0000 7f3d1e20a800 1 -- start start 2024-05-08T18:05:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.785+0000 7f3d1e20a800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee765050 0x55a5ee8eab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.785+0000 7f3d1e20a800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a5ee7f8da0 con 0x55a5ee765050 2024-05-08T18:05:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.785+0000 7f3d1bd02640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee765050 0x55a5ee8eab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.785+0000 7f3d1bd02640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee765050 0x55a5ee8eab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57586/0 (socket says 172.21.15.50:57586) 2024-05-08T18:05:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.785+0000 7f3d1bd02640 1 -- 172.21.15.50:0/250826118 learned_addr learned my addr 172.21.15.50:0/250826118 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.785+0000 7f3d1bd02640 1 -- 172.21.15.50:0/250826118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ee8eb0a0 con 0x55a5ee765050 2024-05-08T18:05:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.786+0000 7f3d1bd02640 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee765050 0x55a5ee8eab60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c000c00 tx=0x7f3d0c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.786+0000 7f3c03fff640 1 -- 172.21.15.50:0/250826118 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d0c012030 con 0x55a5ee765050 2024-05-08T18:05:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.786+0000 7f3c03fff640 1 -- 172.21.15.50:0/250826118 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d0c0014a0 con 0x55a5ee765050 2024-05-08T18:05:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.786+0000 7f3d1e20a800 1 -- 172.21.15.50:0/250826118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a5ee8eb330 con 0x55a5ee765050 2024-05-08T18:05:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.786+0000 7f3c03fff640 1 -- 172.21.15.50:0/250826118 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d0c01a700 con 0x55a5ee765050 2024-05-08T18:05:02.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.786+0000 7f3d1e20a800 1 -- 172.21.15.50:0/250826118 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5ee8edf40 con 0x55a5ee765050 2024-05-08T18:05:02.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.787+0000 7f3c03fff640 1 -- 172.21.15.50:0/250826118 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3d0c015020 con 0x55a5ee765050 2024-05-08T18:05:02.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.787+0000 7f3c03fff640 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3c0403bba0 0x7f3c0403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.787+0000 7f3c03fff640 1 -- 172.21.15.50:0/250826118 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f3d0c020070 con 0x55a5ee765050 2024-05-08T18:05:02.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.787+0000 7f3d1b501640 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3c0403bba0 0x7f3c0403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.787+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a5ee8efcf0 0x55a5ee8f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.787+0000 7f3d1e20a800 1 -- 172.21.15.50:0/250826118 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee8f2620 con 0x55a5ee8efcf0 2024-05-08T18:05:02.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.788+0000 7f3d1c503640 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a5ee8efcf0 0x55a5ee8f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.788+0000 7f3d1b501640 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3c0403bba0 0x7f3c0403e060 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3d04054000 tx=0x7f3d0404e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.788+0000 7f3d1c503640 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a5ee8efcf0 0x55a5ee8f20e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.788+0000 7f3d1c503640 1 -- 172.21.15.50:0/250826118 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee8efcf0 2024-05-08T18:05:02.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.788+0000 7f3d1e20a800 1 -- 172.21.15.50:0/250826118 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee8f2a70 con 0x55a5ee8efcf0 2024-05-08T18:05:02.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.788+0000 7f3d1c503640 1 -- 172.21.15.50:0/250826118 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee8efcf0 2024-05-08T18:05:02.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.788+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a5ee8f4880 0x55a5ee8f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.788+0000 7f3d1e20a800 1 -- 172.21.15.50:0/250826118 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee8f7360 con 0x55a5ee8f4880 2024-05-08T18:05:02.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.789+0000 7f3d1bd02640 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a5ee8f4880 0x55a5ee8f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.789+0000 7f3d1bd02640 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a5ee8f4880 0x55a5ee8f6c70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.789+0000 7f3d1bd02640 1 -- 172.21.15.50:0/250826118 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f3d0c018cd0 con 0x55a5ee8f4880 2024-05-08T18:05:02.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.790+0000 7f3d1e20a800 1 -- 172.21.15.50:0/250826118 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee8f3720 con 0x55a5ee8efcf0 2024-05-08T18:05:02.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.790+0000 7f3d1c503640 1 -- 172.21.15.50:0/250826118 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee8efcf0 2024-05-08T18:05:02.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.790+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a5ee8f87c0 0x55a5ee8fabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.790+0000 7f3d1e20a800 1 -- 172.21.15.50:0/250826118 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee8fb2b0 con 0x55a5ee8f87c0 2024-05-08T18:05:02.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.790+0000 7f3d1b501640 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a5ee8f87c0 0x55a5ee8fabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.790+0000 7f3d1b501640 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a5ee8f87c0 0x55a5ee8fabb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.791+0000 7f3d1b501640 1 -- 172.21.15.50:0/250826118 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3d04052a40 con 0x55a5ee8f87c0 2024-05-08T18:05:02.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.791+0000 7f3d1e20a800 1 Processor -- start 2024-05-08T18:05:02.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.791+0000 7f3d1e20a800 1 -- start start 2024-05-08T18:05:02.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.791+0000 7f3d1e20a800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee909620 0x55a5ee909a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.791+0000 7f3d1e20a800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a5ee6e1330 con 0x55a5ee909620 2024-05-08T18:05:02.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.791+0000 7f3d1bd02640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee909620 0x55a5ee909a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.791+0000 7f3d1bd02640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee909620 0x55a5ee909a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57590/0 (socket says 172.21.15.50:57590) 2024-05-08T18:05:02.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.791+0000 7f3d1bd02640 1 -- 172.21.15.50:0/1993493648 learned_addr learned my addr 172.21.15.50:0/1993493648 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.792+0000 7f3d1bd02640 1 -- 172.21.15.50:0/1993493648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ee909f80 con 0x55a5ee909620 2024-05-08T18:05:02.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.792+0000 7f3d1bd02640 1 --2- 172.21.15.50:0/1993493648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee909620 0x55a5ee909a40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c01afd0 tx=0x7f3d0c024bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.792+0000 7f3c017fa640 1 -- 172.21.15.50:0/1993493648 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d0c00da20 con 0x55a5ee909620 2024-05-08T18:05:02.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.792+0000 7f3c017fa640 1 -- 172.21.15.50:0/1993493648 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d0c049f00 con 0x55a5ee909620 2024-05-08T18:05:02.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.792+0000 7f3c017fa640 1 -- 172.21.15.50:0/1993493648 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d0c04a1e0 con 0x55a5ee909620 2024-05-08T18:05:02.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.792+0000 7f3d1e20a800 1 -- 172.21.15.50:0/1993493648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a5ee90a9f0 con 0x55a5ee909620 2024-05-08T18:05:02.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.792+0000 7f3d1e20a800 1 -- 172.21.15.50:0/1993493648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5ee90aef0 con 0x55a5ee909620 2024-05-08T18:05:02.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.793+0000 7f3c017fa640 1 -- 172.21.15.50:0/1993493648 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3d0c052b90 con 0x55a5ee909620 2024-05-08T18:05:02.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.793+0000 7f3c017fa640 1 --2- 172.21.15.50:0/1993493648 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3bf403bb50 0x7f3bf403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.793+0000 7f3c017fa640 1 -- 172.21.15.50:0/1993493648 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f3d0c03ba00 con 0x55a5ee909620 2024-05-08T18:05:02.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.793+0000 7f3d1b501640 1 --2- 172.21.15.50:0/1993493648 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3bf403bb50 0x7f3bf403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.794+0000 7f3d1e20a800 1 Processor -- start 2024-05-08T18:05:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.794+0000 7f3d1e20a800 1 -- start start 2024-05-08T18:05:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.794+0000 7f3d1b501640 1 --2- 172.21.15.50:0/1993493648 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3bf403bb50 0x7f3bf403e010 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x55a5ee8f2e30 tx=0x7f3d04005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.794+0000 7f3d1e20a800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee916810 0x55a5ee916c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.794+0000 7f3d1e20a800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a5ee8e3670 con 0x55a5ee916810 2024-05-08T18:05:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.794+0000 7f3d1bd02640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee916810 0x55a5ee916c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.794+0000 7f3d1bd02640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee916810 0x55a5ee916c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57596/0 (socket says 172.21.15.50:57596) 2024-05-08T18:05:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.794+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3462425792 learned_addr learned my addr 172.21.15.50:0/3462425792 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.794+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3462425792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ee917200 con 0x55a5ee916810 2024-05-08T18:05:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.794+0000 7f3d1bd02640 1 --2- 172.21.15.50:0/3462425792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee916810 0x55a5ee916c30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c047ba0 tx=0x7f3d0c047c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8100b11ea5139ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.795+0000 7f3bfaffd640 1 -- 172.21.15.50:0/3462425792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d0c056030 con 0x55a5ee916810 2024-05-08T18:05:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.795+0000 7f3bfaffd640 1 -- 172.21.15.50:0/3462425792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d0c0167f0 con 0x55a5ee916810 2024-05-08T18:05:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.795+0000 7f3bfaffd640 1 -- 172.21.15.50:0/3462425792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d0c0522a0 con 0x55a5ee916810 2024-05-08T18:05:02.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.795+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3462425792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee916810 msgr2=0x55a5ee916c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.795+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/3462425792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee916810 0x55a5ee916c30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c047ba0 tx=0x7f3d0c047c90 comp rx=0 tx=0).stop 2024-05-08T18:05:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.795+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3462425792 shutdown_connections 2024-05-08T18:05:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.795+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/3462425792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee916810 0x55a5ee916c30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.795+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3462425792 >> 172.21.15.50:0/3462425792 conn(0x55a5ee912a90 msgr2=0x55a5ee910af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.795+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3462425792 shutdown_connections 2024-05-08T18:05:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.795+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3462425792 wait complete. 2024-05-08T18:05:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.795+0000 7f3d1e20a800 1 Processor -- start 2024-05-08T18:05:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.796+0000 7f3d1e20a800 1 -- start start 2024-05-08T18:05:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.796+0000 7f3d1e20a800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee916810 0x55a5ee91db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.796+0000 7f3d1e20a800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a5ee917870 con 0x55a5ee916810 2024-05-08T18:05:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.796+0000 7f3d1bd02640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee916810 0x55a5ee91db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.796+0000 7f3d1bd02640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee916810 0x55a5ee91db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57610/0 (socket says 172.21.15.50:57610) 2024-05-08T18:05:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.796+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 learned_addr learned my addr 172.21.15.50:0/3058459633 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.796+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5ee91e070 con 0x55a5ee916810 2024-05-08T18:05:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.796+0000 7f3d1bd02640 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee916810 0x55a5ee91db30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0492d0 tx=0x7f3d0c01cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.797+0000 7f3bf97fa640 1 -- 172.21.15.50:0/3058459633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d0c056040 con 0x55a5ee916810 2024-05-08T18:05:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.797+0000 7f3bf97fa640 1 -- 172.21.15.50:0/3058459633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d0c00b130 con 0x55a5ee916810 2024-05-08T18:05:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.797+0000 7f3bf97fa640 1 -- 172.21.15.50:0/3058459633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d0c03e230 con 0x55a5ee916810 2024-05-08T18:05:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.797+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a5ee91e070 con 0x55a5ee916810 2024-05-08T18:05:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.797+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5ee91e070 con 0x55a5ee916810 2024-05-08T18:05:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.798+0000 7f3bf97fa640 1 -- 172.21.15.50:0/3058459633 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3d0c04f140 con 0x55a5ee916810 2024-05-08T18:05:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.798+0000 7f3bf97fa640 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3be403bba0 0x7f3be403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.798+0000 7f3bf97fa640 1 -- 172.21.15.50:0/3058459633 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f3d0c067070 con 0x55a5ee916810 2024-05-08T18:05:02.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.798+0000 7f3d1b501640 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3be403bba0 0x7f3be403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.798+0000 7f3d1e20a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.798+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a5ee929100 0x55a5ee92b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.799+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee92bd10 con 0x55a5ee929100 2024-05-08T18:05:02.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.799+0000 7f3d1b501640 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3be403bba0 0x7f3be403e060 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400a470 tx=0x7f3d0404f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.799+0000 7f3d1c503640 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a5ee929100 0x55a5ee92b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.799+0000 7f3d1c503640 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a5ee929100 0x55a5ee92b5c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.799+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee929100 2024-05-08T18:05:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.799+0000 7f3d1e20a800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.799+0000 7f3d1e20a800 20 realm 2024-05-08T18:05:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.799+0000 7f3d1e20a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.799+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee92c120 con 0x55a5ee929100 2024-05-08T18:05:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.799+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee929100 2024-05-08T18:05:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.799+0000 7f3d1e20a800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.799+0000 7f3d1e20a800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.799+0000 7f3d1e20a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.799+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee92c9f0 con 0x55a5ee929100 2024-05-08T18:05:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.800+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee929100 2024-05-08T18:05:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.800+0000 7f3d1e20a800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.800+0000 7f3d1e20a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.800+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee92cd90 con 0x55a5ee929100 2024-05-08T18:05:02.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.800+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee929100 2024-05-08T18:05:02.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.800+0000 7f3d1e20a800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:02.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.800+0000 7f3d1e20a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.800+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a5ee92daf0 0x55a5ee92fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.800+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee9304f0 con 0x55a5ee92daf0 2024-05-08T18:05:02.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.800+0000 7f3d1bd02640 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a5ee92daf0 0x55a5ee92fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.801+0000 7f3d1bd02640 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a5ee92daf0 0x55a5ee92fee0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.801+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.801+0000 7f3d1e20a800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:02.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.801+0000 7f3d1e20a800 20 searching for the correct realm 2024-05-08T18:05:02.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.801+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a5ee9324a0 0x55a5ee934960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.801+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee935030 con 0x55a5ee9324a0 2024-05-08T18:05:02.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.801+0000 7f3d1b501640 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a5ee9324a0 0x55a5ee934960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.802+0000 7f3d1b501640 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a5ee9324a0 0x55a5ee934960 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.802+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.802+0000 7f3d1b501640 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3d04076040 0x7f3d04083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.802+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee935030 con 0x7f3d04076040 2024-05-08T18:05:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.802+0000 7f3d1c503640 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3d04076040 0x7f3d04083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.802+0000 7f3d1c503640 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3d04076040 0x7f3d04083410 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.803+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d1400a4a0 con 0x7f3d04076040 2024-05-08T18:05:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.803+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee92cd90 con 0x55a5ee92daf0 2024-05-08T18:05:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.803+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.803+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee92cd90 con 0x55a5ee929100 2024-05-08T18:05:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.803+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee929100 2024-05-08T18:05:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.803+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee92cd90 con 0x55a5ee929100 2024-05-08T18:05:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.803+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee929100 2024-05-08T18:05:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.803+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee92cd90 con 0x55a5ee92daf0 2024-05-08T18:05:02.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.804+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.804+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee92cd90 con 0x55a5ee92daf0 2024-05-08T18:05:02.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.804+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v20'1 uv1 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.804+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee92cd90 con 0x7f3d04076040 2024-05-08T18:05:02.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.804+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d1400a4a0 con 0x7f3d04076040 2024-05-08T18:05:02.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.804+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee92cd90 con 0x55a5ee9324a0 2024-05-08T18:05:02.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.805+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.805+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee92cd90 con 0x55a5ee929100 2024-05-08T18:05:02.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.805+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee929100 2024-05-08T18:05:02.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.805+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee92cd90 con 0x55a5ee92daf0 2024-05-08T18:05:02.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.805+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.805+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee92cd90 con 0x55a5ee929100 2024-05-08T18:05:02.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.806+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee929100 2024-05-08T18:05:02.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.806+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee92cd90 con 0x7f3d04076040 2024-05-08T18:05:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.806+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f3d1400a4a0 con 0x7f3d04076040 2024-05-08T18:05:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.806+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee92cd90 con 0x55a5ee9324a0 2024-05-08T18:05:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.806+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.806+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x55a5ee9324a0 2024-05-08T18:05:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.806+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.806+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x55a5ee92daf0 2024-05-08T18:05:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.807+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.807+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x55a5ee9324a0 2024-05-08T18:05:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.807+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.807+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x55a5ee9324a0 2024-05-08T18:05:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.807+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.807+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x55a5ee929100 2024-05-08T18:05:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.808+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee929100 2024-05-08T18:05:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.808+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x55a5ee929100 2024-05-08T18:05:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.808+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee929100 2024-05-08T18:05:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.808+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x55a5ee92daf0 2024-05-08T18:05:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.808+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.808+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x55a5ee9324a0 2024-05-08T18:05:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.808+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.808+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x7f3d04076040 2024-05-08T18:05:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.809+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d1400a4a0 con 0x7f3d04076040 2024-05-08T18:05:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.809+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x55a5ee929100 2024-05-08T18:05:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.809+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee929100 2024-05-08T18:05:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.809+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x55a5ee92daf0 2024-05-08T18:05:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.809+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.809+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x55a5ee9324a0 2024-05-08T18:05:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.810+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.810+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x55a5ee92daf0 2024-05-08T18:05:02.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.810+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.810+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x7f3d04076040 2024-05-08T18:05:02.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.810+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d1400a4a0 con 0x7f3d04076040 2024-05-08T18:05:02.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.810+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x55a5ee9324a0 2024-05-08T18:05:02.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.810+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.810+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x55a5ee92daf0 2024-05-08T18:05:02.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.811+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.811+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x55a5ee929100 2024-05-08T18:05:02.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.811+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee929100 2024-05-08T18:05:02.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.811+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3d04090040 con 0x7f3d04076040 2024-05-08T18:05:02.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.811+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3d1400a4a0 con 0x7f3d04076040 2024-05-08T18:05:02.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.811+0000 7f3d1e20a800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:02.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.811+0000 7f3d1e20a800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:02.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.811+0000 7f3d1e20a800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:02.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.811+0000 7f3d1e20a800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:02.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.811+0000 7f3d1e20a800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:02.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.811+0000 7f3d1e20a800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:02.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.811+0000 7f3d1e20a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.811+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee930db0 con 0x55a5ee929100 2024-05-08T18:05:02.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.811+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee929100 2024-05-08T18:05:02.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee930db0 con 0x55a5ee929100 2024-05-08T18:05:02.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3d1400a4a0 con 0x55a5ee929100 2024-05-08T18:05:02.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:02.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee930db0 con 0x55a5ee9324a0 2024-05-08T18:05:02.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:02.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 20 zone default found 2024-05-08T18:05:02.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 4 Realm: () 2024-05-08T18:05:02.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:02.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:02.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:02.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 20 zonegroup default 2024-05-08T18:05:02.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee931710 con 0x55a5ee92daf0 2024-05-08T18:05:02.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.812+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee936f20 con 0x55a5ee9324a0 2024-05-08T18:05:02.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.813+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.813+0000 7f3d1e20a800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.813+0000 7f3d1e20a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.813+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee9373f0 con 0x55a5ee9324a0 2024-05-08T18:05:02.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.813+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.813+0000 7f3d1e20a800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.813+0000 7f3d1e20a800 20 started sync module instance, tier type = 2024-05-08T18:05:02.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.813+0000 7f3d1e20a800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:02.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.813+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee936330 con 0x7f3d04076040 2024-05-08T18:05:02.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.814+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d1400a4a0 con 0x7f3d04076040 2024-05-08T18:05:02.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.814+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94171160602704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee936330 con 0x7f3d04076040 2024-05-08T18:05:02.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.814+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee937e70 con 0x55a5ee929100 2024-05-08T18:05:02.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.815+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94171160602704] v27'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x7f3d04076040 2024-05-08T18:05:02.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.815+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x55a5ee929100 2024-05-08T18:05:02.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.815+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94171160607648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee936330 con 0x55a5ee929100 2024-05-08T18:05:02.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.815+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee9388c0 con 0x55a5ee9324a0 2024-05-08T18:05:02.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.817+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94171160607648] v27'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x55a5ee929100 2024-05-08T18:05:02.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.817+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.817+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94171160600192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee9362a0 con 0x55a5ee9324a0 2024-05-08T18:05:02.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.817+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee939780 con 0x7f3d04076040 2024-05-08T18:05:02.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.818+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x7f3d04076040 2024-05-08T18:05:02.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.818+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94171160614064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee93a020 con 0x7f3d04076040 2024-05-08T18:05:02.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.818+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee93ab10 con 0x55a5ee929100 2024-05-08T18:05:02.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.818+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94171160600192] v27'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.819+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94171160614064] v27'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x7f3d04076040 2024-05-08T18:05:02.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.819+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x55a5ee929100 2024-05-08T18:05:02.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.819+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94171160618464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee939b20 con 0x55a5ee929100 2024-05-08T18:05:02.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.819+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee93b3b0 con 0x55a5ee92daf0 2024-05-08T18:05:02.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.820+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.820+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94171160620800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee939b20 con 0x55a5ee92daf0 2024-05-08T18:05:02.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.820+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee93c2a0 con 0x55a5ee92daf0 2024-05-08T18:05:02.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.820+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94171160618464] v27'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x55a5ee929100 2024-05-08T18:05:02.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.821+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94171160620800] v27'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.821+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.821+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94171160622928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee938cb0 con 0x55a5ee92daf0 2024-05-08T18:05:02.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.821+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee93c880 con 0x55a5ee92daf0 2024-05-08T18:05:02.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.822+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94171160622928] v27'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.822+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.822+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94171160624496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee93c880 con 0x55a5ee92daf0 2024-05-08T18:05:02.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.822+0000 7f3d1e20a800 20 add_watcher() i=0 2024-05-08T18:05:02.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.822+0000 7f3d1e20a800 20 add_watcher() i=1 2024-05-08T18:05:02.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.822+0000 7f3d1e20a800 20 add_watcher() i=2 2024-05-08T18:05:02.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.822+0000 7f3d1e20a800 20 add_watcher() i=3 2024-05-08T18:05:02.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.822+0000 7f3d1e20a800 20 add_watcher() i=4 2024-05-08T18:05:02.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.822+0000 7f3d1e20a800 20 add_watcher() i=5 2024-05-08T18:05:02.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.822+0000 7f3d1e20a800 20 add_watcher() i=6 2024-05-08T18:05:02.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.823+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94171160624496] v27'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.823+0000 7f3d1e20a800 20 add_watcher() i=7 2024-05-08T18:05:02.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.823+0000 7f3d1e20a800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:02.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.823+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee93fb60 con 0x55a5ee929100 2024-05-08T18:05:02.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.824+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f3d14040040 con 0x55a5ee929100 2024-05-08T18:05:02.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.824+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94171160626496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee93ce10 con 0x55a5ee929100 2024-05-08T18:05:02.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.825+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94171160626496] v27'26 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x55a5ee929100 2024-05-08T18:05:02.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.825+0000 7f3bd1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:02.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.825+0000 7f3d1e20a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:02.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.825+0000 7f3d1e20a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:02.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.825+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee93f360 con 0x55a5ee929100 2024-05-08T18:05:02.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.826+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x55a5ee929100 2024-05-08T18:05:02.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.826+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee948c70 con 0x55a5ee92daf0 2024-05-08T18:05:02.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.827+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'10 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.827+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee948c70 con 0x55a5ee9324a0 2024-05-08T18:05:02.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.828+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'10 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.828+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee949220 con 0x55a5ee929100 2024-05-08T18:05:02.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.829+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'27 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x55a5ee929100 2024-05-08T18:05:02.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.829+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee9496d0 con 0x7f3d04076040 2024-05-08T18:05:02.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.830+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'20 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x7f3d04076040 2024-05-08T18:05:02.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.830+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee9483c0 con 0x55a5ee9324a0 2024-05-08T18:05:02.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.831+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'13 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.831+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee9483c0 con 0x55a5ee929100 2024-05-08T18:05:02.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.833+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'12 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x55a5ee929100 2024-05-08T18:05:02.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.833+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a230 con 0x55a5ee92daf0 2024-05-08T18:05:02.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.834+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'13 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.834+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a230 con 0x7f3d04076040 2024-05-08T18:05:02.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.835+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'10 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x7f3d04076040 2024-05-08T18:05:02.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.835+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x55a5ee9324a0 2024-05-08T18:05:02.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.836+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.836+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x7f3d04076040 2024-05-08T18:05:02.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.837+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'21 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x7f3d04076040 2024-05-08T18:05:02.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.837+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x55a5ee92daf0 2024-05-08T18:05:02.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.838+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'12 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.838+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x7f3d04076040 2024-05-08T18:05:02.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.839+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'10 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x7f3d04076040 2024-05-08T18:05:02.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.839+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x55a5ee92daf0 2024-05-08T18:05:02.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.840+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'19 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.840+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x55a5ee929100 2024-05-08T18:05:02.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.841+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'28 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x55a5ee929100 2024-05-08T18:05:02.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.841+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x55a5ee9324a0 2024-05-08T18:05:02.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.842+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'14 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.842+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a7a0 con 0x7f3d04076040 2024-05-08T18:05:02.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.843+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'22 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x7f3d04076040 2024-05-08T18:05:02.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.843+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a7a0 con 0x55a5ee9324a0 2024-05-08T18:05:02.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.844+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'10 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.844+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a770 con 0x55a5ee929100 2024-05-08T18:05:02.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.845+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x55a5ee929100 2024-05-08T18:05:02.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.845+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a770 con 0x55a5ee929100 2024-05-08T18:05:02.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.846+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'29 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x55a5ee929100 2024-05-08T18:05:02.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.846+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a770 con 0x55a5ee9324a0 2024-05-08T18:05:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.847+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'15 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.847+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x7f3d04076040 2024-05-08T18:05:02.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.848+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'20 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x7f3d04076040 2024-05-08T18:05:02.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.848+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x55a5ee9324a0 2024-05-08T18:05:02.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.849+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.849+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x55a5ee9324a0 2024-05-08T18:05:02.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.850+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.850+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x55a5ee9324a0 2024-05-08T18:05:02.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.851+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'6 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.851+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x55a5ee9324a0 2024-05-08T18:05:02.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.852+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'14 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.852+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x55a5ee92daf0 2024-05-08T18:05:02.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.854+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'12 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.854+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x55a5ee9324a0 2024-05-08T18:05:02.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.855+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'10 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.855+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x55a5ee92daf0 2024-05-08T18:05:02.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.856+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'14 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.856+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x55a5ee9324a0 2024-05-08T18:05:02.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.857+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'16 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.857+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x55a5ee92daf0 2024-05-08T18:05:02.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.858+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'20 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.858+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee94a6e0 con 0x7f3d04076040 2024-05-08T18:05:02.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.859+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x7f3d04076040 2024-05-08T18:05:02.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.860+0000 7f3bd0fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:02.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.860+0000 7f3d1e20a800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:02.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.860+0000 7f3d1e20a800 20 Filter name: none 2024-05-08T18:05:02.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.860+0000 7f3b4a7fc640 20 reqs_thread_entry: start 2024-05-08T18:05:02.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.860+0000 7f3d1e20a800 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-05-08T18:05:02.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.860+0000 7f3d1e20a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.860+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:94 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee975900 con 0x7f3d04076040 2024-05-08T18:05:02.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.860+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(94 bud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x7f3d04076040 2024-05-08T18:05:02.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.860+0000 7f3d1e20a800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.860+0000 7f3d1e20a800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x0 2024-05-08T18:05:02.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.860+0000 7f3d1e20a800 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-05-08T18:05:02.859 INFO:teuthology.orchestra.run.smithi050.stderr:could not remove user: unable to remove user, user does not exist 2024-05-08T18:05:02.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.861+0000 7f3bd1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:02.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.861+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94171160626496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee947070 con 0x55a5ee929100 2024-05-08T18:05:02.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.862+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 94171160626496] v27'30 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x55a5ee929100 2024-05-08T18:05:02.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.862+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94171160602704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee93c540 con 0x7f3d04076040 2024-05-08T18:05:02.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.863+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 94171160602704] v27'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x7f3d04076040 2024-05-08T18:05:02.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.863+0000 7f3d1e20a800 20 remove_watcher() i=0 2024-05-08T18:05:02.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.863+0000 7f3d1e20a800 2 removed watcher, disabling cache 2024-05-08T18:05:02.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.863+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94171160607648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee93c540 con 0x55a5ee929100 2024-05-08T18:05:02.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.864+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 94171160607648] v27'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x55a5ee929100 2024-05-08T18:05:02.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.864+0000 7f3d1e20a800 20 remove_watcher() i=1 2024-05-08T18:05:02.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.864+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94171160600192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee937ba0 con 0x55a5ee9324a0 2024-05-08T18:05:02.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.865+0000 7f3d1b501640 1 -- 172.21.15.50:0/3058459633 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 94171160600192] v27'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d04081070 con 0x55a5ee9324a0 2024-05-08T18:05:02.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.865+0000 7f3d1e20a800 20 remove_watcher() i=2 2024-05-08T18:05:02.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.865+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94171160614064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee93c540 con 0x7f3d04076040 2024-05-08T18:05:02.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.866+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.2 v2:172.21.15.50:6811/3039646617 20 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 94171160614064] v27'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x7f3d04076040 2024-05-08T18:05:02.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.866+0000 7f3d1e20a800 20 remove_watcher() i=3 2024-05-08T18:05:02.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.866+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94171160618464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee935e80 con 0x55a5ee929100 2024-05-08T18:05:02.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.868+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 94171160618464] v27'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d14040040 con 0x55a5ee929100 2024-05-08T18:05:02.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.868+0000 7f3d1e20a800 20 remove_watcher() i=4 2024-05-08T18:05:02.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.868+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94171160620800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee937ba0 con 0x55a5ee92daf0 2024-05-08T18:05:02.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.869+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 94171160620800] v27'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.869+0000 7f3d1e20a800 20 remove_watcher() i=5 2024-05-08T18:05:02.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.869+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94171160622928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee9394b0 con 0x55a5ee92daf0 2024-05-08T18:05:02.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.870+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 94171160622928] v27'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.870+0000 7f3d1e20a800 20 remove_watcher() i=6 2024-05-08T18:05:02.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.870+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94171160624496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a5ee93c540 con 0x55a5ee92daf0 2024-05-08T18:05:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1bd02640 1 -- 172.21.15.50:0/3058459633 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 94171160624496] v27'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3d0c071020 con 0x55a5ee92daf0 2024-05-08T18:05:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 20 remove_watcher() i=7 2024-05-08T18:05:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a5ee92daf0 msgr2=0x55a5ee92fee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a5ee92daf0 0x55a5ee92fee0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a5ee929100 msgr2=0x55a5ee92b5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a5ee929100 0x55a5ee92b5c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3d04076040 msgr2=0x7f3d04083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3d04076040 0x7f3d04083410 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a5ee9324a0 msgr2=0x55a5ee934960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a5ee9324a0 0x55a5ee934960 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3be403bba0 msgr2=0x7f3be403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3be403bba0 0x7f3be403e060 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3d0400a470 tx=0x7f3d0404f000 comp rx=0 tx=0).stop 2024-05-08T18:05:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee916810 msgr2=0x55a5ee91db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1c503640 1 -- 172.21.15.50:0/3058459633 reap_dead start 2024-05-08T18:05:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee916810 0x55a5ee91db30 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c0492d0 tx=0x7f3d0c01cf20 comp rx=0 tx=0).stop 2024-05-08T18:05:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 shutdown_connections 2024-05-08T18:05:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/3058459633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee916810 0x55a5ee91db30 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 >> 172.21.15.50:0/3058459633 conn(0x55a5ee912a90 msgr2=0x55a5ee9100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 shutdown_connections 2024-05-08T18:05:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.871+0000 7f3d1e20a800 1 -- 172.21.15.50:0/3058459633 wait complete. 2024-05-08T18:05:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 -- 172.21.15.50:0/1993493648 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3bf403bb50 msgr2=0x7f3bf403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/1993493648 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3bf403bb50 0x7f3bf403e010 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x55a5ee8f2e30 tx=0x7f3d04005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 -- 172.21.15.50:0/1993493648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee909620 msgr2=0x55a5ee909a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/1993493648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee909620 0x55a5ee909a40 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c01afd0 tx=0x7f3d0c024bb0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 -- 172.21.15.50:0/1993493648 shutdown_connections 2024-05-08T18:05:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/1993493648 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3bf403bb50 0x7f3bf403e010 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/1993493648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee909620 0x55a5ee909a40 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 -- 172.21.15.50:0/1993493648 >> 172.21.15.50:0/1993493648 conn(0x55a5ee8fd6c0 msgr2=0x55a5ee8fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 -- 172.21.15.50:0/1993493648 shutdown_connections 2024-05-08T18:05:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 -- 172.21.15.50:0/1993493648 wait complete. 2024-05-08T18:05:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 -- 172.21.15.50:0/250826118 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a5ee8f4880 msgr2=0x55a5ee8f6c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a5ee8f4880 0x55a5ee8f6c70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 -- 172.21.15.50:0/250826118 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a5ee8efcf0 msgr2=0x55a5ee8f20e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a5ee8efcf0 0x55a5ee8f20e0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 -- 172.21.15.50:0/250826118 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a5ee8f87c0 msgr2=0x55a5ee8fabb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a5ee8f87c0 0x55a5ee8fabb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 -- 172.21.15.50:0/250826118 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3c0403bba0 msgr2=0x7f3c0403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3c0403bba0 0x7f3c0403e060 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3d04054000 tx=0x7f3d0404e000 comp rx=0 tx=0).stop 2024-05-08T18:05:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 -- 172.21.15.50:0/250826118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee765050 msgr2=0x55a5ee8eab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1e20a800 1 --2- 172.21.15.50:0/250826118 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5ee765050 0x55a5ee8eab60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c000c00 tx=0x7f3d0c000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.872+0000 7f3d1c503640 1 -- 172.21.15.50:0/250826118 reap_dead start 2024-05-08T18:05:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.873+0000 7f3d1e20a800 1 -- 172.21.15.50:0/250826118 shutdown_connections 2024-05-08T18:05:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.873+0000 7f3d1e20a800 1 -- 172.21.15.50:0/250826118 >> 172.21.15.50:0/250826118 conn(0x55a5ee7eac70 msgr2=0x55a5ee8dce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.873+0000 7f3d1e20a800 1 -- 172.21.15.50:0/250826118 shutdown_connections 2024-05-08T18:05:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.873+0000 7f3d1e20a800 1 -- 172.21.15.50:0/250826118 wait complete. 2024-05-08T18:05:02.879 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-08T18:05:02.880 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bar'] 2024-05-08T18:05:02.880 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bar'] 2024-05-08T18:05:02.880 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bar 2024-05-08T18:05:02.926 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:02.926 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:02.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.928+0000 7fe907d2b800 1 Processor -- start 2024-05-08T18:05:02.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.929+0000 7fe907d2b800 1 -- start start 2024-05-08T18:05:02.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.929+0000 7fe907d2b800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5564db618550 0x5564db618970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.929+0000 7fe907d2b800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5564db618ff0 con 0x5564db6197b0 2024-05-08T18:05:02.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.929+0000 7fe907d2b800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5564db4b9e20 con 0x5564db618550 2024-05-08T18:05:02.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.929+0000 7fe903ae8640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5564db618550 0x5564db618970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.929+0000 7fe903ae8640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5564db618550 0x5564db618970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57616/0 (socket says 172.21.15.50:57616) 2024-05-08T18:05:02.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.929+0000 7fe9032e7640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5564db6197b0 0x5564db619c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:52984/0 (socket says 172.21.15.50:52984) 2024-05-08T18:05:02.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.929+0000 7fe903ae8640 1 -- 172.21.15.50:0/392217748 learned_addr learned my addr 172.21.15.50:0/392217748 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.929+0000 7fe903ae8640 1 -- 172.21.15.50:0/392217748 >> v1:172.21.15.46:6789/0 conn(0x5564db6197b0 legacy=0x5564db619c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.929+0000 7fe903ae8640 1 -- 172.21.15.50:0/392217748 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564db61a370 con 0x5564db618550 2024-05-08T18:05:02.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.931+0000 7fe903ae8640 1 --2- 172.21.15.50:0/392217748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 0x5564db618970 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400a520 tx=0x7fe8f401d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3327d4fe2ee58b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.931+0000 7fe902ae6640 1 -- 172.21.15.50:0/392217748 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f4057070 con 0x5564db618550 2024-05-08T18:05:02.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.931+0000 7fe902ae6640 1 -- 172.21.15.50:0/392217748 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f4052570 con 0x5564db618550 2024-05-08T18:05:02.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.931+0000 7fe902ae6640 1 -- 172.21.15.50:0/392217748 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f4052870 con 0x5564db618550 2024-05-08T18:05:02.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.932+0000 7fe907d2b800 1 -- 172.21.15.50:0/392217748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 msgr2=0x5564db618970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.932+0000 7fe907d2b800 1 --2- 172.21.15.50:0/392217748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 0x5564db618970 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400a520 tx=0x7fe8f401d010 comp rx=0 tx=0).stop 2024-05-08T18:05:02.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.932+0000 7fe907d2b800 1 -- 172.21.15.50:0/392217748 shutdown_connections 2024-05-08T18:05:02.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.932+0000 7fe907d2b800 1 --2- 172.21.15.50:0/392217748 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 0x5564db618970 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.932+0000 7fe907d2b800 1 -- 172.21.15.50:0/392217748 >> 172.21.15.50:0/392217748 conn(0x5564db613b50 msgr2=0x5564db615f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:02.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.932+0000 7fe907d2b800 1 -- 172.21.15.50:0/392217748 shutdown_connections 2024-05-08T18:05:02.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.932+0000 7fe907d2b800 1 -- 172.21.15.50:0/392217748 wait complete. 2024-05-08T18:05:02.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.947+0000 7fe907d2b800 1 Processor -- start 2024-05-08T18:05:02.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.947+0000 7fe907d2b800 1 -- start start 2024-05-08T18:05:02.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.948+0000 7fe907d2b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 0x5564db58c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.948+0000 7fe907d2b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5564db564a10 con 0x5564db618550 2024-05-08T18:05:02.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.948+0000 7fe903ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 0x5564db58c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.948+0000 7fe903ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 0x5564db58c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57626/0 (socket says 172.21.15.50:57626) 2024-05-08T18:05:02.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.948+0000 7fe903ae8640 1 -- 172.21.15.50:0/102440231 learned_addr learned my addr 172.21.15.50:0/102440231 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.948+0000 7fe903ae8640 1 -- 172.21.15.50:0/102440231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564db70d960 con 0x5564db618550 2024-05-08T18:05:02.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.948+0000 7fe903ae8640 1 --2- 172.21.15.50:0/102440231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 0x5564db58c530 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4055040 tx=0x7fe8f404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d277c0971e2e5906 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.949+0000 7fe7f1e0b640 1 -- 172.21.15.50:0/102440231 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f4057040 con 0x5564db618550 2024-05-08T18:05:02.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.949+0000 7fe7f1e0b640 1 -- 172.21.15.50:0/102440231 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f4052870 con 0x5564db618550 2024-05-08T18:05:02.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.949+0000 7fe7f1e0b640 1 -- 172.21.15.50:0/102440231 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f4052b90 con 0x5564db618550 2024-05-08T18:05:02.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.949+0000 7fe907d2b800 1 -- 172.21.15.50:0/102440231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 msgr2=0x5564db58c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.949+0000 7fe907d2b800 1 --2- 172.21.15.50:0/102440231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 0x5564db58c530 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4055040 tx=0x7fe8f404c000 comp rx=0 tx=0).stop 2024-05-08T18:05:02.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.949+0000 7fe907d2b800 1 -- 172.21.15.50:0/102440231 shutdown_connections 2024-05-08T18:05:02.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.949+0000 7fe907d2b800 1 --2- 172.21.15.50:0/102440231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 0x5564db58c530 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.949+0000 7fe907d2b800 1 -- 172.21.15.50:0/102440231 >> 172.21.15.50:0/102440231 conn(0x5564db613b50 msgr2=0x5564db705450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:02.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.949+0000 7fe907d2b800 1 -- 172.21.15.50:0/102440231 shutdown_connections 2024-05-08T18:05:02.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.949+0000 7fe907d2b800 1 -- 172.21.15.50:0/102440231 wait complete. 2024-05-08T18:05:02.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.950+0000 7fe907d2b800 1 Processor -- start 2024-05-08T18:05:02.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.950+0000 7fe907d2b800 1 -- start start 2024-05-08T18:05:02.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.950+0000 7fe907d2b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 0x5564db7126e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.950+0000 7fe907d2b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5564db621c80 con 0x5564db618550 2024-05-08T18:05:02.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.950+0000 7fe903ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 0x5564db7126e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.950+0000 7fe903ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 0x5564db7126e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57628/0 (socket says 172.21.15.50:57628) 2024-05-08T18:05:02.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.950+0000 7fe903ae8640 1 -- 172.21.15.50:0/4283979286 learned_addr learned my addr 172.21.15.50:0/4283979286 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.950+0000 7fe903ae8640 1 -- 172.21.15.50:0/4283979286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564db712c20 con 0x5564db618550 2024-05-08T18:05:02.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.951+0000 7fe903ae8640 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 0x5564db7126e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4052540 tx=0x7fe8f4003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.951+0000 7fe7ebfff640 1 -- 172.21.15.50:0/4283979286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f4073070 con 0x5564db618550 2024-05-08T18:05:02.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.951+0000 7fe7ebfff640 1 -- 172.21.15.50:0/4283979286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f405d570 con 0x5564db618550 2024-05-08T18:05:02.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.951+0000 7fe907d2b800 1 -- 172.21.15.50:0/4283979286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5564db712eb0 con 0x5564db618550 2024-05-08T18:05:02.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.951+0000 7fe7ebfff640 1 -- 172.21.15.50:0/4283979286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f4057070 con 0x5564db618550 2024-05-08T18:05:02.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.951+0000 7fe907d2b800 1 -- 172.21.15.50:0/4283979286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564db715b10 con 0x5564db618550 2024-05-08T18:05:02.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.952+0000 7fe7ebfff640 1 -- 172.21.15.50:0/4283979286 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fe8f4076070 con 0x5564db618550 2024-05-08T18:05:02.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.952+0000 7fe7ebfff640 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7ec03bba0 0x7fe7ec03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.952+0000 7fe7ebfff640 1 -- 172.21.15.50:0/4283979286 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fe8f4094070 con 0x5564db618550 2024-05-08T18:05:02.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.952+0000 7fe9032e7640 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7ec03bba0 0x7fe7ec03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.952+0000 7fe907d2b800 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5564db6197b0 0x5564db7198d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.952+0000 7fe907d2b800 1 -- 172.21.15.50:0/4283979286 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db719e10 con 0x5564db6197b0 2024-05-08T18:05:02.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.952+0000 7fe9042e9640 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5564db6197b0 0x5564db7198d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.953+0000 7fe9032e7640 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7ec03bba0 0x7fe7ec03e060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8001e30 tx=0x7fe8f8001390 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.953+0000 7fe9042e9640 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5564db6197b0 0x5564db7198d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.953+0000 7fe9042e9640 1 -- 172.21.15.50:0/4283979286 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db6197b0 2024-05-08T18:05:02.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.953+0000 7fe907d2b800 1 -- 172.21.15.50:0/4283979286 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db716ea0 con 0x5564db6197b0 2024-05-08T18:05:02.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.953+0000 7fe9042e9640 1 -- 172.21.15.50:0/4283979286 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db6197b0 2024-05-08T18:05:02.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.953+0000 7fe907d2b800 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5564db71c100 0x5564db71e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.953+0000 7fe907d2b800 1 -- 172.21.15.50:0/4283979286 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db71ebc0 con 0x5564db71c100 2024-05-08T18:05:02.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.954+0000 7fe903ae8640 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5564db71c100 0x5564db71e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.954+0000 7fe903ae8640 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5564db71c100 0x5564db71e4f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.954+0000 7fe903ae8640 1 -- 172.21.15.50:0/4283979286 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db71c100 2024-05-08T18:05:02.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.954+0000 7fe907d2b800 1 -- 172.21.15.50:0/4283979286 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db71afa0 con 0x5564db6197b0 2024-05-08T18:05:02.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.955+0000 7fe9042e9640 1 -- 172.21.15.50:0/4283979286 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db6197b0 2024-05-08T18:05:02.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.955+0000 7fe907d2b800 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5564db720000 0x5564db7223f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.955+0000 7fe907d2b800 1 -- 172.21.15.50:0/4283979286 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db722b30 con 0x5564db720000 2024-05-08T18:05:02.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.955+0000 7fe9032e7640 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5564db720000 0x5564db7223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.955+0000 7fe9032e7640 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5564db720000 0x5564db7223f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.955+0000 7fe9032e7640 1 -- 172.21.15.50:0/4283979286 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fe8f800bd50 con 0x5564db720000 2024-05-08T18:05:02.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.956+0000 7fe907d2b800 1 Processor -- start 2024-05-08T18:05:02.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.956+0000 7fe907d2b800 1 -- start start 2024-05-08T18:05:02.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.956+0000 7fe907d2b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db70b4f0 0x5564db709960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.956+0000 7fe907d2b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5564db50b330 con 0x5564db70b4f0 2024-05-08T18:05:02.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.956+0000 7fe903ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db70b4f0 0x5564db709960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.956+0000 7fe903ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db70b4f0 0x5564db709960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57642/0 (socket says 172.21.15.50:57642) 2024-05-08T18:05:02.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.956+0000 7fe903ae8640 1 -- 172.21.15.50:0/3023791059 learned_addr learned my addr 172.21.15.50:0/3023791059 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.956+0000 7fe903ae8640 1 -- 172.21.15.50:0/3023791059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564db709ea0 con 0x5564db70b4f0 2024-05-08T18:05:02.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.956+0000 7fe903ae8640 1 --2- 172.21.15.50:0/3023791059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db70b4f0 0x5564db709960 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4006e20 tx=0x7fe8f408bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.957+0000 7fe7e97fa640 1 -- 172.21.15.50:0/3023791059 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f408b070 con 0x5564db70b4f0 2024-05-08T18:05:02.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.957+0000 7fe7e97fa640 1 -- 172.21.15.50:0/3023791059 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f4076de0 con 0x5564db70b4f0 2024-05-08T18:05:02.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.957+0000 7fe907d2b800 1 -- 172.21.15.50:0/3023791059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5564db70a130 con 0x5564db70b4f0 2024-05-08T18:05:02.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.957+0000 7fe7e97fa640 1 -- 172.21.15.50:0/3023791059 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f405cc80 con 0x5564db70b4f0 2024-05-08T18:05:02.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.957+0000 7fe907d2b800 1 -- 172.21.15.50:0/3023791059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564db70a630 con 0x5564db70b4f0 2024-05-08T18:05:02.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.957+0000 7fe7e97fa640 1 -- 172.21.15.50:0/3023791059 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fe8f4076360 con 0x5564db70b4f0 2024-05-08T18:05:02.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.958+0000 7fe7e97fa640 1 --2- 172.21.15.50:0/3023791059 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7dc03bb00 0x7fe7dc03dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.958+0000 7fe7e97fa640 1 -- 172.21.15.50:0/3023791059 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fe8f4083070 con 0x5564db70b4f0 2024-05-08T18:05:02.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.958+0000 7fe9032e7640 1 --2- 172.21.15.50:0/3023791059 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7dc03bb00 0x7fe7dc03dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.958+0000 7fe907d2b800 1 Processor -- start 2024-05-08T18:05:02.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.958+0000 7fe907d2b800 1 -- start start 2024-05-08T18:05:02.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.958+0000 7fe9032e7640 1 --2- 172.21.15.50:0/3023791059 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7dc03bb00 0x7fe7dc03dfc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x5564db71a280 tx=0x7fe8f800f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.958+0000 7fe907d2b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db740250 0x5564db740670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.958+0000 7fe907d2b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5564db70dbf0 con 0x5564db740250 2024-05-08T18:05:02.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.959+0000 7fe903ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db740250 0x5564db740670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.959+0000 7fe903ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db740250 0x5564db740670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57652/0 (socket says 172.21.15.50:57652) 2024-05-08T18:05:02.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.959+0000 7fe903ae8640 1 -- 172.21.15.50:0/111832140 learned_addr learned my addr 172.21.15.50:0/111832140 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.959+0000 7fe903ae8640 1 -- 172.21.15.50:0/111832140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564db740c40 con 0x5564db740250 2024-05-08T18:05:02.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.959+0000 7fe903ae8640 1 --2- 172.21.15.50:0/111832140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db740250 0x5564db740670 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4091240 tx=0x7fe8f406cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a000cc48a8c9a9f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.959+0000 7fe7e2ffd640 1 -- 172.21.15.50:0/111832140 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f40773b0 con 0x5564db740250 2024-05-08T18:05:02.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.959+0000 7fe7e2ffd640 1 -- 172.21.15.50:0/111832140 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f408b330 con 0x5564db740250 2024-05-08T18:05:02.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.959+0000 7fe7e2ffd640 1 -- 172.21.15.50:0/111832140 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f4077550 con 0x5564db740250 2024-05-08T18:05:02.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.960+0000 7fe907d2b800 1 -- 172.21.15.50:0/111832140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db740250 msgr2=0x5564db740670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:02.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.960+0000 7fe907d2b800 1 --2- 172.21.15.50:0/111832140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db740250 0x5564db740670 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4091240 tx=0x7fe8f406cb90 comp rx=0 tx=0).stop 2024-05-08T18:05:02.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.960+0000 7fe907d2b800 1 -- 172.21.15.50:0/111832140 shutdown_connections 2024-05-08T18:05:02.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.960+0000 7fe907d2b800 1 --2- 172.21.15.50:0/111832140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db740250 0x5564db740670 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:02.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.960+0000 7fe907d2b800 1 -- 172.21.15.50:0/111832140 >> 172.21.15.50:0/111832140 conn(0x5564db73c5b0 msgr2=0x5564db73a610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:02.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.960+0000 7fe907d2b800 1 -- 172.21.15.50:0/111832140 shutdown_connections 2024-05-08T18:05:02.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.960+0000 7fe907d2b800 1 -- 172.21.15.50:0/111832140 wait complete. 2024-05-08T18:05:02.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.960+0000 7fe907d2b800 1 Processor -- start 2024-05-08T18:05:02.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.960+0000 7fe907d2b800 1 -- start start 2024-05-08T18:05:02.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.960+0000 7fe907d2b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db740250 0x5564db7475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.960+0000 7fe907d2b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5564db7412f0 con 0x5564db740250 2024-05-08T18:05:02.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.960+0000 7fe903ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db740250 0x5564db7475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.960+0000 7fe903ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db740250 0x5564db7475a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57660/0 (socket says 172.21.15.50:57660) 2024-05-08T18:05:02.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.960+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 learned_addr learned my addr 172.21.15.50:0/1332313560 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:02.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.961+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564db747ae0 con 0x5564db740250 2024-05-08T18:05:02.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.961+0000 7fe903ae8640 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db740250 0x5564db7475a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40912e0 tx=0x7fe8f405dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.961+0000 7fe7e17fa640 1 -- 172.21.15.50:0/1332313560 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f408b850 con 0x5564db740250 2024-05-08T18:05:02.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.961+0000 7fe7e17fa640 1 -- 172.21.15.50:0/1332313560 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f406c890 con 0x5564db740250 2024-05-08T18:05:02.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.961+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5564db747ae0 con 0x5564db740250 2024-05-08T18:05:02.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.961+0000 7fe7e17fa640 1 -- 172.21.15.50:0/1332313560 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8f4078590 con 0x5564db740250 2024-05-08T18:05:02.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.961+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564db747ae0 con 0x5564db740250 2024-05-08T18:05:02.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.962+0000 7fe7e17fa640 1 -- 172.21.15.50:0/1332313560 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fe8f40b4020 con 0x5564db740250 2024-05-08T18:05:02.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.962+0000 7fe7e17fa640 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7cc03bb50 0x7fe7cc03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.962+0000 7fe7e17fa640 1 -- 172.21.15.50:0/1332313560 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fe8f40bd9b0 con 0x5564db740250 2024-05-08T18:05:02.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.962+0000 7fe9032e7640 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7cc03bb50 0x7fe7cc03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.963+0000 7fe9032e7640 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7cc03bb50 0x7fe7cc03e010 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5564db722f30 tx=0x7fe8f8010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.963+0000 7fe907d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.963+0000 7fe907d2b800 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5564db752d00 0x5564db7551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.963+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db755910 con 0x5564db752d00 2024-05-08T18:05:02.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.963+0000 7fe9042e9640 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5564db752d00 0x5564db7551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.963+0000 7fe9042e9640 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5564db752d00 0x5564db7551c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.963+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db752d00 2024-05-08T18:05:02.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.963+0000 7fe907d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.963+0000 7fe907d2b800 20 realm 2024-05-08T18:05:02.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.963+0000 7fe907d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.963+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db755d20 con 0x5564db752d00 2024-05-08T18:05:02.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.964+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db752d00 2024-05-08T18:05:02.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.964+0000 7fe907d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.964+0000 7fe907d2b800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.964+0000 7fe907d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.964+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db7565f0 con 0x5564db752d00 2024-05-08T18:05:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.964+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db752d00 2024-05-08T18:05:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.964+0000 7fe907d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.964+0000 7fe907d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.964+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db752d00 2024-05-08T18:05:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.964+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db752d00 2024-05-08T18:05:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.964+0000 7fe907d2b800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.964+0000 7fe907d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.964+0000 7fe907d2b800 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5564db7576f0 0x5564db759ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.964+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db75a0a0 con 0x5564db7576f0 2024-05-08T18:05:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.965+0000 7fe903ae8640 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5564db7576f0 0x5564db759ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.965+0000 7fe903ae8640 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5564db7576f0 0x5564db759ae0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.965+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.965+0000 7fe907d2b800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.965+0000 7fe907d2b800 20 searching for the correct realm 2024-05-08T18:05:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.965+0000 7fe907d2b800 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5564db75c050 0x5564db75e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.965+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db75ebe0 con 0x5564db75c050 2024-05-08T18:05:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.965+0000 7fe9032e7640 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5564db75c050 0x5564db75e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.966+0000 7fe9032e7640 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5564db75c050 0x5564db75e510 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.966+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.966+0000 7fe9032e7640 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fe8f800f070 0x7fe8f802e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.966+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db75ebe0 con 0x7fe8f800f070 2024-05-08T18:05:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.966+0000 7fe9042e9640 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fe8f800f070 0x7fe8f802e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.967+0000 7fe9042e9640 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fe8f800f070 0x7fe8f802e910 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.967+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x7fe8f800f070 2024-05-08T18:05:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.967+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db7576f0 2024-05-08T18:05:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.967+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.967+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db752d00 2024-05-08T18:05:02.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.967+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db752d00 2024-05-08T18:05:02.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.967+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db752d00 2024-05-08T18:05:02.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.968+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db752d00 2024-05-08T18:05:02.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.968+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db7576f0 2024-05-08T18:05:02.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.968+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.968+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db7576f0 2024-05-08T18:05:02.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.968+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v20'1 uv1 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.968+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x7fe8f800f070 2024-05-08T18:05:02.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.968+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x7fe8f800f070 2024-05-08T18:05:02.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.968+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db75c050 2024-05-08T18:05:02.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.969+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.969+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db752d00 2024-05-08T18:05:02.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.969+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db752d00 2024-05-08T18:05:02.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.969+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db7576f0 2024-05-08T18:05:02.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.969+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.969+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db752d00 2024-05-08T18:05:02.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.969+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db752d00 2024-05-08T18:05:02.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.969+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x7fe8f800f070 2024-05-08T18:05:02.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.970+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x7fe8f800f070 2024-05-08T18:05:02.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.970+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db75c050 2024-05-08T18:05:02.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.970+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.970+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db75c050 2024-05-08T18:05:02.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.970+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.970+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db7576f0 2024-05-08T18:05:02.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.970+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.970+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db75c050 2024-05-08T18:05:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.971+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.971+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db75c050 2024-05-08T18:05:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.971+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.971+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db752d00 2024-05-08T18:05:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.971+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db752d00 2024-05-08T18:05:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.971+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db752d00 2024-05-08T18:05:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.971+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db752d00 2024-05-08T18:05:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.971+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db7576f0 2024-05-08T18:05:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.972+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.972+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db75c050 2024-05-08T18:05:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.972+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.972+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x7fe8f800f070 2024-05-08T18:05:02.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.972+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x7fe8f800f070 2024-05-08T18:05:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.972+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db752d00 2024-05-08T18:05:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.972+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db752d00 2024-05-08T18:05:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.972+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db7576f0 2024-05-08T18:05:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.973+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.973+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db75c050 2024-05-08T18:05:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.973+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.973+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db7576f0 2024-05-08T18:05:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.973+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.973+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x7fe8f800f070 2024-05-08T18:05:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.973+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x7fe8f800f070 2024-05-08T18:05:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.973+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db75c050 2024-05-08T18:05:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.974+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.974+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db7576f0 2024-05-08T18:05:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.974+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.974+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x5564db752d00 2024-05-08T18:05:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.974+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db752d00 2024-05-08T18:05:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.974+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5564db756990 con 0x7fe8f800f070 2024-05-08T18:05:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.974+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x7fe8f800f070 2024-05-08T18:05:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.974+0000 7fe907d2b800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.974+0000 7fe907d2b800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.974+0000 7fe907d2b800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.974+0000 7fe907d2b800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.974+0000 7fe907d2b800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.974+0000 7fe907d2b800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.974+0000 7fe907d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.974+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db75a960 con 0x5564db752d00 2024-05-08T18:05:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db752d00 2024-05-08T18:05:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe907d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe907d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db75a960 con 0x5564db752d00 2024-05-08T18:05:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x5564db752d00 2024-05-08T18:05:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe907d2b800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe907d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db75a960 con 0x5564db75c050 2024-05-08T18:05:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe907d2b800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe907d2b800 20 zone default found 2024-05-08T18:05:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe907d2b800 4 Realm: () 2024-05-08T18:05:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe907d2b800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe907d2b800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe907d2b800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe907d2b800 20 zonegroup default 2024-05-08T18:05:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe907d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.975+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db75b2d0 con 0x5564db7576f0 2024-05-08T18:05:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.976+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.976+0000 7fe907d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.976+0000 7fe907d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.976+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db760ad0 con 0x5564db75c050 2024-05-08T18:05:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.976+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.976+0000 7fe907d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.976+0000 7fe907d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.976+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db760f30 con 0x5564db75c050 2024-05-08T18:05:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.976+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.976+0000 7fe907d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.976+0000 7fe907d2b800 20 started sync module instance, tier type = 2024-05-08T18:05:02.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.976+0000 7fe907d2b800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:02.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.976+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db75fee0 con 0x7fe8f800f070 2024-05-08T18:05:02.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.977+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8fc00a4a0 con 0x7fe8f800f070 2024-05-08T18:05:02.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.977+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93891667035136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db75fee0 con 0x7fe8f800f070 2024-05-08T18:05:02.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.977+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db7619f0 con 0x5564db752d00 2024-05-08T18:05:02.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.978+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93891667035136] v27'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x7fe8f800f070 2024-05-08T18:05:02.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.979+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x5564db752d00 2024-05-08T18:05:02.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.979+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93891667039488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db75fee0 con 0x5564db752d00 2024-05-08T18:05:02.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.979+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db762460 con 0x5564db75c050 2024-05-08T18:05:02.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.980+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.980+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93891667042128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db762d30 con 0x5564db75c050 2024-05-08T18:05:02.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.980+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db763810 con 0x7fe8f800f070 2024-05-08T18:05:02.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.980+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93891667039488] v27'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x5564db752d00 2024-05-08T18:05:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.981+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x7fe8f800f070 2024-05-08T18:05:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.981+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93891667047200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db763c00 con 0x7fe8f800f070 2024-05-08T18:05:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.981+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db7645c0 con 0x5564db752d00 2024-05-08T18:05:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.981+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93891667042128] v27'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.982+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x5564db752d00 2024-05-08T18:05:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.982+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93891667044480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db7645c0 con 0x5564db752d00 2024-05-08T18:05:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.982+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db765040 con 0x5564db7576f0 2024-05-08T18:05:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.982+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93891667047200] v27'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x7fe8f800f070 2024-05-08T18:05:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.983+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.983+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93891667032624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db765430 con 0x5564db7576f0 2024-05-08T18:05:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.983+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db765df0 con 0x5564db7576f0 2024-05-08T18:05:02.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.983+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93891667044480] v27'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x5564db752d00 2024-05-08T18:05:02.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.984+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.984+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93891667050768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db766690 con 0x5564db7576f0 2024-05-08T18:05:02.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.985+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db766f60 con 0x5564db7576f0 2024-05-08T18:05:02.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.985+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 93891667032624] v27'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.985+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93891667050768] v27'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.986+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.986+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93891667053392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db7666d0 con 0x5564db7576f0 2024-05-08T18:05:02.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.986+0000 7fe907d2b800 20 add_watcher() i=0 2024-05-08T18:05:02.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.986+0000 7fe907d2b800 20 add_watcher() i=1 2024-05-08T18:05:02.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.986+0000 7fe907d2b800 20 add_watcher() i=2 2024-05-08T18:05:02.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.986+0000 7fe907d2b800 20 add_watcher() i=3 2024-05-08T18:05:02.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.986+0000 7fe907d2b800 20 add_watcher() i=4 2024-05-08T18:05:02.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.986+0000 7fe907d2b800 20 add_watcher() i=5 2024-05-08T18:05:02.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.986+0000 7fe907d2b800 20 add_watcher() i=6 2024-05-08T18:05:02.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.987+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93891667053392] v27'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.987+0000 7fe907d2b800 20 add_watcher() i=7 2024-05-08T18:05:02.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.987+0000 7fe907d2b800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:02.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.987+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db76a060 con 0x5564db752d00 2024-05-08T18:05:02.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.987+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fe8fc040040 con 0x5564db752d00 2024-05-08T18:05:02.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.987+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93891667059168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db7674e0 con 0x5564db752d00 2024-05-08T18:05:02.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.988+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93891667059168] v27'31 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x5564db752d00 2024-05-08T18:05:02.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.988+0000 7fe7b9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:02.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.988+0000 7fe907d2b800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:02.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.988+0000 7fe907d2b800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:02.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.988+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db7727a0 con 0x5564db752d00 2024-05-08T18:05:02.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.989+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x5564db752d00 2024-05-08T18:05:02.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.989+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db772e70 con 0x5564db7576f0 2024-05-08T18:05:02.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.991+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'11 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.991+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db772e70 con 0x5564db75c050 2024-05-08T18:05:02.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.992+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'11 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.992+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db773910 con 0x5564db752d00 2024-05-08T18:05:02.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.992+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'32 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x5564db752d00 2024-05-08T18:05:02.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.993+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db770630 con 0x7fe8f800f070 2024-05-08T18:05:02.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.994+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'23 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x7fe8f800f070 2024-05-08T18:05:02.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.994+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db770630 con 0x5564db75c050 2024-05-08T18:05:02.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.995+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'15 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.995+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774640 con 0x5564db752d00 2024-05-08T18:05:02.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.996+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'13 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x5564db752d00 2024-05-08T18:05:02.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.996+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db7705b0 con 0x5564db7576f0 2024-05-08T18:05:02.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.997+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'15 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.997+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db7705b0 con 0x7fe8f800f070 2024-05-08T18:05:02.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.998+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'11 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x7fe8f800f070 2024-05-08T18:05:02.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.998+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db7705b0 con 0x5564db75c050 2024-05-08T18:05:02.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.999+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:02.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:02.999+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x7fe8f800f070 2024-05-08T18:05:02.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.000+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'24 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x7fe8f800f070 2024-05-08T18:05:02.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.000+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x5564db7576f0 2024-05-08T18:05:02.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.001+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'13 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:02.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.002+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x7fe8f800f070 2024-05-08T18:05:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.002+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'11 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x7fe8f800f070 2024-05-08T18:05:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.002+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x5564db7576f0 2024-05-08T18:05:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.004+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'21 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.004+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db7705b0 con 0x5564db752d00 2024-05-08T18:05:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.005+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'33 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x5564db752d00 2024-05-08T18:05:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.005+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db7705b0 con 0x5564db75c050 2024-05-08T18:05:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.006+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'17 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.006+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x7fe8f800f070 2024-05-08T18:05:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.007+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'25 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x7fe8f800f070 2024-05-08T18:05:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.007+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x5564db75c050 2024-05-08T18:05:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.008+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'11 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.008+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x5564db752d00 2024-05-08T18:05:03.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.009+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x5564db752d00 2024-05-08T18:05:03.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.009+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x5564db752d00 2024-05-08T18:05:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.010+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'34 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x5564db752d00 2024-05-08T18:05:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.010+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x5564db75c050 2024-05-08T18:05:03.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.011+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'18 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:03.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.011+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x7fe8f800f070 2024-05-08T18:05:03.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.012+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'21 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x7fe8f800f070 2024-05-08T18:05:03.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.012+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x5564db75c050 2024-05-08T18:05:03.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.013+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:03.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.013+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x5564db75c050 2024-05-08T18:05:03.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.014+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:03.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.014+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x5564db75c050 2024-05-08T18:05:03.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.015+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'7 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:03.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.015+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x5564db75c050 2024-05-08T18:05:03.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.016+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'16 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:03.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.016+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x5564db7576f0 2024-05-08T18:05:03.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.017+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'13 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:03.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.017+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x5564db75c050 2024-05-08T18:05:03.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.018+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'11 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:03.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.018+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x5564db7576f0 2024-05-08T18:05:03.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.019+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'16 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:03.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.019+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x5564db75c050 2024-05-08T18:05:03.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.020+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'19 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:03.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.020+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x5564db7576f0 2024-05-08T18:05:03.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.021+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'22 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:03.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.021+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db774d30 con 0x7fe8f800f070 2024-05-08T18:05:03.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.022+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x7fe8f800f070 2024-05-08T18:05:03.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.023+0000 7fe7b8fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:03.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.023+0000 7fe907d2b800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:03.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.023+0000 7fe907d2b800 20 Filter name: none 2024-05-08T18:05:03.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.023+0000 7fe7327fc640 20 reqs_thread_entry: start 2024-05-08T18:05:03.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.023+0000 7fe907d2b800 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-05-08T18:05:03.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.023+0000 7fe907d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.023+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:94 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5564db79fc90 con 0x5564db7576f0 2024-05-08T18:05:03.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.023+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(94 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:03.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.023+0000 7fe907d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.023+0000 7fe907d2b800 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-05-08T18:05:03.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.023+0000 7fe907d2b800 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-05-08T18:05:03.022 INFO:teuthology.orchestra.run.smithi050.stderr:could not remove user: unable to remove user, user does not exist 2024-05-08T18:05:03.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.024+0000 7fe7b9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:03.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.024+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93891667059168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db76f5a0 con 0x5564db752d00 2024-05-08T18:05:03.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.025+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 93891667059168] v27'35 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x5564db752d00 2024-05-08T18:05:03.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.025+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93891667035136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db7661e0 con 0x7fe8f800f070 2024-05-08T18:05:03.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.026+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 93891667035136] v27'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x7fe8f800f070 2024-05-08T18:05:03.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.026+0000 7fe907d2b800 20 remove_watcher() i=0 2024-05-08T18:05:03.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.026+0000 7fe907d2b800 2 removed watcher, disabling cache 2024-05-08T18:05:03.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.026+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93891667039488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db760400 con 0x5564db752d00 2024-05-08T18:05:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.028+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 93891667039488] v27'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x5564db752d00 2024-05-08T18:05:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.028+0000 7fe907d2b800 20 remove_watcher() i=1 2024-05-08T18:05:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.028+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93891667042128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db760400 con 0x5564db75c050 2024-05-08T18:05:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.029+0000 7fe9032e7640 1 -- 172.21.15.50:0/1332313560 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 93891667042128] v27'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8f8037040 con 0x5564db75c050 2024-05-08T18:05:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.029+0000 7fe907d2b800 20 remove_watcher() i=2 2024-05-08T18:05:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.029+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93891667047200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db760400 con 0x7fe8f800f070 2024-05-08T18:05:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.030+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 93891667047200] v27'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x7fe8f800f070 2024-05-08T18:05:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.030+0000 7fe907d2b800 20 remove_watcher() i=3 2024-05-08T18:05:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.030+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93891667044480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db763320 con 0x5564db752d00 2024-05-08T18:05:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.031+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 93891667044480] v27'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8fc040040 con 0x5564db752d00 2024-05-08T18:05:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.031+0000 7fe907d2b800 20 remove_watcher() i=4 2024-05-08T18:05:03.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.031+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93891667032624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db760400 con 0x5564db7576f0 2024-05-08T18:05:03.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.032+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 93891667032624] v27'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:03.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.032+0000 7fe907d2b800 20 remove_watcher() i=5 2024-05-08T18:05:03.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.032+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93891667050768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db762880 con 0x5564db7576f0 2024-05-08T18:05:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.033+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 93891667050768] v27'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.033+0000 7fe907d2b800 20 remove_watcher() i=6 2024-05-08T18:05:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.033+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93891667053392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5564db760400 con 0x5564db7576f0 2024-05-08T18:05:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe903ae8640 1 -- 172.21.15.50:0/1332313560 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 93891667053392] v27'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe8f4075030 con 0x5564db7576f0 2024-05-08T18:05:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 20 remove_watcher() i=7 2024-05-08T18:05:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5564db7576f0 msgr2=0x5564db759ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5564db7576f0 0x5564db759ae0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5564db752d00 msgr2=0x5564db7551c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5564db752d00 0x5564db7551c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fe8f800f070 msgr2=0x7fe8f802e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fe8f800f070 0x7fe8f802e910 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5564db75c050 msgr2=0x5564db75e510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5564db75c050 0x5564db75e510 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7cc03bb50 msgr2=0x7fe7cc03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7cc03bb50 0x7fe7cc03e010 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x5564db722f30 tx=0x7fe8f8010000 comp rx=0 tx=0).stop 2024-05-08T18:05:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db740250 msgr2=0x5564db7475a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe9042e9640 1 -- 172.21.15.50:0/1332313560 reap_dead start 2024-05-08T18:05:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db740250 0x5564db7475a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40912e0 tx=0x7fe8f405dde0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 shutdown_connections 2024-05-08T18:05:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 --2- 172.21.15.50:0/1332313560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db740250 0x5564db7475a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 >> 172.21.15.50:0/1332313560 conn(0x5564db73c5b0 msgr2=0x5564db739c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 shutdown_connections 2024-05-08T18:05:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.034+0000 7fe907d2b800 1 -- 172.21.15.50:0/1332313560 wait complete. 2024-05-08T18:05:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 -- 172.21.15.50:0/3023791059 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7dc03bb00 msgr2=0x7fe7dc03dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 --2- 172.21.15.50:0/3023791059 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7dc03bb00 0x7fe7dc03dfc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x5564db71a280 tx=0x7fe8f800f000 comp rx=0 tx=0).stop 2024-05-08T18:05:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 -- 172.21.15.50:0/3023791059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db70b4f0 msgr2=0x5564db709960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 --2- 172.21.15.50:0/3023791059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db70b4f0 0x5564db709960 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4006e20 tx=0x7fe8f408bd80 comp rx=0 tx=0).stop 2024-05-08T18:05:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 -- 172.21.15.50:0/3023791059 shutdown_connections 2024-05-08T18:05:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 --2- 172.21.15.50:0/3023791059 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7dc03bb00 0x7fe7dc03dfc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 --2- 172.21.15.50:0/3023791059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db70b4f0 0x5564db709960 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 -- 172.21.15.50:0/3023791059 >> 172.21.15.50:0/3023791059 conn(0x5564db724f40 msgr2=0x5564db725340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 -- 172.21.15.50:0/3023791059 shutdown_connections 2024-05-08T18:05:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 -- 172.21.15.50:0/3023791059 wait complete. 2024-05-08T18:05:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 -- 172.21.15.50:0/4283979286 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5564db71c100 msgr2=0x5564db71e4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5564db71c100 0x5564db71e4f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 -- 172.21.15.50:0/4283979286 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5564db6197b0 msgr2=0x5564db7198d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5564db6197b0 0x5564db7198d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 -- 172.21.15.50:0/4283979286 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5564db720000 msgr2=0x5564db7223f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5564db720000 0x5564db7223f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 -- 172.21.15.50:0/4283979286 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7ec03bba0 msgr2=0x7fe7ec03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7ec03bba0 0x7fe7ec03e060 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8001e30 tx=0x7fe8f8001390 comp rx=0 tx=0).stop 2024-05-08T18:05:03.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 -- 172.21.15.50:0/4283979286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 msgr2=0x5564db7126e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 --2- 172.21.15.50:0/4283979286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564db618550 0x5564db7126e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4052540 tx=0x7fe8f4003020 comp rx=0 tx=0).stop 2024-05-08T18:05:03.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe9042e9640 1 -- 172.21.15.50:0/4283979286 reap_dead start 2024-05-08T18:05:03.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 -- 172.21.15.50:0/4283979286 shutdown_connections 2024-05-08T18:05:03.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 -- 172.21.15.50:0/4283979286 >> 172.21.15.50:0/4283979286 conn(0x5564db613b50 msgr2=0x5564db703c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 -- 172.21.15.50:0/4283979286 shutdown_connections 2024-05-08T18:05:03.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.035+0000 7fe907d2b800 1 -- 172.21.15.50:0/4283979286 wait complete. 2024-05-08T18:05:03.042 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-08T18:05:03.043 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-05-08T18:05:03.043 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-05-08T18:05:03.043 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-05-08T18:05:03.088 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:03.089 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:03.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.091+0000 7f6bf4f2e800 1 Processor -- start 2024-05-08T18:05:03.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.091+0000 7f6bf4f2e800 1 -- start start 2024-05-08T18:05:03.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.092+0000 7f6bf4f2e800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5632c4235c90 0x5632c4238080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.092+0000 7f6bf4f2e800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5632c42385c0 con 0x5632c4232180 2024-05-08T18:05:03.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.092+0000 7f6bf4f2e800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5632c40d1e20 con 0x5632c4235c90 2024-05-08T18:05:03.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.092+0000 7f6bf045f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5632c4235c90 0x5632c4238080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.092+0000 7f6bf045f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5632c4235c90 0x5632c4238080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57672/0 (socket says 172.21.15.50:57672) 2024-05-08T18:05:03.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.092+0000 7f6bf045f640 1 -- 172.21.15.50:0/4116069866 learned_addr learned my addr 172.21.15.50:0/4116069866 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.092+0000 7f6bf045f640 1 -- 172.21.15.50:0/4116069866 >> v1:172.21.15.46:6789/0 conn(0x5632c4232180 legacy=0x5632c4234570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.092+0000 7f6bf045f640 1 -- 172.21.15.50:0/4116069866 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632c42397a0 con 0x5632c4235c90 2024-05-08T18:05:03.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.094+0000 7f6bf045f640 1 --2- 172.21.15.50:0/4116069866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4235c90 0x5632c4238080 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6be400a520 tx=0x7f6be401d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18dbde0c322ef044 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.094+0000 7f6befc5e640 1 -- 172.21.15.50:0/4116069866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be4057070 con 0x5632c4235c90 2024-05-08T18:05:03.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.094+0000 7f6befc5e640 1 -- 172.21.15.50:0/4116069866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be4052570 con 0x5632c4235c90 2024-05-08T18:05:03.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.094+0000 7f6befc5e640 1 -- 172.21.15.50:0/4116069866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be404ae60 con 0x5632c4235c90 2024-05-08T18:05:03.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.095+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/4116069866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4235c90 msgr2=0x5632c4238080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.095+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/4116069866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4235c90 0x5632c4238080 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6be400a520 tx=0x7f6be401d010 comp rx=0 tx=0).stop 2024-05-08T18:05:03.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.095+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/4116069866 shutdown_connections 2024-05-08T18:05:03.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.095+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/4116069866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4235c90 0x5632c4238080 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.095+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/4116069866 >> 172.21.15.50:0/4116069866 conn(0x5632c422bb50 msgr2=0x5632c422df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.095+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/4116069866 shutdown_connections 2024-05-08T18:05:03.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.095+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/4116069866 wait complete. 2024-05-08T18:05:03.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.110+0000 7f6bf4f2e800 1 Processor -- start 2024-05-08T18:05:03.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.110+0000 7f6bf4f2e800 1 -- start start 2024-05-08T18:05:03.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.110+0000 7f6bf4f2e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4232180 0x5632c4231ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.110+0000 7f6bf4f2e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5632c417ca10 con 0x5632c4232180 2024-05-08T18:05:03.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.111+0000 7f6bf0c60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4232180 0x5632c4231ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.111+0000 7f6bf0c60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4232180 0x5632c4231ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57686/0 (socket says 172.21.15.50:57686) 2024-05-08T18:05:03.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.111+0000 7f6bf0c60640 1 -- 172.21.15.50:0/4266964092 learned_addr learned my addr 172.21.15.50:0/4266964092 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.111+0000 7f6bf0c60640 1 -- 172.21.15.50:0/4266964092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632c4230080 con 0x5632c4232180 2024-05-08T18:05:03.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.111+0000 7f6bf0c60640 1 --2- 172.21.15.50:0/4266964092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4232180 0x5632c4231ac0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6be0001d10 tx=0x7f6be000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c70a3b1bc7ecad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.111+0000 7f6adee0d640 1 -- 172.21.15.50:0/4266964092 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be0001e60 con 0x5632c4232180 2024-05-08T18:05:03.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.111+0000 7f6adee0d640 1 -- 172.21.15.50:0/4266964092 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be000f400 con 0x5632c4232180 2024-05-08T18:05:03.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.111+0000 7f6adee0d640 1 -- 172.21.15.50:0/4266964092 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be000f760 con 0x5632c4232180 2024-05-08T18:05:03.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.112+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/4266964092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4232180 msgr2=0x5632c4231ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.112+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/4266964092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4232180 0x5632c4231ac0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6be0001d10 tx=0x7f6be000b5b0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.112+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/4266964092 shutdown_connections 2024-05-08T18:05:03.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.112+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/4266964092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4232180 0x5632c4231ac0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.112+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/4266964092 >> 172.21.15.50:0/4266964092 conn(0x5632c422bb50 msgr2=0x5632c431e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.112+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/4266964092 shutdown_connections 2024-05-08T18:05:03.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.112+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/4266964092 wait complete. 2024-05-08T18:05:03.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.113+0000 7f6bf4f2e800 1 Processor -- start 2024-05-08T18:05:03.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.113+0000 7f6bf4f2e800 1 -- start start 2024-05-08T18:05:03.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.113+0000 7f6bf4f2e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4232180 0x5632c432c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.113+0000 7f6bf4f2e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5632c4238a90 con 0x5632c4232180 2024-05-08T18:05:03.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.113+0000 7f6bf0c60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4232180 0x5632c432c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.113+0000 7f6bf0c60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4232180 0x5632c432c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57690/0 (socket says 172.21.15.50:57690) 2024-05-08T18:05:03.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.113+0000 7f6bf0c60640 1 -- 172.21.15.50:0/421374029 learned_addr learned my addr 172.21.15.50:0/421374029 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.113+0000 7f6bf0c60640 1 -- 172.21.15.50:0/421374029 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632c432cdb0 con 0x5632c4232180 2024-05-08T18:05:03.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.113+0000 7f6bf0c60640 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4232180 0x5632c432c870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6be000b5f0 tx=0x7f6be0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.114+0000 7f6add60a640 1 -- 172.21.15.50:0/421374029 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be001a480 con 0x5632c4232180 2024-05-08T18:05:03.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.114+0000 7f6add60a640 1 -- 172.21.15.50:0/421374029 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be001a640 con 0x5632c4232180 2024-05-08T18:05:03.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.114+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/421374029 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5632c432d040 con 0x5632c4232180 2024-05-08T18:05:03.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.114+0000 7f6add60a640 1 -- 172.21.15.50:0/421374029 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be001a960 con 0x5632c4232180 2024-05-08T18:05:03.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.114+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/421374029 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632c432fcf0 con 0x5632c4232180 2024-05-08T18:05:03.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.115+0000 7f6add60a640 1 -- 172.21.15.50:0/421374029 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6be0017070 con 0x5632c4232180 2024-05-08T18:05:03.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.115+0000 7f6add60a640 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ad803bba0 0x7f6ad803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.115+0000 7f6add60a640 1 -- 172.21.15.50:0/421374029 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f6be0015020 con 0x5632c4232180 2024-05-08T18:05:03.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.115+0000 7f6bf045f640 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ad803bba0 0x7f6ad803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.115+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632c4235c90 0x5632c4333ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.115+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/421374029 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c4333ff0 con 0x5632c4235c90 2024-05-08T18:05:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.115+0000 7f6bf1461640 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632c4235c90 0x5632c4333ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.115+0000 7f6bf045f640 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ad803bba0 0x7f6ad803e060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6be4054000 tx=0x7f6be404e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.115+0000 7f6bf1461640 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632c4235c90 0x5632c4333ab0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.116+0000 7f6bf1461640 1 -- 172.21.15.50:0/421374029 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c4235c90 2024-05-08T18:05:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.116+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/421374029 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c4331080 con 0x5632c4235c90 2024-05-08T18:05:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.116+0000 7f6bf1461640 1 -- 172.21.15.50:0/421374029 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c4235c90 2024-05-08T18:05:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.116+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632c4336390 0x5632c4338780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.116+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/421374029 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c4338e20 con 0x5632c4336390 2024-05-08T18:05:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.116+0000 7f6bf0c60640 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632c4336390 0x5632c4338780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.117+0000 7f6bf0c60640 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632c4336390 0x5632c4338780 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.117+0000 7f6bf0c60640 1 -- 172.21.15.50:0/421374029 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f6be002e070 con 0x5632c4336390 2024-05-08T18:05:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.117+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/421374029 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c4335200 con 0x5632c4235c90 2024-05-08T18:05:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.117+0000 7f6bf1461640 1 -- 172.21.15.50:0/421374029 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c4235c90 2024-05-08T18:05:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.117+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632c433a280 0x5632c433c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.117+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/421374029 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c433cd60 con 0x5632c433a280 2024-05-08T18:05:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.117+0000 7f6bf045f640 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632c433a280 0x5632c433c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.118+0000 7f6bf045f640 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632c433a280 0x5632c433c670 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.118+0000 7f6bf045f640 1 -- 172.21.15.50:0/421374029 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f6be405a040 con 0x5632c433a280 2024-05-08T18:05:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.118+0000 7f6bf4f2e800 1 Processor -- start 2024-05-08T18:05:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.118+0000 7f6bf4f2e800 1 -- start start 2024-05-08T18:05:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.119+0000 7f6bf4f2e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c434b030 0x5632c434b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.119+0000 7f6bf4f2e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5632c4123330 con 0x5632c434b030 2024-05-08T18:05:03.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.119+0000 7f6bf0c60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c434b030 0x5632c434b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.119+0000 7f6bf0c60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c434b030 0x5632c434b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57702/0 (socket says 172.21.15.50:57702) 2024-05-08T18:05:03.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.119+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2190203340 learned_addr learned my addr 172.21.15.50:0/2190203340 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.119+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2190203340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632c434b990 con 0x5632c434b030 2024-05-08T18:05:03.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.119+0000 7f6bf0c60640 1 --2- 172.21.15.50:0/2190203340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c434b030 0x5632c434b450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6be0051170 tx=0x7f6be001aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.119+0000 7f6ad67fc640 1 -- 172.21.15.50:0/2190203340 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be0017860 con 0x5632c434b030 2024-05-08T18:05:03.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.119+0000 7f6ad67fc640 1 -- 172.21.15.50:0/2190203340 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be004a010 con 0x5632c434b030 2024-05-08T18:05:03.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.119+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2190203340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5632c434c400 con 0x5632c434b030 2024-05-08T18:05:03.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.119+0000 7f6ad67fc640 1 -- 172.21.15.50:0/2190203340 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be004a2f0 con 0x5632c434b030 2024-05-08T18:05:03.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.119+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2190203340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632c434c400 con 0x5632c434b030 2024-05-08T18:05:03.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.120+0000 7f6ad67fc640 1 -- 172.21.15.50:0/2190203340 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6be000e070 con 0x5632c434b030 2024-05-08T18:05:03.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.120+0000 7f6ad67fc640 1 --2- 172.21.15.50:0/2190203340 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ac803bb50 0x7f6ac803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.120+0000 7f6ad67fc640 1 -- 172.21.15.50:0/2190203340 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f6be003cc50 con 0x5632c434b030 2024-05-08T18:05:03.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.120+0000 7f6bf045f640 1 --2- 172.21.15.50:0/2190203340 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ac803bb50 0x7f6ac803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.121+0000 7f6bf4f2e800 1 Processor -- start 2024-05-08T18:05:03.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.121+0000 7f6bf4f2e800 1 -- start start 2024-05-08T18:05:03.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.121+0000 7f6bf045f640 1 --2- 172.21.15.50:0/2190203340 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ac803bb50 0x7f6ac803e010 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x5632c4330fa0 tx=0x7f6be4005ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.121+0000 7f6bf4f2e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4358040 0x5632c4358460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.121+0000 7f6bf4f2e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5632c4230400 con 0x5632c4358040 2024-05-08T18:05:03.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.121+0000 7f6bf0c60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4358040 0x5632c4358460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.121+0000 7f6bf0c60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4358040 0x5632c4358460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57710/0 (socket says 172.21.15.50:57710) 2024-05-08T18:05:03.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.121+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2647145890 learned_addr learned my addr 172.21.15.50:0/2647145890 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.121+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2647145890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632c4358a30 con 0x5632c4358040 2024-05-08T18:05:03.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.122+0000 7f6bf0c60640 1 --2- 172.21.15.50:0/2647145890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4358040 0x5632c4358460 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6be004dfe0 tx=0x7f6be0030a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324ee76aeb3b5cd5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.122+0000 7f6ac7fff640 1 -- 172.21.15.50:0/2647145890 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be0043cf0 con 0x5632c4358040 2024-05-08T18:05:03.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.122+0000 7f6ac7fff640 1 -- 172.21.15.50:0/2647145890 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be004a770 con 0x5632c4358040 2024-05-08T18:05:03.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.122+0000 7f6ac7fff640 1 -- 172.21.15.50:0/2647145890 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be001b830 con 0x5632c4358040 2024-05-08T18:05:03.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.122+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2647145890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4358040 msgr2=0x5632c4358460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.122+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/2647145890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4358040 0x5632c4358460 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6be004dfe0 tx=0x7f6be0030a20 comp rx=0 tx=0).stop 2024-05-08T18:05:03.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.122+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2647145890 shutdown_connections 2024-05-08T18:05:03.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.122+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/2647145890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4358040 0x5632c4358460 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.122+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2647145890 >> 172.21.15.50:0/2647145890 conn(0x5632c43542c0 msgr2=0x5632c4352320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.122+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2647145890 shutdown_connections 2024-05-08T18:05:03.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.122+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2647145890 wait complete. 2024-05-08T18:05:03.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.123+0000 7f6bf4f2e800 1 Processor -- start 2024-05-08T18:05:03.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.123+0000 7f6bf4f2e800 1 -- start start 2024-05-08T18:05:03.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.123+0000 7f6bf4f2e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4358040 0x5632c435f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.123+0000 7f6bf4f2e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5632c43590a0 con 0x5632c4358040 2024-05-08T18:05:03.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.123+0000 7f6bf0c60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4358040 0x5632c435f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.123+0000 7f6bf0c60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4358040 0x5632c435f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57720/0 (socket says 172.21.15.50:57720) 2024-05-08T18:05:03.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.123+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 learned_addr learned my addr 172.21.15.50:0/2192629846 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.123+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632c435f8c0 con 0x5632c4358040 2024-05-08T18:05:03.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.123+0000 7f6bf0c60640 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4358040 0x5632c435f380 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6be004dfe0 tx=0x7f6be002c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.124+0000 7f6ac67fc640 1 -- 172.21.15.50:0/2192629846 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be006d020 con 0x5632c4358040 2024-05-08T18:05:03.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.124+0000 7f6ac67fc640 1 -- 172.21.15.50:0/2192629846 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6be002c060 con 0x5632c4358040 2024-05-08T18:05:03.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.124+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5632c435f8c0 con 0x5632c4358040 2024-05-08T18:05:03.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.124+0000 7f6ac67fc640 1 -- 172.21.15.50:0/2192629846 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6be0027430 con 0x5632c4358040 2024-05-08T18:05:03.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.124+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632c435fdc0 con 0x5632c4358040 2024-05-08T18:05:03.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.125+0000 7f6ac67fc640 1 -- 172.21.15.50:0/2192629846 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6be0070020 con 0x5632c4358040 2024-05-08T18:05:03.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.125+0000 7f6ac67fc640 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6abc03bba0 0x7f6abc03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.125+0000 7f6ac67fc640 1 -- 172.21.15.50:0/2192629846 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f6be0063030 con 0x5632c4358040 2024-05-08T18:05:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.125+0000 7f6bf045f640 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6abc03bba0 0x7f6abc03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.125+0000 7f6bf045f640 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6abc03bba0 0x7f6abc03e060 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6be4052fd0 tx=0x7f6be404f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.126+0000 7f6bf4f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.126+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632c436ab10 0x5632c436cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.126+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c436d720 con 0x5632c436ab10 2024-05-08T18:05:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.126+0000 7f6bf1461640 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632c436ab10 0x5632c436cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.126+0000 7f6bf1461640 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632c436ab10 0x5632c436cfd0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.126+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c436ab10 2024-05-08T18:05:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.126+0000 7f6bf4f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.126+0000 7f6bf4f2e800 20 realm 2024-05-08T18:05:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.126+0000 7f6bf4f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.126+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c436db30 con 0x5632c436ab10 2024-05-08T18:05:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.126+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c436ab10 2024-05-08T18:05:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.127+0000 7f6bf4f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.127+0000 7f6bf4f2e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.127+0000 7f6bf4f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.127+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c436e400 con 0x5632c436ab10 2024-05-08T18:05:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.127+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c436ab10 2024-05-08T18:05:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.127+0000 7f6bf4f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.127+0000 7f6bf4f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.127+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c436e7a0 con 0x5632c436ab10 2024-05-08T18:05:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.127+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c436ab10 2024-05-08T18:05:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.127+0000 7f6bf4f2e800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.127+0000 7f6bf4f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.127+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632c436f500 0x5632c43718f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.127+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c4371f00 con 0x5632c436f500 2024-05-08T18:05:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.127+0000 7f6bf0c60640 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632c436f500 0x5632c43718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.128+0000 7f6bf0c60640 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632c436f500 0x5632c43718f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.128+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.128+0000 7f6bf4f2e800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.128+0000 7f6bf4f2e800 20 searching for the correct realm 2024-05-08T18:05:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.128+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632c4373eb0 0x5632c4376370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.128+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5632c4376a40 con 0x5632c4373eb0 2024-05-08T18:05:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.128+0000 7f6bf045f640 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632c4373eb0 0x5632c4376370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.129+0000 7f6bf045f640 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632c4373eb0 0x5632c4376370 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.129+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.129+0000 7f6bf045f640 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6be4074040 0x7f6be4081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.129+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5632c4376a40 con 0x7f6be4074040 2024-05-08T18:05:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.129+0000 7f6bf1461640 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6be4074040 0x7f6be4081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.129+0000 7f6bf1461640 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6be4074040 0x7f6be4081410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.129+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be800a4a0 con 0x7f6be4074040 2024-05-08T18:05:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.129+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5632c436e7a0 con 0x5632c436f500 2024-05-08T18:05:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.130+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.130+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5632c436e7a0 con 0x5632c436ab10 2024-05-08T18:05:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.130+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c436ab10 2024-05-08T18:05:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.130+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5632c436e7a0 con 0x5632c436ab10 2024-05-08T18:05:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.130+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c436ab10 2024-05-08T18:05:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.130+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5632c436e7a0 con 0x5632c436f500 2024-05-08T18:05:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.131+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.131+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5632c436e7a0 con 0x5632c436f500 2024-05-08T18:05:03.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.131+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v20'1 uv1 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.131+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5632c436e7a0 con 0x7f6be4074040 2024-05-08T18:05:03.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.131+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be800a4a0 con 0x7f6be4074040 2024-05-08T18:05:03.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.131+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5632c436e7a0 con 0x5632c4373eb0 2024-05-08T18:05:03.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.131+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.131+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5632c436e7a0 con 0x5632c436ab10 2024-05-08T18:05:03.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.131+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c436ab10 2024-05-08T18:05:03.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.132+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5632c436e7a0 con 0x5632c436f500 2024-05-08T18:05:03.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.132+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.132+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5632c436e7a0 con 0x5632c436ab10 2024-05-08T18:05:03.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.132+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c436ab10 2024-05-08T18:05:03.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.132+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5632c436e7a0 con 0x7f6be4074040 2024-05-08T18:05:03.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.132+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f6be800a4a0 con 0x7f6be4074040 2024-05-08T18:05:03.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.132+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5632c436e7a0 con 0x5632c4373eb0 2024-05-08T18:05:03.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.132+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.133+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x5632c4373eb0 2024-05-08T18:05:03.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.133+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.133+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x5632c436f500 2024-05-08T18:05:03.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.133+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.133+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x5632c4373eb0 2024-05-08T18:05:03.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.133+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.133+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x5632c4373eb0 2024-05-08T18:05:03.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.133+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.133+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x5632c436ab10 2024-05-08T18:05:03.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.134+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c436ab10 2024-05-08T18:05:03.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.134+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x5632c436ab10 2024-05-08T18:05:03.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.134+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c436ab10 2024-05-08T18:05:03.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.134+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x5632c436f500 2024-05-08T18:05:03.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.134+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.134+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x5632c4373eb0 2024-05-08T18:05:03.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.135+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.135+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x7f6be4074040 2024-05-08T18:05:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.135+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be800a4a0 con 0x7f6be4074040 2024-05-08T18:05:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.135+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x5632c436ab10 2024-05-08T18:05:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.135+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c436ab10 2024-05-08T18:05:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.135+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x5632c436f500 2024-05-08T18:05:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.135+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.135+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x5632c4373eb0 2024-05-08T18:05:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.136+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.136+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x5632c436f500 2024-05-08T18:05:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.136+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.136+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x7f6be4074040 2024-05-08T18:05:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.136+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be800a4a0 con 0x7f6be4074040 2024-05-08T18:05:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.136+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x5632c4373eb0 2024-05-08T18:05:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.137+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.137+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x5632c436f500 2024-05-08T18:05:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.137+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.137+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x5632c436ab10 2024-05-08T18:05:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.137+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c436ab10 2024-05-08T18:05:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.137+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6be408e040 con 0x7f6be4074040 2024-05-08T18:05:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6be800a4a0 con 0x7f6be4074040 2024-05-08T18:05:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c43727c0 con 0x5632c436ab10 2024-05-08T18:05:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c436ab10 2024-05-08T18:05:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c43727c0 con 0x5632c436ab10 2024-05-08T18:05:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6be800a4a0 con 0x5632c436ab10 2024-05-08T18:05:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c43727c0 con 0x5632c4373eb0 2024-05-08T18:05:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:03.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 20 zone default found 2024-05-08T18:05:03.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 4 Realm: () 2024-05-08T18:05:03.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:03.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:03.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:03.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 20 zonegroup default 2024-05-08T18:05:03.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.138+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c4373160 con 0x5632c436f500 2024-05-08T18:05:03.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.139+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.139+0000 7f6bf4f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.139+0000 7f6bf4f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.139+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c4378930 con 0x5632c4373eb0 2024-05-08T18:05:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.139+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.139+0000 7f6bf4f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.139+0000 7f6bf4f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.139+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c4378d90 con 0x5632c4373eb0 2024-05-08T18:05:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.139+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.139+0000 7f6bf4f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.139+0000 7f6bf4f2e800 20 started sync module instance, tier type = 2024-05-08T18:05:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.139+0000 7f6bf4f2e800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.139+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c4377d40 con 0x7f6be4074040 2024-05-08T18:05:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.140+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be800a4a0 con 0x7f6be4074040 2024-05-08T18:05:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.140+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94776040325728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c4377d40 con 0x7f6be4074040 2024-05-08T18:05:03.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.140+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c4379860 con 0x5632c436ab10 2024-05-08T18:05:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.141+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94776040325728] v27'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x7f6be4074040 2024-05-08T18:05:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.142+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x5632c436ab10 2024-05-08T18:05:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.142+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94776040330048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c4377d40 con 0x5632c436ab10 2024-05-08T18:05:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.142+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437a2d0 con 0x5632c4373eb0 2024-05-08T18:05:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.143+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94776040330048] v27'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x5632c436ab10 2024-05-08T18:05:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.143+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.143+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94776040323216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c4377cb0 con 0x5632c4373eb0 2024-05-08T18:05:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.143+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437b190 con 0x7f6be4074040 2024-05-08T18:05:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.144+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x7f6be4074040 2024-05-08T18:05:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.144+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94776040323216] v27'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.144+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94776040337120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437ba30 con 0x7f6be4074040 2024-05-08T18:05:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.144+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437a700 con 0x5632c436ab10 2024-05-08T18:05:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.145+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94776040337120] v27'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x7f6be4074040 2024-05-08T18:05:03.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.145+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x5632c436ab10 2024-05-08T18:05:03.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.145+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94776040338560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437bfc0 con 0x5632c436ab10 2024-05-08T18:05:03.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.145+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437c980 con 0x5632c436f500 2024-05-08T18:05:03.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.146+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94776040338560] v27'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x5632c436ab10 2024-05-08T18:05:03.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.146+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.146+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94776040342736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437cd70 con 0x5632c436f500 2024-05-08T18:05:03.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.146+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437d730 con 0x5632c436f500 2024-05-08T18:05:03.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.147+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94776040342736] v27'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.147+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.147+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94776040339616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437df80 con 0x5632c436f500 2024-05-08T18:05:03.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.147+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437e940 con 0x5632c436f500 2024-05-08T18:05:03.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.148+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94776040339616] v27'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.148+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.149+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94776040350864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437ed60 con 0x5632c436f500 2024-05-08T18:05:03.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.149+0000 7f6bf4f2e800 20 add_watcher() i=0 2024-05-08T18:05:03.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.149+0000 7f6bf4f2e800 20 add_watcher() i=1 2024-05-08T18:05:03.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.149+0000 7f6bf4f2e800 20 add_watcher() i=2 2024-05-08T18:05:03.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.149+0000 7f6bf4f2e800 20 add_watcher() i=3 2024-05-08T18:05:03.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.149+0000 7f6bf4f2e800 20 add_watcher() i=4 2024-05-08T18:05:03.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.149+0000 7f6bf4f2e800 20 add_watcher() i=5 2024-05-08T18:05:03.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.149+0000 7f6bf4f2e800 20 add_watcher() i=6 2024-05-08T18:05:03.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.150+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94776040350864] v27'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.150+0000 7f6bf4f2e800 20 add_watcher() i=7 2024-05-08T18:05:03.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.150+0000 7f6bf4f2e800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:03.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.150+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c4381fb0 con 0x5632c436ab10 2024-05-08T18:05:03.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.150+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f6be8040040 con 0x5632c436ab10 2024-05-08T18:05:03.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.150+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94776040352288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437f2f0 con 0x5632c436ab10 2024-05-08T18:05:03.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.151+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94776040352288] v27'36 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x5632c436ab10 2024-05-08T18:05:03.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.151+0000 7f6aa77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:03.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.151+0000 7f6bf4f2e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:03.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.151+0000 7f6bf4f2e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:03.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.151+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c4381840 con 0x5632c436ab10 2024-05-08T18:05:03.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.152+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x5632c436ab10 2024-05-08T18:05:03.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.152+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438b0c0 con 0x5632c436f500 2024-05-08T18:05:03.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.154+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'12 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.154+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438b0c0 con 0x5632c4373eb0 2024-05-08T18:05:03.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.155+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.155+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438b560 con 0x5632c436ab10 2024-05-08T18:05:03.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.156+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'37 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x5632c436ab10 2024-05-08T18:05:03.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.156+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438bad0 con 0x7f6be4074040 2024-05-08T18:05:03.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.157+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'26 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x7f6be4074040 2024-05-08T18:05:03.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.157+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438c040 con 0x5632c4373eb0 2024-05-08T18:05:03.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.158+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'17 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.158+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438c4f0 con 0x5632c436ab10 2024-05-08T18:05:03.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.159+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'14 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x5632c436ab10 2024-05-08T18:05:03.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.159+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438a7c0 con 0x5632c436f500 2024-05-08T18:05:03.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.160+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'17 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.160+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438a7c0 con 0x7f6be4074040 2024-05-08T18:05:03.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.161+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x7f6be4074040 2024-05-08T18:05:03.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.161+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438ce80 con 0x5632c4373eb0 2024-05-08T18:05:03.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.162+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.162+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438ce80 con 0x7f6be4074040 2024-05-08T18:05:03.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.163+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'27 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x7f6be4074040 2024-05-08T18:05:03.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.163+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438ce80 con 0x5632c436f500 2024-05-08T18:05:03.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.164+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'14 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.164+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438ce80 con 0x7f6be4074040 2024-05-08T18:05:03.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.165+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x7f6be4074040 2024-05-08T18:05:03.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.165+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438ce80 con 0x5632c436f500 2024-05-08T18:05:03.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.167+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'23 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.167+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438ce80 con 0x5632c436ab10 2024-05-08T18:05:03.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.168+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'38 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x5632c436ab10 2024-05-08T18:05:03.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.168+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438ce80 con 0x5632c4373eb0 2024-05-08T18:05:03.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.169+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'20 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.169+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438a060 con 0x7f6be4074040 2024-05-08T18:05:03.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.170+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'28 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x7f6be4074040 2024-05-08T18:05:03.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.170+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438a060 con 0x5632c4373eb0 2024-05-08T18:05:03.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.171+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'12 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.171+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438a060 con 0x5632c436ab10 2024-05-08T18:05:03.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.172+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'20 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x5632c436ab10 2024-05-08T18:05:03.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.172+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438a060 con 0x5632c436ab10 2024-05-08T18:05:03.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.173+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'39 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x5632c436ab10 2024-05-08T18:05:03.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.173+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438a060 con 0x5632c4373eb0 2024-05-08T18:05:03.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.174+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'21 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.174+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438a060 con 0x7f6be4074040 2024-05-08T18:05:03.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.175+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'22 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x7f6be4074040 2024-05-08T18:05:03.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.175+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438a060 con 0x5632c4373eb0 2024-05-08T18:05:03.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.176+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.176+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438a060 con 0x5632c4373eb0 2024-05-08T18:05:03.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.177+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.177+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438a060 con 0x5632c4373eb0 2024-05-08T18:05:03.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.178+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'8 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.178+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438a060 con 0x5632c4373eb0 2024-05-08T18:05:03.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.179+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'18 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.179+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438a060 con 0x5632c436f500 2024-05-08T18:05:03.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.180+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'14 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.180+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438a060 con 0x5632c4373eb0 2024-05-08T18:05:03.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.181+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'12 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.181+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438d400 con 0x5632c436f500 2024-05-08T18:05:03.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.182+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'18 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.182+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438d3e0 con 0x5632c4373eb0 2024-05-08T18:05:03.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.183+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'22 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.183+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438d3e0 con 0x5632c436f500 2024-05-08T18:05:03.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.185+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'24 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.185+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c438a7c0 con 0x7f6be4074040 2024-05-08T18:05:03.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.186+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x7f6be4074040 2024-05-08T18:05:03.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.186+0000 7f6aa67e4640 20 reqs_thread_entry: start 2024-05-08T18:05:03.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.186+0000 7f6bf4f2e800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:03.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.186+0000 7f6bf4f2e800 20 Filter name: none 2024-05-08T18:05:03.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.186+0000 7f6a9f7fe640 20 reqs_thread_entry: start 2024-05-08T18:05:03.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.186+0000 7f6bf4f2e800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-05-08T18:05:03.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.186+0000 7f6bf4f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.186+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5632c43b7f70 con 0x5632c436f500 2024-05-08T18:05:03.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.187+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(94 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.187+0000 7f6bf4f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.187+0000 7f6bf4f2e800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-05-08T18:05:03.201 INFO:teuthology.orchestra.run.smithi050.stderr:could not remove user: 2024-05-08T18:05:03.187+0000 7f6bf4f2e800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-05-08T18:05:03.201 INFO:teuthology.orchestra.run.smithi050.stderr:unable to remove user, user does not exist 2024-05-08T18:05:03.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.188+0000 7f6aa77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:03.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.188+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94776040352288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c4389ba0 con 0x5632c436ab10 2024-05-08T18:05:03.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.188+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 94776040352288] v27'40 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x5632c436ab10 2024-05-08T18:05:03.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.189+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94776040325728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437ea20 con 0x7f6be4074040 2024-05-08T18:05:03.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.190+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 94776040325728] v27'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x7f6be4074040 2024-05-08T18:05:03.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.190+0000 7f6bf4f2e800 20 remove_watcher() i=0 2024-05-08T18:05:03.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.190+0000 7f6bf4f2e800 2 removed watcher, disabling cache 2024-05-08T18:05:03.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.190+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94776040330048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437ea20 con 0x5632c436ab10 2024-05-08T18:05:03.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.191+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 94776040330048] v27'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x5632c436ab10 2024-05-08T18:05:03.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.191+0000 7f6bf4f2e800 20 remove_watcher() i=1 2024-05-08T18:05:03.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.191+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94776040323216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437ea20 con 0x5632c4373eb0 2024-05-08T18:05:03.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.192+0000 7f6bf045f640 1 -- 172.21.15.50:0/2192629846 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 94776040323216] v27'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be407f070 con 0x5632c4373eb0 2024-05-08T18:05:03.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.192+0000 7f6bf4f2e800 20 remove_watcher() i=2 2024-05-08T18:05:03.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.192+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94776040337120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437ea20 con 0x7f6be4074040 2024-05-08T18:05:03.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.193+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 94776040337120] v27'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x7f6be4074040 2024-05-08T18:05:03.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.193+0000 7f6bf4f2e800 20 remove_watcher() i=3 2024-05-08T18:05:03.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.193+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94776040338560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437ea20 con 0x5632c436ab10 2024-05-08T18:05:03.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.194+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 94776040338560] v27'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be8040040 con 0x5632c436ab10 2024-05-08T18:05:03.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.194+0000 7f6bf4f2e800 20 remove_watcher() i=4 2024-05-08T18:05:03.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.194+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94776040342736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437b480 con 0x5632c436f500 2024-05-08T18:05:03.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.195+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 94776040342736] v27'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.195+0000 7f6bf4f2e800 20 remove_watcher() i=5 2024-05-08T18:05:03.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.195+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94776040339616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437ea20 con 0x5632c436f500 2024-05-08T18:05:03.206 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-08T18:05:03.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.196+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 94776040339616] v27'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.196+0000 7f6bf4f2e800 20 remove_watcher() i=6 2024-05-08T18:05:03.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.196+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94776040350864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5632c437ea20 con 0x5632c436f500 2024-05-08T18:05:03.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf0c60640 1 -- 172.21.15.50:0/2192629846 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 94776040350864] v27'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6be0066070 con 0x5632c436f500 2024-05-08T18:05:03.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 20 remove_watcher() i=7 2024-05-08T18:05:03.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632c436f500 msgr2=0x5632c43718f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632c436f500 0x5632c43718f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632c436ab10 msgr2=0x5632c436cfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632c436ab10 0x5632c436cfd0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6be4074040 msgr2=0x7f6be4081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6be4074040 0x7f6be4081410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632c4373eb0 msgr2=0x5632c4376370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632c4373eb0 0x5632c4376370 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6abc03bba0 msgr2=0x7f6abc03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6abc03bba0 0x7f6abc03e060 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6be4052fd0 tx=0x7f6be404f000 comp rx=0 tx=0).stop 2024-05-08T18:05:03.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4358040 msgr2=0x5632c435f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf1461640 1 -- 172.21.15.50:0/2192629846 reap_dead start 2024-05-08T18:05:03.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4358040 0x5632c435f380 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6be004dfe0 tx=0x7f6be002c530 comp rx=0 tx=0).stop 2024-05-08T18:05:03.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 shutdown_connections 2024-05-08T18:05:03.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/2192629846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4358040 0x5632c435f380 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 >> 172.21.15.50:0/2192629846 conn(0x5632c43542c0 msgr2=0x5632c4351920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 shutdown_connections 2024-05-08T18:05:03.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.197+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2192629846 wait complete. 2024-05-08T18:05:03.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2190203340 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ac803bb50 msgr2=0x7f6ac803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/2190203340 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ac803bb50 0x7f6ac803e010 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x5632c4330fa0 tx=0x7f6be4005ac0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2190203340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c434b030 msgr2=0x5632c434b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/2190203340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c434b030 0x5632c434b450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6be0051170 tx=0x7f6be001aed0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2190203340 shutdown_connections 2024-05-08T18:05:03.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/2190203340 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ac803bb50 0x7f6ac803e010 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/2190203340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c434b030 0x5632c434b450 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2190203340 >> 172.21.15.50:0/2190203340 conn(0x5632c433f0d0 msgr2=0x5632c433f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2190203340 shutdown_connections 2024-05-08T18:05:03.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/2190203340 wait complete. 2024-05-08T18:05:03.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/421374029 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632c4336390 msgr2=0x5632c4338780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632c4336390 0x5632c4338780 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/421374029 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632c4235c90 msgr2=0x5632c4333ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632c4235c90 0x5632c4333ab0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/421374029 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632c433a280 msgr2=0x5632c433c670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632c433a280 0x5632c433c670 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/421374029 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ad803bba0 msgr2=0x7f6ad803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6ad803bba0 0x7f6ad803e060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6be4054000 tx=0x7f6be404e000 comp rx=0 tx=0).stop 2024-05-08T18:05:03.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/421374029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4232180 msgr2=0x5632c432c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 --2- 172.21.15.50:0/421374029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632c4232180 0x5632c432c870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6be000b5f0 tx=0x7f6be0000bc0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf1461640 1 -- 172.21.15.50:0/421374029 reap_dead start 2024-05-08T18:05:03.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/421374029 shutdown_connections 2024-05-08T18:05:03.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/421374029 >> 172.21.15.50:0/421374029 conn(0x5632c422bb50 msgr2=0x5632c4230960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/421374029 shutdown_connections 2024-05-08T18:05:03.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.198+0000 7f6bf4f2e800 1 -- 172.21.15.50:0/421374029 wait complete. 2024-05-08T18:05:03.210 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-05-08T18:05:03.211 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-05-08T18:05:03.211 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-05-08T18:05:03.256 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:03.256 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:03.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.259+0000 7f8901a1d800 1 Processor -- start 2024-05-08T18:05:03.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.259+0000 7f8901a1d800 1 -- start start 2024-05-08T18:05:03.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.259+0000 7f8901a1d800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55c8550f4700 0x55c855184210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.259+0000 7f8901a1d800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55c8551884d0 con 0x55c855184780 2024-05-08T18:05:03.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.259+0000 7f8901a1d800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55c855021e20 con 0x55c8550f4700 2024-05-08T18:05:03.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.259+0000 7f88ff515640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55c8550f4700 0x55c855184210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.259+0000 7f88fed14640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55c855184780 0x55c855186c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:52998/0 (socket says 172.21.15.50:52998) 2024-05-08T18:05:03.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.259+0000 7f88fed14640 1 -- 172.21.15.50:0/1065347803 learned_addr learned my addr 172.21.15.50:0/1065347803 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:03.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.260+0000 7f88ff515640 1 -- 172.21.15.50:0/1065347803 >> v1:172.21.15.46:6789/0 conn(0x55c855184780 legacy=0x55c855186c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.260+0000 7f88ff515640 1 -- 172.21.15.50:0/1065347803 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c855189700 con 0x55c8550f4700 2024-05-08T18:05:03.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.261+0000 7f88ff515640 1 --2- 172.21.15.50:0/1065347803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 0x55c855184210 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f88f000a470 tx=0x7f88f001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1d3279874790be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.262+0000 7f88fe513640 1 -- 172.21.15.50:0/1065347803 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88f0057070 con 0x55c8550f4700 2024-05-08T18:05:03.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.262+0000 7f88fe513640 1 -- 172.21.15.50:0/1065347803 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f0052570 con 0x55c8550f4700 2024-05-08T18:05:03.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.262+0000 7f88fe513640 1 -- 172.21.15.50:0/1065347803 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88f0052870 con 0x55c8550f4700 2024-05-08T18:05:03.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.262+0000 7f8901a1d800 1 -- 172.21.15.50:0/1065347803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 msgr2=0x55c855184210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.262+0000 7f8901a1d800 1 --2- 172.21.15.50:0/1065347803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 0x55c855184210 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f88f000a470 tx=0x7f88f001ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:03.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.262+0000 7f8901a1d800 1 -- 172.21.15.50:0/1065347803 shutdown_connections 2024-05-08T18:05:03.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.262+0000 7f8901a1d800 1 --2- 172.21.15.50:0/1065347803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 0x55c855184210 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.262+0000 7f8901a1d800 1 -- 172.21.15.50:0/1065347803 >> 172.21.15.50:0/1065347803 conn(0x55c85517bb50 msgr2=0x55c85517df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.262+0000 7f8901a1d800 1 -- 172.21.15.50:0/1065347803 shutdown_connections 2024-05-08T18:05:03.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.262+0000 7f8901a1d800 1 -- 172.21.15.50:0/1065347803 wait complete. 2024-05-08T18:05:03.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.278+0000 7f8901a1d800 1 Processor -- start 2024-05-08T18:05:03.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.278+0000 7f8901a1d800 1 -- start start 2024-05-08T18:05:03.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.278+0000 7f8901a1d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 0x55c855272b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.278+0000 7f8901a1d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c8550cca10 con 0x55c8550f4700 2024-05-08T18:05:03.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.278+0000 7f88ff515640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 0x55c855272b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.278+0000 7f88ff515640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 0x55c855272b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57740/0 (socket says 172.21.15.50:57740) 2024-05-08T18:05:03.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.278+0000 7f88ff515640 1 -- 172.21.15.50:0/2486921843 learned_addr learned my addr 172.21.15.50:0/2486921843 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.279+0000 7f88ff515640 1 -- 172.21.15.50:0/2486921843 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c855273100 con 0x55c8550f4700 2024-05-08T18:05:03.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.279+0000 7f88ff515640 1 --2- 172.21.15.50:0/2486921843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 0x55c855272b30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f88f0055040 tx=0x7f88f004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db7b9471aeb854e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.279+0000 7f87ed60a640 1 -- 172.21.15.50:0/2486921843 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88f0057070 con 0x55c8550f4700 2024-05-08T18:05:03.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.279+0000 7f87ed60a640 1 -- 172.21.15.50:0/2486921843 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f0063070 con 0x55c8550f4700 2024-05-08T18:05:03.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.279+0000 7f87ed60a640 1 -- 172.21.15.50:0/2486921843 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88f0052720 con 0x55c8550f4700 2024-05-08T18:05:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.280+0000 7f8901a1d800 1 -- 172.21.15.50:0/2486921843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 msgr2=0x55c855272b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.280+0000 7f8901a1d800 1 --2- 172.21.15.50:0/2486921843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 0x55c855272b30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f88f0055040 tx=0x7f88f004c000 comp rx=0 tx=0).stop 2024-05-08T18:05:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.280+0000 7f8901a1d800 1 -- 172.21.15.50:0/2486921843 shutdown_connections 2024-05-08T18:05:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.280+0000 7f8901a1d800 1 --2- 172.21.15.50:0/2486921843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 0x55c855272b30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.280+0000 7f8901a1d800 1 -- 172.21.15.50:0/2486921843 >> 172.21.15.50:0/2486921843 conn(0x55c85517bb50 msgr2=0x55c85526e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.280+0000 7f8901a1d800 1 -- 172.21.15.50:0/2486921843 shutdown_connections 2024-05-08T18:05:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.280+0000 7f8901a1d800 1 -- 172.21.15.50:0/2486921843 wait complete. 2024-05-08T18:05:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.280+0000 7f8901a1d800 1 Processor -- start 2024-05-08T18:05:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.280+0000 7f8901a1d800 1 -- start start 2024-05-08T18:05:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.280+0000 7f8901a1d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 0x55c85527a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.280+0000 7f8901a1d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c855188940 con 0x55c8550f4700 2024-05-08T18:05:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.281+0000 7f88ff515640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 0x55c85527a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.281+0000 7f88ff515640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 0x55c85527a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57744/0 (socket says 172.21.15.50:57744) 2024-05-08T18:05:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.281+0000 7f88ff515640 1 -- 172.21.15.50:0/4075553783 learned_addr learned my addr 172.21.15.50:0/4075553783 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.281+0000 7f88ff515640 1 -- 172.21.15.50:0/4075553783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c85527aa30 con 0x55c8550f4700 2024-05-08T18:05:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.281+0000 7f88ff515640 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 0x55c85527a4f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f88f0055040 tx=0x7f88f0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.281+0000 7f87ebe07640 1 -- 172.21.15.50:0/4075553783 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88f005e480 con 0x55c8550f4700 2024-05-08T18:05:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.281+0000 7f87ebe07640 1 -- 172.21.15.50:0/4075553783 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f006c070 con 0x55c8550f4700 2024-05-08T18:05:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.281+0000 7f8901a1d800 1 -- 172.21.15.50:0/4075553783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c85527acc0 con 0x55c8550f4700 2024-05-08T18:05:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.281+0000 7f87ebe07640 1 -- 172.21.15.50:0/4075553783 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88f005e750 con 0x55c8550f4700 2024-05-08T18:05:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.281+0000 7f8901a1d800 1 -- 172.21.15.50:0/4075553783 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c85527d970 con 0x55c8550f4700 2024-05-08T18:05:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.282+0000 7f87ebe07640 1 -- 172.21.15.50:0/4075553783 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f88f005e8f0 con 0x55c8550f4700 2024-05-08T18:05:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.282+0000 7f87ebe07640 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f87e403bb50 0x7f87e403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.283+0000 7f87ebe07640 1 -- 172.21.15.50:0/4075553783 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f88f0095090 con 0x55c8550f4700 2024-05-08T18:05:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.283+0000 7f88fed14640 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f87e403bb50 0x7f87e403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.283+0000 7f8901a1d800 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c855184780 0x55c855281730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.283+0000 7f8901a1d800 1 -- 172.21.15.50:0/4075553783 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c855281cb0 con 0x55c855184780 2024-05-08T18:05:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.283+0000 7f88ffd16640 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c855184780 0x55c855281730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.283+0000 7f88fed14640 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f87e403bb50 0x7f87e403e010 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f88e8001d70 tx=0x7f88e80012d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.283+0000 7f88ffd16640 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c855184780 0x55c855281730 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.283+0000 7f88ffd16640 1 -- 172.21.15.50:0/4075553783 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c855184780 2024-05-08T18:05:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.283+0000 7f8901a1d800 1 -- 172.21.15.50:0/4075553783 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c855282570 con 0x55c855184780 2024-05-08T18:05:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.284+0000 7f88ffd16640 1 -- 172.21.15.50:0/4075553783 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c855184780 2024-05-08T18:05:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.284+0000 7f8901a1d800 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c855283e80 0x55c855286320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.284+0000 7f8901a1d800 1 -- 172.21.15.50:0/4075553783 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c855286ab0 con 0x55c855283e80 2024-05-08T18:05:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.284+0000 7f88ff515640 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c855283e80 0x55c855286320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.284+0000 7f88ff515640 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c855283e80 0x55c855286320 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.285+0000 7f88ff515640 1 -- 172.21.15.50:0/4075553783 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f88f0071470 con 0x55c855283e80 2024-05-08T18:05:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.285+0000 7f8901a1d800 1 -- 172.21.15.50:0/4075553783 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c855282dc0 con 0x55c855184780 2024-05-08T18:05:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.285+0000 7f88ffd16640 1 -- 172.21.15.50:0/4075553783 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c855184780 2024-05-08T18:05:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.285+0000 7f8901a1d800 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c855287f10 0x55c85528a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.285+0000 7f8901a1d800 1 -- 172.21.15.50:0/4075553783 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c85528aa40 con 0x55c855287f10 2024-05-08T18:05:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.285+0000 7f88fed14640 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c855287f10 0x55c85528a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.285+0000 7f88fed14640 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c855287f10 0x55c85528a300 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.286+0000 7f88fed14640 1 -- 172.21.15.50:0/4075553783 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f88e8009050 con 0x55c855287f10 2024-05-08T18:05:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.286+0000 7f8901a1d800 1 Processor -- start 2024-05-08T18:05:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.286+0000 7f8901a1d800 1 -- start start 2024-05-08T18:05:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.286+0000 7f8901a1d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855298d70 0x55c855299190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.286+0000 7f8901a1d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c855073330 con 0x55c855298d70 2024-05-08T18:05:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.287+0000 7f88ff515640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855298d70 0x55c855299190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.287+0000 7f88ff515640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855298d70 0x55c855299190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57760/0 (socket says 172.21.15.50:57760) 2024-05-08T18:05:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.287+0000 7f88ff515640 1 -- 172.21.15.50:0/2218749589 learned_addr learned my addr 172.21.15.50:0/2218749589 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.287+0000 7f88ff515640 1 -- 172.21.15.50:0/2218749589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8552996d0 con 0x55c855298d70 2024-05-08T18:05:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.287+0000 7f88ff515640 1 --2- 172.21.15.50:0/2218749589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855298d70 0x55c855299190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f88f0060040 tx=0x7f88f0009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.287+0000 7f87e9602640 1 -- 172.21.15.50:0/2218749589 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88f0066910 con 0x55c855298d70 2024-05-08T18:05:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.287+0000 7f87e9602640 1 -- 172.21.15.50:0/2218749589 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f0058040 con 0x55c855298d70 2024-05-08T18:05:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.287+0000 7f87e9602640 1 -- 172.21.15.50:0/2218749589 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88f0066be0 con 0x55c855298d70 2024-05-08T18:05:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.287+0000 7f8901a1d800 1 -- 172.21.15.50:0/2218749589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c85529a140 con 0x55c855298d70 2024-05-08T18:05:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.287+0000 7f8901a1d800 1 -- 172.21.15.50:0/2218749589 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c85529a140 con 0x55c855298d70 2024-05-08T18:05:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.288+0000 7f87e9602640 1 -- 172.21.15.50:0/2218749589 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f88f008c360 con 0x55c855298d70 2024-05-08T18:05:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.288+0000 7f87e9602640 1 --2- 172.21.15.50:0/2218749589 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f87d803bba0 0x7f87d803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.288+0000 7f87e9602640 1 -- 172.21.15.50:0/2218749589 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f88f00073e0 con 0x55c855298d70 2024-05-08T18:05:03.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.288+0000 7f88fed14640 1 --2- 172.21.15.50:0/2218749589 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f87d803bba0 0x7f87d803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.289+0000 7f8901a1d800 1 Processor -- start 2024-05-08T18:05:03.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.289+0000 7f8901a1d800 1 -- start start 2024-05-08T18:05:03.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.289+0000 7f88fed14640 1 --2- 172.21.15.50:0/2218749589 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f87d803bba0 0x7f87d803e060 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f88e800bf40 tx=0x7f88e800f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.289+0000 7f8901a1d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855181870 0x55c855181c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.289+0000 7f8901a1d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c8552733c0 con 0x55c855181870 2024-05-08T18:05:03.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.289+0000 7f88ff515640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855181870 0x55c855181c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.289+0000 7f88ff515640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855181870 0x55c855181c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57772/0 (socket says 172.21.15.50:57772) 2024-05-08T18:05:03.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.289+0000 7f88ff515640 1 -- 172.21.15.50:0/1096052747 learned_addr learned my addr 172.21.15.50:0/1096052747 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.289+0000 7f88ff515640 1 -- 172.21.15.50:0/1096052747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c855180080 con 0x55c855181870 2024-05-08T18:05:03.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.289+0000 7f88ff515640 1 --2- 172.21.15.50:0/1096052747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855181870 0x55c855181c90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f88f0066770 tx=0x7f88f0084f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3192d6627851662 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.289+0000 7f87e2ffd640 1 -- 172.21.15.50:0/1096052747 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88f00a6070 con 0x55c855181870 2024-05-08T18:05:03.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.289+0000 7f87e2ffd640 1 -- 172.21.15.50:0/1096052747 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f0095e40 con 0x55c855181870 2024-05-08T18:05:03.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.289+0000 7f87e2ffd640 1 -- 172.21.15.50:0/1096052747 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88f0096d20 con 0x55c855181870 2024-05-08T18:05:03.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.290+0000 7f8901a1d800 1 -- 172.21.15.50:0/1096052747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855181870 msgr2=0x55c855181c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.290+0000 7f8901a1d800 1 --2- 172.21.15.50:0/1096052747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855181870 0x55c855181c90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f88f0066770 tx=0x7f88f0084f30 comp rx=0 tx=0).stop 2024-05-08T18:05:03.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.290+0000 7f8901a1d800 1 -- 172.21.15.50:0/1096052747 shutdown_connections 2024-05-08T18:05:03.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.290+0000 7f8901a1d800 1 --2- 172.21.15.50:0/1096052747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855181870 0x55c855181c90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.290+0000 7f8901a1d800 1 -- 172.21.15.50:0/1096052747 >> 172.21.15.50:0/1096052747 conn(0x55c8552a2040 msgr2=0x55c8552a00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.290+0000 7f8901a1d800 1 -- 172.21.15.50:0/1096052747 shutdown_connections 2024-05-08T18:05:03.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.290+0000 7f8901a1d800 1 -- 172.21.15.50:0/1096052747 wait complete. 2024-05-08T18:05:03.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.290+0000 7f8901a1d800 1 Processor -- start 2024-05-08T18:05:03.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.290+0000 7f8901a1d800 1 -- start start 2024-05-08T18:05:03.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.290+0000 7f8901a1d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855181870 0x55c8552af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.290+0000 7f8901a1d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c855180490 con 0x55c855181870 2024-05-08T18:05:03.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.291+0000 7f88ff515640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855181870 0x55c8552af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.291+0000 7f88ff515640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855181870 0x55c8552af280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57776/0 (socket says 172.21.15.50:57776) 2024-05-08T18:05:03.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.291+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 learned_addr learned my addr 172.21.15.50:0/3186077643 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.291+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8552af7c0 con 0x55c855181870 2024-05-08T18:05:03.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.291+0000 7f88ff515640 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855181870 0x55c8552af280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f88f0068dc0 tx=0x7f88f0085f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.291+0000 7f87e17fa640 1 -- 172.21.15.50:0/3186077643 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88f00b8030 con 0x55c855181870 2024-05-08T18:05:03.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.291+0000 7f87e17fa640 1 -- 172.21.15.50:0/3186077643 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88f0096d20 con 0x55c855181870 2024-05-08T18:05:03.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.291+0000 7f87e17fa640 1 -- 172.21.15.50:0/3186077643 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88f00953b0 con 0x55c855181870 2024-05-08T18:05:03.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.291+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c8552af7c0 con 0x55c855181870 2024-05-08T18:05:03.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.291+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8552af7c0 con 0x55c855181870 2024-05-08T18:05:03.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.292+0000 7f87e17fa640 1 -- 172.21.15.50:0/3186077643 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f88f00bb070 con 0x55c855181870 2024-05-08T18:05:03.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.292+0000 7f87e17fa640 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f87c803bb50 0x7f87c803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.292+0000 7f87e17fa640 1 -- 172.21.15.50:0/3186077643 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f88f00b9070 con 0x55c855181870 2024-05-08T18:05:03.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.292+0000 7f88fed14640 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f87c803bb50 0x7f87c803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.293+0000 7f88fed14640 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f87c803bb50 0x7f87c803e010 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f88e800c010 tx=0x7f88e8010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.293+0000 7f8901a1d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.293+0000 7f8901a1d800 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c8552ba9d0 0x55c8552bce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.293+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c8552bd5e0 con 0x55c8552ba9d0 2024-05-08T18:05:03.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.293+0000 7f88ffd16640 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c8552ba9d0 0x55c8552bce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.294+0000 7f88ffd16640 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c8552ba9d0 0x55c8552bce90 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.294+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c8552ba9d0 2024-05-08T18:05:03.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.294+0000 7f8901a1d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.294+0000 7f8901a1d800 20 realm 2024-05-08T18:05:03.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.294+0000 7f8901a1d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.294+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c8552bd9f0 con 0x55c8552ba9d0 2024-05-08T18:05:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.294+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c8552ba9d0 2024-05-08T18:05:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.294+0000 7f8901a1d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.294+0000 7f8901a1d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.294+0000 7f8901a1d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.294+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be2c0 con 0x55c8552ba9d0 2024-05-08T18:05:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.294+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c8552ba9d0 2024-05-08T18:05:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.294+0000 7f8901a1d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.294+0000 7f8901a1d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.294+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552ba9d0 2024-05-08T18:05:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.295+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c8552ba9d0 2024-05-08T18:05:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.295+0000 7f8901a1d800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.295+0000 7f8901a1d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.295+0000 7f8901a1d800 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c8552bf3c0 0x55c8552c17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.295+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c1d70 con 0x55c8552bf3c0 2024-05-08T18:05:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.295+0000 7f88ff515640 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c8552bf3c0 0x55c8552c17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.295+0000 7f88ff515640 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c8552bf3c0 0x55c8552c17b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.295+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.296+0000 7f8901a1d800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.296+0000 7f8901a1d800 20 searching for the correct realm 2024-05-08T18:05:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.296+0000 7f8901a1d800 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c8552c3d00 0x55c8552c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.296+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c6890 con 0x55c8552c3d00 2024-05-08T18:05:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.296+0000 7f88fed14640 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c8552c3d00 0x55c8552c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.296+0000 7f88fed14640 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c8552c3d00 0x55c8552c61c0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.296+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.296+0000 7f88fed14640 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f88e8037040 0x7f88e8036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.296+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c6890 con 0x7f88e8037040 2024-05-08T18:05:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.296+0000 7f88ffd16640 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f88e8037040 0x7f88e8036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.297+0000 7f88ffd16640 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f88e8037040 0x7f88e8036410 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.297+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f800a4a0 con 0x7f88e8037040 2024-05-08T18:05:03.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.297+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552bf3c0 2024-05-08T18:05:03.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.297+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.297+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552ba9d0 2024-05-08T18:05:03.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.297+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c8552ba9d0 2024-05-08T18:05:03.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.297+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552ba9d0 2024-05-08T18:05:03.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.298+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c8552ba9d0 2024-05-08T18:05:03.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.298+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552bf3c0 2024-05-08T18:05:03.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.298+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.298+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552bf3c0 2024-05-08T18:05:03.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.298+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v20'1 uv1 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.298+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x7f88e8037040 2024-05-08T18:05:03.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.298+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f800a4a0 con 0x7f88e8037040 2024-05-08T18:05:03.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.298+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552c3d00 2024-05-08T18:05:03.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.299+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.299+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552ba9d0 2024-05-08T18:05:03.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.299+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c8552ba9d0 2024-05-08T18:05:03.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.299+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552bf3c0 2024-05-08T18:05:03.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.299+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.299+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552ba9d0 2024-05-08T18:05:03.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.299+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c8552ba9d0 2024-05-08T18:05:03.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.300+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x7f88e8037040 2024-05-08T18:05:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.300+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f88f800a4a0 con 0x7f88e8037040 2024-05-08T18:05:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.300+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552c3d00 2024-05-08T18:05:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.300+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.300+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552c3d00 2024-05-08T18:05:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.300+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.300+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552bf3c0 2024-05-08T18:05:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.300+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.301+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552c3d00 2024-05-08T18:05:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.301+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.301+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552c3d00 2024-05-08T18:05:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.301+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.301+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552ba9d0 2024-05-08T18:05:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.301+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c8552ba9d0 2024-05-08T18:05:03.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.301+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552ba9d0 2024-05-08T18:05:03.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.301+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c8552ba9d0 2024-05-08T18:05:03.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.301+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552bf3c0 2024-05-08T18:05:03.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.302+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.302+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552c3d00 2024-05-08T18:05:03.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.302+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.302+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x7f88e8037040 2024-05-08T18:05:03.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.302+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f800a4a0 con 0x7f88e8037040 2024-05-08T18:05:03.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.302+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552ba9d0 2024-05-08T18:05:03.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.302+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c8552ba9d0 2024-05-08T18:05:03.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.302+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552bf3c0 2024-05-08T18:05:03.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.303+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.303+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552c3d00 2024-05-08T18:05:03.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.303+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.303+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552bf3c0 2024-05-08T18:05:03.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.303+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.303+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x7f88e8037040 2024-05-08T18:05:03.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.304+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f800a4a0 con 0x7f88e8037040 2024-05-08T18:05:03.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.304+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552c3d00 2024-05-08T18:05:03.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.304+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.304+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552bf3c0 2024-05-08T18:05:03.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.304+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.304+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x55c8552ba9d0 2024-05-08T18:05:03.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.304+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c8552ba9d0 2024-05-08T18:05:03.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c8552be660 con 0x7f88e8037040 2024-05-08T18:05:03.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f88f800a4a0 con 0x7f88e8037040 2024-05-08T18:05:03.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f8901a1d800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:03.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f8901a1d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:03.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f8901a1d800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:03.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f8901a1d800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:03.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f8901a1d800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:03.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f8901a1d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:03.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f8901a1d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c2630 con 0x55c8552ba9d0 2024-05-08T18:05:03.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c8552ba9d0 2024-05-08T18:05:03.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f8901a1d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f8901a1d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c2630 con 0x55c8552ba9d0 2024-05-08T18:05:03.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f88f800a4a0 con 0x55c8552ba9d0 2024-05-08T18:05:03.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f8901a1d800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:03.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f8901a1d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.305+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c2630 con 0x55c8552c3d00 2024-05-08T18:05:03.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f8901a1d800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:03.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f8901a1d800 20 zone default found 2024-05-08T18:05:03.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f8901a1d800 4 Realm: () 2024-05-08T18:05:03.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f8901a1d800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:03.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f8901a1d800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:03.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f8901a1d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:03.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f8901a1d800 20 zonegroup default 2024-05-08T18:05:03.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f8901a1d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c3010 con 0x55c8552bf3c0 2024-05-08T18:05:03.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f8901a1d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f8901a1d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c8ba0 con 0x55c8552c3d00 2024-05-08T18:05:03.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f8901a1d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f8901a1d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.306+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c8f40 con 0x55c8552c3d00 2024-05-08T18:05:03.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.307+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.307+0000 7f8901a1d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.307+0000 7f8901a1d800 20 started sync module instance, tier type = 2024-05-08T18:05:03.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.307+0000 7f8901a1d800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:03.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.307+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c7fe0 con 0x7f88e8037040 2024-05-08T18:05:03.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.308+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f800a4a0 con 0x7f88e8037040 2024-05-08T18:05:03.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.308+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94318910800896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c7fe0 con 0x7f88e8037040 2024-05-08T18:05:03.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.308+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c9790 con 0x55c8552ba9d0 2024-05-08T18:05:03.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.309+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94318910800896] v27'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x7f88e8037040 2024-05-08T18:05:03.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.309+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x55c8552ba9d0 2024-05-08T18:05:03.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.309+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94318910805184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c7fe0 con 0x55c8552ba9d0 2024-05-08T18:05:03.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.309+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552ca200 con 0x55c8552c3d00 2024-05-08T18:05:03.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.310+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94318910805184] v27'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x55c8552ba9d0 2024-05-08T18:05:03.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.310+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.310+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94318910798640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c7f50 con 0x55c8552c3d00 2024-05-08T18:05:03.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.310+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552cb0c0 con 0x7f88e8037040 2024-05-08T18:05:03.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.311+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94318910798640] v27'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.312+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x7f88e8037040 2024-05-08T18:05:03.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.312+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94318910811152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552cb4e0 con 0x7f88e8037040 2024-05-08T18:05:03.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.312+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552cbea0 con 0x55c8552ba9d0 2024-05-08T18:05:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.313+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x55c8552ba9d0 2024-05-08T18:05:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.313+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94318910814704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552cc2c0 con 0x55c8552ba9d0 2024-05-08T18:05:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.313+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552cc920 con 0x55c8552bf3c0 2024-05-08T18:05:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.313+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94318910811152] v27'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x7f88e8037040 2024-05-08T18:05:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.314+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94318910814704] v27'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x55c8552ba9d0 2024-05-08T18:05:03.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.314+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.314+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94318910812208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552ccd40 con 0x55c8552bf3c0 2024-05-08T18:05:03.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.314+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552cd590 con 0x55c8552bf3c0 2024-05-08T18:05:03.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.315+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.315+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94318910821088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552cddb0 con 0x55c8552bf3c0 2024-05-08T18:05:03.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.315+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552ce8d0 con 0x55c8552bf3c0 2024-05-08T18:05:03.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.315+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94318910812208] v27'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.316+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94318910821088] v27'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.316+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.316+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94318910825376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552cd800 con 0x55c8552bf3c0 2024-05-08T18:05:03.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.316+0000 7f8901a1d800 20 add_watcher() i=0 2024-05-08T18:05:03.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.316+0000 7f8901a1d800 20 add_watcher() i=1 2024-05-08T18:05:03.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.316+0000 7f8901a1d800 20 add_watcher() i=2 2024-05-08T18:05:03.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.316+0000 7f8901a1d800 20 add_watcher() i=3 2024-05-08T18:05:03.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.316+0000 7f8901a1d800 20 add_watcher() i=4 2024-05-08T18:05:03.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.316+0000 7f8901a1d800 20 add_watcher() i=5 2024-05-08T18:05:03.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.316+0000 7f8901a1d800 20 add_watcher() i=6 2024-05-08T18:05:03.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.317+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94318910825376] v27'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.317+0000 7f8901a1d800 20 add_watcher() i=7 2024-05-08T18:05:03.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.317+0000 7f8901a1d800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:03.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.317+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c8552d18c0 con 0x55c8552ba9d0 2024-05-08T18:05:03.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.317+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f88f8040040 con 0x55c8552ba9d0 2024-05-08T18:05:03.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.317+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94318910817328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552ced40 con 0x55c8552ba9d0 2024-05-08T18:05:03.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.318+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94318910817328] v27'41 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x55c8552ba9d0 2024-05-08T18:05:03.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.318+0000 7f87b5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:03.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.319+0000 7f8901a1d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:03.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.319+0000 7f8901a1d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:03.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.319+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552da100 con 0x55c8552ba9d0 2024-05-08T18:05:03.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.320+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x55c8552ba9d0 2024-05-08T18:05:03.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.320+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552da750 con 0x55c8552bf3c0 2024-05-08T18:05:03.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.321+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'13 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.321+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552da750 con 0x55c8552c3d00 2024-05-08T18:05:03.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.322+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.322+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552db230 con 0x55c8552ba9d0 2024-05-08T18:05:03.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.323+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'42 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x55c8552ba9d0 2024-05-08T18:05:03.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.323+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552d7f80 con 0x7f88e8037040 2024-05-08T18:05:03.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.324+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'29 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x7f88e8037040 2024-05-08T18:05:03.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.324+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552d7f50 con 0x55c8552c3d00 2024-05-08T18:05:03.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.325+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'19 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.325+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dbf60 con 0x55c8552ba9d0 2024-05-08T18:05:03.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.326+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'15 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x55c8552ba9d0 2024-05-08T18:05:03.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.326+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dbff0 con 0x55c8552bf3c0 2024-05-08T18:05:03.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.327+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'19 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.327+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dc0a0 con 0x7f88e8037040 2024-05-08T18:05:03.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.328+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x7f88e8037040 2024-05-08T18:05:03.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.328+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dc170 con 0x55c8552c3d00 2024-05-08T18:05:03.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.329+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.329+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dc5a0 con 0x7f88e8037040 2024-05-08T18:05:03.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.330+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'30 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x7f88e8037040 2024-05-08T18:05:03.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.330+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dc5a0 con 0x55c8552bf3c0 2024-05-08T18:05:03.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.332+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'15 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.332+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dc5a0 con 0x7f88e8037040 2024-05-08T18:05:03.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.332+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x7f88e8037040 2024-05-08T18:05:03.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.333+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dc5a0 con 0x55c8552bf3c0 2024-05-08T18:05:03.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.334+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'25 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.334+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dc5a0 con 0x55c8552ba9d0 2024-05-08T18:05:03.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.335+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'43 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x55c8552ba9d0 2024-05-08T18:05:03.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.335+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dc770 con 0x55c8552c3d00 2024-05-08T18:05:03.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.336+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'23 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.336+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dcc70 con 0x7f88e8037040 2024-05-08T18:05:03.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.337+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'31 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x7f88e8037040 2024-05-08T18:05:03.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.337+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dcc70 con 0x55c8552c3d00 2024-05-08T18:05:03.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.338+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'13 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.338+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dcc70 con 0x55c8552ba9d0 2024-05-08T18:05:03.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.339+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x55c8552ba9d0 2024-05-08T18:05:03.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.339+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dcc70 con 0x55c8552ba9d0 2024-05-08T18:05:03.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.340+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'44 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x55c8552ba9d0 2024-05-08T18:05:03.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.340+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dcc70 con 0x55c8552c3d00 2024-05-08T18:05:03.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.341+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'24 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.341+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dcc70 con 0x7f88e8037040 2024-05-08T18:05:03.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.342+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'23 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x7f88e8037040 2024-05-08T18:05:03.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.342+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dcc70 con 0x55c8552c3d00 2024-05-08T18:05:03.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.343+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.343+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dcc70 con 0x55c8552c3d00 2024-05-08T18:05:03.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.344+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.344+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dcc70 con 0x55c8552c3d00 2024-05-08T18:05:03.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.345+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'9 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.345+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dcc70 con 0x55c8552c3d00 2024-05-08T18:05:03.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.346+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'20 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.346+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dcc70 con 0x55c8552bf3c0 2024-05-08T18:05:03.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.348+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'15 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.348+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dcc70 con 0x55c8552c3d00 2024-05-08T18:05:03.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.349+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'13 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.349+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dcc70 con 0x55c8552bf3c0 2024-05-08T18:05:03.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.350+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'20 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.350+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dcc70 con 0x55c8552c3d00 2024-05-08T18:05:03.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.351+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'25 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.351+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dcc70 con 0x55c8552bf3c0 2024-05-08T18:05:03.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.352+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'26 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.352+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552dcc70 con 0x7f88e8037040 2024-05-08T18:05:03.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.353+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x7f88e8037040 2024-05-08T18:05:03.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.353+0000 7f87b4fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:03.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.353+0000 7f8901a1d800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:03.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.354+0000 7f8901a1d800 20 Filter name: none 2024-05-08T18:05:03.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.354+0000 7f872e7fc640 20 reqs_thread_entry: start 2024-05-08T18:05:03.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.354+0000 7f8901a1d800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:03.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.354+0000 7f8901a1d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.354+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c855307cf0 con 0x55c8552ba9d0 2024-05-08T18:05:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.354+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(94 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x55c8552ba9d0 2024-05-08T18:05:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.354+0000 7f8901a1d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.354+0000 7f8901a1d800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2024-05-08T18:05:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.354+0000 7f8901a1d800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:03.358 INFO:teuthology.orchestra.run.smithi050.stderr:could not remove user: unable to remove user, user does not exist 2024-05-08T18:05:03.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.355+0000 7f87b5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:03.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.355+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94318910817328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552d6f30 con 0x55c8552ba9d0 2024-05-08T18:05:03.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.356+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 94318910817328] v27'45 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x55c8552ba9d0 2024-05-08T18:05:03.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.356+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94318910800896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552cc430 con 0x7f88e8037040 2024-05-08T18:05:03.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.357+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 94318910800896] v27'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x7f88e8037040 2024-05-08T18:05:03.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.357+0000 7f8901a1d800 20 remove_watcher() i=0 2024-05-08T18:05:03.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.357+0000 7f8901a1d800 2 removed watcher, disabling cache 2024-05-08T18:05:03.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.357+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94318910805184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c8400 con 0x55c8552ba9d0 2024-05-08T18:05:03.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.358+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 94318910805184] v27'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x55c8552ba9d0 2024-05-08T18:05:03.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.358+0000 7f8901a1d800 20 remove_watcher() i=1 2024-05-08T18:05:03.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.358+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94318910798640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c8400 con 0x55c8552c3d00 2024-05-08T18:05:03.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.359+0000 7f88fed14640 1 -- 172.21.15.50:0/3186077643 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 94318910798640] v27'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88e8009050 con 0x55c8552c3d00 2024-05-08T18:05:03.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.359+0000 7f8901a1d800 20 remove_watcher() i=2 2024-05-08T18:05:03.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.359+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94318910811152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c7b30 con 0x7f88e8037040 2024-05-08T18:05:03.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.360+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 94318910811152] v27'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x7f88e8037040 2024-05-08T18:05:03.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.360+0000 7f8901a1d800 20 remove_watcher() i=3 2024-05-08T18:05:03.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.360+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94318910814704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c8400 con 0x55c8552ba9d0 2024-05-08T18:05:03.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.361+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 94318910814704] v27'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f8040040 con 0x55c8552ba9d0 2024-05-08T18:05:03.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.361+0000 7f8901a1d800 20 remove_watcher() i=4 2024-05-08T18:05:03.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.361+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94318910812208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552cac10 con 0x55c8552bf3c0 2024-05-08T18:05:03.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.362+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 94318910812208] v27'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.362+0000 7f8901a1d800 20 remove_watcher() i=5 2024-05-08T18:05:03.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.362+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94318910821088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552c7b30 con 0x55c8552bf3c0 2024-05-08T18:05:03.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.363+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 94318910821088] v27'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.363+0000 7f8901a1d800 20 remove_watcher() i=6 2024-05-08T18:05:03.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.363+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94318910825376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c8552cb9f0 con 0x55c8552bf3c0 2024-05-08T18:05:03.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.364+0000 7f88ff515640 1 -- 172.21.15.50:0/3186077643 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 94318910825376] v27'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f88f0071470 con 0x55c8552bf3c0 2024-05-08T18:05:03.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.364+0000 7f8901a1d800 20 remove_watcher() i=7 2024-05-08T18:05:03.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.364+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c8552bf3c0 msgr2=0x55c8552c17b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.364+0000 7f8901a1d800 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c8552bf3c0 0x55c8552c17b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.364+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c8552ba9d0 msgr2=0x55c8552bce90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.364+0000 7f8901a1d800 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c8552ba9d0 0x55c8552bce90 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.364+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f88e8037040 msgr2=0x7f88e8036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.364+0000 7f8901a1d800 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f88e8037040 0x7f88e8036410 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.364+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c8552c3d00 msgr2=0x55c8552c61c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.364+0000 7f8901a1d800 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c8552c3d00 0x55c8552c61c0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.364+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f87c803bb50 msgr2=0x7f87c803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.364+0000 7f8901a1d800 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f87c803bb50 0x7f87c803e010 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f88e800c010 tx=0x7f88e8010000 comp rx=0 tx=0).stop 2024-05-08T18:05:03.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.364+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855181870 msgr2=0x55c8552af280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.364+0000 7f8901a1d800 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855181870 0x55c8552af280 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f88f0068dc0 tx=0x7f88f0085f10 comp rx=0 tx=0).stop 2024-05-08T18:05:03.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.364+0000 7f88ffd16640 1 -- 172.21.15.50:0/3186077643 reap_dead start 2024-05-08T18:05:03.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 shutdown_connections 2024-05-08T18:05:03.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 --2- 172.21.15.50:0/3186077643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855181870 0x55c8552af280 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 >> 172.21.15.50:0/3186077643 conn(0x55c8552a2040 msgr2=0x55c85529f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 shutdown_connections 2024-05-08T18:05:03.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 -- 172.21.15.50:0/3186077643 wait complete. 2024-05-08T18:05:03.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 -- 172.21.15.50:0/2218749589 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f87d803bba0 msgr2=0x7f87d803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 --2- 172.21.15.50:0/2218749589 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f87d803bba0 0x7f87d803e060 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f88e800bf40 tx=0x7f88e800f000 comp rx=0 tx=0).stop 2024-05-08T18:05:03.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 -- 172.21.15.50:0/2218749589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855298d70 msgr2=0x55c855299190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 --2- 172.21.15.50:0/2218749589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855298d70 0x55c855299190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f88f0060040 tx=0x7f88f0009850 comp rx=0 tx=0).stop 2024-05-08T18:05:03.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 -- 172.21.15.50:0/2218749589 shutdown_connections 2024-05-08T18:05:03.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 --2- 172.21.15.50:0/2218749589 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f87d803bba0 0x7f87d803e060 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 --2- 172.21.15.50:0/2218749589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c855298d70 0x55c855299190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 -- 172.21.15.50:0/2218749589 >> 172.21.15.50:0/2218749589 conn(0x55c85528ce10 msgr2=0x55c85528d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 -- 172.21.15.50:0/2218749589 shutdown_connections 2024-05-08T18:05:03.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 -- 172.21.15.50:0/2218749589 wait complete. 2024-05-08T18:05:03.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 -- 172.21.15.50:0/4075553783 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c855283e80 msgr2=0x55c855286320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c855283e80 0x55c855286320 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 -- 172.21.15.50:0/4075553783 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c855184780 msgr2=0x55c855281730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c855184780 0x55c855281730 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 -- 172.21.15.50:0/4075553783 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c855287f10 msgr2=0x55c85528a300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.365+0000 7f8901a1d800 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c855287f10 0x55c85528a300 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.366+0000 7f8901a1d800 1 -- 172.21.15.50:0/4075553783 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f87e403bb50 msgr2=0x7f87e403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.366+0000 7f8901a1d800 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f87e403bb50 0x7f87e403e010 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f88e8001d70 tx=0x7f88e80012d0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.366+0000 7f8901a1d800 1 -- 172.21.15.50:0/4075553783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 msgr2=0x55c85527a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.366+0000 7f8901a1d800 1 --2- 172.21.15.50:0/4075553783 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c8550f4700 0x55c85527a4f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f88f0055040 tx=0x7f88f0002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.366+0000 7f88ffd16640 1 -- 172.21.15.50:0/4075553783 reap_dead start 2024-05-08T18:05:03.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.366+0000 7f8901a1d800 1 -- 172.21.15.50:0/4075553783 shutdown_connections 2024-05-08T18:05:03.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.366+0000 7f8901a1d800 1 -- 172.21.15.50:0/4075553783 >> 172.21.15.50:0/4075553783 conn(0x55c85517bb50 msgr2=0x55c85526cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.366+0000 7f8901a1d800 1 -- 172.21.15.50:0/4075553783 shutdown_connections 2024-05-08T18:05:03.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.366+0000 7f8901a1d800 1 -- 172.21.15.50:0/4075553783 wait complete. 2024-05-08T18:05:03.379 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-05-08T18:05:03.379 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-05-08T18:05:03.380 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-05-08T18:05:03.380 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2024-05-08T18:05:03.427 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:03.428 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:03.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.430+0000 7f5b18b33800 1 Processor -- start 2024-05-08T18:05:03.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.430+0000 7f5b18b33800 1 -- start start 2024-05-08T18:05:03.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.431+0000 7f5b18b33800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x560778e23bd0 0x560778e2c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.431+0000 7f5b18b33800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x560778d97100 con 0x560778e22090 2024-05-08T18:05:03.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.431+0000 7f5b18b33800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x560778cc3e20 con 0x560778e23bd0 2024-05-08T18:05:03.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.431+0000 7f5b13fed640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x560778e23bd0 0x560778e2c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.431+0000 7f5b147ee640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x560778e22090 0x560778e224b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:53000/0 (socket says 172.21.15.50:53000) 2024-05-08T18:05:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.431+0000 7f5b147ee640 1 -- 172.21.15.50:0/4220745292 learned_addr learned my addr 172.21.15.50:0/4220745292 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.431+0000 7f5b137ec640 1 -- 172.21.15.50:0/4220745292 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1483359403 0 0) 0x560778d97100 con 0x560778e22090 2024-05-08T18:05:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.431+0000 7f5b137ec640 1 -- 172.21.15.50:0/4220745292 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5afc001e10 con 0x560778e22090 2024-05-08T18:05:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.431+0000 7f5b13fed640 1 -- 172.21.15.50:0/4220745292 >> v1:172.21.15.46:6789/0 conn(0x560778e22090 legacy=0x560778e224b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.431+0000 7f5b13fed640 1 -- 172.21.15.50:0/4220745292 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x560778d972d0 con 0x560778e23bd0 2024-05-08T18:05:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.433+0000 7f5b13fed640 1 --2- 172.21.15.50:0/4220745292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e23bd0 0x560778e2c6d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800a470 tx=0x7f5b0801cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853538ed3d860fb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.433+0000 7f5b137ec640 1 -- 172.21.15.50:0/4220745292 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b08057070 con 0x560778e23bd0 2024-05-08T18:05:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.433+0000 7f5b137ec640 1 -- 172.21.15.50:0/4220745292 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b08052570 con 0x560778e23bd0 2024-05-08T18:05:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.433+0000 7f5b137ec640 1 -- 172.21.15.50:0/4220745292 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b08052850 con 0x560778e23bd0 2024-05-08T18:05:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.434+0000 7f5b18b33800 1 -- 172.21.15.50:0/4220745292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e23bd0 msgr2=0x560778e2c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.434+0000 7f5b18b33800 1 --2- 172.21.15.50:0/4220745292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e23bd0 0x560778e2c6d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800a470 tx=0x7f5b0801cdb0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.434+0000 7f5b18b33800 1 -- 172.21.15.50:0/4220745292 shutdown_connections 2024-05-08T18:05:03.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.434+0000 7f5b18b33800 1 --2- 172.21.15.50:0/4220745292 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e23bd0 0x560778e2c6d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.434+0000 7f5b18b33800 1 -- 172.21.15.50:0/4220745292 >> 172.21.15.50:0/4220745292 conn(0x560778e1dbb0 msgr2=0x560778e20010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.434+0000 7f5b18b33800 1 -- 172.21.15.50:0/4220745292 shutdown_connections 2024-05-08T18:05:03.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.434+0000 7f5b18b33800 1 -- 172.21.15.50:0/4220745292 wait complete. 2024-05-08T18:05:03.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.449+0000 7f5b18b33800 1 Processor -- start 2024-05-08T18:05:03.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.449+0000 7f5b18b33800 1 -- start start 2024-05-08T18:05:03.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.450+0000 7f5b18b33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e22090 0x560778f16c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.450+0000 7f5b18b33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560778d6ea30 con 0x560778e22090 2024-05-08T18:05:03.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.450+0000 7f5b147ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e22090 0x560778f16c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.450+0000 7f5b147ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e22090 0x560778f16c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57786/0 (socket says 172.21.15.50:57786) 2024-05-08T18:05:03.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.450+0000 7f5b147ee640 1 -- 172.21.15.50:0/4290701607 learned_addr learned my addr 172.21.15.50:0/4290701607 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.450+0000 7f5b147ee640 1 -- 172.21.15.50:0/4290701607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560778f171d0 con 0x560778e22090 2024-05-08T18:05:03.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.450+0000 7f5b147ee640 1 --2- 172.21.15.50:0/4290701607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e22090 0x560778f16c00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5b04001de0 tx=0x7f5b0400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67214a16d88aa008 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.451+0000 7f5a0260c640 1 -- 172.21.15.50:0/4290701607 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b04012020 con 0x560778e22090 2024-05-08T18:05:03.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.451+0000 7f5a0260c640 1 -- 172.21.15.50:0/4290701607 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0400e470 con 0x560778e22090 2024-05-08T18:05:03.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.451+0000 7f5a0260c640 1 -- 172.21.15.50:0/4290701607 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b0400e7b0 con 0x560778e22090 2024-05-08T18:05:03.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.451+0000 7f5b18b33800 1 -- 172.21.15.50:0/4290701607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e22090 msgr2=0x560778f16c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.451+0000 7f5b18b33800 1 --2- 172.21.15.50:0/4290701607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e22090 0x560778f16c00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5b04001de0 tx=0x7f5b0400b6c0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.451+0000 7f5b18b33800 1 -- 172.21.15.50:0/4290701607 shutdown_connections 2024-05-08T18:05:03.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.451+0000 7f5b18b33800 1 --2- 172.21.15.50:0/4290701607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e22090 0x560778f16c00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.451+0000 7f5b18b33800 1 -- 172.21.15.50:0/4290701607 >> 172.21.15.50:0/4290701607 conn(0x560778e1dbb0 msgr2=0x560778f12520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.451+0000 7f5b18b33800 1 -- 172.21.15.50:0/4290701607 shutdown_connections 2024-05-08T18:05:03.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.451+0000 7f5b18b33800 1 -- 172.21.15.50:0/4290701607 wait complete. 2024-05-08T18:05:03.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.452+0000 7f5b18b33800 1 Processor -- start 2024-05-08T18:05:03.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.452+0000 7f5b18b33800 1 -- start start 2024-05-08T18:05:03.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.452+0000 7f5b18b33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e22090 0x560778f1e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.452+0000 7f5b18b33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560778e2cee0 con 0x560778e22090 2024-05-08T18:05:03.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.452+0000 7f5b147ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e22090 0x560778f1e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.452+0000 7f5b147ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e22090 0x560778f1e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57788/0 (socket says 172.21.15.50:57788) 2024-05-08T18:05:03.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.452+0000 7f5b147ee640 1 -- 172.21.15.50:0/2528792023 learned_addr learned my addr 172.21.15.50:0/2528792023 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.452+0000 7f5b147ee640 1 -- 172.21.15.50:0/2528792023 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560778f1ece0 con 0x560778e22090 2024-05-08T18:05:03.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.453+0000 7f5b147ee640 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e22090 0x560778f1e7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400b700 tx=0x7f5b040040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.453+0000 7f5a00e09640 1 -- 172.21.15.50:0/2528792023 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b04012030 con 0x560778e22090 2024-05-08T18:05:03.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.453+0000 7f5a00e09640 1 -- 172.21.15.50:0/2528792023 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0401a590 con 0x560778e22090 2024-05-08T18:05:03.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.453+0000 7f5b18b33800 1 -- 172.21.15.50:0/2528792023 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560778f1ef70 con 0x560778e22090 2024-05-08T18:05:03.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.453+0000 7f5a00e09640 1 -- 172.21.15.50:0/2528792023 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b0401a870 con 0x560778e22090 2024-05-08T18:05:03.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.453+0000 7f5b18b33800 1 -- 172.21.15.50:0/2528792023 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560778f21bd0 con 0x560778e22090 2024-05-08T18:05:03.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.453+0000 7f5b18b33800 1 Processor -- start 2024-05-08T18:05:03.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.453+0000 7f5b18b33800 1 -- start start 2024-05-08T18:05:03.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.454+0000 7f5b18b33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f2bde0 0x560778f2c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.454+0000 7f5a00e09640 1 -- 172.21.15.50:0/2528792023 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f5b04015020 con 0x560778e22090 2024-05-08T18:05:03.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.454+0000 7f5b18b33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560778d15330 con 0x560778f2bde0 2024-05-08T18:05:03.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.454+0000 7f5b14fef640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f2bde0 0x560778f2c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.454+0000 7f5b14fef640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f2bde0 0x560778f2c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57796/0 (socket says 172.21.15.50:57796) 2024-05-08T18:05:03.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.454+0000 7f5b14fef640 1 -- 172.21.15.50:0/3800000122 learned_addr learned my addr 172.21.15.50:0/3800000122 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.454+0000 7f5a00e09640 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f59fc03bb50 0x7f59fc03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.454+0000 7f5a00e09640 1 -- 172.21.15.50:0/2528792023 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f5b04020030 con 0x560778e22090 2024-05-08T18:05:03.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.454+0000 7f5b147ee640 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f59fc03bb50 0x7f59fc03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.454+0000 7f5b14fef640 1 -- 172.21.15.50:0/3800000122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560778f2c740 con 0x560778f2bde0 2024-05-08T18:05:03.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.454+0000 7f5b14fef640 1 --2- 172.21.15.50:0/3800000122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f2bde0 0x560778f2c200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00a430 tx=0x7f5b0c00a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5f6350ed67e771 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.454+0000 7f5b147ee640 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f59fc03bb50 0x7f59fc03e010 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5b04051070 tx=0x7f5b04017e70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.454+0000 7f59fa7fc640 1 -- 172.21.15.50:0/3800000122 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b0c015070 con 0x560778f2bde0 2024-05-08T18:05:03.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.454+0000 7f59fa7fc640 1 -- 172.21.15.50:0/3800000122 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0c001670 con 0x560778f2bde0 2024-05-08T18:05:03.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.454+0000 7f59fa7fc640 1 -- 172.21.15.50:0/3800000122 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b0c001990 con 0x560778f2bde0 2024-05-08T18:05:03.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.455+0000 7f5b18b33800 1 -- 172.21.15.50:0/3800000122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f2bde0 msgr2=0x560778f2c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.455+0000 7f5b18b33800 1 --2- 172.21.15.50:0/3800000122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f2bde0 0x560778f2c200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00a430 tx=0x7f5b0c00a8f0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.455+0000 7f5b18b33800 1 -- 172.21.15.50:0/3800000122 shutdown_connections 2024-05-08T18:05:03.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.455+0000 7f5b18b33800 1 --2- 172.21.15.50:0/3800000122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f2bde0 0x560778f2c200 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.455+0000 7f5b18b33800 1 -- 172.21.15.50:0/3800000122 >> 172.21.15.50:0/3800000122 conn(0x560778e23bd0 msgr2=0x560778f28880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.455+0000 7f5b18b33800 1 -- 172.21.15.50:0/3800000122 shutdown_connections 2024-05-08T18:05:03.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.455+0000 7f5b18b33800 1 -- 172.21.15.50:0/3800000122 wait complete. 2024-05-08T18:05:03.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.455+0000 7f5b18b33800 1 Processor -- start 2024-05-08T18:05:03.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.455+0000 7f5b18b33800 1 -- start start 2024-05-08T18:05:03.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.455+0000 7f5b18b33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f2bde0 0x560778f33340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.455+0000 7f5b18b33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560778f2d0d0 con 0x560778f2bde0 2024-05-08T18:05:03.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.456+0000 7f5b14fef640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f2bde0 0x560778f33340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.456+0000 7f5b14fef640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f2bde0 0x560778f33340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57800/0 (socket says 172.21.15.50:57800) 2024-05-08T18:05:03.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.456+0000 7f5b14fef640 1 -- 172.21.15.50:0/934779061 learned_addr learned my addr 172.21.15.50:0/934779061 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.456+0000 7f5b14fef640 1 -- 172.21.15.50:0/934779061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560778f33880 con 0x560778f2bde0 2024-05-08T18:05:03.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.456+0000 7f5b14fef640 1 --2- 172.21.15.50:0/934779061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f2bde0 0x560778f33340 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00a930 tx=0x7f5b0c003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.456+0000 7f59f8ff9640 1 -- 172.21.15.50:0/934779061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b0c020020 con 0x560778f2bde0 2024-05-08T18:05:03.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.456+0000 7f59f8ff9640 1 -- 172.21.15.50:0/934779061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b0c0108b0 con 0x560778f2bde0 2024-05-08T18:05:03.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.456+0000 7f59f8ff9640 1 -- 172.21.15.50:0/934779061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b0c015040 con 0x560778f2bde0 2024-05-08T18:05:03.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.456+0000 7f5b18b33800 1 -- 172.21.15.50:0/934779061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560778f33b10 con 0x560778f2bde0 2024-05-08T18:05:03.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.456+0000 7f5b18b33800 1 -- 172.21.15.50:0/934779061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560778f367c0 con 0x560778f2bde0 2024-05-08T18:05:03.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.457+0000 7f59f8ff9640 1 -- 172.21.15.50:0/934779061 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f5b0c023020 con 0x560778f2bde0 2024-05-08T18:05:03.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.458+0000 7f59f8ff9640 1 --2- 172.21.15.50:0/934779061 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f59ec03bba0 0x7f59ec03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.458+0000 7f59f8ff9640 1 -- 172.21.15.50:0/934779061 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f5b0c0522f0 con 0x560778f2bde0 2024-05-08T18:05:03.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.458+0000 7f5b13fed640 1 --2- 172.21.15.50:0/934779061 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f59ec03bba0 0x7f59ec03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.458+0000 7f5b18b33800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:03.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.458+0000 7f5b18b33800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:03.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.458+0000 7f5b18b33800 1 Processor -- start 2024-05-08T18:05:03.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.458+0000 7f5b18b33800 1 -- start start 2024-05-08T18:05:03.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.458+0000 7f5b13fed640 1 --2- 172.21.15.50:0/934779061 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f59ec03bba0 0x7f59ec03e060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5b08054000 tx=0x7f5b0804e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.458+0000 7f5b18b33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f4d9b0 0x560778f4ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.458+0000 7f5b18b33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560778f17550 con 0x560778f4d9b0 2024-05-08T18:05:03.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.459+0000 7f5b147ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f4d9b0 0x560778f4ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.459+0000 7f5b147ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f4d9b0 0x560778f4ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57806/0 (socket says 172.21.15.50:57806) 2024-05-08T18:05:03.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.459+0000 7f5b147ee640 1 -- 172.21.15.50:0/550651480 learned_addr learned my addr 172.21.15.50:0/550651480 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.459+0000 7f5b147ee640 1 -- 172.21.15.50:0/550651480 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560778f4e310 con 0x560778f4d9b0 2024-05-08T18:05:03.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.459+0000 7f5b147ee640 1 --2- 172.21.15.50:0/550651480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f4d9b0 0x560778f4ddd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5b04000fb0 tx=0x7f5b0400d530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.459+0000 7f59d97e2640 1 -- 172.21.15.50:0/550651480 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b04041070 con 0x560778f4d9b0 2024-05-08T18:05:03.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.459+0000 7f59d97e2640 1 -- 172.21.15.50:0/550651480 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b04018950 con 0x560778f4d9b0 2024-05-08T18:05:03.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.459+0000 7f5b18b33800 1 -- 172.21.15.50:0/550651480 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560778f4ed80 con 0x560778f4d9b0 2024-05-08T18:05:03.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.459+0000 7f59d97e2640 1 -- 172.21.15.50:0/550651480 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5b04018c30 con 0x560778f4d9b0 2024-05-08T18:05:03.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.459+0000 7f5b18b33800 1 -- 172.21.15.50:0/550651480 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560778f4f280 con 0x560778f4d9b0 2024-05-08T18:05:03.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.460+0000 7f59d97e2640 1 -- 172.21.15.50:0/550651480 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f5b04044070 con 0x560778f4d9b0 2024-05-08T18:05:03.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.460+0000 7f59d97e2640 1 --2- 172.21.15.50:0/550651480 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f594c03bb00 0x7f594c03dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.460+0000 7f59d97e2640 1 -- 172.21.15.50:0/550651480 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f5b04047cb0 con 0x560778f4d9b0 2024-05-08T18:05:03.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.460+0000 7f5b13fed640 1 --2- 172.21.15.50:0/550651480 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f594c03bb00 0x7f594c03dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.461+0000 7f59d2ffd640 20 reqs_thread_entry: start 2024-05-08T18:05:03.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.461+0000 7f5b18b33800 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x560778f5b3b0 0x560778f5d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.461+0000 7f5b18b33800 1 -- 172.21.15.50:0/2528792023 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560778f5ded0 con 0x560778f5b3b0 2024-05-08T18:05:03.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.461+0000 7f5b13fed640 1 --2- 172.21.15.50:0/550651480 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f594c03bb00 0x7f594c03dfc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5b0801cdf0 tx=0x7f5b08005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.461+0000 7f5b14fef640 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x560778f5b3b0 0x560778f5d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.461+0000 7f5b14fef640 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x560778f5b3b0 0x560778f5d800 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.461+0000 7f5b14fef640 1 -- 172.21.15.50:0/2528792023 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5b0c010d70 con 0x560778f5b3b0 2024-05-08T18:05:03.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.461+0000 7f5b18b33800 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x560778f5fcf0 0x560778f620e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.461+0000 7f5b18b33800 1 -- 172.21.15.50:0/2528792023 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:2 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560778f627b0 con 0x560778f5fcf0 2024-05-08T18:05:03.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.462+0000 7f5b147ee640 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x560778f5fcf0 0x560778f620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.462+0000 7f5b147ee640 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x560778f5fcf0 0x560778f620e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.462+0000 7f5b147ee640 1 -- 172.21.15.50:0/2528792023 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(2 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f5b0404d070 con 0x560778f5fcf0 2024-05-08T18:05:03.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.462+0000 7f5b18b33800 1 -- 172.21.15.50:0/2528792023 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [stat,call version.check_conds in=122b,call version.inc in=38b,writefull 0~993 in=993b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560778f67640 con 0x560778f5fcf0 2024-05-08T18:05:03.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.464+0000 7f5b147ee640 1 -- 172.21.15.50:0/2528792023 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [stat,call,call,writefull 0~993] v27'2 uv2 ondisk = 0) ==== 316+0+0 (crc 0 0 0) 0x7f5b0404d070 con 0x560778f5fcf0 2024-05-08T18:05:03.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.464+0000 7f5b18b33800 1 -- 172.21.15.50:0/934779061 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f59ec03bba0 msgr2=0x7f59ec03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.464+0000 7f5b18b33800 1 --2- 172.21.15.50:0/934779061 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f59ec03bba0 0x7f59ec03e060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5b08054000 tx=0x7f5b0804e000 comp rx=0 tx=0).stop 2024-05-08T18:05:03.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.464+0000 7f5b18b33800 1 -- 172.21.15.50:0/934779061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f2bde0 msgr2=0x560778f33340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.464+0000 7f5b18b33800 1 --2- 172.21.15.50:0/934779061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f2bde0 0x560778f33340 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00a930 tx=0x7f5b0c003370 comp rx=0 tx=0).stop 2024-05-08T18:05:03.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.464+0000 7f5b18b33800 1 -- 172.21.15.50:0/934779061 shutdown_connections 2024-05-08T18:05:03.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.464+0000 7f5b18b33800 1 --2- 172.21.15.50:0/934779061 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f59ec03bba0 0x7f59ec03e060 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.464+0000 7f5b18b33800 1 --2- 172.21.15.50:0/934779061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f2bde0 0x560778f33340 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.464+0000 7f5b18b33800 1 -- 172.21.15.50:0/934779061 >> 172.21.15.50:0/934779061 conn(0x560778e23bd0 msgr2=0x560778f25ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 -- 172.21.15.50:0/934779061 shutdown_connections 2024-05-08T18:05:03.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 -- 172.21.15.50:0/934779061 wait complete. 2024-05-08T18:05:03.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 -- 172.21.15.50:0/550651480 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f594c03bb00 msgr2=0x7f594c03dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 --2- 172.21.15.50:0/550651480 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f594c03bb00 0x7f594c03dfc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5b0801cdf0 tx=0x7f5b08005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:03.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 -- 172.21.15.50:0/550651480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f4d9b0 msgr2=0x560778f4ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 --2- 172.21.15.50:0/550651480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f4d9b0 0x560778f4ddd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5b04000fb0 tx=0x7f5b0400d530 comp rx=0 tx=0).stop 2024-05-08T18:05:03.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 -- 172.21.15.50:0/550651480 shutdown_connections 2024-05-08T18:05:03.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 --2- 172.21.15.50:0/550651480 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f594c03bb00 0x7f594c03dfc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 --2- 172.21.15.50:0/550651480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778f4d9b0 0x560778f4ddd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 -- 172.21.15.50:0/550651480 >> 172.21.15.50:0/550651480 conn(0x560778f42460 msgr2=0x560778f42e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 -- 172.21.15.50:0/550651480 shutdown_connections 2024-05-08T18:05:03.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 -- 172.21.15.50:0/550651480 wait complete. 2024-05-08T18:05:03.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 -- 172.21.15.50:0/2528792023 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x560778f5fcf0 msgr2=0x560778f620e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x560778f5fcf0 0x560778f620e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 -- 172.21.15.50:0/2528792023 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x560778f5b3b0 msgr2=0x560778f5d800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x560778f5b3b0 0x560778f5d800 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 -- 172.21.15.50:0/2528792023 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f59fc03bb50 msgr2=0x7f59fc03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f59fc03bb50 0x7f59fc03e010 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5b04051070 tx=0x7f5b04017e70 comp rx=0 tx=0).stop 2024-05-08T18:05:03.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 -- 172.21.15.50:0/2528792023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e22090 msgr2=0x560778f1e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e22090 0x560778f1e7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5b0400b700 tx=0x7f5b040040e0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 -- 172.21.15.50:0/2528792023 shutdown_connections 2024-05-08T18:05:03.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x560778f5fcf0 0x560778f620e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x560778f5b3b0 0x560778f5d800 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f59fc03bb50 0x7f59fc03e010 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 --2- 172.21.15.50:0/2528792023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560778e22090 0x560778f1e7a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 -- 172.21.15.50:0/2528792023 >> 172.21.15.50:0/2528792023 conn(0x560778e1dbb0 msgr2=0x560778f10d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.465+0000 7f5b18b33800 1 -- 172.21.15.50:0/2528792023 shutdown_connections 2024-05-08T18:05:03.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.466+0000 7f5b18b33800 1 -- 172.21.15.50:0/2528792023 wait complete. 2024-05-08T18:05:03.472 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"1c041ffd-e40a-486a-88e5-3547b85128ea","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-05-08T18:05:03.473 DEBUG:tasks.util.rgw: json result: {'id': '1c041ffd-e40a-486a-88e5-3547b85128ea', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-05-08T18:05:03.474 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2024-05-08T18:05:03.474 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2024-05-08T18:05:03.474 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2024-05-08T18:05:03.517 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:03.518 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:03.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.520+0000 7f7f73535800 1 Processor -- start 2024-05-08T18:05:03.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.520+0000 7f7f73535800 1 -- start start 2024-05-08T18:05:03.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.521+0000 7f7f73535800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ec1171c170 0x55ec1171e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.521+0000 7f7f73535800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55ec117225b0 con 0x55ec1171eaa0 2024-05-08T18:05:03.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.521+0000 7f7f73535800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55ec115bbe20 con 0x55ec1171c170 2024-05-08T18:05:03.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.521+0000 7f7f6f1ae640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ec1171c170 0x55ec1171e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.521+0000 7f7f6e9ad640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55ec1171eaa0 0x55ec11720e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:53014/0 (socket says 172.21.15.50:53014) 2024-05-08T18:05:03.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.521+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/3538488331 learned_addr learned my addr 172.21.15.50:0/3538488331 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:03.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.521+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/3538488331 >> v1:172.21.15.46:6789/0 conn(0x55ec1171eaa0 legacy=0x55ec11720e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.521+0000 7f7f6e1ac640 1 -- 172.21.15.50:0/3538488331 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 325903428 0 0) 0x55ec117225b0 con 0x55ec1171eaa0 2024-05-08T18:05:03.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.521+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/3538488331 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec11723790 con 0x55ec1171c170 2024-05-08T18:05:03.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.523+0000 7f7f6f1ae640 1 --2- 172.21.15.50:0/3538488331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 0x55ec1171e560 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000a470 tx=0x7f7f6001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1333041ca8029f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.523+0000 7f7f6e1ac640 1 -- 172.21.15.50:0/3538488331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f60057070 con 0x55ec1171c170 2024-05-08T18:05:03.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.523+0000 7f7f6e1ac640 1 -- 172.21.15.50:0/3538488331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f60052570 con 0x55ec1171c170 2024-05-08T18:05:03.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.523+0000 7f7f6e1ac640 1 -- 172.21.15.50:0/3538488331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f60052870 con 0x55ec1171c170 2024-05-08T18:05:03.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.524+0000 7f7f73535800 1 -- 172.21.15.50:0/3538488331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 msgr2=0x55ec1171e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.524+0000 7f7f73535800 1 --2- 172.21.15.50:0/3538488331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 0x55ec1171e560 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7f6000a470 tx=0x7f7f6001ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:03.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.524+0000 7f7f73535800 1 -- 172.21.15.50:0/3538488331 shutdown_connections 2024-05-08T18:05:03.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.524+0000 7f7f73535800 1 --2- 172.21.15.50:0/3538488331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 0x55ec1171e560 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.524+0000 7f7f73535800 1 -- 172.21.15.50:0/3538488331 >> 172.21.15.50:0/3538488331 conn(0x55ec11715ac0 msgr2=0x55ec11717f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.524+0000 7f7f73535800 1 -- 172.21.15.50:0/3538488331 shutdown_connections 2024-05-08T18:05:03.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.524+0000 7f7f73535800 1 -- 172.21.15.50:0/3538488331 wait complete. 2024-05-08T18:05:03.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.539+0000 7f7f73535800 1 Processor -- start 2024-05-08T18:05:03.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.539+0000 7f7f73535800 1 -- start start 2024-05-08T18:05:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.539+0000 7f7f73535800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 0x55ec1171bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.539+0000 7f7f73535800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec11666a00 con 0x55ec1171c170 2024-05-08T18:05:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.539+0000 7f7f6f1ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 0x55ec1171bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.539+0000 7f7f6f1ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 0x55ec1171bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57826/0 (socket says 172.21.15.50:57826) 2024-05-08T18:05:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.539+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/727857864 learned_addr learned my addr 172.21.15.50:0/727857864 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.540+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/727857864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec1171a070 con 0x55ec1171c170 2024-05-08T18:05:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.540+0000 7f7f6f1ae640 1 --2- 172.21.15.50:0/727857864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 0x55ec1171bab0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7f60055040 tx=0x7f7f6004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3216ad0d151c636f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.540+0000 7f7e5d60a640 1 -- 172.21.15.50:0/727857864 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f60057070 con 0x55ec1171c170 2024-05-08T18:05:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.540+0000 7f7e5d60a640 1 -- 172.21.15.50:0/727857864 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f60063070 con 0x55ec1171c170 2024-05-08T18:05:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.540+0000 7f7e5d60a640 1 -- 172.21.15.50:0/727857864 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f60052720 con 0x55ec1171c170 2024-05-08T18:05:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.541+0000 7f7f73535800 1 -- 172.21.15.50:0/727857864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 msgr2=0x55ec1171bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.541+0000 7f7f73535800 1 --2- 172.21.15.50:0/727857864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 0x55ec1171bab0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7f60055040 tx=0x7f7f6004c000 comp rx=0 tx=0).stop 2024-05-08T18:05:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.541+0000 7f7f73535800 1 -- 172.21.15.50:0/727857864 shutdown_connections 2024-05-08T18:05:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.541+0000 7f7f73535800 1 --2- 172.21.15.50:0/727857864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 0x55ec1171bab0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.541+0000 7f7f73535800 1 -- 172.21.15.50:0/727857864 >> 172.21.15.50:0/727857864 conn(0x55ec11715ac0 msgr2=0x55ec118082c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.541+0000 7f7f73535800 1 -- 172.21.15.50:0/727857864 shutdown_connections 2024-05-08T18:05:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.541+0000 7f7f73535800 1 -- 172.21.15.50:0/727857864 wait complete. 2024-05-08T18:05:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.541+0000 7f7f73535800 1 Processor -- start 2024-05-08T18:05:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.541+0000 7f7f73535800 1 -- start start 2024-05-08T18:05:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.541+0000 7f7f73535800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 0x55ec118165f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.541+0000 7f7f73535800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec11722a80 con 0x55ec1171c170 2024-05-08T18:05:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.542+0000 7f7f6f1ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 0x55ec118165f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.542+0000 7f7f6f1ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 0x55ec118165f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57834/0 (socket says 172.21.15.50:57834) 2024-05-08T18:05:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.542+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/609574955 learned_addr learned my addr 172.21.15.50:0/609574955 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.542+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/609574955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec11816b30 con 0x55ec1171c170 2024-05-08T18:05:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.542+0000 7f7f6f1ae640 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 0x55ec118165f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7f60060040 tx=0x7f7f60002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.542+0000 7f7e5be07640 1 -- 172.21.15.50:0/609574955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f60052e30 con 0x55ec1171c170 2024-05-08T18:05:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.542+0000 7f7e5be07640 1 -- 172.21.15.50:0/609574955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f6006c070 con 0x55ec1171c170 2024-05-08T18:05:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.543+0000 7f7f73535800 1 -- 172.21.15.50:0/609574955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec11816dc0 con 0x55ec1171c170 2024-05-08T18:05:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.543+0000 7f7e5be07640 1 -- 172.21.15.50:0/609574955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f6005e5f0 con 0x55ec1171c170 2024-05-08T18:05:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.543+0000 7f7f73535800 1 -- 172.21.15.50:0/609574955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec11819a70 con 0x55ec1171c170 2024-05-08T18:05:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.543+0000 7f7e5be07640 1 -- 172.21.15.50:0/609574955 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7f6005e790 con 0x55ec1171c170 2024-05-08T18:05:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.544+0000 7f7e5be07640 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e5403bb50 0x7f7e5403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.544+0000 7f7e5be07640 1 -- 172.21.15.50:0/609574955 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7f60094e20 con 0x55ec1171c170 2024-05-08T18:05:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.544+0000 7f7f6e9ad640 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e5403bb50 0x7f7e5403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.544+0000 7f7f73535800 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec1171eaa0 0x55ec1181d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.544+0000 7f7f73535800 1 -- 172.21.15.50:0/609574955 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1181ddb0 con 0x55ec1171eaa0 2024-05-08T18:05:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.544+0000 7f7f6f9af640 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec1171eaa0 0x55ec1181d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.544+0000 7f7f6e9ad640 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e5403bb50 0x7f7e5403e010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7f64000c00 tx=0x7f7f640013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.544+0000 7f7f6f9af640 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec1171eaa0 0x55ec1181d830 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.544+0000 7f7f6f9af640 1 -- 172.21.15.50:0/609574955 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec1171eaa0 2024-05-08T18:05:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.545+0000 7f7f73535800 1 -- 172.21.15.50:0/609574955 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1181ade0 con 0x55ec1171eaa0 2024-05-08T18:05:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.545+0000 7f7f6f9af640 1 -- 172.21.15.50:0/609574955 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec1171eaa0 2024-05-08T18:05:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.545+0000 7f7f73535800 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec118200a0 0x55ec11822490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.545+0000 7f7f73535800 1 -- 172.21.15.50:0/609574955 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec11822b80 con 0x55ec118200a0 2024-05-08T18:05:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.545+0000 7f7f6f1ae640 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec118200a0 0x55ec11822490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.545+0000 7f7f6f1ae640 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec118200a0 0x55ec11822490 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.546+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/609574955 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7f60009b60 con 0x55ec118200a0 2024-05-08T18:05:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.546+0000 7f7f73535800 1 -- 172.21.15.50:0/609574955 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1181ef40 con 0x55ec1171eaa0 2024-05-08T18:05:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.546+0000 7f7f6f9af640 1 -- 172.21.15.50:0/609574955 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec1171eaa0 2024-05-08T18:05:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.546+0000 7f7f73535800 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec11823fe0 0x55ec118263d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.546+0000 7f7f73535800 1 -- 172.21.15.50:0/609574955 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec11826ad0 con 0x55ec11823fe0 2024-05-08T18:05:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.546+0000 7f7f6e9ad640 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec11823fe0 0x55ec118263d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.546+0000 7f7f6e9ad640 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec11823fe0 0x55ec118263d0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.546+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/609574955 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7f6400f670 con 0x55ec11823fe0 2024-05-08T18:05:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.547+0000 7f7f73535800 1 Processor -- start 2024-05-08T18:05:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.547+0000 7f7f73535800 1 -- start start 2024-05-08T18:05:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.547+0000 7f7f73535800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11834e40 0x55ec11835260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.547+0000 7f7f73535800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec1160d330 con 0x55ec11834e40 2024-05-08T18:05:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.547+0000 7f7f6f1ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11834e40 0x55ec11835260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.547+0000 7f7f6f1ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11834e40 0x55ec11835260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57844/0 (socket says 172.21.15.50:57844) 2024-05-08T18:05:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.547+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/1018658836 learned_addr learned my addr 172.21.15.50:0/1018658836 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.548+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/1018658836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec118357a0 con 0x55ec11834e40 2024-05-08T18:05:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.548+0000 7f7f6f1ae640 1 --2- 172.21.15.50:0/1018658836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11834e40 0x55ec11835260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7f60093210 tx=0x7f7f6005c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.548+0000 7f7e59602640 1 -- 172.21.15.50:0/1018658836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f60089070 con 0x55ec11834e40 2024-05-08T18:05:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.548+0000 7f7e59602640 1 -- 172.21.15.50:0/1018658836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f60058040 con 0x55ec11834e40 2024-05-08T18:05:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.548+0000 7f7e59602640 1 -- 172.21.15.50:0/1018658836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f600095d0 con 0x55ec11834e40 2024-05-08T18:05:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.548+0000 7f7f73535800 1 -- 172.21.15.50:0/1018658836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec11836210 con 0x55ec11834e40 2024-05-08T18:05:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.548+0000 7f7f73535800 1 -- 172.21.15.50:0/1018658836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec11836710 con 0x55ec11834e40 2024-05-08T18:05:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.549+0000 7f7e59602640 1 -- 172.21.15.50:0/1018658836 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7f6008c070 con 0x55ec11834e40 2024-05-08T18:05:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.549+0000 7f7e59602640 1 --2- 172.21.15.50:0/1018658836 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e4803bb50 0x7f7e4803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.549+0000 7f7e59602640 1 -- 172.21.15.50:0/1018658836 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7f6007b050 con 0x55ec11834e40 2024-05-08T18:05:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.549+0000 7f7f6e9ad640 1 --2- 172.21.15.50:0/1018658836 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e4803bb50 0x7f7e4803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.549+0000 7f7f73535800 1 Processor -- start 2024-05-08T18:05:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.549+0000 7f7f73535800 1 -- start start 2024-05-08T18:05:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.549+0000 7f7f6e9ad640 1 --2- 172.21.15.50:0/1018658836 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e4803bb50 0x7f7e4803e010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55ec1181e650 tx=0x7f7f64006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.550+0000 7f7f73535800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11841fc0 0x55ec118423e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.550+0000 7f7f73535800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec1171a330 con 0x55ec11841fc0 2024-05-08T18:05:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.550+0000 7f7f6f1ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11841fc0 0x55ec118423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.550+0000 7f7f6f1ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11841fc0 0x55ec118423e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57848/0 (socket says 172.21.15.50:57848) 2024-05-08T18:05:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.550+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/391163324 learned_addr learned my addr 172.21.15.50:0/391163324 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.550+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/391163324 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec118429b0 con 0x55ec11841fc0 2024-05-08T18:05:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.550+0000 7f7f6f1ae640 1 --2- 172.21.15.50:0/391163324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11841fc0 0x55ec118423e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7f600936a0 tx=0x7f7f600936d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae30c713501739b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.550+0000 7f7e52ffd640 1 -- 172.21.15.50:0/391163324 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f6005cdd0 con 0x55ec11841fc0 2024-05-08T18:05:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.550+0000 7f7e52ffd640 1 -- 172.21.15.50:0/391163324 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f6006bca0 con 0x55ec11841fc0 2024-05-08T18:05:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.550+0000 7f7e52ffd640 1 -- 172.21.15.50:0/391163324 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f60088170 con 0x55ec11841fc0 2024-05-08T18:05:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.551+0000 7f7f73535800 1 -- 172.21.15.50:0/391163324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11841fc0 msgr2=0x55ec118423e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.551+0000 7f7f73535800 1 --2- 172.21.15.50:0/391163324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11841fc0 0x55ec118423e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7f600936a0 tx=0x7f7f600936d0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.551+0000 7f7f73535800 1 -- 172.21.15.50:0/391163324 shutdown_connections 2024-05-08T18:05:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.551+0000 7f7f73535800 1 --2- 172.21.15.50:0/391163324 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11841fc0 0x55ec118423e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.551+0000 7f7f73535800 1 -- 172.21.15.50:0/391163324 >> 172.21.15.50:0/391163324 conn(0x55ec1183e2d0 msgr2=0x55ec1183c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.551+0000 7f7f73535800 1 -- 172.21.15.50:0/391163324 shutdown_connections 2024-05-08T18:05:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.551+0000 7f7f73535800 1 -- 172.21.15.50:0/391163324 wait complete. 2024-05-08T18:05:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.551+0000 7f7f73535800 1 Processor -- start 2024-05-08T18:05:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.551+0000 7f7f73535800 1 -- start start 2024-05-08T18:05:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.551+0000 7f7f73535800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11841fc0 0x55ec11849410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.551+0000 7f7f73535800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec11843040 con 0x55ec11841fc0 2024-05-08T18:05:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.552+0000 7f7f6f1ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11841fc0 0x55ec11849410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.552+0000 7f7f6f1ae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11841fc0 0x55ec11849410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57850/0 (socket says 172.21.15.50:57850) 2024-05-08T18:05:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.552+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 learned_addr learned my addr 172.21.15.50:0/2527091964 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.552+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec11849950 con 0x55ec11841fc0 2024-05-08T18:05:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.552+0000 7f7f6f1ae640 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11841fc0 0x55ec11849410 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7f60068010 tx=0x7f7f6006ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.552+0000 7f7e517fa640 1 -- 172.21.15.50:0/2527091964 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f600b8070 con 0x55ec11841fc0 2024-05-08T18:05:03.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.552+0000 7f7e517fa640 1 -- 172.21.15.50:0/2527091964 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f600ad040 con 0x55ec11841fc0 2024-05-08T18:05:03.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.552+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec11849950 con 0x55ec11841fc0 2024-05-08T18:05:03.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.552+0000 7f7e517fa640 1 -- 172.21.15.50:0/2527091964 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f60071ad0 con 0x55ec11841fc0 2024-05-08T18:05:03.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.552+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec11849950 con 0x55ec11841fc0 2024-05-08T18:05:03.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.553+0000 7f7e517fa640 1 -- 172.21.15.50:0/2527091964 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7f600bb070 con 0x55ec11841fc0 2024-05-08T18:05:03.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.553+0000 7f7e517fa640 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e3803bb50 0x7f7e3803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.553+0000 7f7e517fa640 1 -- 172.21.15.50:0/2527091964 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7f600a3030 con 0x55ec11841fc0 2024-05-08T18:05:03.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.553+0000 7f7f6e9ad640 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e3803bb50 0x7f7e3803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.554+0000 7f7f73535800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.554+0000 7f7f73535800 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec11854ac0 0x55ec11856f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.554+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec118576d0 con 0x55ec11854ac0 2024-05-08T18:05:03.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.554+0000 7f7f6f9af640 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec11854ac0 0x55ec11856f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.554+0000 7f7f6e9ad640 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e3803bb50 0x7f7e3803e010 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400ffd0 tx=0x7f7f64023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.554+0000 7f7f6f9af640 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec11854ac0 0x55ec11856f80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.554+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec11854ac0 2024-05-08T18:05:03.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.554+0000 7f7f73535800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.554+0000 7f7f73535800 20 realm 2024-05-08T18:05:03.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.554+0000 7f7f73535800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.554+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec11857ae0 con 0x55ec11854ac0 2024-05-08T18:05:03.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.555+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec11854ac0 2024-05-08T18:05:03.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.555+0000 7f7f73535800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.555+0000 7f7f73535800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:03.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.555+0000 7f7f73535800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.555+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec118583b0 con 0x55ec11854ac0 2024-05-08T18:05:03.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.555+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec11854ac0 2024-05-08T18:05:03.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.555+0000 7f7f73535800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.555+0000 7f7f73535800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.555+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11854ac0 2024-05-08T18:05:03.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.555+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec11854ac0 2024-05-08T18:05:03.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.555+0000 7f7f73535800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:03.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.555+0000 7f7f73535800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.555+0000 7f7f73535800 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec11859240 0x55ec1185b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.555+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1185bd90 con 0x55ec11859240 2024-05-08T18:05:03.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.555+0000 7f7f6f1ae640 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec11859240 0x55ec1185b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.556+0000 7f7f6f1ae640 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec11859240 0x55ec1185b630 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.556+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.556+0000 7f7f73535800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:03.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.556+0000 7f7f73535800 20 searching for the correct realm 2024-05-08T18:05:03.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.556+0000 7f7f73535800 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec1185de40 0x55ec11860300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.556+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec118609d0 con 0x55ec1185de40 2024-05-08T18:05:03.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.556+0000 7f7f6e9ad640 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec1185de40 0x55ec11860300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.556+0000 7f7f6e9ad640 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec1185de40 0x55ec11860300 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.557+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.557+0000 7f7f6e9ad640 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7f6402e910 0x7f7f6403a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.557+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec118609d0 con 0x7f7f6402e910 2024-05-08T18:05:03.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.557+0000 7f7f6f9af640 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7f6402e910 0x7f7f6403a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.557+0000 7f7f6f9af640 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7f6402e910 0x7f7f6403a3c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.557+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f6800a4a0 con 0x7f7f6402e910 2024-05-08T18:05:03.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.557+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11859240 2024-05-08T18:05:03.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.558+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.558+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11854ac0 2024-05-08T18:05:03.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.558+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec11854ac0 2024-05-08T18:05:03.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.558+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11854ac0 2024-05-08T18:05:03.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.558+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec11854ac0 2024-05-08T18:05:03.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.558+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11859240 2024-05-08T18:05:03.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.559+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.559+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11859240 2024-05-08T18:05:03.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.559+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.559+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x7f7f6402e910 2024-05-08T18:05:03.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.559+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f6800a4a0 con 0x7f7f6402e910 2024-05-08T18:05:03.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.559+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec1185de40 2024-05-08T18:05:03.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.559+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.559+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11854ac0 2024-05-08T18:05:03.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.560+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec11854ac0 2024-05-08T18:05:03.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.560+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11859240 2024-05-08T18:05:03.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.560+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.560+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11854ac0 2024-05-08T18:05:03.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.560+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec11854ac0 2024-05-08T18:05:03.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.560+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x7f7f6402e910 2024-05-08T18:05:03.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.560+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7f6800a4a0 con 0x7f7f6402e910 2024-05-08T18:05:03.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.560+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec1185de40 2024-05-08T18:05:03.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.561+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.561+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec1185de40 2024-05-08T18:05:03.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.561+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.561+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11859240 2024-05-08T18:05:03.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.561+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.561+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec1185de40 2024-05-08T18:05:03.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.561+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.561+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec1185de40 2024-05-08T18:05:03.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.562+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.562+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11854ac0 2024-05-08T18:05:03.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.562+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec11854ac0 2024-05-08T18:05:03.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.562+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11854ac0 2024-05-08T18:05:03.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.562+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec11854ac0 2024-05-08T18:05:03.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.562+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11859240 2024-05-08T18:05:03.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.562+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.562+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec1185de40 2024-05-08T18:05:03.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.563+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.563+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x7f7f6402e910 2024-05-08T18:05:03.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.563+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f6800a4a0 con 0x7f7f6402e910 2024-05-08T18:05:03.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.563+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11854ac0 2024-05-08T18:05:03.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.563+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec11854ac0 2024-05-08T18:05:03.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.563+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11859240 2024-05-08T18:05:03.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.563+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.564+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec1185de40 2024-05-08T18:05:03.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.564+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.564+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11859240 2024-05-08T18:05:03.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.564+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.564+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x7f7f6402e910 2024-05-08T18:05:03.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.564+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f6800a4a0 con 0x7f7f6402e910 2024-05-08T18:05:03.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.564+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec1185de40 2024-05-08T18:05:03.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.564+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.564+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11859240 2024-05-08T18:05:03.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.565+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.565+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x55ec11854ac0 2024-05-08T18:05:03.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.565+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec11854ac0 2024-05-08T18:05:03.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.565+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ec11858750 con 0x7f7f6402e910 2024-05-08T18:05:03.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.565+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f6800a4a0 con 0x7f7f6402e910 2024-05-08T18:05:03.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.565+0000 7f7f73535800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:03.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.565+0000 7f7f73535800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:03.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.565+0000 7f7f73535800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:03.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.565+0000 7f7f73535800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:03.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.565+0000 7f7f73535800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:03.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.565+0000 7f7f73535800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:03.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.565+0000 7f7f73535800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.565+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1185c740 con 0x55ec11854ac0 2024-05-08T18:05:03.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec11854ac0 2024-05-08T18:05:03.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1185c740 con 0x55ec11854ac0 2024-05-08T18:05:03.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7f6800a4a0 con 0x55ec11854ac0 2024-05-08T18:05:03.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:03.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1185c710 con 0x55ec1185de40 2024-05-08T18:05:03.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:03.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 20 zone default found 2024-05-08T18:05:03.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 4 Realm: () 2024-05-08T18:05:03.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:03.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:03.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:03.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 20 zonegroup default 2024-05-08T18:05:03.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1185cfe0 con 0x55ec11859240 2024-05-08T18:05:03.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.566+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec11862830 con 0x55ec1185de40 2024-05-08T18:05:03.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.567+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.567+0000 7f7f73535800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.567+0000 7f7f73535800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.567+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec11862bd0 con 0x55ec1185de40 2024-05-08T18:05:03.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.567+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.567+0000 7f7f73535800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.567+0000 7f7f73535800 20 started sync module instance, tier type = 2024-05-08T18:05:03.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.567+0000 7f7f73535800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:03.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.567+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11861cd0 con 0x7f7f6402e910 2024-05-08T18:05:03.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.568+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f6800a4a0 con 0x7f7f6402e910 2024-05-08T18:05:03.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.568+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94472394645968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11861cd0 con 0x7f7f6402e910 2024-05-08T18:05:03.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.568+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11863620 con 0x55ec11854ac0 2024-05-08T18:05:03.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.569+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94472394645968] v27'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x7f7f6402e910 2024-05-08T18:05:03.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.569+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.569+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94472394649872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11861cd0 con 0x55ec11854ac0 2024-05-08T18:05:03.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.569+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec118640e0 con 0x55ec1185de40 2024-05-08T18:05:03.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.570+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.570+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94472394652592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11864980 con 0x55ec1185de40 2024-05-08T18:05:03.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.570+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11865460 con 0x7f7f6402e910 2024-05-08T18:05:03.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.571+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94472394649872] v27'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.571+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94472394652592] v27'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.572+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x7f7f6402e910 2024-05-08T18:05:03.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.572+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94472394654928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11861820 con 0x7f7f6402e910 2024-05-08T18:05:03.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.572+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11865d00 con 0x55ec11854ac0 2024-05-08T18:05:03.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.573+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.573+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94472394659840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11865c20 con 0x55ec11854ac0 2024-05-08T18:05:03.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.573+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11866c00 con 0x55ec11859240 2024-05-08T18:05:03.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.573+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94472394654928] v27'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x7f7f6402e910 2024-05-08T18:05:03.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.574+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94472394659840] v27'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.574+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.574+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94472394662128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11866710 con 0x55ec11859240 2024-05-08T18:05:03.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.574+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec118674a0 con 0x55ec11859240 2024-05-08T18:05:03.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.575+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.575+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94472394666448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11867c20 con 0x55ec11859240 2024-05-08T18:05:03.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.575+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11868720 con 0x55ec11859240 2024-05-08T18:05:03.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.575+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94472394662128] v27'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.576+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94472394666448] v27'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.576+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.576+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94472394670608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11867670 con 0x55ec11859240 2024-05-08T18:05:03.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.576+0000 7f7f73535800 20 add_watcher() i=0 2024-05-08T18:05:03.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.576+0000 7f7f73535800 20 add_watcher() i=1 2024-05-08T18:05:03.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.576+0000 7f7f73535800 20 add_watcher() i=2 2024-05-08T18:05:03.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.576+0000 7f7f73535800 20 add_watcher() i=3 2024-05-08T18:05:03.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.576+0000 7f7f73535800 20 add_watcher() i=4 2024-05-08T18:05:03.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.576+0000 7f7f73535800 20 add_watcher() i=5 2024-05-08T18:05:03.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.576+0000 7f7f73535800 20 add_watcher() i=6 2024-05-08T18:05:03.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.577+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94472394670608] v27'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.577+0000 7f7f73535800 20 add_watcher() i=7 2024-05-08T18:05:03.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.577+0000 7f7f73535800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:03.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.577+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1186b710 con 0x55ec11854ac0 2024-05-08T18:05:03.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.577+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.577+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94472394657648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11868b90 con 0x55ec11854ac0 2024-05-08T18:05:03.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.578+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94472394657648] v27'46 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.579+0000 7f7e25fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:03.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.579+0000 7f7f73535800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:03.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.579+0000 7f7f73535800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:03.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.579+0000 7f7f73535800 5 note: GC not initialized 2024-05-08T18:05:03.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.579+0000 7f7e24fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:03.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.579+0000 7f7f73535800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:03.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.579+0000 7f7f73535800 20 Filter name: none 2024-05-08T18:05:03.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.579+0000 7f7d9e7fc640 20 reqs_thread_entry: start 2024-05-08T18:05:03.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.579+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f190 con 0x55ec11854ac0 2024-05-08T18:05:03.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.579+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.580+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f190 con 0x7f7f6402e910 2024-05-08T18:05:03.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.580+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x7f7f6402e910 2024-05-08T18:05:03.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.580+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f150 con 0x55ec11854ac0 2024-05-08T18:05:03.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.580+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.580+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f150 con 0x55ec1185de40 2024-05-08T18:05:03.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.580+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.580+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f660 con 0x55ec1185de40 2024-05-08T18:05:03.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.580+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.580+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f660 con 0x55ec1185de40 2024-05-08T18:05:03.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.581+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.581+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f660 con 0x55ec11854ac0 2024-05-08T18:05:03.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.581+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.581+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f660 con 0x55ec11854ac0 2024-05-08T18:05:03.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.581+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.581+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec1185de40 2024-05-08T18:05:03.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.581+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.581+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec1185de40 2024-05-08T18:05:03.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.582+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.582+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec11859240 2024-05-08T18:05:03.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.582+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(72 usage.10 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.582+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec11854ac0 2024-05-08T18:05:03.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.582+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.582+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x7f7f6402e910 2024-05-08T18:05:03.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.582+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x7f7f6402e910 2024-05-08T18:05:03.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.582+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec1185de40 2024-05-08T18:05:03.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.583+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.583+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec11859240 2024-05-08T18:05:03.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.583+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.583+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec11859240 2024-05-08T18:05:03.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.583+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.583+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec1185de40 2024-05-08T18:05:03.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.583+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.583+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f650 con 0x55ec11859240 2024-05-08T18:05:03.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.584+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(79 usage.17 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.584+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec11854ac0 2024-05-08T18:05:03.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.584+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.584+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec1185de40 2024-05-08T18:05:03.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.584+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.584+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec11859240 2024-05-08T18:05:03.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.585+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.585+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x7f7f6402e910 2024-05-08T18:05:03.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.585+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x7f7f6402e910 2024-05-08T18:05:03.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.585+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec11859240 2024-05-08T18:05:03.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.585+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.585+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec11854ac0 2024-05-08T18:05:03.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.585+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.585+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec11854ac0 2024-05-08T18:05:03.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.585+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.585+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec1185de40 2024-05-08T18:05:03.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.586+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.586+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec11854ac0 2024-05-08T18:05:03.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.586+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.586+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec11854ac0 2024-05-08T18:05:03.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.586+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.586+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec1185de40 2024-05-08T18:05:03.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.586+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.586+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec1185de40 2024-05-08T18:05:03.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.587+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.587+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189f610 con 0x55ec1185de40 2024-05-08T18:05:03.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.587+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.587+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ec118a0000 con 0x55ec11859240 2024-05-08T18:05:03.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.587+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.588+0000 7f7e25fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:03.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.588+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94472394657648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec1189dba0 con 0x55ec11854ac0 2024-05-08T18:05:03.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.589+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94472394657648] v27'47 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.589+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94472394645968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11864f70 con 0x7f7f6402e910 2024-05-08T18:05:03.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.590+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94472394645968] v27'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x7f7f6402e910 2024-05-08T18:05:03.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.590+0000 7f7f73535800 20 remove_watcher() i=0 2024-05-08T18:05:03.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.590+0000 7f7f73535800 2 removed watcher, disabling cache 2024-05-08T18:05:03.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.590+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94472394649872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11864f70 con 0x55ec11854ac0 2024-05-08T18:05:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.591+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 32 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94472394649872] v27'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.591+0000 7f7f73535800 20 remove_watcher() i=1 2024-05-08T18:05:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.591+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94472394652592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11864f70 con 0x55ec1185de40 2024-05-08T18:05:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.592+0000 7f7f6e9ad640 1 -- 172.21.15.50:0/2527091964 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94472394652592] v27'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f64037040 con 0x55ec1185de40 2024-05-08T18:05:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.592+0000 7f7f73535800 20 remove_watcher() i=2 2024-05-08T18:05:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.592+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94472394654928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11864f70 con 0x7f7f6402e910 2024-05-08T18:05:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.593+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94472394654928] v27'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x7f7f6402e910 2024-05-08T18:05:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.593+0000 7f7f73535800 20 remove_watcher() i=3 2024-05-08T18:05:03.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.593+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94472394659840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec118644d0 con 0x55ec11854ac0 2024-05-08T18:05:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.594+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 <== osd.1 v2:172.21.15.50:6808/2141923372 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94472394659840] v27'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f68040040 con 0x55ec11854ac0 2024-05-08T18:05:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.594+0000 7f7f73535800 20 remove_watcher() i=4 2024-05-08T18:05:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.594+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94472394662128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11864f70 con 0x55ec11859240 2024-05-08T18:05:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.595+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94472394662128] v27'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.595+0000 7f7f73535800 20 remove_watcher() i=5 2024-05-08T18:05:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.595+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94472394666448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11865800 con 0x55ec11859240 2024-05-08T18:05:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.596+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94472394666448] v27'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.596+0000 7f7f73535800 20 remove_watcher() i=6 2024-05-08T18:05:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.596+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94472394670608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ec11865800 con 0x55ec11859240 2024-05-08T18:05:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f6f1ae640 1 -- 172.21.15.50:0/2527091964 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94472394670608] v27'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f600ba030 con 0x55ec11859240 2024-05-08T18:05:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f73535800 20 remove_watcher() i=7 2024-05-08T18:05:03.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec11859240 msgr2=0x55ec1185b630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f73535800 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec11859240 0x55ec1185b630 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec11854ac0 msgr2=0x55ec11856f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f73535800 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec11854ac0 0x55ec11856f80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7f6402e910 msgr2=0x7f7f6403a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f73535800 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7f6402e910 0x7f7f6403a3c0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec1185de40 msgr2=0x55ec11860300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f73535800 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec1185de40 0x55ec11860300 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e3803bb50 msgr2=0x7f7e3803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f73535800 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e3803bb50 0x7f7e3803e010 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400ffd0 tx=0x7f7f64023040 comp rx=0 tx=0).stop 2024-05-08T18:05:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11841fc0 msgr2=0x55ec11849410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f73535800 1 --2- 172.21.15.50:0/2527091964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11841fc0 0x55ec11849410 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7f60068010 tx=0x7f7f6006ffa0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 reap_dead start 2024-05-08T18:05:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f6f9af640 1 -- 172.21.15.50:0/2527091964 reap_dead start 2024-05-08T18:05:03.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 shutdown_connections 2024-05-08T18:05:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 >> 172.21.15.50:0/2527091964 conn(0x55ec1183e2d0 msgr2=0x55ec1183b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.597+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 shutdown_connections 2024-05-08T18:05:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 -- 172.21.15.50:0/2527091964 wait complete. 2024-05-08T18:05:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 -- 172.21.15.50:0/1018658836 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e4803bb50 msgr2=0x7f7e4803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 --2- 172.21.15.50:0/1018658836 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e4803bb50 0x7f7e4803e010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55ec1181e650 tx=0x7f7f64006000 comp rx=0 tx=0).stop 2024-05-08T18:05:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 -- 172.21.15.50:0/1018658836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11834e40 msgr2=0x55ec11835260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 --2- 172.21.15.50:0/1018658836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11834e40 0x55ec11835260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7f60093210 tx=0x7f7f6005c670 comp rx=0 tx=0).stop 2024-05-08T18:05:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 -- 172.21.15.50:0/1018658836 shutdown_connections 2024-05-08T18:05:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 --2- 172.21.15.50:0/1018658836 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e4803bb50 0x7f7e4803e010 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 --2- 172.21.15.50:0/1018658836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec11834e40 0x55ec11835260 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 -- 172.21.15.50:0/1018658836 >> 172.21.15.50:0/1018658836 conn(0x55ec11828ee0 msgr2=0x55ec118292e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 -- 172.21.15.50:0/1018658836 shutdown_connections 2024-05-08T18:05:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 -- 172.21.15.50:0/1018658836 wait complete. 2024-05-08T18:05:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 -- 172.21.15.50:0/609574955 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec118200a0 msgr2=0x55ec11822490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec118200a0 0x55ec11822490 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 -- 172.21.15.50:0/609574955 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec1171eaa0 msgr2=0x55ec1181d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec1171eaa0 0x55ec1181d830 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 -- 172.21.15.50:0/609574955 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec11823fe0 msgr2=0x55ec118263d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec11823fe0 0x55ec118263d0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 -- 172.21.15.50:0/609574955 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e5403bb50 msgr2=0x7f7e5403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e5403bb50 0x7f7e5403e010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7f64000c00 tx=0x7f7f640013b0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 -- 172.21.15.50:0/609574955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 msgr2=0x55ec118165f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f73535800 1 --2- 172.21.15.50:0/609574955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec1171c170 0x55ec118165f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7f60060040 tx=0x7f7f60002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.598+0000 7f7f6f9af640 1 -- 172.21.15.50:0/609574955 reap_dead start 2024-05-08T18:05:03.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.599+0000 7f7f73535800 1 -- 172.21.15.50:0/609574955 shutdown_connections 2024-05-08T18:05:03.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.599+0000 7f7f73535800 1 -- 172.21.15.50:0/609574955 >> 172.21.15.50:0/609574955 conn(0x55ec11715ac0 msgr2=0x55ec11807e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.599+0000 7f7f73535800 1 -- 172.21.15.50:0/609574955 shutdown_connections 2024-05-08T18:05:03.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.599+0000 7f7f73535800 1 -- 172.21.15.50:0/609574955 wait complete. 2024-05-08T18:05:03.610 INFO:teuthology.orchestra.run.smithi050.stdout:{"entries":[],"summary":[]} 2024-05-08T18:05:03.610 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2024-05-08T18:05:03.611 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-05-08T18:05:03.611 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-05-08T18:05:03.611 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-05-08T18:05:03.698 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:03.698 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:03.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.701+0000 7f6753b0a800 1 Processor -- start 2024-05-08T18:05:03.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.701+0000 7f6753b0a800 1 -- start start 2024-05-08T18:05:03.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.701+0000 7f6753b0a800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5575b897fad0 0x5575b89885d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.701+0000 7f6753b0a800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5575b88f3010 con 0x5575b897df90 2024-05-08T18:05:03.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.701+0000 7f6753b0a800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5575b881fe20 con 0x5575b897fad0 2024-05-08T18:05:03.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.702+0000 7f6748e01640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5575b897fad0 0x5575b89885d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.702+0000 7f6751602640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5575b897df90 0x5575b897e3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:53028/0 (socket says 172.21.15.50:53028) 2024-05-08T18:05:03.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.702+0000 7f6748e01640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5575b897fad0 0x5575b89885d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57852/0 (socket says 172.21.15.50:57852) 2024-05-08T18:05:03.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.702+0000 7f6751602640 1 -- 172.21.15.50:0/1322002919 learned_addr learned my addr 172.21.15.50:0/1322002919 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:03.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.702+0000 7f6748e01640 1 -- 172.21.15.50:0/1322002919 >> v1:172.21.15.46:6789/0 conn(0x5575b897df90 legacy=0x5575b897e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.702+0000 7f6750e01640 1 -- 172.21.15.50:0/1322002919 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2950140758 0 0) 0x5575b88f3010 con 0x5575b897df90 2024-05-08T18:05:03.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.702+0000 7f6748e01640 1 -- 172.21.15.50:0/1322002919 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575b8989b20 con 0x5575b897fad0 2024-05-08T18:05:03.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.704+0000 7f6748e01640 1 --2- 172.21.15.50:0/1322002919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897fad0 0x5575b89885d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f673c00a470 tx=0x7f673c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7757f1e10033d0b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.704+0000 7f6750e01640 1 -- 172.21.15.50:0/1322002919 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f673c057070 con 0x5575b897fad0 2024-05-08T18:05:03.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.704+0000 7f6750e01640 1 -- 172.21.15.50:0/1322002919 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f673c04adc0 con 0x5575b897fad0 2024-05-08T18:05:03.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.704+0000 7f6750e01640 1 -- 172.21.15.50:0/1322002919 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f673c052590 con 0x5575b897fad0 2024-05-08T18:05:03.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.705+0000 7f6753b0a800 1 -- 172.21.15.50:0/1322002919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897fad0 msgr2=0x5575b89885d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.705+0000 7f6753b0a800 1 --2- 172.21.15.50:0/1322002919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897fad0 0x5575b89885d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f673c00a470 tx=0x7f673c01ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:03.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.705+0000 7f6753b0a800 1 -- 172.21.15.50:0/1322002919 shutdown_connections 2024-05-08T18:05:03.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.705+0000 7f6753b0a800 1 --2- 172.21.15.50:0/1322002919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897fad0 0x5575b89885d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.705+0000 7f6753b0a800 1 -- 172.21.15.50:0/1322002919 >> 172.21.15.50:0/1322002919 conn(0x5575b8979ad0 msgr2=0x5575b897bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.705+0000 7f6753b0a800 1 -- 172.21.15.50:0/1322002919 shutdown_connections 2024-05-08T18:05:03.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.705+0000 7f6753b0a800 1 -- 172.21.15.50:0/1322002919 wait complete. 2024-05-08T18:05:03.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.720+0000 7f6753b0a800 1 Processor -- start 2024-05-08T18:05:03.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.720+0000 7f6753b0a800 1 -- start start 2024-05-08T18:05:03.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.720+0000 7f6753b0a800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897df90 0x5575b8a72c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.720+0000 7f6753b0a800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5575b88caa10 con 0x5575b897df90 2024-05-08T18:05:03.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.721+0000 7f6751602640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897df90 0x5575b8a72c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.721+0000 7f6751602640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897df90 0x5575b8a72c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57864/0 (socket says 172.21.15.50:57864) 2024-05-08T18:05:03.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.721+0000 7f6751602640 1 -- 172.21.15.50:0/2667412791 learned_addr learned my addr 172.21.15.50:0/2667412791 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.721+0000 7f6751602640 1 -- 172.21.15.50:0/2667412791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575b8a731e0 con 0x5575b897df90 2024-05-08T18:05:03.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.721+0000 7f6751602640 1 --2- 172.21.15.50:0/2667412791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897df90 0x5575b8a72c10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6744001e70 tx=0x7f674400b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9012d566c8a7c41a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.721+0000 7f663f606640 1 -- 172.21.15.50:0/2667412791 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6744008070 con 0x5575b897df90 2024-05-08T18:05:03.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.721+0000 7f663f606640 1 -- 172.21.15.50:0/2667412791 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f674400d500 con 0x5575b897df90 2024-05-08T18:05:03.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.721+0000 7f663f606640 1 -- 172.21.15.50:0/2667412791 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f674400d8a0 con 0x5575b897df90 2024-05-08T18:05:03.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.722+0000 7f6753b0a800 1 -- 172.21.15.50:0/2667412791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897df90 msgr2=0x5575b8a72c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.722+0000 7f6753b0a800 1 --2- 172.21.15.50:0/2667412791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897df90 0x5575b8a72c10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6744001e70 tx=0x7f674400b6a0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.722+0000 7f6753b0a800 1 -- 172.21.15.50:0/2667412791 shutdown_connections 2024-05-08T18:05:03.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.722+0000 7f6753b0a800 1 --2- 172.21.15.50:0/2667412791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897df90 0x5575b8a72c10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.722+0000 7f6753b0a800 1 -- 172.21.15.50:0/2667412791 >> 172.21.15.50:0/2667412791 conn(0x5575b8979ad0 msgr2=0x5575b8a6e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.722+0000 7f6753b0a800 1 -- 172.21.15.50:0/2667412791 shutdown_connections 2024-05-08T18:05:03.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.722+0000 7f6753b0a800 1 -- 172.21.15.50:0/2667412791 wait complete. 2024-05-08T18:05:03.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.722+0000 7f6753b0a800 1 Processor -- start 2024-05-08T18:05:03.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.722+0000 7f6753b0a800 1 -- start start 2024-05-08T18:05:03.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.723+0000 7f6753b0a800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897df90 0x5575b8a7a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.723+0000 7f6753b0a800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5575b8988de0 con 0x5575b897df90 2024-05-08T18:05:03.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.723+0000 7f6751602640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897df90 0x5575b8a7a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.723+0000 7f6751602640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897df90 0x5575b8a7a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57868/0 (socket says 172.21.15.50:57868) 2024-05-08T18:05:03.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.723+0000 7f6751602640 1 -- 172.21.15.50:0/3948179424 learned_addr learned my addr 172.21.15.50:0/3948179424 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.723+0000 7f6751602640 1 -- 172.21.15.50:0/3948179424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575b8a7acf0 con 0x5575b897df90 2024-05-08T18:05:03.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.723+0000 7f6751602640 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897df90 0x5575b8a7a7b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x5575b8a739a0 tx=0x7f6744000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.723+0000 7f663de03640 1 -- 172.21.15.50:0/3948179424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f674400f980 con 0x5575b897df90 2024-05-08T18:05:03.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.723+0000 7f663de03640 1 -- 172.21.15.50:0/3948179424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f674400fb20 con 0x5575b897df90 2024-05-08T18:05:03.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.723+0000 7f663de03640 1 -- 172.21.15.50:0/3948179424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f674400fe00 con 0x5575b897df90 2024-05-08T18:05:03.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.723+0000 7f6753b0a800 1 -- 172.21.15.50:0/3948179424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5575b8a7af80 con 0x5575b897df90 2024-05-08T18:05:03.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.723+0000 7f6753b0a800 1 -- 172.21.15.50:0/3948179424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575b8a7dc30 con 0x5575b897df90 2024-05-08T18:05:03.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.724+0000 7f663de03640 1 -- 172.21.15.50:0/3948179424 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6744017040 con 0x5575b897df90 2024-05-08T18:05:03.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.724+0000 7f663de03640 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f663803bba0 0x7f663803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.725+0000 7f663de03640 1 -- 172.21.15.50:0/3948179424 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f6744015020 con 0x5575b897df90 2024-05-08T18:05:03.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.725+0000 7f6748e01640 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f663803bba0 0x7f663803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.725+0000 7f6753b0a800 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5575b897fad0 0x5575b8a819f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.725+0000 7f6753b0a800 1 -- 172.21.15.50:0/3948179424 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8a81f30 con 0x5575b897fad0 2024-05-08T18:05:03.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.725+0000 7f6751e03640 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5575b897fad0 0x5575b8a819f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.725+0000 7f6748e01640 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f663803bba0 0x7f663803e060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f673c054000 tx=0x7f673c04e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.725+0000 7f6751e03640 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5575b897fad0 0x5575b8a819f0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.725+0000 7f6751e03640 1 -- 172.21.15.50:0/3948179424 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b897fad0 2024-05-08T18:05:03.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.725+0000 7f6753b0a800 1 -- 172.21.15.50:0/3948179424 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8a7efc0 con 0x5575b897fad0 2024-05-08T18:05:03.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.726+0000 7f6751e03640 1 -- 172.21.15.50:0/3948179424 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b897fad0 2024-05-08T18:05:03.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.726+0000 7f6753b0a800 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5575b8a84240 0x5575b8a86630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.726+0000 7f6753b0a800 1 -- 172.21.15.50:0/3948179424 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8a86d20 con 0x5575b8a84240 2024-05-08T18:05:03.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.726+0000 7f6751602640 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5575b8a84240 0x5575b8a86630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.726+0000 7f6751602640 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5575b8a84240 0x5575b8a86630 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.727+0000 7f6751602640 1 -- 172.21.15.50:0/3948179424 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f674402d070 con 0x5575b8a84240 2024-05-08T18:05:03.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.727+0000 7f6753b0a800 1 -- 172.21.15.50:0/3948179424 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8a830e0 con 0x5575b897fad0 2024-05-08T18:05:03.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.727+0000 7f6751e03640 1 -- 172.21.15.50:0/3948179424 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b897fad0 2024-05-08T18:05:03.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.727+0000 7f6753b0a800 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5575b8a88180 0x5575b8a8a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.727+0000 7f6753b0a800 1 -- 172.21.15.50:0/3948179424 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8a8ac60 con 0x5575b8a88180 2024-05-08T18:05:03.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.727+0000 7f6748e01640 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5575b8a88180 0x5575b8a8a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.727+0000 7f6748e01640 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5575b8a88180 0x5575b8a8a570 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.728+0000 7f6748e01640 1 -- 172.21.15.50:0/3948179424 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f673c05a040 con 0x5575b8a88180 2024-05-08T18:05:03.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.728+0000 7f6753b0a800 1 Processor -- start 2024-05-08T18:05:03.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.728+0000 7f6753b0a800 1 -- start start 2024-05-08T18:05:03.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.728+0000 7f6753b0a800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8a98f90 0x5575b8a993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.728+0000 7f6753b0a800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5575b8871330 con 0x5575b8a98f90 2024-05-08T18:05:03.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.729+0000 7f6751602640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8a98f90 0x5575b8a993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.729+0000 7f6751602640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8a98f90 0x5575b8a993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57870/0 (socket says 172.21.15.50:57870) 2024-05-08T18:05:03.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.729+0000 7f6751602640 1 -- 172.21.15.50:0/3576925111 learned_addr learned my addr 172.21.15.50:0/3576925111 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.729+0000 7f6751602640 1 -- 172.21.15.50:0/3576925111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575b8a998f0 con 0x5575b8a98f90 2024-05-08T18:05:03.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.729+0000 7f6751602640 1 --2- 172.21.15.50:0/3576925111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8a98f90 0x5575b8a993b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f674400ea70 tx=0x7f674401ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.729+0000 7f6636ffd640 1 -- 172.21.15.50:0/3576925111 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f674401a540 con 0x5575b8a98f90 2024-05-08T18:05:03.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.729+0000 7f6636ffd640 1 -- 172.21.15.50:0/3576925111 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f674404a320 con 0x5575b8a98f90 2024-05-08T18:05:03.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.729+0000 7f6753b0a800 1 -- 172.21.15.50:0/3576925111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5575b8a9a360 con 0x5575b8a98f90 2024-05-08T18:05:03.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.729+0000 7f6636ffd640 1 -- 172.21.15.50:0/3576925111 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67440183d0 con 0x5575b8a98f90 2024-05-08T18:05:03.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.729+0000 7f6753b0a800 1 -- 172.21.15.50:0/3576925111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575b8a9a860 con 0x5575b8a98f90 2024-05-08T18:05:03.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.730+0000 7f6636ffd640 1 -- 172.21.15.50:0/3576925111 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6744018570 con 0x5575b8a98f90 2024-05-08T18:05:03.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.730+0000 7f6636ffd640 1 --2- 172.21.15.50:0/3576925111 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f662c03bbf0 0x7f662c03e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.730+0000 7f6636ffd640 1 -- 172.21.15.50:0/3576925111 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f674404ae00 con 0x5575b8a98f90 2024-05-08T18:05:03.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.730+0000 7f6748e01640 1 --2- 172.21.15.50:0/3576925111 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f662c03bbf0 0x7f662c03e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.731+0000 7f6753b0a800 1 Processor -- start 2024-05-08T18:05:03.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.731+0000 7f6753b0a800 1 -- start start 2024-05-08T18:05:03.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.731+0000 7f6748e01640 1 --2- 172.21.15.50:0/3576925111 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f662c03bbf0 0x7f662c03e0b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5575b8a830b0 tx=0x7f673c005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.731+0000 7f6753b0a800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8aa6330 0x5575b8aa6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.731+0000 7f6753b0a800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5575b8a73560 con 0x5575b8aa6330 2024-05-08T18:05:03.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.731+0000 7f6751602640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8aa6330 0x5575b8aa6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.731+0000 7f6751602640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8aa6330 0x5575b8aa6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57882/0 (socket says 172.21.15.50:57882) 2024-05-08T18:05:03.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.731+0000 7f6751602640 1 -- 172.21.15.50:0/4027870369 learned_addr learned my addr 172.21.15.50:0/4027870369 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.731+0000 7f6751602640 1 -- 172.21.15.50:0/4027870369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575b8aa6d20 con 0x5575b8aa6330 2024-05-08T18:05:03.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.731+0000 7f6751602640 1 --2- 172.21.15.50:0/4027870369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8aa6330 0x5575b8aa6750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6744051fd0 tx=0x7f674401cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb15e886ca8a2e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.731+0000 7f6634ff9640 1 -- 172.21.15.50:0/4027870369 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f674403ed10 con 0x5575b8aa6330 2024-05-08T18:05:03.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.731+0000 7f6634ff9640 1 -- 172.21.15.50:0/4027870369 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6744043930 con 0x5575b8aa6330 2024-05-08T18:05:03.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.732+0000 7f6634ff9640 1 -- 172.21.15.50:0/4027870369 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6744051910 con 0x5575b8aa6330 2024-05-08T18:05:03.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.732+0000 7f6753b0a800 1 -- 172.21.15.50:0/4027870369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8aa6330 msgr2=0x5575b8aa6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.732+0000 7f6753b0a800 1 --2- 172.21.15.50:0/4027870369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8aa6330 0x5575b8aa6750 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6744051fd0 tx=0x7f674401cf30 comp rx=0 tx=0).stop 2024-05-08T18:05:03.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.732+0000 7f6753b0a800 1 -- 172.21.15.50:0/4027870369 shutdown_connections 2024-05-08T18:05:03.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.732+0000 7f6753b0a800 1 --2- 172.21.15.50:0/4027870369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8aa6330 0x5575b8aa6750 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.732+0000 7f6753b0a800 1 -- 172.21.15.50:0/4027870369 >> 172.21.15.50:0/4027870369 conn(0x5575b8aa25b0 msgr2=0x5575b8aa0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.732+0000 7f6753b0a800 1 -- 172.21.15.50:0/4027870369 shutdown_connections 2024-05-08T18:05:03.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.732+0000 7f6753b0a800 1 -- 172.21.15.50:0/4027870369 wait complete. 2024-05-08T18:05:03.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.732+0000 7f6753b0a800 1 Processor -- start 2024-05-08T18:05:03.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.732+0000 7f6753b0a800 1 -- start start 2024-05-08T18:05:03.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.733+0000 7f6753b0a800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8aa6330 0x5575b8aad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.733+0000 7f6753b0a800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5575b8aa7390 con 0x5575b8aa6330 2024-05-08T18:05:03.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.733+0000 7f6751602640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8aa6330 0x5575b8aad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.733+0000 7f6751602640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8aa6330 0x5575b8aad6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57886/0 (socket says 172.21.15.50:57886) 2024-05-08T18:05:03.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.733+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 learned_addr learned my addr 172.21.15.50:0/2180545650 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.733+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575b8aadc10 con 0x5575b8aa6330 2024-05-08T18:05:03.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.733+0000 7f6751602640 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8aa6330 0x5575b8aad6d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6744051890 tx=0x7f6744055040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.733+0000 7f6622ffd640 1 -- 172.21.15.50:0/2180545650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67440336b0 con 0x5575b8aa6330 2024-05-08T18:05:03.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.733+0000 7f6622ffd640 1 -- 172.21.15.50:0/2180545650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6744033890 con 0x5575b8aa6330 2024-05-08T18:05:03.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.733+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5575b8aadc10 con 0x5575b8aa6330 2024-05-08T18:05:03.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.733+0000 7f6622ffd640 1 -- 172.21.15.50:0/2180545650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6744033bb0 con 0x5575b8aa6330 2024-05-08T18:05:03.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.734+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575b8aadc10 con 0x5575b8aa6330 2024-05-08T18:05:03.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.734+0000 7f6622ffd640 1 -- 172.21.15.50:0/2180545650 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6744033d50 con 0x5575b8aa6330 2024-05-08T18:05:03.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.735+0000 7f6622ffd640 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f661c03bba0 0x7f661c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.735+0000 7f6622ffd640 1 -- 172.21.15.50:0/2180545650 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f674406cc50 con 0x5575b8aa6330 2024-05-08T18:05:03.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.735+0000 7f6748e01640 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f661c03bba0 0x7f661c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.735+0000 7f6753b0a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.735+0000 7f6753b0a800 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5575b8ab8e30 0x5575b8abb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.735+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abba40 con 0x5575b8ab8e30 2024-05-08T18:05:03.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.735+0000 7f6751e03640 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5575b8ab8e30 0x5575b8abb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.735+0000 7f6748e01640 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f661c03bba0 0x7f661c03e060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f673c05dfd0 tx=0x7f673c04f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.735+0000 7f6751e03640 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5575b8ab8e30 0x5575b8abb2f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b8ab8e30 2024-05-08T18:05:03.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6753b0a800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6753b0a800 20 realm 2024-05-08T18:05:03.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6753b0a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abbe50 con 0x5575b8ab8e30 2024-05-08T18:05:03.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b8ab8e30 2024-05-08T18:05:03.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6753b0a800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6753b0a800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:03.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6753b0a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abc720 con 0x5575b8ab8e30 2024-05-08T18:05:03.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b8ab8e30 2024-05-08T18:05:03.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6753b0a800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6753b0a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ab8e30 2024-05-08T18:05:03.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b8ab8e30 2024-05-08T18:05:03.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6753b0a800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:03.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.736+0000 7f6753b0a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.737+0000 7f6753b0a800 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5575b8abd5f0 0x5575b8abf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.737+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac0140 con 0x5575b8abd5f0 2024-05-08T18:05:03.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.737+0000 7f6751602640 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5575b8abd5f0 0x5575b8abf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.737+0000 7f6751602640 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5575b8abd5f0 0x5575b8abf9e0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.737+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.737+0000 7f6753b0a800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:03.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.737+0000 7f6753b0a800 20 searching for the correct realm 2024-05-08T18:05:03.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.737+0000 7f6753b0a800 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5575b8ac21f0 0x5575b8ac46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.737+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac4d80 con 0x5575b8ac21f0 2024-05-08T18:05:03.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.737+0000 7f6748e01640 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5575b8ac21f0 0x5575b8ac46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.738+0000 7f6748e01640 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5575b8ac21f0 0x5575b8ac46b0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.738+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f673c070030 con 0x5575b8ac21f0 2024-05-08T18:05:03.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.738+0000 7f6748e01640 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f673c083040 0x7f673c084430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.738+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac4d80 con 0x7f673c083040 2024-05-08T18:05:03.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.738+0000 7f6751e03640 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f673c083040 0x7f673c084430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.738+0000 7f6751e03640 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f673c083040 0x7f673c084430 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.739+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674c00a4a0 con 0x7f673c083040 2024-05-08T18:05:03.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.739+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8abd5f0 2024-05-08T18:05:03.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.739+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.739+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ab8e30 2024-05-08T18:05:03.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.739+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b8ab8e30 2024-05-08T18:05:03.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.739+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ab8e30 2024-05-08T18:05:03.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.739+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b8ab8e30 2024-05-08T18:05:03.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.739+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8abd5f0 2024-05-08T18:05:03.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.740+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.740+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8abd5f0 2024-05-08T18:05:03.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.740+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.740+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x7f673c083040 2024-05-08T18:05:03.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.740+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674c00a4a0 con 0x7f673c083040 2024-05-08T18:05:03.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.740+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ac21f0 2024-05-08T18:05:03.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.740+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f673c070030 con 0x5575b8ac21f0 2024-05-08T18:05:03.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.740+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ab8e30 2024-05-08T18:05:03.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.741+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b8ab8e30 2024-05-08T18:05:03.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.741+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8abd5f0 2024-05-08T18:05:03.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.741+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.741+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ab8e30 2024-05-08T18:05:03.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.741+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b8ab8e30 2024-05-08T18:05:03.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.741+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x7f673c083040 2024-05-08T18:05:03.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.742+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f674c00a4a0 con 0x7f673c083040 2024-05-08T18:05:03.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.742+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ac21f0 2024-05-08T18:05:03.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.742+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f673c070030 con 0x5575b8ac21f0 2024-05-08T18:05:03.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.742+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ac21f0 2024-05-08T18:05:03.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.742+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f673c070030 con 0x5575b8ac21f0 2024-05-08T18:05:03.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.742+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8abd5f0 2024-05-08T18:05:03.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.742+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.742+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ac21f0 2024-05-08T18:05:03.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.743+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f673c070030 con 0x5575b8ac21f0 2024-05-08T18:05:03.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.743+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ac21f0 2024-05-08T18:05:03.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.743+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f673c070030 con 0x5575b8ac21f0 2024-05-08T18:05:03.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.743+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ab8e30 2024-05-08T18:05:03.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.743+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b8ab8e30 2024-05-08T18:05:03.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.743+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ab8e30 2024-05-08T18:05:03.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.743+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b8ab8e30 2024-05-08T18:05:03.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.743+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8abd5f0 2024-05-08T18:05:03.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.744+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.744+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ac21f0 2024-05-08T18:05:03.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.744+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f673c070030 con 0x5575b8ac21f0 2024-05-08T18:05:03.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.744+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x7f673c083040 2024-05-08T18:05:03.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.744+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674c00a4a0 con 0x7f673c083040 2024-05-08T18:05:03.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.744+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ab8e30 2024-05-08T18:05:03.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.744+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b8ab8e30 2024-05-08T18:05:03.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.744+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8abd5f0 2024-05-08T18:05:03.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.745+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.745+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ac21f0 2024-05-08T18:05:03.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.745+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f673c070030 con 0x5575b8ac21f0 2024-05-08T18:05:03.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.745+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8abd5f0 2024-05-08T18:05:03.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.745+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.745+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x7f673c083040 2024-05-08T18:05:03.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.746+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674c00a4a0 con 0x7f673c083040 2024-05-08T18:05:03.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.746+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ac21f0 2024-05-08T18:05:03.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.746+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f673c070030 con 0x5575b8ac21f0 2024-05-08T18:05:03.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.746+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8abd5f0 2024-05-08T18:05:03.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.746+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.746+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x5575b8ab8e30 2024-05-08T18:05:03.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.746+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b8ab8e30 2024-05-08T18:05:03.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.746+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5575b8abcac0 con 0x7f673c083040 2024-05-08T18:05:03.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f674c00a4a0 con 0x7f673c083040 2024-05-08T18:05:03.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:03.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:03.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:03.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:03.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:03.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:03.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac0af0 con 0x5575b8ab8e30 2024-05-08T18:05:03.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b8ab8e30 2024-05-08T18:05:03.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac0af0 con 0x5575b8ab8e30 2024-05-08T18:05:03.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f674c00a4a0 con 0x5575b8ab8e30 2024-05-08T18:05:03.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:03.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac0ac0 con 0x5575b8ac21f0 2024-05-08T18:05:03.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f673c070030 con 0x5575b8ac21f0 2024-05-08T18:05:03.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:03.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 20 zone default found 2024-05-08T18:05:03.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 4 Realm: () 2024-05-08T18:05:03.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:03.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:03.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:03.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 20 zonegroup default 2024-05-08T18:05:03.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.747+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac1390 con 0x5575b8abd5f0 2024-05-08T18:05:03.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.748+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.748+0000 7f6753b0a800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.748+0000 7f6753b0a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.748+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac6be0 con 0x5575b8ac21f0 2024-05-08T18:05:03.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.748+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f673c070030 con 0x5575b8ac21f0 2024-05-08T18:05:03.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.748+0000 7f6753b0a800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.748+0000 7f6753b0a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.748+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac6f80 con 0x5575b8ac21f0 2024-05-08T18:05:03.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.748+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f673c070030 con 0x5575b8ac21f0 2024-05-08T18:05:03.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.748+0000 7f6753b0a800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.748+0000 7f6753b0a800 20 started sync module instance, tier type = 2024-05-08T18:05:03.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.748+0000 7f6753b0a800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:03.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.748+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac6080 con 0x7f673c083040 2024-05-08T18:05:03.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.749+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674c00a4a0 con 0x7f673c083040 2024-05-08T18:05:03.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.749+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93964097840544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac6080 con 0x7f673c083040 2024-05-08T18:05:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.749+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac7940 con 0x5575b8ab8e30 2024-05-08T18:05:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.750+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93964097840544] v27'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674c040040 con 0x7f673c083040 2024-05-08T18:05:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.751+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674c040040 con 0x5575b8ab8e30 2024-05-08T18:05:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.751+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93964097844848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac6080 con 0x5575b8ab8e30 2024-05-08T18:05:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.751+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac83b0 con 0x5575b8ac21f0 2024-05-08T18:05:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.752+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93964097844848] v27'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674c040040 con 0x5575b8ab8e30 2024-05-08T18:05:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.752+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f673c070030 con 0x5575b8ac21f0 2024-05-08T18:05:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.752+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93964097838032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac5ff0 con 0x5575b8ac21f0 2024-05-08T18:05:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.752+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac9270 con 0x7f673c083040 2024-05-08T18:05:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.753+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93964097838032] v27'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f673c070030 con 0x5575b8ac21f0 2024-05-08T18:05:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.753+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674c040040 con 0x7f673c083040 2024-05-08T18:05:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.753+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93964097850816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ac91e0 con 0x7f673c083040 2024-05-08T18:05:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.753+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8aca000 con 0x5575b8ab8e30 2024-05-08T18:05:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.754+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674c040040 con 0x5575b8ab8e30 2024-05-08T18:05:03.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.754+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93964097854224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8aca000 con 0x5575b8ab8e30 2024-05-08T18:05:03.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.754+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8acaf60 con 0x5575b8abd5f0 2024-05-08T18:05:03.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.754+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93964097850816] v27'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674c040040 con 0x7f673c083040 2024-05-08T18:05:03.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.755+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.755+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93964097853024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8acb350 con 0x5575b8abd5f0 2024-05-08T18:05:03.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.755+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93964097854224] v27'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674c040040 con 0x5575b8ab8e30 2024-05-08T18:05:03.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.755+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8aca3f0 con 0x5575b8abd5f0 2024-05-08T18:05:03.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.756+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93964097853024] v27'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.756+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.756+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93964097861728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8acc130 con 0x5575b8abd5f0 2024-05-08T18:05:03.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.757+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8acc980 con 0x5575b8abd5f0 2024-05-08T18:05:03.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.757+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93964097861728] v27'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.757+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.757+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93964097864912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8accda0 con 0x5575b8abd5f0 2024-05-08T18:05:03.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.757+0000 7f6753b0a800 20 add_watcher() i=0 2024-05-08T18:05:03.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.757+0000 7f6753b0a800 20 add_watcher() i=1 2024-05-08T18:05:03.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.757+0000 7f6753b0a800 20 add_watcher() i=2 2024-05-08T18:05:03.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.757+0000 7f6753b0a800 20 add_watcher() i=3 2024-05-08T18:05:03.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.757+0000 7f6753b0a800 20 add_watcher() i=4 2024-05-08T18:05:03.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.757+0000 7f6753b0a800 20 add_watcher() i=5 2024-05-08T18:05:03.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.757+0000 7f6753b0a800 20 add_watcher() i=6 2024-05-08T18:05:03.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.759+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93964097864912] v27'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.759+0000 7f6753b0a800 20 add_watcher() i=7 2024-05-08T18:05:03.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.759+0000 7f6753b0a800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:03.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.759+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8ad0060 con 0x5575b8ab8e30 2024-05-08T18:05:03.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.759+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f674c040040 con 0x5575b8ab8e30 2024-05-08T18:05:03.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.759+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93964097866336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8acd330 con 0x5575b8ab8e30 2024-05-08T18:05:03.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.760+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93964097866336] v27'48 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f674c040040 con 0x5575b8ab8e30 2024-05-08T18:05:03.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.760+0000 7f66077e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:03.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.760+0000 7f6753b0a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:03.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.760+0000 7f6753b0a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:03.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.760+0000 7f6753b0a800 5 note: GC not initialized 2024-05-08T18:05:03.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.760+0000 7f66067e4640 20 reqs_thread_entry: start 2024-05-08T18:05:03.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.761+0000 7f6753b0a800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:03.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.761+0000 7f6753b0a800 20 Filter name: none 2024-05-08T18:05:03.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.761+0000 7f6577fff640 20 reqs_thread_entry: start 2024-05-08T18:05:03.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.761+0000 7f6753b0a800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.761+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5575b8b03110 con 0x5575b8ab8e30 2024-05-08T18:05:03.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.761+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f674c040040 con 0x5575b8ab8e30 2024-05-08T18:05:03.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.761+0000 7f6753b0a800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.777 INFO:teuthology.orchestra.run.smithi050.stderr:could not fetch user info: no user info saved 2024-05-08T18:05:03.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.762+0000 7f66077e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:03.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.762+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93964097866336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8b030d0 con 0x5575b8ab8e30 2024-05-08T18:05:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.763+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 93964097866336] v27'49 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f674c040040 con 0x5575b8ab8e30 2024-05-08T18:05:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.763+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93964097840544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8b030d0 con 0x7f673c083040 2024-05-08T18:05:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.764+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 93964097840544] v27'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674c040040 con 0x7f673c083040 2024-05-08T18:05:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.764+0000 7f6753b0a800 20 remove_watcher() i=0 2024-05-08T18:05:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.764+0000 7f6753b0a800 2 removed watcher, disabling cache 2024-05-08T18:05:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.764+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93964097844848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8b030d0 con 0x5575b8ab8e30 2024-05-08T18:05:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.765+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 93964097844848] v27'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674c040040 con 0x5575b8ab8e30 2024-05-08T18:05:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.765+0000 7f6753b0a800 20 remove_watcher() i=1 2024-05-08T18:05:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.765+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93964097838032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8b030d0 con 0x5575b8ac21f0 2024-05-08T18:05:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.766+0000 7f6748e01640 1 -- 172.21.15.50:0/2180545650 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 93964097838032] v27'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f673c070030 con 0x5575b8ac21f0 2024-05-08T18:05:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.766+0000 7f6753b0a800 20 remove_watcher() i=2 2024-05-08T18:05:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.766+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93964097850816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8b030d0 con 0x7f673c083040 2024-05-08T18:05:03.783 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T18:05:03.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.767+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 93964097850816] v27'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674c040040 con 0x7f673c083040 2024-05-08T18:05:03.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.767+0000 7f6753b0a800 20 remove_watcher() i=3 2024-05-08T18:05:03.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.767+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93964097854224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8b030d0 con 0x5575b8ab8e30 2024-05-08T18:05:03.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.768+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 93964097854224] v27'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674c040040 con 0x5575b8ab8e30 2024-05-08T18:05:03.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.768+0000 7f6753b0a800 20 remove_watcher() i=4 2024-05-08T18:05:03.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.768+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93964097853024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8b030d0 con 0x5575b8abd5f0 2024-05-08T18:05:03.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.769+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 93964097853024] v27'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.769+0000 7f6753b0a800 20 remove_watcher() i=5 2024-05-08T18:05:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.769+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93964097861728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8b030d0 con 0x5575b8abd5f0 2024-05-08T18:05:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.770+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 93964097861728] v27'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.770+0000 7f6753b0a800 20 remove_watcher() i=6 2024-05-08T18:05:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.770+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93964097864912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5575b8b030d0 con 0x5575b8abd5f0 2024-05-08T18:05:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.771+0000 7f6751602640 1 -- 172.21.15.50:0/2180545650 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 93964097864912] v27'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f674402d070 con 0x5575b8abd5f0 2024-05-08T18:05:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.771+0000 7f6753b0a800 20 remove_watcher() i=7 2024-05-08T18:05:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.771+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5575b8abd5f0 msgr2=0x5575b8abf9e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.771+0000 7f6753b0a800 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5575b8abd5f0 0x5575b8abf9e0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.771+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5575b8ab8e30 msgr2=0x5575b8abb2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.771+0000 7f6753b0a800 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5575b8ab8e30 0x5575b8abb2f0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.771+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f673c083040 msgr2=0x7f673c084430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.771+0000 7f6753b0a800 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f673c083040 0x7f673c084430 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.771+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5575b8ac21f0 msgr2=0x5575b8ac46b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.771+0000 7f6753b0a800 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5575b8ac21f0 0x5575b8ac46b0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.771+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f661c03bba0 msgr2=0x7f661c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.771+0000 7f6753b0a800 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f661c03bba0 0x7f661c03e060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f673c05dfd0 tx=0x7f673c04f000 comp rx=0 tx=0).stop 2024-05-08T18:05:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.771+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8aa6330 msgr2=0x5575b8aad6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.771+0000 7f6751e03640 1 -- 172.21.15.50:0/2180545650 reap_dead start 2024-05-08T18:05:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.771+0000 7f6753b0a800 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8aa6330 0x5575b8aad6d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6744051890 tx=0x7f6744055040 comp rx=0 tx=0).stop 2024-05-08T18:05:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.771+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 shutdown_connections 2024-05-08T18:05:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 --2- 172.21.15.50:0/2180545650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8aa6330 0x5575b8aad6d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 >> 172.21.15.50:0/2180545650 conn(0x5575b8aa25b0 msgr2=0x5575b8a9fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 shutdown_connections 2024-05-08T18:05:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 -- 172.21.15.50:0/2180545650 wait complete. 2024-05-08T18:05:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 -- 172.21.15.50:0/3576925111 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f662c03bbf0 msgr2=0x7f662c03e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 --2- 172.21.15.50:0/3576925111 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f662c03bbf0 0x7f662c03e0b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5575b8a830b0 tx=0x7f673c005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 -- 172.21.15.50:0/3576925111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8a98f90 msgr2=0x5575b8a993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 --2- 172.21.15.50:0/3576925111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8a98f90 0x5575b8a993b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f674400ea70 tx=0x7f674401ab20 comp rx=0 tx=0).stop 2024-05-08T18:05:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 -- 172.21.15.50:0/3576925111 shutdown_connections 2024-05-08T18:05:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 --2- 172.21.15.50:0/3576925111 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f662c03bbf0 0x7f662c03e0b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 --2- 172.21.15.50:0/3576925111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b8a98f90 0x5575b8a993b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 -- 172.21.15.50:0/3576925111 >> 172.21.15.50:0/3576925111 conn(0x5575b8a8d030 msgr2=0x5575b8a8d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 -- 172.21.15.50:0/3576925111 shutdown_connections 2024-05-08T18:05:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 -- 172.21.15.50:0/3576925111 wait complete. 2024-05-08T18:05:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 -- 172.21.15.50:0/3948179424 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5575b8a84240 msgr2=0x5575b8a86630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5575b8a84240 0x5575b8a86630 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 -- 172.21.15.50:0/3948179424 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5575b897fad0 msgr2=0x5575b8a819f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5575b897fad0 0x5575b8a819f0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 -- 172.21.15.50:0/3948179424 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5575b8a88180 msgr2=0x5575b8a8a570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5575b8a88180 0x5575b8a8a570 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.772+0000 7f6753b0a800 1 -- 172.21.15.50:0/3948179424 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f663803bba0 msgr2=0x7f663803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.773+0000 7f6753b0a800 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f663803bba0 0x7f663803e060 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f673c054000 tx=0x7f673c04e000 comp rx=0 tx=0).stop 2024-05-08T18:05:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.773+0000 7f6753b0a800 1 -- 172.21.15.50:0/3948179424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897df90 msgr2=0x5575b8a7a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.773+0000 7f6753b0a800 1 --2- 172.21.15.50:0/3948179424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5575b897df90 0x5575b8a7a7b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x5575b8a739a0 tx=0x7f6744000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.773+0000 7f6751e03640 1 -- 172.21.15.50:0/3948179424 reap_dead start 2024-05-08T18:05:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.773+0000 7f6753b0a800 1 -- 172.21.15.50:0/3948179424 shutdown_connections 2024-05-08T18:05:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.773+0000 7f6753b0a800 1 -- 172.21.15.50:0/3948179424 >> 172.21.15.50:0/3948179424 conn(0x5575b8979ad0 msgr2=0x5575b8a6cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.773+0000 7f6753b0a800 1 -- 172.21.15.50:0/3948179424 shutdown_connections 2024-05-08T18:05:03.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.773+0000 7f6753b0a800 1 -- 172.21.15.50:0/3948179424 wait complete. 2024-05-08T18:05:03.788 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2024-05-08T18:05:03.788 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2024-05-08T18:05:03.788 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid foo --display-name Foo --email foo@foo.com --access-key 9te6NH5mcdcq0Tc5i8i1 --secret Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu --max-buckets 4 2024-05-08T18:05:03.845 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:03.845 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:03.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.848+0000 7fa27781f800 1 Processor -- start 2024-05-08T18:05:03.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.848+0000 7fa27781f800 1 -- start start 2024-05-08T18:05:03.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.848+0000 7fa27781f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55d0a72afa10 0x55d0a72b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.848+0000 7fa27781f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d0a72b23d0 con 0x55d0a72ab580 2024-05-08T18:05:03.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.848+0000 7fa27781f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55d0a714ce20 con 0x55d0a72afa10 2024-05-08T18:05:03.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.848+0000 7fa274b1d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55d0a72afa10 0x55d0a72b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.848+0000 7fa27531e640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55d0a72ab580 0x55d0a72ab9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:53042/0 (socket says 172.21.15.50:53042) 2024-05-08T18:05:03.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.848+0000 7fa27531e640 1 -- 172.21.15.50:0/1101659439 learned_addr learned my addr 172.21.15.50:0/1101659439 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:03.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.849+0000 7fa274b1d640 1 -- 172.21.15.50:0/1101659439 >> v1:172.21.15.46:6789/0 conn(0x55d0a72ab580 legacy=0x55d0a72ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.849+0000 7fa267fff640 1 -- 172.21.15.50:0/1101659439 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1006783304 0 0) 0x55d0a72b23d0 con 0x55d0a72ab580 2024-05-08T18:05:03.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.849+0000 7fa274b1d640 1 -- 172.21.15.50:0/1101659439 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0a72b3600 con 0x55d0a72afa10 2024-05-08T18:05:03.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.850+0000 7fa274b1d640 1 --2- 172.21.15.50:0/1101659439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72afa10 0x55d0a72b1e00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00a470 tx=0x7fa26c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fba01a45ab3a0f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.851+0000 7fa267fff640 1 -- 172.21.15.50:0/1101659439 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa26c057070 con 0x55d0a72afa10 2024-05-08T18:05:03.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.851+0000 7fa267fff640 1 -- 172.21.15.50:0/1101659439 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa26c052570 con 0x55d0a72afa10 2024-05-08T18:05:03.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.851+0000 7fa267fff640 1 -- 172.21.15.50:0/1101659439 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa26c052870 con 0x55d0a72afa10 2024-05-08T18:05:03.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.851+0000 7fa27781f800 1 -- 172.21.15.50:0/1101659439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72afa10 msgr2=0x55d0a72b1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.851+0000 7fa27781f800 1 --2- 172.21.15.50:0/1101659439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72afa10 0x55d0a72b1e00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00a470 tx=0x7fa26c01ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:03.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.851+0000 7fa27781f800 1 -- 172.21.15.50:0/1101659439 shutdown_connections 2024-05-08T18:05:03.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.851+0000 7fa27781f800 1 --2- 172.21.15.50:0/1101659439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72afa10 0x55d0a72b1e00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.851+0000 7fa27781f800 1 -- 172.21.15.50:0/1101659439 >> 172.21.15.50:0/1101659439 conn(0x55d0a72a6ae0 msgr2=0x55d0a72a8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.851+0000 7fa27781f800 1 -- 172.21.15.50:0/1101659439 shutdown_connections 2024-05-08T18:05:03.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.851+0000 7fa27781f800 1 -- 172.21.15.50:0/1101659439 wait complete. 2024-05-08T18:05:03.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.866+0000 7fa27781f800 1 Processor -- start 2024-05-08T18:05:03.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.867+0000 7fa27781f800 1 -- start start 2024-05-08T18:05:03.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.867+0000 7fa27781f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72ab580 0x55d0a721f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.867+0000 7fa27781f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d0a71f7b20 con 0x55d0a72ab580 2024-05-08T18:05:03.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.867+0000 7fa27531e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72ab580 0x55d0a721f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.867+0000 7fa27531e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72ab580 0x55d0a721f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57894/0 (socket says 172.21.15.50:57894) 2024-05-08T18:05:03.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.867+0000 7fa27531e640 1 -- 172.21.15.50:0/1715283958 learned_addr learned my addr 172.21.15.50:0/1715283958 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.867+0000 7fa27531e640 1 -- 172.21.15.50:0/1715283958 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0a739e540 con 0x55d0a72ab580 2024-05-08T18:05:03.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.868+0000 7fa27531e640 1 --2- 172.21.15.50:0/1715283958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72ab580 0x55d0a721f640 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa268001ea0 tx=0x7fa26800b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f08b0676aca6317 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.868+0000 7fa163606640 1 -- 172.21.15.50:0/1715283958 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa268008070 con 0x55d0a72ab580 2024-05-08T18:05:03.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.868+0000 7fa163606640 1 -- 172.21.15.50:0/1715283958 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa26800d500 con 0x55d0a72ab580 2024-05-08T18:05:03.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.868+0000 7fa163606640 1 -- 172.21.15.50:0/1715283958 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa26800d880 con 0x55d0a72ab580 2024-05-08T18:05:03.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.868+0000 7fa27781f800 1 -- 172.21.15.50:0/1715283958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72ab580 msgr2=0x55d0a721f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.868+0000 7fa27781f800 1 --2- 172.21.15.50:0/1715283958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72ab580 0x55d0a721f640 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa268001ea0 tx=0x7fa26800b6a0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.868+0000 7fa27781f800 1 -- 172.21.15.50:0/1715283958 shutdown_connections 2024-05-08T18:05:03.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.868+0000 7fa27781f800 1 --2- 172.21.15.50:0/1715283958 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72ab580 0x55d0a721f640 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.869+0000 7fa27781f800 1 -- 172.21.15.50:0/1715283958 >> 172.21.15.50:0/1715283958 conn(0x55d0a72a6ae0 msgr2=0x55d0a7398240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.869+0000 7fa27781f800 1 -- 172.21.15.50:0/1715283958 shutdown_connections 2024-05-08T18:05:03.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.869+0000 7fa27781f800 1 -- 172.21.15.50:0/1715283958 wait complete. 2024-05-08T18:05:03.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.869+0000 7fa27781f800 1 Processor -- start 2024-05-08T18:05:03.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.869+0000 7fa27781f800 1 -- start start 2024-05-08T18:05:03.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.869+0000 7fa27781f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72ab580 0x55d0a73a77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.869+0000 7fa27781f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d0a72b28a0 con 0x55d0a72ab580 2024-05-08T18:05:03.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.869+0000 7fa27531e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72ab580 0x55d0a73a77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.870+0000 7fa27531e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72ab580 0x55d0a73a77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57906/0 (socket says 172.21.15.50:57906) 2024-05-08T18:05:03.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.870+0000 7fa27531e640 1 -- 172.21.15.50:0/396229249 learned_addr learned my addr 172.21.15.50:0/396229249 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.870+0000 7fa27531e640 1 -- 172.21.15.50:0/396229249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0a73a7d20 con 0x55d0a72ab580 2024-05-08T18:05:03.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.870+0000 7fa27531e640 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72ab580 0x55d0a73a77e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa2680012f0 tx=0x7fa268000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.870+0000 7fa161e03640 1 -- 172.21.15.50:0/396229249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa26800f9d0 con 0x55d0a72ab580 2024-05-08T18:05:03.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.870+0000 7fa161e03640 1 -- 172.21.15.50:0/396229249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa26800fb70 con 0x55d0a72ab580 2024-05-08T18:05:03.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.870+0000 7fa27781f800 1 -- 172.21.15.50:0/396229249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d0a73a7fb0 con 0x55d0a72ab580 2024-05-08T18:05:03.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.870+0000 7fa161e03640 1 -- 172.21.15.50:0/396229249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa26800fe50 con 0x55d0a72ab580 2024-05-08T18:05:03.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.870+0000 7fa27781f800 1 -- 172.21.15.50:0/396229249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0a73aac60 con 0x55d0a72ab580 2024-05-08T18:05:03.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.871+0000 7fa161e03640 1 -- 172.21.15.50:0/396229249 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa268017040 con 0x55d0a72ab580 2024-05-08T18:05:03.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.871+0000 7fa161e03640 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa15c03bcf0 0x7fa15c03e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.871+0000 7fa161e03640 1 -- 172.21.15.50:0/396229249 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fa268015020 con 0x55d0a72ab580 2024-05-08T18:05:03.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.871+0000 7fa274b1d640 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa15c03bcf0 0x7fa15c03e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.872+0000 7fa27781f800 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d0a72afa10 0x55d0a73aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.872+0000 7fa27781f800 1 -- 172.21.15.50:0/396229249 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73aefb0 con 0x55d0a72afa10 2024-05-08T18:05:03.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.872+0000 7fa275b1f640 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d0a72afa10 0x55d0a73aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.872+0000 7fa274b1d640 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa15c03bcf0 0x7fa15c03e1b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa26c054000 tx=0x7fa26c04e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.872+0000 7fa275b1f640 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d0a72afa10 0x55d0a73aea20 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.872+0000 7fa275b1f640 1 -- 172.21.15.50:0/396229249 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a72afa10 2024-05-08T18:05:03.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.872+0000 7fa27781f800 1 -- 172.21.15.50:0/396229249 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73abff0 con 0x55d0a72afa10 2024-05-08T18:05:03.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.873+0000 7fa275b1f640 1 -- 172.21.15.50:0/396229249 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a72afa10 2024-05-08T18:05:03.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.873+0000 7fa27781f800 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d0a73b12e0 0x55d0a73b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.873+0000 7fa27781f800 1 -- 172.21.15.50:0/396229249 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73b3d70 con 0x55d0a73b12e0 2024-05-08T18:05:03.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.873+0000 7fa27531e640 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d0a73b12e0 0x55d0a73b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.873+0000 7fa27531e640 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d0a73b12e0 0x55d0a73b36d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.873+0000 7fa27531e640 1 -- 172.21.15.50:0/396229249 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fa26802d070 con 0x55d0a73b12e0 2024-05-08T18:05:03.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.874+0000 7fa27781f800 1 -- 172.21.15.50:0/396229249 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73b0180 con 0x55d0a72afa10 2024-05-08T18:05:03.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.874+0000 7fa275b1f640 1 -- 172.21.15.50:0/396229249 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a72afa10 2024-05-08T18:05:03.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.874+0000 7fa27781f800 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d0a73b51d0 0x55d0a73b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.874+0000 7fa27781f800 1 -- 172.21.15.50:0/396229249 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73b7cc0 con 0x55d0a73b51d0 2024-05-08T18:05:03.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.874+0000 7fa274b1d640 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d0a73b51d0 0x55d0a73b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.874+0000 7fa274b1d640 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d0a73b51d0 0x55d0a73b75c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.874+0000 7fa274b1d640 1 -- 172.21.15.50:0/396229249 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fa26c0528e0 con 0x55d0a73b51d0 2024-05-08T18:05:03.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.875+0000 7fa27781f800 1 Processor -- start 2024-05-08T18:05:03.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.875+0000 7fa27781f800 1 -- start start 2024-05-08T18:05:03.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.875+0000 7fa27781f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73c5ff0 0x55d0a73c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.875+0000 7fa27781f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d0a719e330 con 0x55d0a73c5ff0 2024-05-08T18:05:03.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.875+0000 7fa27531e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73c5ff0 0x55d0a73c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.875+0000 7fa27531e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73c5ff0 0x55d0a73c6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57908/0 (socket says 172.21.15.50:57908) 2024-05-08T18:05:03.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.875+0000 7fa27531e640 1 -- 172.21.15.50:0/1236337339 learned_addr learned my addr 172.21.15.50:0/1236337339 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.876+0000 7fa27531e640 1 -- 172.21.15.50:0/1236337339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0a73c6950 con 0x55d0a73c5ff0 2024-05-08T18:05:03.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.876+0000 7fa27531e640 1 --2- 172.21.15.50:0/1236337339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73c5ff0 0x55d0a73c6410 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa26800b6c0 tx=0x7fa268051f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.876+0000 7fa15affd640 1 -- 172.21.15.50:0/1236337339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa26801a3c0 con 0x55d0a73c5ff0 2024-05-08T18:05:03.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.876+0000 7fa27781f800 1 -- 172.21.15.50:0/1236337339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d0a73c73c0 con 0x55d0a73c5ff0 2024-05-08T18:05:03.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.876+0000 7fa15affd640 1 -- 172.21.15.50:0/1236337339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa26801a560 con 0x55d0a73c5ff0 2024-05-08T18:05:03.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.876+0000 7fa15affd640 1 -- 172.21.15.50:0/1236337339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa268035060 con 0x55d0a73c5ff0 2024-05-08T18:05:03.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.876+0000 7fa27781f800 1 -- 172.21.15.50:0/1236337339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0a73c78c0 con 0x55d0a73c5ff0 2024-05-08T18:05:03.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.877+0000 7fa15affd640 1 -- 172.21.15.50:0/1236337339 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa26800e710 con 0x55d0a73c5ff0 2024-05-08T18:05:03.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.877+0000 7fa15affd640 1 --2- 172.21.15.50:0/1236337339 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa14c03bab0 0x7fa14c03df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.877+0000 7fa15affd640 1 -- 172.21.15.50:0/1236337339 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fa26803d490 con 0x55d0a73c5ff0 2024-05-08T18:05:03.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.877+0000 7fa274b1d640 1 --2- 172.21.15.50:0/1236337339 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa14c03bab0 0x7fa14c03df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.877+0000 7fa27781f800 1 Processor -- start 2024-05-08T18:05:03.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.877+0000 7fa27781f800 1 -- start start 2024-05-08T18:05:03.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.878+0000 7fa274b1d640 1 --2- 172.21.15.50:0/1236337339 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa14c03bab0 0x7fa14c03df70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55d0a73abf10 tx=0x7fa26c005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.878+0000 7fa27781f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73d3320 0x55d0a73d3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.878+0000 7fa27781f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d0a73a2cd0 con 0x55d0a73d3320 2024-05-08T18:05:03.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.878+0000 7fa27531e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73d3320 0x55d0a73d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.878+0000 7fa27531e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73d3320 0x55d0a73d3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57916/0 (socket says 172.21.15.50:57916) 2024-05-08T18:05:03.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.878+0000 7fa27531e640 1 -- 172.21.15.50:0/3313584592 learned_addr learned my addr 172.21.15.50:0/3313584592 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.878+0000 7fa27531e640 1 -- 172.21.15.50:0/3313584592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0a73d3d10 con 0x55d0a73d3320 2024-05-08T18:05:03.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.878+0000 7fa27531e640 1 --2- 172.21.15.50:0/3313584592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73d3320 0x55d0a73d3740 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa26800c010 tx=0x7fa268041d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a723b9dc987247df server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.878+0000 7fa158ff9640 1 -- 172.21.15.50:0/3313584592 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa268041e30 con 0x55d0a73d3320 2024-05-08T18:05:03.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.878+0000 7fa158ff9640 1 -- 172.21.15.50:0/3313584592 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa26803b9e0 con 0x55d0a73d3320 2024-05-08T18:05:03.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.878+0000 7fa158ff9640 1 -- 172.21.15.50:0/3313584592 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa268052cf0 con 0x55d0a73d3320 2024-05-08T18:05:03.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.879+0000 7fa27781f800 1 -- 172.21.15.50:0/3313584592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73d3320 msgr2=0x55d0a73d3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.879+0000 7fa27781f800 1 --2- 172.21.15.50:0/3313584592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73d3320 0x55d0a73d3740 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa26800c010 tx=0x7fa268041d00 comp rx=0 tx=0).stop 2024-05-08T18:05:03.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.879+0000 7fa27781f800 1 -- 172.21.15.50:0/3313584592 shutdown_connections 2024-05-08T18:05:03.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.879+0000 7fa27781f800 1 --2- 172.21.15.50:0/3313584592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73d3320 0x55d0a73d3740 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.879+0000 7fa27781f800 1 -- 172.21.15.50:0/3313584592 >> 172.21.15.50:0/3313584592 conn(0x55d0a73cf5f0 msgr2=0x55d0a73cd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.879+0000 7fa27781f800 1 -- 172.21.15.50:0/3313584592 shutdown_connections 2024-05-08T18:05:03.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.879+0000 7fa27781f800 1 -- 172.21.15.50:0/3313584592 wait complete. 2024-05-08T18:05:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.879+0000 7fa27781f800 1 Processor -- start 2024-05-08T18:05:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.879+0000 7fa27781f800 1 -- start start 2024-05-08T18:05:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.879+0000 7fa27781f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73d3320 0x55d0a73da740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.879+0000 7fa27781f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d0a73d43e0 con 0x55d0a73d3320 2024-05-08T18:05:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.880+0000 7fa27531e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73d3320 0x55d0a73da740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.880+0000 7fa27531e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73d3320 0x55d0a73da740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57922/0 (socket says 172.21.15.50:57922) 2024-05-08T18:05:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.880+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 learned_addr learned my addr 172.21.15.50:0/1890431860 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.880+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0a73dac80 con 0x55d0a73d3320 2024-05-08T18:05:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.880+0000 7fa27531e640 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73d3320 0x55d0a73da740 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa268041b40 tx=0x7fa268031fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.880+0000 7fa14affd640 1 -- 172.21.15.50:0/1890431860 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa268070070 con 0x55d0a73d3320 2024-05-08T18:05:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.880+0000 7fa14affd640 1 -- 172.21.15.50:0/1890431860 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa268018db0 con 0x55d0a73d3320 2024-05-08T18:05:03.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.880+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d0a73dac80 con 0x55d0a73d3320 2024-05-08T18:05:03.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.880+0000 7fa14affd640 1 -- 172.21.15.50:0/1890431860 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa268057050 con 0x55d0a73d3320 2024-05-08T18:05:03.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.880+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0a73dac80 con 0x55d0a73d3320 2024-05-08T18:05:03.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.881+0000 7fa14affd640 1 -- 172.21.15.50:0/1890431860 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa268073070 con 0x55d0a73d3320 2024-05-08T18:05:03.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.881+0000 7fa14affd640 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa14003bba0 0x7fa14003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.881+0000 7fa14affd640 1 -- 172.21.15.50:0/1890431860 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fa268076ca0 con 0x55d0a73d3320 2024-05-08T18:05:03.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.881+0000 7fa274b1d640 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa14003bba0 0x7fa14003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.882+0000 7fa274b1d640 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa14003bba0 0x7fa14003e060 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa26c052fd0 tx=0x7fa26c04f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.882+0000 7fa27781f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.882+0000 7fa27781f800 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d0a73e5d00 0x55d0a73e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.882+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73e8910 con 0x55d0a73e5d00 2024-05-08T18:05:03.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.882+0000 7fa275b1f640 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d0a73e5d00 0x55d0a73e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.882+0000 7fa275b1f640 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d0a73e5d00 0x55d0a73e81c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a73e5d00 2024-05-08T18:05:03.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa27781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa27781f800 20 realm 2024-05-08T18:05:03.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa27781f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73e8d20 con 0x55d0a73e5d00 2024-05-08T18:05:03.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a73e5d00 2024-05-08T18:05:03.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa27781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa27781f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:03.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa27781f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73e95f0 con 0x55d0a73e5d00 2024-05-08T18:05:03.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a73e5d00 2024-05-08T18:05:03.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa27781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa27781f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73e9990 con 0x55d0a73e5d00 2024-05-08T18:05:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a73e5d00 2024-05-08T18:05:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa27781f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa27781f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa27781f800 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d0a73ea6f0 0x55d0a73ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.883+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73ed0f0 con 0x55d0a73ea6f0 2024-05-08T18:05:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.884+0000 7fa27531e640 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d0a73ea6f0 0x55d0a73ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.884+0000 7fa27531e640 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d0a73ea6f0 0x55d0a73ecae0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.884+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.884+0000 7fa27781f800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.884+0000 7fa27781f800 20 searching for the correct realm 2024-05-08T18:05:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.884+0000 7fa27781f800 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d0a73ef0a0 0x55d0a73f1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.884+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f1c30 con 0x55d0a73ef0a0 2024-05-08T18:05:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.885+0000 7fa274b1d640 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d0a73ef0a0 0x55d0a73f1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.885+0000 7fa274b1d640 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d0a73ef0a0 0x55d0a73f1560 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.885+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa26c081070 con 0x55d0a73ef0a0 2024-05-08T18:05:03.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.885+0000 7fa274b1d640 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa26c076040 0x7fa26c05e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.885+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f1c30 con 0x7fa26c076040 2024-05-08T18:05:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.885+0000 7fa275b1f640 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa26c076040 0x7fa26c05e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.885+0000 7fa275b1f640 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa26c076040 0x7fa26c05e480 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.886+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa27000a4a0 con 0x7fa26c076040 2024-05-08T18:05:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.886+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73e9990 con 0x55d0a73ea6f0 2024-05-08T18:05:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.886+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.886+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73e9990 con 0x55d0a73e5d00 2024-05-08T18:05:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.886+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a73e5d00 2024-05-08T18:05:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.886+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73e9990 con 0x55d0a73e5d00 2024-05-08T18:05:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.887+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a73e5d00 2024-05-08T18:05:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.887+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73e9990 con 0x55d0a73ea6f0 2024-05-08T18:05:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.887+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.887+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73e9990 con 0x55d0a73ea6f0 2024-05-08T18:05:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.887+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.887+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73e9990 con 0x7fa26c076040 2024-05-08T18:05:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.887+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa27000a4a0 con 0x7fa26c076040 2024-05-08T18:05:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.887+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73e9990 con 0x55d0a73ef0a0 2024-05-08T18:05:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.888+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa26c081070 con 0x55d0a73ef0a0 2024-05-08T18:05:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.888+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73e9990 con 0x55d0a73e5d00 2024-05-08T18:05:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.888+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a73e5d00 2024-05-08T18:05:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.888+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73e9990 con 0x55d0a73ea6f0 2024-05-08T18:05:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.888+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.888+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73e9990 con 0x55d0a73e5d00 2024-05-08T18:05:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.888+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a73e5d00 2024-05-08T18:05:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.888+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73e9990 con 0x7fa26c076040 2024-05-08T18:05:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.888+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fa27000a4a0 con 0x7fa26c076040 2024-05-08T18:05:03.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.888+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73e9990 con 0x55d0a73ef0a0 2024-05-08T18:05:03.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.889+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa26c081070 con 0x55d0a73ef0a0 2024-05-08T18:05:03.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.889+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x55d0a73ef0a0 2024-05-08T18:05:03.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.889+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa26c081070 con 0x55d0a73ef0a0 2024-05-08T18:05:03.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.889+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x55d0a73ea6f0 2024-05-08T18:05:03.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.889+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.889+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x55d0a73ef0a0 2024-05-08T18:05:03.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.889+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa26c081070 con 0x55d0a73ef0a0 2024-05-08T18:05:03.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.889+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x55d0a73ef0a0 2024-05-08T18:05:03.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.890+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fa26c081070 con 0x55d0a73ef0a0 2024-05-08T18:05:03.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.890+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x55d0a73e5d00 2024-05-08T18:05:03.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.890+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a73e5d00 2024-05-08T18:05:03.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.890+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x55d0a73e5d00 2024-05-08T18:05:03.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.890+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a73e5d00 2024-05-08T18:05:03.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.890+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x55d0a73ea6f0 2024-05-08T18:05:03.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.890+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.890+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x55d0a73ef0a0 2024-05-08T18:05:03.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.891+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa26c081070 con 0x55d0a73ef0a0 2024-05-08T18:05:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.891+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x7fa26c076040 2024-05-08T18:05:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.891+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa27000a4a0 con 0x7fa26c076040 2024-05-08T18:05:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.891+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x55d0a73e5d00 2024-05-08T18:05:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.891+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a73e5d00 2024-05-08T18:05:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.891+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x55d0a73ea6f0 2024-05-08T18:05:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.892+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.892+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x55d0a73ef0a0 2024-05-08T18:05:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.892+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa26c081070 con 0x55d0a73ef0a0 2024-05-08T18:05:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.892+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x55d0a73ea6f0 2024-05-08T18:05:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.892+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.892+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x7fa26c076040 2024-05-08T18:05:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.893+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa27000a4a0 con 0x7fa26c076040 2024-05-08T18:05:03.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.893+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x55d0a73ef0a0 2024-05-08T18:05:03.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.893+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa26c081070 con 0x55d0a73ef0a0 2024-05-08T18:05:03.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.893+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x55d0a73ea6f0 2024-05-08T18:05:03.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.893+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.893+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x55d0a73e5d00 2024-05-08T18:05:03.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.893+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a73e5d00 2024-05-08T18:05:03.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.893+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fa26c090040 con 0x7fa26c076040 2024-05-08T18:05:03.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa27000a4a0 con 0x7fa26c076040 2024-05-08T18:05:03.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:03.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:03.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:03.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:03.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:03.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:03.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73ed9b0 con 0x55d0a73e5d00 2024-05-08T18:05:03.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a73e5d00 2024-05-08T18:05:03.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73ed9b0 con 0x55d0a73e5d00 2024-05-08T18:05:03.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa27000a4a0 con 0x55d0a73e5d00 2024-05-08T18:05:03.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:03.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73ed9b0 con 0x55d0a73ef0a0 2024-05-08T18:05:03.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fa26c081070 con 0x55d0a73ef0a0 2024-05-08T18:05:03.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:03.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 20 zone default found 2024-05-08T18:05:03.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 4 Realm: () 2024-05-08T18:05:03.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:03.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:03.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:03.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 20 zonegroup default 2024-05-08T18:05:03.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.894+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73ee2e0 con 0x55d0a73ea6f0 2024-05-08T18:05:03.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.895+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.895+0000 7fa27781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.895+0000 7fa27781f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.895+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f3af0 con 0x55d0a73ef0a0 2024-05-08T18:05:03.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.895+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa26c081070 con 0x55d0a73ef0a0 2024-05-08T18:05:03.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.895+0000 7fa27781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.895+0000 7fa27781f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.895+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f3fc0 con 0x55d0a73ef0a0 2024-05-08T18:05:03.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.895+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa26c081070 con 0x55d0a73ef0a0 2024-05-08T18:05:03.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.895+0000 7fa27781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.895+0000 7fa27781f800 20 started sync module instance, tier type = 2024-05-08T18:05:03.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.895+0000 7fa27781f800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:03.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.895+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f2f30 con 0x7fa26c076040 2024-05-08T18:05:03.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.896+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa27000a4a0 con 0x7fa26c076040 2024-05-08T18:05:03.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.896+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94354647495760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f2f30 con 0x7fa26c076040 2024-05-08T18:05:03.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.896+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f4a40 con 0x55d0a73e5d00 2024-05-08T18:05:03.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.897+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94354647495760] v27'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa270040040 con 0x7fa26c076040 2024-05-08T18:05:03.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.898+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa270040040 con 0x55d0a73e5d00 2024-05-08T18:05:03.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.898+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94354647500656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f2f30 con 0x55d0a73e5d00 2024-05-08T18:05:03.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.898+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f54e0 con 0x55d0a73ef0a0 2024-05-08T18:05:03.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.899+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94354647500656] v27'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa270040040 con 0x55d0a73e5d00 2024-05-08T18:05:03.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.899+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa26c081070 con 0x55d0a73ef0a0 2024-05-08T18:05:03.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.899+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94354647493248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f2ea0 con 0x55d0a73ef0a0 2024-05-08T18:05:03.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.899+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f63a0 con 0x7fa26c076040 2024-05-08T18:05:03.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.900+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa270040040 con 0x7fa26c076040 2024-05-08T18:05:03.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.900+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94354647507152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f6c40 con 0x7fa26c076040 2024-05-08T18:05:03.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.900+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f7670 con 0x55d0a73e5d00 2024-05-08T18:05:03.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.900+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94354647493248] v27'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa26c081070 con 0x55d0a73ef0a0 2024-05-08T18:05:03.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.901+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa270040040 con 0x55d0a73e5d00 2024-05-08T18:05:03.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.901+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94354647505104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f7630 con 0x55d0a73e5d00 2024-05-08T18:05:03.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.901+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94354647507152] v27'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa270040040 con 0x7fa26c076040 2024-05-08T18:05:03.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.901+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f6790 con 0x55d0a73ea6f0 2024-05-08T18:05:03.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.902+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.902+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94354647505104] v27'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa270040040 con 0x55d0a73e5d00 2024-05-08T18:05:03.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.902+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94354647514112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f8020 con 0x55d0a73ea6f0 2024-05-08T18:05:03.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.902+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f89e0 con 0x55d0a73ea6f0 2024-05-08T18:05:03.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.903+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94354647514112] v27'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.903+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.903+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94354647511424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f9230 con 0x55d0a73ea6f0 2024-05-08T18:05:03.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.903+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f9a80 con 0x55d0a73ea6f0 2024-05-08T18:05:03.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.904+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94354647511424] v27'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.904+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.904+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94354647520720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73f9ea0 con 0x55d0a73ea6f0 2024-05-08T18:05:03.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.904+0000 7fa27781f800 20 add_watcher() i=0 2024-05-08T18:05:03.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.904+0000 7fa27781f800 20 add_watcher() i=1 2024-05-08T18:05:03.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.904+0000 7fa27781f800 20 add_watcher() i=2 2024-05-08T18:05:03.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.904+0000 7fa27781f800 20 add_watcher() i=3 2024-05-08T18:05:03.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.904+0000 7fa27781f800 20 add_watcher() i=4 2024-05-08T18:05:03.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.904+0000 7fa27781f800 20 add_watcher() i=5 2024-05-08T18:05:03.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.904+0000 7fa27781f800 20 add_watcher() i=6 2024-05-08T18:05:03.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.905+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94354647520720] v27'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.905+0000 7fa27781f800 20 add_watcher() i=7 2024-05-08T18:05:03.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.905+0000 7fa27781f800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:03.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.905+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73fd1a0 con 0x55d0a73e5d00 2024-05-08T18:05:03.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.906+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fa270040040 con 0x55d0a73e5d00 2024-05-08T18:05:03.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.906+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94354647522144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a73fa430 con 0x55d0a73e5d00 2024-05-08T18:05:03.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.907+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94354647522144] v27'50 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa270040040 con 0x55d0a73e5d00 2024-05-08T18:05:03.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.907+0000 7fa12bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.907+0000 7fa27781f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.907+0000 7fa27781f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.907+0000 7fa27781f800 5 note: GC not initialized 2024-05-08T18:05:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.907+0000 7fa12afe5640 20 reqs_thread_entry: start 2024-05-08T18:05:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 20 Filter name: none 2024-05-08T18:05:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa128fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a74307e0 con 0x55d0a73e5d00 2024-05-08T18:05:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7fa270040040 con 0x55d0a73e5d00 2024-05-08T18:05:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2024-05-08T18:05:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2024-05-08T18:05:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.3 8:c162d5d1:users.email::foo@foo.com:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a742ef80 con 0x55d0a73e5d00 2024-05-08T18:05:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 foo@foo.com [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 197+0+0 (crc 0 0 0) 0x7fa270040040 con 0x55d0a73e5d00 2024-05-08T18:05:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x0 2024-05-08T18:05:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-05-08T18:05:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : miss 2024-05-08T18:05:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.908+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:64 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a742f370 con 0x55d0a73ea6f0 2024-05-08T18:05:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.909+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(64 9te6NH5mcdcq0Tc5i8i1 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.909+0000 7fa27781f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:03.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.909+0000 7fa27781f800 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x0 2024-05-08T18:05:03.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.909+0000 7fa27781f800 10 adding default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-05-08T18:05:03.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.909+0000 7fa27781f800 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2024-05-08T18:05:03.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.909+0000 7fa27781f800 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2024-05-08T18:05:03.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.909+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a7430290 con 0x55d0a73e5d00 2024-05-08T18:05:03.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.910+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 foo [delete,create,call,writefull 0~391] v27'1 uv1 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7fa270040040 con 0x55d0a73e5d00 2024-05-08T18:05:03.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.910+0000 7fa27781f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:03.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.910+0000 7fa27781f800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:03.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.910+0000 7fa27781f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-05-08T18:05:03.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.910+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify cookie 94354647742384 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a7434360 con 0x55d0a73e5d00 2024-05-08T18:05:03.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.910+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== watch-notify(notify (1) cookie 94354647505104 notify 115964116992 ret 0) ==== 609+0+0 (crc 0 0 0) 0x55d0a742fdc0 con 0x55d0a73e5d00 2024-05-08T18:05:03.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.910+0000 7fa158ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116992 cookie 94354647505104 notifier 4284 bl.length()=567 2024-05-08T18:05:03.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.910+0000 7fa158ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:03.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.910+0000 7fa158ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:03.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.910+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(66 notify.4 [notify cookie 94354647742384 out=8b] v0'0 uv24 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fa270040040 con 0x55d0a73e5d00 2024-05-08T18:05:03.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.910+0000 7fa158ff9640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fa138003ea0 con 0x55d0a73e5d00 2024-05-08T18:05:03.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.911+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(67 notify.4 [notify-ack] v0'0 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa270040040 con 0x55d0a73e5d00 2024-05-08T18:05:03.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.911+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== watch-notify(notify_complete (2) cookie 94354647742384 notify 115964116992 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d0a742f540 con 0x55d0a73e5d00 2024-05-08T18:05:03.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.911+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a742f7b0 con 0x55d0a73e5d00 2024-05-08T18:05:03.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.912+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(68 foo@foo.com [delete,create,writefull 0~7] v27'1 uv1 ondisk = 0) ==== 239+0+0 (crc 0 0 0) 0x7fa270040040 con 0x55d0a73e5d00 2024-05-08T18:05:03.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.912+0000 7fa27781f800 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2024-05-08T18:05:03.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.912+0000 7fa27781f800 10 moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-05-08T18:05:03.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.912+0000 7fa27781f800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2024-05-08T18:05:03.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.912+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [notify cookie 94354647742384 in=181b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a7434790 con 0x7fa26c076040 2024-05-08T18:05:03.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.913+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== watch-notify(notify (1) cookie 94354647507152 notify 115964116992 ret 0) ==== 211+0+0 (crc 0 0 0) 0x55d0a742f540 con 0x7fa26c076040 2024-05-08T18:05:03.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.913+0000 7fa158ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116992 cookie 94354647507152 notifier 4284 bl.length()=169 2024-05-08T18:05:03.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.913+0000 7fa158ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2024-05-08T18:05:03.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.913+0000 7fa158ff9640 10 rgw watcher librados: moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-05-08T18:05:03.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.913+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(69 notify.3 [notify cookie 94354647742384 out=8b] v0'0 uv24 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fa270040040 con 0x7fa26c076040 2024-05-08T18:05:03.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.913+0000 7fa158ff9640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fa138003ea0 con 0x7fa26c076040 2024-05-08T18:05:03.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.913+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== watch-notify(notify_complete (2) cookie 94354647742384 notify 115964116992 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fa270038450 con 0x7fa26c076040 2024-05-08T18:05:03.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.913+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(70 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa270040040 con 0x7fa26c076040 2024-05-08T18:05:03.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.913+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a742f7b0 con 0x55d0a73ea6f0 2024-05-08T18:05:03.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.914+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(71 9te6NH5mcdcq0Tc5i8i1 [delete,create,writefull 0~7] v27'1 uv1 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.914+0000 7fa27781f800 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2024-05-08T18:05:03.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.914+0000 7fa27781f800 10 moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-05-08T18:05:03.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.914+0000 7fa27781f800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2024-05-08T18:05:03.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.914+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 7.e 7:7759931f:::notify.3:head [notify cookie 94354647742384 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d0a7433530 con 0x7fa26c076040 2024-05-08T18:05:03.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.915+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== watch-notify(notify (1) cookie 94354647507152 notify 115964116993 ret 0) ==== 219+0+0 (crc 0 0 0) 0x7fa270038450 con 0x7fa26c076040 2024-05-08T18:05:03.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.915+0000 7fa14bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116993 cookie 94354647507152 notifier 4284 bl.length()=177 2024-05-08T18:05:03.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.915+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(72 notify.3 [notify cookie 94354647742384 out=8b] v0'0 uv24 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fa270040040 con 0x7fa26c076040 2024-05-08T18:05:03.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.915+0000 7fa14bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2024-05-08T18:05:03.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.915+0000 7fa14bfff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-05-08T18:05:03.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.915+0000 7fa14bfff640 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fa09c003400 con 0x7fa26c076040 2024-05-08T18:05:03.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.915+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== watch-notify(notify_complete (2) cookie 94354647742384 notify 115964116993 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fa270038830 con 0x7fa26c076040 2024-05-08T18:05:03.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.915+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(73 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa270040040 con 0x7fa26c076040 2024-05-08T18:05:03.949 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:03.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.916+0000 7fa12bfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:03.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.916+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:74 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94354647522144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a7433530 con 0x55d0a73e5d00 2024-05-08T18:05:03.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.917+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(74 data_loggenerations_metadata [watch unwatch cookie 94354647522144] v27'51 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa270040040 con 0x55d0a73e5d00 2024-05-08T18:05:03.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.917+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:75 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94354647495760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a7433530 con 0x7fa26c076040 2024-05-08T18:05:03.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.918+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(75 notify.0 [watch unwatch cookie 94354647495760] v27'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa270040040 con 0x7fa26c076040 2024-05-08T18:05:03.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.918+0000 7fa27781f800 20 remove_watcher() i=0 2024-05-08T18:05:03.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.918+0000 7fa27781f800 2 removed watcher, disabling cache 2024-05-08T18:05:03.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.918+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94354647500656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a7433530 con 0x55d0a73e5d00 2024-05-08T18:05:03.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.919+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(76 notify.1 [watch unwatch cookie 94354647500656] v27'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa270040040 con 0x55d0a73e5d00 2024-05-08T18:05:03.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.919+0000 7fa27781f800 20 remove_watcher() i=1 2024-05-08T18:05:03.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.919+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94354647493248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a7433530 con 0x55d0a73ef0a0 2024-05-08T18:05:03.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.920+0000 7fa274b1d640 1 -- 172.21.15.50:0/1890431860 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(77 notify.2 [watch unwatch cookie 94354647493248] v27'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa26c081070 con 0x55d0a73ef0a0 2024-05-08T18:05:03.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.920+0000 7fa27781f800 20 remove_watcher() i=2 2024-05-08T18:05:03.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.920+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94354647507152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a7433530 con 0x7fa26c076040 2024-05-08T18:05:03.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.921+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.2 v2:172.21.15.50:6811/3039646617 20 ==== osd_op_reply(78 notify.3 [watch unwatch cookie 94354647507152] v27'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa270040040 con 0x7fa26c076040 2024-05-08T18:05:03.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.921+0000 7fa27781f800 20 remove_watcher() i=3 2024-05-08T18:05:03.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.921+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:79 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94354647505104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a7433530 con 0x55d0a73e5d00 2024-05-08T18:05:03.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.922+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(79 notify.4 [watch unwatch cookie 94354647505104] v27'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa270040040 con 0x55d0a73e5d00 2024-05-08T18:05:03.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.922+0000 7fa27781f800 20 remove_watcher() i=4 2024-05-08T18:05:03.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.922+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:80 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94354647514112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a7433530 con 0x55d0a73ea6f0 2024-05-08T18:05:03.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.923+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(80 notify.5 [watch unwatch cookie 94354647514112] v27'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.923+0000 7fa27781f800 20 remove_watcher() i=5 2024-05-08T18:05:03.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.923+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:81 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94354647511424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a7433530 con 0x55d0a73ea6f0 2024-05-08T18:05:03.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.924+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(81 notify.6 [watch unwatch cookie 94354647511424] v27'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.924+0000 7fa27781f800 20 remove_watcher() i=6 2024-05-08T18:05:03.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.924+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:82 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94354647520720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d0a7433530 con 0x55d0a73ea6f0 2024-05-08T18:05:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27531e640 1 -- 172.21.15.50:0/1890431860 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(82 notify.7 [watch unwatch cookie 94354647520720] v27'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa268072030 con 0x55d0a73ea6f0 2024-05-08T18:05:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 20 remove_watcher() i=7 2024-05-08T18:05:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d0a73ea6f0 msgr2=0x55d0a73ecae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d0a73ea6f0 0x55d0a73ecae0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d0a73e5d00 msgr2=0x55d0a73e81c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d0a73e5d00 0x55d0a73e81c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa26c076040 msgr2=0x7fa26c05e480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa26c076040 0x7fa26c05e480 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d0a73ef0a0 msgr2=0x55d0a73f1560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d0a73ef0a0 0x55d0a73f1560 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa14003bba0 msgr2=0x7fa14003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa14003bba0 0x7fa14003e060 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa26c052fd0 tx=0x7fa26c04f000 comp rx=0 tx=0).stop 2024-05-08T18:05:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73d3320 msgr2=0x55d0a73da740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73d3320 0x55d0a73da740 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa268041b40 tx=0x7fa268031fc0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa275b1f640 1 -- 172.21.15.50:0/1890431860 reap_dead start 2024-05-08T18:05:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 shutdown_connections 2024-05-08T18:05:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 --2- 172.21.15.50:0/1890431860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73d3320 0x55d0a73da740 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 >> 172.21.15.50:0/1890431860 conn(0x55d0a73cf5f0 msgr2=0x55d0a73ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 shutdown_connections 2024-05-08T18:05:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.925+0000 7fa27781f800 1 -- 172.21.15.50:0/1890431860 wait complete. 2024-05-08T18:05:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 -- 172.21.15.50:0/1236337339 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa14c03bab0 msgr2=0x7fa14c03df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 --2- 172.21.15.50:0/1236337339 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa14c03bab0 0x7fa14c03df70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55d0a73abf10 tx=0x7fa26c005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 -- 172.21.15.50:0/1236337339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73c5ff0 msgr2=0x55d0a73c6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 --2- 172.21.15.50:0/1236337339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73c5ff0 0x55d0a73c6410 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa26800b6c0 tx=0x7fa268051f40 comp rx=0 tx=0).stop 2024-05-08T18:05:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 -- 172.21.15.50:0/1236337339 shutdown_connections 2024-05-08T18:05:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 --2- 172.21.15.50:0/1236337339 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa14c03bab0 0x7fa14c03df70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 --2- 172.21.15.50:0/1236337339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a73c5ff0 0x55d0a73c6410 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 -- 172.21.15.50:0/1236337339 >> 172.21.15.50:0/1236337339 conn(0x55d0a73ba090 msgr2=0x55d0a73ba490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 -- 172.21.15.50:0/1236337339 shutdown_connections 2024-05-08T18:05:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 -- 172.21.15.50:0/1236337339 wait complete. 2024-05-08T18:05:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 -- 172.21.15.50:0/396229249 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d0a73b12e0 msgr2=0x55d0a73b36d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d0a73b12e0 0x55d0a73b36d0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 -- 172.21.15.50:0/396229249 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d0a72afa10 msgr2=0x55d0a73aea20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d0a72afa10 0x55d0a73aea20 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 -- 172.21.15.50:0/396229249 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d0a73b51d0 msgr2=0x55d0a73b75c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d0a73b51d0 0x55d0a73b75c0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 -- 172.21.15.50:0/396229249 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa15c03bcf0 msgr2=0x7fa15c03e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa15c03bcf0 0x7fa15c03e1b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa26c054000 tx=0x7fa26c04e000 comp rx=0 tx=0).stop 2024-05-08T18:05:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 -- 172.21.15.50:0/396229249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72ab580 msgr2=0x55d0a73a77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 --2- 172.21.15.50:0/396229249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d0a72ab580 0x55d0a73a77e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa2680012f0 tx=0x7fa268000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa275b1f640 1 -- 172.21.15.50:0/396229249 reap_dead start 2024-05-08T18:05:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 -- 172.21.15.50:0/396229249 shutdown_connections 2024-05-08T18:05:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 -- 172.21.15.50:0/396229249 >> 172.21.15.50:0/396229249 conn(0x55d0a72a6ae0 msgr2=0x55d0a7396a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 -- 172.21.15.50:0/396229249 shutdown_connections 2024-05-08T18:05:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:03.926+0000 7fa27781f800 1 -- 172.21.15.50:0/396229249 wait complete. 2024-05-08T18:05:03.965 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:03.965 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2024-05-08T18:05:03.966 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2024-05-08T18:05:03.966 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --email foo@foo.com 2024-05-08T18:05:04.017 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:04.017 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:04.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.020+0000 7f3efdd33800 1 Processor -- start 2024-05-08T18:05:04.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.020+0000 7f3efdd33800 1 -- start start 2024-05-08T18:05:04.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.020+0000 7f3efdd33800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55efe475e710 0x55efe475eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.020+0000 7f3efdd33800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55efe475f1b0 con 0x55efe475f8d0 2024-05-08T18:05:04.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.020+0000 7f3efdd33800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55efe45fee20 con 0x55efe475e710 2024-05-08T18:05:04.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.021+0000 7f3ef99ca640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55efe475e710 0x55efe475eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.021+0000 7f3ef99ca640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55efe475e710 0x55efe475eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57934/0 (socket says 172.21.15.50:57934) 2024-05-08T18:05:04.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.021+0000 7f3ef99ca640 1 -- 172.21.15.50:0/1939937954 learned_addr learned my addr 172.21.15.50:0/1939937954 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.021+0000 7f3ef91c9640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55efe475f8d0 0x55efe475fd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:53058/0 (socket says 172.21.15.50:53058) 2024-05-08T18:05:04.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.021+0000 7f3ef99ca640 1 -- 172.21.15.50:0/1939937954 >> v1:172.21.15.46:6789/0 conn(0x55efe475f8d0 legacy=0x55efe475fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.021+0000 7f3ef99ca640 1 -- 172.21.15.50:0/1939937954 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe4760490 con 0x55efe475e710 2024-05-08T18:05:04.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.021+0000 7f3ef89c8640 1 -- 172.21.15.50:0/1939937954 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 496665270 0 0) 0x55efe475f1b0 con 0x55efe475f8d0 2024-05-08T18:05:04.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.023+0000 7f3ef99ca640 1 --2- 172.21.15.50:0/1939937954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 0x55efe475eb30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00a520 tx=0x7f3eec01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc5d2cdf6101264 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.023+0000 7f3ef89c8640 1 -- 172.21.15.50:0/1939937954 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eec057070 con 0x55efe475e710 2024-05-08T18:05:04.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.023+0000 7f3ef89c8640 1 -- 172.21.15.50:0/1939937954 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eec052570 con 0x55efe475e710 2024-05-08T18:05:04.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.023+0000 7f3ef89c8640 1 -- 172.21.15.50:0/1939937954 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eec052870 con 0x55efe475e710 2024-05-08T18:05:04.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.023+0000 7f3efdd33800 1 -- 172.21.15.50:0/1939937954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 msgr2=0x55efe475eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.023+0000 7f3efdd33800 1 --2- 172.21.15.50:0/1939937954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 0x55efe475eb30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00a520 tx=0x7f3eec01d010 comp rx=0 tx=0).stop 2024-05-08T18:05:04.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.023+0000 7f3efdd33800 1 -- 172.21.15.50:0/1939937954 shutdown_connections 2024-05-08T18:05:04.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.023+0000 7f3efdd33800 1 --2- 172.21.15.50:0/1939937954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 0x55efe475eb30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.023+0000 7f3efdd33800 1 -- 172.21.15.50:0/1939937954 >> 172.21.15.50:0/1939937954 conn(0x55efe4759c90 msgr2=0x55efe475c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.024+0000 7f3efdd33800 1 -- 172.21.15.50:0/1939937954 shutdown_connections 2024-05-08T18:05:04.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.024+0000 7f3efdd33800 1 -- 172.21.15.50:0/1939937954 wait complete. 2024-05-08T18:05:04.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.039+0000 7f3efdd33800 1 Processor -- start 2024-05-08T18:05:04.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.039+0000 7f3efdd33800 1 -- start start 2024-05-08T18:05:04.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.039+0000 7f3efdd33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 0x55efe46d3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.039+0000 7f3efdd33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55efe46a9a30 con 0x55efe475e710 2024-05-08T18:05:04.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.039+0000 7f3ef99ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 0x55efe46d3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.039+0000 7f3ef99ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 0x55efe46d3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57936/0 (socket says 172.21.15.50:57936) 2024-05-08T18:05:04.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.039+0000 7f3ef99ca640 1 -- 172.21.15.50:0/460614476 learned_addr learned my addr 172.21.15.50:0/460614476 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.039+0000 7f3ef99ca640 1 -- 172.21.15.50:0/460614476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe46d24c0 con 0x55efe475e710 2024-05-08T18:05:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.040+0000 7f3ef99ca640 1 --2- 172.21.15.50:0/460614476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 0x55efe46d3f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3eec055040 tx=0x7f3eec04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b8557c590981a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.040+0000 7f3de7e07640 1 -- 172.21.15.50:0/460614476 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eec057040 con 0x55efe475e710 2024-05-08T18:05:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.040+0000 7f3de7e07640 1 -- 172.21.15.50:0/460614476 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eec052870 con 0x55efe475e710 2024-05-08T18:05:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.040+0000 7f3de7e07640 1 -- 172.21.15.50:0/460614476 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eec052b90 con 0x55efe475e710 2024-05-08T18:05:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.040+0000 7f3efdd33800 1 -- 172.21.15.50:0/460614476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 msgr2=0x55efe46d3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.040+0000 7f3efdd33800 1 --2- 172.21.15.50:0/460614476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 0x55efe46d3f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3eec055040 tx=0x7f3eec04c000 comp rx=0 tx=0).stop 2024-05-08T18:05:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.041+0000 7f3efdd33800 1 -- 172.21.15.50:0/460614476 shutdown_connections 2024-05-08T18:05:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.041+0000 7f3efdd33800 1 --2- 172.21.15.50:0/460614476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 0x55efe46d3f00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.041+0000 7f3efdd33800 1 -- 172.21.15.50:0/460614476 >> 172.21.15.50:0/460614476 conn(0x55efe4759c90 msgr2=0x55efe484b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.041+0000 7f3efdd33800 1 -- 172.21.15.50:0/460614476 shutdown_connections 2024-05-08T18:05:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.041+0000 7f3efdd33800 1 -- 172.21.15.50:0/460614476 wait complete. 2024-05-08T18:05:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.041+0000 7f3efdd33800 1 Processor -- start 2024-05-08T18:05:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.041+0000 7f3efdd33800 1 -- start start 2024-05-08T18:05:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.041+0000 7f3efdd33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 0x55efe48596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.041+0000 7f3efdd33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55efe4767da0 con 0x55efe475e710 2024-05-08T18:05:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.041+0000 7f3ef99ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 0x55efe48596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.041+0000 7f3ef99ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 0x55efe48596f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57948/0 (socket says 172.21.15.50:57948) 2024-05-08T18:05:04.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.041+0000 7f3ef99ca640 1 -- 172.21.15.50:0/1600366031 learned_addr learned my addr 172.21.15.50:0/1600366031 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.042+0000 7f3ef99ca640 1 -- 172.21.15.50:0/1600366031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe4859c30 con 0x55efe475e710 2024-05-08T18:05:04.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.042+0000 7f3ef99ca640 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 0x55efe48596f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0524f0 tx=0x7f3eec003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.042+0000 7f3de6604640 1 -- 172.21.15.50:0/1600366031 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eec073070 con 0x55efe475e710 2024-05-08T18:05:04.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.042+0000 7f3de6604640 1 -- 172.21.15.50:0/1600366031 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eec05d570 con 0x55efe475e710 2024-05-08T18:05:04.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.042+0000 7f3efdd33800 1 -- 172.21.15.50:0/1600366031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55efe4859ec0 con 0x55efe475e710 2024-05-08T18:05:04.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.042+0000 7f3de6604640 1 -- 172.21.15.50:0/1600366031 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eec057070 con 0x55efe475e710 2024-05-08T18:05:04.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.042+0000 7f3efdd33800 1 -- 172.21.15.50:0/1600366031 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efe485cb70 con 0x55efe475e710 2024-05-08T18:05:04.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.043+0000 7f3de6604640 1 -- 172.21.15.50:0/1600366031 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3eec076070 con 0x55efe475e710 2024-05-08T18:05:04.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.043+0000 7f3de6604640 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3de003bba0 0x7f3de003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.043+0000 7f3de6604640 1 -- 172.21.15.50:0/1600366031 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f3eec0941a0 con 0x55efe475e710 2024-05-08T18:05:04.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.043+0000 7f3ef91c9640 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3de003bba0 0x7f3de003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.043+0000 7f3efdd33800 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55efe475f8d0 0x55efe4860930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.043+0000 7f3efdd33800 1 -- 172.21.15.50:0/1600366031 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe4860e70 con 0x55efe475f8d0 2024-05-08T18:05:04.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.044+0000 7f3efa1cb640 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55efe475f8d0 0x55efe4860930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.044+0000 7f3ef91c9640 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3de003bba0 0x7f3de003e060 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0000c00 tx=0x7f3ef00013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.044+0000 7f3efa1cb640 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55efe475f8d0 0x55efe4860930 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.044+0000 7f3efa1cb640 1 -- 172.21.15.50:0/1600366031 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe475f8d0 2024-05-08T18:05:04.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.044+0000 7f3efdd33800 1 -- 172.21.15.50:0/1600366031 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe485dee0 con 0x55efe475f8d0 2024-05-08T18:05:04.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.044+0000 7f3efa1cb640 1 -- 172.21.15.50:0/1600366031 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe475f8d0 2024-05-08T18:05:04.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.045+0000 7f3efdd33800 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55efe4863190 0x55efe4865580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.045+0000 7f3efdd33800 1 -- 172.21.15.50:0/1600366031 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe4865c70 con 0x55efe4863190 2024-05-08T18:05:04.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.045+0000 7f3ef99ca640 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55efe4863190 0x55efe4865580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.045+0000 7f3ef99ca640 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55efe4863190 0x55efe4865580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.045+0000 7f3ef99ca640 1 -- 172.21.15.50:0/1600366031 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f3eec075030 con 0x55efe4863190 2024-05-08T18:05:04.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.045+0000 7f3efdd33800 1 -- 172.21.15.50:0/1600366031 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe4862030 con 0x55efe475f8d0 2024-05-08T18:05:04.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.046+0000 7f3efa1cb640 1 -- 172.21.15.50:0/1600366031 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe475f8d0 2024-05-08T18:05:04.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.046+0000 7f3efdd33800 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55efe48670d0 0x55efe48694c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.046+0000 7f3efdd33800 1 -- 172.21.15.50:0/1600366031 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe4869bc0 con 0x55efe48670d0 2024-05-08T18:05:04.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.046+0000 7f3ef91c9640 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55efe48670d0 0x55efe48694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.046+0000 7f3ef91c9640 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55efe48670d0 0x55efe48694c0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.046+0000 7f3ef91c9640 1 -- 172.21.15.50:0/1600366031 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3ef000f670 con 0x55efe48670d0 2024-05-08T18:05:04.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.047+0000 7f3efdd33800 1 Processor -- start 2024-05-08T18:05:04.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.047+0000 7f3efdd33800 1 -- start start 2024-05-08T18:05:04.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.047+0000 7f3efdd33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe4877f30 0x55efe4878350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.047+0000 7f3efdd33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55efe4650330 con 0x55efe4877f30 2024-05-08T18:05:04.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.047+0000 7f3ef99ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe4877f30 0x55efe4878350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.047+0000 7f3ef99ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe4877f30 0x55efe4878350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57958/0 (socket says 172.21.15.50:57958) 2024-05-08T18:05:04.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.047+0000 7f3ef99ca640 1 -- 172.21.15.50:0/1944046135 learned_addr learned my addr 172.21.15.50:0/1944046135 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.047+0000 7f3ef99ca640 1 -- 172.21.15.50:0/1944046135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe4878890 con 0x55efe4877f30 2024-05-08T18:05:04.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.047+0000 7f3ef99ca640 1 --2- 172.21.15.50:0/1944046135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe4877f30 0x55efe4878350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3eec05c720 tx=0x7f3eec053f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.048+0000 7f3ddf7fe640 1 -- 172.21.15.50:0/1944046135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eec076ba0 con 0x55efe4877f30 2024-05-08T18:05:04.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.048+0000 7f3ddf7fe640 1 -- 172.21.15.50:0/1944046135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eec076d40 con 0x55efe4877f30 2024-05-08T18:05:04.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.048+0000 7f3efdd33800 1 -- 172.21.15.50:0/1944046135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55efe4879300 con 0x55efe4877f30 2024-05-08T18:05:04.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.048+0000 7f3ddf7fe640 1 -- 172.21.15.50:0/1944046135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eec05ca70 con 0x55efe4877f30 2024-05-08T18:05:04.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.048+0000 7f3efdd33800 1 -- 172.21.15.50:0/1944046135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efe4879800 con 0x55efe4877f30 2024-05-08T18:05:04.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.048+0000 7f3ddf7fe640 1 -- 172.21.15.50:0/1944046135 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3eec05cc10 con 0x55efe4877f30 2024-05-08T18:05:04.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.049+0000 7f3ddf7fe640 1 --2- 172.21.15.50:0/1944046135 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dd403bba0 0x7f3dd403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.049+0000 7f3ddf7fe640 1 -- 172.21.15.50:0/1944046135 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f3eec086c80 con 0x55efe4877f30 2024-05-08T18:05:04.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.049+0000 7f3ef91c9640 1 --2- 172.21.15.50:0/1944046135 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dd403bba0 0x7f3dd403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.049+0000 7f3efdd33800 1 Processor -- start 2024-05-08T18:05:04.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.049+0000 7f3efdd33800 1 -- start start 2024-05-08T18:05:04.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.049+0000 7f3ef91c9640 1 --2- 172.21.15.50:0/1944046135 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dd403bba0 0x7f3dd403e060 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55efe4861740 tx=0x7f3ef0006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.049+0000 7f3efdd33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe48850e0 0x55efe4885500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.049+0000 7f3efdd33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55efe46d2780 con 0x55efe48850e0 2024-05-08T18:05:04.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.049+0000 7f3ef99ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe48850e0 0x55efe4885500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.049+0000 7f3ef99ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe48850e0 0x55efe4885500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57970/0 (socket says 172.21.15.50:57970) 2024-05-08T18:05:04.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.049+0000 7f3ef99ca640 1 -- 172.21.15.50:0/2182668699 learned_addr learned my addr 172.21.15.50:0/2182668699 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.050+0000 7f3ef99ca640 1 -- 172.21.15.50:0/2182668699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe4885ad0 con 0x55efe48850e0 2024-05-08T18:05:04.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.050+0000 7f3ef99ca640 1 --2- 172.21.15.50:0/2182668699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe48850e0 0x55efe4885500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3eec056f50 tx=0x7f3eec078f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3863961e429bbd75 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.050+0000 7f3ddd7fa640 1 -- 172.21.15.50:0/2182668699 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eec078440 con 0x55efe48850e0 2024-05-08T18:05:04.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.050+0000 7f3ddd7fa640 1 -- 172.21.15.50:0/2182668699 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eec053930 con 0x55efe48850e0 2024-05-08T18:05:04.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.050+0000 7f3ddd7fa640 1 -- 172.21.15.50:0/2182668699 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eec076770 con 0x55efe48850e0 2024-05-08T18:05:04.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.050+0000 7f3efdd33800 1 -- 172.21.15.50:0/2182668699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe48850e0 msgr2=0x55efe4885500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.050+0000 7f3efdd33800 1 --2- 172.21.15.50:0/2182668699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe48850e0 0x55efe4885500 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3eec056f50 tx=0x7f3eec078f90 comp rx=0 tx=0).stop 2024-05-08T18:05:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.050+0000 7f3efdd33800 1 -- 172.21.15.50:0/2182668699 shutdown_connections 2024-05-08T18:05:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.050+0000 7f3efdd33800 1 --2- 172.21.15.50:0/2182668699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe48850e0 0x55efe4885500 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.050+0000 7f3efdd33800 1 -- 172.21.15.50:0/2182668699 >> 172.21.15.50:0/2182668699 conn(0x55efe48813f0 msgr2=0x55efe487f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.050+0000 7f3efdd33800 1 -- 172.21.15.50:0/2182668699 shutdown_connections 2024-05-08T18:05:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.050+0000 7f3efdd33800 1 -- 172.21.15.50:0/2182668699 wait complete. 2024-05-08T18:05:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.051+0000 7f3efdd33800 1 Processor -- start 2024-05-08T18:05:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.051+0000 7f3efdd33800 1 -- start start 2024-05-08T18:05:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.051+0000 7f3efdd33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe48850e0 0x55efe488c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.051+0000 7f3efdd33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55efe4886160 con 0x55efe48850e0 2024-05-08T18:05:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.051+0000 7f3ef99ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe48850e0 0x55efe488c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.051+0000 7f3ef99ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe48850e0 0x55efe488c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57986/0 (socket says 172.21.15.50:57986) 2024-05-08T18:05:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.051+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 learned_addr learned my addr 172.21.15.50:0/566230765 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.051+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efe488ca30 con 0x55efe48850e0 2024-05-08T18:05:04.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.052+0000 7f3ef99ca640 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe48850e0 0x55efe488c4f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3eec078fd0 tx=0x7f3eec088fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.052+0000 7f3dcb7fe640 1 -- 172.21.15.50:0/566230765 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eec078440 con 0x55efe48850e0 2024-05-08T18:05:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.052+0000 7f3dcb7fe640 1 -- 172.21.15.50:0/566230765 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eec0b1070 con 0x55efe48850e0 2024-05-08T18:05:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.052+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55efe488ca30 con 0x55efe48850e0 2024-05-08T18:05:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.052+0000 7f3dcb7fe640 1 -- 172.21.15.50:0/566230765 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eec054080 con 0x55efe48850e0 2024-05-08T18:05:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.052+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efe488ca30 con 0x55efe48850e0 2024-05-08T18:05:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.052+0000 7f3dcb7fe640 1 -- 172.21.15.50:0/566230765 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3eec054220 con 0x55efe48850e0 2024-05-08T18:05:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.053+0000 7f3dcb7fe640 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dc403bba0 0x7f3dc403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.053+0000 7f3dcb7fe640 1 -- 172.21.15.50:0/566230765 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f3eec0bd9b0 con 0x55efe48850e0 2024-05-08T18:05:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.053+0000 7f3ef91c9640 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dc403bba0 0x7f3dc403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.053+0000 7f3ef91c9640 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dc403bba0 0x7f3dc403e060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55efe4869fc0 tx=0x7f3ef0023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.053+0000 7f3efdd33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.053+0000 7f3efdd33800 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55efe4897af0 0x55efe4899fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.053+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe489a700 con 0x55efe4897af0 2024-05-08T18:05:04.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.054+0000 7f3efa1cb640 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55efe4897af0 0x55efe4899fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.054+0000 7f3efa1cb640 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55efe4897af0 0x55efe4899fb0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.054+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe4897af0 2024-05-08T18:05:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.054+0000 7f3efdd33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.054+0000 7f3efdd33800 20 realm 2024-05-08T18:05:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.054+0000 7f3efdd33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.054+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe489ab10 con 0x55efe4897af0 2024-05-08T18:05:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.054+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe4897af0 2024-05-08T18:05:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.054+0000 7f3efdd33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.054+0000 7f3efdd33800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.054+0000 7f3efdd33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.054+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe489b3e0 con 0x55efe4897af0 2024-05-08T18:05:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.055+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe4897af0 2024-05-08T18:05:04.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.055+0000 7f3efdd33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.055+0000 7f3efdd33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.055+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe489b780 con 0x55efe4897af0 2024-05-08T18:05:04.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.055+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe4897af0 2024-05-08T18:05:04.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.055+0000 7f3efdd33800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:04.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.055+0000 7f3efdd33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.055+0000 7f3efdd33800 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55efe489c4e0 0x55efe489e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.055+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe489eee0 con 0x55efe489c4e0 2024-05-08T18:05:04.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.055+0000 7f3ef99ca640 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55efe489c4e0 0x55efe489e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.056+0000 7f3ef99ca640 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55efe489c4e0 0x55efe489e8d0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.056+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.056+0000 7f3efdd33800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:04.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.056+0000 7f3efdd33800 20 searching for the correct realm 2024-05-08T18:05:04.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.056+0000 7f3efdd33800 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55efe48a0e90 0x55efe48a3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.075 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.056+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a3a20 con 0x55efe48a0e90 2024-05-08T18:05:04.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.056+0000 7f3ef91c9640 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55efe48a0e90 0x55efe48a3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.056+0000 7f3ef91c9640 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55efe48a0e90 0x55efe48a3350 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.057+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef0037040 con 0x55efe48a0e90 2024-05-08T18:05:04.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.057+0000 7f3ef91c9640 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3ef002ea80 0x7f3ef003a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.057+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a3a20 con 0x7f3ef002ea80 2024-05-08T18:05:04.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.057+0000 7f3efa1cb640 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3ef002ea80 0x7f3ef003a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.057+0000 7f3efa1cb640 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3ef002ea80 0x7f3ef003a3c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.057+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef400a4a0 con 0x7f3ef002ea80 2024-05-08T18:05:04.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.057+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55efe489b780 con 0x55efe489c4e0 2024-05-08T18:05:04.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.058+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.058+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55efe489b780 con 0x55efe4897af0 2024-05-08T18:05:04.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.058+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe4897af0 2024-05-08T18:05:04.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.058+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55efe489b780 con 0x55efe4897af0 2024-05-08T18:05:04.076 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.058+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe4897af0 2024-05-08T18:05:04.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.058+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55efe489b780 con 0x55efe489c4e0 2024-05-08T18:05:04.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.058+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.058+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55efe489b780 con 0x55efe489c4e0 2024-05-08T18:05:04.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.059+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.059+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x7f3ef002ea80 2024-05-08T18:05:04.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.059+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef400a4a0 con 0x7f3ef002ea80 2024-05-08T18:05:04.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.059+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe48a0e90 2024-05-08T18:05:04.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.059+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef0037040 con 0x55efe48a0e90 2024-05-08T18:05:04.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.059+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe4897af0 2024-05-08T18:05:04.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.059+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe4897af0 2024-05-08T18:05:04.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.059+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe489c4e0 2024-05-08T18:05:04.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.060+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.060+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe4897af0 2024-05-08T18:05:04.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.060+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe4897af0 2024-05-08T18:05:04.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.060+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x7f3ef002ea80 2024-05-08T18:05:04.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.060+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f3ef400a4a0 con 0x7f3ef002ea80 2024-05-08T18:05:04.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.060+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe48a0e90 2024-05-08T18:05:04.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.060+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef0037040 con 0x55efe48a0e90 2024-05-08T18:05:04.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.060+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe48a0e90 2024-05-08T18:05:04.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.061+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef0037040 con 0x55efe48a0e90 2024-05-08T18:05:04.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.061+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe489c4e0 2024-05-08T18:05:04.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.061+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.061+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe48a0e90 2024-05-08T18:05:04.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.061+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef0037040 con 0x55efe48a0e90 2024-05-08T18:05:04.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.061+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe48a0e90 2024-05-08T18:05:04.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.061+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f3ef0037040 con 0x55efe48a0e90 2024-05-08T18:05:04.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.061+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe4897af0 2024-05-08T18:05:04.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.062+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe4897af0 2024-05-08T18:05:04.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.062+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe4897af0 2024-05-08T18:05:04.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.062+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe4897af0 2024-05-08T18:05:04.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.062+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe489c4e0 2024-05-08T18:05:04.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.063+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.063+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe48a0e90 2024-05-08T18:05:04.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.063+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef0037040 con 0x55efe48a0e90 2024-05-08T18:05:04.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.063+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x7f3ef002ea80 2024-05-08T18:05:04.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.063+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef400a4a0 con 0x7f3ef002ea80 2024-05-08T18:05:04.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.063+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe4897af0 2024-05-08T18:05:04.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.063+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe4897af0 2024-05-08T18:05:04.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.063+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe489c4e0 2024-05-08T18:05:04.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.064+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.064+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe48a0e90 2024-05-08T18:05:04.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.064+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef0037040 con 0x55efe48a0e90 2024-05-08T18:05:04.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.064+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe489c4e0 2024-05-08T18:05:04.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.064+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.064+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x7f3ef002ea80 2024-05-08T18:05:04.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.064+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef400a4a0 con 0x7f3ef002ea80 2024-05-08T18:05:04.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.064+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe48a0e90 2024-05-08T18:05:04.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.065+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef0037040 con 0x55efe48a0e90 2024-05-08T18:05:04.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.065+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe489c4e0 2024-05-08T18:05:04.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.065+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.065+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x55efe4897af0 2024-05-08T18:05:04.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.065+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe4897af0 2024-05-08T18:05:04.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.065+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3eec0ca080 con 0x7f3ef002ea80 2024-05-08T18:05:04.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.065+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef400a4a0 con 0x7f3ef002ea80 2024-05-08T18:05:04.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.065+0000 7f3efdd33800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:04.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.065+0000 7f3efdd33800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:04.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.065+0000 7f3efdd33800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:04.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.065+0000 7f3efdd33800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:04.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.065+0000 7f3efdd33800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:04.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.065+0000 7f3efdd33800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:04.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.065+0000 7f3efdd33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.065+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe489f7a0 con 0x55efe4897af0 2024-05-08T18:05:04.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe4897af0 2024-05-08T18:05:04.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efdd33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efdd33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe489f7a0 con 0x55efe4897af0 2024-05-08T18:05:04.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3ef400a4a0 con 0x55efe4897af0 2024-05-08T18:05:04.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efdd33800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:04.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efdd33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe489f7a0 con 0x55efe48a0e90 2024-05-08T18:05:04.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f3ef0037040 con 0x55efe48a0e90 2024-05-08T18:05:04.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efdd33800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:04.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efdd33800 20 zone default found 2024-05-08T18:05:04.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efdd33800 4 Realm: () 2024-05-08T18:05:04.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efdd33800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:04.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efdd33800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:04.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efdd33800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:04.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efdd33800 20 zonegroup default 2024-05-08T18:05:04.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efdd33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.066+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a0100 con 0x55efe489c4e0 2024-05-08T18:05:04.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.067+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.067+0000 7f3efdd33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.067+0000 7f3efdd33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.067+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a5910 con 0x55efe48a0e90 2024-05-08T18:05:04.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.067+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3ef0037040 con 0x55efe48a0e90 2024-05-08T18:05:04.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.067+0000 7f3efdd33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.067+0000 7f3efdd33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.067+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a5e10 con 0x55efe48a0e90 2024-05-08T18:05:04.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.067+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3ef0037040 con 0x55efe48a0e90 2024-05-08T18:05:04.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.067+0000 7f3efdd33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.067+0000 7f3efdd33800 20 started sync module instance, tier type = 2024-05-08T18:05:04.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.067+0000 7f3efdd33800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:04.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.067+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a4d20 con 0x7f3ef002ea80 2024-05-08T18:05:04.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.068+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef400a4a0 con 0x7f3ef002ea80 2024-05-08T18:05:04.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.068+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94488819814976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a4d20 con 0x7f3ef002ea80 2024-05-08T18:05:04.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.068+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a68f0 con 0x55efe4897af0 2024-05-08T18:05:04.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.069+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94488819814976] v27'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef4040040 con 0x7f3ef002ea80 2024-05-08T18:05:04.101 DEBUG:teuthology.orchestra.run:got remote process result: 240 2024-05-08T18:05:04.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.070+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef4040040 con 0x55efe4897af0 2024-05-08T18:05:04.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.070+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94488819819488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a4d20 con 0x55efe4897af0 2024-05-08T18:05:04.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.070+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a7380 con 0x55efe48a0e90 2024-05-08T18:05:04.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.071+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94488819819488] v27'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef4040040 con 0x55efe4897af0 2024-05-08T18:05:04.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.071+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef0037040 con 0x55efe48a0e90 2024-05-08T18:05:04.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.071+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94488819812464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a4c90 con 0x55efe48a0e90 2024-05-08T18:05:04.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.071+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a8240 con 0x7f3ef002ea80 2024-05-08T18:05:04.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.072+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94488819812464] v27'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef0037040 con 0x55efe48a0e90 2024-05-08T18:05:04.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.072+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef4040040 con 0x7f3ef002ea80 2024-05-08T18:05:04.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.072+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94488819826064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a8660 con 0x7f3ef002ea80 2024-05-08T18:05:04.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.072+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a9020 con 0x55efe4897af0 2024-05-08T18:05:04.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.073+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94488819826064] v27'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef4040040 con 0x7f3ef002ea80 2024-05-08T18:05:04.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.073+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef4040040 con 0x55efe4897af0 2024-05-08T18:05:04.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.074+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94488819827600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a9020 con 0x55efe4897af0 2024-05-08T18:05:04.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.074+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a99e0 con 0x55efe489c4e0 2024-05-08T18:05:04.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.074+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94488819827600] v27'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef4040040 con 0x55efe4897af0 2024-05-08T18:05:04.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.075+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.075+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94488819829616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a8f90 con 0x55efe489c4e0 2024-05-08T18:05:04.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.075+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48aa760 con 0x55efe489c4e0 2024-05-08T18:05:04.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.076+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.076+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94488819835440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48aaf90 con 0x55efe489c4e0 2024-05-08T18:05:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.076+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48abab0 con 0x55efe489c4e0 2024-05-08T18:05:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.076+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94488819829616] v27'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.077+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94488819835440] v27'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.077+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.077+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94488819840384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48aa9e0 con 0x55efe489c4e0 2024-05-08T18:05:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.077+0000 7f3efdd33800 20 add_watcher() i=0 2024-05-08T18:05:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.077+0000 7f3efdd33800 20 add_watcher() i=1 2024-05-08T18:05:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.077+0000 7f3efdd33800 20 add_watcher() i=2 2024-05-08T18:05:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.077+0000 7f3efdd33800 20 add_watcher() i=3 2024-05-08T18:05:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.077+0000 7f3efdd33800 20 add_watcher() i=4 2024-05-08T18:05:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.077+0000 7f3efdd33800 20 add_watcher() i=5 2024-05-08T18:05:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.077+0000 7f3efdd33800 20 add_watcher() i=6 2024-05-08T18:05:04.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.078+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94488819840384] v27'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.078+0000 7f3efdd33800 20 add_watcher() i=7 2024-05-08T18:05:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.078+0000 7f3efdd33800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.078+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe48aeaa0 con 0x55efe4897af0 2024-05-08T18:05:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.078+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f3ef4040040 con 0x55efe4897af0 2024-05-08T18:05:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.078+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94488819834240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48abf50 con 0x55efe4897af0 2024-05-08T18:05:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.079+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94488819834240] v27'52 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3ef4040040 con 0x55efe4897af0 2024-05-08T18:05:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.079+0000 7f3dabfff640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.079+0000 7f3efdd33800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.079+0000 7f3efdd33800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.079+0000 7f3efdd33800 5 note: GC not initialized 2024-05-08T18:05:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.080+0000 7f3daaffd640 20 reqs_thread_entry: start 2024-05-08T18:05:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.080+0000 7f3efdd33800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.080+0000 7f3efdd33800 20 Filter name: none 2024-05-08T18:05:04.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.080+0000 7f3da8ff9640 20 reqs_thread_entry: start 2024-05-08T18:05:04.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.080+0000 7f3efdd33800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-05-08T18:05:04.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.080+0000 7f3efdd33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.080+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe48e1f50 con 0x55efe489c4e0 2024-05-08T18:05:04.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.080+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.080+0000 7f3efdd33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.080+0000 7f3efdd33800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-05-08T18:05:04.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.080+0000 7f3efdd33800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-05-08T18:05:04.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.080+0000 7f3efdd33800 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2024-05-08T18:05:04.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.080+0000 7f3efdd33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.080+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.3 8:c162d5d1:users.email::foo@foo.com:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe48e06f0 con 0x55efe4897af0 2024-05-08T18:05:04.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.081+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(63 foo@foo.com [stat out=16b,read 0~7 out=7b] v0'0 uv1 ondisk = 0) ==== 197+0+23 (crc 0 0 0) 0x7f3ef4040040 con 0x55efe4897af0 2024-05-08T18:05:04.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.081+0000 7f3efdd33800 20 rados_obj.operate() r=0 bl.length=7 2024-05-08T18:05:04.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.081+0000 7f3efdd33800 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x5 2024-05-08T18:05:04.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.081+0000 7f3efdd33800 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-05-08T18:05:04.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.081+0000 7f3efdd33800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:04.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.081+0000 7f3efdd33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.081+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55efe48e0ae0 con 0x55efe4897af0 2024-05-08T18:05:04.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.081+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(64 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f3ef4040040 con 0x55efe4897af0 2024-05-08T18:05:04.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.081+0000 7f3efdd33800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:04.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.081+0000 7f3efdd33800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:04.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.081+0000 7f3efdd33800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:04.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.081+0000 7f3efdd33800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:04.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.081+0000 7f3efdd33800 10 chain_cache_entry: cache_locator=default.rgw.meta+users.uid+foo 2024-05-08T18:05:04.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.081+0000 7f3efdd33800 20 get_user_info_from_index found 0 xattrs 2024-05-08T18:05:04.107 INFO:teuthology.orchestra.run.smithi050.stderr:could not create user: unable to parse parameters, email: foo@foo.com is the email address of an existing user 2024-05-08T18:05:04.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.082+0000 7f3dabfff640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:04.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.082+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94488819834240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe4891750 con 0x55efe4897af0 2024-05-08T18:05:04.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.083+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94488819834240] v27'53 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3ef4040040 con 0x55efe4897af0 2024-05-08T18:05:04.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.083+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94488819814976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a9d80 con 0x7f3ef002ea80 2024-05-08T18:05:04.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.084+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94488819814976] v27'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef4040040 con 0x7f3ef002ea80 2024-05-08T18:05:04.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.084+0000 7f3efdd33800 20 remove_watcher() i=0 2024-05-08T18:05:04.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.084+0000 7f3efdd33800 2 removed watcher, disabling cache 2024-05-08T18:05:04.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.084+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94488819819488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a9d80 con 0x55efe4897af0 2024-05-08T18:05:04.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.085+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94488819819488] v27'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef4040040 con 0x55efe4897af0 2024-05-08T18:05:04.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.085+0000 7f3efdd33800 20 remove_watcher() i=1 2024-05-08T18:05:04.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.085+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94488819812464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a63e0 con 0x55efe48a0e90 2024-05-08T18:05:04.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.086+0000 7f3ef91c9640 1 -- 172.21.15.50:0/566230765 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94488819812464] v27'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef0037040 con 0x55efe48a0e90 2024-05-08T18:05:04.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.086+0000 7f3efdd33800 20 remove_watcher() i=2 2024-05-08T18:05:04.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.086+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94488819826064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a9d80 con 0x7f3ef002ea80 2024-05-08T18:05:04.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.087+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94488819826064] v27'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef4040040 con 0x7f3ef002ea80 2024-05-08T18:05:04.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.087+0000 7f3efdd33800 20 remove_watcher() i=3 2024-05-08T18:05:04.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.087+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94488819827600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a4870 con 0x55efe4897af0 2024-05-08T18:05:04.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.088+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94488819827600] v27'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef4040040 con 0x55efe4897af0 2024-05-08T18:05:04.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.088+0000 7f3efdd33800 20 remove_watcher() i=4 2024-05-08T18:05:04.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.088+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94488819829616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a7d90 con 0x55efe489c4e0 2024-05-08T18:05:04.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.089+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94488819829616] v27'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.089+0000 7f3efdd33800 20 remove_watcher() i=5 2024-05-08T18:05:04.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.089+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94488819835440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a9d80 con 0x55efe489c4e0 2024-05-08T18:05:04.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.090+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94488819835440] v27'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.090+0000 7f3efdd33800 20 remove_watcher() i=6 2024-05-08T18:05:04.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.090+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94488819840384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55efe48a8390 con 0x55efe489c4e0 2024-05-08T18:05:04.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3ef99ca640 1 -- 172.21.15.50:0/566230765 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94488819840384] v27'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eec075030 con 0x55efe489c4e0 2024-05-08T18:05:04.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 20 remove_watcher() i=7 2024-05-08T18:05:04.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55efe489c4e0 msgr2=0x55efe489e8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55efe489c4e0 0x55efe489e8d0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55efe4897af0 msgr2=0x55efe4899fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55efe4897af0 0x55efe4899fb0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3ef002ea80 msgr2=0x7f3ef003a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3ef002ea80 0x7f3ef003a3c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55efe48a0e90 msgr2=0x55efe48a3350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55efe48a0e90 0x55efe48a3350 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dc403bba0 msgr2=0x7f3dc403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dc403bba0 0x7f3dc403e060 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x55efe4869fc0 tx=0x7f3ef0023040 comp rx=0 tx=0).stop 2024-05-08T18:05:04.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe48850e0 msgr2=0x55efe488c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 1 --2- 172.21.15.50:0/566230765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe48850e0 0x55efe488c4f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3eec078fd0 tx=0x7f3eec088fc0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 reap_dead start 2024-05-08T18:05:04.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efa1cb640 1 -- 172.21.15.50:0/566230765 reap_dead start 2024-05-08T18:05:04.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 shutdown_connections 2024-05-08T18:05:04.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 >> 172.21.15.50:0/566230765 conn(0x55efe48813f0 msgr2=0x55efe487ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 shutdown_connections 2024-05-08T18:05:04.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.091+0000 7f3efdd33800 1 -- 172.21.15.50:0/566230765 wait complete. 2024-05-08T18:05:04.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 -- 172.21.15.50:0/1944046135 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dd403bba0 msgr2=0x7f3dd403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 --2- 172.21.15.50:0/1944046135 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dd403bba0 0x7f3dd403e060 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55efe4861740 tx=0x7f3ef0006000 comp rx=0 tx=0).stop 2024-05-08T18:05:04.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 -- 172.21.15.50:0/1944046135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe4877f30 msgr2=0x55efe4878350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 --2- 172.21.15.50:0/1944046135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe4877f30 0x55efe4878350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3eec05c720 tx=0x7f3eec053f30 comp rx=0 tx=0).stop 2024-05-08T18:05:04.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 -- 172.21.15.50:0/1944046135 shutdown_connections 2024-05-08T18:05:04.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 --2- 172.21.15.50:0/1944046135 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dd403bba0 0x7f3dd403e060 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 --2- 172.21.15.50:0/1944046135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe4877f30 0x55efe4878350 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 -- 172.21.15.50:0/1944046135 >> 172.21.15.50:0/1944046135 conn(0x55efe486bfd0 msgr2=0x55efe486c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 -- 172.21.15.50:0/1944046135 shutdown_connections 2024-05-08T18:05:04.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 -- 172.21.15.50:0/1944046135 wait complete. 2024-05-08T18:05:04.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 -- 172.21.15.50:0/1600366031 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55efe4863190 msgr2=0x55efe4865580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55efe4863190 0x55efe4865580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 -- 172.21.15.50:0/1600366031 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55efe475f8d0 msgr2=0x55efe4860930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55efe475f8d0 0x55efe4860930 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 -- 172.21.15.50:0/1600366031 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55efe48670d0 msgr2=0x55efe48694c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55efe48670d0 0x55efe48694c0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 -- 172.21.15.50:0/1600366031 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3de003bba0 msgr2=0x7f3de003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3de003bba0 0x7f3de003e060 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0000c00 tx=0x7f3ef00013b0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 -- 172.21.15.50:0/1600366031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 msgr2=0x55efe48596f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 --2- 172.21.15.50:0/1600366031 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55efe475e710 0x55efe48596f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0524f0 tx=0x7f3eec003020 comp rx=0 tx=0).stop 2024-05-08T18:05:04.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efa1cb640 1 -- 172.21.15.50:0/1600366031 reap_dead start 2024-05-08T18:05:04.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 -- 172.21.15.50:0/1600366031 shutdown_connections 2024-05-08T18:05:04.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 -- 172.21.15.50:0/1600366031 >> 172.21.15.50:0/1600366031 conn(0x55efe4759c90 msgr2=0x55efe46d2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 -- 172.21.15.50:0/1600366031 shutdown_connections 2024-05-08T18:05:04.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.092+0000 7f3efdd33800 1 -- 172.21.15.50:0/1600366031 wait complete. 2024-05-08T18:05:04.113 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-05-08T18:05:04.113 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-05-08T18:05:04.113 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-05-08T18:05:04.159 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:04.159 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:04.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.161+0000 7f9cf6213800 1 Processor -- start 2024-05-08T18:05:04.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.161+0000 7f9cf6213800 1 -- start start 2024-05-08T18:05:04.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.162+0000 7f9cf6213800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56162ba349e0 0x56162ba36dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.162+0000 7f9cf6213800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56162ba373a0 con 0x56162ba30550 2024-05-08T18:05:04.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.162+0000 7f9cf6213800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x56162b8d1e20 con 0x56162ba349e0 2024-05-08T18:05:04.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.162+0000 7f9cf0fd4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56162ba349e0 0x56162ba36dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.162+0000 7f9cf17d5640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x56162ba30550 0x56162ba30970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:53074/0 (socket says 172.21.15.50:53074) 2024-05-08T18:05:04.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.162+0000 7f9cf17d5640 1 -- 172.21.15.50:0/3064885749 learned_addr learned my addr 172.21.15.50:0/3064885749 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:04.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.162+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/3064885749 >> v1:172.21.15.46:6789/0 conn(0x56162ba30550 legacy=0x56162ba30970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.162+0000 7f9ce3fff640 1 -- 172.21.15.50:0/3064885749 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3326617260 0 0) 0x56162ba373a0 con 0x56162ba30550 2024-05-08T18:05:04.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.162+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/3064885749 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162ba385d0 con 0x56162ba349e0 2024-05-08T18:05:04.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.164+0000 7f9cf0fd4640 1 --2- 172.21.15.50:0/3064885749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba349e0 0x56162ba36dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800a470 tx=0x7f9ce801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61a88611778cd34 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.164+0000 7f9ce3fff640 1 -- 172.21.15.50:0/3064885749 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ce8057070 con 0x56162ba349e0 2024-05-08T18:05:04.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.164+0000 7f9ce3fff640 1 -- 172.21.15.50:0/3064885749 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce8052570 con 0x56162ba349e0 2024-05-08T18:05:04.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.164+0000 7f9ce3fff640 1 -- 172.21.15.50:0/3064885749 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ce8052870 con 0x56162ba349e0 2024-05-08T18:05:04.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.165+0000 7f9cf6213800 1 -- 172.21.15.50:0/3064885749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba349e0 msgr2=0x56162ba36dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.165+0000 7f9cf6213800 1 --2- 172.21.15.50:0/3064885749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba349e0 0x56162ba36dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9ce800a470 tx=0x7f9ce801ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:04.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.165+0000 7f9cf6213800 1 -- 172.21.15.50:0/3064885749 shutdown_connections 2024-05-08T18:05:04.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.165+0000 7f9cf6213800 1 --2- 172.21.15.50:0/3064885749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba349e0 0x56162ba36dd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.165+0000 7f9cf6213800 1 -- 172.21.15.50:0/3064885749 >> 172.21.15.50:0/3064885749 conn(0x56162ba2bb50 msgr2=0x56162ba2df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.165+0000 7f9cf6213800 1 -- 172.21.15.50:0/3064885749 shutdown_connections 2024-05-08T18:05:04.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.165+0000 7f9cf6213800 1 -- 172.21.15.50:0/3064885749 wait complete. 2024-05-08T18:05:04.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.180+0000 7f9cf6213800 1 Processor -- start 2024-05-08T18:05:04.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.180+0000 7f9cf6213800 1 -- start start 2024-05-08T18:05:04.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.180+0000 7f9cf6213800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba30550 0x56162bb22980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.180+0000 7f9cf6213800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56162b97ca10 con 0x56162ba30550 2024-05-08T18:05:04.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.180+0000 7f9cf17d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba30550 0x56162bb22980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.180+0000 7f9cf17d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba30550 0x56162bb22980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58008/0 (socket says 172.21.15.50:58008) 2024-05-08T18:05:04.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.180+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1186382063 learned_addr learned my addr 172.21.15.50:0/1186382063 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.180+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1186382063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162bb22f50 con 0x56162ba30550 2024-05-08T18:05:04.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.181+0000 7f9cf17d5640 1 --2- 172.21.15.50:0/1186382063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba30550 0x56162bb22980 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4001ea0 tx=0x7f9ce400b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445403ed3e4cb631 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.181+0000 7f9bdf606640 1 -- 172.21.15.50:0/1186382063 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ce4008070 con 0x56162ba30550 2024-05-08T18:05:04.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.181+0000 7f9bdf606640 1 -- 172.21.15.50:0/1186382063 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce400d500 con 0x56162ba30550 2024-05-08T18:05:04.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.181+0000 7f9bdf606640 1 -- 172.21.15.50:0/1186382063 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ce400d880 con 0x56162ba30550 2024-05-08T18:05:04.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.181+0000 7f9cf6213800 1 -- 172.21.15.50:0/1186382063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba30550 msgr2=0x56162bb22980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.181+0000 7f9cf6213800 1 --2- 172.21.15.50:0/1186382063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba30550 0x56162bb22980 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4001ea0 tx=0x7f9ce400b6a0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.182+0000 7f9cf6213800 1 -- 172.21.15.50:0/1186382063 shutdown_connections 2024-05-08T18:05:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.182+0000 7f9cf6213800 1 --2- 172.21.15.50:0/1186382063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba30550 0x56162bb22980 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.182+0000 7f9cf6213800 1 -- 172.21.15.50:0/1186382063 >> 172.21.15.50:0/1186382063 conn(0x56162ba2bb50 msgr2=0x56162bb1e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.182+0000 7f9cf6213800 1 -- 172.21.15.50:0/1186382063 shutdown_connections 2024-05-08T18:05:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.182+0000 7f9cf6213800 1 -- 172.21.15.50:0/1186382063 wait complete. 2024-05-08T18:05:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.182+0000 7f9cf6213800 1 Processor -- start 2024-05-08T18:05:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.182+0000 7f9cf6213800 1 -- start start 2024-05-08T18:05:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.182+0000 7f9cf6213800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba30550 0x56162bb2a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.182+0000 7f9cf6213800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56162ba37870 con 0x56162ba30550 2024-05-08T18:05:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.182+0000 7f9cf17d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba30550 0x56162bb2a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.182+0000 7f9cf17d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba30550 0x56162bb2a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58020/0 (socket says 172.21.15.50:58020) 2024-05-08T18:05:04.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.182+0000 7f9cf17d5640 1 -- 172.21.15.50:0/818295857 learned_addr learned my addr 172.21.15.50:0/818295857 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.183+0000 7f9cf17d5640 1 -- 172.21.15.50:0/818295857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162bb2aa60 con 0x56162ba30550 2024-05-08T18:05:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.183+0000 7f9cf17d5640 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba30550 0x56162bb2a520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40012f0 tx=0x7f9ce4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.183+0000 7f9bdde03640 1 -- 172.21.15.50:0/818295857 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ce400f9d0 con 0x56162ba30550 2024-05-08T18:05:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.183+0000 7f9bdde03640 1 -- 172.21.15.50:0/818295857 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce400fb70 con 0x56162ba30550 2024-05-08T18:05:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.183+0000 7f9cf6213800 1 -- 172.21.15.50:0/818295857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56162bb2acf0 con 0x56162ba30550 2024-05-08T18:05:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.183+0000 7f9bdde03640 1 -- 172.21.15.50:0/818295857 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ce400fe50 con 0x56162ba30550 2024-05-08T18:05:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.183+0000 7f9cf6213800 1 -- 172.21.15.50:0/818295857 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162bb2d9a0 con 0x56162ba30550 2024-05-08T18:05:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.184+0000 7f9bdde03640 1 -- 172.21.15.50:0/818295857 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9ce4017040 con 0x56162ba30550 2024-05-08T18:05:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.184+0000 7f9bdde03640 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9bd803bd40 0x7f9bd803e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.184+0000 7f9bdde03640 1 -- 172.21.15.50:0/818295857 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f9ce4015020 con 0x56162ba30550 2024-05-08T18:05:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.184+0000 7f9cf0fd4640 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9bd803bd40 0x7f9bd803e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.185+0000 7f9cf6213800 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56162ba349e0 0x56162bb31760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.185+0000 7f9cf6213800 1 -- 172.21.15.50:0/818295857 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb31ce0 con 0x56162ba349e0 2024-05-08T18:05:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.185+0000 7f9cf1fd6640 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56162ba349e0 0x56162bb31760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.185+0000 7f9cf0fd4640 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9bd803bd40 0x7f9bd803e200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8054000 tx=0x7f9ce804e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.185+0000 7f9cf1fd6640 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56162ba349e0 0x56162bb31760 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.185+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/818295857 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162ba349e0 2024-05-08T18:05:04.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.185+0000 7f9cf6213800 1 -- 172.21.15.50:0/818295857 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb2ed30 con 0x56162ba349e0 2024-05-08T18:05:04.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.186+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/818295857 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162ba349e0 2024-05-08T18:05:04.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.186+0000 7f9cf6213800 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56162bb34030 0x56162bb36420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.186+0000 7f9cf6213800 1 -- 172.21.15.50:0/818295857 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb36ac0 con 0x56162bb34030 2024-05-08T18:05:04.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.186+0000 7f9cf17d5640 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56162bb34030 0x56162bb36420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.186+0000 7f9cf17d5640 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56162bb34030 0x56162bb36420 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.186+0000 7f9cf17d5640 1 -- 172.21.15.50:0/818295857 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f9ce402d070 con 0x56162bb34030 2024-05-08T18:05:04.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.186+0000 7f9cf6213800 1 -- 172.21.15.50:0/818295857 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb32ed0 con 0x56162ba349e0 2024-05-08T18:05:04.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.187+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/818295857 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162ba349e0 2024-05-08T18:05:04.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.187+0000 7f9cf6213800 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56162bb37f20 0x56162bb3a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.187+0000 7f9cf6213800 1 -- 172.21.15.50:0/818295857 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb3aa10 con 0x56162bb37f20 2024-05-08T18:05:04.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.187+0000 7f9cf0fd4640 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56162bb37f20 0x56162bb3a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.187+0000 7f9cf0fd4640 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56162bb37f20 0x56162bb3a310 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.187+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/818295857 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9ce8052940 con 0x56162bb37f20 2024-05-08T18:05:04.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.188+0000 7f9cf6213800 1 Processor -- start 2024-05-08T18:05:04.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.188+0000 7f9cf6213800 1 -- start start 2024-05-08T18:05:04.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.188+0000 7f9cf6213800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb48d80 0x56162bb491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.188+0000 7f9cf6213800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56162b923330 con 0x56162bb48d80 2024-05-08T18:05:04.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.188+0000 7f9cf17d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb48d80 0x56162bb491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.188+0000 7f9cf17d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb48d80 0x56162bb491a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58032/0 (socket says 172.21.15.50:58032) 2024-05-08T18:05:04.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.188+0000 7f9cf17d5640 1 -- 172.21.15.50:0/264768877 learned_addr learned my addr 172.21.15.50:0/264768877 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.189+0000 7f9cf17d5640 1 -- 172.21.15.50:0/264768877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162bb496e0 con 0x56162bb48d80 2024-05-08T18:05:04.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.189+0000 7f9cf17d5640 1 --2- 172.21.15.50:0/264768877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb48d80 0x56162bb491a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4051820 tx=0x7f9ce4051f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.189+0000 7f9bd6ffd640 1 -- 172.21.15.50:0/264768877 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ce401a3c0 con 0x56162bb48d80 2024-05-08T18:05:04.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.189+0000 7f9bd6ffd640 1 -- 172.21.15.50:0/264768877 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce4047070 con 0x56162bb48d80 2024-05-08T18:05:04.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.189+0000 7f9cf6213800 1 -- 172.21.15.50:0/264768877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56162bb4a150 con 0x56162bb48d80 2024-05-08T18:05:04.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.189+0000 7f9bd6ffd640 1 -- 172.21.15.50:0/264768877 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ce40183d0 con 0x56162bb48d80 2024-05-08T18:05:04.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.189+0000 7f9cf6213800 1 -- 172.21.15.50:0/264768877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162bb4a650 con 0x56162bb48d80 2024-05-08T18:05:04.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.190+0000 7f9bd6ffd640 1 -- 172.21.15.50:0/264768877 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9ce4018570 con 0x56162bb48d80 2024-05-08T18:05:04.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.190+0000 7f9bd6ffd640 1 --2- 172.21.15.50:0/264768877 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9bc803bba0 0x7f9bc803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.190+0000 7f9bd6ffd640 1 -- 172.21.15.50:0/264768877 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f9ce403d550 con 0x56162bb48d80 2024-05-08T18:05:04.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.190+0000 7f9cf0fd4640 1 --2- 172.21.15.50:0/264768877 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9bc803bba0 0x7f9bc803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.190+0000 7f9cf6213800 1 Processor -- start 2024-05-08T18:05:04.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.190+0000 7f9cf6213800 1 -- start start 2024-05-08T18:05:04.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.190+0000 7f9cf0fd4640 1 --2- 172.21.15.50:0/264768877 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9bc803bba0 0x7f9bc803e060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x56162bb2ec50 tx=0x7f9ce8005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.191+0000 7f9cf6213800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb56090 0x56162bb564b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.191+0000 7f9cf6213800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56162bb232d0 con 0x56162bb56090 2024-05-08T18:05:04.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.191+0000 7f9cf17d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb56090 0x56162bb564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.191+0000 7f9cf17d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb56090 0x56162bb564b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58036/0 (socket says 172.21.15.50:58036) 2024-05-08T18:05:04.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.191+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1792666766 learned_addr learned my addr 172.21.15.50:0/1792666766 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.191+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1792666766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162bb56a80 con 0x56162bb56090 2024-05-08T18:05:04.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.191+0000 7f9cf17d5640 1 --2- 172.21.15.50:0/1792666766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb56090 0x56162bb564b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9ce401b750 tx=0x7f9ce402f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b9f66804d8cc0d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.191+0000 7f9bd4ff9640 1 -- 172.21.15.50:0/1792666766 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ce4057070 con 0x56162bb56090 2024-05-08T18:05:04.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.191+0000 7f9bd4ff9640 1 -- 172.21.15.50:0/1792666766 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce4026e10 con 0x56162bb56090 2024-05-08T18:05:04.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.191+0000 7f9bd4ff9640 1 -- 172.21.15.50:0/1792666766 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ce4035070 con 0x56162bb56090 2024-05-08T18:05:04.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.192+0000 7f9cf6213800 1 -- 172.21.15.50:0/1792666766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb56090 msgr2=0x56162bb564b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.192+0000 7f9cf6213800 1 --2- 172.21.15.50:0/1792666766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb56090 0x56162bb564b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9ce401b750 tx=0x7f9ce402f740 comp rx=0 tx=0).stop 2024-05-08T18:05:04.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.192+0000 7f9cf6213800 1 -- 172.21.15.50:0/1792666766 shutdown_connections 2024-05-08T18:05:04.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.192+0000 7f9cf6213800 1 --2- 172.21.15.50:0/1792666766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb56090 0x56162bb564b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.192+0000 7f9cf6213800 1 -- 172.21.15.50:0/1792666766 >> 172.21.15.50:0/1792666766 conn(0x56162bb52360 msgr2=0x56162bb503c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.192+0000 7f9cf6213800 1 -- 172.21.15.50:0/1792666766 shutdown_connections 2024-05-08T18:05:04.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.192+0000 7f9cf6213800 1 -- 172.21.15.50:0/1792666766 wait complete. 2024-05-08T18:05:04.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.192+0000 7f9cf6213800 1 Processor -- start 2024-05-08T18:05:04.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.192+0000 7f9cf6213800 1 -- start start 2024-05-08T18:05:04.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.192+0000 7f9cf6213800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb56090 0x56162bb5d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.192+0000 7f9cf6213800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56162bb570e0 con 0x56162bb56090 2024-05-08T18:05:04.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.192+0000 7f9cf17d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb56090 0x56162bb5d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.192+0000 7f9cf17d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb56090 0x56162bb5d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58040/0 (socket says 172.21.15.50:58040) 2024-05-08T18:05:04.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.192+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 learned_addr learned my addr 172.21.15.50:0/1671013527 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.193+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162bb5d8b0 con 0x56162bb56090 2024-05-08T18:05:04.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.193+0000 7f9cf17d5640 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb56090 0x56162bb5d370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400efd0 tx=0x7f9ce403b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.193+0000 7f9bc6ffd640 1 -- 172.21.15.50:0/1671013527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ce4019610 con 0x56162bb56090 2024-05-08T18:05:04.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.193+0000 7f9bc6ffd640 1 -- 172.21.15.50:0/1671013527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ce4066030 con 0x56162bb56090 2024-05-08T18:05:04.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.193+0000 7f9bc6ffd640 1 -- 172.21.15.50:0/1671013527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ce4003cd0 con 0x56162bb56090 2024-05-08T18:05:04.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.193+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56162bb5d8b0 con 0x56162bb56090 2024-05-08T18:05:04.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.193+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162bb5d8b0 con 0x56162bb56090 2024-05-08T18:05:04.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.194+0000 7f9bc6ffd640 1 -- 172.21.15.50:0/1671013527 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9ce405c070 con 0x56162bb56090 2024-05-08T18:05:04.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.194+0000 7f9bc6ffd640 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9bbc03bba0 0x7f9bbc03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.194+0000 7f9bc6ffd640 1 -- 172.21.15.50:0/1671013527 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f9ce405a020 con 0x56162bb56090 2024-05-08T18:05:04.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.194+0000 7f9cf0fd4640 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9bbc03bba0 0x7f9bbc03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.195+0000 7f9cf0fd4640 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9bbc03bba0 0x7f9bbc03e060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8052fd0 tx=0x7f9ce804f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.195+0000 7f9cf6213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.195+0000 7f9cf6213800 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56162bb68b20 0x56162bb6afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.195+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6b730 con 0x56162bb68b20 2024-05-08T18:05:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.195+0000 7f9cf1fd6640 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56162bb68b20 0x56162bb6afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.195+0000 7f9cf1fd6640 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56162bb68b20 0x56162bb6afe0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162bb68b20 2024-05-08T18:05:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf6213800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf6213800 20 realm 2024-05-08T18:05:04.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf6213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6bb40 con 0x56162bb68b20 2024-05-08T18:05:04.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162bb68b20 2024-05-08T18:05:04.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf6213800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf6213800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:04.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf6213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6c410 con 0x56162bb68b20 2024-05-08T18:05:04.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162bb68b20 2024-05-08T18:05:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf6213800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf6213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6c7b0 con 0x56162bb68b20 2024-05-08T18:05:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162bb68b20 2024-05-08T18:05:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf6213800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf6213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf6213800 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56162bb6d2a0 0x56162bb6f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.196+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6fdf0 con 0x56162bb6d2a0 2024-05-08T18:05:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.197+0000 7f9cf17d5640 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56162bb6d2a0 0x56162bb6f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.197+0000 7f9cf17d5640 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56162bb6d2a0 0x56162bb6f690 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.197+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.197+0000 7f9cf6213800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.197+0000 7f9cf6213800 20 searching for the correct realm 2024-05-08T18:05:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.197+0000 7f9cf6213800 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56162bb71ea0 0x56162bb74360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.197+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56162bb74a30 con 0x56162bb71ea0 2024-05-08T18:05:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.197+0000 7f9cf0fd4640 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56162bb71ea0 0x56162bb74360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.198+0000 7f9cf0fd4640 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56162bb71ea0 0x56162bb74360 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.198+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9ce8081070 con 0x56162bb71ea0 2024-05-08T18:05:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.198+0000 7f9cf0fd4640 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9ce8076040 0x7f9ce8083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.198+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56162bb74a30 con 0x7f9ce8076040 2024-05-08T18:05:04.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.198+0000 7f9cf1fd6640 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9ce8076040 0x7f9ce8083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.198+0000 7f9cf1fd6640 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9ce8076040 0x7f9ce8083410 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.198+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9cec00a4a0 con 0x7f9ce8076040 2024-05-08T18:05:04.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.198+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6c7b0 con 0x56162bb6d2a0 2024-05-08T18:05:04.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.199+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.199+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6c7b0 con 0x56162bb68b20 2024-05-08T18:05:04.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.199+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162bb68b20 2024-05-08T18:05:04.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.199+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6c7b0 con 0x56162bb68b20 2024-05-08T18:05:04.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.199+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162bb68b20 2024-05-08T18:05:04.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.199+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6c7b0 con 0x56162bb6d2a0 2024-05-08T18:05:04.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.199+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.199+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6c7b0 con 0x56162bb6d2a0 2024-05-08T18:05:04.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.200+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.200+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6c7b0 con 0x7f9ce8076040 2024-05-08T18:05:04.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.200+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9cec00a4a0 con 0x7f9ce8076040 2024-05-08T18:05:04.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.200+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6c7b0 con 0x56162bb71ea0 2024-05-08T18:05:04.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.200+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9ce8081070 con 0x56162bb71ea0 2024-05-08T18:05:04.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.200+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6c7b0 con 0x56162bb68b20 2024-05-08T18:05:04.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.200+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162bb68b20 2024-05-08T18:05:04.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.200+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6c7b0 con 0x56162bb6d2a0 2024-05-08T18:05:04.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.201+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.201+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6c7b0 con 0x56162bb68b20 2024-05-08T18:05:04.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.201+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162bb68b20 2024-05-08T18:05:04.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.201+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6c7b0 con 0x7f9ce8076040 2024-05-08T18:05:04.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.201+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9cec00a4a0 con 0x7f9ce8076040 2024-05-08T18:05:04.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.201+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56162bb6c7b0 con 0x56162bb71ea0 2024-05-08T18:05:04.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.201+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9ce8081070 con 0x56162bb71ea0 2024-05-08T18:05:04.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.201+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x56162bb71ea0 2024-05-08T18:05:04.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.202+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9ce8081070 con 0x56162bb71ea0 2024-05-08T18:05:04.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.202+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x56162bb6d2a0 2024-05-08T18:05:04.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.202+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.202+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x56162bb71ea0 2024-05-08T18:05:04.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.202+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9ce8081070 con 0x56162bb71ea0 2024-05-08T18:05:04.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.202+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x56162bb71ea0 2024-05-08T18:05:04.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.202+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f9ce8081070 con 0x56162bb71ea0 2024-05-08T18:05:04.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.202+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x56162bb68b20 2024-05-08T18:05:04.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.203+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162bb68b20 2024-05-08T18:05:04.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.203+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x56162bb68b20 2024-05-08T18:05:04.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.203+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162bb68b20 2024-05-08T18:05:04.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.203+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x56162bb6d2a0 2024-05-08T18:05:04.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.203+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.203+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x56162bb71ea0 2024-05-08T18:05:04.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.203+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9ce8081070 con 0x56162bb71ea0 2024-05-08T18:05:04.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.203+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x7f9ce8076040 2024-05-08T18:05:04.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.204+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9cec00a4a0 con 0x7f9ce8076040 2024-05-08T18:05:04.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.204+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x56162bb68b20 2024-05-08T18:05:04.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.204+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162bb68b20 2024-05-08T18:05:04.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.204+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x56162bb6d2a0 2024-05-08T18:05:04.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.204+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.204+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x56162bb71ea0 2024-05-08T18:05:04.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.204+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9ce8081070 con 0x56162bb71ea0 2024-05-08T18:05:04.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.204+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x56162bb6d2a0 2024-05-08T18:05:04.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.205+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.205+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x7f9ce8076040 2024-05-08T18:05:04.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.205+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9cec00a4a0 con 0x7f9ce8076040 2024-05-08T18:05:04.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.205+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x56162bb71ea0 2024-05-08T18:05:04.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.205+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9ce8081070 con 0x56162bb71ea0 2024-05-08T18:05:04.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.205+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x56162bb6d2a0 2024-05-08T18:05:04.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.206+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.206+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x56162bb68b20 2024-05-08T18:05:04.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.206+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162bb68b20 2024-05-08T18:05:04.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.206+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9ce8090040 con 0x7f9ce8076040 2024-05-08T18:05:04.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.206+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9cec00a4a0 con 0x7f9ce8076040 2024-05-08T18:05:04.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.206+0000 7f9cf6213800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:04.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.206+0000 7f9cf6213800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:04.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.206+0000 7f9cf6213800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:04.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.206+0000 7f9cf6213800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:04.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.206+0000 7f9cf6213800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:04.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.206+0000 7f9cf6213800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:04.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.206+0000 7f9cf6213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.206+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb707a0 con 0x56162bb68b20 2024-05-08T18:05:04.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162bb68b20 2024-05-08T18:05:04.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf6213800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf6213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb707a0 con 0x56162bb68b20 2024-05-08T18:05:04.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9cec00a4a0 con 0x56162bb68b20 2024-05-08T18:05:04.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf6213800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:04.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf6213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb70770 con 0x56162bb71ea0 2024-05-08T18:05:04.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f9ce8081070 con 0x56162bb71ea0 2024-05-08T18:05:04.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf6213800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:04.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf6213800 20 zone default found 2024-05-08T18:05:04.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf6213800 4 Realm: () 2024-05-08T18:05:04.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf6213800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:04.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf6213800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:04.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf6213800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:04.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf6213800 20 zonegroup default 2024-05-08T18:05:04.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf6213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.207+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb71040 con 0x56162bb6d2a0 2024-05-08T18:05:04.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.208+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.208+0000 7f9cf6213800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.208+0000 7f9cf6213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.208+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb76890 con 0x56162bb71ea0 2024-05-08T18:05:04.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.208+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9ce8081070 con 0x56162bb71ea0 2024-05-08T18:05:04.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.208+0000 7f9cf6213800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.208+0000 7f9cf6213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.208+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb76c30 con 0x56162bb71ea0 2024-05-08T18:05:04.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.208+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9ce8081070 con 0x56162bb71ea0 2024-05-08T18:05:04.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.208+0000 7f9cf6213800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.208+0000 7f9cf6213800 20 started sync module instance, tier type = 2024-05-08T18:05:04.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.208+0000 7f9cf6213800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:04.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.208+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb75d30 con 0x7f9ce8076040 2024-05-08T18:05:04.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.209+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9cec00a4a0 con 0x7f9ce8076040 2024-05-08T18:05:04.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.209+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94653222707840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb75d30 con 0x7f9ce8076040 2024-05-08T18:05:04.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.209+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb77650 con 0x56162bb68b20 2024-05-08T18:05:04.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.210+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94653222707840] v27'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9cec040040 con 0x7f9ce8076040 2024-05-08T18:05:04.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.211+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9cec040040 con 0x56162bb68b20 2024-05-08T18:05:04.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.211+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94653222711616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb75d30 con 0x56162bb68b20 2024-05-08T18:05:04.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.211+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb780f0 con 0x56162bb71ea0 2024-05-08T18:05:04.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.212+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94653222711616] v27'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9cec040040 con 0x56162bb68b20 2024-05-08T18:05:04.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.212+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9ce8081070 con 0x56162bb71ea0 2024-05-08T18:05:04.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.212+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94653222705280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb75ca0 con 0x56162bb71ea0 2024-05-08T18:05:04.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.212+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb78fb0 con 0x7f9ce8076040 2024-05-08T18:05:04.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.213+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94653222705280] v27'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9ce8081070 con 0x56162bb71ea0 2024-05-08T18:05:04.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.213+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9cec040040 con 0x7f9ce8076040 2024-05-08T18:05:04.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.213+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94653222718208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb793d0 con 0x7f9ce8076040 2024-05-08T18:05:04.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.213+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb79ca0 con 0x56162bb68b20 2024-05-08T18:05:04.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.214+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9cec040040 con 0x56162bb68b20 2024-05-08T18:05:04.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.214+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94653222720464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb797f0 con 0x56162bb68b20 2024-05-08T18:05:04.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.214+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb7ac70 con 0x56162bb6d2a0 2024-05-08T18:05:04.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.215+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94653222718208] v27'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9cec040040 con 0x7f9ce8076040 2024-05-08T18:05:04.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.215+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94653222720464] v27'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9cec040040 con 0x56162bb68b20 2024-05-08T18:05:04.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.216+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.216+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94653222723728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb78f20 con 0x56162bb6d2a0 2024-05-08T18:05:04.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.216+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb7b510 con 0x56162bb6d2a0 2024-05-08T18:05:04.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.217+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.217+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94653222729904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb7c180 con 0x56162bb6d2a0 2024-05-08T18:05:04.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.217+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb7cb80 con 0x56162bb6d2a0 2024-05-08T18:05:04.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.217+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94653222723728] v27'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.218+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94653222729904] v27'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.218+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.218+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94653222730960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb7c6f0 con 0x56162bb6d2a0 2024-05-08T18:05:04.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.218+0000 7f9cf6213800 20 add_watcher() i=0 2024-05-08T18:05:04.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.218+0000 7f9cf6213800 20 add_watcher() i=1 2024-05-08T18:05:04.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.218+0000 7f9cf6213800 20 add_watcher() i=2 2024-05-08T18:05:04.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.218+0000 7f9cf6213800 20 add_watcher() i=3 2024-05-08T18:05:04.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.218+0000 7f9cf6213800 20 add_watcher() i=4 2024-05-08T18:05:04.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.218+0000 7f9cf6213800 20 add_watcher() i=5 2024-05-08T18:05:04.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.218+0000 7f9cf6213800 20 add_watcher() i=6 2024-05-08T18:05:04.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.219+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94653222730960] v27'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.219+0000 7f9cf6213800 20 add_watcher() i=7 2024-05-08T18:05:04.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.219+0000 7f9cf6213800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:04.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.219+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bb7fd30 con 0x56162bb68b20 2024-05-08T18:05:04.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.219+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9cec040040 con 0x56162bb68b20 2024-05-08T18:05:04.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.219+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94653222725472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb7d100 con 0x56162bb68b20 2024-05-08T18:05:04.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.220+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94653222725472] v27'54 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9cec040040 con 0x56162bb68b20 2024-05-08T18:05:04.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.220+0000 7f9ba77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:04.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.220+0000 7f9cf6213800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:04.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.220+0000 7f9cf6213800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:04.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.220+0000 7f9cf6213800 5 note: GC not initialized 2024-05-08T18:05:04.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.220+0000 7f9ba67e4640 20 reqs_thread_entry: start 2024-05-08T18:05:04.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.221+0000 7f9cf6213800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:04.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.221+0000 7f9cf6213800 20 Filter name: none 2024-05-08T18:05:04.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.221+0000 7f9b17fff640 20 reqs_thread_entry: start 2024-05-08T18:05:04.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.221+0000 7f9cf6213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.221+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56162bbb2de0 con 0x56162bb68b20 2024-05-08T18:05:04.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.221+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f9cec040040 con 0x56162bb68b20 2024-05-08T18:05:04.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.221+0000 7f9cf6213800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:04.235 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:04.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.222+0000 7f9ba77e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:04.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.222+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94653222725472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb655c0 con 0x56162bb68b20 2024-05-08T18:05:04.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.223+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94653222725472] v27'55 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9cec040040 con 0x56162bb68b20 2024-05-08T18:05:04.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.223+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94653222707840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb7a760 con 0x7f9ce8076040 2024-05-08T18:05:04.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.224+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94653222707840] v27'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9cec040040 con 0x7f9ce8076040 2024-05-08T18:05:04.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.224+0000 7f9cf6213800 20 remove_watcher() i=0 2024-05-08T18:05:04.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.224+0000 7f9cf6213800 2 removed watcher, disabling cache 2024-05-08T18:05:04.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.224+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94653222711616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb76280 con 0x56162bb68b20 2024-05-08T18:05:04.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.225+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94653222711616] v27'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9cec040040 con 0x56162bb68b20 2024-05-08T18:05:04.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.226+0000 7f9cf6213800 20 remove_watcher() i=1 2024-05-08T18:05:04.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.226+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94653222705280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb77140 con 0x56162bb71ea0 2024-05-08T18:05:04.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.226+0000 7f9cf0fd4640 1 -- 172.21.15.50:0/1671013527 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94653222705280] v27'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9ce8081070 con 0x56162bb71ea0 2024-05-08T18:05:04.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.226+0000 7f9cf6213800 20 remove_watcher() i=2 2024-05-08T18:05:04.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.226+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94653222718208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb77140 con 0x7f9ce8076040 2024-05-08T18:05:04.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.227+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94653222718208] v27'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9cec040040 con 0x7f9ce8076040 2024-05-08T18:05:04.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.227+0000 7f9cf6213800 20 remove_watcher() i=3 2024-05-08T18:05:04.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.227+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94653222720464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb78b00 con 0x56162bb68b20 2024-05-08T18:05:04.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.228+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94653222720464] v27'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9cec040040 con 0x56162bb68b20 2024-05-08T18:05:04.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.228+0000 7f9cf6213800 20 remove_watcher() i=4 2024-05-08T18:05:04.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.228+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94653222723728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb77140 con 0x56162bb6d2a0 2024-05-08T18:05:04.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.229+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94653222723728] v27'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.229+0000 7f9cf6213800 20 remove_watcher() i=5 2024-05-08T18:05:04.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.229+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94653222729904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb793d0 con 0x56162bb6d2a0 2024-05-08T18:05:04.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.230+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94653222729904] v27'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.230+0000 7f9cf6213800 20 remove_watcher() i=6 2024-05-08T18:05:04.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.230+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94653222730960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56162bb79370 con 0x56162bb6d2a0 2024-05-08T18:05:04.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.231+0000 7f9cf17d5640 1 -- 172.21.15.50:0/1671013527 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94653222730960] v27'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9ce4072070 con 0x56162bb6d2a0 2024-05-08T18:05:04.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.231+0000 7f9cf6213800 20 remove_watcher() i=7 2024-05-08T18:05:04.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.231+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56162bb6d2a0 msgr2=0x56162bb6f690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.231+0000 7f9cf6213800 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56162bb6d2a0 0x56162bb6f690 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.231+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56162bb68b20 msgr2=0x56162bb6afe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.231+0000 7f9cf6213800 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56162bb68b20 0x56162bb6afe0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.231+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9ce8076040 msgr2=0x7f9ce8083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.231+0000 7f9cf6213800 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9ce8076040 0x7f9ce8083410 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.231+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56162bb71ea0 msgr2=0x56162bb74360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.231+0000 7f9cf6213800 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56162bb71ea0 0x56162bb74360 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9bbc03bba0 msgr2=0x7f9bbc03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9bbc03bba0 0x7f9bbc03e060 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8052fd0 tx=0x7f9ce804f000 comp rx=0 tx=0).stop 2024-05-08T18:05:04.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb56090 msgr2=0x56162bb5d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 --2- 172.21.15.50:0/1671013527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb56090 0x56162bb5d370 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400efd0 tx=0x7f9ce403b8f0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 reap_dead start 2024-05-08T18:05:04.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/1671013527 reap_dead start 2024-05-08T18:05:04.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 shutdown_connections 2024-05-08T18:05:04.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 >> 172.21.15.50:0/1671013527 conn(0x56162bb52360 msgr2=0x56162bb4f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 shutdown_connections 2024-05-08T18:05:04.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 -- 172.21.15.50:0/1671013527 wait complete. 2024-05-08T18:05:04.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 -- 172.21.15.50:0/264768877 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9bc803bba0 msgr2=0x7f9bc803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 --2- 172.21.15.50:0/264768877 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9bc803bba0 0x7f9bc803e060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x56162bb2ec50 tx=0x7f9ce8005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:04.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 -- 172.21.15.50:0/264768877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb48d80 msgr2=0x56162bb491a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 --2- 172.21.15.50:0/264768877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb48d80 0x56162bb491a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4051820 tx=0x7f9ce4051f00 comp rx=0 tx=0).stop 2024-05-08T18:05:04.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 -- 172.21.15.50:0/264768877 shutdown_connections 2024-05-08T18:05:04.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 --2- 172.21.15.50:0/264768877 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9bc803bba0 0x7f9bc803e060 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 --2- 172.21.15.50:0/264768877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162bb48d80 0x56162bb491a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 -- 172.21.15.50:0/264768877 >> 172.21.15.50:0/264768877 conn(0x56162bb3ce20 msgr2=0x56162bb3d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 -- 172.21.15.50:0/264768877 shutdown_connections 2024-05-08T18:05:04.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.232+0000 7f9cf6213800 1 -- 172.21.15.50:0/264768877 wait complete. 2024-05-08T18:05:04.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.233+0000 7f9cf6213800 1 -- 172.21.15.50:0/818295857 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56162bb34030 msgr2=0x56162bb36420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.233+0000 7f9cf6213800 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56162bb34030 0x56162bb36420 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.233+0000 7f9cf6213800 1 -- 172.21.15.50:0/818295857 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56162ba349e0 msgr2=0x56162bb31760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.233+0000 7f9cf6213800 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56162ba349e0 0x56162bb31760 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.233+0000 7f9cf6213800 1 -- 172.21.15.50:0/818295857 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56162bb37f20 msgr2=0x56162bb3a310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.233+0000 7f9cf6213800 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56162bb37f20 0x56162bb3a310 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.233+0000 7f9cf6213800 1 -- 172.21.15.50:0/818295857 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9bd803bd40 msgr2=0x7f9bd803e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.233+0000 7f9cf6213800 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9bd803bd40 0x7f9bd803e200 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8054000 tx=0x7f9ce804e000 comp rx=0 tx=0).stop 2024-05-08T18:05:04.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.233+0000 7f9cf6213800 1 -- 172.21.15.50:0/818295857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba30550 msgr2=0x56162bb2a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.233+0000 7f9cf6213800 1 --2- 172.21.15.50:0/818295857 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56162ba30550 0x56162bb2a520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9ce40012f0 tx=0x7f9ce4000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.233+0000 7f9cf1fd6640 1 -- 172.21.15.50:0/818295857 reap_dead start 2024-05-08T18:05:04.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.233+0000 7f9cf6213800 1 -- 172.21.15.50:0/818295857 shutdown_connections 2024-05-08T18:05:04.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.233+0000 7f9cf6213800 1 -- 172.21.15.50:0/818295857 >> 172.21.15.50:0/818295857 conn(0x56162ba2bb50 msgr2=0x56162bb1caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.233+0000 7f9cf6213800 1 -- 172.21.15.50:0/818295857 shutdown_connections 2024-05-08T18:05:04.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.233+0000 7f9cf6213800 1 -- 172.21.15.50:0/818295857 wait complete. 2024-05-08T18:05:04.247 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:04.247 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2024-05-08T18:05:04.247 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2024-05-08T18:05:04.247 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2024-05-08T18:05:04.294 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:04.294 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:04.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.297+0000 7fcb93b46800 1 Processor -- start 2024-05-08T18:05:04.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.297+0000 7fcb93b46800 1 -- start start 2024-05-08T18:05:04.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.297+0000 7fcb93b46800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x561628984930 0x561628984d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.297+0000 7fcb93b46800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x561628985380 con 0x561628987970 2024-05-08T18:05:04.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.297+0000 7fcb93b46800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x561628826e20 con 0x561628984930 2024-05-08T18:05:04.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.298+0000 7fcb9163e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x561628984930 0x561628984d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.298+0000 7fcb90e3d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x561628987970 0x561628987e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:53080/0 (socket says 172.21.15.50:53080) 2024-05-08T18:05:04.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.298+0000 7fcb90e3d640 1 -- 172.21.15.50:0/4238742201 learned_addr learned my addr 172.21.15.50:0/4238742201 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:04.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.298+0000 7fcb9163e640 1 --2- 172.21.15.50:0/4238742201 >> v2:172.21.15.46:3300/0 conn(0x561628984930 0x561628984d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58048/0 (socket says 172.21.15.50:58048) 2024-05-08T18:05:04.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.298+0000 7fcb9163e640 1 -- 172.21.15.50:0/4238742201 >> v1:172.21.15.46:6789/0 conn(0x561628987970 legacy=0x561628987e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.298+0000 7fcb9163e640 1 -- 172.21.15.50:0/4238742201 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x561628988530 con 0x561628984930 2024-05-08T18:05:04.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.298+0000 7fcb83fff640 1 -- 172.21.15.50:0/4238742201 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2297624573 0 0) 0x561628985380 con 0x561628987970 2024-05-08T18:05:04.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.300+0000 7fcb9163e640 1 --2- 172.21.15.50:0/4238742201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628984930 0x561628984d50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400a470 tx=0x7fcb8401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7d7e0ff3cb0cb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.300+0000 7fcb83fff640 1 -- 172.21.15.50:0/4238742201 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb84057070 con 0x561628984930 2024-05-08T18:05:04.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.300+0000 7fcb83fff640 1 -- 172.21.15.50:0/4238742201 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb8404adc0 con 0x561628984930 2024-05-08T18:05:04.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.300+0000 7fcb83fff640 1 -- 172.21.15.50:0/4238742201 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb84052590 con 0x561628984930 2024-05-08T18:05:04.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.300+0000 7fcb93b46800 1 -- 172.21.15.50:0/4238742201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628984930 msgr2=0x561628984d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.300+0000 7fcb93b46800 1 --2- 172.21.15.50:0/4238742201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628984930 0x561628984d50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400a470 tx=0x7fcb8401ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:04.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.300+0000 7fcb93b46800 1 -- 172.21.15.50:0/4238742201 shutdown_connections 2024-05-08T18:05:04.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.300+0000 7fcb93b46800 1 --2- 172.21.15.50:0/4238742201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628984930 0x561628984d50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.300+0000 7fcb93b46800 1 -- 172.21.15.50:0/4238742201 >> 172.21.15.50:0/4238742201 conn(0x5616289802d0 msgr2=0x5616289826f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.300+0000 7fcb93b46800 1 -- 172.21.15.50:0/4238742201 shutdown_connections 2024-05-08T18:05:04.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.300+0000 7fcb93b46800 1 -- 172.21.15.50:0/4238742201 wait complete. 2024-05-08T18:05:04.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.316+0000 7fcb93b46800 1 Processor -- start 2024-05-08T18:05:04.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.316+0000 7fcb93b46800 1 -- start start 2024-05-08T18:05:04.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.316+0000 7fcb93b46800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628984930 0x561628a77a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.316+0000 7fcb93b46800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5616288d1a10 con 0x561628984930 2024-05-08T18:05:04.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.316+0000 7fcb9163e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628984930 0x561628a77a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.316+0000 7fcb9163e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628984930 0x561628a77a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58050/0 (socket says 172.21.15.50:58050) 2024-05-08T18:05:04.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.316+0000 7fcb9163e640 1 -- 172.21.15.50:0/427460996 learned_addr learned my addr 172.21.15.50:0/427460996 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.317+0000 7fcb9163e640 1 -- 172.21.15.50:0/427460996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561628a78030 con 0x561628984930 2024-05-08T18:05:04.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.317+0000 7fcb9163e640 1 --2- 172.21.15.50:0/427460996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628984930 0x561628a77a60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcb84055040 tx=0x7fcb8404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581fef29d09f0122 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.317+0000 7fca7f606640 1 -- 172.21.15.50:0/427460996 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb84057040 con 0x561628984930 2024-05-08T18:05:04.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.317+0000 7fca7f606640 1 -- 172.21.15.50:0/427460996 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb84062070 con 0x561628984930 2024-05-08T18:05:04.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.317+0000 7fca7f606640 1 -- 172.21.15.50:0/427460996 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb8405d470 con 0x561628984930 2024-05-08T18:05:04.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.318+0000 7fcb93b46800 1 -- 172.21.15.50:0/427460996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628984930 msgr2=0x561628a77a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.318+0000 7fcb93b46800 1 --2- 172.21.15.50:0/427460996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628984930 0x561628a77a60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcb84055040 tx=0x7fcb8404c000 comp rx=0 tx=0).stop 2024-05-08T18:05:04.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.318+0000 7fcb93b46800 1 -- 172.21.15.50:0/427460996 shutdown_connections 2024-05-08T18:05:04.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.318+0000 7fcb93b46800 1 --2- 172.21.15.50:0/427460996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628984930 0x561628a77a60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.318+0000 7fcb93b46800 1 -- 172.21.15.50:0/427460996 >> 172.21.15.50:0/427460996 conn(0x5616289802d0 msgr2=0x561628a73380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.318+0000 7fcb93b46800 1 -- 172.21.15.50:0/427460996 shutdown_connections 2024-05-08T18:05:04.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.318+0000 7fcb93b46800 1 -- 172.21.15.50:0/427460996 wait complete. 2024-05-08T18:05:04.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.318+0000 7fcb93b46800 1 Processor -- start 2024-05-08T18:05:04.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.318+0000 7fcb93b46800 1 -- start start 2024-05-08T18:05:04.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.318+0000 7fcb93b46800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628987970 0x561628a7f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.318+0000 7fcb93b46800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56162898fde0 con 0x561628987970 2024-05-08T18:05:04.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.319+0000 7fcb9163e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628987970 0x561628a7f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.319+0000 7fcb9163e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628987970 0x561628a7f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58056/0 (socket says 172.21.15.50:58056) 2024-05-08T18:05:04.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.319+0000 7fcb9163e640 1 -- 172.21.15.50:0/3978824395 learned_addr learned my addr 172.21.15.50:0/3978824395 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.319+0000 7fcb9163e640 1 -- 172.21.15.50:0/3978824395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561628a7fc40 con 0x561628987970 2024-05-08T18:05:04.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.319+0000 7fcb9163e640 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628987970 0x561628a7f700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcb8401ce60 tx=0x7fcb84002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.319+0000 7fca7de03640 1 -- 172.21.15.50:0/3978824395 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb8406b070 con 0x561628987970 2024-05-08T18:05:04.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.319+0000 7fca7de03640 1 -- 172.21.15.50:0/3978824395 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb8405d830 con 0x561628987970 2024-05-08T18:05:04.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.319+0000 7fcb93b46800 1 -- 172.21.15.50:0/3978824395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561628a7fed0 con 0x561628987970 2024-05-08T18:05:04.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.319+0000 7fca7de03640 1 -- 172.21.15.50:0/3978824395 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb84057070 con 0x561628987970 2024-05-08T18:05:04.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.319+0000 7fcb93b46800 1 -- 172.21.15.50:0/3978824395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561628a82b80 con 0x561628987970 2024-05-08T18:05:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.320+0000 7fca7de03640 1 -- 172.21.15.50:0/3978824395 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fcb8405db50 con 0x561628987970 2024-05-08T18:05:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.320+0000 7fca7de03640 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca7803bba0 0x7fca7803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.320+0000 7fca7de03640 1 -- 172.21.15.50:0/3978824395 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fcb840942c0 con 0x561628987970 2024-05-08T18:05:04.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.321+0000 7fcb90e3d640 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca7803bba0 0x7fca7803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.321+0000 7fcb93b46800 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561628a84930 0x561628a86d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.321+0000 7fcb93b46800 1 -- 172.21.15.50:0/3978824395 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628a87260 con 0x561628a84930 2024-05-08T18:05:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.321+0000 7fcb91e3f640 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561628a84930 0x561628a86d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.321+0000 7fcb90e3d640 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca7803bba0 0x7fca7803e060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcb88000c00 tx=0x7fcb880013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.321+0000 7fcb91e3f640 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561628a84930 0x561628a86d20 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.321+0000 7fcb91e3f640 1 -- 172.21.15.50:0/3978824395 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628a84930 2024-05-08T18:05:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.321+0000 7fcb93b46800 1 -- 172.21.15.50:0/3978824395 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628a83eb0 con 0x561628a84930 2024-05-08T18:05:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.322+0000 7fcb91e3f640 1 -- 172.21.15.50:0/3978824395 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628a84930 2024-05-08T18:05:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.322+0000 7fcb93b46800 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x561628a89530 0x561628a8b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.322+0000 7fcb93b46800 1 -- 172.21.15.50:0/3978824395 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628a8bff0 con 0x561628a89530 2024-05-08T18:05:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.322+0000 7fcb9163e640 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x561628a89530 0x561628a8b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.322+0000 7fcb9163e640 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x561628a89530 0x561628a8b920 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.322+0000 7fcb9163e640 1 -- 172.21.15.50:0/3978824395 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fcb8405a040 con 0x561628a89530 2024-05-08T18:05:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.322+0000 7fcb93b46800 1 -- 172.21.15.50:0/3978824395 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628a883a0 con 0x561628a84930 2024-05-08T18:05:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.323+0000 7fcb91e3f640 1 -- 172.21.15.50:0/3978824395 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628a84930 2024-05-08T18:05:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.323+0000 7fcb93b46800 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x561628a8d430 0x561628a8f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.323+0000 7fcb93b46800 1 -- 172.21.15.50:0/3978824395 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628a8fef0 con 0x561628a8d430 2024-05-08T18:05:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.323+0000 7fcb90e3d640 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x561628a8d430 0x561628a8f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.323+0000 7fcb90e3d640 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x561628a8d430 0x561628a8f820 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.323+0000 7fcb90e3d640 1 -- 172.21.15.50:0/3978824395 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fcb8800f670 con 0x561628a8d430 2024-05-08T18:05:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.324+0000 7fcb93b46800 1 Processor -- start 2024-05-08T18:05:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.324+0000 7fcb93b46800 1 -- start start 2024-05-08T18:05:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.324+0000 7fcb93b46800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628a9e260 0x561628a9e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.324+0000 7fcb93b46800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561628878330 con 0x561628a9e260 2024-05-08T18:05:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.324+0000 7fcb9163e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628a9e260 0x561628a9e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.324+0000 7fcb9163e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628a9e260 0x561628a9e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58058/0 (socket says 172.21.15.50:58058) 2024-05-08T18:05:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.324+0000 7fcb9163e640 1 -- 172.21.15.50:0/1400580033 learned_addr learned my addr 172.21.15.50:0/1400580033 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.325+0000 7fcb9163e640 1 -- 172.21.15.50:0/1400580033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561628a9ebc0 con 0x561628a9e260 2024-05-08T18:05:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.325+0000 7fcb9163e640 1 --2- 172.21.15.50:0/1400580033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628a9e260 0x561628a9e680 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcb8406e860 tx=0x7fcb84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.325+0000 7fca76ffd640 1 -- 172.21.15.50:0/1400580033 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb84087070 con 0x561628a9e260 2024-05-08T18:05:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.325+0000 7fca76ffd640 1 -- 172.21.15.50:0/1400580033 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb840707a0 con 0x561628a9e260 2024-05-08T18:05:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.325+0000 7fca76ffd640 1 -- 172.21.15.50:0/1400580033 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb84009560 con 0x561628a9e260 2024-05-08T18:05:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.325+0000 7fcb93b46800 1 -- 172.21.15.50:0/1400580033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561628a9f630 con 0x561628a9e260 2024-05-08T18:05:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.325+0000 7fcb93b46800 1 -- 172.21.15.50:0/1400580033 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561628a9fb30 con 0x561628a9e260 2024-05-08T18:05:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.326+0000 7fca76ffd640 1 -- 172.21.15.50:0/1400580033 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fcb8408a070 con 0x561628a9e260 2024-05-08T18:05:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.326+0000 7fca76ffd640 1 --2- 172.21.15.50:0/1400580033 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca6803bb00 0x7fca6803dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.326+0000 7fca76ffd640 1 -- 172.21.15.50:0/1400580033 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fcb84054090 con 0x561628a9e260 2024-05-08T18:05:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.326+0000 7fcb90e3d640 1 --2- 172.21.15.50:0/1400580033 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca6803bb00 0x7fca6803dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.326+0000 7fcb93b46800 1 Processor -- start 2024-05-08T18:05:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.326+0000 7fcb93b46800 1 -- start start 2024-05-08T18:05:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.326+0000 7fcb90e3d640 1 --2- 172.21.15.50:0/1400580033 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca6803bb00 0x7fca6803dfc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x561628a87ae0 tx=0x7fcb88006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.326+0000 7fcb93b46800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628aab410 0x561628aab830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.326+0000 7fcb93b46800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561628a782f0 con 0x561628aab410 2024-05-08T18:05:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.327+0000 7fcb9163e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628aab410 0x561628aab830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.327+0000 7fcb9163e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628aab410 0x561628aab830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58070/0 (socket says 172.21.15.50:58070) 2024-05-08T18:05:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.327+0000 7fcb9163e640 1 -- 172.21.15.50:0/3304638136 learned_addr learned my addr 172.21.15.50:0/3304638136 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.327+0000 7fcb9163e640 1 -- 172.21.15.50:0/3304638136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561628aabe00 con 0x561628aab410 2024-05-08T18:05:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.327+0000 7fcb9163e640 1 --2- 172.21.15.50:0/3304638136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628aab410 0x561628aab830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcb8406e810 tx=0x7fcb8404f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d6b991d109cfdb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.327+0000 7fca74ff9640 1 -- 172.21.15.50:0/3304638136 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb8407bd10 con 0x561628aab410 2024-05-08T18:05:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.327+0000 7fca74ff9640 1 -- 172.21.15.50:0/3304638136 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb840927f0 con 0x561628aab410 2024-05-08T18:05:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.327+0000 7fca74ff9640 1 -- 172.21.15.50:0/3304638136 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb84079820 con 0x561628aab410 2024-05-08T18:05:04.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.328+0000 7fcb93b46800 1 -- 172.21.15.50:0/3304638136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628aab410 msgr2=0x561628aab830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.328+0000 7fcb93b46800 1 --2- 172.21.15.50:0/3304638136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628aab410 0x561628aab830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcb8406e810 tx=0x7fcb8404f360 comp rx=0 tx=0).stop 2024-05-08T18:05:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.328+0000 7fcb93b46800 1 -- 172.21.15.50:0/3304638136 shutdown_connections 2024-05-08T18:05:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.328+0000 7fcb93b46800 1 --2- 172.21.15.50:0/3304638136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628aab410 0x561628aab830 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.328+0000 7fcb93b46800 1 -- 172.21.15.50:0/3304638136 >> 172.21.15.50:0/3304638136 conn(0x561628aa76e0 msgr2=0x561628aa5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.328+0000 7fcb93b46800 1 -- 172.21.15.50:0/3304638136 shutdown_connections 2024-05-08T18:05:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.328+0000 7fcb93b46800 1 -- 172.21.15.50:0/3304638136 wait complete. 2024-05-08T18:05:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.328+0000 7fcb93b46800 1 Processor -- start 2024-05-08T18:05:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.328+0000 7fcb93b46800 1 -- start start 2024-05-08T18:05:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.328+0000 7fcb93b46800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628aab410 0x561628ab2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.328+0000 7fcb93b46800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561628aac460 con 0x561628aab410 2024-05-08T18:05:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.328+0000 7fcb9163e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628aab410 0x561628ab2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.328+0000 7fcb9163e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628aab410 0x561628ab2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58084/0 (socket says 172.21.15.50:58084) 2024-05-08T18:05:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.328+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 learned_addr learned my addr 172.21.15.50:0/2627356195 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.329+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561628ab2c70 con 0x561628aab410 2024-05-08T18:05:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.329+0000 7fcb9163e640 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628aab410 0x561628ab2730 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcb8408dfd0 tx=0x7fcb84057f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.329+0000 7fca66ffd640 1 -- 172.21.15.50:0/2627356195 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb840ba070 con 0x561628aab410 2024-05-08T18:05:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.329+0000 7fca66ffd640 1 -- 172.21.15.50:0/2627356195 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb840af070 con 0x561628aab410 2024-05-08T18:05:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.329+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561628ab2c70 con 0x561628aab410 2024-05-08T18:05:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.329+0000 7fca66ffd640 1 -- 172.21.15.50:0/2627356195 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb84082860 con 0x561628aab410 2024-05-08T18:05:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.329+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561628ab2c70 con 0x561628aab410 2024-05-08T18:05:04.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.330+0000 7fca66ffd640 1 -- 172.21.15.50:0/2627356195 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fcb840bd070 con 0x561628aab410 2024-05-08T18:05:04.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.330+0000 7fca66ffd640 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca5c03bba0 0x7fca5c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.330+0000 7fca66ffd640 1 -- 172.21.15.50:0/2627356195 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fcb840c0ca0 con 0x561628aab410 2024-05-08T18:05:04.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.330+0000 7fcb90e3d640 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca5c03bba0 0x7fca5c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.330+0000 7fcb90e3d640 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca5c03bba0 0x7fca5c03e060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800ffd0 tx=0x7fcb88023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.331+0000 7fcb93b46800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.331+0000 7fcb93b46800 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561628abdef0 0x561628ac03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.331+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628ac0b00 con 0x561628abdef0 2024-05-08T18:05:04.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.331+0000 7fcb91e3f640 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561628abdef0 0x561628ac03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.331+0000 7fcb91e3f640 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561628abdef0 0x561628ac03b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.331+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628abdef0 2024-05-08T18:05:04.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.331+0000 7fcb93b46800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.331+0000 7fcb93b46800 20 realm 2024-05-08T18:05:04.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.331+0000 7fcb93b46800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.331+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628ac0f10 con 0x561628abdef0 2024-05-08T18:05:04.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.332+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628abdef0 2024-05-08T18:05:04.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.332+0000 7fcb93b46800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.332+0000 7fcb93b46800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:04.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.332+0000 7fcb93b46800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.332+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628ac17e0 con 0x561628abdef0 2024-05-08T18:05:04.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.332+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628abdef0 2024-05-08T18:05:04.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.332+0000 7fcb93b46800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.332+0000 7fcb93b46800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.332+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628abdef0 2024-05-08T18:05:04.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.332+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628abdef0 2024-05-08T18:05:04.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.332+0000 7fcb93b46800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:04.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.332+0000 7fcb93b46800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.332+0000 7fcb93b46800 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x561628ac28e0 0x561628ac4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.332+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628ac52e0 con 0x561628ac28e0 2024-05-08T18:05:04.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.332+0000 7fcb9163e640 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x561628ac28e0 0x561628ac4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.333+0000 7fcb9163e640 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x561628ac28e0 0x561628ac4cd0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.333+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.333+0000 7fcb93b46800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:04.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.333+0000 7fcb93b46800 20 searching for the correct realm 2024-05-08T18:05:04.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.333+0000 7fcb93b46800 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x561628ac7290 0x561628ac9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.333+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac9e20 con 0x561628ac7290 2024-05-08T18:05:04.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.333+0000 7fcb90e3d640 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x561628ac7290 0x561628ac9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.334+0000 7fcb90e3d640 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x561628ac7290 0x561628ac9750 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.334+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb88037040 con 0x561628ac7290 2024-05-08T18:05:04.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.334+0000 7fcb90e3d640 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fcb8802e910 0x7fcb8803a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.334+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac9e20 con 0x7fcb8802e910 2024-05-08T18:05:04.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.334+0000 7fcb91e3f640 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fcb8802e910 0x7fcb8803a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.334+0000 7fcb91e3f640 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fcb8802e910 0x7fcb8803a3c0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.334+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x7fcb8802e910 2024-05-08T18:05:04.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.334+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac28e0 2024-05-08T18:05:04.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.335+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.335+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628abdef0 2024-05-08T18:05:04.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.335+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628abdef0 2024-05-08T18:05:04.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.335+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628abdef0 2024-05-08T18:05:04.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.335+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628abdef0 2024-05-08T18:05:04.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.335+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac28e0 2024-05-08T18:05:04.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.336+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.336+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac28e0 2024-05-08T18:05:04.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.336+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.336+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x7fcb8802e910 2024-05-08T18:05:04.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.336+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x7fcb8802e910 2024-05-08T18:05:04.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.336+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac7290 2024-05-08T18:05:04.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.336+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb88037040 con 0x561628ac7290 2024-05-08T18:05:04.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.336+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628abdef0 2024-05-08T18:05:04.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.337+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628abdef0 2024-05-08T18:05:04.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.337+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac28e0 2024-05-08T18:05:04.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.337+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.337+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628abdef0 2024-05-08T18:05:04.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.337+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628abdef0 2024-05-08T18:05:04.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.337+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x7fcb8802e910 2024-05-08T18:05:04.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.337+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x7fcb8802e910 2024-05-08T18:05:04.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.337+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac7290 2024-05-08T18:05:04.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.338+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb88037040 con 0x561628ac7290 2024-05-08T18:05:04.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.338+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac7290 2024-05-08T18:05:04.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.338+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb88037040 con 0x561628ac7290 2024-05-08T18:05:04.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.338+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac28e0 2024-05-08T18:05:04.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.338+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.338+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac7290 2024-05-08T18:05:04.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.338+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb88037040 con 0x561628ac7290 2024-05-08T18:05:04.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.338+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac7290 2024-05-08T18:05:04.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.339+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fcb88037040 con 0x561628ac7290 2024-05-08T18:05:04.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.339+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628abdef0 2024-05-08T18:05:04.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.339+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628abdef0 2024-05-08T18:05:04.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.339+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628abdef0 2024-05-08T18:05:04.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.339+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628abdef0 2024-05-08T18:05:04.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.339+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac28e0 2024-05-08T18:05:04.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.340+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.340+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac7290 2024-05-08T18:05:04.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.340+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb88037040 con 0x561628ac7290 2024-05-08T18:05:04.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.340+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x7fcb8802e910 2024-05-08T18:05:04.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.340+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x7fcb8802e910 2024-05-08T18:05:04.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.340+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628abdef0 2024-05-08T18:05:04.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.340+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628abdef0 2024-05-08T18:05:04.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.340+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac28e0 2024-05-08T18:05:04.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.341+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.341+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac7290 2024-05-08T18:05:04.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.341+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb88037040 con 0x561628ac7290 2024-05-08T18:05:04.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.341+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac28e0 2024-05-08T18:05:04.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.341+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.341+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x7fcb8802e910 2024-05-08T18:05:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.341+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x7fcb8802e910 2024-05-08T18:05:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.341+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac7290 2024-05-08T18:05:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.342+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb88037040 con 0x561628ac7290 2024-05-08T18:05:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.342+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628ac28e0 2024-05-08T18:05:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.342+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.342+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x561628abdef0 2024-05-08T18:05:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.342+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628abdef0 2024-05-08T18:05:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.342+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561628ac1b80 con 0x7fcb8802e910 2024-05-08T18:05:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x7fcb8802e910 2024-05-08T18:05:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:04.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628ac5ba0 con 0x561628abdef0 2024-05-08T18:05:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628abdef0 2024-05-08T18:05:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628ac5ba0 con 0x561628abdef0 2024-05-08T18:05:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x561628abdef0 2024-05-08T18:05:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628ac5ba0 con 0x561628ac7290 2024-05-08T18:05:04.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fcb88037040 con 0x561628ac7290 2024-05-08T18:05:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 20 zone default found 2024-05-08T18:05:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 4 Realm: () 2024-05-08T18:05:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.343+0000 7fcb93b46800 20 zonegroup default 2024-05-08T18:05:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.344+0000 7fcb93b46800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.344+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628ac6500 con 0x561628ac28e0 2024-05-08T18:05:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.344+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.344+0000 7fcb93b46800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.344+0000 7fcb93b46800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.344+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628acbd10 con 0x561628ac7290 2024-05-08T18:05:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.344+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fcb88037040 con 0x561628ac7290 2024-05-08T18:05:04.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.344+0000 7fcb93b46800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.344+0000 7fcb93b46800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.344+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628acc200 con 0x561628ac7290 2024-05-08T18:05:04.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.344+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fcb88037040 con 0x561628ac7290 2024-05-08T18:05:04.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.344+0000 7fcb93b46800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.345+0000 7fcb93b46800 20 started sync module instance, tier type = 2024-05-08T18:05:04.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.345+0000 7fcb93b46800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:04.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.345+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628acb120 con 0x7fcb8802e910 2024-05-08T18:05:04.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.346+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb8c00a4a0 con 0x7fcb8802e910 2024-05-08T18:05:04.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.346+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94653171676704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628acb120 con 0x7fcb8802e910 2024-05-08T18:05:04.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.346+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628acccc0 con 0x561628abdef0 2024-05-08T18:05:04.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.347+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94653171676704] v27'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb8c040040 con 0x7fcb8802e910 2024-05-08T18:05:04.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.347+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb8c040040 con 0x561628abdef0 2024-05-08T18:05:04.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.347+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94653171681232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628acb120 con 0x561628abdef0 2024-05-08T18:05:04.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.347+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628acd770 con 0x561628ac7290 2024-05-08T18:05:04.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.348+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb88037040 con 0x561628ac7290 2024-05-08T18:05:04.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.348+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94653171681232] v27'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb8c040040 con 0x561628abdef0 2024-05-08T18:05:04.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.348+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94653171683920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628acdb60 con 0x561628ac7290 2024-05-08T18:05:04.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.348+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628ace580 con 0x7fcb8802e910 2024-05-08T18:05:04.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.349+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94653171683920] v27'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb88037040 con 0x561628ac7290 2024-05-08T18:05:04.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.349+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb8c040040 con 0x7fcb8802e910 2024-05-08T18:05:04.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.349+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94653171687536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628ace970 con 0x7fcb8802e910 2024-05-08T18:05:04.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.349+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628acf370 con 0x561628abdef0 2024-05-08T18:05:04.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.350+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb8c040040 con 0x561628abdef0 2024-05-08T18:05:04.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.350+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94653171691136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628acf2a0 con 0x561628abdef0 2024-05-08T18:05:04.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.350+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628ad0280 con 0x561628ac28e0 2024-05-08T18:05:04.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.350+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94653171687536] v27'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb8c040040 con 0x7fcb8802e910 2024-05-08T18:05:04.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.351+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.351+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94653171691136] v27'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb8c040040 con 0x561628abdef0 2024-05-08T18:05:04.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.351+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94653171674224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628ad0670 con 0x561628ac28e0 2024-05-08T18:05:04.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.351+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628acf760 con 0x561628ac28e0 2024-05-08T18:05:04.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.352+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.352+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94653171693408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628ad1030 con 0x561628ac28e0 2024-05-08T18:05:04.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.352+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628ad1a70 con 0x561628ac28e0 2024-05-08T18:05:04.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.352+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94653171674224] v27'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.353+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v27'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.353+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94653171693408] v27'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.353+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94653171701056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628ad1a30 con 0x561628ac28e0 2024-05-08T18:05:04.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.353+0000 7fcb93b46800 20 add_watcher() i=0 2024-05-08T18:05:04.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.353+0000 7fcb93b46800 20 add_watcher() i=1 2024-05-08T18:05:04.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.353+0000 7fcb93b46800 20 add_watcher() i=2 2024-05-08T18:05:04.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.353+0000 7fcb93b46800 20 add_watcher() i=3 2024-05-08T18:05:04.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.353+0000 7fcb93b46800 20 add_watcher() i=4 2024-05-08T18:05:04.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.353+0000 7fcb93b46800 20 add_watcher() i=5 2024-05-08T18:05:04.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.353+0000 7fcb93b46800 20 add_watcher() i=6 2024-05-08T18:05:04.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.354+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94653171701056] v27'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.354+0000 7fcb93b46800 20 add_watcher() i=7 2024-05-08T18:05:04.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.354+0000 7fcb93b46800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:04.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.354+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628ad4a60 con 0x561628abdef0 2024-05-08T18:05:04.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.355+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fcb8c040040 con 0x561628abdef0 2024-05-08T18:05:04.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.355+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94653171694976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628ad1ee0 con 0x561628abdef0 2024-05-08T18:05:04.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.356+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94653171694976] v27'56 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fcb8c040040 con 0x561628abdef0 2024-05-08T18:05:04.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.356+0000 7fca47fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:04.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.356+0000 7fcb93b46800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:04.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.356+0000 7fcb93b46800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:04.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.356+0000 7fcb93b46800 5 note: GC not initialized 2024-05-08T18:05:04.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.356+0000 7fca46fe5640 20 reqs_thread_entry: start 2024-05-08T18:05:04.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.356+0000 7fcb93b46800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:04.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.356+0000 7fcb93b46800 20 Filter name: none 2024-05-08T18:05:04.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.356+0000 7fca44fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:04.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.356+0000 7fcb93b46800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:04.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.356+0000 7fcb93b46800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.356+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628b07e60 con 0x561628abdef0 2024-05-08T18:05:04.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.357+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fcb8c040040 con 0x561628abdef0 2024-05-08T18:05:04.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.357+0000 7fcb93b46800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:04.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.357+0000 7fcb93b46800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:04.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.357+0000 7fcb93b46800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:04.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.357+0000 7fcb93b46800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:04.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.357+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628b0a8f0 con 0x7fcb8802e910 2024-05-08T18:05:04.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.357+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 155+0+0 (crc 0 0 0) 0x7fcb8c040040 con 0x7fcb8802e910 2024-05-08T18:05:04.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.357+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628b0e100 con 0x561628abdef0 2024-05-08T18:05:04.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.358+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~391] v27'2 uv2 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fcb8c040040 con 0x561628abdef0 2024-05-08T18:05:04.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.358+0000 7fcb93b46800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:04.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.358+0000 7fcb93b46800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:04.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.358+0000 7fcb93b46800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-05-08T18:05:04.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.358+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94653171948224 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561628b098e0 con 0x561628abdef0 2024-05-08T18:05:04.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.359+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== watch-notify(notify (1) cookie 94653171691136 notify 115964116993 ret 0) ==== 609+0+0 (crc 0 0 0) 0x7fcb8c018db0 con 0x561628abdef0 2024-05-08T18:05:04.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.359+0000 7fca74ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116993 cookie 94653171691136 notifier 4305 bl.length()=567 2024-05-08T18:05:04.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.359+0000 7fca74ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:04.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.359+0000 7fca74ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:04.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.359+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(65 notify.4 [notify cookie 94653171948224 out=8b] v0'0 uv33 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fcb8c040040 con 0x561628abdef0 2024-05-08T18:05:04.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.359+0000 7fca74ff9640 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fca54003ea0 con 0x561628abdef0 2024-05-08T18:05:04.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.359+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb8c040040 con 0x561628abdef0 2024-05-08T18:05:04.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.359+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== watch-notify(notify_complete (2) cookie 94653171948224 notify 115964116993 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fcb8c038450 con 0x561628abdef0 2024-05-08T18:05:04.373 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":1,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:04.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.360+0000 7fca47fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:04.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.360+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94653171694976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628b0cea0 con 0x561628abdef0 2024-05-08T18:05:04.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.361+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(67 data_loggenerations_metadata [watch unwatch cookie 94653171694976] v27'57 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fcb8c040040 con 0x561628abdef0 2024-05-08T18:05:04.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.361+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94653171676704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628acfd80 con 0x7fcb8802e910 2024-05-08T18:05:04.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.362+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(68 notify.0 [watch unwatch cookie 94653171676704] v27'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb8c040040 con 0x7fcb8802e910 2024-05-08T18:05:04.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.362+0000 7fcb93b46800 20 remove_watcher() i=0 2024-05-08T18:05:04.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.362+0000 7fcb93b46800 2 removed watcher, disabling cache 2024-05-08T18:05:04.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.362+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94653171681232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628acb620 con 0x561628abdef0 2024-05-08T18:05:04.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.363+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(69 notify.1 [watch unwatch cookie 94653171681232] v27'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb8c040040 con 0x561628abdef0 2024-05-08T18:05:04.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.363+0000 7fcb93b46800 20 remove_watcher() i=1 2024-05-08T18:05:04.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.363+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94653171683920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628acb620 con 0x561628ac7290 2024-05-08T18:05:04.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.364+0000 7fcb90e3d640 1 -- 172.21.15.50:0/2627356195 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(70 notify.2 [watch unwatch cookie 94653171683920] v27'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb88037040 con 0x561628ac7290 2024-05-08T18:05:04.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.364+0000 7fcb93b46800 20 remove_watcher() i=2 2024-05-08T18:05:04.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.364+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94653171687536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628acd250 con 0x7fcb8802e910 2024-05-08T18:05:04.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.365+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(71 notify.3 [watch unwatch cookie 94653171687536] v27'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb8c040040 con 0x7fcb8802e910 2024-05-08T18:05:04.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.365+0000 7fcb93b46800 20 remove_watcher() i=3 2024-05-08T18:05:04.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.365+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94653171691136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628ace070 con 0x561628abdef0 2024-05-08T18:05:04.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.365+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(72 notify.4 [watch unwatch cookie 94653171691136] v27'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb8c040040 con 0x561628abdef0 2024-05-08T18:05:04.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.365+0000 7fcb93b46800 20 remove_watcher() i=4 2024-05-08T18:05:04.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.365+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94653171674224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628acd250 con 0x561628ac28e0 2024-05-08T18:05:04.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.366+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(73 notify.5 [watch unwatch cookie 94653171674224] v27'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.366+0000 7fcb93b46800 20 remove_watcher() i=5 2024-05-08T18:05:04.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.366+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94653171693408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628acee80 con 0x561628ac28e0 2024-05-08T18:05:04.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.367+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(74 notify.6 [watch unwatch cookie 94653171693408] v27'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.367+0000 7fcb93b46800 20 remove_watcher() i=6 2024-05-08T18:05:04.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.367+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94653171701056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561628acd250 con 0x561628ac28e0 2024-05-08T18:05:04.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.368+0000 7fcb9163e640 1 -- 172.21.15.50:0/2627356195 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(75 notify.7 [watch unwatch cookie 94653171701056] v27'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcb840bc030 con 0x561628ac28e0 2024-05-08T18:05:04.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 20 remove_watcher() i=7 2024-05-08T18:05:04.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x561628ac28e0 msgr2=0x561628ac4cd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x561628ac28e0 0x561628ac4cd0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561628abdef0 msgr2=0x561628ac03b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561628abdef0 0x561628ac03b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fcb8802e910 msgr2=0x7fcb8803a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fcb8802e910 0x7fcb8803a3c0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x561628ac7290 msgr2=0x561628ac9750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x561628ac7290 0x561628ac9750 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca5c03bba0 msgr2=0x7fca5c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca5c03bba0 0x7fca5c03e060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800ffd0 tx=0x7fcb88023040 comp rx=0 tx=0).stop 2024-05-08T18:05:04.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb91e3f640 1 -- 172.21.15.50:0/2627356195 reap_dead start 2024-05-08T18:05:04.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628aab410 msgr2=0x561628ab2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628aab410 0x561628ab2730 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcb8408dfd0 tx=0x7fcb84057f20 comp rx=0 tx=0).stop 2024-05-08T18:05:04.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 shutdown_connections 2024-05-08T18:05:04.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 --2- 172.21.15.50:0/2627356195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628aab410 0x561628ab2730 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 >> 172.21.15.50:0/2627356195 conn(0x561628aa76e0 msgr2=0x561628aa4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 shutdown_connections 2024-05-08T18:05:04.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 -- 172.21.15.50:0/2627356195 wait complete. 2024-05-08T18:05:04.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 -- 172.21.15.50:0/1400580033 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca6803bb00 msgr2=0x7fca6803dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 --2- 172.21.15.50:0/1400580033 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca6803bb00 0x7fca6803dfc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x561628a87ae0 tx=0x7fcb88006000 comp rx=0 tx=0).stop 2024-05-08T18:05:04.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 -- 172.21.15.50:0/1400580033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628a9e260 msgr2=0x561628a9e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 --2- 172.21.15.50:0/1400580033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628a9e260 0x561628a9e680 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcb8406e860 tx=0x7fcb84009e30 comp rx=0 tx=0).stop 2024-05-08T18:05:04.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 -- 172.21.15.50:0/1400580033 shutdown_connections 2024-05-08T18:05:04.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 --2- 172.21.15.50:0/1400580033 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca6803bb00 0x7fca6803dfc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 --2- 172.21.15.50:0/1400580033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628a9e260 0x561628a9e680 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 -- 172.21.15.50:0/1400580033 >> 172.21.15.50:0/1400580033 conn(0x561628a92300 msgr2=0x561628a92700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 -- 172.21.15.50:0/1400580033 shutdown_connections 2024-05-08T18:05:04.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.369+0000 7fcb93b46800 1 -- 172.21.15.50:0/1400580033 wait complete. 2024-05-08T18:05:04.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.370+0000 7fcb93b46800 1 -- 172.21.15.50:0/3978824395 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x561628a89530 msgr2=0x561628a8b920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.370+0000 7fcb93b46800 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x561628a89530 0x561628a8b920 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.370+0000 7fcb93b46800 1 -- 172.21.15.50:0/3978824395 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561628a84930 msgr2=0x561628a86d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.370+0000 7fcb93b46800 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561628a84930 0x561628a86d20 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.370+0000 7fcb93b46800 1 -- 172.21.15.50:0/3978824395 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x561628a8d430 msgr2=0x561628a8f820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.370+0000 7fcb93b46800 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x561628a8d430 0x561628a8f820 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.370+0000 7fcb93b46800 1 -- 172.21.15.50:0/3978824395 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca7803bba0 msgr2=0x7fca7803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.370+0000 7fcb93b46800 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fca7803bba0 0x7fca7803e060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fcb88000c00 tx=0x7fcb880013b0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.370+0000 7fcb93b46800 1 -- 172.21.15.50:0/3978824395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628987970 msgr2=0x561628a7f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.370+0000 7fcb93b46800 1 --2- 172.21.15.50:0/3978824395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561628987970 0x561628a7f700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcb8401ce60 tx=0x7fcb84002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.370+0000 7fcb91e3f640 1 -- 172.21.15.50:0/3978824395 reap_dead start 2024-05-08T18:05:04.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.370+0000 7fcb93b46800 1 -- 172.21.15.50:0/3978824395 shutdown_connections 2024-05-08T18:05:04.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.370+0000 7fcb93b46800 1 -- 172.21.15.50:0/3978824395 >> 172.21.15.50:0/3978824395 conn(0x5616289802d0 msgr2=0x561628a71b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.370+0000 7fcb93b46800 1 -- 172.21.15.50:0/3978824395 shutdown_connections 2024-05-08T18:05:04.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.370+0000 7fcb93b46800 1 -- 172.21.15.50:0/3978824395 wait complete. 2024-05-08T18:05:04.384 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:04.385 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-05-08T18:05:04.385 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-05-08T18:05:04.385 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-05-08T18:05:04.430 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:04.430 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.433+0000 7f58d3726800 1 Processor -- start 2024-05-08T18:05:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.433+0000 7f58d3726800 1 -- start start 2024-05-08T18:05:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.433+0000 7f58d3726800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55d668aa2e30 0x55d668aa5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.433+0000 7f58d3726800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d668aa5760 con 0x55d668a9cbe0 2024-05-08T18:05:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.433+0000 7f58d3726800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55d66893ce20 con 0x55d668aa2e30 2024-05-08T18:05:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.433+0000 7f58ceb9f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55d668aa2e30 0x55d668aa5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.434+0000 7f58cf3a0640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55d668a9cbe0 0x55d668a9d000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:53086/0 (socket says 172.21.15.50:53086) 2024-05-08T18:05:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.434+0000 7f58cf3a0640 1 -- 172.21.15.50:0/3271914144 learned_addr learned my addr 172.21.15.50:0/3271914144 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:04.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.434+0000 7f58ce39e640 1 -- 172.21.15.50:0/3271914144 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 986665638 0 0) 0x55d668aa5760 con 0x55d668a9cbe0 2024-05-08T18:05:04.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.434+0000 7f58ce39e640 1 -- 172.21.15.50:0/3271914144 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58b8001e10 con 0x55d668a9cbe0 2024-05-08T18:05:04.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.434+0000 7f58ceb9f640 1 -- 172.21.15.50:0/3271914144 >> v1:172.21.15.46:6789/0 conn(0x55d668a9cbe0 legacy=0x55d668a9d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.434+0000 7f58ceb9f640 1 -- 172.21.15.50:0/3271914144 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d668aa6940 con 0x55d668aa2e30 2024-05-08T18:05:04.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.436+0000 7f58ceb9f640 1 --2- 172.21.15.50:0/3271914144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668aa2e30 0x55d668aa5220 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f58c400a470 tx=0x7f58c401cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24757fa32a47f6d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.436+0000 7f58ce39e640 1 -- 172.21.15.50:0/3271914144 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c4057070 con 0x55d668aa2e30 2024-05-08T18:05:04.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.436+0000 7f58ce39e640 1 -- 172.21.15.50:0/3271914144 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c4052570 con 0x55d668aa2e30 2024-05-08T18:05:04.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.436+0000 7f58ce39e640 1 -- 172.21.15.50:0/3271914144 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c4052870 con 0x55d668aa2e30 2024-05-08T18:05:04.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.436+0000 7f58d3726800 1 -- 172.21.15.50:0/3271914144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668aa2e30 msgr2=0x55d668aa5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.436+0000 7f58d3726800 1 --2- 172.21.15.50:0/3271914144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668aa2e30 0x55d668aa5220 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f58c400a470 tx=0x7f58c401cdb0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.436+0000 7f58d3726800 1 -- 172.21.15.50:0/3271914144 shutdown_connections 2024-05-08T18:05:04.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.436+0000 7f58d3726800 1 --2- 172.21.15.50:0/3271914144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668aa2e30 0x55d668aa5220 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.436+0000 7f58d3726800 1 -- 172.21.15.50:0/3271914144 >> 172.21.15.50:0/3271914144 conn(0x55d668a962d0 msgr2=0x55d668a986f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.436+0000 7f58d3726800 1 -- 172.21.15.50:0/3271914144 shutdown_connections 2024-05-08T18:05:04.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.436+0000 7f58d3726800 1 -- 172.21.15.50:0/3271914144 wait complete. 2024-05-08T18:05:04.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.451+0000 7f58d3726800 1 Processor -- start 2024-05-08T18:05:04.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.452+0000 7f58d3726800 1 -- start start 2024-05-08T18:05:04.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.452+0000 7f58d3726800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668a9cbe0 0x55d668b8fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.452+0000 7f58d3726800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d6689e7a10 con 0x55d668a9cbe0 2024-05-08T18:05:04.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.452+0000 7f58cf3a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668a9cbe0 0x55d668b8fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.452+0000 7f58cf3a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668a9cbe0 0x55d668b8fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58108/0 (socket says 172.21.15.50:58108) 2024-05-08T18:05:04.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.452+0000 7f58cf3a0640 1 -- 172.21.15.50:0/3516561412 learned_addr learned my addr 172.21.15.50:0/3516561412 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.452+0000 7f58cf3a0640 1 -- 172.21.15.50:0/3516561412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d668b900a0 con 0x55d668a9cbe0 2024-05-08T18:05:04.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.453+0000 7f58cf3a0640 1 --2- 172.21.15.50:0/3516561412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668a9cbe0 0x55d668b8fad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f58c0001e30 tx=0x7f58c000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7e5760ef3d9a85 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.453+0000 7f57bd60a640 1 -- 172.21.15.50:0/3516561412 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c0012020 con 0x55d668a9cbe0 2024-05-08T18:05:04.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.453+0000 7f57bd60a640 1 -- 172.21.15.50:0/3516561412 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c000e470 con 0x55d668a9cbe0 2024-05-08T18:05:04.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.453+0000 7f57bd60a640 1 -- 172.21.15.50:0/3516561412 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c000e790 con 0x55d668a9cbe0 2024-05-08T18:05:04.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.453+0000 7f58d3726800 1 -- 172.21.15.50:0/3516561412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668a9cbe0 msgr2=0x55d668b8fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.453+0000 7f58d3726800 1 --2- 172.21.15.50:0/3516561412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668a9cbe0 0x55d668b8fad0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f58c0001e30 tx=0x7f58c000b780 comp rx=0 tx=0).stop 2024-05-08T18:05:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.454+0000 7f58d3726800 1 -- 172.21.15.50:0/3516561412 shutdown_connections 2024-05-08T18:05:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.454+0000 7f58d3726800 1 --2- 172.21.15.50:0/3516561412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668a9cbe0 0x55d668b8fad0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.454+0000 7f58d3726800 1 -- 172.21.15.50:0/3516561412 >> 172.21.15.50:0/3516561412 conn(0x55d668a962d0 msgr2=0x55d668b8b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.454+0000 7f58d3726800 1 -- 172.21.15.50:0/3516561412 shutdown_connections 2024-05-08T18:05:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.454+0000 7f58d3726800 1 -- 172.21.15.50:0/3516561412 wait complete. 2024-05-08T18:05:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.454+0000 7f58d3726800 1 Processor -- start 2024-05-08T18:05:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.454+0000 7f58d3726800 1 -- start start 2024-05-08T18:05:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.454+0000 7f58d3726800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668a9cbe0 0x55d668b976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.454+0000 7f58d3726800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d668aa5c30 con 0x55d668a9cbe0 2024-05-08T18:05:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.454+0000 7f58cf3a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668a9cbe0 0x55d668b976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.454+0000 7f58cf3a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668a9cbe0 0x55d668b976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58120/0 (socket says 172.21.15.50:58120) 2024-05-08T18:05:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.454+0000 7f58cf3a0640 1 -- 172.21.15.50:0/904568313 learned_addr learned my addr 172.21.15.50:0/904568313 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.455+0000 7f58cf3a0640 1 -- 172.21.15.50:0/904568313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d668b97be0 con 0x55d668a9cbe0 2024-05-08T18:05:04.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.455+0000 7f58cf3a0640 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668a9cbe0 0x55d668b976a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f58c0001e30 tx=0x7f58c00040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.455+0000 7f57bbe07640 1 -- 172.21.15.50:0/904568313 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c0012030 con 0x55d668a9cbe0 2024-05-08T18:05:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.455+0000 7f57bbe07640 1 -- 172.21.15.50:0/904568313 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c001a7a0 con 0x55d668a9cbe0 2024-05-08T18:05:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.455+0000 7f58d3726800 1 -- 172.21.15.50:0/904568313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d668b97e70 con 0x55d668a9cbe0 2024-05-08T18:05:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.455+0000 7f57bbe07640 1 -- 172.21.15.50:0/904568313 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c001aaa0 con 0x55d668a9cbe0 2024-05-08T18:05:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.455+0000 7f58d3726800 1 -- 172.21.15.50:0/904568313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d668b9ab20 con 0x55d668a9cbe0 2024-05-08T18:05:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.456+0000 7f57bbe07640 1 -- 172.21.15.50:0/904568313 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f58c0015020 con 0x55d668a9cbe0 2024-05-08T18:05:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.456+0000 7f57bbe07640 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f57b403bba0 0x7f57b403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.456+0000 7f57bbe07640 1 -- 172.21.15.50:0/904568313 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f58c0020070 con 0x55d668a9cbe0 2024-05-08T18:05:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.456+0000 7f58ceb9f640 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f57b403bba0 0x7f57b403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.456+0000 7f58d3726800 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d668aa2e30 0x55d668b9e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.456+0000 7f58d3726800 1 -- 172.21.15.50:0/904568313 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668b9ee20 con 0x55d668aa2e30 2024-05-08T18:05:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.456+0000 7f58cfba1640 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d668aa2e30 0x55d668b9e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.457+0000 7f58ceb9f640 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f57b403bba0 0x7f57b403e060 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f58c400a470 tx=0x7f58c404e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.457+0000 7f58cfba1640 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d668aa2e30 0x55d668b9e8e0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.457+0000 7f58cfba1640 1 -- 172.21.15.50:0/904568313 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668aa2e30 2024-05-08T18:05:04.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.457+0000 7f58d3726800 1 -- 172.21.15.50:0/904568313 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668b9beb0 con 0x55d668aa2e30 2024-05-08T18:05:04.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.457+0000 7f58cfba1640 1 -- 172.21.15.50:0/904568313 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668aa2e30 2024-05-08T18:05:04.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.457+0000 7f58d3726800 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d668ba1180 0x55d668ba3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.457+0000 7f58d3726800 1 -- 172.21.15.50:0/904568313 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668ba3c60 con 0x55d668ba1180 2024-05-08T18:05:04.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.458+0000 7f58cf3a0640 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d668ba1180 0x55d668ba3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.458+0000 7f58cf3a0640 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d668ba1180 0x55d668ba3570 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.458+0000 7f58cf3a0640 1 -- 172.21.15.50:0/904568313 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f58c0018430 con 0x55d668ba1180 2024-05-08T18:05:04.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.458+0000 7f58d3726800 1 -- 172.21.15.50:0/904568313 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668ba0020 con 0x55d668aa2e30 2024-05-08T18:05:04.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.458+0000 7f58cfba1640 1 -- 172.21.15.50:0/904568313 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668aa2e30 2024-05-08T18:05:04.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.458+0000 7f58d3726800 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d668ba50c0 0x55d668ba74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.458+0000 7f58d3726800 1 -- 172.21.15.50:0/904568313 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668ba7bb0 con 0x55d668ba50c0 2024-05-08T18:05:04.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.459+0000 7f58ceb9f640 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d668ba50c0 0x55d668ba74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.459+0000 7f58ceb9f640 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d668ba50c0 0x55d668ba74b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.459+0000 7f58ceb9f640 1 -- 172.21.15.50:0/904568313 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f58c405a040 con 0x55d668ba50c0 2024-05-08T18:05:04.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.459+0000 7f58d3726800 1 Processor -- start 2024-05-08T18:05:04.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.460+0000 7f58d3726800 1 -- start start 2024-05-08T18:05:04.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.460+0000 7f58d3726800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bb5ea0 0x55d668bb62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.460+0000 7f58d3726800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d66898e330 con 0x55d668bb5ea0 2024-05-08T18:05:04.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.460+0000 7f58cf3a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bb5ea0 0x55d668bb62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.460+0000 7f58cf3a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bb5ea0 0x55d668bb62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58136/0 (socket says 172.21.15.50:58136) 2024-05-08T18:05:04.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.460+0000 7f58cf3a0640 1 -- 172.21.15.50:0/76728392 learned_addr learned my addr 172.21.15.50:0/76728392 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.460+0000 7f58cf3a0640 1 -- 172.21.15.50:0/76728392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d668bb6800 con 0x55d668bb5ea0 2024-05-08T18:05:04.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.460+0000 7f58cf3a0640 1 --2- 172.21.15.50:0/76728392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bb5ea0 0x55d668bb62c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f58c000b7c0 tx=0x7f58c0018c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.460+0000 7f57b9602640 1 -- 172.21.15.50:0/76728392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c004a170 con 0x55d668bb5ea0 2024-05-08T18:05:04.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.460+0000 7f57b9602640 1 -- 172.21.15.50:0/76728392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c0047070 con 0x55d668bb5ea0 2024-05-08T18:05:04.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.460+0000 7f58d3726800 1 -- 172.21.15.50:0/76728392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d668bb7270 con 0x55d668bb5ea0 2024-05-08T18:05:04.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.460+0000 7f57b9602640 1 -- 172.21.15.50:0/76728392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c004a440 con 0x55d668bb5ea0 2024-05-08T18:05:04.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.461+0000 7f58d3726800 1 -- 172.21.15.50:0/76728392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d668bb7270 con 0x55d668bb5ea0 2024-05-08T18:05:04.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.461+0000 7f57b9602640 1 -- 172.21.15.50:0/76728392 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f58c00163b0 con 0x55d668bb5ea0 2024-05-08T18:05:04.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.461+0000 7f57b9602640 1 --2- 172.21.15.50:0/76728392 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f57a803bb50 0x7f57a803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.461+0000 7f57b9602640 1 -- 172.21.15.50:0/76728392 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f58c0038d30 con 0x55d668bb5ea0 2024-05-08T18:05:04.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.462+0000 7f58ceb9f640 1 --2- 172.21.15.50:0/76728392 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f57a803bb50 0x7f57a803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.462+0000 7f58d3726800 1 Processor -- start 2024-05-08T18:05:04.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.462+0000 7f58d3726800 1 -- start start 2024-05-08T18:05:04.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.462+0000 7f58ceb9f640 1 --2- 172.21.15.50:0/76728392 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f57a803bb50 0x7f57a803e010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55d668b9f730 tx=0x7f58c4005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.462+0000 7f58d3726800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bc2e10 0x55d668bc3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.462+0000 7f58d3726800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d668b90420 con 0x55d668bc2e10 2024-05-08T18:05:04.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.462+0000 7f58cf3a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bc2e10 0x55d668bc3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.462+0000 7f58cf3a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bc2e10 0x55d668bc3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58150/0 (socket says 172.21.15.50:58150) 2024-05-08T18:05:04.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.462+0000 7f58cf3a0640 1 -- 172.21.15.50:0/3442548345 learned_addr learned my addr 172.21.15.50:0/3442548345 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.463+0000 7f58cf3a0640 1 -- 172.21.15.50:0/3442548345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d668bc3800 con 0x55d668bc2e10 2024-05-08T18:05:04.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.463+0000 7f58cf3a0640 1 --2- 172.21.15.50:0/3442548345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bc2e10 0x55d668bc3230 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f58c0018700 tx=0x7f58c004e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4af52c03209d7cc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.463+0000 7f57b2ffd640 1 -- 172.21.15.50:0/3442548345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c0025cc0 con 0x55d668bc2e10 2024-05-08T18:05:04.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.463+0000 7f57b2ffd640 1 -- 172.21.15.50:0/3442548345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c0049320 con 0x55d668bc2e10 2024-05-08T18:05:04.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.463+0000 7f57b2ffd640 1 -- 172.21.15.50:0/3442548345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c00514a0 con 0x55d668bc2e10 2024-05-08T18:05:04.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.463+0000 7f58d3726800 1 -- 172.21.15.50:0/3442548345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bc2e10 msgr2=0x55d668bc3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.463+0000 7f58d3726800 1 --2- 172.21.15.50:0/3442548345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bc2e10 0x55d668bc3230 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f58c0018700 tx=0x7f58c004e440 comp rx=0 tx=0).stop 2024-05-08T18:05:04.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.463+0000 7f58d3726800 1 -- 172.21.15.50:0/3442548345 shutdown_connections 2024-05-08T18:05:04.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.463+0000 7f58d3726800 1 --2- 172.21.15.50:0/3442548345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bc2e10 0x55d668bc3230 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.463+0000 7f58d3726800 1 -- 172.21.15.50:0/3442548345 >> 172.21.15.50:0/3442548345 conn(0x55d668bbf170 msgr2=0x55d668bbd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.463+0000 7f58d3726800 1 -- 172.21.15.50:0/3442548345 shutdown_connections 2024-05-08T18:05:04.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.463+0000 7f58d3726800 1 -- 172.21.15.50:0/3442548345 wait complete. 2024-05-08T18:05:04.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.464+0000 7f58d3726800 1 Processor -- start 2024-05-08T18:05:04.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.464+0000 7f58d3726800 1 -- start start 2024-05-08T18:05:04.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.464+0000 7f58d3726800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bc2e10 0x55d668bca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.464+0000 7f58d3726800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d668bc3eb0 con 0x55d668bc2e10 2024-05-08T18:05:04.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.464+0000 7f58cf3a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bc2e10 0x55d668bca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.464+0000 7f58cf3a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bc2e10 0x55d668bca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58156/0 (socket says 172.21.15.50:58156) 2024-05-08T18:05:04.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.464+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 learned_addr learned my addr 172.21.15.50:0/2279819261 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.464+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d668bca710 con 0x55d668bc2e10 2024-05-08T18:05:04.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.465+0000 7f58cf3a0640 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bc2e10 0x55d668bca1d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f58c00494d0 tx=0x7f58c00388f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.465+0000 7f57b17fa640 1 -- 172.21.15.50:0/2279819261 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c006c070 con 0x55d668bc2e10 2024-05-08T18:05:04.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.465+0000 7f57b17fa640 1 -- 172.21.15.50:0/2279819261 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c002cd00 con 0x55d668bc2e10 2024-05-08T18:05:04.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.465+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d668bca710 con 0x55d668bc2e10 2024-05-08T18:05:04.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.465+0000 7f57b17fa640 1 -- 172.21.15.50:0/2279819261 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f58c000d5f0 con 0x55d668bc2e10 2024-05-08T18:05:04.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.465+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d668bca710 con 0x55d668bc2e10 2024-05-08T18:05:04.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.466+0000 7f57b17fa640 1 -- 172.21.15.50:0/2279819261 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f58c006f070 con 0x55d668bc2e10 2024-05-08T18:05:04.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.466+0000 7f57b17fa640 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f579803bba0 0x7f579803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.466+0000 7f57b17fa640 1 -- 172.21.15.50:0/2279819261 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f58c0062070 con 0x55d668bc2e10 2024-05-08T18:05:04.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.466+0000 7f58ceb9f640 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f579803bba0 0x7f579803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.466+0000 7f58d3726800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.466+0000 7f58d3726800 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d668bd5910 0x55d668bd7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.466+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd8520 con 0x55d668bd5910 2024-05-08T18:05:04.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.466+0000 7f58cfba1640 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d668bd5910 0x55d668bd7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.467+0000 7f58ceb9f640 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f579803bba0 0x7f579803e060 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55d668ba7fb0 tx=0x7f58c404f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.467+0000 7f58cfba1640 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d668bd5910 0x55d668bd7dd0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.467+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668bd5910 2024-05-08T18:05:04.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.467+0000 7f58d3726800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.467+0000 7f58d3726800 20 realm 2024-05-08T18:05:04.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.467+0000 7f58d3726800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.467+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd8930 con 0x55d668bd5910 2024-05-08T18:05:04.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.467+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668bd5910 2024-05-08T18:05:04.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.467+0000 7f58d3726800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.467+0000 7f58d3726800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:04.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.467+0000 7f58d3726800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.467+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd9200 con 0x55d668bd5910 2024-05-08T18:05:04.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.467+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668bd5910 2024-05-08T18:05:04.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.467+0000 7f58d3726800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.467+0000 7f58d3726800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.467+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bd5910 2024-05-08T18:05:04.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.468+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668bd5910 2024-05-08T18:05:04.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.468+0000 7f58d3726800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:04.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.468+0000 7f58d3726800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.468+0000 7f58d3726800 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d668bda090 0x55d668bdc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.468+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668bdcbe0 con 0x55d668bda090 2024-05-08T18:05:04.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.468+0000 7f58cf3a0640 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d668bda090 0x55d668bdc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.469+0000 7f58cf3a0640 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d668bda090 0x55d668bdc480 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.469+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.469+0000 7f58d3726800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:04.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.469+0000 7f58d3726800 20 searching for the correct realm 2024-05-08T18:05:04.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.469+0000 7f58d3726800 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d668bdec80 0x55d668be1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.469+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668be1810 con 0x55d668bdec80 2024-05-08T18:05:04.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.469+0000 7f58ceb9f640 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d668bdec80 0x55d668be1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.469+0000 7f58ceb9f640 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d668bdec80 0x55d668be1140 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.470+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c4070030 con 0x55d668bdec80 2024-05-08T18:05:04.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.470+0000 7f58ceb9f640 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f58c4083040 0x7f58c4075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.470+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668be1810 con 0x7f58c4083040 2024-05-08T18:05:04.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.470+0000 7f58cfba1640 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f58c4083040 0x7f58c4075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.470+0000 7f58cfba1640 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f58c4083040 0x7f58c4075430 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.470+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c800a4a0 con 0x7f58c4083040 2024-05-08T18:05:04.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.470+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bda090 2024-05-08T18:05:04.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.471+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.471+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bd5910 2024-05-08T18:05:04.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.471+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668bd5910 2024-05-08T18:05:04.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.471+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bd5910 2024-05-08T18:05:04.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.471+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668bd5910 2024-05-08T18:05:04.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.471+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bda090 2024-05-08T18:05:04.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.472+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.472+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bda090 2024-05-08T18:05:04.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.472+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.472+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x7f58c4083040 2024-05-08T18:05:04.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.472+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c800a4a0 con 0x7f58c4083040 2024-05-08T18:05:04.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.472+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bdec80 2024-05-08T18:05:04.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.473+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c4070030 con 0x55d668bdec80 2024-05-08T18:05:04.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.473+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bd5910 2024-05-08T18:05:04.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.473+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668bd5910 2024-05-08T18:05:04.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.473+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bda090 2024-05-08T18:05:04.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.473+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.473+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bd5910 2024-05-08T18:05:04.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.473+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668bd5910 2024-05-08T18:05:04.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.473+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x7f58c4083040 2024-05-08T18:05:04.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.474+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f58c800a4a0 con 0x7f58c4083040 2024-05-08T18:05:04.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.474+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bdec80 2024-05-08T18:05:04.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.474+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c4070030 con 0x55d668bdec80 2024-05-08T18:05:04.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.474+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bdec80 2024-05-08T18:05:04.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.474+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c4070030 con 0x55d668bdec80 2024-05-08T18:05:04.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.474+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bda090 2024-05-08T18:05:04.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.474+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.475+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bdec80 2024-05-08T18:05:04.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.475+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c4070030 con 0x55d668bdec80 2024-05-08T18:05:04.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.475+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bdec80 2024-05-08T18:05:04.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.475+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f58c4070030 con 0x55d668bdec80 2024-05-08T18:05:04.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.475+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bd5910 2024-05-08T18:05:04.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.475+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668bd5910 2024-05-08T18:05:04.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.475+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bd5910 2024-05-08T18:05:04.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.475+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668bd5910 2024-05-08T18:05:04.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.475+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bda090 2024-05-08T18:05:04.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.476+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.476+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bdec80 2024-05-08T18:05:04.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.476+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c4070030 con 0x55d668bdec80 2024-05-08T18:05:04.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.476+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x7f58c4083040 2024-05-08T18:05:04.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.476+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c800a4a0 con 0x7f58c4083040 2024-05-08T18:05:04.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.476+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bd5910 2024-05-08T18:05:04.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.476+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668bd5910 2024-05-08T18:05:04.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.476+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bda090 2024-05-08T18:05:04.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.477+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.477+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bdec80 2024-05-08T18:05:04.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.477+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c4070030 con 0x55d668bdec80 2024-05-08T18:05:04.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.477+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bda090 2024-05-08T18:05:04.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.477+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.477+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x7f58c4083040 2024-05-08T18:05:04.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.478+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c800a4a0 con 0x7f58c4083040 2024-05-08T18:05:04.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.478+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bdec80 2024-05-08T18:05:04.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.478+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c4070030 con 0x55d668bdec80 2024-05-08T18:05:04.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.478+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bda090 2024-05-08T18:05:04.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.478+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.478+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x55d668bd5910 2024-05-08T18:05:04.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668bd5910 2024-05-08T18:05:04.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d668bd95a0 con 0x7f58c4083040 2024-05-08T18:05:04.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f58c800a4a0 con 0x7f58c4083040 2024-05-08T18:05:04.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58d3726800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:04.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58d3726800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:04.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58d3726800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:04.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58d3726800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:04.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58d3726800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:04.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58d3726800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:04.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58d3726800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668bdd580 con 0x55d668bd5910 2024-05-08T18:05:04.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668bd5910 2024-05-08T18:05:04.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58d3726800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58d3726800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668bdd580 con 0x55d668bd5910 2024-05-08T18:05:04.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f58c800a4a0 con 0x55d668bd5910 2024-05-08T18:05:04.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58d3726800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58d3726800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.479+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668bdd530 con 0x55d668bdec80 2024-05-08T18:05:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.480+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f58c4070030 con 0x55d668bdec80 2024-05-08T18:05:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.480+0000 7f58d3726800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.480+0000 7f58d3726800 20 zone default found 2024-05-08T18:05:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.480+0000 7f58d3726800 4 Realm: () 2024-05-08T18:05:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.480+0000 7f58d3726800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.480+0000 7f58d3726800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.480+0000 7f58d3726800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.480+0000 7f58d3726800 20 zonegroup default 2024-05-08T18:05:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.480+0000 7f58d3726800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.480+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668bdde20 con 0x55d668bda090 2024-05-08T18:05:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.480+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.480+0000 7f58d3726800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.480+0000 7f58d3726800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.480+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668be3670 con 0x55d668bdec80 2024-05-08T18:05:04.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.481+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f58c4070030 con 0x55d668bdec80 2024-05-08T18:05:04.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.481+0000 7f58d3726800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.481+0000 7f58d3726800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.481+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668be3a10 con 0x55d668bdec80 2024-05-08T18:05:04.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.481+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f58c4070030 con 0x55d668bdec80 2024-05-08T18:05:04.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.481+0000 7f58d3726800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.481+0000 7f58d3726800 20 started sync module instance, tier type = 2024-05-08T18:05:04.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.481+0000 7f58d3726800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:04.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.481+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be2b10 con 0x7f58c4083040 2024-05-08T18:05:04.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.482+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c800a4a0 con 0x7f58c4083040 2024-05-08T18:05:04.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.482+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94379368656912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be2b10 con 0x7f58c4083040 2024-05-08T18:05:04.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.482+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be44c0 con 0x55d668bd5910 2024-05-08T18:05:04.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.483+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c8040040 con 0x55d668bd5910 2024-05-08T18:05:04.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.483+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94379368656912] v27'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c8040040 con 0x7f58c4083040 2024-05-08T18:05:04.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.483+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94379368660896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be44c0 con 0x55d668bd5910 2024-05-08T18:05:04.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.483+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be2680 con 0x55d668bdec80 2024-05-08T18:05:04.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.484+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94379368660896] v27'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c8040040 con 0x55d668bd5910 2024-05-08T18:05:04.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.484+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c4070030 con 0x55d668bdec80 2024-05-08T18:05:04.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.484+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94379368663216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be5460 con 0x55d668bdec80 2024-05-08T18:05:04.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.484+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be5e20 con 0x7f58c4083040 2024-05-08T18:05:04.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.485+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94379368663216] v27'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c4070030 con 0x55d668bdec80 2024-05-08T18:05:04.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.485+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c8040040 con 0x7f58c4083040 2024-05-08T18:05:04.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.485+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94379368667504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be6240 con 0x7f58c4083040 2024-05-08T18:05:04.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.486+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be6c00 con 0x55d668bd5910 2024-05-08T18:05:04.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.486+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c8040040 con 0x55d668bd5910 2024-05-08T18:05:04.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.486+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94379368671056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be7020 con 0x55d668bd5910 2024-05-08T18:05:04.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.486+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be7aa0 con 0x55d668bda090 2024-05-08T18:05:04.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.487+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94379368667504] v27'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c8040040 con 0x7f58c4083040 2024-05-08T18:05:04.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.488+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.488+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94379368671056] v27'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c8040040 con 0x55d668bd5910 2024-05-08T18:05:04.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.488+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94379368674704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be7e90 con 0x55d668bda090 2024-05-08T18:05:04.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.488+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be6b70 con 0x55d668bda090 2024-05-08T18:05:04.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.489+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.489+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94379368678576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be8f00 con 0x55d668bda090 2024-05-08T18:05:04.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.489+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be99f0 con 0x55d668bda090 2024-05-08T18:05:04.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.489+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94379368674704] v27'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.489+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v27'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.489+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94379368682688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be99b0 con 0x55d668bda090 2024-05-08T18:05:04.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.490+0000 7f58d3726800 20 add_watcher() i=0 2024-05-08T18:05:04.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.490+0000 7f58d3726800 20 add_watcher() i=1 2024-05-08T18:05:04.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.490+0000 7f58d3726800 20 add_watcher() i=2 2024-05-08T18:05:04.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.490+0000 7f58d3726800 20 add_watcher() i=3 2024-05-08T18:05:04.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.490+0000 7f58d3726800 20 add_watcher() i=4 2024-05-08T18:05:04.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.490+0000 7f58d3726800 20 add_watcher() i=5 2024-05-08T18:05:04.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.490+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94379368678576] v27'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.490+0000 7f58d3726800 20 add_watcher() i=6 2024-05-08T18:05:04.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.491+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94379368682688] v27'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.491+0000 7f58d3726800 20 add_watcher() i=7 2024-05-08T18:05:04.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.491+0000 7f58d3726800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:04.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.491+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668be8990 con 0x55d668bd5910 2024-05-08T18:05:04.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.491+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f58c8040040 con 0x55d668bd5910 2024-05-08T18:05:04.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.491+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94379368668560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be8950 con 0x55d668bd5910 2024-05-08T18:05:04.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.492+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94379368668560] v27'58 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f58c8040040 con 0x55d668bd5910 2024-05-08T18:05:04.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.492+0000 7f5785fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:04.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.492+0000 7f58d3726800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:04.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.492+0000 7f58d3726800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:04.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.492+0000 7f58d3726800 5 note: GC not initialized 2024-05-08T18:05:04.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.493+0000 7f5784fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:04.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.493+0000 7f58d3726800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:04.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.493+0000 7f58d3726800 20 Filter name: none 2024-05-08T18:05:04.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.493+0000 7f56fe7fc640 20 reqs_thread_entry: start 2024-05-08T18:05:04.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.493+0000 7f58d3726800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.493+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d668c1f970 con 0x55d668bd5910 2024-05-08T18:05:04.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.493+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f58c8040040 con 0x55d668bd5910 2024-05-08T18:05:04.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.493+0000 7f58d3726800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:04.501 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":1,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:04.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.494+0000 7f5785fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:04.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.494+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94379368668560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668c1e8c0 con 0x55d668bd5910 2024-05-08T18:05:04.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.495+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94379368668560] v27'59 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f58c8040040 con 0x55d668bd5910 2024-05-08T18:05:04.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.495+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94379368656912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be5d90 con 0x7f58c4083040 2024-05-08T18:05:04.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.496+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94379368656912] v27'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c8040040 con 0x7f58c4083040 2024-05-08T18:05:04.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.496+0000 7f58d3726800 20 remove_watcher() i=0 2024-05-08T18:05:04.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.496+0000 7f58d3726800 2 removed watcher, disabling cache 2024-05-08T18:05:04.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.496+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94379368660896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be5d90 con 0x55d668bd5910 2024-05-08T18:05:04.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.497+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94379368660896] v27'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c8040040 con 0x55d668bd5910 2024-05-08T18:05:04.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.497+0000 7f58d3726800 20 remove_watcher() i=1 2024-05-08T18:05:04.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.497+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94379368663216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be3fa0 con 0x55d668bdec80 2024-05-08T18:05:04.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.498+0000 7f58ceb9f640 1 -- 172.21.15.50:0/2279819261 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94379368663216] v27'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c4070030 con 0x55d668bdec80 2024-05-08T18:05:04.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.498+0000 7f58d3726800 20 remove_watcher() i=2 2024-05-08T18:05:04.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.498+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94379368667504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be5d90 con 0x7f58c4083040 2024-05-08T18:05:04.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.500+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94379368667504] v27'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c8040040 con 0x7f58c4083040 2024-05-08T18:05:04.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.500+0000 7f58d3726800 20 remove_watcher() i=3 2024-05-08T18:05:04.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.500+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94379368671056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be48b0 con 0x55d668bd5910 2024-05-08T18:05:04.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.501+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94379368671056] v27'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c8040040 con 0x55d668bd5910 2024-05-08T18:05:04.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.501+0000 7f58d3726800 20 remove_watcher() i=4 2024-05-08T18:05:04.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.501+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94379368674704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be6750 con 0x55d668bda090 2024-05-08T18:05:04.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.501+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94379368674704] v27'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.502+0000 7f58d3726800 20 remove_watcher() i=5 2024-05-08T18:05:04.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.502+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94379368678576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be48b0 con 0x55d668bda090 2024-05-08T18:05:04.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.502+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94379368678576] v27'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.503+0000 7f58d3726800 20 remove_watcher() i=6 2024-05-08T18:05:04.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.503+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94379368682688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d668be6750 con 0x55d668bda090 2024-05-08T18:05:04.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58cf3a0640 1 -- 172.21.15.50:0/2279819261 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94379368682688] v27'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f58c0078070 con 0x55d668bda090 2024-05-08T18:05:04.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 20 remove_watcher() i=7 2024-05-08T18:05:04.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d668bda090 msgr2=0x55d668bdc480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d668bda090 0x55d668bdc480 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d668bd5910 msgr2=0x55d668bd7dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d668bd5910 0x55d668bd7dd0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f58c4083040 msgr2=0x7f58c4075430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f58c4083040 0x7f58c4075430 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d668bdec80 msgr2=0x55d668be1140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d668bdec80 0x55d668be1140 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f579803bba0 msgr2=0x7f579803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f579803bba0 0x7f579803e060 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55d668ba7fb0 tx=0x7f58c404f000 comp rx=0 tx=0).stop 2024-05-08T18:05:04.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bc2e10 msgr2=0x55d668bca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58cfba1640 1 -- 172.21.15.50:0/2279819261 reap_dead start 2024-05-08T18:05:04.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bc2e10 0x55d668bca1d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f58c00494d0 tx=0x7f58c00388f0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 shutdown_connections 2024-05-08T18:05:04.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 --2- 172.21.15.50:0/2279819261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bc2e10 0x55d668bca1d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 >> 172.21.15.50:0/2279819261 conn(0x55d668bbf170 msgr2=0x55d668bbc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 shutdown_connections 2024-05-08T18:05:04.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.504+0000 7f58d3726800 1 -- 172.21.15.50:0/2279819261 wait complete. 2024-05-08T18:05:04.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 -- 172.21.15.50:0/76728392 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f57a803bb50 msgr2=0x7f57a803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 --2- 172.21.15.50:0/76728392 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f57a803bb50 0x7f57a803e010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55d668b9f730 tx=0x7f58c4005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:04.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 -- 172.21.15.50:0/76728392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bb5ea0 msgr2=0x55d668bb62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 --2- 172.21.15.50:0/76728392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bb5ea0 0x55d668bb62c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f58c000b7c0 tx=0x7f58c0018c20 comp rx=0 tx=0).stop 2024-05-08T18:05:04.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 -- 172.21.15.50:0/76728392 shutdown_connections 2024-05-08T18:05:04.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 --2- 172.21.15.50:0/76728392 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f57a803bb50 0x7f57a803e010 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 --2- 172.21.15.50:0/76728392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668bb5ea0 0x55d668bb62c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 -- 172.21.15.50:0/76728392 >> 172.21.15.50:0/76728392 conn(0x55d668ba9f40 msgr2=0x55d668baa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 -- 172.21.15.50:0/76728392 shutdown_connections 2024-05-08T18:05:04.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 -- 172.21.15.50:0/76728392 wait complete. 2024-05-08T18:05:04.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 -- 172.21.15.50:0/904568313 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d668ba1180 msgr2=0x55d668ba3570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d668ba1180 0x55d668ba3570 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 -- 172.21.15.50:0/904568313 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d668aa2e30 msgr2=0x55d668b9e8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d668aa2e30 0x55d668b9e8e0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 -- 172.21.15.50:0/904568313 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d668ba50c0 msgr2=0x55d668ba74b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d668ba50c0 0x55d668ba74b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 -- 172.21.15.50:0/904568313 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f57b403bba0 msgr2=0x7f57b403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f57b403bba0 0x7f57b403e060 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f58c400a470 tx=0x7f58c404e000 comp rx=0 tx=0).stop 2024-05-08T18:05:04.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 -- 172.21.15.50:0/904568313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668a9cbe0 msgr2=0x55d668b976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 --2- 172.21.15.50:0/904568313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d668a9cbe0 0x55d668b976a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f58c0001e30 tx=0x7f58c00040e0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58cfba1640 1 -- 172.21.15.50:0/904568313 reap_dead start 2024-05-08T18:05:04.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 -- 172.21.15.50:0/904568313 shutdown_connections 2024-05-08T18:05:04.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 -- 172.21.15.50:0/904568313 >> 172.21.15.50:0/904568313 conn(0x55d668a962d0 msgr2=0x55d668b89bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 -- 172.21.15.50:0/904568313 shutdown_connections 2024-05-08T18:05:04.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.505+0000 7f58d3726800 1 -- 172.21.15.50:0/904568313 wait complete. 2024-05-08T18:05:04.513 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:04.514 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2024-05-08T18:05:04.514 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2024-05-08T18:05:04.514 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2024-05-08T18:05:04.559 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:04.560 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:04.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.562+0000 7f14f3040800 1 Processor -- start 2024-05-08T18:05:04.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.562+0000 7f14f3040800 1 -- start start 2024-05-08T18:05:04.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.562+0000 7f14f3040800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5577898c0700 0x557789950210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.562+0000 7f14f3040800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5577899544d0 con 0x557789950780 2024-05-08T18:05:04.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.562+0000 7f14f3040800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5577897ede20 con 0x5577898c0700 2024-05-08T18:05:04.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.563+0000 7f14edcd5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5577898c0700 0x557789950210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.563+0000 7f14ed4d4640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x557789950780 0x557789952c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:53092/0 (socket says 172.21.15.50:53092) 2024-05-08T18:05:04.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.563+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3482378534 learned_addr learned my addr 172.21.15.50:0/3482378534 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:04.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.563+0000 7f14edcd5640 1 -- 172.21.15.50:0/3482378534 >> v1:172.21.15.46:6789/0 conn(0x557789950780 legacy=0x557789952c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.563+0000 7f14edcd5640 1 -- 172.21.15.50:0/3482378534 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x557789955700 con 0x5577898c0700 2024-05-08T18:05:04.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.565+0000 7f14edcd5640 1 --2- 172.21.15.50:0/3482378534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 0x557789950210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f14e000a470 tx=0x7f14e001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c1042fe2f39625 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.565+0000 7f14eccd3640 1 -- 172.21.15.50:0/3482378534 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e0057070 con 0x5577898c0700 2024-05-08T18:05:04.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.565+0000 7f14eccd3640 1 -- 172.21.15.50:0/3482378534 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e0052570 con 0x5577898c0700 2024-05-08T18:05:04.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.565+0000 7f14eccd3640 1 -- 172.21.15.50:0/3482378534 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e0052870 con 0x5577898c0700 2024-05-08T18:05:04.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.566+0000 7f14f3040800 1 -- 172.21.15.50:0/3482378534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 msgr2=0x557789950210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.566+0000 7f14f3040800 1 --2- 172.21.15.50:0/3482378534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 0x557789950210 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f14e000a470 tx=0x7f14e001ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:04.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.566+0000 7f14f3040800 1 -- 172.21.15.50:0/3482378534 shutdown_connections 2024-05-08T18:05:04.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.566+0000 7f14f3040800 1 --2- 172.21.15.50:0/3482378534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 0x557789950210 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.566+0000 7f14f3040800 1 -- 172.21.15.50:0/3482378534 >> 172.21.15.50:0/3482378534 conn(0x557789947b50 msgr2=0x557789949f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.566+0000 7f14f3040800 1 -- 172.21.15.50:0/3482378534 shutdown_connections 2024-05-08T18:05:04.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.566+0000 7f14f3040800 1 -- 172.21.15.50:0/3482378534 wait complete. 2024-05-08T18:05:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.581+0000 7f14f3040800 1 Processor -- start 2024-05-08T18:05:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.581+0000 7f14f3040800 1 -- start start 2024-05-08T18:05:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.581+0000 7f14f3040800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 0x55778994db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.581+0000 7f14f3040800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557789898a10 con 0x5577898c0700 2024-05-08T18:05:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.581+0000 7f14edcd5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 0x55778994db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.581+0000 7f14edcd5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 0x55778994db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58168/0 (socket says 172.21.15.50:58168) 2024-05-08T18:05:04.580 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.581+0000 7f14edcd5640 1 -- 172.21.15.50:0/3561266640 learned_addr learned my addr 172.21.15.50:0/3561266640 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.582+0000 7f14edcd5640 1 -- 172.21.15.50:0/3561266640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55778994c100 con 0x5577898c0700 2024-05-08T18:05:04.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.582+0000 7f14edcd5640 1 --2- 172.21.15.50:0/3561266640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 0x55778994db40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f14e0055040 tx=0x7f14e004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c178039d568db952 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.582+0000 7f13dbe07640 1 -- 172.21.15.50:0/3561266640 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e0057070 con 0x5577898c0700 2024-05-08T18:05:04.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.582+0000 7f13dbe07640 1 -- 172.21.15.50:0/3561266640 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e0063070 con 0x5577898c0700 2024-05-08T18:05:04.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.582+0000 7f13dbe07640 1 -- 172.21.15.50:0/3561266640 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e0052720 con 0x5577898c0700 2024-05-08T18:05:04.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.583+0000 7f14f3040800 1 -- 172.21.15.50:0/3561266640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 msgr2=0x55778994db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.583+0000 7f14f3040800 1 --2- 172.21.15.50:0/3561266640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 0x55778994db40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f14e0055040 tx=0x7f14e004c000 comp rx=0 tx=0).stop 2024-05-08T18:05:04.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.583+0000 7f14f3040800 1 -- 172.21.15.50:0/3561266640 shutdown_connections 2024-05-08T18:05:04.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.583+0000 7f14f3040800 1 --2- 172.21.15.50:0/3561266640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 0x55778994db40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.583+0000 7f14f3040800 1 -- 172.21.15.50:0/3561266640 >> 172.21.15.50:0/3561266640 conn(0x557789947b50 msgr2=0x557789a3a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.583+0000 7f14f3040800 1 -- 172.21.15.50:0/3561266640 shutdown_connections 2024-05-08T18:05:04.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.583+0000 7f14f3040800 1 -- 172.21.15.50:0/3561266640 wait complete. 2024-05-08T18:05:04.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.583+0000 7f14f3040800 1 Processor -- start 2024-05-08T18:05:04.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.583+0000 7f14f3040800 1 -- start start 2024-05-08T18:05:04.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.583+0000 7f14f3040800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 0x557789a487e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.583+0000 7f14f3040800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557789954940 con 0x5577898c0700 2024-05-08T18:05:04.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.584+0000 7f14edcd5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 0x557789a487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.584+0000 7f14edcd5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 0x557789a487e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58184/0 (socket says 172.21.15.50:58184) 2024-05-08T18:05:04.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.584+0000 7f14edcd5640 1 -- 172.21.15.50:0/3544177276 learned_addr learned my addr 172.21.15.50:0/3544177276 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.584+0000 7f14edcd5640 1 -- 172.21.15.50:0/3544177276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557789a48d20 con 0x5577898c0700 2024-05-08T18:05:04.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.584+0000 7f14edcd5640 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 0x557789a487e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f14e0060040 tx=0x7f14e0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.584+0000 7f13da604640 1 -- 172.21.15.50:0/3544177276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e0052e30 con 0x5577898c0700 2024-05-08T18:05:04.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.585+0000 7f13da604640 1 -- 172.21.15.50:0/3544177276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e006c070 con 0x5577898c0700 2024-05-08T18:05:04.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.585+0000 7f13da604640 1 -- 172.21.15.50:0/3544177276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e005e5f0 con 0x5577898c0700 2024-05-08T18:05:04.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.585+0000 7f14f3040800 1 -- 172.21.15.50:0/3544177276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557789a48fb0 con 0x5577898c0700 2024-05-08T18:05:04.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.585+0000 7f14f3040800 1 -- 172.21.15.50:0/3544177276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557789a4bc60 con 0x5577898c0700 2024-05-08T18:05:04.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.586+0000 7f13da604640 1 -- 172.21.15.50:0/3544177276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f14e005e790 con 0x5577898c0700 2024-05-08T18:05:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.586+0000 7f13da604640 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f13d403bba0 0x7f13d403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.586+0000 7f13da604640 1 -- 172.21.15.50:0/3544177276 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f14e0094e20 con 0x5577898c0700 2024-05-08T18:05:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.586+0000 7f14ed4d4640 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f13d403bba0 0x7f13d403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.586+0000 7f14f3040800 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557789950780 0x557789a4fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.586+0000 7f14f3040800 1 -- 172.21.15.50:0/3544177276 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a4ff60 con 0x557789950780 2024-05-08T18:05:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.586+0000 7f14ee4d6640 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557789950780 0x557789a4fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.586+0000 7f14ed4d4640 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f13d403bba0 0x7f13d403e060 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f14e4001d70 tx=0x7f14e40012d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.587+0000 7f14ee4d6640 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557789950780 0x557789a4fa20 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.587+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3544177276 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789950780 2024-05-08T18:05:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.587+0000 7f14f3040800 1 -- 172.21.15.50:0/3544177276 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a507e0 con 0x557789950780 2024-05-08T18:05:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.587+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3544177276 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789950780 2024-05-08T18:05:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.587+0000 7f14f3040800 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557789a520f0 0x557789a54590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.587+0000 7f14f3040800 1 -- 172.21.15.50:0/3544177276 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a54d20 con 0x557789a520f0 2024-05-08T18:05:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.588+0000 7f14edcd5640 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557789a520f0 0x557789a54590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.588+0000 7f14edcd5640 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557789a520f0 0x557789a54590 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.588+0000 7f14edcd5640 1 -- 172.21.15.50:0/3544177276 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f14e0009b60 con 0x557789a520f0 2024-05-08T18:05:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.588+0000 7f14f3040800 1 -- 172.21.15.50:0/3544177276 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a51030 con 0x557789950780 2024-05-08T18:05:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.589+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3544177276 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789950780 2024-05-08T18:05:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.589+0000 7f14f3040800 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557789a56180 0x557789a58570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.589+0000 7f14f3040800 1 -- 172.21.15.50:0/3544177276 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a58cb0 con 0x557789a56180 2024-05-08T18:05:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.589+0000 7f14ed4d4640 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557789a56180 0x557789a58570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.589+0000 7f14ed4d4640 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557789a56180 0x557789a58570 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.589+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3544177276 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f14e4009050 con 0x557789a56180 2024-05-08T18:05:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.590+0000 7f14f3040800 1 Processor -- start 2024-05-08T18:05:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.590+0000 7f14f3040800 1 -- start start 2024-05-08T18:05:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.590+0000 7f14f3040800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a66fe0 0x557789a67400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.590+0000 7f14f3040800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55778983f330 con 0x557789a66fe0 2024-05-08T18:05:04.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.590+0000 7f14edcd5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a66fe0 0x557789a67400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.590+0000 7f14edcd5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a66fe0 0x557789a67400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58192/0 (socket says 172.21.15.50:58192) 2024-05-08T18:05:04.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.590+0000 7f14edcd5640 1 -- 172.21.15.50:0/2703806367 learned_addr learned my addr 172.21.15.50:0/2703806367 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.590+0000 7f14edcd5640 1 -- 172.21.15.50:0/2703806367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557789a67940 con 0x557789a66fe0 2024-05-08T18:05:04.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.591+0000 7f14edcd5640 1 --2- 172.21.15.50:0/2703806367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a66fe0 0x557789a67400 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f14e00931b0 tx=0x7f14e005c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.591+0000 7f13d37fe640 1 -- 172.21.15.50:0/2703806367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e00095d0 con 0x557789a66fe0 2024-05-08T18:05:04.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.591+0000 7f13d37fe640 1 -- 172.21.15.50:0/2703806367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e0058040 con 0x557789a66fe0 2024-05-08T18:05:04.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.591+0000 7f14f3040800 1 -- 172.21.15.50:0/2703806367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557789a683b0 con 0x557789a66fe0 2024-05-08T18:05:04.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.591+0000 7f13d37fe640 1 -- 172.21.15.50:0/2703806367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e00098c0 con 0x557789a66fe0 2024-05-08T18:05:04.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.591+0000 7f14f3040800 1 -- 172.21.15.50:0/2703806367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557789a688b0 con 0x557789a66fe0 2024-05-08T18:05:04.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.592+0000 7f13d37fe640 1 -- 172.21.15.50:0/2703806367 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f14e0071990 con 0x557789a66fe0 2024-05-08T18:05:04.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.592+0000 7f13d37fe640 1 --2- 172.21.15.50:0/2703806367 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f13c803bb50 0x7f13c803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.592+0000 7f13d37fe640 1 -- 172.21.15.50:0/2703806367 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f14e0091c50 con 0x557789a66fe0 2024-05-08T18:05:04.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.592+0000 7f14ed4d4640 1 --2- 172.21.15.50:0/2703806367 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f13c803bb50 0x7f13c803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.592+0000 7f14f3040800 1 Processor -- start 2024-05-08T18:05:04.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.592+0000 7f14f3040800 1 -- start start 2024-05-08T18:05:04.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.592+0000 7f14ed4d4640 1 --2- 172.21.15.50:0/2703806367 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f13c803bb50 0x7f13c803e010 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f14e400bf40 tx=0x7f14e400f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.592+0000 7f14f3040800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a741a0 0x557789a745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.592+0000 7f14f3040800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55778994c3c0 con 0x557789a741a0 2024-05-08T18:05:04.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.593+0000 7f14edcd5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a741a0 0x557789a745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.593+0000 7f14edcd5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a741a0 0x557789a745c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58202/0 (socket says 172.21.15.50:58202) 2024-05-08T18:05:04.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.593+0000 7f14edcd5640 1 -- 172.21.15.50:0/48862533 learned_addr learned my addr 172.21.15.50:0/48862533 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.593+0000 7f14edcd5640 1 -- 172.21.15.50:0/48862533 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557789a74b90 con 0x557789a741a0 2024-05-08T18:05:04.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.593+0000 7f14edcd5640 1 --2- 172.21.15.50:0/48862533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a741a0 0x557789a745c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f14e0088fd0 tx=0x7f14e009b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1714bc6248c418 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.593+0000 7f13d17fa640 1 -- 172.21.15.50:0/48862533 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e0067d50 con 0x557789a741a0 2024-05-08T18:05:04.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.593+0000 7f13d17fa640 1 -- 172.21.15.50:0/48862533 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e007a3f0 con 0x557789a741a0 2024-05-08T18:05:04.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.593+0000 7f13d17fa640 1 -- 172.21.15.50:0/48862533 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e0074120 con 0x557789a741a0 2024-05-08T18:05:04.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.594+0000 7f14f3040800 1 -- 172.21.15.50:0/48862533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a741a0 msgr2=0x557789a745c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.594+0000 7f14f3040800 1 --2- 172.21.15.50:0/48862533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a741a0 0x557789a745c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f14e0088fd0 tx=0x7f14e009b490 comp rx=0 tx=0).stop 2024-05-08T18:05:04.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.594+0000 7f14f3040800 1 -- 172.21.15.50:0/48862533 shutdown_connections 2024-05-08T18:05:04.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.594+0000 7f14f3040800 1 --2- 172.21.15.50:0/48862533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a741a0 0x557789a745c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.594+0000 7f14f3040800 1 -- 172.21.15.50:0/48862533 >> 172.21.15.50:0/48862533 conn(0x557789a704b0 msgr2=0x557789a6e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.594+0000 7f14f3040800 1 -- 172.21.15.50:0/48862533 shutdown_connections 2024-05-08T18:05:04.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.594+0000 7f14f3040800 1 -- 172.21.15.50:0/48862533 wait complete. 2024-05-08T18:05:04.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.594+0000 7f14f3040800 1 Processor -- start 2024-05-08T18:05:04.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.594+0000 7f14f3040800 1 -- start start 2024-05-08T18:05:04.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.594+0000 7f14f3040800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a741a0 0x557789a7b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.594+0000 7f14f3040800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557789a75220 con 0x557789a741a0 2024-05-08T18:05:04.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.595+0000 7f14edcd5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a741a0 0x557789a7b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.595+0000 7f14edcd5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a741a0 0x557789a7b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58208/0 (socket says 172.21.15.50:58208) 2024-05-08T18:05:04.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.595+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 learned_addr learned my addr 172.21.15.50:0/3177244585 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.595+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557789a7ba50 con 0x557789a741a0 2024-05-08T18:05:04.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.595+0000 7f14edcd5640 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a741a0 0x557789a7b510 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f14e0091f80 tx=0x7f14e0059fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.595+0000 7f13bf7fe640 1 -- 172.21.15.50:0/3177244585 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e007a8a0 con 0x557789a741a0 2024-05-08T18:05:04.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.595+0000 7f13bf7fe640 1 -- 172.21.15.50:0/3177244585 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14e007aa40 con 0x557789a741a0 2024-05-08T18:05:04.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.595+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557789a7ba50 con 0x557789a741a0 2024-05-08T18:05:04.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.595+0000 7f13bf7fe640 1 -- 172.21.15.50:0/3177244585 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14e005c910 con 0x557789a741a0 2024-05-08T18:05:04.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.595+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557789a7ba50 con 0x557789a741a0 2024-05-08T18:05:04.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.596+0000 7f13bf7fe640 1 -- 172.21.15.50:0/3177244585 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f14e0089a70 con 0x557789a741a0 2024-05-08T18:05:04.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.596+0000 7f13bf7fe640 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f13b803bb50 0x7f13b803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.596+0000 7f13bf7fe640 1 -- 172.21.15.50:0/3177244585 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f14e00a3070 con 0x557789a741a0 2024-05-08T18:05:04.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.596+0000 7f14ed4d4640 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f13b803bb50 0x7f13b803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.597+0000 7f14ed4d4640 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f13b803bb50 0x7f13b803e010 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f14e400c010 tx=0x7f14e4010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.597+0000 7f14f3040800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.597+0000 7f14f3040800 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557789a86ca0 0x557789a89160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.597+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a898b0 con 0x557789a86ca0 2024-05-08T18:05:04.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.597+0000 7f14ee4d6640 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557789a86ca0 0x557789a89160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.598+0000 7f14ee4d6640 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557789a86ca0 0x557789a89160 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.598+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789a86ca0 2024-05-08T18:05:04.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.598+0000 7f14f3040800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.598+0000 7f14f3040800 20 realm 2024-05-08T18:05:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.598+0000 7f14f3040800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.598+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a89cc0 con 0x557789a86ca0 2024-05-08T18:05:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.598+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789a86ca0 2024-05-08T18:05:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.598+0000 7f14f3040800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.598+0000 7f14f3040800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.598+0000 7f14f3040800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.598+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a590 con 0x557789a86ca0 2024-05-08T18:05:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.598+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789a86ca0 2024-05-08T18:05:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.598+0000 7f14f3040800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.598+0000 7f14f3040800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.598+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a86ca0 2024-05-08T18:05:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.599+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789a86ca0 2024-05-08T18:05:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.599+0000 7f14f3040800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:04.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.599+0000 7f14f3040800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.599+0000 7f14f3040800 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557789a8b690 0x557789a8da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.599+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a8e040 con 0x557789a8b690 2024-05-08T18:05:04.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.599+0000 7f14edcd5640 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557789a8b690 0x557789a8da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.599+0000 7f14edcd5640 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557789a8b690 0x557789a8da80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.599+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.599+0000 7f14f3040800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:04.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.599+0000 7f14f3040800 20 searching for the correct realm 2024-05-08T18:05:04.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.599+0000 7f14f3040800 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557789a8fff0 0x557789a924b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.599+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a92b80 con 0x557789a8fff0 2024-05-08T18:05:04.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.600+0000 7f14ed4d4640 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557789a8fff0 0x557789a924b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.600+0000 7f14ed4d4640 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557789a8fff0 0x557789a924b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.600+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e4009050 con 0x557789a8fff0 2024-05-08T18:05:04.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.600+0000 7f14ed4d4640 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f14e4037040 0x7f14e4036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.600+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a92b80 con 0x7f14e4037040 2024-05-08T18:05:04.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.600+0000 7f14ee4d6640 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f14e4037040 0x7f14e4036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.601+0000 7f14ee4d6640 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f14e4037040 0x7f14e4036410 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.601+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e800a4a0 con 0x7f14e4037040 2024-05-08T18:05:04.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.601+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8b690 2024-05-08T18:05:04.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.601+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.601+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a86ca0 2024-05-08T18:05:04.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.601+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789a86ca0 2024-05-08T18:05:04.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.601+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a86ca0 2024-05-08T18:05:04.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.602+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789a86ca0 2024-05-08T18:05:04.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.602+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8b690 2024-05-08T18:05:04.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.602+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.602+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8b690 2024-05-08T18:05:04.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.602+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.602+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x7f14e4037040 2024-05-08T18:05:04.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.602+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e800a4a0 con 0x7f14e4037040 2024-05-08T18:05:04.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.602+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8fff0 2024-05-08T18:05:04.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.603+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e4009050 con 0x557789a8fff0 2024-05-08T18:05:04.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.603+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a86ca0 2024-05-08T18:05:04.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.603+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789a86ca0 2024-05-08T18:05:04.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.603+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8b690 2024-05-08T18:05:04.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.603+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.603+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a86ca0 2024-05-08T18:05:04.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.603+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789a86ca0 2024-05-08T18:05:04.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.603+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x7f14e4037040 2024-05-08T18:05:04.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.604+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f14e800a4a0 con 0x7f14e4037040 2024-05-08T18:05:04.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.604+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8fff0 2024-05-08T18:05:04.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.604+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e4009050 con 0x557789a8fff0 2024-05-08T18:05:04.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.604+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8fff0 2024-05-08T18:05:04.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.604+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e4009050 con 0x557789a8fff0 2024-05-08T18:05:04.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.604+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8b690 2024-05-08T18:05:04.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.604+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.604+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8fff0 2024-05-08T18:05:04.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.605+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e4009050 con 0x557789a8fff0 2024-05-08T18:05:04.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.605+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8fff0 2024-05-08T18:05:04.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.605+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f14e4009050 con 0x557789a8fff0 2024-05-08T18:05:04.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.605+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a86ca0 2024-05-08T18:05:04.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.605+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789a86ca0 2024-05-08T18:05:04.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.605+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a86ca0 2024-05-08T18:05:04.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.605+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789a86ca0 2024-05-08T18:05:04.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.605+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8b690 2024-05-08T18:05:04.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.606+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.606+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8fff0 2024-05-08T18:05:04.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.606+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e4009050 con 0x557789a8fff0 2024-05-08T18:05:04.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.606+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x7f14e4037040 2024-05-08T18:05:04.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.606+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e800a4a0 con 0x7f14e4037040 2024-05-08T18:05:04.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.606+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a86ca0 2024-05-08T18:05:04.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.606+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789a86ca0 2024-05-08T18:05:04.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.606+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8b690 2024-05-08T18:05:04.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.607+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.607+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8fff0 2024-05-08T18:05:04.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.607+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e4009050 con 0x557789a8fff0 2024-05-08T18:05:04.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.607+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8b690 2024-05-08T18:05:04.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.608+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.608+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x7f14e4037040 2024-05-08T18:05:04.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.608+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e800a4a0 con 0x7f14e4037040 2024-05-08T18:05:04.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.608+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8fff0 2024-05-08T18:05:04.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.608+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e4009050 con 0x557789a8fff0 2024-05-08T18:05:04.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.608+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a8b690 2024-05-08T18:05:04.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.608+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.608+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x557789a86ca0 2024-05-08T18:05:04.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789a86ca0 2024-05-08T18:05:04.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557789a8a930 con 0x7f14e4037040 2024-05-08T18:05:04.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f14e800a4a0 con 0x7f14e4037040 2024-05-08T18:05:04.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14f3040800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:04.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14f3040800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:04.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14f3040800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:04.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14f3040800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:04.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14f3040800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:04.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14f3040800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:04.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14f3040800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a8e900 con 0x557789a86ca0 2024-05-08T18:05:04.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789a86ca0 2024-05-08T18:05:04.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14f3040800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14f3040800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a8e900 con 0x557789a86ca0 2024-05-08T18:05:04.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f14e800a4a0 con 0x557789a86ca0 2024-05-08T18:05:04.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14f3040800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:04.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14f3040800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.609+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a8e900 con 0x557789a8fff0 2024-05-08T18:05:04.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f14e4009050 con 0x557789a8fff0 2024-05-08T18:05:04.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14f3040800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:04.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14f3040800 20 zone default found 2024-05-08T18:05:04.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14f3040800 4 Realm: () 2024-05-08T18:05:04.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14f3040800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:04.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14f3040800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:04.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14f3040800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:04.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14f3040800 20 zonegroup default 2024-05-08T18:05:04.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14f3040800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a8f2e0 con 0x557789a8b690 2024-05-08T18:05:04.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14f3040800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14f3040800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a94e90 con 0x557789a8fff0 2024-05-08T18:05:04.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f14e4009050 con 0x557789a8fff0 2024-05-08T18:05:04.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14f3040800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14f3040800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.610+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a95230 con 0x557789a8fff0 2024-05-08T18:05:04.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.611+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f14e4009050 con 0x557789a8fff0 2024-05-08T18:05:04.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.611+0000 7f14f3040800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.611+0000 7f14f3040800 20 started sync module instance, tier type = 2024-05-08T18:05:04.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.611+0000 7f14f3040800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:04.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.611+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a942d0 con 0x7f14e4037040 2024-05-08T18:05:04.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.612+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e800a4a0 con 0x7f14e4037040 2024-05-08T18:05:04.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.612+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93971899041520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a942d0 con 0x7f14e4037040 2024-05-08T18:05:04.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.612+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a95a80 con 0x557789a86ca0 2024-05-08T18:05:04.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.613+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93971899041520] v27'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e8040040 con 0x7f14e4037040 2024-05-08T18:05:04.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.613+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e8040040 con 0x557789a86ca0 2024-05-08T18:05:04.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.613+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93971899045808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a942d0 con 0x557789a86ca0 2024-05-08T18:05:04.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.613+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a964f0 con 0x557789a8fff0 2024-05-08T18:05:04.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.614+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93971899045808] v27'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e8040040 con 0x557789a86ca0 2024-05-08T18:05:04.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.614+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e4009050 con 0x557789a8fff0 2024-05-08T18:05:04.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.614+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93971899039264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a94240 con 0x557789a8fff0 2024-05-08T18:05:04.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.614+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a973b0 con 0x7f14e4037040 2024-05-08T18:05:04.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.615+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93971899039264] v27'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e4009050 con 0x557789a8fff0 2024-05-08T18:05:04.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.615+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e8040040 con 0x7f14e4037040 2024-05-08T18:05:04.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.615+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93971899051776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a977d0 con 0x7f14e4037040 2024-05-08T18:05:04.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.615+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a98190 con 0x557789a86ca0 2024-05-08T18:05:04.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.616+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93971899051776] v27'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e8040040 con 0x7f14e4037040 2024-05-08T18:05:04.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.616+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e8040040 con 0x557789a86ca0 2024-05-08T18:05:04.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.616+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93971899053312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a98190 con 0x557789a86ca0 2024-05-08T18:05:04.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.617+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a98b50 con 0x557789a8b690 2024-05-08T18:05:04.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.618+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.618+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93971899057824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a993a0 con 0x557789a8b690 2024-05-08T18:05:04.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.618+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a99e00 con 0x557789a8b690 2024-05-08T18:05:04.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.618+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93971899053312] v27'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e8040040 con 0x557789a86ca0 2024-05-08T18:05:04.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.618+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93971899057824] v27'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.619+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.619+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93971899055328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a9a1f0 con 0x557789a8b690 2024-05-08T18:05:04.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.619+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a9aa40 con 0x557789a8b690 2024-05-08T18:05:04.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.620+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93971899055328] v27'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.620+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.620+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93971899064736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a9aa70 con 0x557789a8b690 2024-05-08T18:05:04.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.620+0000 7f14f3040800 20 add_watcher() i=0 2024-05-08T18:05:04.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.620+0000 7f14f3040800 20 add_watcher() i=1 2024-05-08T18:05:04.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.620+0000 7f14f3040800 20 add_watcher() i=2 2024-05-08T18:05:04.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.620+0000 7f14f3040800 20 add_watcher() i=3 2024-05-08T18:05:04.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.620+0000 7f14f3040800 20 add_watcher() i=4 2024-05-08T18:05:04.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.620+0000 7f14f3040800 20 add_watcher() i=5 2024-05-08T18:05:04.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.620+0000 7f14f3040800 20 add_watcher() i=6 2024-05-08T18:05:04.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.621+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93971899064736] v27'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.621+0000 7f14f3040800 20 add_watcher() i=7 2024-05-08T18:05:04.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.621+0000 7f14f3040800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:04.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.621+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789a9ddd0 con 0x557789a86ca0 2024-05-08T18:05:04.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.621+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f14e8040040 con 0x557789a86ca0 2024-05-08T18:05:04.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.621+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93971899066160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a9b000 con 0x557789a86ca0 2024-05-08T18:05:04.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.622+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93971899066160] v27'60 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f14e8040040 con 0x557789a86ca0 2024-05-08T18:05:04.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.622+0000 7f139f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:04.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.622+0000 7f14f3040800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:04.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.622+0000 7f14f3040800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:04.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.622+0000 7f14f3040800 5 note: GC not initialized 2024-05-08T18:05:04.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.622+0000 7f139e7fc640 20 reqs_thread_entry: start 2024-05-08T18:05:04.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.623+0000 7f14f3040800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:04.630 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.623+0000 7f14f3040800 20 Filter name: none 2024-05-08T18:05:04.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.623+0000 7f130ffff640 20 reqs_thread_entry: start 2024-05-08T18:05:04.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.623+0000 7f14f3040800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:04.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.623+0000 7f14f3040800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.623+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789ad1290 con 0x557789a86ca0 2024-05-08T18:05:04.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.623+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f14e8040040 con 0x557789a86ca0 2024-05-08T18:05:04.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.623+0000 7f14f3040800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:04.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.623+0000 7f14f3040800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:04.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.623+0000 7f14f3040800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:04.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.623+0000 7f14f3040800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:04.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.623+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789ad0db0 con 0x7f14e4037040 2024-05-08T18:05:04.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.624+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 155+0+0 (crc 0 0 0) 0x7f14e8040040 con 0x7f14e4037040 2024-05-08T18:05:04.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.624+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789ad7530 con 0x557789a86ca0 2024-05-08T18:05:04.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.625+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~391] v27'3 uv3 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f14e8040040 con 0x557789a86ca0 2024-05-08T18:05:04.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.625+0000 7f14f3040800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:04.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.625+0000 7f14f3040800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:04.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.625+0000 7f14f3040800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-05-08T18:05:04.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.625+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 93971899314624 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557789ad4250 con 0x557789a86ca0 2024-05-08T18:05:04.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.625+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== watch-notify(notify (1) cookie 93971899053312 notify 115964116994 ret 0) ==== 609+0+0 (crc 0 0 0) 0x7f14e8018db0 con 0x557789a86ca0 2024-05-08T18:05:04.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.625+0000 7f13d17fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116994 cookie 93971899053312 notifier 4319 bl.length()=567 2024-05-08T18:05:04.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.625+0000 7f13d17fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:04.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.625+0000 7f13d17fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:04.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.625+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(65 notify.4 [notify cookie 93971899314624 out=8b] v0'0 uv39 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f14e8040040 con 0x557789a86ca0 2024-05-08T18:05:04.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.625+0000 7f13d17fa640 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f13b0003ea0 con 0x557789a86ca0 2024-05-08T18:05:04.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.625+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e8040040 con 0x557789a86ca0 2024-05-08T18:05:04.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.626+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== watch-notify(notify_complete (2) cookie 93971899314624 notify 115964116994 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f14e8038450 con 0x557789a86ca0 2024-05-08T18:05:04.633 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:04.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.626+0000 7f139f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:04.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.627+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93971899066160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789ad4000 con 0x557789a86ca0 2024-05-08T18:05:04.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.628+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(67 data_loggenerations_metadata [watch unwatch cookie 93971899066160] v27'61 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f14e8040040 con 0x557789a86ca0 2024-05-08T18:05:04.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.628+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93971899041520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a9a730 con 0x7f14e4037040 2024-05-08T18:05:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.629+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(68 notify.0 [watch unwatch cookie 93971899041520] v27'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e8040040 con 0x7f14e4037040 2024-05-08T18:05:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.629+0000 7f14f3040800 20 remove_watcher() i=0 2024-05-08T18:05:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.629+0000 7f14f3040800 2 removed watcher, disabling cache 2024-05-08T18:05:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.629+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93971899045808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a9a730 con 0x557789a86ca0 2024-05-08T18:05:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.630+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(69 notify.1 [watch unwatch cookie 93971899045808] v27'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e8040040 con 0x557789a86ca0 2024-05-08T18:05:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.630+0000 7f14f3040800 20 remove_watcher() i=1 2024-05-08T18:05:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.630+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93971899039264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a9a730 con 0x557789a8fff0 2024-05-08T18:05:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.631+0000 7f14ed4d4640 1 -- 172.21.15.50:0/3177244585 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(70 notify.2 [watch unwatch cookie 93971899039264] v27'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e4009050 con 0x557789a8fff0 2024-05-08T18:05:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.631+0000 7f14f3040800 20 remove_watcher() i=2 2024-05-08T18:05:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.631+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93971899051776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a93e20 con 0x7f14e4037040 2024-05-08T18:05:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.632+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(71 notify.3 [watch unwatch cookie 93971899051776] v27'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e8040040 con 0x7f14e4037040 2024-05-08T18:05:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.632+0000 7f14f3040800 20 remove_watcher() i=3 2024-05-08T18:05:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.632+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93971899053312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a96f00 con 0x557789a86ca0 2024-05-08T18:05:04.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.633+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(72 notify.4 [watch unwatch cookie 93971899053312] v27'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e8040040 con 0x557789a86ca0 2024-05-08T18:05:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.633+0000 7f14f3040800 20 remove_watcher() i=4 2024-05-08T18:05:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.633+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93971899057824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a93e20 con 0x557789a8b690 2024-05-08T18:05:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.634+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(73 notify.5 [watch unwatch cookie 93971899057824] v27'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.634+0000 7f14f3040800 20 remove_watcher() i=5 2024-05-08T18:05:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.634+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93971899055328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a97500 con 0x557789a8b690 2024-05-08T18:05:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.635+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(74 notify.6 [watch unwatch cookie 93971899055328] v27'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.635+0000 7f14f3040800 20 remove_watcher() i=6 2024-05-08T18:05:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.635+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93971899064736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557789a96f00 con 0x557789a8b690 2024-05-08T18:05:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.636+0000 7f14edcd5640 1 -- 172.21.15.50:0/3177244585 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(75 notify.7 [watch unwatch cookie 93971899064736] v27'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f14e00c0020 con 0x557789a8b690 2024-05-08T18:05:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.636+0000 7f14f3040800 20 remove_watcher() i=7 2024-05-08T18:05:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557789a8b690 msgr2=0x557789a8da80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557789a8b690 0x557789a8da80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557789a86ca0 msgr2=0x557789a89160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557789a86ca0 0x557789a89160 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f14e4037040 msgr2=0x7f14e4036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f14e4037040 0x7f14e4036410 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557789a8fff0 msgr2=0x557789a924b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557789a8fff0 0x557789a924b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f13b803bb50 msgr2=0x7f13b803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f13b803bb50 0x7f13b803e010 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f14e400c010 tx=0x7f14e4010000 comp rx=0 tx=0).stop 2024-05-08T18:05:04.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a741a0 msgr2=0x557789a7b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 --2- 172.21.15.50:0/3177244585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a741a0 0x557789a7b510 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f14e0091f80 tx=0x7f14e0059fa0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 reap_dead start 2024-05-08T18:05:04.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3177244585 reap_dead start 2024-05-08T18:05:04.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 shutdown_connections 2024-05-08T18:05:04.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 >> 172.21.15.50:0/3177244585 conn(0x557789a704b0 msgr2=0x557789a6db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 shutdown_connections 2024-05-08T18:05:04.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 -- 172.21.15.50:0/3177244585 wait complete. 2024-05-08T18:05:04.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 -- 172.21.15.50:0/2703806367 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f13c803bb50 msgr2=0x7f13c803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 --2- 172.21.15.50:0/2703806367 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f13c803bb50 0x7f13c803e010 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f14e400bf40 tx=0x7f14e400f000 comp rx=0 tx=0).stop 2024-05-08T18:05:04.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 -- 172.21.15.50:0/2703806367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a66fe0 msgr2=0x557789a67400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 --2- 172.21.15.50:0/2703806367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a66fe0 0x557789a67400 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f14e00931b0 tx=0x7f14e005c380 comp rx=0 tx=0).stop 2024-05-08T18:05:04.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 -- 172.21.15.50:0/2703806367 shutdown_connections 2024-05-08T18:05:04.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 --2- 172.21.15.50:0/2703806367 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f13c803bb50 0x7f13c803e010 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 --2- 172.21.15.50:0/2703806367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557789a66fe0 0x557789a67400 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 -- 172.21.15.50:0/2703806367 >> 172.21.15.50:0/2703806367 conn(0x557789a5b080 msgr2=0x557789a5b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 -- 172.21.15.50:0/2703806367 shutdown_connections 2024-05-08T18:05:04.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.637+0000 7f14f3040800 1 -- 172.21.15.50:0/2703806367 wait complete. 2024-05-08T18:05:04.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.638+0000 7f14f3040800 1 -- 172.21.15.50:0/3544177276 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557789a520f0 msgr2=0x557789a54590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.638+0000 7f14f3040800 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557789a520f0 0x557789a54590 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.638+0000 7f14f3040800 1 -- 172.21.15.50:0/3544177276 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557789950780 msgr2=0x557789a4fa20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.638+0000 7f14f3040800 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557789950780 0x557789a4fa20 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.638+0000 7f14f3040800 1 -- 172.21.15.50:0/3544177276 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557789a56180 msgr2=0x557789a58570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.638+0000 7f14f3040800 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557789a56180 0x557789a58570 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.638+0000 7f14f3040800 1 -- 172.21.15.50:0/3544177276 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f13d403bba0 msgr2=0x7f13d403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.638+0000 7f14f3040800 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f13d403bba0 0x7f13d403e060 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f14e4001d70 tx=0x7f14e40012d0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.638+0000 7f14f3040800 1 -- 172.21.15.50:0/3544177276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 msgr2=0x557789a487e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.638+0000 7f14f3040800 1 --2- 172.21.15.50:0/3544177276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577898c0700 0x557789a487e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f14e0060040 tx=0x7f14e0002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.638+0000 7f14ee4d6640 1 -- 172.21.15.50:0/3544177276 reap_dead start 2024-05-08T18:05:04.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.638+0000 7f14f3040800 1 -- 172.21.15.50:0/3544177276 shutdown_connections 2024-05-08T18:05:04.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.638+0000 7f14f3040800 1 -- 172.21.15.50:0/3544177276 >> 172.21.15.50:0/3544177276 conn(0x557789947b50 msgr2=0x557789a3a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.638+0000 7f14f3040800 1 -- 172.21.15.50:0/3544177276 shutdown_connections 2024-05-08T18:05:04.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.638+0000 7f14f3040800 1 -- 172.21.15.50:0/3544177276 wait complete. 2024-05-08T18:05:04.645 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:04.645 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-05-08T18:05:04.645 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-05-08T18:05:04.646 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-05-08T18:05:04.691 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:04.691 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:04.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.694+0000 7f2a1392f800 1 Processor -- start 2024-05-08T18:05:04.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.694+0000 7f2a1392f800 1 -- start start 2024-05-08T18:05:04.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.694+0000 7f2a1392f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55bc4a188cb0 0x55bc4a18b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.694+0000 7f2a1392f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55bc4a18b5e0 con 0x55bc4a1851a0 2024-05-08T18:05:04.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.694+0000 7f2a1392f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55bc4a024e20 con 0x55bc4a188cb0 2024-05-08T18:05:04.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.694+0000 7f2a0eebd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55bc4a188cb0 0x55bc4a18b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.694+0000 7f2a0f6be640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55bc4a1851a0 0x55bc4a187590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:53094/0 (socket says 172.21.15.50:53094) 2024-05-08T18:05:04.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.694+0000 7f2a0f6be640 1 -- 172.21.15.50:0/2843434999 learned_addr learned my addr 172.21.15.50:0/2843434999 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:04.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.694+0000 7f2a0eebd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55bc4a188cb0 0x55bc4a18b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:58224/0 (socket says 172.21.15.50:58224) 2024-05-08T18:05:04.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.694+0000 7f2a0eebd640 1 -- 172.21.15.50:0/2843434999 >> v1:172.21.15.46:6789/0 conn(0x55bc4a1851a0 legacy=0x55bc4a187590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.694+0000 7f2a0e6bc640 1 -- 172.21.15.50:0/2843434999 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3473209844 0 0) 0x55bc4a18b5e0 con 0x55bc4a1851a0 2024-05-08T18:05:04.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.695+0000 7f2a0eebd640 1 -- 172.21.15.50:0/2843434999 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc4a18c7c0 con 0x55bc4a188cb0 2024-05-08T18:05:04.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.696+0000 7f2a0eebd640 1 --2- 172.21.15.50:0/2843434999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a188cb0 0x55bc4a18b0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400a470 tx=0x7f2a0401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b93fa9932035ad0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.697+0000 7f2a0e6bc640 1 -- 172.21.15.50:0/2843434999 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a04057070 con 0x55bc4a188cb0 2024-05-08T18:05:04.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.697+0000 7f2a0e6bc640 1 -- 172.21.15.50:0/2843434999 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a04052570 con 0x55bc4a188cb0 2024-05-08T18:05:04.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.697+0000 7f2a0e6bc640 1 -- 172.21.15.50:0/2843434999 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a04052870 con 0x55bc4a188cb0 2024-05-08T18:05:04.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.697+0000 7f2a1392f800 1 -- 172.21.15.50:0/2843434999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a188cb0 msgr2=0x55bc4a18b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.697+0000 7f2a1392f800 1 --2- 172.21.15.50:0/2843434999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a188cb0 0x55bc4a18b0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400a470 tx=0x7f2a0401ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:04.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.697+0000 7f2a1392f800 1 -- 172.21.15.50:0/2843434999 shutdown_connections 2024-05-08T18:05:04.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.697+0000 7f2a1392f800 1 --2- 172.21.15.50:0/2843434999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a188cb0 0x55bc4a18b0a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.697+0000 7f2a1392f800 1 -- 172.21.15.50:0/2843434999 >> 172.21.15.50:0/2843434999 conn(0x55bc4a17ead0 msgr2=0x55bc4a180f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.697+0000 7f2a1392f800 1 -- 172.21.15.50:0/2843434999 shutdown_connections 2024-05-08T18:05:04.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.697+0000 7f2a1392f800 1 -- 172.21.15.50:0/2843434999 wait complete. 2024-05-08T18:05:04.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.712+0000 7f2a1392f800 1 Processor -- start 2024-05-08T18:05:04.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.712+0000 7f2a1392f800 1 -- start start 2024-05-08T18:05:04.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.712+0000 7f2a1392f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a1851a0 0x55bc4a2759f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.712+0000 7f2a1392f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bc4a0cfa10 con 0x55bc4a1851a0 2024-05-08T18:05:04.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.713+0000 7f2a0f6be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a1851a0 0x55bc4a2759f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.713+0000 7f2a0f6be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a1851a0 0x55bc4a2759f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36472/0 (socket says 172.21.15.50:36472) 2024-05-08T18:05:04.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.713+0000 7f2a0f6be640 1 -- 172.21.15.50:0/2729004458 learned_addr learned my addr 172.21.15.50:0/2729004458 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.713+0000 7f2a0f6be640 1 -- 172.21.15.50:0/2729004458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc4a275fc0 con 0x55bc4a1851a0 2024-05-08T18:05:04.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.713+0000 7f2a0f6be640 1 --2- 172.21.15.50:0/2729004458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a1851a0 0x55bc4a2759f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2a00001e70 tx=0x7f2a0000b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b898191b30d360b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.713+0000 7f28fd60a640 1 -- 172.21.15.50:0/2729004458 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a00008070 con 0x55bc4a1851a0 2024-05-08T18:05:04.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.713+0000 7f28fd60a640 1 -- 172.21.15.50:0/2729004458 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0000d500 con 0x55bc4a1851a0 2024-05-08T18:05:04.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.713+0000 7f28fd60a640 1 -- 172.21.15.50:0/2729004458 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a0000d8a0 con 0x55bc4a1851a0 2024-05-08T18:05:04.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.714+0000 7f2a1392f800 1 -- 172.21.15.50:0/2729004458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a1851a0 msgr2=0x55bc4a2759f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.714+0000 7f2a1392f800 1 --2- 172.21.15.50:0/2729004458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a1851a0 0x55bc4a2759f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2a00001e70 tx=0x7f2a0000b6a0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.714+0000 7f2a1392f800 1 -- 172.21.15.50:0/2729004458 shutdown_connections 2024-05-08T18:05:04.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.714+0000 7f2a1392f800 1 --2- 172.21.15.50:0/2729004458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a1851a0 0x55bc4a2759f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.714+0000 7f2a1392f800 1 -- 172.21.15.50:0/2729004458 >> 172.21.15.50:0/2729004458 conn(0x55bc4a17ead0 msgr2=0x55bc4a271310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.714+0000 7f2a1392f800 1 -- 172.21.15.50:0/2729004458 shutdown_connections 2024-05-08T18:05:04.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.714+0000 7f2a1392f800 1 -- 172.21.15.50:0/2729004458 wait complete. 2024-05-08T18:05:04.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.714+0000 7f2a1392f800 1 Processor -- start 2024-05-08T18:05:04.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.715+0000 7f2a1392f800 1 -- start start 2024-05-08T18:05:04.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.715+0000 7f2a1392f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a1851a0 0x55bc4a27d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.715+0000 7f2a1392f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bc4a18bab0 con 0x55bc4a1851a0 2024-05-08T18:05:04.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.715+0000 7f2a0f6be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a1851a0 0x55bc4a27d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.715+0000 7f2a0f6be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a1851a0 0x55bc4a27d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36486/0 (socket says 172.21.15.50:36486) 2024-05-08T18:05:04.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.715+0000 7f2a0f6be640 1 -- 172.21.15.50:0/2630968313 learned_addr learned my addr 172.21.15.50:0/2630968313 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.715+0000 7f2a0f6be640 1 -- 172.21.15.50:0/2630968313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc4a27dab0 con 0x55bc4a1851a0 2024-05-08T18:05:04.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.715+0000 7f2a0f6be640 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a1851a0 0x55bc4a27d570 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55bc4a276780 tx=0x7f2a00000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.716+0000 7f28fbe07640 1 -- 172.21.15.50:0/2630968313 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a0000f980 con 0x55bc4a1851a0 2024-05-08T18:05:04.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.716+0000 7f28fbe07640 1 -- 172.21.15.50:0/2630968313 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0000fb20 con 0x55bc4a1851a0 2024-05-08T18:05:04.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.716+0000 7f2a1392f800 1 -- 172.21.15.50:0/2630968313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bc4a27dd40 con 0x55bc4a1851a0 2024-05-08T18:05:04.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.716+0000 7f28fbe07640 1 -- 172.21.15.50:0/2630968313 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a0000fe00 con 0x55bc4a1851a0 2024-05-08T18:05:04.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.716+0000 7f2a1392f800 1 -- 172.21.15.50:0/2630968313 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc4a2809a0 con 0x55bc4a1851a0 2024-05-08T18:05:04.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.717+0000 7f28fbe07640 1 -- 172.21.15.50:0/2630968313 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2a00017040 con 0x55bc4a1851a0 2024-05-08T18:05:04.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.717+0000 7f28fbe07640 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f28f403bba0 0x7f28f403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.717+0000 7f28fbe07640 1 -- 172.21.15.50:0/2630968313 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f2a00015020 con 0x55bc4a1851a0 2024-05-08T18:05:04.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.717+0000 7f2a0eebd640 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f28f403bba0 0x7f28f403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.717+0000 7f2a1392f800 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bc4a188cb0 0x55bc4a284760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.717+0000 7f2a1392f800 1 -- 172.21.15.50:0/2630968313 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a284ca0 con 0x55bc4a188cb0 2024-05-08T18:05:04.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.717+0000 7f2a0febf640 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bc4a188cb0 0x55bc4a284760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.718+0000 7f2a0febf640 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bc4a188cb0 0x55bc4a284760 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.718+0000 7f2a0eebd640 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f28f403bba0 0x7f28f403e060 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2a04054000 tx=0x7f2a0404e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.718+0000 7f2a0febf640 1 -- 172.21.15.50:0/2630968313 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a188cb0 2024-05-08T18:05:04.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.718+0000 7f2a1392f800 1 -- 172.21.15.50:0/2630968313 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a281d10 con 0x55bc4a188cb0 2024-05-08T18:05:04.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.718+0000 7f2a0febf640 1 -- 172.21.15.50:0/2630968313 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a188cb0 2024-05-08T18:05:04.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.718+0000 7f2a1392f800 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bc4a287020 0x55bc4a289410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.718+0000 7f2a1392f800 1 -- 172.21.15.50:0/2630968313 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a289ab0 con 0x55bc4a287020 2024-05-08T18:05:04.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.719+0000 7f2a0f6be640 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bc4a287020 0x55bc4a289410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.719+0000 7f2a0f6be640 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bc4a287020 0x55bc4a289410 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.719+0000 7f2a0f6be640 1 -- 172.21.15.50:0/2630968313 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f2a0002d070 con 0x55bc4a287020 2024-05-08T18:05:04.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.719+0000 7f2a1392f800 1 -- 172.21.15.50:0/2630968313 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a285ec0 con 0x55bc4a188cb0 2024-05-08T18:05:04.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.719+0000 7f2a0febf640 1 -- 172.21.15.50:0/2630968313 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a188cb0 2024-05-08T18:05:04.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.719+0000 7f2a1392f800 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bc4a28af10 0x55bc4a28d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.719+0000 7f2a1392f800 1 -- 172.21.15.50:0/2630968313 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a28da00 con 0x55bc4a28af10 2024-05-08T18:05:04.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.720+0000 7f2a0eebd640 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bc4a28af10 0x55bc4a28d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.720+0000 7f2a0eebd640 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bc4a28af10 0x55bc4a28d300 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.720+0000 7f2a0eebd640 1 -- 172.21.15.50:0/2630968313 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f2a04052940 con 0x55bc4a28af10 2024-05-08T18:05:04.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.721+0000 7f2a1392f800 1 Processor -- start 2024-05-08T18:05:04.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.721+0000 7f2a1392f800 1 -- start start 2024-05-08T18:05:04.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.721+0000 7f2a1392f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a184d40 0x55bc4a1831b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.721+0000 7f2a1392f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bc4a076330 con 0x55bc4a184d40 2024-05-08T18:05:04.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.721+0000 7f2a0f6be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a184d40 0x55bc4a1831b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.721+0000 7f2a0f6be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a184d40 0x55bc4a1831b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36496/0 (socket says 172.21.15.50:36496) 2024-05-08T18:05:04.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.721+0000 7f2a0f6be640 1 -- 172.21.15.50:0/2782400280 learned_addr learned my addr 172.21.15.50:0/2782400280 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.721+0000 7f2a0f6be640 1 -- 172.21.15.50:0/2782400280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc4a1836f0 con 0x55bc4a184d40 2024-05-08T18:05:04.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.721+0000 7f2a0f6be640 1 --2- 172.21.15.50:0/2782400280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a184d40 0x55bc4a1831b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2a000517c0 tx=0x7f2a0001ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.722+0000 7f28f9602640 1 -- 172.21.15.50:0/2782400280 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a0004a350 con 0x55bc4a184d40 2024-05-08T18:05:04.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.722+0000 7f28f9602640 1 -- 172.21.15.50:0/2782400280 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a000501b0 con 0x55bc4a184d40 2024-05-08T18:05:04.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.722+0000 7f2a1392f800 1 -- 172.21.15.50:0/2782400280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bc4a183980 con 0x55bc4a184d40 2024-05-08T18:05:04.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.722+0000 7f28f9602640 1 -- 172.21.15.50:0/2782400280 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a000183d0 con 0x55bc4a184d40 2024-05-08T18:05:04.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.722+0000 7f2a1392f800 1 -- 172.21.15.50:0/2782400280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc4a183e80 con 0x55bc4a184d40 2024-05-08T18:05:04.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.722+0000 7f28f9602640 1 -- 172.21.15.50:0/2782400280 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2a00018570 con 0x55bc4a184d40 2024-05-08T18:05:04.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.722+0000 7f28f9602640 1 --2- 172.21.15.50:0/2782400280 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f28e803bba0 0x7f28e803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.723+0000 7f28f9602640 1 -- 172.21.15.50:0/2782400280 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f2a0003c610 con 0x55bc4a184d40 2024-05-08T18:05:04.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.723+0000 7f2a0eebd640 1 --2- 172.21.15.50:0/2782400280 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f28e803bba0 0x7f28e803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.723+0000 7f2a1392f800 1 Processor -- start 2024-05-08T18:05:04.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.723+0000 7f2a1392f800 1 -- start start 2024-05-08T18:05:04.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.723+0000 7f2a0eebd640 1 --2- 172.21.15.50:0/2782400280 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f28e803bba0 0x7f28e803e060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55bc4a281c50 tx=0x7f2a04005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.723+0000 7f2a1392f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a2ab1b0 0x55bc4a2ab5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.723+0000 7f2a1392f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bc4a276340 con 0x55bc4a2ab1b0 2024-05-08T18:05:04.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.723+0000 7f2a0f6be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a2ab1b0 0x55bc4a2ab5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.723+0000 7f2a0f6be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a2ab1b0 0x55bc4a2ab5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36512/0 (socket says 172.21.15.50:36512) 2024-05-08T18:05:04.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.723+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3770365849 learned_addr learned my addr 172.21.15.50:0/3770365849 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.723+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3770365849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc4a2abba0 con 0x55bc4a2ab1b0 2024-05-08T18:05:04.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.724+0000 7f2a0f6be640 1 --2- 172.21.15.50:0/3770365849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a2ab1b0 0x55bc4a2ab5d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2a0003aa50 tx=0x7f2a00042d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d66942efd97b96 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.724+0000 7f28f2ffd640 1 -- 172.21.15.50:0/3770365849 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a0002f5a0 con 0x55bc4a2ab1b0 2024-05-08T18:05:04.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.724+0000 7f28f2ffd640 1 -- 172.21.15.50:0/3770365849 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a00019470 con 0x55bc4a2ab1b0 2024-05-08T18:05:04.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.724+0000 7f28f2ffd640 1 -- 172.21.15.50:0/3770365849 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a00026e60 con 0x55bc4a2ab1b0 2024-05-08T18:05:04.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.724+0000 7f2a1392f800 1 -- 172.21.15.50:0/3770365849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a2ab1b0 msgr2=0x55bc4a2ab5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.724+0000 7f2a1392f800 1 --2- 172.21.15.50:0/3770365849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a2ab1b0 0x55bc4a2ab5d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2a0003aa50 tx=0x7f2a00042d50 comp rx=0 tx=0).stop 2024-05-08T18:05:04.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.724+0000 7f2a1392f800 1 -- 172.21.15.50:0/3770365849 shutdown_connections 2024-05-08T18:05:04.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.724+0000 7f2a1392f800 1 --2- 172.21.15.50:0/3770365849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a2ab1b0 0x55bc4a2ab5d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.724+0000 7f2a1392f800 1 -- 172.21.15.50:0/3770365849 >> 172.21.15.50:0/3770365849 conn(0x55bc4a2a74d0 msgr2=0x55bc4a2a5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.724+0000 7f2a1392f800 1 -- 172.21.15.50:0/3770365849 shutdown_connections 2024-05-08T18:05:04.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.724+0000 7f2a1392f800 1 -- 172.21.15.50:0/3770365849 wait complete. 2024-05-08T18:05:04.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.725+0000 7f2a1392f800 1 Processor -- start 2024-05-08T18:05:04.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.725+0000 7f2a1392f800 1 -- start start 2024-05-08T18:05:04.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.725+0000 7f2a1392f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a2ab1b0 0x55bc4a2b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.725+0000 7f2a1392f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bc4a2ac140 con 0x55bc4a2ab1b0 2024-05-08T18:05:04.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.725+0000 7f2a0f6be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a2ab1b0 0x55bc4a2b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.725+0000 7f2a0f6be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a2ab1b0 0x55bc4a2b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36520/0 (socket says 172.21.15.50:36520) 2024-05-08T18:05:04.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.725+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 learned_addr learned my addr 172.21.15.50:0/3897975864 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.725+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc4a2b2a60 con 0x55bc4a2ab1b0 2024-05-08T18:05:04.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.725+0000 7f2a0f6be640 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a2ab1b0 0x55bc4a2b2520 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000bbf0 tx=0x7f2a0003a9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.726+0000 7f28f17fa640 1 -- 172.21.15.50:0/3897975864 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a0002f5a0 con 0x55bc4a2ab1b0 2024-05-08T18:05:04.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.726+0000 7f28f17fa640 1 -- 172.21.15.50:0/3897975864 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0001cb60 con 0x55bc4a2ab1b0 2024-05-08T18:05:04.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.726+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bc4a2b2a60 con 0x55bc4a2ab1b0 2024-05-08T18:05:04.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.726+0000 7f28f17fa640 1 -- 172.21.15.50:0/3897975864 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a0001ce40 con 0x55bc4a2ab1b0 2024-05-08T18:05:04.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.726+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc4a2b2a60 con 0x55bc4a2ab1b0 2024-05-08T18:05:04.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.726+0000 7f28f17fa640 1 -- 172.21.15.50:0/3897975864 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2a0005b070 con 0x55bc4a2ab1b0 2024-05-08T18:05:04.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.727+0000 7f28f17fa640 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f28d803bba0 0x7f28d803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.727+0000 7f28f17fa640 1 -- 172.21.15.50:0/3897975864 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f2a00065070 con 0x55bc4a2ab1b0 2024-05-08T18:05:04.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.727+0000 7f2a0eebd640 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f28d803bba0 0x7f28d803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.727+0000 7f2a1392f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.727+0000 7f2a1392f800 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bc4a2bdc40 0x55bc4a2c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.727+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c0850 con 0x55bc4a2bdc40 2024-05-08T18:05:04.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.727+0000 7f2a0febf640 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bc4a2bdc40 0x55bc4a2c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.727+0000 7f2a0eebd640 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f28d803bba0 0x7f28d803e060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2a04052fd0 tx=0x7f2a0404f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.728+0000 7f2a0febf640 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bc4a2bdc40 0x55bc4a2c0100 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.728+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.728+0000 7f2a1392f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.728+0000 7f2a1392f800 20 realm 2024-05-08T18:05:04.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.728+0000 7f2a1392f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.728+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c0c60 con 0x55bc4a2bdc40 2024-05-08T18:05:04.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.728+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.728+0000 7f2a1392f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.728+0000 7f2a1392f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:04.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.728+0000 7f2a1392f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.728+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c1530 con 0x55bc4a2bdc40 2024-05-08T18:05:04.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.728+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.728+0000 7f2a1392f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.728+0000 7f2a1392f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.728+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c18d0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.729+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.729+0000 7f2a1392f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:04.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.729+0000 7f2a1392f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.729+0000 7f2a1392f800 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bc4a2c23c0 0x55bc4a2c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.729+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c4f10 con 0x55bc4a2c23c0 2024-05-08T18:05:04.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.729+0000 7f2a0f6be640 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bc4a2c23c0 0x55bc4a2c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.729+0000 7f2a0f6be640 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bc4a2c23c0 0x55bc4a2c47b0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.729+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.729+0000 7f2a1392f800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:04.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.729+0000 7f2a1392f800 20 searching for the correct realm 2024-05-08T18:05:04.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.729+0000 7f2a1392f800 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bc4a2c6fc0 0x55bc4a2c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.729+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c9b50 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.730+0000 7f2a0eebd640 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bc4a2c6fc0 0x55bc4a2c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.730+0000 7f2a0eebd640 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bc4a2c6fc0 0x55bc4a2c9480 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.730+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a04081070 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.730+0000 7f2a0eebd640 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2a04076040 0x7f2a04083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.730+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c9b50 con 0x7f2a04076040 2024-05-08T18:05:04.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.730+0000 7f2a0febf640 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2a04076040 0x7f2a04083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.731+0000 7f2a0febf640 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2a04076040 0x7f2a04083410 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.731+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0800a4a0 con 0x7f2a04076040 2024-05-08T18:05:04.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.731+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c18d0 con 0x55bc4a2c23c0 2024-05-08T18:05:04.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.731+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.731+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c18d0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.731+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.731+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c18d0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.731+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.731+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c18d0 con 0x55bc4a2c23c0 2024-05-08T18:05:04.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.732+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.732+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c18d0 con 0x55bc4a2c23c0 2024-05-08T18:05:04.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.732+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.732+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c18d0 con 0x7f2a04076040 2024-05-08T18:05:04.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.732+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0800a4a0 con 0x7f2a04076040 2024-05-08T18:05:04.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.732+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c18d0 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.733+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a04081070 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.733+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c18d0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.733+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.733+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c18d0 con 0x55bc4a2c23c0 2024-05-08T18:05:04.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.733+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.733+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c18d0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.733+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.733+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c18d0 con 0x7f2a04076040 2024-05-08T18:05:04.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.734+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2a0800a4a0 con 0x7f2a04076040 2024-05-08T18:05:04.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.734+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c18d0 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.734+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a04081070 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.734+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.734+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a04081070 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.734+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x55bc4a2c23c0 2024-05-08T18:05:04.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.734+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.734+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.735+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a04081070 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.735+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.735+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f2a04081070 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.735+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x55bc4a2bdc40 2024-05-08T18:05:04.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.735+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.735+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x55bc4a2bdc40 2024-05-08T18:05:04.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.735+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.735+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x55bc4a2c23c0 2024-05-08T18:05:04.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.736+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.736+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.736+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a04081070 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.736+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x7f2a04076040 2024-05-08T18:05:04.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.736+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0800a4a0 con 0x7f2a04076040 2024-05-08T18:05:04.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.736+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x55bc4a2bdc40 2024-05-08T18:05:04.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.736+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.736+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x55bc4a2c23c0 2024-05-08T18:05:04.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.737+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.737+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.737+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a04081070 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.737+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x55bc4a2c23c0 2024-05-08T18:05:04.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.738+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.738+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x7f2a04076040 2024-05-08T18:05:04.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.738+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0800a4a0 con 0x7f2a04076040 2024-05-08T18:05:04.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.738+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.738+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a04081070 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.738+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x55bc4a2c23c0 2024-05-08T18:05:04.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x55bc4a2bdc40 2024-05-08T18:05:04.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f2a04090040 con 0x7f2a04076040 2024-05-08T18:05:04.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a0800a4a0 con 0x7f2a04076040 2024-05-08T18:05:04.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a1392f800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:04.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a1392f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:04.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a1392f800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:04.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a1392f800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:04.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a1392f800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:04.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a1392f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:04.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a1392f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c58c0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a1392f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a1392f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.739+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c58c0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.740+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2a0800a4a0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.740+0000 7f2a1392f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:04.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.740+0000 7f2a1392f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.740+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c5890 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.740+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f2a04081070 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.740+0000 7f2a1392f800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:04.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.740+0000 7f2a1392f800 20 zone default found 2024-05-08T18:05:04.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.740+0000 7f2a1392f800 4 Realm: () 2024-05-08T18:05:04.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.740+0000 7f2a1392f800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:04.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.740+0000 7f2a1392f800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:04.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.740+0000 7f2a1392f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:04.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.740+0000 7f2a1392f800 20 zonegroup default 2024-05-08T18:05:04.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.740+0000 7f2a1392f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.740+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2c6160 con 0x55bc4a2c23c0 2024-05-08T18:05:04.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.741+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.741+0000 7f2a1392f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.741+0000 7f2a1392f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.741+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2cb9b0 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.741+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2a04081070 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.741+0000 7f2a1392f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.741+0000 7f2a1392f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.741+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2cbd50 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.741+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2a04081070 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.741+0000 7f2a1392f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.741+0000 7f2a1392f800 20 started sync module instance, tier type = 2024-05-08T18:05:04.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.741+0000 7f2a1392f800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:04.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.741+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2cae50 con 0x7f2a04076040 2024-05-08T18:05:04.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.742+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a0800a4a0 con 0x7f2a04076040 2024-05-08T18:05:04.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.742+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94267186656112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2cae50 con 0x7f2a04076040 2024-05-08T18:05:04.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.742+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2cc750 con 0x55bc4a2bdc40 2024-05-08T18:05:04.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.743+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94267186656112] v27'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a08040040 con 0x7f2a04076040 2024-05-08T18:05:04.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.743+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a08040040 con 0x55bc4a2bdc40 2024-05-08T18:05:04.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.744+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94267186659936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2cae50 con 0x55bc4a2bdc40 2024-05-08T18:05:04.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.744+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2cd200 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.744+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94267186659936] v27'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a08040040 con 0x55bc4a2bdc40 2024-05-08T18:05:04.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.744+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a04081070 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.745+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94267186653600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2cadc0 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.745+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2ce0c0 con 0x7f2a04076040 2024-05-08T18:05:04.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.745+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94267186653600] v27'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a04081070 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.746+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a08040040 con 0x7f2a04076040 2024-05-08T18:05:04.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.746+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94267186666512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2ce030 con 0x7f2a04076040 2024-05-08T18:05:04.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.746+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2cee80 con 0x55bc4a2bdc40 2024-05-08T18:05:04.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.747+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a08040040 con 0x55bc4a2bdc40 2024-05-08T18:05:04.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.747+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94267186669920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2cee80 con 0x55bc4a2bdc40 2024-05-08T18:05:04.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.747+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94267186666512] v27'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a08040040 con 0x7f2a04076040 2024-05-08T18:05:04.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.747+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2cfdc0 con 0x55bc4a2c23c0 2024-05-08T18:05:04.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.748+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94267186669920] v27'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a08040040 con 0x55bc4a2bdc40 2024-05-08T18:05:04.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.748+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.748+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94267186668800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2cf270 con 0x55bc4a2c23c0 2024-05-08T18:05:04.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.748+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2d0610 con 0x55bc4a2c23c0 2024-05-08T18:05:04.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.749+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94267186668800] v27'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.749+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.749+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94267186676064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2d0a30 con 0x55bc4a2c23c0 2024-05-08T18:05:04.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.749+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2d13f0 con 0x55bc4a2c23c0 2024-05-08T18:05:04.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.750+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v27'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.750+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94267186676064] v27'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.750+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94267186679616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2d1810 con 0x55bc4a2c23c0 2024-05-08T18:05:04.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.750+0000 7f2a1392f800 20 add_watcher() i=0 2024-05-08T18:05:04.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.750+0000 7f2a1392f800 20 add_watcher() i=1 2024-05-08T18:05:04.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.750+0000 7f2a1392f800 20 add_watcher() i=2 2024-05-08T18:05:04.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.750+0000 7f2a1392f800 20 add_watcher() i=3 2024-05-08T18:05:04.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.750+0000 7f2a1392f800 20 add_watcher() i=4 2024-05-08T18:05:04.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.750+0000 7f2a1392f800 20 add_watcher() i=5 2024-05-08T18:05:04.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.750+0000 7f2a1392f800 20 add_watcher() i=6 2024-05-08T18:05:04.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.751+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94267186679616] v27'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.751+0000 7f2a1392f800 20 add_watcher() i=7 2024-05-08T18:05:04.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.751+0000 7f2a1392f800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:04.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.751+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2d1810 con 0x55bc4a2bdc40 2024-05-08T18:05:04.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.751+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f2a08040040 con 0x55bc4a2bdc40 2024-05-08T18:05:04.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.751+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94267186680672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2d20e0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.752+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94267186680672] v27'62 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2a08040040 con 0x55bc4a2bdc40 2024-05-08T18:05:04.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.752+0000 7f28c5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:04.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.753+0000 7f2a1392f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:04.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.753+0000 7f2a1392f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:04.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.753+0000 7f2a1392f800 5 note: GC not initialized 2024-05-08T18:05:04.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.753+0000 7f28c4fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:04.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.753+0000 7f2a1392f800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:04.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.753+0000 7f2a1392f800 20 Filter name: none 2024-05-08T18:05:04.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.753+0000 7f283e7fc640 20 reqs_thread_entry: start 2024-05-08T18:05:04.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.753+0000 7f2a1392f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.753+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a3078e0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.754+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f2a08040040 con 0x55bc4a2bdc40 2024-05-08T18:05:04.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.754+0000 7f2a1392f800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:04.762 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:04.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.754+0000 7f28c5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:04.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.754+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94267186680672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2bb0a0 con 0x55bc4a2bdc40 2024-05-08T18:05:04.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.755+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94267186680672] v27'63 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2a08040040 con 0x55bc4a2bdc40 2024-05-08T18:05:04.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.755+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94267186656112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2d1360 con 0x7f2a04076040 2024-05-08T18:05:04.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.756+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94267186656112] v27'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a08040040 con 0x7f2a04076040 2024-05-08T18:05:04.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.756+0000 7f2a1392f800 20 remove_watcher() i=0 2024-05-08T18:05:04.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.756+0000 7f2a1392f800 2 removed watcher, disabling cache 2024-05-08T18:05:04.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.756+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94267186659936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2d1360 con 0x55bc4a2bdc40 2024-05-08T18:05:04.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.757+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94267186659936] v27'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a08040040 con 0x55bc4a2bdc40 2024-05-08T18:05:04.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.757+0000 7f2a1392f800 20 remove_watcher() i=1 2024-05-08T18:05:04.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.757+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94267186653600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2cc260 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.758+0000 7f2a0eebd640 1 -- 172.21.15.50:0/3897975864 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94267186653600] v27'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a04081070 con 0x55bc4a2c6fc0 2024-05-08T18:05:04.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.758+0000 7f2a1392f800 20 remove_watcher() i=2 2024-05-08T18:05:04.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.758+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94267186666512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2ca9a0 con 0x7f2a04076040 2024-05-08T18:05:04.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.759+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94267186666512] v27'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a08040040 con 0x7f2a04076040 2024-05-08T18:05:04.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.759+0000 7f2a1392f800 20 remove_watcher() i=3 2024-05-08T18:05:04.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.759+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94267186669920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2cc260 con 0x55bc4a2bdc40 2024-05-08T18:05:04.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.760+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94267186669920] v27'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a08040040 con 0x55bc4a2bdc40 2024-05-08T18:05:04.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.760+0000 7f2a1392f800 20 remove_watcher() i=4 2024-05-08T18:05:04.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.760+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94267186668800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2cdc10 con 0x55bc4a2c23c0 2024-05-08T18:05:04.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.761+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94267186668800] v27'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.761+0000 7f2a1392f800 20 remove_watcher() i=5 2024-05-08T18:05:04.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.761+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94267186676064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2ca9a0 con 0x55bc4a2c23c0 2024-05-08T18:05:04.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.762+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94267186676064] v27'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.762+0000 7f2a1392f800 20 remove_watcher() i=6 2024-05-08T18:05:04.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.762+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94267186679616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bc4a2ce960 con 0x55bc4a2c23c0 2024-05-08T18:05:04.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.763+0000 7f2a0f6be640 1 -- 172.21.15.50:0/3897975864 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94267186679616] v27'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a0006f020 con 0x55bc4a2c23c0 2024-05-08T18:05:04.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.763+0000 7f2a1392f800 20 remove_watcher() i=7 2024-05-08T18:05:04.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.763+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bc4a2c23c0 msgr2=0x55bc4a2c47b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.763+0000 7f2a1392f800 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bc4a2c23c0 0x55bc4a2c47b0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.763+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bc4a2bdc40 msgr2=0x55bc4a2c0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.763+0000 7f2a1392f800 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bc4a2bdc40 0x55bc4a2c0100 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.763+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2a04076040 msgr2=0x7f2a04083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.763+0000 7f2a1392f800 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2a04076040 0x7f2a04083410 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.763+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bc4a2c6fc0 msgr2=0x55bc4a2c9480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.763+0000 7f2a1392f800 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bc4a2c6fc0 0x55bc4a2c9480 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.763+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f28d803bba0 msgr2=0x7f28d803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.763+0000 7f2a1392f800 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f28d803bba0 0x7f28d803e060 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2a04052fd0 tx=0x7f2a0404f000 comp rx=0 tx=0).stop 2024-05-08T18:05:04.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a2ab1b0 msgr2=0x55bc4a2b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a0febf640 1 -- 172.21.15.50:0/3897975864 reap_dead start 2024-05-08T18:05:04.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a2ab1b0 0x55bc4a2b2520 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000bbf0 tx=0x7f2a0003a9b0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 shutdown_connections 2024-05-08T18:05:04.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 --2- 172.21.15.50:0/3897975864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a2ab1b0 0x55bc4a2b2520 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 >> 172.21.15.50:0/3897975864 conn(0x55bc4a2a74d0 msgr2=0x55bc4a2a4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 shutdown_connections 2024-05-08T18:05:04.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 -- 172.21.15.50:0/3897975864 wait complete. 2024-05-08T18:05:04.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 -- 172.21.15.50:0/2782400280 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f28e803bba0 msgr2=0x7f28e803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 --2- 172.21.15.50:0/2782400280 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f28e803bba0 0x7f28e803e060 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55bc4a281c50 tx=0x7f2a04005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:04.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 -- 172.21.15.50:0/2782400280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a184d40 msgr2=0x55bc4a1831b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 --2- 172.21.15.50:0/2782400280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a184d40 0x55bc4a1831b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2a000517c0 tx=0x7f2a0001ab20 comp rx=0 tx=0).stop 2024-05-08T18:05:04.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 -- 172.21.15.50:0/2782400280 shutdown_connections 2024-05-08T18:05:04.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 --2- 172.21.15.50:0/2782400280 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f28e803bba0 0x7f28e803e060 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 --2- 172.21.15.50:0/2782400280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a184d40 0x55bc4a1831b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 -- 172.21.15.50:0/2782400280 >> 172.21.15.50:0/2782400280 conn(0x55bc4a28fd90 msgr2=0x55bc4a290190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 -- 172.21.15.50:0/2782400280 shutdown_connections 2024-05-08T18:05:04.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 -- 172.21.15.50:0/2782400280 wait complete. 2024-05-08T18:05:04.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.764+0000 7f2a1392f800 1 -- 172.21.15.50:0/2630968313 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bc4a287020 msgr2=0x55bc4a289410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.765+0000 7f2a1392f800 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bc4a287020 0x55bc4a289410 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.765+0000 7f2a1392f800 1 -- 172.21.15.50:0/2630968313 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bc4a188cb0 msgr2=0x55bc4a284760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.765+0000 7f2a1392f800 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bc4a188cb0 0x55bc4a284760 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.765+0000 7f2a1392f800 1 -- 172.21.15.50:0/2630968313 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bc4a28af10 msgr2=0x55bc4a28d300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.765+0000 7f2a1392f800 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bc4a28af10 0x55bc4a28d300 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.765+0000 7f2a1392f800 1 -- 172.21.15.50:0/2630968313 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f28f403bba0 msgr2=0x7f28f403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.765+0000 7f2a1392f800 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f28f403bba0 0x7f28f403e060 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2a04054000 tx=0x7f2a0404e000 comp rx=0 tx=0).stop 2024-05-08T18:05:04.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.765+0000 7f2a1392f800 1 -- 172.21.15.50:0/2630968313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a1851a0 msgr2=0x55bc4a27d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.765+0000 7f2a1392f800 1 --2- 172.21.15.50:0/2630968313 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bc4a1851a0 0x55bc4a27d570 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55bc4a276780 tx=0x7f2a00000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.765+0000 7f2a0febf640 1 -- 172.21.15.50:0/2630968313 reap_dead start 2024-05-08T18:05:04.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.765+0000 7f2a1392f800 1 -- 172.21.15.50:0/2630968313 shutdown_connections 2024-05-08T18:05:04.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.765+0000 7f2a1392f800 1 -- 172.21.15.50:0/2630968313 >> 172.21.15.50:0/2630968313 conn(0x55bc4a17ead0 msgr2=0x55bc4a26fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.765+0000 7f2a1392f800 1 -- 172.21.15.50:0/2630968313 shutdown_connections 2024-05-08T18:05:04.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.765+0000 7f2a1392f800 1 -- 172.21.15.50:0/2630968313 wait complete. 2024-05-08T18:05:04.773 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:04.773 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2024-05-08T18:05:04.773 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2024-05-08T18:05:04.773 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key create --uid foo --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh 2024-05-08T18:05:04.823 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:04.823 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:04.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.826+0000 7f3550933800 1 Processor -- start 2024-05-08T18:05:04.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.826+0000 7f3550933800 1 -- start start 2024-05-08T18:05:04.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.826+0000 7f3550933800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55a51a95bcc0 0x55a51a9f05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.826+0000 7f3550933800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a51a95c170 con 0x55a51a95d050 2024-05-08T18:05:04.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.826+0000 7f3550933800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55a51a887e20 con 0x55a51a95bcc0 2024-05-08T18:05:04.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.826+0000 7f354beea640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55a51a95bcc0 0x55a51a9f05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.826+0000 7f354c6eb640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55a51a95d050 0x55a51a95b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:42978/0 (socket says 172.21.15.50:42978) 2024-05-08T18:05:04.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.826+0000 7f354c6eb640 1 -- 172.21.15.50:0/3679778178 learned_addr learned my addr 172.21.15.50:0/3679778178 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:04.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.826+0000 7f354beea640 1 -- 172.21.15.50:0/3679778178 >> v1:172.21.15.46:6789/0 conn(0x55a51a95d050 legacy=0x55a51a95b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.827+0000 7f354b6e9640 1 -- 172.21.15.50:0/3679778178 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2311050926 0 0) 0x55a51a95c170 con 0x55a51a95d050 2024-05-08T18:05:04.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.827+0000 7f354beea640 1 -- 172.21.15.50:0/3679778178 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a51a95c390 con 0x55a51a95bcc0 2024-05-08T18:05:04.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.829+0000 7f354beea640 1 --2- 172.21.15.50:0/3679778178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 0x55a51a9f05d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f354000a470 tx=0x7f354001cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5fe6a984c3c81b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.829+0000 7f354b6e9640 1 -- 172.21.15.50:0/3679778178 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3540057070 con 0x55a51a95bcc0 2024-05-08T18:05:04.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.829+0000 7f354b6e9640 1 -- 172.21.15.50:0/3679778178 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3540052570 con 0x55a51a95bcc0 2024-05-08T18:05:04.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.829+0000 7f354b6e9640 1 -- 172.21.15.50:0/3679778178 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3540052870 con 0x55a51a95bcc0 2024-05-08T18:05:04.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.829+0000 7f3550933800 1 -- 172.21.15.50:0/3679778178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 msgr2=0x55a51a9f05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.829+0000 7f3550933800 1 --2- 172.21.15.50:0/3679778178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 0x55a51a9f05d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f354000a470 tx=0x7f354001cf20 comp rx=0 tx=0).stop 2024-05-08T18:05:04.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.829+0000 7f3550933800 1 -- 172.21.15.50:0/3679778178 shutdown_connections 2024-05-08T18:05:04.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.829+0000 7f3550933800 1 --2- 172.21.15.50:0/3679778178 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 0x55a51a9f05d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.829+0000 7f3550933800 1 -- 172.21.15.50:0/3679778178 >> 172.21.15.50:0/3679778178 conn(0x55a51a9e2c90 msgr2=0x55a51a9e50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.829+0000 7f3550933800 1 -- 172.21.15.50:0/3679778178 shutdown_connections 2024-05-08T18:05:04.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.829+0000 7f3550933800 1 -- 172.21.15.50:0/3679778178 wait complete. 2024-05-08T18:05:04.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.844+0000 7f3550933800 1 Processor -- start 2024-05-08T18:05:04.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.845+0000 7f3550933800 1 -- start start 2024-05-08T18:05:04.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.845+0000 7f3550933800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 0x55a51aadac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.845+0000 7f3550933800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a51a932a30 con 0x55a51a95bcc0 2024-05-08T18:05:04.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.845+0000 7f354c6eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 0x55a51aadac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.845+0000 7f354c6eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 0x55a51aadac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36544/0 (socket says 172.21.15.50:36544) 2024-05-08T18:05:04.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.845+0000 7f354c6eb640 1 -- 172.21.15.50:0/3885095785 learned_addr learned my addr 172.21.15.50:0/3885095785 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.845+0000 7f354c6eb640 1 -- 172.21.15.50:0/3885095785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a51aadb250 con 0x55a51a95bcc0 2024-05-08T18:05:04.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.846+0000 7f354c6eb640 1 --2- 172.21.15.50:0/3885095785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 0x55a51aadac80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f353c001db0 tx=0x7f353c00b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db16b89175073fc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.846+0000 7f343a60c640 1 -- 172.21.15.50:0/3885095785 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f353c012020 con 0x55a51a95bcc0 2024-05-08T18:05:04.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.846+0000 7f343a60c640 1 -- 172.21.15.50:0/3885095785 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353c00e5c0 con 0x55a51a95bcc0 2024-05-08T18:05:04.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.846+0000 7f343a60c640 1 -- 172.21.15.50:0/3885095785 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f353c00e8e0 con 0x55a51a95bcc0 2024-05-08T18:05:04.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.846+0000 7f3550933800 1 -- 172.21.15.50:0/3885095785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 msgr2=0x55a51aadac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.846+0000 7f3550933800 1 --2- 172.21.15.50:0/3885095785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 0x55a51aadac80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f353c001db0 tx=0x7f353c00b6c0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.847+0000 7f3550933800 1 -- 172.21.15.50:0/3885095785 shutdown_connections 2024-05-08T18:05:04.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.847+0000 7f3550933800 1 --2- 172.21.15.50:0/3885095785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 0x55a51aadac80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.847+0000 7f3550933800 1 -- 172.21.15.50:0/3885095785 >> 172.21.15.50:0/3885095785 conn(0x55a51a9e2c90 msgr2=0x55a51aad65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.847+0000 7f3550933800 1 -- 172.21.15.50:0/3885095785 shutdown_connections 2024-05-08T18:05:04.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.847+0000 7f3550933800 1 -- 172.21.15.50:0/3885095785 wait complete. 2024-05-08T18:05:04.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.847+0000 7f3550933800 1 Processor -- start 2024-05-08T18:05:04.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.847+0000 7f3550933800 1 -- start start 2024-05-08T18:05:04.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.847+0000 7f3550933800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 0x55a51aae27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.847+0000 7f3550933800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a51a9f0e10 con 0x55a51a95bcc0 2024-05-08T18:05:04.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.847+0000 7f354c6eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 0x55a51aae27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.847+0000 7f354c6eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 0x55a51aae27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36558/0 (socket says 172.21.15.50:36558) 2024-05-08T18:05:04.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.847+0000 7f354c6eb640 1 -- 172.21.15.50:0/2301741688 learned_addr learned my addr 172.21.15.50:0/2301741688 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.848+0000 7f354c6eb640 1 -- 172.21.15.50:0/2301741688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a51aae2d10 con 0x55a51a95bcc0 2024-05-08T18:05:04.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.848+0000 7f354c6eb640 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 0x55a51aae27d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f353c000c00 tx=0x7f353c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.848+0000 7f3438e09640 1 -- 172.21.15.50:0/2301741688 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f353c012030 con 0x55a51a95bcc0 2024-05-08T18:05:04.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.848+0000 7f3438e09640 1 -- 172.21.15.50:0/2301741688 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353c01a750 con 0x55a51a95bcc0 2024-05-08T18:05:04.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.848+0000 7f3550933800 1 -- 172.21.15.50:0/2301741688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a51aae2fa0 con 0x55a51a95bcc0 2024-05-08T18:05:04.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.848+0000 7f3438e09640 1 -- 172.21.15.50:0/2301741688 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f353c01aa70 con 0x55a51a95bcc0 2024-05-08T18:05:04.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.848+0000 7f3550933800 1 -- 172.21.15.50:0/2301741688 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a51aae5c50 con 0x55a51a95bcc0 2024-05-08T18:05:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.849+0000 7f3438e09640 1 -- 172.21.15.50:0/2301741688 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f353c015020 con 0x55a51a95bcc0 2024-05-08T18:05:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.849+0000 7f3438e09640 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f343403bba0 0x7f343403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.849+0000 7f3438e09640 1 -- 172.21.15.50:0/2301741688 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f353c020030 con 0x55a51a95bcc0 2024-05-08T18:05:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.849+0000 7f354beea640 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f343403bba0 0x7f343403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.849+0000 7f3550933800 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a51a95d050 0x55a51aae9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.849+0000 7f3550933800 1 -- 172.21.15.50:0/2301741688 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51aae9f50 con 0x55a51a95d050 2024-05-08T18:05:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.849+0000 7f354ceec640 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a51a95d050 0x55a51aae9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.850+0000 7f354beea640 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f343403bba0 0x7f343403e060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3540054000 tx=0x7f354004e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.850+0000 7f354ceec640 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a51a95d050 0x55a51aae9a10 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.850+0000 7f354ceec640 1 -- 172.21.15.50:0/2301741688 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51a95d050 2024-05-08T18:05:04.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.850+0000 7f3550933800 1 -- 172.21.15.50:0/2301741688 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51aae6fc0 con 0x55a51a95d050 2024-05-08T18:05:04.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.850+0000 7f354ceec640 1 -- 172.21.15.50:0/2301741688 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51a95d050 2024-05-08T18:05:04.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.850+0000 7f3550933800 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a51aaec270 0x55a51aaee660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.850+0000 7f3550933800 1 -- 172.21.15.50:0/2301741688 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51aaeed50 con 0x55a51aaec270 2024-05-08T18:05:04.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.851+0000 7f354c6eb640 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a51aaec270 0x55a51aaee660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.851+0000 7f354c6eb640 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a51aaec270 0x55a51aaee660 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.851+0000 7f354c6eb640 1 -- 172.21.15.50:0/2301741688 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f353c0430c0 con 0x55a51aaec270 2024-05-08T18:05:04.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.851+0000 7f3550933800 1 -- 172.21.15.50:0/2301741688 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51aaeb110 con 0x55a51a95d050 2024-05-08T18:05:04.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.852+0000 7f354ceec640 1 -- 172.21.15.50:0/2301741688 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51a95d050 2024-05-08T18:05:04.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.852+0000 7f3550933800 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a51aaf01b0 0x55a51aaf25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.852+0000 7f3550933800 1 -- 172.21.15.50:0/2301741688 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51aaf2ca0 con 0x55a51aaf01b0 2024-05-08T18:05:04.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.852+0000 7f354beea640 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a51aaf01b0 0x55a51aaf25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.852+0000 7f354beea640 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a51aaf01b0 0x55a51aaf25a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.852+0000 7f354beea640 1 -- 172.21.15.50:0/2301741688 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3540052940 con 0x55a51aaf01b0 2024-05-08T18:05:04.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.853+0000 7f3550933800 1 Processor -- start 2024-05-08T18:05:04.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.853+0000 7f3550933800 1 -- start start 2024-05-08T18:05:04.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.853+0000 7f3550933800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab00f90 0x55a51ab013b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.853+0000 7f3550933800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a51a8d9330 con 0x55a51ab00f90 2024-05-08T18:05:04.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.853+0000 7f354c6eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab00f90 0x55a51ab013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.853+0000 7f354c6eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab00f90 0x55a51ab013b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36562/0 (socket says 172.21.15.50:36562) 2024-05-08T18:05:04.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.853+0000 7f354c6eb640 1 -- 172.21.15.50:0/2292700628 learned_addr learned my addr 172.21.15.50:0/2292700628 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.853+0000 7f354c6eb640 1 -- 172.21.15.50:0/2292700628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a51ab018f0 con 0x55a51ab00f90 2024-05-08T18:05:04.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.854+0000 7f354c6eb640 1 --2- 172.21.15.50:0/2292700628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab00f90 0x55a51ab013b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f353c00b700 tx=0x7f353c040f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.854+0000 7f3431ffb640 1 -- 172.21.15.50:0/2292700628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f353c049e20 con 0x55a51ab00f90 2024-05-08T18:05:04.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.854+0000 7f3431ffb640 1 -- 172.21.15.50:0/2292700628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353c049fc0 con 0x55a51ab00f90 2024-05-08T18:05:04.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.854+0000 7f3550933800 1 -- 172.21.15.50:0/2292700628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a51ab02360 con 0x55a51ab00f90 2024-05-08T18:05:04.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.854+0000 7f3431ffb640 1 -- 172.21.15.50:0/2292700628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f353c04a2a0 con 0x55a51ab00f90 2024-05-08T18:05:04.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.854+0000 7f3550933800 1 -- 172.21.15.50:0/2292700628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a51ab02860 con 0x55a51ab00f90 2024-05-08T18:05:04.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.854+0000 7f3431ffb640 1 -- 172.21.15.50:0/2292700628 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f353c0167f0 con 0x55a51ab00f90 2024-05-08T18:05:04.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.855+0000 7f3431ffb640 1 --2- 172.21.15.50:0/2292700628 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f342403bba0 0x7f342403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.855+0000 7f3431ffb640 1 -- 172.21.15.50:0/2292700628 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f353c049070 con 0x55a51ab00f90 2024-05-08T18:05:04.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.855+0000 7f354beea640 1 --2- 172.21.15.50:0/2292700628 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f342403bba0 0x7f342403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.855+0000 7f3550933800 1 Processor -- start 2024-05-08T18:05:04.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.855+0000 7f3550933800 1 -- start start 2024-05-08T18:05:04.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.855+0000 7f354beea640 1 --2- 172.21.15.50:0/2292700628 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f342403bba0 0x7f342403e060 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55a51aaea820 tx=0x7f3540005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.855+0000 7f3550933800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab0e0b0 0x55a51ab0e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.855+0000 7f3550933800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a51aadb5d0 con 0x55a51ab0e0b0 2024-05-08T18:05:04.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.856+0000 7f354c6eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab0e0b0 0x55a51ab0e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.856+0000 7f354c6eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab0e0b0 0x55a51ab0e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36568/0 (socket says 172.21.15.50:36568) 2024-05-08T18:05:04.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.856+0000 7f354c6eb640 1 -- 172.21.15.50:0/2969143512 learned_addr learned my addr 172.21.15.50:0/2969143512 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.856+0000 7f354c6eb640 1 -- 172.21.15.50:0/2969143512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a51ab0eaa0 con 0x55a51ab0e0b0 2024-05-08T18:05:04.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.856+0000 7f354c6eb640 1 --2- 172.21.15.50:0/2969143512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab0e0b0 0x55a51ab0e4d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f353c016d70 tx=0x7f353c0499c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d072cd970089f93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.856+0000 7f342b7fe640 1 -- 172.21.15.50:0/2969143512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f353c060070 con 0x55a51ab0e0b0 2024-05-08T18:05:04.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.856+0000 7f342b7fe640 1 -- 172.21.15.50:0/2969143512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353c03c070 con 0x55a51ab0e0b0 2024-05-08T18:05:04.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.856+0000 7f342b7fe640 1 -- 172.21.15.50:0/2969143512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f353c03dce0 con 0x55a51ab0e0b0 2024-05-08T18:05:04.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.857+0000 7f3550933800 1 -- 172.21.15.50:0/2969143512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab0e0b0 msgr2=0x55a51ab0e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.857+0000 7f3550933800 1 --2- 172.21.15.50:0/2969143512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab0e0b0 0x55a51ab0e4d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f353c016d70 tx=0x7f353c0499c0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.857+0000 7f3550933800 1 -- 172.21.15.50:0/2969143512 shutdown_connections 2024-05-08T18:05:04.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.857+0000 7f3550933800 1 --2- 172.21.15.50:0/2969143512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab0e0b0 0x55a51ab0e4d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.857+0000 7f3550933800 1 -- 172.21.15.50:0/2969143512 >> 172.21.15.50:0/2969143512 conn(0x55a51ab0a410 msgr2=0x55a51ab08470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.857+0000 7f3550933800 1 -- 172.21.15.50:0/2969143512 shutdown_connections 2024-05-08T18:05:04.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.857+0000 7f3550933800 1 -- 172.21.15.50:0/2969143512 wait complete. 2024-05-08T18:05:04.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.857+0000 7f3550933800 1 Processor -- start 2024-05-08T18:05:04.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.857+0000 7f3550933800 1 -- start start 2024-05-08T18:05:04.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.857+0000 7f3550933800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab0e0b0 0x55a51ab15480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.857+0000 7f3550933800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a51ab0f150 con 0x55a51ab0e0b0 2024-05-08T18:05:04.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.857+0000 7f354c6eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab0e0b0 0x55a51ab15480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.857+0000 7f354c6eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab0e0b0 0x55a51ab15480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36572/0 (socket says 172.21.15.50:36572) 2024-05-08T18:05:04.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.857+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 learned_addr learned my addr 172.21.15.50:0/3238547732 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.858+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a51ab159c0 con 0x55a51ab0e0b0 2024-05-08T18:05:04.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.858+0000 7f354c6eb640 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab0e0b0 0x55a51ab15480 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f353c055010 tx=0x7f353c04fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.858+0000 7f3429ffb640 1 -- 172.21.15.50:0/3238547732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f353c06a070 con 0x55a51ab0e0b0 2024-05-08T18:05:04.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.858+0000 7f3429ffb640 1 -- 172.21.15.50:0/3238547732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353c03de40 con 0x55a51ab0e0b0 2024-05-08T18:05:04.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.858+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a51ab159c0 con 0x55a51ab0e0b0 2024-05-08T18:05:04.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.858+0000 7f3429ffb640 1 -- 172.21.15.50:0/3238547732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f353c060040 con 0x55a51ab0e0b0 2024-05-08T18:05:04.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.858+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a51ab159c0 con 0x55a51ab0e0b0 2024-05-08T18:05:04.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.859+0000 7f3429ffb640 1 -- 172.21.15.50:0/3238547732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f353c06d070 con 0x55a51ab0e0b0 2024-05-08T18:05:04.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.859+0000 7f3429ffb640 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f341c03bb50 0x7f341c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.859+0000 7f3429ffb640 1 -- 172.21.15.50:0/3238547732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f353c058070 con 0x55a51ab0e0b0 2024-05-08T18:05:04.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.859+0000 7f354beea640 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f341c03bb50 0x7f341c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.860+0000 7f3550933800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.860+0000 7f354beea640 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f341c03bb50 0x7f341c03e010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3540052fd0 tx=0x7f354004f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.860+0000 7f3550933800 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a51ab20a90 0x55a51ab22f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.860+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab236a0 con 0x55a51ab20a90 2024-05-08T18:05:04.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.860+0000 7f354ceec640 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a51ab20a90 0x55a51ab22f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.860+0000 7f354ceec640 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a51ab20a90 0x55a51ab22f50 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.860+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51ab20a90 2024-05-08T18:05:04.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.860+0000 7f3550933800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.860+0000 7f3550933800 20 realm 2024-05-08T18:05:04.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.860+0000 7f3550933800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.860+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab23ab0 con 0x55a51ab20a90 2024-05-08T18:05:04.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.861+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51ab20a90 2024-05-08T18:05:04.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.861+0000 7f3550933800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.861+0000 7f3550933800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:04.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.861+0000 7f3550933800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.861+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab24380 con 0x55a51ab20a90 2024-05-08T18:05:04.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.861+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51ab20a90 2024-05-08T18:05:04.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.861+0000 7f3550933800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.861+0000 7f3550933800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.861+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab24720 con 0x55a51ab20a90 2024-05-08T18:05:04.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.861+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51ab20a90 2024-05-08T18:05:04.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.861+0000 7f3550933800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:04.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.861+0000 7f3550933800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.861+0000 7f3550933800 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a51ab25480 0x55a51ab27870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.861+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab27e80 con 0x55a51ab25480 2024-05-08T18:05:04.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.861+0000 7f354c6eb640 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a51ab25480 0x55a51ab27870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.862+0000 7f354c6eb640 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a51ab25480 0x55a51ab27870 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.862+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.862+0000 7f3550933800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:04.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.862+0000 7f3550933800 20 searching for the correct realm 2024-05-08T18:05:04.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.862+0000 7f3550933800 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a51ab29e30 0x55a51ab2c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.862+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab2c9c0 con 0x55a51ab29e30 2024-05-08T18:05:04.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.862+0000 7f354beea640 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a51ab29e30 0x55a51ab2c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.863+0000 7f354beea640 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a51ab29e30 0x55a51ab2c2f0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.863+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.863+0000 7f354beea640 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3540076040 0x7f3540083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.863+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab2c9c0 con 0x7f3540076040 2024-05-08T18:05:04.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.863+0000 7f354ceec640 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3540076040 0x7f3540083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.863+0000 7f354ceec640 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3540076040 0x7f3540083410 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.864+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f354400a4a0 con 0x7f3540076040 2024-05-08T18:05:04.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.864+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab24720 con 0x55a51ab25480 2024-05-08T18:05:04.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.864+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.864+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab24720 con 0x55a51ab20a90 2024-05-08T18:05:04.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.864+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51ab20a90 2024-05-08T18:05:04.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.864+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab24720 con 0x55a51ab20a90 2024-05-08T18:05:04.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.864+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51ab20a90 2024-05-08T18:05:04.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.864+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab24720 con 0x55a51ab25480 2024-05-08T18:05:04.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.865+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.865+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab24720 con 0x55a51ab25480 2024-05-08T18:05:04.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.865+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.865+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab24720 con 0x7f3540076040 2024-05-08T18:05:04.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.865+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f354400a4a0 con 0x7f3540076040 2024-05-08T18:05:04.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.865+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab24720 con 0x55a51ab29e30 2024-05-08T18:05:04.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.865+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.865+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab24720 con 0x55a51ab20a90 2024-05-08T18:05:04.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.866+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51ab20a90 2024-05-08T18:05:04.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.866+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab24720 con 0x55a51ab25480 2024-05-08T18:05:04.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.866+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.866+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab24720 con 0x55a51ab20a90 2024-05-08T18:05:04.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.866+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51ab20a90 2024-05-08T18:05:04.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.866+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab24720 con 0x7f3540076040 2024-05-08T18:05:04.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.866+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f354400a4a0 con 0x7f3540076040 2024-05-08T18:05:04.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.866+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab24720 con 0x55a51ab29e30 2024-05-08T18:05:04.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.867+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.867+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x55a51ab29e30 2024-05-08T18:05:04.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.867+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.867+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x55a51ab25480 2024-05-08T18:05:04.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.867+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.867+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x55a51ab29e30 2024-05-08T18:05:04.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.868+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.868+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x55a51ab29e30 2024-05-08T18:05:04.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.868+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.868+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x55a51ab20a90 2024-05-08T18:05:04.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.868+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51ab20a90 2024-05-08T18:05:04.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.868+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x55a51ab20a90 2024-05-08T18:05:04.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.868+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51ab20a90 2024-05-08T18:05:04.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.868+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x55a51ab25480 2024-05-08T18:05:04.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.869+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.869+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x55a51ab29e30 2024-05-08T18:05:04.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.869+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.869+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x7f3540076040 2024-05-08T18:05:04.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.869+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f354400a4a0 con 0x7f3540076040 2024-05-08T18:05:04.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.869+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x55a51ab20a90 2024-05-08T18:05:04.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.869+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51ab20a90 2024-05-08T18:05:04.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.869+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x55a51ab25480 2024-05-08T18:05:04.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.870+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.870+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x55a51ab29e30 2024-05-08T18:05:04.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.870+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.870+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x55a51ab25480 2024-05-08T18:05:04.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.870+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.870+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x7f3540076040 2024-05-08T18:05:04.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.871+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f354400a4a0 con 0x7f3540076040 2024-05-08T18:05:04.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.871+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x55a51ab29e30 2024-05-08T18:05:04.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.871+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.871+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x55a51ab25480 2024-05-08T18:05:04.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.871+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.871+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x55a51ab20a90 2024-05-08T18:05:04.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.871+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51ab20a90 2024-05-08T18:05:04.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.871+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3540090040 con 0x7f3540076040 2024-05-08T18:05:04.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.871+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f354400a4a0 con 0x7f3540076040 2024-05-08T18:05:04.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.871+0000 7f3550933800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:04.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:04.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:04.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:04.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:04.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:04.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab28740 con 0x55a51ab20a90 2024-05-08T18:05:04.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51ab20a90 2024-05-08T18:05:04.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab28740 con 0x55a51ab20a90 2024-05-08T18:05:04.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f354400a4a0 con 0x55a51ab20a90 2024-05-08T18:05:04.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:04.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab28740 con 0x55a51ab29e30 2024-05-08T18:05:04.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:04.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 20 zone default found 2024-05-08T18:05:04.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 4 Realm: () 2024-05-08T18:05:04.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:04.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:04.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:04.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 20 zonegroup default 2024-05-08T18:05:04.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.872+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab29040 con 0x55a51ab25480 2024-05-08T18:05:04.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.873+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.873+0000 7f3550933800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.873+0000 7f3550933800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.873+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab2e850 con 0x55a51ab29e30 2024-05-08T18:05:04.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.873+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.873+0000 7f3550933800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.873+0000 7f3550933800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.873+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab2ece0 con 0x55a51ab29e30 2024-05-08T18:05:04.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.873+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.873+0000 7f3550933800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.873+0000 7f3550933800 20 started sync module instance, tier type = 2024-05-08T18:05:04.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.873+0000 7f3550933800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:04.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.873+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab2dcc0 con 0x7f3540076040 2024-05-08T18:05:04.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.874+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f354400a4a0 con 0x7f3540076040 2024-05-08T18:05:04.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.874+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94167605895648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab2dcc0 con 0x7f3540076040 2024-05-08T18:05:04.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.874+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab2f7a0 con 0x55a51ab20a90 2024-05-08T18:05:04.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.875+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3544040040 con 0x55a51ab20a90 2024-05-08T18:05:04.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.875+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94167605895648] v27'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3544040040 con 0x7f3540076040 2024-05-08T18:05:04.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.875+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94167605899952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab2f770 con 0x55a51ab20a90 2024-05-08T18:05:04.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.875+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab2d850 con 0x55a51ab29e30 2024-05-08T18:05:04.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.876+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94167605899952] v27'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3544040040 con 0x55a51ab20a90 2024-05-08T18:05:04.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.876+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.876+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94167605902224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab30710 con 0x55a51ab29e30 2024-05-08T18:05:04.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.876+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab310d0 con 0x7f3540076040 2024-05-08T18:05:04.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.877+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94167605902224] v27'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.878+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3544040040 con 0x7f3540076040 2024-05-08T18:05:04.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.878+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94167605906464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab314f0 con 0x7f3540076040 2024-05-08T18:05:04.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.878+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab31eb0 con 0x55a51ab20a90 2024-05-08T18:05:04.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.879+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3544040040 con 0x55a51ab20a90 2024-05-08T18:05:04.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.879+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94167605910016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab322d0 con 0x55a51ab20a90 2024-05-08T18:05:04.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.879+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab32d70 con 0x55a51ab25480 2024-05-08T18:05:04.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.879+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94167605906464] v27'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3544040040 con 0x7f3540076040 2024-05-08T18:05:04.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.880+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94167605910016] v27'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3544040040 con 0x55a51ab20a90 2024-05-08T18:05:04.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.880+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.880+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94167605907520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab322d0 con 0x55a51ab25480 2024-05-08T18:05:04.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.880+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab33610 con 0x55a51ab25480 2024-05-08T18:05:04.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.881+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.881+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94167605916480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab33dd0 con 0x55a51ab25480 2024-05-08T18:05:04.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.881+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab348c0 con 0x55a51ab25480 2024-05-08T18:05:04.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.881+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94167605907520] v27'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.882+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94167605916480] v27'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.882+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.882+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94167605920656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab33820 con 0x55a51ab25480 2024-05-08T18:05:04.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.882+0000 7f3550933800 20 add_watcher() i=0 2024-05-08T18:05:04.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.882+0000 7f3550933800 20 add_watcher() i=1 2024-05-08T18:05:04.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.882+0000 7f3550933800 20 add_watcher() i=2 2024-05-08T18:05:04.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.882+0000 7f3550933800 20 add_watcher() i=3 2024-05-08T18:05:04.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.882+0000 7f3550933800 20 add_watcher() i=4 2024-05-08T18:05:04.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.882+0000 7f3550933800 20 add_watcher() i=5 2024-05-08T18:05:04.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.882+0000 7f3550933800 20 add_watcher() i=6 2024-05-08T18:05:04.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.883+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94167605920656] v27'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.883+0000 7f3550933800 20 add_watcher() i=7 2024-05-08T18:05:04.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.883+0000 7f3550933800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:04.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.883+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab378b0 con 0x55a51ab20a90 2024-05-08T18:05:04.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.883+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f3544040040 con 0x55a51ab20a90 2024-05-08T18:05:04.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.883+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94167605911072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab34d60 con 0x55a51ab20a90 2024-05-08T18:05:04.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.884+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94167605911072] v27'64 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3544040040 con 0x55a51ab20a90 2024-05-08T18:05:04.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.884+0000 7f34027e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:04.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.884+0000 7f3550933800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:04.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.884+0000 7f3550933800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:04.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.885+0000 7f3550933800 5 note: GC not initialized 2024-05-08T18:05:04.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.885+0000 7f34017e2640 20 reqs_thread_entry: start 2024-05-08T18:05:04.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.885+0000 7f3550933800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:04.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.885+0000 7f3550933800 20 Filter name: none 2024-05-08T18:05:04.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.885+0000 7f3372ffd640 20 reqs_thread_entry: start 2024-05-08T18:05:04.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.885+0000 7f3550933800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:04.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.885+0000 7f3550933800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.885+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab6adf0 con 0x55a51ab20a90 2024-05-08T18:05:04.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.885+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f3544040040 con 0x55a51ab20a90 2024-05-08T18:05:04.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.885+0000 7f3550933800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:04.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.885+0000 7f3550933800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:04.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.885+0000 7f3550933800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:04.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.885+0000 7f3550933800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:04.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.886+0000 7f3550933800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-05-08T18:05:04.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.886+0000 7f3550933800 20 rados->read ofs=0 len=0 2024-05-08T18:05:04.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.886+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab6b860 con 0x55a51ab25480 2024-05-08T18:05:04.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.886+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.886+0000 7f3550933800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:04.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.886+0000 7f3550933800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-05-08T18:05:04.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.886+0000 7f3550933800 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-05-08T18:05:04.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.886+0000 7f3550933800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-05-08T18:05:04.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.886+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~502 in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab71260 con 0x55a51ab20a90 2024-05-08T18:05:04.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.887+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~502] v27'4 uv4 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f3544040040 con 0x55a51ab20a90 2024-05-08T18:05:04.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.887+0000 7f3550933800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:04.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.887+0000 7f3550933800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:04.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.887+0000 7f3550933800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-05-08T18:05:04.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.887+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94167606168608 in=690b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab6c8a0 con 0x55a51ab20a90 2024-05-08T18:05:04.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.888+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== watch-notify(notify (1) cookie 94167605910016 notify 115964116995 ret 0) ==== 720+0+0 (crc 0 0 0) 0x7f3544038450 con 0x55a51ab20a90 2024-05-08T18:05:04.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.888+0000 7f342b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116995 cookie 94167605910016 notifier 4333 bl.length()=678 2024-05-08T18:05:04.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.888+0000 7f342b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:04.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.888+0000 7f342b7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:04.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.888+0000 7f342b7fe640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f3410003ea0 con 0x55a51ab20a90 2024-05-08T18:05:04.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.888+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(65 notify.4 [notify cookie 94167606168608 out=8b] v0'0 uv45 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3544040040 con 0x55a51ab20a90 2024-05-08T18:05:04.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.888+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== watch-notify(notify_complete (2) cookie 94167606168608 notify 115964116995 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f3544038620 con 0x55a51ab20a90 2024-05-08T18:05:04.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.888+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3544040040 con 0x55a51ab20a90 2024-05-08T18:05:04.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.888+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab70c20 con 0x55a51ab25480 2024-05-08T18:05:04.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.889+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v27'1 uv1 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.889+0000 7f3550933800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-05-08T18:05:04.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.889+0000 7f3550933800 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-05-08T18:05:04.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.889+0000 7f3550933800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-05-08T18:05:04.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.889+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94167606168608 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab6c3f0 con 0x55a51ab29e30 2024-05-08T18:05:04.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.890+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== watch-notify(notify (1) cookie 94167605902224 notify 115964116992 ret 0) ==== 219+0+0 (crc 0 0 0) 0x7f354008a040 con 0x55a51ab29e30 2024-05-08T18:05:04.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.890+0000 7f342affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116992 cookie 94167605902224 notifier 4333 bl.length()=177 2024-05-08T18:05:04.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.890+0000 7f342affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-05-08T18:05:04.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.890+0000 7f342affd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-05-08T18:05:04.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.890+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(68 notify.2 [notify cookie 94167606168608 out=8b] v0'0 uv45 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.890+0000 7f342affd640 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f3380003400 con 0x55a51ab29e30 2024-05-08T18:05:04.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.890+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.890+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== watch-notify(notify_complete (2) cookie 94167606168608 notify 115964116992 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a51a95c390 con 0x55a51ab29e30 2024-05-08T18:05:04.898 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"},{"user":"foo","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-05-08T18:05:04.887254Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:04.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.891+0000 7f34027e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:04.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.891+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94167605911072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab6c820 con 0x55a51ab20a90 2024-05-08T18:05:04.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.892+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94167605911072] v27'65 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3544040040 con 0x55a51ab20a90 2024-05-08T18:05:04.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.892+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94167605895648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab31e20 con 0x7f3540076040 2024-05-08T18:05:04.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.893+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94167605895648] v27'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3544040040 con 0x7f3540076040 2024-05-08T18:05:04.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.893+0000 7f3550933800 20 remove_watcher() i=0 2024-05-08T18:05:04.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.893+0000 7f3550933800 2 removed watcher, disabling cache 2024-05-08T18:05:04.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.893+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94167605899952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab31e20 con 0x55a51ab20a90 2024-05-08T18:05:04.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.894+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94167605899952] v27'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3544040040 con 0x55a51ab20a90 2024-05-08T18:05:04.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.894+0000 7f3550933800 20 remove_watcher() i=1 2024-05-08T18:05:04.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.894+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94167605902224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab31e20 con 0x55a51ab29e30 2024-05-08T18:05:04.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.895+0000 7f354beea640 1 -- 172.21.15.50:0/3238547732 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94167605902224] v27'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3540081070 con 0x55a51ab29e30 2024-05-08T18:05:04.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.895+0000 7f3550933800 20 remove_watcher() i=2 2024-05-08T18:05:04.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.895+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94167605906464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab2fb90 con 0x7f3540076040 2024-05-08T18:05:04.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.896+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94167605906464] v27'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3544040040 con 0x7f3540076040 2024-05-08T18:05:04.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.896+0000 7f3550933800 20 remove_watcher() i=3 2024-05-08T18:05:04.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.896+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94167605910016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab31e20 con 0x55a51ab20a90 2024-05-08T18:05:04.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.897+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94167605910016] v27'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3544040040 con 0x55a51ab20a90 2024-05-08T18:05:04.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.897+0000 7f3550933800 20 remove_watcher() i=4 2024-05-08T18:05:04.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.897+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94167605907520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab30c20 con 0x55a51ab25480 2024-05-08T18:05:04.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.898+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94167605907520] v27'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.898+0000 7f3550933800 20 remove_watcher() i=5 2024-05-08T18:05:04.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.898+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94167605916480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab2fb90 con 0x55a51ab25480 2024-05-08T18:05:04.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.899+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94167605916480] v27'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.899+0000 7f3550933800 20 remove_watcher() i=6 2024-05-08T18:05:04.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.899+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94167605920656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a51ab33340 con 0x55a51ab25480 2024-05-08T18:05:04.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f354c6eb640 1 -- 172.21.15.50:0/3238547732 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94167605920656] v27'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f353c076070 con 0x55a51ab25480 2024-05-08T18:05:04.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 20 remove_watcher() i=7 2024-05-08T18:05:04.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a51ab25480 msgr2=0x55a51ab27870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a51ab25480 0x55a51ab27870 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a51ab20a90 msgr2=0x55a51ab22f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a51ab20a90 0x55a51ab22f50 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3540076040 msgr2=0x7f3540083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3540076040 0x7f3540083410 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a51ab29e30 msgr2=0x55a51ab2c2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a51ab29e30 0x55a51ab2c2f0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f341c03bb50 msgr2=0x7f341c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f341c03bb50 0x7f341c03e010 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3540052fd0 tx=0x7f354004f000 comp rx=0 tx=0).stop 2024-05-08T18:05:04.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab0e0b0 msgr2=0x55a51ab15480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f354ceec640 1 -- 172.21.15.50:0/3238547732 reap_dead start 2024-05-08T18:05:04.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab0e0b0 0x55a51ab15480 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f353c055010 tx=0x7f353c04fcf0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 shutdown_connections 2024-05-08T18:05:04.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 --2- 172.21.15.50:0/3238547732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab0e0b0 0x55a51ab15480 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 >> 172.21.15.50:0/3238547732 conn(0x55a51ab0a410 msgr2=0x55a51ab07a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 shutdown_connections 2024-05-08T18:05:04.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.900+0000 7f3550933800 1 -- 172.21.15.50:0/3238547732 wait complete. 2024-05-08T18:05:04.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 -- 172.21.15.50:0/2292700628 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f342403bba0 msgr2=0x7f342403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 --2- 172.21.15.50:0/2292700628 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f342403bba0 0x7f342403e060 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55a51aaea820 tx=0x7f3540005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:04.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 -- 172.21.15.50:0/2292700628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab00f90 msgr2=0x55a51ab013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 --2- 172.21.15.50:0/2292700628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab00f90 0x55a51ab013b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f353c00b700 tx=0x7f353c040f50 comp rx=0 tx=0).stop 2024-05-08T18:05:04.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 -- 172.21.15.50:0/2292700628 shutdown_connections 2024-05-08T18:05:04.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 --2- 172.21.15.50:0/2292700628 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f342403bba0 0x7f342403e060 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 --2- 172.21.15.50:0/2292700628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51ab00f90 0x55a51ab013b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 -- 172.21.15.50:0/2292700628 >> 172.21.15.50:0/2292700628 conn(0x55a51aaf5030 msgr2=0x55a51aaf5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 -- 172.21.15.50:0/2292700628 shutdown_connections 2024-05-08T18:05:04.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 -- 172.21.15.50:0/2292700628 wait complete. 2024-05-08T18:05:04.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 -- 172.21.15.50:0/2301741688 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a51aaec270 msgr2=0x55a51aaee660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a51aaec270 0x55a51aaee660 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 -- 172.21.15.50:0/2301741688 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a51a95d050 msgr2=0x55a51aae9a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a51a95d050 0x55a51aae9a10 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 -- 172.21.15.50:0/2301741688 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a51aaf01b0 msgr2=0x55a51aaf25a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a51aaf01b0 0x55a51aaf25a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 -- 172.21.15.50:0/2301741688 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f343403bba0 msgr2=0x7f343403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f343403bba0 0x7f343403e060 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3540054000 tx=0x7f354004e000 comp rx=0 tx=0).stop 2024-05-08T18:05:04.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 -- 172.21.15.50:0/2301741688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 msgr2=0x55a51aae27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 --2- 172.21.15.50:0/2301741688 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a51a95bcc0 0x55a51aae27d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f353c000c00 tx=0x7f353c000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f354ceec640 1 -- 172.21.15.50:0/2301741688 reap_dead start 2024-05-08T18:05:04.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 -- 172.21.15.50:0/2301741688 shutdown_connections 2024-05-08T18:05:04.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 -- 172.21.15.50:0/2301741688 >> 172.21.15.50:0/2301741688 conn(0x55a51a9e2c90 msgr2=0x55a51aad4da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 -- 172.21.15.50:0/2301741688 shutdown_connections 2024-05-08T18:05:04.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.901+0000 7f3550933800 1 -- 172.21.15.50:0/2301741688 wait complete. 2024-05-08T18:05:04.909 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-05-08T18:05:04.887254Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:04.909 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-05-08T18:05:04.910 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-05-08T18:05:04.910 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-05-08T18:05:04.955 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:04.955 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:04.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.958+0000 7f2ab043f800 1 Processor -- start 2024-05-08T18:05:04.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.958+0000 7f2ab043f800 1 -- start start 2024-05-08T18:05:04.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.958+0000 7f2ab043f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5588cc91e550 0x5588cc91e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.958+0000 7f2ab043f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5588cc91eff0 con 0x5588cc91f7b0 2024-05-08T18:05:04.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.958+0000 7f2ab043f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5588cc7bfe20 con 0x5588cc91e550 2024-05-08T18:05:04.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.959+0000 7f2aadf37640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5588cc91e550 0x5588cc91e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.959+0000 7f2aad736640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5588cc91f7b0 0x5588cc91fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:42992/0 (socket says 172.21.15.50:42992) 2024-05-08T18:05:04.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.959+0000 7f2aad736640 1 -- 172.21.15.50:0/1564260658 learned_addr learned my addr 172.21.15.50:0/1564260658 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:04.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.959+0000 7f2aadf37640 1 -- 172.21.15.50:0/1564260658 >> v1:172.21.15.46:6789/0 conn(0x5588cc91f7b0 legacy=0x5588cc91fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.959+0000 7f2aadf37640 1 -- 172.21.15.50:0/1564260658 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588cc920370 con 0x5588cc91e550 2024-05-08T18:05:04.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.959+0000 7f2aacf35640 1 -- 172.21.15.50:0/1564260658 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4056053079 0 0) 0x5588cc91eff0 con 0x5588cc91f7b0 2024-05-08T18:05:04.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.961+0000 7f2aadf37640 1 --2- 172.21.15.50:0/1564260658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 0x5588cc91e970 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000a470 tx=0x7f2aa001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7f27fc5a3b8729 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.961+0000 7f2aacf35640 1 -- 172.21.15.50:0/1564260658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa0057070 con 0x5588cc91e550 2024-05-08T18:05:04.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.961+0000 7f2aacf35640 1 -- 172.21.15.50:0/1564260658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa004adc0 con 0x5588cc91e550 2024-05-08T18:05:04.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.961+0000 7f2aacf35640 1 -- 172.21.15.50:0/1564260658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa0052590 con 0x5588cc91e550 2024-05-08T18:05:04.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.962+0000 7f2ab043f800 1 -- 172.21.15.50:0/1564260658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 msgr2=0x5588cc91e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.962+0000 7f2ab043f800 1 --2- 172.21.15.50:0/1564260658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 0x5588cc91e970 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000a470 tx=0x7f2aa001ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:04.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.962+0000 7f2ab043f800 1 -- 172.21.15.50:0/1564260658 shutdown_connections 2024-05-08T18:05:04.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.962+0000 7f2ab043f800 1 --2- 172.21.15.50:0/1564260658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 0x5588cc91e970 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.962+0000 7f2ab043f800 1 -- 172.21.15.50:0/1564260658 >> 172.21.15.50:0/1564260658 conn(0x5588cc919b50 msgr2=0x5588cc91bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.962+0000 7f2ab043f800 1 -- 172.21.15.50:0/1564260658 shutdown_connections 2024-05-08T18:05:04.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.962+0000 7f2ab043f800 1 -- 172.21.15.50:0/1564260658 wait complete. 2024-05-08T18:05:04.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.977+0000 7f2ab043f800 1 Processor -- start 2024-05-08T18:05:04.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.977+0000 7f2ab043f800 1 -- start start 2024-05-08T18:05:04.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.977+0000 7f2ab043f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 0x5588cca0f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.977+0000 7f2ab043f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5588cc86aa10 con 0x5588cc91e550 2024-05-08T18:05:04.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.977+0000 7f2aadf37640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 0x5588cca0f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.977+0000 7f2aadf37640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 0x5588cca0f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36600/0 (socket says 172.21.15.50:36600) 2024-05-08T18:05:04.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.978+0000 7f2aadf37640 1 -- 172.21.15.50:0/1995122817 learned_addr learned my addr 172.21.15.50:0/1995122817 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.978+0000 7f2aadf37640 1 -- 172.21.15.50:0/1995122817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588cca0fe90 con 0x5588cc91e550 2024-05-08T18:05:04.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.978+0000 7f2aadf37640 1 --2- 172.21.15.50:0/1995122817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 0x5588cca0f8c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0055040 tx=0x7f2aa004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6cd550ea9e4b029 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.978+0000 7f299be07640 1 -- 172.21.15.50:0/1995122817 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa0063070 con 0x5588cc91e550 2024-05-08T18:05:04.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.978+0000 7f299be07640 1 -- 172.21.15.50:0/1995122817 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa000a230 con 0x5588cc91e550 2024-05-08T18:05:04.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.978+0000 7f299be07640 1 -- 172.21.15.50:0/1995122817 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa0057040 con 0x5588cc91e550 2024-05-08T18:05:04.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.979+0000 7f2ab043f800 1 -- 172.21.15.50:0/1995122817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 msgr2=0x5588cca0f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:04.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.979+0000 7f2ab043f800 1 --2- 172.21.15.50:0/1995122817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 0x5588cca0f8c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0055040 tx=0x7f2aa004c000 comp rx=0 tx=0).stop 2024-05-08T18:05:04.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.979+0000 7f2ab043f800 1 -- 172.21.15.50:0/1995122817 shutdown_connections 2024-05-08T18:05:04.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.979+0000 7f2ab043f800 1 --2- 172.21.15.50:0/1995122817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 0x5588cca0f8c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:04.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.979+0000 7f2ab043f800 1 -- 172.21.15.50:0/1995122817 >> 172.21.15.50:0/1995122817 conn(0x5588cc919b50 msgr2=0x5588cca0b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:04.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.979+0000 7f2ab043f800 1 -- 172.21.15.50:0/1995122817 shutdown_connections 2024-05-08T18:05:04.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.979+0000 7f2ab043f800 1 -- 172.21.15.50:0/1995122817 wait complete. 2024-05-08T18:05:04.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.979+0000 7f2ab043f800 1 Processor -- start 2024-05-08T18:05:04.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.979+0000 7f2ab043f800 1 -- start start 2024-05-08T18:05:04.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.980+0000 7f2ab043f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 0x5588cca17410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.980+0000 7f2ab043f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5588cc927c80 con 0x5588cc91e550 2024-05-08T18:05:04.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.980+0000 7f2aadf37640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 0x5588cca17410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.980+0000 7f2aadf37640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 0x5588cca17410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36602/0 (socket says 172.21.15.50:36602) 2024-05-08T18:05:04.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.980+0000 7f2aadf37640 1 -- 172.21.15.50:0/1960165379 learned_addr learned my addr 172.21.15.50:0/1960165379 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.980+0000 7f2aadf37640 1 -- 172.21.15.50:0/1960165379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588cca17950 con 0x5588cc91e550 2024-05-08T18:05:04.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.980+0000 7f2aadf37640 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 0x5588cca17410 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0061040 tx=0x7f2aa0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.981+0000 7f299a604640 1 -- 172.21.15.50:0/1960165379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa006c070 con 0x5588cc91e550 2024-05-08T18:05:04.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.981+0000 7f299a604640 1 -- 172.21.15.50:0/1960165379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa005d7f0 con 0x5588cc91e550 2024-05-08T18:05:04.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.981+0000 7f2ab043f800 1 -- 172.21.15.50:0/1960165379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5588cca17be0 con 0x5588cc91e550 2024-05-08T18:05:04.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.981+0000 7f299a604640 1 -- 172.21.15.50:0/1960165379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa0063040 con 0x5588cc91e550 2024-05-08T18:05:04.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.981+0000 7f2ab043f800 1 -- 172.21.15.50:0/1960165379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588cca1a840 con 0x5588cc91e550 2024-05-08T18:05:04.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.982+0000 7f299a604640 1 -- 172.21.15.50:0/1960165379 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2aa0066070 con 0x5588cc91e550 2024-05-08T18:05:04.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.982+0000 7f299a604640 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f299403bba0 0x7f299403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.982+0000 7f299a604640 1 -- 172.21.15.50:0/1960165379 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f2aa0093860 con 0x5588cc91e550 2024-05-08T18:05:04.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.982+0000 7f2aad736640 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f299403bba0 0x7f299403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.982+0000 7f2ab043f800 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5588cc91f7b0 0x5588cca1e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.982+0000 7f2ab043f800 1 -- 172.21.15.50:0/1960165379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca1eb80 con 0x5588cc91f7b0 2024-05-08T18:05:04.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.982+0000 7f2aae738640 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5588cc91f7b0 0x5588cca1e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.983+0000 7f2aad736640 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f299403bba0 0x7f299403e060 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4000c00 tx=0x7f2aa40013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.983+0000 7f2aae738640 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5588cc91f7b0 0x5588cca1e600 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.983+0000 7f2aae738640 1 -- 172.21.15.50:0/1960165379 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cc91f7b0 2024-05-08T18:05:04.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.983+0000 7f2ab043f800 1 -- 172.21.15.50:0/1960165379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca1bbb0 con 0x5588cc91f7b0 2024-05-08T18:05:04.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.983+0000 7f2aae738640 1 -- 172.21.15.50:0/1960165379 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cc91f7b0 2024-05-08T18:05:04.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.983+0000 7f2ab043f800 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5588cca20ea0 0x5588cca23290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.983+0000 7f2ab043f800 1 -- 172.21.15.50:0/1960165379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca23930 con 0x5588cca20ea0 2024-05-08T18:05:04.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.983+0000 7f2aadf37640 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5588cca20ea0 0x5588cca23290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.984+0000 7f2aadf37640 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5588cca20ea0 0x5588cca23290 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.984+0000 7f2aadf37640 1 -- 172.21.15.50:0/1960165379 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca20ea0 2024-05-08T18:05:04.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.984+0000 7f2ab043f800 1 -- 172.21.15.50:0/1960165379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca1fd40 con 0x5588cc91f7b0 2024-05-08T18:05:04.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.984+0000 7f2aae738640 1 -- 172.21.15.50:0/1960165379 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cc91f7b0 2024-05-08T18:05:04.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.984+0000 7f2ab043f800 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5588cca24d90 0x5588cca27180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.984+0000 7f2ab043f800 1 -- 172.21.15.50:0/1960165379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca27880 con 0x5588cca24d90 2024-05-08T18:05:04.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.984+0000 7f2aad736640 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5588cca24d90 0x5588cca27180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.985+0000 7f2aad736640 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5588cca24d90 0x5588cca27180 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.985+0000 7f2aad736640 1 -- 172.21.15.50:0/1960165379 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f2aa400f670 con 0x5588cca24d90 2024-05-08T18:05:04.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.985+0000 7f2ab043f800 1 Processor -- start 2024-05-08T18:05:04.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.985+0000 7f2ab043f800 1 -- start start 2024-05-08T18:05:04.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.986+0000 7f2ab043f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca37890 0x5588cca39c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:04.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.986+0000 7f2ab043f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5588cc811330 con 0x5588cca37890 2024-05-08T18:05:04.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.986+0000 7f2aadf37640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca37890 0x5588cca39c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:04.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.986+0000 7f2aadf37640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca37890 0x5588cca39c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36610/0 (socket says 172.21.15.50:36610) 2024-05-08T18:05:04.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.986+0000 7f2aadf37640 1 -- 172.21.15.50:0/2670683205 learned_addr learned my addr 172.21.15.50:0/2670683205 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:04.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.986+0000 7f2aadf37640 1 -- 172.21.15.50:0/2670683205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588cc8927f0 con 0x5588cca37890 2024-05-08T18:05:04.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.986+0000 7f2aadf37640 1 --2- 172.21.15.50:0/2670683205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca37890 0x5588cca39c80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0070b50 tx=0x7f2aa0070eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:04.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.986+0000 7f29937fe640 1 -- 172.21.15.50:0/2670683205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa005daf0 con 0x5588cca37890 2024-05-08T18:05:04.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.986+0000 7f29937fe640 1 -- 172.21.15.50:0/2670683205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa005dc90 con 0x5588cca37890 2024-05-08T18:05:04.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.986+0000 7f29937fe640 1 -- 172.21.15.50:0/2670683205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa00095d0 con 0x5588cca37890 2024-05-08T18:05:04.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.986+0000 7f2ab043f800 1 -- 172.21.15.50:0/2670683205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5588cca3a1c0 con 0x5588cca37890 2024-05-08T18:05:04.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.986+0000 7f2ab043f800 1 -- 172.21.15.50:0/2670683205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588cca3a660 con 0x5588cca37890 2024-05-08T18:05:05.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.987+0000 7f29937fe640 1 -- 172.21.15.50:0/2670683205 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2aa0009770 con 0x5588cca37890 2024-05-08T18:05:05.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.987+0000 7f29937fe640 1 --2- 172.21.15.50:0/2670683205 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f298803bbf0 0x7f298803e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.987+0000 7f29937fe640 1 -- 172.21.15.50:0/2670683205 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f2aa0080070 con 0x5588cca37890 2024-05-08T18:05:05.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.987+0000 7f2aad736640 1 --2- 172.21.15.50:0/2670683205 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f298803bbf0 0x7f298803e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.988+0000 7f2ab043f800 1 Processor -- start 2024-05-08T18:05:05.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.988+0000 7f2ab043f800 1 -- start start 2024-05-08T18:05:05.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.988+0000 7f2aad736640 1 --2- 172.21.15.50:0/2670683205 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f298803bbf0 0x7f298803e0b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x5588cca1baf0 tx=0x7f2aa4006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.988+0000 7f2ab043f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca36e80 0x5588cca372a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.988+0000 7f2ab043f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5588cca10150 con 0x5588cca36e80 2024-05-08T18:05:05.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.988+0000 7f2aadf37640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca36e80 0x5588cca372a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.988+0000 7f2aadf37640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca36e80 0x5588cca372a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36620/0 (socket says 172.21.15.50:36620) 2024-05-08T18:05:05.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.988+0000 7f2aadf37640 1 -- 172.21.15.50:0/2182733453 learned_addr learned my addr 172.21.15.50:0/2182733453 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.989+0000 7f2aadf37640 1 -- 172.21.15.50:0/2182733453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588cca35680 con 0x5588cca36e80 2024-05-08T18:05:05.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.989+0000 7f2aadf37640 1 --2- 172.21.15.50:0/2182733453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca36e80 0x5588cca372a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0005820 tx=0x7f2aa0051690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ea508ed2f47b62 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.989+0000 7f29917fa640 1 -- 172.21.15.50:0/2182733453 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa00a2020 con 0x5588cca36e80 2024-05-08T18:05:05.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.989+0000 7f29917fa640 1 -- 172.21.15.50:0/2182733453 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa0093650 con 0x5588cca36e80 2024-05-08T18:05:05.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.989+0000 7f29917fa640 1 -- 172.21.15.50:0/2182733453 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa005b7f0 con 0x5588cca36e80 2024-05-08T18:05:05.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.989+0000 7f2ab043f800 1 -- 172.21.15.50:0/2182733453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca36e80 msgr2=0x5588cca372a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.989+0000 7f2ab043f800 1 --2- 172.21.15.50:0/2182733453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca36e80 0x5588cca372a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0005820 tx=0x7f2aa0051690 comp rx=0 tx=0).stop 2024-05-08T18:05:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.989+0000 7f2ab043f800 1 -- 172.21.15.50:0/2182733453 shutdown_connections 2024-05-08T18:05:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.989+0000 7f2ab043f800 1 --2- 172.21.15.50:0/2182733453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca36e80 0x5588cca372a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.989+0000 7f2ab043f800 1 -- 172.21.15.50:0/2182733453 >> 172.21.15.50:0/2182733453 conn(0x5588cca400e0 msgr2=0x5588cca3e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.989+0000 7f2ab043f800 1 -- 172.21.15.50:0/2182733453 shutdown_connections 2024-05-08T18:05:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.989+0000 7f2ab043f800 1 -- 172.21.15.50:0/2182733453 wait complete. 2024-05-08T18:05:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.990+0000 7f2ab043f800 1 Processor -- start 2024-05-08T18:05:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.990+0000 7f2ab043f800 1 -- start start 2024-05-08T18:05:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.990+0000 7f2ab043f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca36e80 0x5588cca4d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.990+0000 7f2ab043f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5588cca35a90 con 0x5588cca36e80 2024-05-08T18:05:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.990+0000 7f2aadf37640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca36e80 0x5588cca4d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.990+0000 7f2aadf37640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca36e80 0x5588cca4d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36628/0 (socket says 172.21.15.50:36628) 2024-05-08T18:05:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.990+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 learned_addr learned my addr 172.21.15.50:0/3095528756 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.991+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588cca4d8e0 con 0x5588cca36e80 2024-05-08T18:05:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.991+0000 7f2aadf37640 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca36e80 0x5588cca4d3a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0066fd0 tx=0x7f2aa00606b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.991+0000 7f297f7fe640 1 -- 172.21.15.50:0/3095528756 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa005ba10 con 0x5588cca36e80 2024-05-08T18:05:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.991+0000 7f297f7fe640 1 -- 172.21.15.50:0/3095528756 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2aa005bbb0 con 0x5588cca36e80 2024-05-08T18:05:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.991+0000 7f297f7fe640 1 -- 172.21.15.50:0/3095528756 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2aa0060030 con 0x5588cca36e80 2024-05-08T18:05:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.991+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5588cca4d8e0 con 0x5588cca36e80 2024-05-08T18:05:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.991+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588cca4d8e0 con 0x5588cca36e80 2024-05-08T18:05:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.992+0000 7f297f7fe640 1 -- 172.21.15.50:0/3095528756 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2aa00b9060 con 0x5588cca36e80 2024-05-08T18:05:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.992+0000 7f297f7fe640 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f297803bb50 0x7f297803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.992+0000 7f297f7fe640 1 -- 172.21.15.50:0/3095528756 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f2aa00bcc60 con 0x5588cca36e80 2024-05-08T18:05:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.992+0000 7f2aad736640 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f297803bb50 0x7f297803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.992+0000 7f2ab043f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.992+0000 7f2ab043f800 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5588cca589e0 0x5588cca5aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.992+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5b5f0 con 0x5588cca589e0 2024-05-08T18:05:05.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.993+0000 7f2aae738640 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5588cca589e0 0x5588cca5aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.993+0000 7f2aad736640 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f297803bb50 0x7f297803e010 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x5588cca27c80 tx=0x7f2aa4023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.993+0000 7f2aae738640 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5588cca589e0 0x5588cca5aea0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.993+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cca589e0 2024-05-08T18:05:05.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.993+0000 7f2ab043f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.993+0000 7f2ab043f800 20 realm 2024-05-08T18:05:05.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.993+0000 7f2ab043f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.993+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5ba00 con 0x5588cca589e0 2024-05-08T18:05:05.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.993+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cca589e0 2024-05-08T18:05:05.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.993+0000 7f2ab043f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.993+0000 7f2ab043f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:05.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.993+0000 7f2ab043f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.993+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c2d0 con 0x5588cca589e0 2024-05-08T18:05:05.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.994+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cca589e0 2024-05-08T18:05:05.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.994+0000 7f2ab043f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.994+0000 7f2ab043f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.994+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca589e0 2024-05-08T18:05:05.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.994+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cca589e0 2024-05-08T18:05:05.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.994+0000 7f2ab043f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:05.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.994+0000 7f2ab043f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.994+0000 7f2ab043f800 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5588cca5d160 0x5588cca5f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.994+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5fcb0 con 0x5588cca5d160 2024-05-08T18:05:05.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.994+0000 7f2aadf37640 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5588cca5d160 0x5588cca5f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.995+0000 7f2aadf37640 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5588cca5d160 0x5588cca5f550 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.995+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.995+0000 7f2ab043f800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:05.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.995+0000 7f2ab043f800 20 searching for the correct realm 2024-05-08T18:05:05.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.995+0000 7f2ab043f800 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5588cca61d50 0x5588cca64210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.995+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca648e0 con 0x5588cca61d50 2024-05-08T18:05:05.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.995+0000 7f2aad736640 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5588cca61d50 0x5588cca64210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.995+0000 7f2aad736640 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5588cca61d50 0x5588cca64210 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.996+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa4037040 con 0x5588cca61d50 2024-05-08T18:05:05.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.996+0000 7f2aad736640 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2aa402ea80 0x7f2aa403a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.996+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca648e0 con 0x7f2aa402ea80 2024-05-08T18:05:05.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.996+0000 7f2aae738640 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2aa402ea80 0x7f2aa403a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.996+0000 7f2aae738640 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2aa402ea80 0x7f2aa403a3c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.996+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa800a4a0 con 0x7f2aa402ea80 2024-05-08T18:05:05.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.996+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca5d160 2024-05-08T18:05:05.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.997+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.997+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca589e0 2024-05-08T18:05:05.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.997+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cca589e0 2024-05-08T18:05:05.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.997+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca589e0 2024-05-08T18:05:05.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.997+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cca589e0 2024-05-08T18:05:05.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.997+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca5d160 2024-05-08T18:05:05.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.997+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.997+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca5d160 2024-05-08T18:05:05.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.998+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.998+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x7f2aa402ea80 2024-05-08T18:05:05.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.998+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa800a4a0 con 0x7f2aa402ea80 2024-05-08T18:05:05.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.998+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca61d50 2024-05-08T18:05:05.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.998+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa4037040 con 0x5588cca61d50 2024-05-08T18:05:05.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.998+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca589e0 2024-05-08T18:05:05.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.998+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cca589e0 2024-05-08T18:05:05.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.998+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca5d160 2024-05-08T18:05:05.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.999+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.999+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca589e0 2024-05-08T18:05:05.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.999+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cca589e0 2024-05-08T18:05:05.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.999+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x7f2aa402ea80 2024-05-08T18:05:05.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.999+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2aa800a4a0 con 0x7f2aa402ea80 2024-05-08T18:05:05.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.999+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca61d50 2024-05-08T18:05:05.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.999+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa4037040 con 0x5588cca61d50 2024-05-08T18:05:05.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:04.999+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca61d50 2024-05-08T18:05:05.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.000+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa4037040 con 0x5588cca61d50 2024-05-08T18:05:05.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.000+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca5d160 2024-05-08T18:05:05.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.000+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.000+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca61d50 2024-05-08T18:05:05.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.000+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa4037040 con 0x5588cca61d50 2024-05-08T18:05:05.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.000+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca61d50 2024-05-08T18:05:05.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.000+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f2aa4037040 con 0x5588cca61d50 2024-05-08T18:05:05.013 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.000+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca589e0 2024-05-08T18:05:05.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.001+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cca589e0 2024-05-08T18:05:05.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.001+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca589e0 2024-05-08T18:05:05.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.001+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cca589e0 2024-05-08T18:05:05.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.001+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca5d160 2024-05-08T18:05:05.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.001+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.001+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca61d50 2024-05-08T18:05:05.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.001+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa4037040 con 0x5588cca61d50 2024-05-08T18:05:05.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.001+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x7f2aa402ea80 2024-05-08T18:05:05.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.002+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa800a4a0 con 0x7f2aa402ea80 2024-05-08T18:05:05.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.002+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca589e0 2024-05-08T18:05:05.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.002+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cca589e0 2024-05-08T18:05:05.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.002+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca5d160 2024-05-08T18:05:05.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.002+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.002+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca61d50 2024-05-08T18:05:05.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.002+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa4037040 con 0x5588cca61d50 2024-05-08T18:05:05.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.002+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca5d160 2024-05-08T18:05:05.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.003+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.003+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x7f2aa402ea80 2024-05-08T18:05:05.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.003+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa800a4a0 con 0x7f2aa402ea80 2024-05-08T18:05:05.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.003+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca61d50 2024-05-08T18:05:05.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.003+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa4037040 con 0x5588cca61d50 2024-05-08T18:05:05.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.003+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca5d160 2024-05-08T18:05:05.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.004+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.004+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x5588cca589e0 2024-05-08T18:05:05.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.004+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cca589e0 2024-05-08T18:05:05.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.004+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5588cca5c670 con 0x7f2aa402ea80 2024-05-08T18:05:05.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.004+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2aa800a4a0 con 0x7f2aa402ea80 2024-05-08T18:05:05.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.004+0000 7f2ab043f800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:05.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.004+0000 7f2ab043f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:05.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.004+0000 7f2ab043f800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:05.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.004+0000 7f2ab043f800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:05.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.004+0000 7f2ab043f800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:05.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.004+0000 7f2ab043f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:05.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.004+0000 7f2ab043f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.004+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca60650 con 0x5588cca589e0 2024-05-08T18:05:05.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cca589e0 2024-05-08T18:05:05.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2ab043f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2ab043f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca60650 con 0x5588cca589e0 2024-05-08T18:05:05.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2aa800a4a0 con 0x5588cca589e0 2024-05-08T18:05:05.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2ab043f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:05.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2ab043f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca60650 con 0x5588cca61d50 2024-05-08T18:05:05.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f2aa4037040 con 0x5588cca61d50 2024-05-08T18:05:05.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2ab043f800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:05.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2ab043f800 20 zone default found 2024-05-08T18:05:05.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2ab043f800 4 Realm: () 2024-05-08T18:05:05.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2ab043f800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:05.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2ab043f800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:05.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2ab043f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:05.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2ab043f800 20 zonegroup default 2024-05-08T18:05:05.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2ab043f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.005+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca60ef0 con 0x5588cca5d160 2024-05-08T18:05:05.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.006+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.006+0000 7f2ab043f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.006+0000 7f2ab043f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.006+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca66740 con 0x5588cca61d50 2024-05-08T18:05:05.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.006+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2aa4037040 con 0x5588cca61d50 2024-05-08T18:05:05.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.006+0000 7f2ab043f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.006+0000 7f2ab043f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.006+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca66ae0 con 0x5588cca61d50 2024-05-08T18:05:05.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.007+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2aa4037040 con 0x5588cca61d50 2024-05-08T18:05:05.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.007+0000 7f2ab043f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.007+0000 7f2ab043f800 20 started sync module instance, tier type = 2024-05-08T18:05:05.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.007+0000 7f2ab043f800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:05.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.007+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca65be0 con 0x7f2aa402ea80 2024-05-08T18:05:05.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.008+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa800a4a0 con 0x7f2aa402ea80 2024-05-08T18:05:05.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.008+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94046037369088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca65be0 con 0x7f2aa402ea80 2024-05-08T18:05:05.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.008+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca67520 con 0x5588cca589e0 2024-05-08T18:05:05.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.009+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa8040040 con 0x5588cca589e0 2024-05-08T18:05:05.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.009+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94046037372912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca67520 con 0x5588cca589e0 2024-05-08T18:05:05.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.009+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94046037369088] v27'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa8040040 con 0x7f2aa402ea80 2024-05-08T18:05:05.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.009+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca65730 con 0x5588cca61d50 2024-05-08T18:05:05.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.010+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa4037040 con 0x5588cca61d50 2024-05-08T18:05:05.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.010+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94046037377040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca68990 con 0x5588cca61d50 2024-05-08T18:05:05.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.010+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94046037372912] v27'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa8040040 con 0x5588cca589e0 2024-05-08T18:05:05.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.010+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca67910 con 0x7f2aa402ea80 2024-05-08T18:05:05.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.011+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94046037377040] v27'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa4037040 con 0x5588cca61d50 2024-05-08T18:05:05.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.011+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa8040040 con 0x7f2aa402ea80 2024-05-08T18:05:05.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.011+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94046037378272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca69350 con 0x7f2aa402ea80 2024-05-08T18:05:05.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.011+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca69d10 con 0x5588cca589e0 2024-05-08T18:05:05.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.012+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa8040040 con 0x5588cca589e0 2024-05-08T18:05:05.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.012+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94046037383264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca6a130 con 0x5588cca589e0 2024-05-08T18:05:05.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.012+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca6ab70 con 0x5588cca5d160 2024-05-08T18:05:05.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.012+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94046037378272] v27'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa8040040 con 0x7f2aa402ea80 2024-05-08T18:05:05.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.013+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94046037383264] v27'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa8040040 con 0x5588cca589e0 2024-05-08T18:05:05.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.013+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.013+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94046037380768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca6a130 con 0x5588cca5d160 2024-05-08T18:05:05.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.013+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca6b440 con 0x5588cca5d160 2024-05-08T18:05:05.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.014+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.014+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94046037389072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca6b820 con 0x5588cca5d160 2024-05-08T18:05:05.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.014+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca6c320 con 0x5588cca5d160 2024-05-08T18:05:05.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.014+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94046037380768] v27'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.015+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94046037389072] v27'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.015+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.015+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94046037392912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca6b330 con 0x5588cca5d160 2024-05-08T18:05:05.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.015+0000 7f2ab043f800 20 add_watcher() i=0 2024-05-08T18:05:05.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.015+0000 7f2ab043f800 20 add_watcher() i=1 2024-05-08T18:05:05.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.015+0000 7f2ab043f800 20 add_watcher() i=2 2024-05-08T18:05:05.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.015+0000 7f2ab043f800 20 add_watcher() i=3 2024-05-08T18:05:05.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.015+0000 7f2ab043f800 20 add_watcher() i=4 2024-05-08T18:05:05.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.015+0000 7f2ab043f800 20 add_watcher() i=5 2024-05-08T18:05:05.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.015+0000 7f2ab043f800 20 add_watcher() i=6 2024-05-08T18:05:05.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.016+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94046037392912] v27'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.016+0000 7f2ab043f800 20 add_watcher() i=7 2024-05-08T18:05:05.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.016+0000 7f2ab043f800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:05.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.016+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588cca6f310 con 0x5588cca589e0 2024-05-08T18:05:05.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.016+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f2aa8040040 con 0x5588cca589e0 2024-05-08T18:05:05.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.017+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94046037384320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca6c790 con 0x5588cca589e0 2024-05-08T18:05:05.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.018+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94046037384320] v27'66 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2aa8040040 con 0x5588cca589e0 2024-05-08T18:05:05.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.018+0000 7f295f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:05.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.018+0000 7f2ab043f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:05.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.018+0000 7f2ab043f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:05.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.018+0000 7f2ab043f800 5 note: GC not initialized 2024-05-08T18:05:05.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.018+0000 7f295e7fc640 20 reqs_thread_entry: start 2024-05-08T18:05:05.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.018+0000 7f2ab043f800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:05.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.018+0000 7f2ab043f800 20 Filter name: none 2024-05-08T18:05:05.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.018+0000 7f28cffff640 20 reqs_thread_entry: start 2024-05-08T18:05:05.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.018+0000 7f2ab043f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.018+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5588ccaa2250 con 0x5588cca589e0 2024-05-08T18:05:05.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.019+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~502 out=502b,getxattrs out=75b] v0'0 uv4 ondisk = 0) ==== 231+0+625 (crc 0 0 0) 0x7f2aa8040040 con 0x5588cca589e0 2024-05-08T18:05:05.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.019+0000 7f2ab043f800 20 rados_obj.operate() r=0 bl.length=502 2024-05-08T18:05:05.028 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"},{"user":"foo","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-05-08T18:05:04.887254Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:05.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.020+0000 7f295f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:05.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.020+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94046037384320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588ccaa0e20 con 0x5588cca589e0 2024-05-08T18:05:05.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.021+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94046037384320] v27'67 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2aa8040040 con 0x5588cca589e0 2024-05-08T18:05:05.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.021+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94046037369088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca69c80 con 0x7f2aa402ea80 2024-05-08T18:05:05.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.022+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94046037369088] v27'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa8040040 con 0x7f2aa402ea80 2024-05-08T18:05:05.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.022+0000 7f2ab043f800 20 remove_watcher() i=0 2024-05-08T18:05:05.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.022+0000 7f2ab043f800 2 removed watcher, disabling cache 2024-05-08T18:05:05.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.022+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94046037372912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca660e0 con 0x5588cca589e0 2024-05-08T18:05:05.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.023+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94046037372912] v27'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa8040040 con 0x5588cca589e0 2024-05-08T18:05:05.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.023+0000 7f2ab043f800 20 remove_watcher() i=1 2024-05-08T18:05:05.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.023+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94046037377040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca66ff0 con 0x5588cca61d50 2024-05-08T18:05:05.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.024+0000 7f2aad736640 1 -- 172.21.15.50:0/3095528756 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94046037377040] v27'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa4037040 con 0x5588cca61d50 2024-05-08T18:05:05.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.024+0000 7f2ab043f800 20 remove_watcher() i=2 2024-05-08T18:05:05.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.024+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94046037378272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca68010 con 0x7f2aa402ea80 2024-05-08T18:05:05.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.025+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94046037378272] v27'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa8040040 con 0x7f2aa402ea80 2024-05-08T18:05:05.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.025+0000 7f2ab043f800 20 remove_watcher() i=3 2024-05-08T18:05:05.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.025+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94046037383264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca684e0 con 0x5588cca589e0 2024-05-08T18:05:05.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.026+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94046037383264] v27'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa8040040 con 0x5588cca589e0 2024-05-08T18:05:05.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.026+0000 7f2ab043f800 20 remove_watcher() i=4 2024-05-08T18:05:05.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.026+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94046037380768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca684e0 con 0x5588cca5d160 2024-05-08T18:05:05.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.026+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94046037380768] v27'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.026+0000 7f2ab043f800 20 remove_watcher() i=5 2024-05-08T18:05:05.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.026+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94046037389072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca68ea0 con 0x5588cca5d160 2024-05-08T18:05:05.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.027+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94046037389072] v27'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.027+0000 7f2ab043f800 20 remove_watcher() i=6 2024-05-08T18:05:05.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.027+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94046037392912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5588cca68ea0 con 0x5588cca5d160 2024-05-08T18:05:05.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.028+0000 7f2aadf37640 1 -- 172.21.15.50:0/3095528756 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94046037392912] v27'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2aa006e070 con 0x5588cca5d160 2024-05-08T18:05:05.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.028+0000 7f2ab043f800 20 remove_watcher() i=7 2024-05-08T18:05:05.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5588cca5d160 msgr2=0x5588cca5f550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5588cca5d160 0x5588cca5f550 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5588cca589e0 msgr2=0x5588cca5aea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5588cca589e0 0x5588cca5aea0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2aa402ea80 msgr2=0x7f2aa403a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2aa402ea80 0x7f2aa403a3c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5588cca61d50 msgr2=0x5588cca64210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5588cca61d50 0x5588cca64210 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f297803bb50 msgr2=0x7f297803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f297803bb50 0x7f297803e010 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x5588cca27c80 tx=0x7f2aa4023040 comp rx=0 tx=0).stop 2024-05-08T18:05:05.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2aae738640 1 -- 172.21.15.50:0/3095528756 reap_dead start 2024-05-08T18:05:05.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca36e80 msgr2=0x5588cca4d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca36e80 0x5588cca4d3a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0066fd0 tx=0x7f2aa00606b0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 shutdown_connections 2024-05-08T18:05:05.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 --2- 172.21.15.50:0/3095528756 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca36e80 0x5588cca4d3a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 >> 172.21.15.50:0/3095528756 conn(0x5588cca400e0 msgr2=0x5588cca3d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 shutdown_connections 2024-05-08T18:05:05.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 -- 172.21.15.50:0/3095528756 wait complete. 2024-05-08T18:05:05.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 -- 172.21.15.50:0/2670683205 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f298803bbf0 msgr2=0x7f298803e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 --2- 172.21.15.50:0/2670683205 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f298803bbf0 0x7f298803e0b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x5588cca1baf0 tx=0x7f2aa4006000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 -- 172.21.15.50:0/2670683205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca37890 msgr2=0x5588cca39c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 --2- 172.21.15.50:0/2670683205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca37890 0x5588cca39c80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0070b50 tx=0x7f2aa0070eb0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 -- 172.21.15.50:0/2670683205 shutdown_connections 2024-05-08T18:05:05.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 --2- 172.21.15.50:0/2670683205 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f298803bbf0 0x7f298803e0b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 --2- 172.21.15.50:0/2670683205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cca37890 0x5588cca39c80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 -- 172.21.15.50:0/2670683205 >> 172.21.15.50:0/2670683205 conn(0x5588cca29c90 msgr2=0x5588cca2a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 -- 172.21.15.50:0/2670683205 shutdown_connections 2024-05-08T18:05:05.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.029+0000 7f2ab043f800 1 -- 172.21.15.50:0/2670683205 wait complete. 2024-05-08T18:05:05.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.030+0000 7f2ab043f800 1 -- 172.21.15.50:0/1960165379 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5588cca20ea0 msgr2=0x5588cca23290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.030+0000 7f2ab043f800 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5588cca20ea0 0x5588cca23290 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.030+0000 7f2ab043f800 1 -- 172.21.15.50:0/1960165379 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5588cc91f7b0 msgr2=0x5588cca1e600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.030+0000 7f2ab043f800 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5588cc91f7b0 0x5588cca1e600 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.030+0000 7f2ab043f800 1 -- 172.21.15.50:0/1960165379 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5588cca24d90 msgr2=0x5588cca27180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.030+0000 7f2ab043f800 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5588cca24d90 0x5588cca27180 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.030+0000 7f2ab043f800 1 -- 172.21.15.50:0/1960165379 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f299403bba0 msgr2=0x7f299403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.030+0000 7f2ab043f800 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f299403bba0 0x7f299403e060 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4000c00 tx=0x7f2aa40013b0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.030+0000 7f2ab043f800 1 -- 172.21.15.50:0/1960165379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 msgr2=0x5588cca17410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.030+0000 7f2ab043f800 1 --2- 172.21.15.50:0/1960165379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5588cc91e550 0x5588cca17410 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0061040 tx=0x7f2aa0002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.030+0000 7f2aae738640 1 -- 172.21.15.50:0/1960165379 reap_dead start 2024-05-08T18:05:05.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.030+0000 7f2ab043f800 1 -- 172.21.15.50:0/1960165379 shutdown_connections 2024-05-08T18:05:05.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.030+0000 7f2ab043f800 1 -- 172.21.15.50:0/1960165379 >> 172.21.15.50:0/1960165379 conn(0x5588cc919b50 msgr2=0x5588cca099e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.030+0000 7f2ab043f800 1 -- 172.21.15.50:0/1960165379 shutdown_connections 2024-05-08T18:05:05.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.030+0000 7f2ab043f800 1 -- 172.21.15.50:0/1960165379 wait complete. 2024-05-08T18:05:05.039 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-05-08T18:05:04.887254Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:05.040 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2024-05-08T18:05:05.040 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2024-05-08T18:05:05.040 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --uid foo --access-key p5YnriCv1nAtykxBrupQ 2024-05-08T18:05:05.087 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:05.088 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:05.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.090+0000 7f65e941f800 1 Processor -- start 2024-05-08T18:05:05.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.090+0000 7f65e941f800 1 -- start start 2024-05-08T18:05:05.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.090+0000 7f65e941f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55a90eab7b10 0x55a90eac0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.090+0000 7f65e941f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a90ea2a760 con 0x55a90eab5fd0 2024-05-08T18:05:05.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.091+0000 7f65e941f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55a90e957e20 con 0x55a90eab7b10 2024-05-08T18:05:05.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.091+0000 7f65dffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55a90eab7b10 0x55a90eac0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.091+0000 7f65e6f1e640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55a90eab5fd0 0x55a90eab63f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:42996/0 (socket says 172.21.15.50:42996) 2024-05-08T18:05:05.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.091+0000 7f65dffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55a90eab7b10 0x55a90eac0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36636/0 (socket says 172.21.15.50:36636) 2024-05-08T18:05:05.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.091+0000 7f65e6f1e640 1 -- 172.21.15.50:0/784745769 learned_addr learned my addr 172.21.15.50:0/784745769 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:05.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.091+0000 7f65dffff640 1 -- 172.21.15.50:0/784745769 >> v1:172.21.15.46:6789/0 conn(0x55a90eab5fd0 legacy=0x55a90eab63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.091+0000 7f65dffff640 1 -- 172.21.15.50:0/784745769 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a90ea2a930 con 0x55a90eab7b10 2024-05-08T18:05:05.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.093+0000 7f65dffff640 1 --2- 172.21.15.50:0/784745769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab7b10 0x55a90eac0610 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f65d000a470 tx=0x7f65d001cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214b7db6c1275fb4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.093+0000 7f65e671d640 1 -- 172.21.15.50:0/784745769 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d0057070 con 0x55a90eab7b10 2024-05-08T18:05:05.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.093+0000 7f65e671d640 1 -- 172.21.15.50:0/784745769 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d004ac70 con 0x55a90eab7b10 2024-05-08T18:05:05.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.093+0000 7f65e671d640 1 -- 172.21.15.50:0/784745769 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d0052440 con 0x55a90eab7b10 2024-05-08T18:05:05.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.094+0000 7f65e941f800 1 -- 172.21.15.50:0/784745769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab7b10 msgr2=0x55a90eac0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.094+0000 7f65e941f800 1 --2- 172.21.15.50:0/784745769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab7b10 0x55a90eac0610 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f65d000a470 tx=0x7f65d001cce0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.094+0000 7f65e941f800 1 -- 172.21.15.50:0/784745769 shutdown_connections 2024-05-08T18:05:05.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.094+0000 7f65e941f800 1 --2- 172.21.15.50:0/784745769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab7b10 0x55a90eac0610 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.094+0000 7f65e941f800 1 -- 172.21.15.50:0/784745769 >> 172.21.15.50:0/784745769 conn(0x55a90eab1b70 msgr2=0x55a90eab3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.094+0000 7f65e941f800 1 -- 172.21.15.50:0/784745769 shutdown_connections 2024-05-08T18:05:05.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.094+0000 7f65e941f800 1 -- 172.21.15.50:0/784745769 wait complete. 2024-05-08T18:05:05.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.109+0000 7f65e941f800 1 Processor -- start 2024-05-08T18:05:05.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.109+0000 7f65e941f800 1 -- start start 2024-05-08T18:05:05.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.109+0000 7f65e941f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab5fd0 0x55a90ebaadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.109+0000 7f65e941f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a90ea02a30 con 0x55a90eab5fd0 2024-05-08T18:05:05.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.109+0000 7f65e6f1e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab5fd0 0x55a90ebaadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.110+0000 7f65e6f1e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab5fd0 0x55a90ebaadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36648/0 (socket says 172.21.15.50:36648) 2024-05-08T18:05:05.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.110+0000 7f65e6f1e640 1 -- 172.21.15.50:0/102992482 learned_addr learned my addr 172.21.15.50:0/102992482 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.110+0000 7f65e6f1e640 1 -- 172.21.15.50:0/102992482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a90ebab390 con 0x55a90eab5fd0 2024-05-08T18:05:05.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.110+0000 7f65e6f1e640 1 --2- 172.21.15.50:0/102992482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab5fd0 0x55a90ebaadc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f65d800b510 tx=0x7f65d800b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc4078440846ff7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.110+0000 7f64d4e09640 1 -- 172.21.15.50:0/102992482 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d8008070 con 0x55a90eab5fd0 2024-05-08T18:05:05.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.110+0000 7f64d4e09640 1 -- 172.21.15.50:0/102992482 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d800d5c0 con 0x55a90eab5fd0 2024-05-08T18:05:05.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.110+0000 7f64d4e09640 1 -- 172.21.15.50:0/102992482 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d800d900 con 0x55a90eab5fd0 2024-05-08T18:05:05.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.111+0000 7f65e941f800 1 -- 172.21.15.50:0/102992482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab5fd0 msgr2=0x55a90ebaadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.111+0000 7f65e941f800 1 --2- 172.21.15.50:0/102992482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab5fd0 0x55a90ebaadc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f65d800b510 tx=0x7f65d800b9d0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.111+0000 7f65e941f800 1 -- 172.21.15.50:0/102992482 shutdown_connections 2024-05-08T18:05:05.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.111+0000 7f65e941f800 1 --2- 172.21.15.50:0/102992482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab5fd0 0x55a90ebaadc0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.111+0000 7f65e941f800 1 -- 172.21.15.50:0/102992482 >> 172.21.15.50:0/102992482 conn(0x55a90eab1b70 msgr2=0x55a90eba66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.111+0000 7f65e941f800 1 -- 172.21.15.50:0/102992482 shutdown_connections 2024-05-08T18:05:05.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.111+0000 7f65e941f800 1 -- 172.21.15.50:0/102992482 wait complete. 2024-05-08T18:05:05.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.111+0000 7f65e941f800 1 Processor -- start 2024-05-08T18:05:05.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.111+0000 7f65e941f800 1 -- start start 2024-05-08T18:05:05.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.112+0000 7f65e941f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab5fd0 0x55a90ebb2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.112+0000 7f65e941f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a90eac0e20 con 0x55a90eab5fd0 2024-05-08T18:05:05.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.112+0000 7f65e6f1e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab5fd0 0x55a90ebb2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.112+0000 7f65e6f1e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab5fd0 0x55a90ebb2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36656/0 (socket says 172.21.15.50:36656) 2024-05-08T18:05:05.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.112+0000 7f65e6f1e640 1 -- 172.21.15.50:0/3826150559 learned_addr learned my addr 172.21.15.50:0/3826150559 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.112+0000 7f65e6f1e640 1 -- 172.21.15.50:0/3826150559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a90ebb2ec0 con 0x55a90eab5fd0 2024-05-08T18:05:05.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.112+0000 7f65e6f1e640 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab5fd0 0x55a90ebb2980 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55a90ebabb50 tx=0x7f65d8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.112+0000 7f64d3606640 1 -- 172.21.15.50:0/3826150559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d800fab0 con 0x55a90eab5fd0 2024-05-08T18:05:05.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.112+0000 7f64d3606640 1 -- 172.21.15.50:0/3826150559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d800fc50 con 0x55a90eab5fd0 2024-05-08T18:05:05.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.112+0000 7f65e941f800 1 -- 172.21.15.50:0/3826150559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a90ebb3150 con 0x55a90eab5fd0 2024-05-08T18:05:05.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.112+0000 7f64d3606640 1 -- 172.21.15.50:0/3826150559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d8019470 con 0x55a90eab5fd0 2024-05-08T18:05:05.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.112+0000 7f65e941f800 1 -- 172.21.15.50:0/3826150559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a90ebb5d60 con 0x55a90eab5fd0 2024-05-08T18:05:05.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.113+0000 7f64d3606640 1 -- 172.21.15.50:0/3826150559 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f65d8017040 con 0x55a90eab5fd0 2024-05-08T18:05:05.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.113+0000 7f64d3606640 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f64cc03bba0 0x7f64cc03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.114+0000 7f64d3606640 1 -- 172.21.15.50:0/3826150559 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f65d8015020 con 0x55a90eab5fd0 2024-05-08T18:05:05.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.114+0000 7f65dffff640 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f64cc03bba0 0x7f64cc03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.114+0000 7f65e941f800 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a90eab7b10 0x55a90ebb9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.114+0000 7f65e941f800 1 -- 172.21.15.50:0/3826150559 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebba0a0 con 0x55a90eab7b10 2024-05-08T18:05:05.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.114+0000 7f65e771f640 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a90eab7b10 0x55a90ebb9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.114+0000 7f65dffff640 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f64cc03bba0 0x7f64cc03e060 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f65d0054000 tx=0x7f65d004e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.114+0000 7f65e771f640 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a90eab7b10 0x55a90ebb9b20 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.114+0000 7f65e771f640 1 -- 172.21.15.50:0/3826150559 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90eab7b10 2024-05-08T18:05:05.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.114+0000 7f65e941f800 1 -- 172.21.15.50:0/3826150559 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebb70f0 con 0x55a90eab7b10 2024-05-08T18:05:05.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.115+0000 7f65e771f640 1 -- 172.21.15.50:0/3826150559 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90eab7b10 2024-05-08T18:05:05.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.115+0000 7f65e941f800 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a90ebbc3d0 0x55a90ebbe7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.115+0000 7f65e941f800 1 -- 172.21.15.50:0/3826150559 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebbeeb0 con 0x55a90ebbc3d0 2024-05-08T18:05:05.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.115+0000 7f65e6f1e640 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a90ebbc3d0 0x55a90ebbe7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.115+0000 7f65e6f1e640 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a90ebbc3d0 0x55a90ebbe7c0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.116+0000 7f65e6f1e640 1 -- 172.21.15.50:0/3826150559 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f65d802d070 con 0x55a90ebbc3d0 2024-05-08T18:05:05.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.116+0000 7f65e941f800 1 -- 172.21.15.50:0/3826150559 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebbb240 con 0x55a90eab7b10 2024-05-08T18:05:05.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.116+0000 7f65e771f640 1 -- 172.21.15.50:0/3826150559 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90eab7b10 2024-05-08T18:05:05.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.116+0000 7f65e941f800 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a90ebc0310 0x55a90ebc2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.116+0000 7f65e941f800 1 -- 172.21.15.50:0/3826150559 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebc2df0 con 0x55a90ebc0310 2024-05-08T18:05:05.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.116+0000 7f65dffff640 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a90ebc0310 0x55a90ebc2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.116+0000 7f65dffff640 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a90ebc0310 0x55a90ebc2700 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.117+0000 7f65dffff640 1 -- 172.21.15.50:0/3826150559 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f65d005a040 con 0x55a90ebc0310 2024-05-08T18:05:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.117+0000 7f65e941f800 1 Processor -- start 2024-05-08T18:05:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.117+0000 7f65e941f800 1 -- start start 2024-05-08T18:05:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.117+0000 7f65e941f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebd1080 0x55a90ebd14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.117+0000 7f65e941f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a90e9a9330 con 0x55a90ebd1080 2024-05-08T18:05:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.117+0000 7f65e6f1e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebd1080 0x55a90ebd14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.117+0000 7f65e6f1e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebd1080 0x55a90ebd14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36668/0 (socket says 172.21.15.50:36668) 2024-05-08T18:05:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.117+0000 7f65e6f1e640 1 -- 172.21.15.50:0/2885892188 learned_addr learned my addr 172.21.15.50:0/2885892188 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.118+0000 7f65e6f1e640 1 -- 172.21.15.50:0/2885892188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a90ebd19e0 con 0x55a90ebd1080 2024-05-08T18:05:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.118+0000 7f65e6f1e640 1 --2- 172.21.15.50:0/2885892188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebd1080 0x55a90ebd14a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f65d8051990 tx=0x7f65d80533e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.118+0000 7f64d0e01640 1 -- 172.21.15.50:0/2885892188 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d8035070 con 0x55a90ebd1080 2024-05-08T18:05:05.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.118+0000 7f64d0e01640 1 -- 172.21.15.50:0/2885892188 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d804a460 con 0x55a90ebd1080 2024-05-08T18:05:05.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.118+0000 7f65e941f800 1 -- 172.21.15.50:0/2885892188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a90ebd2450 con 0x55a90ebd1080 2024-05-08T18:05:05.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.118+0000 7f64d0e01640 1 -- 172.21.15.50:0/2885892188 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d801c9d0 con 0x55a90ebd1080 2024-05-08T18:05:05.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.118+0000 7f65e941f800 1 -- 172.21.15.50:0/2885892188 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a90ebd2950 con 0x55a90ebd1080 2024-05-08T18:05:05.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.119+0000 7f64d0e01640 1 -- 172.21.15.50:0/2885892188 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f65d8038070 con 0x55a90ebd1080 2024-05-08T18:05:05.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.119+0000 7f64d0e01640 1 --2- 172.21.15.50:0/2885892188 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f64c003bba0 0x7f64c003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.119+0000 7f64d0e01640 1 -- 172.21.15.50:0/2885892188 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f65d804a050 con 0x55a90ebd1080 2024-05-08T18:05:05.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.119+0000 7f65dffff640 1 --2- 172.21.15.50:0/2885892188 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f64c003bba0 0x7f64c003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.119+0000 7f65e941f800 1 Processor -- start 2024-05-08T18:05:05.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.119+0000 7f65e941f800 1 -- start start 2024-05-08T18:05:05.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.119+0000 7f65dffff640 1 --2- 172.21.15.50:0/2885892188 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f64c003bba0 0x7f64c003e060 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55a90ebba980 tx=0x7f65d0005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.119+0000 7f65e941f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebde300 0x55a90ebde720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.119+0000 7f65e941f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a90ebab710 con 0x55a90ebde300 2024-05-08T18:05:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.120+0000 7f65e6f1e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebde300 0x55a90ebde720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.120+0000 7f65e6f1e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebde300 0x55a90ebde720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36672/0 (socket says 172.21.15.50:36672) 2024-05-08T18:05:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.120+0000 7f65e6f1e640 1 -- 172.21.15.50:0/2453388661 learned_addr learned my addr 172.21.15.50:0/2453388661 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.120+0000 7f65e6f1e640 1 -- 172.21.15.50:0/2453388661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a90ebdecf0 con 0x55a90ebde300 2024-05-08T18:05:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.120+0000 7f65e6f1e640 1 --2- 172.21.15.50:0/2453388661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebde300 0x55a90ebde720 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f65d801aa80 tx=0x7f65d8018750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197eaab4aff85320 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.120+0000 7f64ca7fc640 1 -- 172.21.15.50:0/2453388661 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d802fce0 con 0x55a90ebde300 2024-05-08T18:05:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.120+0000 7f64ca7fc640 1 -- 172.21.15.50:0/2453388661 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d8031cf0 con 0x55a90ebde300 2024-05-08T18:05:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.120+0000 7f64ca7fc640 1 -- 172.21.15.50:0/2453388661 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d8019610 con 0x55a90ebde300 2024-05-08T18:05:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.121+0000 7f65e941f800 1 -- 172.21.15.50:0/2453388661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebde300 msgr2=0x55a90ebde720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.121+0000 7f65e941f800 1 --2- 172.21.15.50:0/2453388661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebde300 0x55a90ebde720 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f65d801aa80 tx=0x7f65d8018750 comp rx=0 tx=0).stop 2024-05-08T18:05:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.121+0000 7f65e941f800 1 -- 172.21.15.50:0/2453388661 shutdown_connections 2024-05-08T18:05:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.121+0000 7f65e941f800 1 --2- 172.21.15.50:0/2453388661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebde300 0x55a90ebde720 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.121+0000 7f65e941f800 1 -- 172.21.15.50:0/2453388661 >> 172.21.15.50:0/2453388661 conn(0x55a90ebda660 msgr2=0x55a90ebd86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.121+0000 7f65e941f800 1 -- 172.21.15.50:0/2453388661 shutdown_connections 2024-05-08T18:05:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.121+0000 7f65e941f800 1 -- 172.21.15.50:0/2453388661 wait complete. 2024-05-08T18:05:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.121+0000 7f65e941f800 1 Processor -- start 2024-05-08T18:05:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.121+0000 7f65e941f800 1 -- start start 2024-05-08T18:05:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.121+0000 7f65e941f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebde300 0x55a90ebe5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.121+0000 7f65e941f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a90ebdf3a0 con 0x55a90ebde300 2024-05-08T18:05:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.121+0000 7f65e6f1e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebde300 0x55a90ebe5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.121+0000 7f65e6f1e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebde300 0x55a90ebe5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36682/0 (socket says 172.21.15.50:36682) 2024-05-08T18:05:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.121+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 learned_addr learned my addr 172.21.15.50:0/709671018 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.122+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a90ebe5cc0 con 0x55a90ebde300 2024-05-08T18:05:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.122+0000 7f65e6f1e640 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebde300 0x55a90ebe5780 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f65d8053570 tx=0x7f65d8046fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.122+0000 7f64c8ff9640 1 -- 172.21.15.50:0/709671018 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d80305b0 con 0x55a90ebde300 2024-05-08T18:05:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.122+0000 7f64c8ff9640 1 -- 172.21.15.50:0/709671018 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65d8019610 con 0x55a90ebde300 2024-05-08T18:05:05.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.122+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a90ebe5cc0 con 0x55a90ebde300 2024-05-08T18:05:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.122+0000 7f64c8ff9640 1 -- 172.21.15.50:0/709671018 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f65d80470c0 con 0x55a90ebde300 2024-05-08T18:05:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.122+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a90ebe6160 con 0x55a90ebde300 2024-05-08T18:05:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.123+0000 7f64c8ff9640 1 -- 172.21.15.50:0/709671018 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f65d8047260 con 0x55a90ebde300 2024-05-08T18:05:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.123+0000 7f64c8ff9640 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f64b003bba0 0x7f64b003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.123+0000 7f64c8ff9640 1 -- 172.21.15.50:0/709671018 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f65d8059070 con 0x55a90ebde300 2024-05-08T18:05:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.123+0000 7f65dffff640 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f64b003bba0 0x7f64b003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.123+0000 7f65dffff640 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f64b003bba0 0x7f64b003e060 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f65d0052fd0 tx=0x7f65d004f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.124+0000 7f65e941f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.124+0000 7f65e941f800 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a90ebf0e40 0x55a90ebf3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.124+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf3a50 con 0x55a90ebf0e40 2024-05-08T18:05:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.124+0000 7f65e771f640 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a90ebf0e40 0x55a90ebf3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.124+0000 7f65e771f640 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a90ebf0e40 0x55a90ebf3300 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.124+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90ebf0e40 2024-05-08T18:05:05.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.124+0000 7f65e941f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.124+0000 7f65e941f800 20 realm 2024-05-08T18:05:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.124+0000 7f65e941f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.124+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf3e60 con 0x55a90ebf0e40 2024-05-08T18:05:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.124+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90ebf0e40 2024-05-08T18:05:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.125+0000 7f65e941f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.125+0000 7f65e941f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.125+0000 7f65e941f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.125+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4730 con 0x55a90ebf0e40 2024-05-08T18:05:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.125+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90ebf0e40 2024-05-08T18:05:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.125+0000 7f65e941f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.125+0000 7f65e941f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.125+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf0e40 2024-05-08T18:05:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.125+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90ebf0e40 2024-05-08T18:05:05.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.125+0000 7f65e941f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.125+0000 7f65e941f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.125+0000 7f65e941f800 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a90ebf5830 0x55a90ebf7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.125+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf8230 con 0x55a90ebf5830 2024-05-08T18:05:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.125+0000 7f65e6f1e640 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a90ebf5830 0x55a90ebf7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.126+0000 7f65e6f1e640 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a90ebf5830 0x55a90ebf7c20 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.126+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.126+0000 7f65e941f800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.126+0000 7f65e941f800 20 searching for the correct realm 2024-05-08T18:05:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.126+0000 7f65e941f800 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a90ebfa1e0 0x55a90ebfc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.126+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebfcd70 con 0x55a90ebfa1e0 2024-05-08T18:05:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.126+0000 7f65dffff640 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a90ebfa1e0 0x55a90ebfc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.126+0000 7f65dffff640 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a90ebfa1e0 0x55a90ebfc6a0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.127+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.127+0000 7f65dffff640 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f65d0082040 0x7f65d0074930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.127+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebfcd70 con 0x7f65d0082040 2024-05-08T18:05:05.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.127+0000 7f65e771f640 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f65d0082040 0x7f65d0074930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.127+0000 7f65e771f640 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f65d0082040 0x7f65d0074930 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.127+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65e000a4a0 con 0x7f65d0082040 2024-05-08T18:05:05.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.127+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf5830 2024-05-08T18:05:05.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.127+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.127+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf0e40 2024-05-08T18:05:05.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.128+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90ebf0e40 2024-05-08T18:05:05.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.128+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf0e40 2024-05-08T18:05:05.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.128+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90ebf0e40 2024-05-08T18:05:05.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.128+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf5830 2024-05-08T18:05:05.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.128+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.128+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf5830 2024-05-08T18:05:05.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.129+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.129+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x7f65d0082040 2024-05-08T18:05:05.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.129+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65e000a4a0 con 0x7f65d0082040 2024-05-08T18:05:05.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.129+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebfa1e0 2024-05-08T18:05:05.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.129+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.129+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf0e40 2024-05-08T18:05:05.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.129+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90ebf0e40 2024-05-08T18:05:05.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.129+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf5830 2024-05-08T18:05:05.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.130+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.130+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf0e40 2024-05-08T18:05:05.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.130+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90ebf0e40 2024-05-08T18:05:05.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.130+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x7f65d0082040 2024-05-08T18:05:05.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.130+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f65e000a4a0 con 0x7f65d0082040 2024-05-08T18:05:05.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.130+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebfa1e0 2024-05-08T18:05:05.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.130+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.130+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebfa1e0 2024-05-08T18:05:05.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.131+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.131+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf5830 2024-05-08T18:05:05.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.131+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.131+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebfa1e0 2024-05-08T18:05:05.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.131+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.131+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebfa1e0 2024-05-08T18:05:05.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.131+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.131+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf0e40 2024-05-08T18:05:05.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.132+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90ebf0e40 2024-05-08T18:05:05.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.132+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf0e40 2024-05-08T18:05:05.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.132+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90ebf0e40 2024-05-08T18:05:05.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.132+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf5830 2024-05-08T18:05:05.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.132+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.132+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebfa1e0 2024-05-08T18:05:05.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.132+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.132+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x7f65d0082040 2024-05-08T18:05:05.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.133+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65e000a4a0 con 0x7f65d0082040 2024-05-08T18:05:05.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.133+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf0e40 2024-05-08T18:05:05.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.133+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90ebf0e40 2024-05-08T18:05:05.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.133+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf5830 2024-05-08T18:05:05.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.133+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.133+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebfa1e0 2024-05-08T18:05:05.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.133+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.133+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf5830 2024-05-08T18:05:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.134+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.134+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x7f65d0082040 2024-05-08T18:05:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.134+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65e000a4a0 con 0x7f65d0082040 2024-05-08T18:05:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.134+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebfa1e0 2024-05-08T18:05:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.134+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.134+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf5830 2024-05-08T18:05:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.134+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.134+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x55a90ebf0e40 2024-05-08T18:05:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90ebf0e40 2024-05-08T18:05:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf4ad0 con 0x7f65d0082040 2024-05-08T18:05:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f65e000a4a0 con 0x7f65d0082040 2024-05-08T18:05:05.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e941f800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e941f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e941f800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e941f800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e941f800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e941f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e941f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf8af0 con 0x55a90ebf0e40 2024-05-08T18:05:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90ebf0e40 2024-05-08T18:05:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e941f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e941f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf8af0 con 0x55a90ebf0e40 2024-05-08T18:05:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f65e000a4a0 con 0x55a90ebf0e40 2024-05-08T18:05:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e941f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.135+0000 7f65e941f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf8af0 con 0x55a90ebfa1e0 2024-05-08T18:05:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e941f800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e941f800 20 zone default found 2024-05-08T18:05:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e941f800 4 Realm: () 2024-05-08T18:05:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e941f800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e941f800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e941f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e941f800 20 zonegroup default 2024-05-08T18:05:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e941f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebf9420 con 0x55a90ebf5830 2024-05-08T18:05:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e941f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e941f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebfec20 con 0x55a90ebfa1e0 2024-05-08T18:05:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e941f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e941f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.136+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebff080 con 0x55a90ebfa1e0 2024-05-08T18:05:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.137+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.137+0000 7f65e941f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.137+0000 7f65e941f800 20 started sync module instance, tier type = 2024-05-08T18:05:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.137+0000 7f65e941f800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.137+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebfe070 con 0x7f65d0082040 2024-05-08T18:05:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.138+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65e000a4a0 con 0x7f65d0082040 2024-05-08T18:05:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.138+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94184585291152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebfe070 con 0x7f65d0082040 2024-05-08T18:05:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.138+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebffb70 con 0x55a90ebf0e40 2024-05-08T18:05:05.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.139+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94184585291152] v27'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65e0040040 con 0x7f65d0082040 2024-05-08T18:05:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.139+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65e0040040 con 0x55a90ebf0e40 2024-05-08T18:05:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.139+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94184585295456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebfe070 con 0x55a90ebf0e40 2024-05-08T18:05:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.139+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec005f0 con 0x55a90ebfa1e0 2024-05-08T18:05:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.140+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94184585295456] v27'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65e0040040 con 0x55a90ebf0e40 2024-05-08T18:05:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.140+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.140+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94184585288640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebfdfe0 con 0x55a90ebfa1e0 2024-05-08T18:05:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.140+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec014e0 con 0x7f65d0082040 2024-05-08T18:05:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.141+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94184585288640] v27'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.141+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65e0040040 con 0x7f65d0082040 2024-05-08T18:05:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.141+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94184585302064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec01900 con 0x7f65d0082040 2024-05-08T18:05:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.141+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec022c0 con 0x55a90ebf0e40 2024-05-08T18:05:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.142+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65e0040040 con 0x55a90ebf0e40 2024-05-08T18:05:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.142+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94184585305616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec026e0 con 0x55a90ebf0e40 2024-05-08T18:05:05.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.142+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec031e0 con 0x55a90ebf5830 2024-05-08T18:05:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.142+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94184585302064] v27'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65e0040040 con 0x7f65d0082040 2024-05-08T18:05:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.143+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.143+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94184585309392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec035d0 con 0x55a90ebf5830 2024-05-08T18:05:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.143+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94184585305616] v27'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65e0040040 con 0x55a90ebf0e40 2024-05-08T18:05:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.143+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec03f90 con 0x55a90ebf5830 2024-05-08T18:05:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.144+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94184585309392] v27'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.144+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.144+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94184585312992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec04330 con 0x55a90ebf5830 2024-05-08T18:05:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.144+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec04bc0 con 0x55a90ebf5830 2024-05-08T18:05:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.145+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94184585312992] v27'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.145+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.145+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94184585306672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec04da0 con 0x55a90ebf5830 2024-05-08T18:05:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.145+0000 7f65e941f800 20 add_watcher() i=0 2024-05-08T18:05:05.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.145+0000 7f65e941f800 20 add_watcher() i=1 2024-05-08T18:05:05.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.145+0000 7f65e941f800 20 add_watcher() i=2 2024-05-08T18:05:05.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.145+0000 7f65e941f800 20 add_watcher() i=3 2024-05-08T18:05:05.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.145+0000 7f65e941f800 20 add_watcher() i=4 2024-05-08T18:05:05.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.145+0000 7f65e941f800 20 add_watcher() i=5 2024-05-08T18:05:05.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.145+0000 7f65e941f800 20 add_watcher() i=6 2024-05-08T18:05:05.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.146+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94184585306672] v27'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.146+0000 7f65e941f800 20 add_watcher() i=7 2024-05-08T18:05:05.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.146+0000 7f65e941f800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:05.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.146+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec08170 con 0x55a90ebf0e40 2024-05-08T18:05:05.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.147+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f65e0040040 con 0x55a90ebf0e40 2024-05-08T18:05:05.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.147+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94184585317520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec05560 con 0x55a90ebf0e40 2024-05-08T18:05:05.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.148+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94184585317520] v27'68 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f65e0040040 con 0x55a90ebf0e40 2024-05-08T18:05:05.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.148+0000 7f649d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:05.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.148+0000 7f65e941f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:05.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.148+0000 7f65e941f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:05.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.148+0000 7f65e941f800 5 note: GC not initialized 2024-05-08T18:05:05.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.148+0000 7f6417fff640 20 reqs_thread_entry: start 2024-05-08T18:05:05.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.148+0000 7f65e941f800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:05.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.148+0000 7f65e941f800 20 Filter name: none 2024-05-08T18:05:05.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.148+0000 7f6415ffb640 20 reqs_thread_entry: start 2024-05-08T18:05:05.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.148+0000 7f65e941f800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:05.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.148+0000 7f65e941f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.149+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec3b530 con 0x55a90ebf0e40 2024-05-08T18:05:05.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.149+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~502 out=502b,getxattrs out=75b] v0'0 uv4 ondisk = 0) ==== 231+0+625 (crc 0 0 0) 0x7f65e0040040 con 0x55a90ebf0e40 2024-05-08T18:05:05.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.149+0000 7f65e941f800 20 rados_obj.operate() r=0 bl.length=502 2024-05-08T18:05:05.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.149+0000 7f65e941f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:05.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.149+0000 7f65e941f800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:05.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.149+0000 7f65e941f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:05.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.149+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec41c90 con 0x55a90ebf0e40 2024-05-08T18:05:05.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.150+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~391] v27'5 uv5 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f65e0040040 con 0x55a90ebf0e40 2024-05-08T18:05:05.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.150+0000 7f65e941f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:05.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.150+0000 7f65e941f800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:05.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.150+0000 7f65e941f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-05-08T18:05:05.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.150+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94184585565776 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec3d2d0 con 0x55a90ebf0e40 2024-05-08T18:05:05.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.150+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== watch-notify(notify (1) cookie 94184585305616 notify 115964116996 ret 0) ==== 609+0+0 (crc 0 0 0) 0x7f65e0018d00 con 0x55a90ebf0e40 2024-05-08T18:05:05.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.150+0000 7f64ca7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116996 cookie 94184585305616 notifier 4347 bl.length()=567 2024-05-08T18:05:05.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.150+0000 7f64ca7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:05.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.150+0000 7f64ca7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:05.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.151+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(64 notify.4 [notify cookie 94184585565776 out=8b] v0'0 uv51 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f65e0040040 con 0x55a90ebf0e40 2024-05-08T18:05:05.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.151+0000 7f64ca7fc640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f64ac003ea0 con 0x55a90ebf0e40 2024-05-08T18:05:05.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.151+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== watch-notify(notify_complete (2) cookie 94184585565776 notify 115964116996 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f65e0038450 con 0x55a90ebf0e40 2024-05-08T18:05:05.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.151+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65e0040040 con 0x55a90ebf0e40 2024-05-08T18:05:05.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.151+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:66 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec41650 con 0x55a90ebf5830 2024-05-08T18:05:05.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.152+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(66 p5YnriCv1nAtykxBrupQ [delete] v27'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.152+0000 7f65e941f800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-05-08T18:05:05.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.152+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [notify cookie 94184585565776 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec3ce20 con 0x55a90ebfa1e0 2024-05-08T18:05:05.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.152+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== watch-notify(notify (1) cookie 94184585288640 notify 115964116993 ret 0) ==== 212+0+0 (crc 0 0 0) 0x7f65d008c040 con 0x55a90ebfa1e0 2024-05-08T18:05:05.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.152+0000 7f64c9ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116993 cookie 94184585288640 notifier 4347 bl.length()=170 2024-05-08T18:05:05.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.152+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 notify.2 [notify cookie 94184585565776 out=8b] v0'0 uv51 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.152+0000 7f64c9ffb640 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f641c0032c0 con 0x55a90ebfa1e0 2024-05-08T18:05:05.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.153+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== watch-notify(notify_complete (2) cookie 94184585565776 notify 115964116993 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a90ea2a930 con 0x55a90ebfa1e0 2024-05-08T18:05:05.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.153+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(68 notify.2 [notify-ack] v0'0 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.160 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:05.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.153+0000 7f649d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:05.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.153+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94184585317520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec3d230 con 0x55a90ebf0e40 2024-05-08T18:05:05.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.154+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94184585317520] v27'69 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f65e0040040 con 0x55a90ebf0e40 2024-05-08T18:05:05.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.154+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94184585291152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec04c90 con 0x7f65d0082040 2024-05-08T18:05:05.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.155+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94184585291152] v27'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65e0040040 con 0x7f65d0082040 2024-05-08T18:05:05.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.155+0000 7f65e941f800 20 remove_watcher() i=0 2024-05-08T18:05:05.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.155+0000 7f65e941f800 2 removed watcher, disabling cache 2024-05-08T18:05:05.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.155+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94184585295456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec04c90 con 0x55a90ebf0e40 2024-05-08T18:05:05.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.156+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94184585295456] v27'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65e0040040 con 0x55a90ebf0e40 2024-05-08T18:05:05.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.156+0000 7f65e941f800 20 remove_watcher() i=1 2024-05-08T18:05:05.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.156+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94184585288640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec04c90 con 0x55a90ebfa1e0 2024-05-08T18:05:05.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.157+0000 7f65dffff640 1 -- 172.21.15.50:0/709671018 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94184585288640] v27'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65d0071070 con 0x55a90ebfa1e0 2024-05-08T18:05:05.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.157+0000 7f65e941f800 20 remove_watcher() i=2 2024-05-08T18:05:05.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.157+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94184585302064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebfdbc0 con 0x7f65d0082040 2024-05-08T18:05:05.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.158+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94184585302064] v27'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65e0040040 con 0x7f65d0082040 2024-05-08T18:05:05.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.158+0000 7f65e941f800 20 remove_watcher() i=3 2024-05-08T18:05:05.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.158+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94184585305616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec04c90 con 0x55a90ebf0e40 2024-05-08T18:05:05.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.159+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94184585305616] v27'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65e0040040 con 0x55a90ebf0e40 2024-05-08T18:05:05.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.159+0000 7f65e941f800 20 remove_watcher() i=4 2024-05-08T18:05:05.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.159+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94184585309392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec01030 con 0x55a90ebf5830 2024-05-08T18:05:05.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.160+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94184585309392] v27'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.160+0000 7f65e941f800 20 remove_watcher() i=5 2024-05-08T18:05:05.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.160+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94184585312992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ebfdbc0 con 0x55a90ebf5830 2024-05-08T18:05:05.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.161+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94184585312992] v27'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.161+0000 7f65e941f800 20 remove_watcher() i=6 2024-05-08T18:05:05.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.161+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94184585306672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a90ec01e10 con 0x55a90ebf5830 2024-05-08T18:05:05.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.162+0000 7f65e6f1e640 1 -- 172.21.15.50:0/709671018 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94184585306672] v27'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f65d8037030 con 0x55a90ebf5830 2024-05-08T18:05:05.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.162+0000 7f65e941f800 20 remove_watcher() i=7 2024-05-08T18:05:05.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a90ebf5830 msgr2=0x55a90ebf7c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a90ebf5830 0x55a90ebf7c20 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a90ebf0e40 msgr2=0x55a90ebf3300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a90ebf0e40 0x55a90ebf3300 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f65d0082040 msgr2=0x7f65d0074930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f65d0082040 0x7f65d0074930 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a90ebfa1e0 msgr2=0x55a90ebfc6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a90ebfa1e0 0x55a90ebfc6a0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f64b003bba0 msgr2=0x7f64b003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f64b003bba0 0x7f64b003e060 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f65d0052fd0 tx=0x7f65d004f000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebde300 msgr2=0x55a90ebe5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e771f640 1 -- 172.21.15.50:0/709671018 reap_dead start 2024-05-08T18:05:05.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebde300 0x55a90ebe5780 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f65d8053570 tx=0x7f65d8046fa0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 shutdown_connections 2024-05-08T18:05:05.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 --2- 172.21.15.50:0/709671018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebde300 0x55a90ebe5780 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 >> 172.21.15.50:0/709671018 conn(0x55a90ebda660 msgr2=0x55a90ebd7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 shutdown_connections 2024-05-08T18:05:05.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 -- 172.21.15.50:0/709671018 wait complete. 2024-05-08T18:05:05.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 -- 172.21.15.50:0/2885892188 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f64c003bba0 msgr2=0x7f64c003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 --2- 172.21.15.50:0/2885892188 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f64c003bba0 0x7f64c003e060 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x55a90ebba980 tx=0x7f65d0005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:05.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 -- 172.21.15.50:0/2885892188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebd1080 msgr2=0x55a90ebd14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 --2- 172.21.15.50:0/2885892188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebd1080 0x55a90ebd14a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f65d8051990 tx=0x7f65d80533e0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 -- 172.21.15.50:0/2885892188 shutdown_connections 2024-05-08T18:05:05.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 --2- 172.21.15.50:0/2885892188 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f64c003bba0 0x7f64c003e060 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 --2- 172.21.15.50:0/2885892188 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90ebd1080 0x55a90ebd14a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 -- 172.21.15.50:0/2885892188 >> 172.21.15.50:0/2885892188 conn(0x55a90ebc5120 msgr2=0x55a90ebc5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 -- 172.21.15.50:0/2885892188 shutdown_connections 2024-05-08T18:05:05.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.163+0000 7f65e941f800 1 -- 172.21.15.50:0/2885892188 wait complete. 2024-05-08T18:05:05.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.164+0000 7f65e941f800 1 -- 172.21.15.50:0/3826150559 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a90ebbc3d0 msgr2=0x55a90ebbe7c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.164+0000 7f65e941f800 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a90ebbc3d0 0x55a90ebbe7c0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.164+0000 7f65e941f800 1 -- 172.21.15.50:0/3826150559 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a90eab7b10 msgr2=0x55a90ebb9b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.164+0000 7f65e941f800 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a90eab7b10 0x55a90ebb9b20 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.164+0000 7f65e941f800 1 -- 172.21.15.50:0/3826150559 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a90ebc0310 msgr2=0x55a90ebc2700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.164+0000 7f65e941f800 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a90ebc0310 0x55a90ebc2700 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.164+0000 7f65e941f800 1 -- 172.21.15.50:0/3826150559 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f64cc03bba0 msgr2=0x7f64cc03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.164+0000 7f65e941f800 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f64cc03bba0 0x7f64cc03e060 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f65d0054000 tx=0x7f65d004e000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.164+0000 7f65e941f800 1 -- 172.21.15.50:0/3826150559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab5fd0 msgr2=0x55a90ebb2980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.164+0000 7f65e941f800 1 --2- 172.21.15.50:0/3826150559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a90eab5fd0 0x55a90ebb2980 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55a90ebabb50 tx=0x7f65d8000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.164+0000 7f65e771f640 1 -- 172.21.15.50:0/3826150559 reap_dead start 2024-05-08T18:05:05.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.164+0000 7f65e941f800 1 -- 172.21.15.50:0/3826150559 shutdown_connections 2024-05-08T18:05:05.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.164+0000 7f65e941f800 1 -- 172.21.15.50:0/3826150559 >> 172.21.15.50:0/3826150559 conn(0x55a90eab1b70 msgr2=0x55a90eba4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.164+0000 7f65e941f800 1 -- 172.21.15.50:0/3826150559 shutdown_connections 2024-05-08T18:05:05.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.164+0000 7f65e941f800 1 -- 172.21.15.50:0/3826150559 wait complete. 2024-05-08T18:05:05.172 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:05.172 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2024-05-08T18:05:05.172 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2024-05-08T18:05:05.173 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo1 --access full 2024-05-08T18:05:05.220 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:05.221 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:05.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.223+0000 7f1b47f33800 1 Processor -- start 2024-05-08T18:05:05.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.223+0000 7f1b47f33800 1 -- start start 2024-05-08T18:05:05.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.223+0000 7f1b47f33800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ff40f76cb0 0x55ff40f790a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.223+0000 7f1b47f33800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55ff40f795e0 con 0x55ff40f731a0 2024-05-08T18:05:05.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.223+0000 7f1b47f33800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55ff40e12e20 con 0x55ff40f76cb0 2024-05-08T18:05:05.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.224+0000 7f1b433ed640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ff40f76cb0 0x55ff40f790a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.224+0000 7f1b433ed640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ff40f76cb0 0x55ff40f790a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36694/0 (socket says 172.21.15.50:36694) 2024-05-08T18:05:05.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.224+0000 7f1b43bee640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55ff40f731a0 0x55ff40f75590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43002/0 (socket says 172.21.15.50:43002) 2024-05-08T18:05:05.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.224+0000 7f1b433ed640 1 -- 172.21.15.50:0/1984673494 learned_addr learned my addr 172.21.15.50:0/1984673494 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.224+0000 7f1b433ed640 1 -- 172.21.15.50:0/1984673494 >> v1:172.21.15.46:6789/0 conn(0x55ff40f731a0 legacy=0x55ff40f75590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.224+0000 7f1b433ed640 1 -- 172.21.15.50:0/1984673494 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff40f7a7c0 con 0x55ff40f76cb0 2024-05-08T18:05:05.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.224+0000 7f1b42bec640 1 -- 172.21.15.50:0/1984673494 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4007980781 0 0) 0x55ff40f795e0 con 0x55ff40f731a0 2024-05-08T18:05:05.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.226+0000 7f1b433ed640 1 --2- 172.21.15.50:0/1984673494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f76cb0 0x55ff40f790a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00a520 tx=0x7f1b2c01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b5d8e73692f484 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.226+0000 7f1b42bec640 1 -- 172.21.15.50:0/1984673494 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b2c057070 con 0x55ff40f76cb0 2024-05-08T18:05:05.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.226+0000 7f1b42bec640 1 -- 172.21.15.50:0/1984673494 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b2c052570 con 0x55ff40f76cb0 2024-05-08T18:05:05.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.226+0000 7f1b42bec640 1 -- 172.21.15.50:0/1984673494 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b2c04ae60 con 0x55ff40f76cb0 2024-05-08T18:05:05.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.227+0000 7f1b47f33800 1 -- 172.21.15.50:0/1984673494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f76cb0 msgr2=0x55ff40f790a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.227+0000 7f1b47f33800 1 --2- 172.21.15.50:0/1984673494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f76cb0 0x55ff40f790a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c00a520 tx=0x7f1b2c01d010 comp rx=0 tx=0).stop 2024-05-08T18:05:05.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.227+0000 7f1b47f33800 1 -- 172.21.15.50:0/1984673494 shutdown_connections 2024-05-08T18:05:05.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.227+0000 7f1b47f33800 1 --2- 172.21.15.50:0/1984673494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f76cb0 0x55ff40f790a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.227+0000 7f1b47f33800 1 -- 172.21.15.50:0/1984673494 >> 172.21.15.50:0/1984673494 conn(0x55ff40f6cb70 msgr2=0x55ff40f6ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.227+0000 7f1b47f33800 1 -- 172.21.15.50:0/1984673494 shutdown_connections 2024-05-08T18:05:05.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.227+0000 7f1b47f33800 1 -- 172.21.15.50:0/1984673494 wait complete. 2024-05-08T18:05:05.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.242+0000 7f1b47f33800 1 Processor -- start 2024-05-08T18:05:05.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.242+0000 7f1b47f33800 1 -- start start 2024-05-08T18:05:05.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.242+0000 7f1b47f33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f731a0 0x55ff40f72ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.242+0000 7f1b47f33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ff40ebda30 con 0x55ff40f731a0 2024-05-08T18:05:05.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.243+0000 7f1b43bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f731a0 0x55ff40f72ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.243+0000 7f1b43bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f731a0 0x55ff40f72ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36702/0 (socket says 172.21.15.50:36702) 2024-05-08T18:05:05.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.243+0000 7f1b43bee640 1 -- 172.21.15.50:0/2903785328 learned_addr learned my addr 172.21.15.50:0/2903785328 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.243+0000 7f1b43bee640 1 -- 172.21.15.50:0/2903785328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff40f710a0 con 0x55ff40f731a0 2024-05-08T18:05:05.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.243+0000 7f1b43bee640 1 --2- 172.21.15.50:0/2903785328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f731a0 0x55ff40f72ae0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1b34001e70 tx=0x7f1b3400b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=778e46155cff0598 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.243+0000 7f1a31e0b640 1 -- 172.21.15.50:0/2903785328 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b34008070 con 0x55ff40f731a0 2024-05-08T18:05:05.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.243+0000 7f1a31e0b640 1 -- 172.21.15.50:0/2903785328 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3400d500 con 0x55ff40f731a0 2024-05-08T18:05:05.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.243+0000 7f1a31e0b640 1 -- 172.21.15.50:0/2903785328 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b3400d8a0 con 0x55ff40f731a0 2024-05-08T18:05:05.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.244+0000 7f1b47f33800 1 -- 172.21.15.50:0/2903785328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f731a0 msgr2=0x55ff40f72ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.244+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2903785328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f731a0 0x55ff40f72ae0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1b34001e70 tx=0x7f1b3400b6a0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.244+0000 7f1b47f33800 1 -- 172.21.15.50:0/2903785328 shutdown_connections 2024-05-08T18:05:05.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.244+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2903785328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f731a0 0x55ff40f72ae0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.244+0000 7f1b47f33800 1 -- 172.21.15.50:0/2903785328 >> 172.21.15.50:0/2903785328 conn(0x55ff40f6cb70 msgr2=0x55ff4105f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.244+0000 7f1b47f33800 1 -- 172.21.15.50:0/2903785328 shutdown_connections 2024-05-08T18:05:05.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.244+0000 7f1b47f33800 1 -- 172.21.15.50:0/2903785328 wait complete. 2024-05-08T18:05:05.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.244+0000 7f1b47f33800 1 Processor -- start 2024-05-08T18:05:05.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.244+0000 7f1b47f33800 1 -- start start 2024-05-08T18:05:05.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.244+0000 7f1b47f33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f731a0 0x55ff4106d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.244+0000 7f1b47f33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ff40f79ab0 con 0x55ff40f731a0 2024-05-08T18:05:05.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.245+0000 7f1b43bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f731a0 0x55ff4106d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.245+0000 7f1b43bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f731a0 0x55ff4106d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36708/0 (socket says 172.21.15.50:36708) 2024-05-08T18:05:05.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.245+0000 7f1b43bee640 1 -- 172.21.15.50:0/2275072066 learned_addr learned my addr 172.21.15.50:0/2275072066 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.245+0000 7f1b43bee640 1 -- 172.21.15.50:0/2275072066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff4106dc80 con 0x55ff40f731a0 2024-05-08T18:05:05.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.245+0000 7f1b43bee640 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f731a0 0x55ff4106d740 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55ff40f72110 tx=0x7f1b34000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.245+0000 7f1a2bfff640 1 -- 172.21.15.50:0/2275072066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b3400f980 con 0x55ff40f731a0 2024-05-08T18:05:05.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.245+0000 7f1a2bfff640 1 -- 172.21.15.50:0/2275072066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3400fb20 con 0x55ff40f731a0 2024-05-08T18:05:05.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.245+0000 7f1b47f33800 1 -- 172.21.15.50:0/2275072066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ff4106df10 con 0x55ff40f731a0 2024-05-08T18:05:05.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.245+0000 7f1a2bfff640 1 -- 172.21.15.50:0/2275072066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b3400fe00 con 0x55ff40f731a0 2024-05-08T18:05:05.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.246+0000 7f1b47f33800 1 -- 172.21.15.50:0/2275072066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff41070bc0 con 0x55ff40f731a0 2024-05-08T18:05:05.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.246+0000 7f1a2bfff640 1 -- 172.21.15.50:0/2275072066 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f1b34017040 con 0x55ff40f731a0 2024-05-08T18:05:05.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.247+0000 7f1a2bfff640 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1a2c03bba0 0x7f1a2c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.247+0000 7f1a2bfff640 1 -- 172.21.15.50:0/2275072066 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f1b34015020 con 0x55ff40f731a0 2024-05-08T18:05:05.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.247+0000 7f1b433ed640 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1a2c03bba0 0x7f1a2c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.247+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ff40f76cb0 0x55ff41074980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.247+0000 7f1b47f33800 1 -- 172.21.15.50:0/2275072066 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff41074ec0 con 0x55ff40f76cb0 2024-05-08T18:05:05.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.247+0000 7f1b443ef640 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ff40f76cb0 0x55ff41074980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.247+0000 7f1b433ed640 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1a2c03bba0 0x7f1a2c03e060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c054000 tx=0x7f1b2c04e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.247+0000 7f1b443ef640 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ff40f76cb0 0x55ff41074980 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.248+0000 7f1b443ef640 1 -- 172.21.15.50:0/2275072066 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff40f76cb0 2024-05-08T18:05:05.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.248+0000 7f1b47f33800 1 -- 172.21.15.50:0/2275072066 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff41071f50 con 0x55ff40f76cb0 2024-05-08T18:05:05.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.248+0000 7f1b443ef640 1 -- 172.21.15.50:0/2275072066 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff40f76cb0 2024-05-08T18:05:05.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.248+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ff410771d0 0x55ff410795c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.248+0000 7f1b47f33800 1 -- 172.21.15.50:0/2275072066 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff41079cb0 con 0x55ff410771d0 2024-05-08T18:05:05.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.248+0000 7f1b43bee640 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ff410771d0 0x55ff410795c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.248+0000 7f1b43bee640 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ff410771d0 0x55ff410795c0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.249+0000 7f1b43bee640 1 -- 172.21.15.50:0/2275072066 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f1b3402d070 con 0x55ff410771d0 2024-05-08T18:05:05.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.249+0000 7f1b47f33800 1 -- 172.21.15.50:0/2275072066 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff41076070 con 0x55ff40f76cb0 2024-05-08T18:05:05.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.249+0000 7f1b443ef640 1 -- 172.21.15.50:0/2275072066 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff40f76cb0 2024-05-08T18:05:05.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.249+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ff4107b110 0x55ff4107d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.249+0000 7f1b47f33800 1 -- 172.21.15.50:0/2275072066 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff4107dc00 con 0x55ff4107b110 2024-05-08T18:05:05.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.249+0000 7f1b433ed640 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ff4107b110 0x55ff4107d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.250+0000 7f1b433ed640 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ff4107b110 0x55ff4107d500 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.250+0000 7f1b433ed640 1 -- 172.21.15.50:0/2275072066 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f1b2c05a040 con 0x55ff4107b110 2024-05-08T18:05:05.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.250+0000 7f1b47f33800 1 Processor -- start 2024-05-08T18:05:05.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.250+0000 7f1b47f33800 1 -- start start 2024-05-08T18:05:05.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.250+0000 7f1b47f33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff4108bf70 0x55ff4108c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.251+0000 7f1b47f33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ff40e64330 con 0x55ff4108bf70 2024-05-08T18:05:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.251+0000 7f1b43bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff4108bf70 0x55ff4108c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.251+0000 7f1b43bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff4108bf70 0x55ff4108c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36720/0 (socket says 172.21.15.50:36720) 2024-05-08T18:05:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.251+0000 7f1b43bee640 1 -- 172.21.15.50:0/3837651347 learned_addr learned my addr 172.21.15.50:0/3837651347 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.251+0000 7f1b43bee640 1 -- 172.21.15.50:0/3837651347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff4108c8d0 con 0x55ff4108bf70 2024-05-08T18:05:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.251+0000 7f1b43bee640 1 --2- 172.21.15.50:0/3837651347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff4108bf70 0x55ff4108c390 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400ecf0 tx=0x7f1b3401ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.251+0000 7f1a297fa640 1 -- 172.21.15.50:0/3837651347 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b3404a350 con 0x55ff4108bf70 2024-05-08T18:05:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.251+0000 7f1a297fa640 1 -- 172.21.15.50:0/3837651347 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3401a3c0 con 0x55ff4108bf70 2024-05-08T18:05:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.251+0000 7f1b47f33800 1 -- 172.21.15.50:0/3837651347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ff4108d340 con 0x55ff4108bf70 2024-05-08T18:05:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.251+0000 7f1a297fa640 1 -- 172.21.15.50:0/3837651347 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b340501b0 con 0x55ff4108bf70 2024-05-08T18:05:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.251+0000 7f1b47f33800 1 -- 172.21.15.50:0/3837651347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff4108d840 con 0x55ff4108bf70 2024-05-08T18:05:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.252+0000 7f1a297fa640 1 -- 172.21.15.50:0/3837651347 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f1b34051b70 con 0x55ff4108bf70 2024-05-08T18:05:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.252+0000 7f1a297fa640 1 --2- 172.21.15.50:0/3837651347 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1a1c03bb50 0x7f1a1c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.252+0000 7f1a297fa640 1 -- 172.21.15.50:0/3837651347 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f1b3403c940 con 0x55ff4108bf70 2024-05-08T18:05:05.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.252+0000 7f1b433ed640 1 --2- 172.21.15.50:0/3837651347 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1a1c03bb50 0x7f1a1c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.253+0000 7f1b47f33800 1 Processor -- start 2024-05-08T18:05:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.253+0000 7f1b47f33800 1 -- start start 2024-05-08T18:05:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.253+0000 7f1b433ed640 1 --2- 172.21.15.50:0/3837651347 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1a1c03bb50 0x7f1a1c03e010 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55ff41075780 tx=0x7f1b2c005ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.253+0000 7f1b47f33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff410991f0 0x55ff41099610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.253+0000 7f1b47f33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ff40f71420 con 0x55ff410991f0 2024-05-08T18:05:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.253+0000 7f1b43bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff410991f0 0x55ff41099610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.253+0000 7f1b43bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff410991f0 0x55ff41099610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36736/0 (socket says 172.21.15.50:36736) 2024-05-08T18:05:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.253+0000 7f1b43bee640 1 -- 172.21.15.50:0/403852405 learned_addr learned my addr 172.21.15.50:0/403852405 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.253+0000 7f1b43bee640 1 -- 172.21.15.50:0/403852405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff41099be0 con 0x55ff410991f0 2024-05-08T18:05:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.254+0000 7f1b43bee640 1 --2- 172.21.15.50:0/403852405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff410991f0 0x55ff41099610 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1b3403de80 tx=0x7f1b3402f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a210bed1ea7f107c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.254+0000 7f1a22ffd640 1 -- 172.21.15.50:0/403852405 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b34042c90 con 0x55ff410991f0 2024-05-08T18:05:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.254+0000 7f1a22ffd640 1 -- 172.21.15.50:0/403852405 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3402f5a0 con 0x55ff410991f0 2024-05-08T18:05:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.254+0000 7f1a22ffd640 1 -- 172.21.15.50:0/403852405 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b34026d30 con 0x55ff410991f0 2024-05-08T18:05:05.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.254+0000 7f1b47f33800 1 -- 172.21.15.50:0/403852405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff410991f0 msgr2=0x55ff41099610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.254+0000 7f1b47f33800 1 --2- 172.21.15.50:0/403852405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff410991f0 0x55ff41099610 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1b3403de80 tx=0x7f1b3402f8c0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.254+0000 7f1b47f33800 1 -- 172.21.15.50:0/403852405 shutdown_connections 2024-05-08T18:05:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.254+0000 7f1b47f33800 1 --2- 172.21.15.50:0/403852405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff410991f0 0x55ff41099610 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.254+0000 7f1b47f33800 1 -- 172.21.15.50:0/403852405 >> 172.21.15.50:0/403852405 conn(0x55ff41095550 msgr2=0x55ff410935b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.254+0000 7f1b47f33800 1 -- 172.21.15.50:0/403852405 shutdown_connections 2024-05-08T18:05:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.254+0000 7f1b47f33800 1 -- 172.21.15.50:0/403852405 wait complete. 2024-05-08T18:05:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.255+0000 7f1b47f33800 1 Processor -- start 2024-05-08T18:05:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.255+0000 7f1b47f33800 1 -- start start 2024-05-08T18:05:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.255+0000 7f1b47f33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff410991f0 0x55ff410a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.255+0000 7f1b47f33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ff4109a290 con 0x55ff410991f0 2024-05-08T18:05:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.255+0000 7f1b43bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff410991f0 0x55ff410a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.255+0000 7f1b43bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff410991f0 0x55ff410a05d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36748/0 (socket says 172.21.15.50:36748) 2024-05-08T18:05:05.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.255+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 learned_addr learned my addr 172.21.15.50:0/2288427555 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.255+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff410a0b10 con 0x55ff410991f0 2024-05-08T18:05:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.255+0000 7f1b43bee640 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff410991f0 0x55ff410a05d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1b3403de80 tx=0x7f1b34016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.256+0000 7f1a217fa640 1 -- 172.21.15.50:0/2288427555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b34019680 con 0x55ff410991f0 2024-05-08T18:05:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.256+0000 7f1a217fa640 1 -- 172.21.15.50:0/2288427555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3402fa30 con 0x55ff410991f0 2024-05-08T18:05:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.256+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ff410a0b10 con 0x55ff410991f0 2024-05-08T18:05:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.256+0000 7f1a217fa640 1 -- 172.21.15.50:0/2288427555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b3402fbe0 con 0x55ff410991f0 2024-05-08T18:05:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.256+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff410a0b10 con 0x55ff410991f0 2024-05-08T18:05:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.256+0000 7f1a217fa640 1 -- 172.21.15.50:0/2288427555 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f1b34039400 con 0x55ff410991f0 2024-05-08T18:05:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.257+0000 7f1a217fa640 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1a0c03bba0 0x7f1a0c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.257+0000 7f1a217fa640 1 -- 172.21.15.50:0/2288427555 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f1b34065070 con 0x55ff410991f0 2024-05-08T18:05:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.257+0000 7f1b433ed640 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1a0c03bba0 0x7f1a0c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.257+0000 7f1b433ed640 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1a0c03bba0 0x7f1a0c03e060 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c000c00 tx=0x7f1b2c04f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.257+0000 7f1b47f33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.257+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ff410abca0 0x55ff410ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.257+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff410ae8b0 con 0x55ff410abca0 2024-05-08T18:05:05.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.257+0000 7f1b443ef640 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ff410abca0 0x55ff410ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.258+0000 7f1b443ef640 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ff410abca0 0x55ff410ae160 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.258+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff410abca0 2024-05-08T18:05:05.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.258+0000 7f1b47f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.258+0000 7f1b47f33800 20 realm 2024-05-08T18:05:05.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.258+0000 7f1b47f33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.258+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff410aecc0 con 0x55ff410abca0 2024-05-08T18:05:05.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.258+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff410abca0 2024-05-08T18:05:05.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.258+0000 7f1b47f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.258+0000 7f1b47f33800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:05.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.258+0000 7f1b47f33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.258+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff410af590 con 0x55ff410abca0 2024-05-08T18:05:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.258+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff410abca0 2024-05-08T18:05:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.258+0000 7f1b47f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.258+0000 7f1b47f33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.258+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff410af930 con 0x55ff410abca0 2024-05-08T18:05:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.259+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff410abca0 2024-05-08T18:05:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.259+0000 7f1b47f33800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.259+0000 7f1b47f33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.259+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ff410b0690 0x55ff410b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.259+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff410b3090 con 0x55ff410b0690 2024-05-08T18:05:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.259+0000 7f1b43bee640 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ff410b0690 0x55ff410b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.259+0000 7f1b43bee640 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ff410b0690 0x55ff410b2a80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.259+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.259+0000 7f1b47f33800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.259+0000 7f1b47f33800 20 searching for the correct realm 2024-05-08T18:05:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.259+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ff410b5040 0x55ff410b7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.259+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ff410b7bd0 con 0x55ff410b5040 2024-05-08T18:05:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.260+0000 7f1b433ed640 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ff410b5040 0x55ff410b7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.260+0000 7f1b433ed640 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ff410b5040 0x55ff410b7500 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.260+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b2c07f070 con 0x55ff410b5040 2024-05-08T18:05:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.260+0000 7f1b433ed640 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f1b2c074040 0x7f1b2c081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.260+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ff410b7bd0 con 0x7f1b2c074040 2024-05-08T18:05:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.260+0000 7f1b443ef640 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f1b2c074040 0x7f1b2c081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.261+0000 7f1b443ef640 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f1b2c074040 0x7f1b2c081410 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.261+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x7f1b2c074040 2024-05-08T18:05:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.261+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ff410af930 con 0x55ff410b0690 2024-05-08T18:05:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.261+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.261+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ff410af930 con 0x55ff410abca0 2024-05-08T18:05:05.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.261+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff410abca0 2024-05-08T18:05:05.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.261+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ff410af930 con 0x55ff410abca0 2024-05-08T18:05:05.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.261+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff410abca0 2024-05-08T18:05:05.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.261+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ff410af930 con 0x55ff410b0690 2024-05-08T18:05:05.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.262+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.262+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ff410af930 con 0x55ff410b0690 2024-05-08T18:05:05.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.262+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.262+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ff410af930 con 0x7f1b2c074040 2024-05-08T18:05:05.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.262+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x7f1b2c074040 2024-05-08T18:05:05.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.262+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ff410af930 con 0x55ff410b5040 2024-05-08T18:05:05.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.262+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b2c07f070 con 0x55ff410b5040 2024-05-08T18:05:05.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.262+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ff410af930 con 0x55ff410abca0 2024-05-08T18:05:05.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.263+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff410abca0 2024-05-08T18:05:05.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.263+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ff410af930 con 0x55ff410b0690 2024-05-08T18:05:05.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.263+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.263+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ff410af930 con 0x55ff410abca0 2024-05-08T18:05:05.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.263+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff410abca0 2024-05-08T18:05:05.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.263+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ff410af930 con 0x7f1b2c074040 2024-05-08T18:05:05.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.263+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x7f1b2c074040 2024-05-08T18:05:05.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.263+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ff410af930 con 0x55ff410b5040 2024-05-08T18:05:05.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.264+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b2c07f070 con 0x55ff410b5040 2024-05-08T18:05:05.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.264+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x55ff410b5040 2024-05-08T18:05:05.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.264+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b2c07f070 con 0x55ff410b5040 2024-05-08T18:05:05.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.264+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x55ff410b0690 2024-05-08T18:05:05.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.264+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.264+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x55ff410b5040 2024-05-08T18:05:05.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.264+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b2c07f070 con 0x55ff410b5040 2024-05-08T18:05:05.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.264+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x55ff410b5040 2024-05-08T18:05:05.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.264+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f1b2c07f070 con 0x55ff410b5040 2024-05-08T18:05:05.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.264+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x55ff410abca0 2024-05-08T18:05:05.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.265+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff410abca0 2024-05-08T18:05:05.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.265+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x55ff410abca0 2024-05-08T18:05:05.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.265+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff410abca0 2024-05-08T18:05:05.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.265+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x55ff410b0690 2024-05-08T18:05:05.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.265+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.265+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x55ff410b5040 2024-05-08T18:05:05.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.265+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b2c07f070 con 0x55ff410b5040 2024-05-08T18:05:05.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.265+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x7f1b2c074040 2024-05-08T18:05:05.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.265+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x7f1b2c074040 2024-05-08T18:05:05.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.266+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x55ff410abca0 2024-05-08T18:05:05.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.266+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff410abca0 2024-05-08T18:05:05.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.266+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x55ff410b0690 2024-05-08T18:05:05.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.266+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.266+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x55ff410b5040 2024-05-08T18:05:05.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.266+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b2c07f070 con 0x55ff410b5040 2024-05-08T18:05:05.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.266+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x55ff410b0690 2024-05-08T18:05:05.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.267+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.267+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x7f1b2c074040 2024-05-08T18:05:05.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.267+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x7f1b2c074040 2024-05-08T18:05:05.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.267+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x55ff410b5040 2024-05-08T18:05:05.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.267+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b2c07f070 con 0x55ff410b5040 2024-05-08T18:05:05.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.267+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x55ff410b0690 2024-05-08T18:05:05.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.267+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.267+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x55ff410abca0 2024-05-08T18:05:05.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff410abca0 2024-05-08T18:05:05.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f1b2c08e040 con 0x7f1b2c074040 2024-05-08T18:05:05.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x7f1b2c074040 2024-05-08T18:05:05.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b47f33800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:05.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b47f33800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:05.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b47f33800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:05.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b47f33800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:05.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b47f33800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:05.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b47f33800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:05.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b47f33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff410b3950 con 0x55ff410abca0 2024-05-08T18:05:05.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff410abca0 2024-05-08T18:05:05.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b47f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b47f33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff410b3950 con 0x55ff410abca0 2024-05-08T18:05:05.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x55ff410abca0 2024-05-08T18:05:05.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b47f33800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:05.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b47f33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.268+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff410b3950 con 0x55ff410b5040 2024-05-08T18:05:05.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.269+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f1b2c07f070 con 0x55ff410b5040 2024-05-08T18:05:05.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.269+0000 7f1b47f33800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:05.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.269+0000 7f1b47f33800 20 zone default found 2024-05-08T18:05:05.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.269+0000 7f1b47f33800 4 Realm: () 2024-05-08T18:05:05.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.269+0000 7f1b47f33800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:05.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.269+0000 7f1b47f33800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:05.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.269+0000 7f1b47f33800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:05.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.269+0000 7f1b47f33800 20 zonegroup default 2024-05-08T18:05:05.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.269+0000 7f1b47f33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.269+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff410b4280 con 0x55ff410b0690 2024-05-08T18:05:05.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.269+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.269+0000 7f1b47f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.269+0000 7f1b47f33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.269+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff410b9a90 con 0x55ff410b5040 2024-05-08T18:05:05.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.269+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f1b2c07f070 con 0x55ff410b5040 2024-05-08T18:05:05.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.270+0000 7f1b47f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.270+0000 7f1b47f33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.270+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff410b9f20 con 0x55ff410b5040 2024-05-08T18:05:05.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.270+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f1b2c07f070 con 0x55ff410b5040 2024-05-08T18:05:05.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.270+0000 7f1b47f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.270+0000 7f1b47f33800 20 started sync module instance, tier type = 2024-05-08T18:05:05.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.270+0000 7f1b47f33800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:05.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.270+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410b8ed0 con 0x7f1b2c074040 2024-05-08T18:05:05.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.271+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3c00a4a0 con 0x7f1b2c074040 2024-05-08T18:05:05.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.271+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94554796299216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410b8ed0 con 0x7f1b2c074040 2024-05-08T18:05:05.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.271+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410ba9e0 con 0x55ff410abca0 2024-05-08T18:05:05.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.272+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94554796299216] v27'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x7f1b2c074040 2024-05-08T18:05:05.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.272+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x55ff410abca0 2024-05-08T18:05:05.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.272+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94554796303600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410b8ed0 con 0x55ff410abca0 2024-05-08T18:05:05.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.272+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410bb470 con 0x55ff410b5040 2024-05-08T18:05:05.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.273+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94554796303600] v27'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x55ff410abca0 2024-05-08T18:05:05.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.273+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b2c07f070 con 0x55ff410b5040 2024-05-08T18:05:05.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.273+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94554796296736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410b8e40 con 0x55ff410b5040 2024-05-08T18:05:05.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.273+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410bc330 con 0x7f1b2c074040 2024-05-08T18:05:05.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.274+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94554796296736] v27'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b2c07f070 con 0x55ff410b5040 2024-05-08T18:05:05.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.274+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x7f1b2c074040 2024-05-08T18:05:05.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.274+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94554796310144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410bc2a0 con 0x7f1b2c074040 2024-05-08T18:05:05.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.274+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410bd0f0 con 0x55ff410abca0 2024-05-08T18:05:05.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.276+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x55ff410abca0 2024-05-08T18:05:05.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.276+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94554796313552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410bd0f0 con 0x55ff410abca0 2024-05-08T18:05:05.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.276+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410be030 con 0x55ff410b0690 2024-05-08T18:05:05.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.276+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94554796310144] v27'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x7f1b2c074040 2024-05-08T18:05:05.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.277+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94554796313552] v27'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x55ff410abca0 2024-05-08T18:05:05.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.277+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.277+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94554796315872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410bc6f0 con 0x55ff410b0690 2024-05-08T18:05:05.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.277+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410be8d0 con 0x55ff410b0690 2024-05-08T18:05:05.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.277+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94554796315872] v27'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.278+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.278+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94554796317440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410bf120 con 0x55ff410b0690 2024-05-08T18:05:05.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.278+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410bfae0 con 0x55ff410b0690 2024-05-08T18:05:05.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.279+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94554796317440] v27'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.279+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.279+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94554796324400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410bff00 con 0x55ff410b0690 2024-05-08T18:05:05.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.279+0000 7f1b47f33800 20 add_watcher() i=0 2024-05-08T18:05:05.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.279+0000 7f1b47f33800 20 add_watcher() i=1 2024-05-08T18:05:05.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.279+0000 7f1b47f33800 20 add_watcher() i=2 2024-05-08T18:05:05.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.279+0000 7f1b47f33800 20 add_watcher() i=3 2024-05-08T18:05:05.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.279+0000 7f1b47f33800 20 add_watcher() i=4 2024-05-08T18:05:05.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.279+0000 7f1b47f33800 20 add_watcher() i=5 2024-05-08T18:05:05.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.279+0000 7f1b47f33800 20 add_watcher() i=6 2024-05-08T18:05:05.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.280+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94554796324400] v27'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.280+0000 7f1b47f33800 20 add_watcher() i=7 2024-05-08T18:05:05.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.280+0000 7f1b47f33800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:05.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.280+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff410c31c0 con 0x55ff410abca0 2024-05-08T18:05:05.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.280+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f1b3c040040 con 0x55ff410abca0 2024-05-08T18:05:05.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.280+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94554796325824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410c0490 con 0x55ff410abca0 2024-05-08T18:05:05.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.281+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94554796325824] v27'70 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x55ff410abca0 2024-05-08T18:05:05.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.281+0000 7f19f9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:05.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.281+0000 7f1b47f33800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:05.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.281+0000 7f1b47f33800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:05.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.281+0000 7f1b47f33800 5 note: GC not initialized 2024-05-08T18:05:05.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.281+0000 7f19f8fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:05.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.281+0000 7f1b47f33800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:05.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f1b47f33800 20 Filter name: none 2024-05-08T18:05:05.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f19727fc640 20 reqs_thread_entry: start 2024-05-08T18:05:05.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f1b47f33800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:05.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f1b47f33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff410f67f0 con 0x55ff410abca0 2024-05-08T18:05:05.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f1b3c040040 con 0x55ff410abca0 2024-05-08T18:05:05.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f1b47f33800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:05.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f1b47f33800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:05.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f1b47f33800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:05.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f1b47f33800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:05.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f1b47f33800 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : miss 2024-05-08T18:05:05.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f1b47f33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.8 8:17d68d04:users.swift::foo%3afoo1:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff410f7260 con 0x55ff410abca0 2024-05-08T18:05:05.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 foo:foo1 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 194+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x55ff410abca0 2024-05-08T18:05:05.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f1b47f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f1b47f33800 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x0 2024-05-08T18:05:05.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.282+0000 7f1b47f33800 10 adding default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-05-08T18:05:05.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.283+0000 7f1b47f33800 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : hit (negative entry) 2024-05-08T18:05:05.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.283+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~556 in=556b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410fccc0 con 0x55ff410abca0 2024-05-08T18:05:05.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.284+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~556] v27'6 uv6 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x55ff410abca0 2024-05-08T18:05:05.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.284+0000 7f1b47f33800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:05.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.284+0000 7f1b47f33800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:05.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.284+0000 7f1b47f33800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-05-08T18:05:05.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.284+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94554796574336 in=744b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff410f8270 con 0x55ff410abca0 2024-05-08T18:05:05.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.284+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== watch-notify(notify (1) cookie 94554796313552 notify 115964116997 ret 0) ==== 774+0+0 (crc 0 0 0) 0x7f1b3c018cd0 con 0x55ff410abca0 2024-05-08T18:05:05.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.284+0000 7f1a22ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116997 cookie 94554796313552 notifier 4354 bl.length()=732 2024-05-08T18:05:05.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.284+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(65 notify.4 [notify cookie 94554796574336 out=8b] v0'0 uv54 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f1b3c040040 con 0x55ff410abca0 2024-05-08T18:05:05.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.284+0000 7f1a22ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:05.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.284+0000 7f1a22ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:05.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.284+0000 7f1a22ffd640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f1a08003ea0 con 0x55ff410abca0 2024-05-08T18:05:05.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.284+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x55ff410abca0 2024-05-08T18:05:05.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.284+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== watch-notify(notify_complete (2) cookie 94554796574336 notify 115964116997 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f1b3c038450 con 0x55ff410abca0 2024-05-08T18:05:05.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.284+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410fc680 con 0x55ff410abca0 2024-05-08T18:05:05.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.286+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(67 foo:foo1 [delete,create,writefull 0~7] v27'1 uv1 ondisk = 0) ==== 236+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x55ff410abca0 2024-05-08T18:05:05.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.286+0000 7f1b47f33800 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2024-05-08T18:05:05.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.286+0000 7f1b47f33800 10 moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-05-08T18:05:05.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.286+0000 7f1b47f33800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2024-05-08T18:05:05.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.286+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94554796574336 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ff410f7dc0 con 0x7f1b2c074040 2024-05-08T18:05:05.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.286+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== watch-notify(notify (1) cookie 94554796299216 notify 115964116994 ret 0) ==== 208+0+0 (crc 0 0 0) 0x7f1b3c038450 con 0x7f1b2c074040 2024-05-08T18:05:05.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.286+0000 7f1a22ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116994 cookie 94554796299216 notifier 4354 bl.length()=166 2024-05-08T18:05:05.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.286+0000 7f1a22ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2024-05-08T18:05:05.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.286+0000 7f1a22ffd640 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-05-08T18:05:05.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.286+0000 7f1a22ffd640 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f1a08003ea0 con 0x7f1b2c074040 2024-05-08T18:05:05.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.286+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(68 notify.0 [notify cookie 94554796574336 out=8b] v0'0 uv54 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f1b3c040040 con 0x7f1b2c074040 2024-05-08T18:05:05.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.286+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== watch-notify(notify_complete (2) cookie 94554796574336 notify 115964116994 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f1b3c0386f0 con 0x7f1b2c074040 2024-05-08T18:05:05.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.286+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(69 notify.0 [notify-ack] v0'0 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x7f1b2c074040 2024-05-08T18:05:05.296 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"vDzZuBVsJ1sSPV9wc9BFdG5KDBUSvDNNEWQFwucZ","active":true,"create_date":"2024-05-08T18:05:05.283685Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:05.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.287+0000 7f19f9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:05.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.287+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94554796325824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410c2410 con 0x55ff410abca0 2024-05-08T18:05:05.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.288+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94554796325824] v27'71 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x55ff410abca0 2024-05-08T18:05:05.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.288+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94554796299216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410bfbc0 con 0x7f1b2c074040 2024-05-08T18:05:05.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.289+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94554796299216] v27'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x7f1b2c074040 2024-05-08T18:05:05.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.289+0000 7f1b47f33800 20 remove_watcher() i=0 2024-05-08T18:05:05.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.289+0000 7f1b47f33800 2 removed watcher, disabling cache 2024-05-08T18:05:05.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.289+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94554796303600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410bfbc0 con 0x55ff410abca0 2024-05-08T18:05:05.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.290+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94554796303600] v27'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x55ff410abca0 2024-05-08T18:05:05.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.290+0000 7f1b47f33800 20 remove_watcher() i=1 2024-05-08T18:05:05.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.290+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94554796296736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410ba4f0 con 0x55ff410b5040 2024-05-08T18:05:05.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.291+0000 7f1b433ed640 1 -- 172.21.15.50:0/2288427555 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94554796296736] v27'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b2c07f070 con 0x55ff410b5040 2024-05-08T18:05:05.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.291+0000 7f1b47f33800 20 remove_watcher() i=2 2024-05-08T18:05:05.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.291+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94554796310144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410bfbc0 con 0x7f1b2c074040 2024-05-08T18:05:05.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.292+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94554796310144] v27'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x7f1b2c074040 2024-05-08T18:05:05.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.292+0000 7f1b47f33800 20 remove_watcher() i=3 2024-05-08T18:05:05.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.292+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94554796313552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410b8a20 con 0x55ff410abca0 2024-05-08T18:05:05.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.293+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94554796313552] v27'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3c040040 con 0x55ff410abca0 2024-05-08T18:05:05.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.293+0000 7f1b47f33800 20 remove_watcher() i=4 2024-05-08T18:05:05.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.293+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94554796315872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410ba4f0 con 0x55ff410b0690 2024-05-08T18:05:05.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.294+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94554796315872] v27'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.294+0000 7f1b47f33800 20 remove_watcher() i=5 2024-05-08T18:05:05.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.294+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94554796317440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410bbe80 con 0x55ff410b0690 2024-05-08T18:05:05.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.295+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94554796317440] v27'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.295+0000 7f1b47f33800 20 remove_watcher() i=6 2024-05-08T18:05:05.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.295+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94554796324400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ff410bfbc0 con 0x55ff410b0690 2024-05-08T18:05:05.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b43bee640 1 -- 172.21.15.50:0/2288427555 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94554796324400] v27'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3406e020 con 0x55ff410b0690 2024-05-08T18:05:05.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b47f33800 20 remove_watcher() i=7 2024-05-08T18:05:05.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ff410b0690 msgr2=0x55ff410b2a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ff410b0690 0x55ff410b2a80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ff410abca0 msgr2=0x55ff410ae160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ff410abca0 0x55ff410ae160 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f1b2c074040 msgr2=0x7f1b2c081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f1b2c074040 0x7f1b2c081410 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ff410b5040 msgr2=0x55ff410b7500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ff410b5040 0x55ff410b7500 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1a0c03bba0 msgr2=0x7f1a0c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1a0c03bba0 0x7f1a0c03e060 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c000c00 tx=0x7f1b2c04f000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff410991f0 msgr2=0x55ff410a05d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2288427555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff410991f0 0x55ff410a05d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1b3403de80 tx=0x7f1b34016000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 reap_dead start 2024-05-08T18:05:05.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b443ef640 1 -- 172.21.15.50:0/2288427555 reap_dead start 2024-05-08T18:05:05.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 shutdown_connections 2024-05-08T18:05:05.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.296+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 >> 172.21.15.50:0/2288427555 conn(0x55ff41095550 msgr2=0x55ff41092bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 shutdown_connections 2024-05-08T18:05:05.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 -- 172.21.15.50:0/2288427555 wait complete. 2024-05-08T18:05:05.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 -- 172.21.15.50:0/3837651347 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1a1c03bb50 msgr2=0x7f1a1c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 --2- 172.21.15.50:0/3837651347 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1a1c03bb50 0x7f1a1c03e010 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x55ff41075780 tx=0x7f1b2c005ac0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 -- 172.21.15.50:0/3837651347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff4108bf70 msgr2=0x55ff4108c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 --2- 172.21.15.50:0/3837651347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff4108bf70 0x55ff4108c390 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400ecf0 tx=0x7f1b3401ab20 comp rx=0 tx=0).stop 2024-05-08T18:05:05.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 -- 172.21.15.50:0/3837651347 shutdown_connections 2024-05-08T18:05:05.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 --2- 172.21.15.50:0/3837651347 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1a1c03bb50 0x7f1a1c03e010 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 --2- 172.21.15.50:0/3837651347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff4108bf70 0x55ff4108c390 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 -- 172.21.15.50:0/3837651347 >> 172.21.15.50:0/3837651347 conn(0x55ff41080010 msgr2=0x55ff41080410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 -- 172.21.15.50:0/3837651347 shutdown_connections 2024-05-08T18:05:05.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 -- 172.21.15.50:0/3837651347 wait complete. 2024-05-08T18:05:05.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 -- 172.21.15.50:0/2275072066 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ff410771d0 msgr2=0x55ff410795c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ff410771d0 0x55ff410795c0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 -- 172.21.15.50:0/2275072066 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ff40f76cb0 msgr2=0x55ff41074980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ff40f76cb0 0x55ff41074980 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 -- 172.21.15.50:0/2275072066 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ff4107b110 msgr2=0x55ff4107d500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ff4107b110 0x55ff4107d500 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 -- 172.21.15.50:0/2275072066 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1a2c03bba0 msgr2=0x7f1a2c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f1a2c03bba0 0x7f1a2c03e060 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c054000 tx=0x7f1b2c04e000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 -- 172.21.15.50:0/2275072066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f731a0 msgr2=0x55ff4106d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 --2- 172.21.15.50:0/2275072066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ff40f731a0 0x55ff4106d740 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55ff40f72110 tx=0x7f1b34000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b443ef640 1 -- 172.21.15.50:0/2275072066 reap_dead start 2024-05-08T18:05:05.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 -- 172.21.15.50:0/2275072066 shutdown_connections 2024-05-08T18:05:05.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.297+0000 7f1b47f33800 1 -- 172.21.15.50:0/2275072066 >> 172.21.15.50:0/2275072066 conn(0x55ff40f6cb70 msgr2=0x55ff40f71980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.298+0000 7f1b47f33800 1 -- 172.21.15.50:0/2275072066 shutdown_connections 2024-05-08T18:05:05.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.298+0000 7f1b47f33800 1 -- 172.21.15.50:0/2275072066 wait complete. 2024-05-08T18:05:05.307 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'vDzZuBVsJ1sSPV9wc9BFdG5KDBUSvDNNEWQFwucZ', 'active': True, 'create_date': '2024-05-08T18:05:05.283685Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:05.307 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2024-05-08T18:05:05.307 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2024-05-08T18:05:05.308 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser modify --subuser foo:foo1 --secret gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL --key-type swift 2024-05-08T18:05:05.357 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:05.358 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:05.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.360+0000 7f789af31800 1 Processor -- start 2024-05-08T18:05:05.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.360+0000 7f789af31800 1 -- start start 2024-05-08T18:05:05.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.360+0000 7f789af31800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56039d1bd710 0x56039d1bdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.361+0000 7f789af31800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56039d1be1b0 con 0x56039d1be8d0 2024-05-08T18:05:05.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.361+0000 7f789af31800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x56039d05de20 con 0x56039d1bd710 2024-05-08T18:05:05.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.361+0000 7f7896c28640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56039d1bd710 0x56039d1bdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.361+0000 7f7896427640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x56039d1be8d0 0x56039d1bed50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43008/0 (socket says 172.21.15.50:43008) 2024-05-08T18:05:05.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.361+0000 7f7896c28640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56039d1bd710 0x56039d1bdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36760/0 (socket says 172.21.15.50:36760) 2024-05-08T18:05:05.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.361+0000 7f7896427640 1 -- 172.21.15.50:0/3577664674 learned_addr learned my addr 172.21.15.50:0/3577664674 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:05.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.361+0000 7f7896c28640 1 -- 172.21.15.50:0/3577664674 >> v1:172.21.15.46:6789/0 conn(0x56039d1be8d0 legacy=0x56039d1bed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.361+0000 7f7896c28640 1 -- 172.21.15.50:0/3577664674 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039d1bf490 con 0x56039d1bd710 2024-05-08T18:05:05.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.361+0000 7f7895c26640 1 -- 172.21.15.50:0/3577664674 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 860495614 0 0) 0x56039d1be1b0 con 0x56039d1be8d0 2024-05-08T18:05:05.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.363+0000 7f7896c28640 1 --2- 172.21.15.50:0/3577664674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 0x56039d1bdb30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f788800a470 tx=0x7f788801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea919869cf17cdbd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.363+0000 7f7895c26640 1 -- 172.21.15.50:0/3577664674 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7888057070 con 0x56039d1bd710 2024-05-08T18:05:05.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.363+0000 7f7895c26640 1 -- 172.21.15.50:0/3577664674 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f788804adc0 con 0x56039d1bd710 2024-05-08T18:05:05.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.363+0000 7f7895c26640 1 -- 172.21.15.50:0/3577664674 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7888052590 con 0x56039d1bd710 2024-05-08T18:05:05.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.364+0000 7f789af31800 1 -- 172.21.15.50:0/3577664674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 msgr2=0x56039d1bdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.364+0000 7f789af31800 1 --2- 172.21.15.50:0/3577664674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 0x56039d1bdb30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f788800a470 tx=0x7f788801ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:05.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.364+0000 7f789af31800 1 -- 172.21.15.50:0/3577664674 shutdown_connections 2024-05-08T18:05:05.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.364+0000 7f789af31800 1 --2- 172.21.15.50:0/3577664674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 0x56039d1bdb30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.364+0000 7f789af31800 1 -- 172.21.15.50:0/3577664674 >> 172.21.15.50:0/3577664674 conn(0x56039d1b8c90 msgr2=0x56039d1bb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.364+0000 7f789af31800 1 -- 172.21.15.50:0/3577664674 shutdown_connections 2024-05-08T18:05:05.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.364+0000 7f789af31800 1 -- 172.21.15.50:0/3577664674 wait complete. 2024-05-08T18:05:05.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.379+0000 7f789af31800 1 Processor -- start 2024-05-08T18:05:05.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.379+0000 7f789af31800 1 -- start start 2024-05-08T18:05:05.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.379+0000 7f789af31800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 0x56039d2aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.379+0000 7f789af31800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56039d108a30 con 0x56039d1bd710 2024-05-08T18:05:05.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.380+0000 7f7896c28640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 0x56039d2aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.380+0000 7f7896c28640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 0x56039d2aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36772/0 (socket says 172.21.15.50:36772) 2024-05-08T18:05:05.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.380+0000 7f7896c28640 1 -- 172.21.15.50:0/793449439 learned_addr learned my addr 172.21.15.50:0/793449439 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.380+0000 7f7896c28640 1 -- 172.21.15.50:0/793449439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039d2af0e0 con 0x56039d1bd710 2024-05-08T18:05:05.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.380+0000 7f7896c28640 1 --2- 172.21.15.50:0/793449439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 0x56039d2aeb10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7888055040 tx=0x7f788804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678405cefb244d43 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.380+0000 7f7784e09640 1 -- 172.21.15.50:0/793449439 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7888057040 con 0x56039d1bd710 2024-05-08T18:05:05.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.380+0000 7f7784e09640 1 -- 172.21.15.50:0/793449439 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7888062070 con 0x56039d1bd710 2024-05-08T18:05:05.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.380+0000 7f7784e09640 1 -- 172.21.15.50:0/793449439 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f788805d470 con 0x56039d1bd710 2024-05-08T18:05:05.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.381+0000 7f789af31800 1 -- 172.21.15.50:0/793449439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 msgr2=0x56039d2aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.381+0000 7f789af31800 1 --2- 172.21.15.50:0/793449439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 0x56039d2aeb10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7888055040 tx=0x7f788804c000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.381+0000 7f789af31800 1 -- 172.21.15.50:0/793449439 shutdown_connections 2024-05-08T18:05:05.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.381+0000 7f789af31800 1 --2- 172.21.15.50:0/793449439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 0x56039d2aeb10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.381+0000 7f789af31800 1 -- 172.21.15.50:0/793449439 >> 172.21.15.50:0/793449439 conn(0x56039d1b8c90 msgr2=0x56039d2aa430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.381+0000 7f789af31800 1 -- 172.21.15.50:0/793449439 shutdown_connections 2024-05-08T18:05:05.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.381+0000 7f789af31800 1 -- 172.21.15.50:0/793449439 wait complete. 2024-05-08T18:05:05.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.381+0000 7f789af31800 1 Processor -- start 2024-05-08T18:05:05.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.381+0000 7f789af31800 1 -- start start 2024-05-08T18:05:05.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.382+0000 7f789af31800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 0x56039d2b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.382+0000 7f789af31800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56039d1c6da0 con 0x56039d1bd710 2024-05-08T18:05:05.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.382+0000 7f7896c28640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 0x56039d2b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.382+0000 7f7896c28640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 0x56039d2b65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36786/0 (socket says 172.21.15.50:36786) 2024-05-08T18:05:05.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.382+0000 7f7896c28640 1 -- 172.21.15.50:0/1480485585 learned_addr learned my addr 172.21.15.50:0/1480485585 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.382+0000 7f7896c28640 1 -- 172.21.15.50:0/1480485585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039d2b6b00 con 0x56039d1bd710 2024-05-08T18:05:05.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.382+0000 7f7896c28640 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 0x56039d2b65c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f788801ce60 tx=0x7f7888002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.383+0000 7f7783606640 1 -- 172.21.15.50:0/1480485585 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f788806b070 con 0x56039d1bd710 2024-05-08T18:05:05.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.383+0000 7f7783606640 1 -- 172.21.15.50:0/1480485585 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f788805d970 con 0x56039d1bd710 2024-05-08T18:05:05.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.383+0000 7f789af31800 1 -- 172.21.15.50:0/1480485585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56039d2b6d90 con 0x56039d1bd710 2024-05-08T18:05:05.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.383+0000 7f7783606640 1 -- 172.21.15.50:0/1480485585 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7888057070 con 0x56039d1bd710 2024-05-08T18:05:05.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.383+0000 7f789af31800 1 -- 172.21.15.50:0/1480485585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56039d2b99f0 con 0x56039d1bd710 2024-05-08T18:05:05.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.384+0000 7f7783606640 1 -- 172.21.15.50:0/1480485585 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f788805dc90 con 0x56039d1bd710 2024-05-08T18:05:05.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.384+0000 7f7783606640 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f777c03bba0 0x7f777c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.384+0000 7f7783606640 1 -- 172.21.15.50:0/1480485585 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7888094510 con 0x56039d1bd710 2024-05-08T18:05:05.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.384+0000 7f7896427640 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f777c03bba0 0x7f777c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.384+0000 7f789af31800 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56039d1be8d0 0x56039d2bd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.384+0000 7f789af31800 1 -- 172.21.15.50:0/1480485585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d2bdcf0 con 0x56039d1be8d0 2024-05-08T18:05:05.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.384+0000 7f7897429640 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56039d1be8d0 0x56039d2bd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.385+0000 7f7896427640 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f777c03bba0 0x7f777c03e060 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7880000c00 tx=0x7f78800013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.385+0000 7f7897429640 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56039d1be8d0 0x56039d2bd7b0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.385+0000 7f7897429640 1 -- 172.21.15.50:0/1480485585 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d1be8d0 2024-05-08T18:05:05.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.385+0000 7f789af31800 1 -- 172.21.15.50:0/1480485585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d2bad60 con 0x56039d1be8d0 2024-05-08T18:05:05.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.385+0000 7f7897429640 1 -- 172.21.15.50:0/1480485585 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d1be8d0 2024-05-08T18:05:05.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.385+0000 7f789af31800 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56039d2bffe0 0x56039d2c23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.385+0000 7f789af31800 1 -- 172.21.15.50:0/1480485585 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d2c2ac0 con 0x56039d2bffe0 2024-05-08T18:05:05.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.386+0000 7f7896c28640 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56039d2bffe0 0x56039d2c23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.386+0000 7f7896c28640 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56039d2bffe0 0x56039d2c23d0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.386+0000 7f7896c28640 1 -- 172.21.15.50:0/1480485585 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f788805a040 con 0x56039d2bffe0 2024-05-08T18:05:05.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.386+0000 7f789af31800 1 -- 172.21.15.50:0/1480485585 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d2bee50 con 0x56039d1be8d0 2024-05-08T18:05:05.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.387+0000 7f7897429640 1 -- 172.21.15.50:0/1480485585 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d1be8d0 2024-05-08T18:05:05.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.387+0000 7f789af31800 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56039d2c3f20 0x56039d2c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.400 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.387+0000 7f789af31800 1 -- 172.21.15.50:0/1480485585 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d2c6a00 con 0x56039d2c3f20 2024-05-08T18:05:05.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.387+0000 7f7896427640 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56039d2c3f20 0x56039d2c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.387+0000 7f7896427640 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56039d2c3f20 0x56039d2c6310 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.387+0000 7f7896427640 1 -- 172.21.15.50:0/1480485585 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f788000f670 con 0x56039d2c3f20 2024-05-08T18:05:05.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.388+0000 7f789af31800 1 Processor -- start 2024-05-08T18:05:05.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.388+0000 7f789af31800 1 -- start start 2024-05-08T18:05:05.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.388+0000 7f789af31800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d2d4d30 0x56039d2d5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.388+0000 7f789af31800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56039d0af330 con 0x56039d2d4d30 2024-05-08T18:05:05.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.388+0000 7f7896c28640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d2d4d30 0x56039d2d5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.388+0000 7f7896c28640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d2d4d30 0x56039d2d5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36794/0 (socket says 172.21.15.50:36794) 2024-05-08T18:05:05.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.388+0000 7f7896c28640 1 -- 172.21.15.50:0/2463955542 learned_addr learned my addr 172.21.15.50:0/2463955542 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.388+0000 7f7896c28640 1 -- 172.21.15.50:0/2463955542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039d2d5690 con 0x56039d2d4d30 2024-05-08T18:05:05.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.388+0000 7f7896c28640 1 --2- 172.21.15.50:0/2463955542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d2d4d30 0x56039d2d5150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7888092960 tx=0x7f78880096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.389+0000 7f7780e01640 1 -- 172.21.15.50:0/2463955542 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f788808b070 con 0x56039d2d4d30 2024-05-08T18:05:05.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.389+0000 7f7780e01640 1 -- 172.21.15.50:0/2463955542 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7888092b70 con 0x56039d2d4d30 2024-05-08T18:05:05.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.389+0000 7f789af31800 1 -- 172.21.15.50:0/2463955542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56039d2d6100 con 0x56039d2d4d30 2024-05-08T18:05:05.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.389+0000 7f7780e01640 1 -- 172.21.15.50:0/2463955542 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7888092e50 con 0x56039d2d4d30 2024-05-08T18:05:05.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.389+0000 7f789af31800 1 -- 172.21.15.50:0/2463955542 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56039d2d6600 con 0x56039d2d4d30 2024-05-08T18:05:05.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.389+0000 7f7780e01640 1 -- 172.21.15.50:0/2463955542 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f788808e070 con 0x56039d2d4d30 2024-05-08T18:05:05.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.390+0000 7f7780e01640 1 --2- 172.21.15.50:0/2463955542 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f777003bd40 0x7f777003e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.390+0000 7f7780e01640 1 -- 172.21.15.50:0/2463955542 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7888082040 con 0x56039d2d4d30 2024-05-08T18:05:05.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.390+0000 7f7896427640 1 --2- 172.21.15.50:0/2463955542 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f777003bd40 0x7f777003e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.390+0000 7f789af31800 1 Processor -- start 2024-05-08T18:05:05.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.390+0000 7f789af31800 1 -- start start 2024-05-08T18:05:05.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.390+0000 7f7896427640 1 --2- 172.21.15.50:0/2463955542 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f777003bd40 0x7f777003e200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x56039d2be590 tx=0x7f7880006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.390+0000 7f789af31800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d132bb0 0x56039d132fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.390+0000 7f789af31800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56039d2af3a0 con 0x56039d132bb0 2024-05-08T18:05:05.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.390+0000 7f7896c28640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d132bb0 0x56039d132fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.390+0000 7f7896c28640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d132bb0 0x56039d132fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36800/0 (socket says 172.21.15.50:36800) 2024-05-08T18:05:05.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.390+0000 7f7896c28640 1 -- 172.21.15.50:0/1403414486 learned_addr learned my addr 172.21.15.50:0/1403414486 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.391+0000 7f7896c28640 1 -- 172.21.15.50:0/1403414486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039d1313b0 con 0x56039d132bb0 2024-05-08T18:05:05.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.391+0000 7f7896c28640 1 --2- 172.21.15.50:0/1403414486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d132bb0 0x56039d132fd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7888070b40 tx=0x7f7888086590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517d84d6494383b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.391+0000 7f777a7fc640 1 -- 172.21.15.50:0/1403414486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7888079750 con 0x56039d132bb0 2024-05-08T18:05:05.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.391+0000 7f777a7fc640 1 -- 172.21.15.50:0/1403414486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78880a3070 con 0x56039d132bb0 2024-05-08T18:05:05.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.391+0000 7f777a7fc640 1 -- 172.21.15.50:0/1403414486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78880793f0 con 0x56039d132bb0 2024-05-08T18:05:05.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.391+0000 7f789af31800 1 -- 172.21.15.50:0/1403414486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d132bb0 msgr2=0x56039d132fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.391+0000 7f789af31800 1 --2- 172.21.15.50:0/1403414486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d132bb0 0x56039d132fd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7888070b40 tx=0x7f7888086590 comp rx=0 tx=0).stop 2024-05-08T18:05:05.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.391+0000 7f789af31800 1 -- 172.21.15.50:0/1403414486 shutdown_connections 2024-05-08T18:05:05.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.391+0000 7f789af31800 1 --2- 172.21.15.50:0/1403414486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d132bb0 0x56039d132fd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.391+0000 7f789af31800 1 -- 172.21.15.50:0/1403414486 >> 172.21.15.50:0/1403414486 conn(0x56039d2de250 msgr2=0x56039d2dc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.391+0000 7f789af31800 1 -- 172.21.15.50:0/1403414486 shutdown_connections 2024-05-08T18:05:05.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.391+0000 7f789af31800 1 -- 172.21.15.50:0/1403414486 wait complete. 2024-05-08T18:05:05.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.392+0000 7f789af31800 1 Processor -- start 2024-05-08T18:05:05.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.392+0000 7f789af31800 1 -- start start 2024-05-08T18:05:05.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.392+0000 7f789af31800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d132bb0 0x56039d2eb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.392+0000 7f789af31800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56039d1317c0 con 0x56039d132bb0 2024-05-08T18:05:05.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.392+0000 7f7896c28640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d132bb0 0x56039d2eb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.392+0000 7f7896c28640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d132bb0 0x56039d2eb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36812/0 (socket says 172.21.15.50:36812) 2024-05-08T18:05:05.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.392+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 learned_addr learned my addr 172.21.15.50:0/2815681201 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.392+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039d2eba20 con 0x56039d132bb0 2024-05-08T18:05:05.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.393+0000 7f7896c28640 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d132bb0 0x56039d2eb4e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7888067fa0 tx=0x7f788807fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.393+0000 7f7778ff9640 1 -- 172.21.15.50:0/2815681201 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78880b8070 con 0x56039d132bb0 2024-05-08T18:05:05.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.393+0000 7f7778ff9640 1 -- 172.21.15.50:0/2815681201 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7888087560 con 0x56039d132bb0 2024-05-08T18:05:05.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.393+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56039d2eba20 con 0x56039d132bb0 2024-05-08T18:05:05.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.393+0000 7f7778ff9640 1 -- 172.21.15.50:0/2815681201 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7888079520 con 0x56039d132bb0 2024-05-08T18:05:05.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.393+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56039d2ebec0 con 0x56039d132bb0 2024-05-08T18:05:05.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.393+0000 7f7778ff9640 1 -- 172.21.15.50:0/2815681201 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f78880bb070 con 0x56039d132bb0 2024-05-08T18:05:05.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.394+0000 7f7778ff9640 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f776003bb50 0x7f776003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.394+0000 7f7778ff9640 1 -- 172.21.15.50:0/2815681201 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f78880af030 con 0x56039d132bb0 2024-05-08T18:05:05.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.394+0000 7f7896427640 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f776003bb50 0x7f776003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.394+0000 7f7896427640 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f776003bb50 0x7f776003e010 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f788000ffd0 tx=0x7f7880023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.394+0000 7f789af31800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.394+0000 7f789af31800 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56039d2f6ab0 0x56039d2f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.394+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d2f96c0 con 0x56039d2f6ab0 2024-05-08T18:05:05.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.395+0000 7f7897429640 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56039d2f6ab0 0x56039d2f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.395+0000 7f7897429640 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56039d2f6ab0 0x56039d2f8f70 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.395+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d2f6ab0 2024-05-08T18:05:05.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.395+0000 7f789af31800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.395+0000 7f789af31800 20 realm 2024-05-08T18:05:05.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.395+0000 7f789af31800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.395+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d2f9ad0 con 0x56039d2f6ab0 2024-05-08T18:05:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.395+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d2f6ab0 2024-05-08T18:05:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.395+0000 7f789af31800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.395+0000 7f789af31800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.395+0000 7f789af31800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.395+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa3a0 con 0x56039d2f6ab0 2024-05-08T18:05:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.396+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d2f6ab0 2024-05-08T18:05:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.396+0000 7f789af31800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.396+0000 7f789af31800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.396+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2f6ab0 2024-05-08T18:05:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.396+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d2f6ab0 2024-05-08T18:05:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.396+0000 7f789af31800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:05.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.396+0000 7f789af31800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.396+0000 7f789af31800 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56039d2fb4a0 0x56039d2fd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.396+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fdea0 con 0x56039d2fb4a0 2024-05-08T18:05:05.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.396+0000 7f7896c28640 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56039d2fb4a0 0x56039d2fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.396+0000 7f7896c28640 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56039d2fb4a0 0x56039d2fd890 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.397+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.397+0000 7f789af31800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:05.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.397+0000 7f789af31800 20 searching for the correct realm 2024-05-08T18:05:05.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.397+0000 7f789af31800 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56039d2ffe30 0x56039d3022f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.397+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d3029c0 con 0x56039d2ffe30 2024-05-08T18:05:05.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.397+0000 7f7896427640 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56039d2ffe30 0x56039d3022f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.397+0000 7f7896427640 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56039d2ffe30 0x56039d3022f0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.397+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7880037040 con 0x56039d2ffe30 2024-05-08T18:05:05.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.397+0000 7f7896427640 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f788002e910 0x7f788003a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.397+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d3029c0 con 0x7f788002e910 2024-05-08T18:05:05.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.397+0000 7f7897429640 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f788002e910 0x7f788003a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.398+0000 7f7897429640 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f788002e910 0x7f788003a3c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.398+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f789000a4a0 con 0x7f788002e910 2024-05-08T18:05:05.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.398+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2fb4a0 2024-05-08T18:05:05.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.398+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.398+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2f6ab0 2024-05-08T18:05:05.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.398+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d2f6ab0 2024-05-08T18:05:05.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.399+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2f6ab0 2024-05-08T18:05:05.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.399+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d2f6ab0 2024-05-08T18:05:05.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.399+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2fb4a0 2024-05-08T18:05:05.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.399+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.399+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2fb4a0 2024-05-08T18:05:05.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.400+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.400+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x7f788002e910 2024-05-08T18:05:05.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.400+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f789000a4a0 con 0x7f788002e910 2024-05-08T18:05:05.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.400+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2ffe30 2024-05-08T18:05:05.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.400+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7880037040 con 0x56039d2ffe30 2024-05-08T18:05:05.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.400+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2f6ab0 2024-05-08T18:05:05.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.400+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d2f6ab0 2024-05-08T18:05:05.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.400+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2fb4a0 2024-05-08T18:05:05.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.401+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.401+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2f6ab0 2024-05-08T18:05:05.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.401+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d2f6ab0 2024-05-08T18:05:05.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.401+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x7f788002e910 2024-05-08T18:05:05.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.401+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f789000a4a0 con 0x7f788002e910 2024-05-08T18:05:05.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.401+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2ffe30 2024-05-08T18:05:05.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.401+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7880037040 con 0x56039d2ffe30 2024-05-08T18:05:05.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.401+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2ffe30 2024-05-08T18:05:05.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.401+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7880037040 con 0x56039d2ffe30 2024-05-08T18:05:05.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.401+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2fb4a0 2024-05-08T18:05:05.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.402+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.402+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2ffe30 2024-05-08T18:05:05.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.402+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7880037040 con 0x56039d2ffe30 2024-05-08T18:05:05.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.402+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2ffe30 2024-05-08T18:05:05.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.402+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7880037040 con 0x56039d2ffe30 2024-05-08T18:05:05.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.402+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2f6ab0 2024-05-08T18:05:05.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.402+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d2f6ab0 2024-05-08T18:05:05.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.402+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2f6ab0 2024-05-08T18:05:05.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.403+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d2f6ab0 2024-05-08T18:05:05.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.403+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2fb4a0 2024-05-08T18:05:05.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.403+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.403+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2ffe30 2024-05-08T18:05:05.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.403+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7880037040 con 0x56039d2ffe30 2024-05-08T18:05:05.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.403+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x7f788002e910 2024-05-08T18:05:05.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.404+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f789000a4a0 con 0x7f788002e910 2024-05-08T18:05:05.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.404+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2f6ab0 2024-05-08T18:05:05.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.404+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d2f6ab0 2024-05-08T18:05:05.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.404+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2fb4a0 2024-05-08T18:05:05.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.404+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.404+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2ffe30 2024-05-08T18:05:05.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.404+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7880037040 con 0x56039d2ffe30 2024-05-08T18:05:05.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.404+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2fb4a0 2024-05-08T18:05:05.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.405+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.405+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x7f788002e910 2024-05-08T18:05:05.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.405+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f789000a4a0 con 0x7f788002e910 2024-05-08T18:05:05.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.405+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2ffe30 2024-05-08T18:05:05.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.405+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7880037040 con 0x56039d2ffe30 2024-05-08T18:05:05.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.405+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2fb4a0 2024-05-08T18:05:05.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.405+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.406+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x56039d2f6ab0 2024-05-08T18:05:05.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.406+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d2f6ab0 2024-05-08T18:05:05.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.406+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fa740 con 0x7f788002e910 2024-05-08T18:05:05.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.406+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f789000a4a0 con 0x7f788002e910 2024-05-08T18:05:05.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.406+0000 7f789af31800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:05.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.406+0000 7f789af31800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:05.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.406+0000 7f789af31800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:05.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.406+0000 7f789af31800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:05.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.406+0000 7f789af31800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:05.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.406+0000 7f789af31800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:05.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.406+0000 7f789af31800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.406+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fe760 con 0x56039d2f6ab0 2024-05-08T18:05:05.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.406+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d2f6ab0 2024-05-08T18:05:05.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.406+0000 7f789af31800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.406+0000 7f789af31800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.406+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fe760 con 0x56039d2f6ab0 2024-05-08T18:05:05.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f789000a4a0 con 0x56039d2f6ab0 2024-05-08T18:05:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f789af31800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f789af31800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fe760 con 0x56039d2ffe30 2024-05-08T18:05:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7880037040 con 0x56039d2ffe30 2024-05-08T18:05:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f789af31800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f789af31800 20 zone default found 2024-05-08T18:05:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f789af31800 4 Realm: () 2024-05-08T18:05:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f789af31800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f789af31800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f789af31800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f789af31800 20 zonegroup default 2024-05-08T18:05:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f789af31800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d2ff110 con 0x56039d2fb4a0 2024-05-08T18:05:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f789af31800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f789af31800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.407+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d2fec70 con 0x56039d2ffe30 2024-05-08T18:05:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.408+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7880037040 con 0x56039d2ffe30 2024-05-08T18:05:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.408+0000 7f789af31800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.408+0000 7f789af31800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.408+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d304d70 con 0x56039d2ffe30 2024-05-08T18:05:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.408+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7880037040 con 0x56039d2ffe30 2024-05-08T18:05:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.408+0000 7f789af31800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.408+0000 7f789af31800 20 started sync module instance, tier type = 2024-05-08T18:05:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.408+0000 7f789af31800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.408+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d304160 con 0x7f788002e910 2024-05-08T18:05:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.409+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f789000a4a0 con 0x7f788002e910 2024-05-08T18:05:05.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.409+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94573522077280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d304160 con 0x7f788002e910 2024-05-08T18:05:05.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.409+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d3058b0 con 0x56039d2f6ab0 2024-05-08T18:05:05.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.410+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94573522077280] v27'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7890040040 con 0x7f788002e910 2024-05-08T18:05:05.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.410+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7890040040 con 0x56039d2f6ab0 2024-05-08T18:05:05.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.410+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94573522080672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d304160 con 0x56039d2f6ab0 2024-05-08T18:05:05.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.410+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d306370 con 0x56039d2ffe30 2024-05-08T18:05:05.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.411+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94573522080672] v27'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7890040040 con 0x56039d2f6ab0 2024-05-08T18:05:05.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.412+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7880037040 con 0x56039d2ffe30 2024-05-08T18:05:05.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.412+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94573522074800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d3040d0 con 0x56039d2ffe30 2024-05-08T18:05:05.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.412+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d307230 con 0x7f788002e910 2024-05-08T18:05:05.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.412+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94573522074800] v27'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7880037040 con 0x56039d2ffe30 2024-05-08T18:05:05.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.413+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7890040040 con 0x7f788002e910 2024-05-08T18:05:05.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.413+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94573522087296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d3071a0 con 0x7f788002e910 2024-05-08T18:05:05.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.413+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d307fa0 con 0x56039d2f6ab0 2024-05-08T18:05:05.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.414+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7890040040 con 0x56039d2f6ab0 2024-05-08T18:05:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.414+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94573522090704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d307fa0 con 0x56039d2f6ab0 2024-05-08T18:05:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.414+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d308f00 con 0x56039d2fb4a0 2024-05-08T18:05:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.414+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94573522087296] v27'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7890040040 con 0x7f788002e910 2024-05-08T18:05:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.415+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94573522090704] v27'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7890040040 con 0x56039d2f6ab0 2024-05-08T18:05:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.415+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.415+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94573522089504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d308a10 con 0x56039d2fb4a0 2024-05-08T18:05:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.415+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d3098c0 con 0x56039d2fb4a0 2024-05-08T18:05:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.415+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94573522089504] v27'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.416+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.416+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94573522092944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d3098c0 con 0x56039d2fb4a0 2024-05-08T18:05:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.416+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d30a110 con 0x56039d2fb4a0 2024-05-08T18:05:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.417+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94573522092944] v27'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.417+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.417+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94573522097680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d30a060 con 0x56039d2fb4a0 2024-05-08T18:05:05.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.417+0000 7f789af31800 20 add_watcher() i=0 2024-05-08T18:05:05.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.417+0000 7f789af31800 20 add_watcher() i=1 2024-05-08T18:05:05.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.417+0000 7f789af31800 20 add_watcher() i=2 2024-05-08T18:05:05.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.417+0000 7f789af31800 20 add_watcher() i=3 2024-05-08T18:05:05.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.417+0000 7f789af31800 20 add_watcher() i=4 2024-05-08T18:05:05.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.417+0000 7f789af31800 20 add_watcher() i=5 2024-05-08T18:05:05.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.417+0000 7f789af31800 20 add_watcher() i=6 2024-05-08T18:05:05.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.418+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94573522097680] v27'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.418+0000 7f789af31800 20 add_watcher() i=7 2024-05-08T18:05:05.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.418+0000 7f789af31800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:05.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.418+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d30d5d0 con 0x56039d2f6ab0 2024-05-08T18:05:05.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.418+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7890040040 con 0x56039d2f6ab0 2024-05-08T18:05:05.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.418+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94573522098928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d30aaf0 con 0x56039d2f6ab0 2024-05-08T18:05:05.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.419+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94573522098928] v27'72 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7890040040 con 0x56039d2f6ab0 2024-05-08T18:05:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.419+0000 7f774d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.419+0000 7f789af31800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.419+0000 7f789af31800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.419+0000 7f789af31800 5 note: GC not initialized 2024-05-08T18:05:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.419+0000 7f76c7fff640 20 reqs_thread_entry: start 2024-05-08T18:05:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.419+0000 7f789af31800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.420+0000 7f789af31800 20 Filter name: none 2024-05-08T18:05:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.420+0000 7f76c5ffb640 20 reqs_thread_entry: start 2024-05-08T18:05:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.420+0000 7f789af31800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.420+0000 7f789af31800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.420+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d340a50 con 0x56039d2f6ab0 2024-05-08T18:05:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.420+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv6 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7f7890040040 con 0x56039d2f6ab0 2024-05-08T18:05:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.420+0000 7f789af31800 20 rados_obj.operate() r=0 bl.length=556 2024-05-08T18:05:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.420+0000 7f789af31800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:05.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.420+0000 7f789af31800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:05.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.420+0000 7f789af31800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:05.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.420+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~556 in=556b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d347270 con 0x56039d2f6ab0 2024-05-08T18:05:05.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.421+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~556] v27'7 uv7 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f7890040040 con 0x56039d2f6ab0 2024-05-08T18:05:05.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.421+0000 7f789af31800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:05.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.421+0000 7f789af31800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:05.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.421+0000 7f789af31800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-05-08T18:05:05.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.421+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94573522349104 in=744b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56039d3428f0 con 0x56039d2f6ab0 2024-05-08T18:05:05.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.422+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== watch-notify(notify (1) cookie 94573522090704 notify 115964116998 ret 0) ==== 774+0+0 (crc 0 0 0) 0x7f7890018a30 con 0x56039d2f6ab0 2024-05-08T18:05:05.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.422+0000 7f777a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116998 cookie 94573522090704 notifier 4361 bl.length()=732 2024-05-08T18:05:05.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.422+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(64 notify.4 [notify cookie 94573522349104 out=8b] v0'0 uv57 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7890040040 con 0x56039d2f6ab0 2024-05-08T18:05:05.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.422+0000 7f777a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:05.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.422+0000 7f777a7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:05.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.422+0000 7f777a7fc640 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f775c003ea0 con 0x56039d2f6ab0 2024-05-08T18:05:05.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.422+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7890040040 con 0x56039d2f6ab0 2024-05-08T18:05:05.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.422+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== watch-notify(notify_complete (2) cookie 94573522349104 notify 115964116998 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f7890018c00 con 0x56039d2f6ab0 2024-05-08T18:05:05.431 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:05.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.423+0000 7f774d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:05.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.423+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94573522098928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d342d20 con 0x56039d2f6ab0 2024-05-08T18:05:05.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.424+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94573522098928] v27'73 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7890040040 con 0x56039d2f6ab0 2024-05-08T18:05:05.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.424+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94573522077280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d342d20 con 0x7f788002e910 2024-05-08T18:05:05.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.425+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94573522077280] v27'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7890040040 con 0x7f788002e910 2024-05-08T18:05:05.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.425+0000 7f789af31800 20 remove_watcher() i=0 2024-05-08T18:05:05.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.425+0000 7f789af31800 2 removed watcher, disabling cache 2024-05-08T18:05:05.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.425+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94573522080672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d304660 con 0x56039d2f6ab0 2024-05-08T18:05:05.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.426+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94573522080672] v27'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7890040040 con 0x56039d2f6ab0 2024-05-08T18:05:05.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.426+0000 7f789af31800 20 remove_watcher() i=1 2024-05-08T18:05:05.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.426+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94573522074800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d3053a0 con 0x56039d2ffe30 2024-05-08T18:05:05.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.427+0000 7f7896427640 1 -- 172.21.15.50:0/2815681201 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94573522074800] v27'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7880037040 con 0x56039d2ffe30 2024-05-08T18:05:05.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.427+0000 7f789af31800 20 remove_watcher() i=2 2024-05-08T18:05:05.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.427+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94573522087296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d303cb0 con 0x7f788002e910 2024-05-08T18:05:05.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.428+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94573522087296] v27'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7890040040 con 0x7f788002e910 2024-05-08T18:05:05.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.428+0000 7f789af31800 20 remove_watcher() i=3 2024-05-08T18:05:05.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.428+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94573522090704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d303cb0 con 0x56039d2f6ab0 2024-05-08T18:05:05.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.429+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94573522090704] v27'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7890040040 con 0x56039d2f6ab0 2024-05-08T18:05:05.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.429+0000 7f789af31800 20 remove_watcher() i=4 2024-05-08T18:05:05.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.429+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94573522089504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d307ad0 con 0x56039d2fb4a0 2024-05-08T18:05:05.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.430+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94573522089504] v27'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.430+0000 7f789af31800 20 remove_watcher() i=5 2024-05-08T18:05:05.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.430+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94573522092944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d303cb0 con 0x56039d2fb4a0 2024-05-08T18:05:05.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.431+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94573522092944] v27'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.431+0000 7f789af31800 20 remove_watcher() i=6 2024-05-08T18:05:05.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.431+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94573522097680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56039d307620 con 0x56039d2fb4a0 2024-05-08T18:05:05.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f7896c28640 1 -- 172.21.15.50:0/2815681201 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94573522097680] v27'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f78880ba030 con 0x56039d2fb4a0 2024-05-08T18:05:05.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 20 remove_watcher() i=7 2024-05-08T18:05:05.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56039d2fb4a0 msgr2=0x56039d2fd890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56039d2fb4a0 0x56039d2fd890 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56039d2f6ab0 msgr2=0x56039d2f8f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56039d2f6ab0 0x56039d2f8f70 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f788002e910 msgr2=0x7f788003a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f788002e910 0x7f788003a3c0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56039d2ffe30 msgr2=0x56039d3022f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56039d2ffe30 0x56039d3022f0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f776003bb50 msgr2=0x7f776003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f776003bb50 0x7f776003e010 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f788000ffd0 tx=0x7f7880023040 comp rx=0 tx=0).stop 2024-05-08T18:05:05.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d132bb0 msgr2=0x56039d2eb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 1 --2- 172.21.15.50:0/2815681201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d132bb0 0x56039d2eb4e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7888067fa0 tx=0x7f788807fed0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 reap_dead start 2024-05-08T18:05:05.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f7897429640 1 -- 172.21.15.50:0/2815681201 reap_dead start 2024-05-08T18:05:05.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 shutdown_connections 2024-05-08T18:05:05.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 >> 172.21.15.50:0/2815681201 conn(0x56039d2de250 msgr2=0x56039d2db8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 shutdown_connections 2024-05-08T18:05:05.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.432+0000 7f789af31800 1 -- 172.21.15.50:0/2815681201 wait complete. 2024-05-08T18:05:05.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 -- 172.21.15.50:0/2463955542 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f777003bd40 msgr2=0x7f777003e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 --2- 172.21.15.50:0/2463955542 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f777003bd40 0x7f777003e200 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x56039d2be590 tx=0x7f7880006000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 -- 172.21.15.50:0/2463955542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d2d4d30 msgr2=0x56039d2d5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 --2- 172.21.15.50:0/2463955542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d2d4d30 0x56039d2d5150 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7888092960 tx=0x7f78880096e0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 -- 172.21.15.50:0/2463955542 shutdown_connections 2024-05-08T18:05:05.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 --2- 172.21.15.50:0/2463955542 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f777003bd40 0x7f777003e200 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 --2- 172.21.15.50:0/2463955542 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d2d4d30 0x56039d2d5150 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 -- 172.21.15.50:0/2463955542 >> 172.21.15.50:0/2463955542 conn(0x56039d2c8dd0 msgr2=0x56039d2c91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 -- 172.21.15.50:0/2463955542 shutdown_connections 2024-05-08T18:05:05.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 -- 172.21.15.50:0/2463955542 wait complete. 2024-05-08T18:05:05.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 -- 172.21.15.50:0/1480485585 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56039d2bffe0 msgr2=0x56039d2c23d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56039d2bffe0 0x56039d2c23d0 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 -- 172.21.15.50:0/1480485585 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56039d1be8d0 msgr2=0x56039d2bd7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56039d1be8d0 0x56039d2bd7b0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 -- 172.21.15.50:0/1480485585 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56039d2c3f20 msgr2=0x56039d2c6310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56039d2c3f20 0x56039d2c6310 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 -- 172.21.15.50:0/1480485585 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f777c03bba0 msgr2=0x7f777c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f777c03bba0 0x7f777c03e060 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7880000c00 tx=0x7f78800013b0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 -- 172.21.15.50:0/1480485585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 msgr2=0x56039d2b65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 --2- 172.21.15.50:0/1480485585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56039d1bd710 0x56039d2b65c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f788801ce60 tx=0x7f7888002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f7897429640 1 -- 172.21.15.50:0/1480485585 reap_dead start 2024-05-08T18:05:05.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 -- 172.21.15.50:0/1480485585 shutdown_connections 2024-05-08T18:05:05.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 -- 172.21.15.50:0/1480485585 >> 172.21.15.50:0/1480485585 conn(0x56039d1b8c90 msgr2=0x56039d2a8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 -- 172.21.15.50:0/1480485585 shutdown_connections 2024-05-08T18:05:05.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.433+0000 7f789af31800 1 -- 172.21.15.50:0/1480485585 wait complete. 2024-05-08T18:05:05.442 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:05.443 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-05-08T18:05:05.443 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-05-08T18:05:05.443 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-05-08T18:05:05.488 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:05.488 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:05.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.491+0000 7f25eeda2800 1 Processor -- start 2024-05-08T18:05:05.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.491+0000 7f25eeda2800 1 -- start start 2024-05-08T18:05:05.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.491+0000 7f25eeda2800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5610073ec6f0 0x5610073ecb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.491+0000 7f25eeda2800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5610073ed190 con 0x5610073ed8b0 2024-05-08T18:05:05.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.491+0000 7f25eeda2800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x56100728ce20 con 0x5610073ec6f0 2024-05-08T18:05:05.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.492+0000 7f25ec8a1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5610073ec6f0 0x5610073ecb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.492+0000 7f25ec0a0640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5610073ed8b0 0x5610073edd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43010/0 (socket says 172.21.15.50:43010) 2024-05-08T18:05:05.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.492+0000 7f25ec0a0640 1 -- 172.21.15.50:0/59738989 learned_addr learned my addr 172.21.15.50:0/59738989 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:05.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.492+0000 7f25ec8a1640 1 -- 172.21.15.50:0/59738989 >> v1:172.21.15.46:6789/0 conn(0x5610073ed8b0 legacy=0x5610073edd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.492+0000 7f25ec8a1640 1 -- 172.21.15.50:0/59738989 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610073ee470 con 0x5610073ec6f0 2024-05-08T18:05:05.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.494+0000 7f25ec8a1640 1 --2- 172.21.15.50:0/59738989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 0x5610073ecb10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00a470 tx=0x7f25dc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95324fd1dfa3ba9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.494+0000 7f25eb89f640 1 -- 172.21.15.50:0/59738989 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25dc057070 con 0x5610073ec6f0 2024-05-08T18:05:05.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.494+0000 7f25eb89f640 1 -- 172.21.15.50:0/59738989 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25dc04adc0 con 0x5610073ec6f0 2024-05-08T18:05:05.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.494+0000 7f25eb89f640 1 -- 172.21.15.50:0/59738989 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25dc052590 con 0x5610073ec6f0 2024-05-08T18:05:05.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.494+0000 7f25eeda2800 1 -- 172.21.15.50:0/59738989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 msgr2=0x5610073ecb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.494+0000 7f25eeda2800 1 --2- 172.21.15.50:0/59738989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 0x5610073ecb10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f25dc00a470 tx=0x7f25dc01ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:05.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.495+0000 7f25eeda2800 1 -- 172.21.15.50:0/59738989 shutdown_connections 2024-05-08T18:05:05.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.495+0000 7f25eeda2800 1 --2- 172.21.15.50:0/59738989 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 0x5610073ecb10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.495+0000 7f25eeda2800 1 -- 172.21.15.50:0/59738989 >> 172.21.15.50:0/59738989 conn(0x5610073e7c70 msgr2=0x5610073ea0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.495+0000 7f25eeda2800 1 -- 172.21.15.50:0/59738989 shutdown_connections 2024-05-08T18:05:05.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.495+0000 7f25eeda2800 1 -- 172.21.15.50:0/59738989 wait complete. 2024-05-08T18:05:05.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.510+0000 7f25eeda2800 1 Processor -- start 2024-05-08T18:05:05.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.510+0000 7f25eeda2800 1 -- start start 2024-05-08T18:05:05.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.510+0000 7f25eeda2800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 0x561007361ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.510+0000 7f25eeda2800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561007337a10 con 0x5610073ec6f0 2024-05-08T18:05:05.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.510+0000 7f25ec8a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 0x561007361ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.510+0000 7f25ec8a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 0x561007361ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36842/0 (socket says 172.21.15.50:36842) 2024-05-08T18:05:05.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.510+0000 7f25ec8a1640 1 -- 172.21.15.50:0/3215467249 learned_addr learned my addr 172.21.15.50:0/3215467249 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.510+0000 7f25ec8a1640 1 -- 172.21.15.50:0/3215467249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610073604a0 con 0x5610073ec6f0 2024-05-08T18:05:05.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.510+0000 7f25ec8a1640 1 --2- 172.21.15.50:0/3215467249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 0x561007361ee0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f25dc055040 tx=0x7f25dc04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2302762a653199 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.511+0000 7f24dae0d640 1 -- 172.21.15.50:0/3215467249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25dc057040 con 0x5610073ec6f0 2024-05-08T18:05:05.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.511+0000 7f24dae0d640 1 -- 172.21.15.50:0/3215467249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25dc062070 con 0x5610073ec6f0 2024-05-08T18:05:05.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.511+0000 7f24dae0d640 1 -- 172.21.15.50:0/3215467249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25dc05d470 con 0x5610073ec6f0 2024-05-08T18:05:05.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.511+0000 7f25eeda2800 1 -- 172.21.15.50:0/3215467249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 msgr2=0x561007361ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.511+0000 7f25eeda2800 1 --2- 172.21.15.50:0/3215467249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 0x561007361ee0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f25dc055040 tx=0x7f25dc04c000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.511+0000 7f25eeda2800 1 -- 172.21.15.50:0/3215467249 shutdown_connections 2024-05-08T18:05:05.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.511+0000 7f25eeda2800 1 --2- 172.21.15.50:0/3215467249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 0x561007361ee0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.511+0000 7f25eeda2800 1 -- 172.21.15.50:0/3215467249 >> 172.21.15.50:0/3215467249 conn(0x5610073e7c70 msgr2=0x5610074d9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.511+0000 7f25eeda2800 1 -- 172.21.15.50:0/3215467249 shutdown_connections 2024-05-08T18:05:05.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.511+0000 7f25eeda2800 1 -- 172.21.15.50:0/3215467249 wait complete. 2024-05-08T18:05:05.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.512+0000 7f25eeda2800 1 Processor -- start 2024-05-08T18:05:05.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.512+0000 7f25eeda2800 1 -- start start 2024-05-08T18:05:05.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.512+0000 7f25eeda2800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 0x5610074e7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.512+0000 7f25eeda2800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5610073f5d80 con 0x5610073ec6f0 2024-05-08T18:05:05.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.512+0000 7f25ec8a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 0x5610074e7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.512+0000 7f25ec8a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 0x5610074e7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36858/0 (socket says 172.21.15.50:36858) 2024-05-08T18:05:05.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.512+0000 7f25ec8a1640 1 -- 172.21.15.50:0/235681224 learned_addr learned my addr 172.21.15.50:0/235681224 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.513+0000 7f25ec8a1640 1 -- 172.21.15.50:0/235681224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610074e7e40 con 0x5610073ec6f0 2024-05-08T18:05:05.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.513+0000 7f25ec8a1640 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 0x5610074e7900 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f25dc01ce60 tx=0x7f25dc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.513+0000 7f24d960a640 1 -- 172.21.15.50:0/235681224 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25dc06b070 con 0x5610073ec6f0 2024-05-08T18:05:05.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.513+0000 7f24d960a640 1 -- 172.21.15.50:0/235681224 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25dc05d970 con 0x5610073ec6f0 2024-05-08T18:05:05.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.513+0000 7f24d960a640 1 -- 172.21.15.50:0/235681224 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25dc057070 con 0x5610073ec6f0 2024-05-08T18:05:05.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.513+0000 7f25eeda2800 1 -- 172.21.15.50:0/235681224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5610074e80d0 con 0x5610073ec6f0 2024-05-08T18:05:05.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.513+0000 7f25eeda2800 1 -- 172.21.15.50:0/235681224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610074ead80 con 0x5610073ec6f0 2024-05-08T18:05:05.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.514+0000 7f24d960a640 1 -- 172.21.15.50:0/235681224 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f25dc05dc90 con 0x5610073ec6f0 2024-05-08T18:05:05.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.514+0000 7f24d960a640 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f24d403bba0 0x7f24d403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.514+0000 7f24d960a640 1 -- 172.21.15.50:0/235681224 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f25dc0945f0 con 0x5610073ec6f0 2024-05-08T18:05:05.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.514+0000 7f25ec0a0640 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f24d403bba0 0x7f24d403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.514+0000 7f25eeda2800 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5610073ed8b0 0x5610074eeb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.514+0000 7f25eeda2800 1 -- 172.21.15.50:0/235681224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5610074ef080 con 0x5610073ed8b0 2024-05-08T18:05:05.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.514+0000 7f25ed0a2640 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5610073ed8b0 0x5610074eeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.514+0000 7f25ec0a0640 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f24d403bba0 0x7f24d403e060 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f25e0001d70 tx=0x7f25e00012d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.515+0000 7f25ed0a2640 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5610073ed8b0 0x5610074eeb40 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.515+0000 7f25ed0a2640 1 -- 172.21.15.50:0/235681224 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f25e400a4a0 con 0x5610073ed8b0 2024-05-08T18:05:05.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.515+0000 7f25eeda2800 1 -- 172.21.15.50:0/235681224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5610074ef920 con 0x5610073ed8b0 2024-05-08T18:05:05.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.515+0000 7f25ed0a2640 1 -- 172.21.15.50:0/235681224 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f25e400a4a0 con 0x5610073ed8b0 2024-05-08T18:05:05.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.515+0000 7f25eeda2800 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5610074f1270 0x5610074f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.515+0000 7f25eeda2800 1 -- 172.21.15.50:0/235681224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5610074f3e90 con 0x5610074f1270 2024-05-08T18:05:05.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.516+0000 7f25ec8a1640 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5610074f1270 0x5610074f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.516+0000 7f25ec8a1640 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5610074f1270 0x5610074f3700 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.516+0000 7f25ec8a1640 1 -- 172.21.15.50:0/235681224 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f25dc05a040 con 0x5610074f1270 2024-05-08T18:05:05.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.516+0000 7f25eeda2800 1 -- 172.21.15.50:0/235681224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5610074f0170 con 0x5610073ed8b0 2024-05-08T18:05:05.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.517+0000 7f25ed0a2640 1 -- 172.21.15.50:0/235681224 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f25e400a4a0 con 0x5610073ed8b0 2024-05-08T18:05:05.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.517+0000 7f25eeda2800 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5610074f52f0 0x5610074f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.517+0000 7f25eeda2800 1 -- 172.21.15.50:0/235681224 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5610074f7da0 con 0x5610074f52f0 2024-05-08T18:05:05.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.517+0000 7f25ec0a0640 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5610074f52f0 0x5610074f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.517+0000 7f25ec0a0640 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5610074f52f0 0x5610074f76e0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.517+0000 7f25ec0a0640 1 -- 172.21.15.50:0/235681224 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f25e0009050 con 0x5610074f52f0 2024-05-08T18:05:05.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.518+0000 7f25eeda2800 1 Processor -- start 2024-05-08T18:05:05.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.518+0000 7f25eeda2800 1 -- start start 2024-05-08T18:05:05.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.518+0000 7f25eeda2800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561007506110 0x561007506530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.518+0000 7f25eeda2800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5610072de330 con 0x561007506110 2024-05-08T18:05:05.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.518+0000 7f25ec8a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561007506110 0x561007506530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.518+0000 7f25ec8a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561007506110 0x561007506530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36864/0 (socket says 172.21.15.50:36864) 2024-05-08T18:05:05.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.518+0000 7f25ec8a1640 1 -- 172.21.15.50:0/2335670654 learned_addr learned my addr 172.21.15.50:0/2335670654 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.518+0000 7f25ec8a1640 1 -- 172.21.15.50:0/2335670654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561007506a70 con 0x561007506110 2024-05-08T18:05:05.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.518+0000 7f25ec8a1640 1 --2- 172.21.15.50:0/2335670654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561007506110 0x561007506530 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f25dc05b7e0 tx=0x7f25dc05b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.518+0000 7f24d27fc640 1 -- 172.21.15.50:0/2335670654 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25dc070e40 con 0x561007506110 2024-05-08T18:05:05.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.518+0000 7f24d27fc640 1 -- 172.21.15.50:0/2335670654 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25dc009410 con 0x561007506110 2024-05-08T18:05:05.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.518+0000 7f25eeda2800 1 -- 172.21.15.50:0/2335670654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5610075074e0 con 0x561007506110 2024-05-08T18:05:05.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.518+0000 7f24d27fc640 1 -- 172.21.15.50:0/2335670654 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25dc0097a0 con 0x561007506110 2024-05-08T18:05:05.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.519+0000 7f25eeda2800 1 -- 172.21.15.50:0/2335670654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610075079e0 con 0x561007506110 2024-05-08T18:05:05.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.519+0000 7f24d27fc640 1 -- 172.21.15.50:0/2335670654 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f25dc009940 con 0x561007506110 2024-05-08T18:05:05.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.519+0000 7f24d27fc640 1 --2- 172.21.15.50:0/2335670654 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f24c403bba0 0x7f24c403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.519+0000 7f24d27fc640 1 -- 172.21.15.50:0/2335670654 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f25dc088ce0 con 0x561007506110 2024-05-08T18:05:05.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.520+0000 7f25ec0a0640 1 --2- 172.21.15.50:0/2335670654 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f24c403bba0 0x7f24c403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.520+0000 7f25eeda2800 1 Processor -- start 2024-05-08T18:05:05.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.520+0000 7f25eeda2800 1 -- start start 2024-05-08T18:05:05.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.520+0000 7f25eeda2800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610075132c0 0x5610075136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.520+0000 7f25eeda2800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561007360760 con 0x5610075132c0 2024-05-08T18:05:05.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.520+0000 7f25ec0a0640 1 --2- 172.21.15.50:0/2335670654 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f24c403bba0 0x7f24c403e060 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f25e000bf40 tx=0x7f25e000f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.520+0000 7f25ec8a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610075132c0 0x5610075136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.520+0000 7f25ec8a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610075132c0 0x5610075136a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36866/0 (socket says 172.21.15.50:36866) 2024-05-08T18:05:05.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.520+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1552789897 learned_addr learned my addr 172.21.15.50:0/1552789897 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.520+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1552789897 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561007513c70 con 0x5610075132c0 2024-05-08T18:05:05.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.521+0000 7f25ec8a1640 1 --2- 172.21.15.50:0/1552789897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610075132c0 0x5610075136a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f25dc07dbc0 tx=0x7f25dc094f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bca929c3196e216 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.521+0000 7f24c3fff640 1 -- 172.21.15.50:0/1552789897 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25dc0a7030 con 0x5610075132c0 2024-05-08T18:05:05.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.521+0000 7f24c3fff640 1 -- 172.21.15.50:0/1552789897 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25dc091580 con 0x5610075132c0 2024-05-08T18:05:05.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.521+0000 7f24c3fff640 1 -- 172.21.15.50:0/1552789897 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25dc06e6c0 con 0x5610075132c0 2024-05-08T18:05:05.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.521+0000 7f25eeda2800 1 -- 172.21.15.50:0/1552789897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610075132c0 msgr2=0x5610075136a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.521+0000 7f25eeda2800 1 --2- 172.21.15.50:0/1552789897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610075132c0 0x5610075136a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f25dc07dbc0 tx=0x7f25dc094f10 comp rx=0 tx=0).stop 2024-05-08T18:05:05.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.521+0000 7f25eeda2800 1 -- 172.21.15.50:0/1552789897 shutdown_connections 2024-05-08T18:05:05.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.521+0000 7f25eeda2800 1 --2- 172.21.15.50:0/1552789897 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610075132c0 0x5610075136a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.521+0000 7f25eeda2800 1 -- 172.21.15.50:0/1552789897 >> 172.21.15.50:0/1552789897 conn(0x56100750f590 msgr2=0x56100750d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.521+0000 7f25eeda2800 1 -- 172.21.15.50:0/1552789897 shutdown_connections 2024-05-08T18:05:05.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.521+0000 7f25eeda2800 1 -- 172.21.15.50:0/1552789897 wait complete. 2024-05-08T18:05:05.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.522+0000 7f25eeda2800 1 Processor -- start 2024-05-08T18:05:05.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.522+0000 7f25eeda2800 1 -- start start 2024-05-08T18:05:05.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.522+0000 7f25eeda2800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610075132c0 0x56100751a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.522+0000 7f25eeda2800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561007514380 con 0x5610075132c0 2024-05-08T18:05:05.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.522+0000 7f25ec8a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610075132c0 0x56100751a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.522+0000 7f25ec8a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610075132c0 0x56100751a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36872/0 (socket says 172.21.15.50:36872) 2024-05-08T18:05:05.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.522+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 learned_addr learned my addr 172.21.15.50:0/1386954147 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.522+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56100751ab20 con 0x5610075132c0 2024-05-08T18:05:05.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.522+0000 7f25ec8a1640 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610075132c0 0x56100751a5e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f25dc07dbc0 tx=0x7f25dc002ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.522+0000 7f24c27fc640 1 -- 172.21.15.50:0/1386954147 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25dc06ea20 con 0x5610075132c0 2024-05-08T18:05:05.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.522+0000 7f24c27fc640 1 -- 172.21.15.50:0/1386954147 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25dc07d750 con 0x5610075132c0 2024-05-08T18:05:05.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.522+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56100751ab20 con 0x5610075132c0 2024-05-08T18:05:05.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.523+0000 7f24c27fc640 1 -- 172.21.15.50:0/1386954147 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f25dc06ebc0 con 0x5610075132c0 2024-05-08T18:05:05.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.523+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56100751ab20 con 0x5610075132c0 2024-05-08T18:05:05.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.524+0000 7f24c27fc640 1 -- 172.21.15.50:0/1386954147 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f25dc0b5020 con 0x5610075132c0 2024-05-08T18:05:05.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.524+0000 7f24c27fc640 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f24b803bb00 0x7f24b803dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.524+0000 7f24c27fc640 1 -- 172.21.15.50:0/1386954147 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f25dc0a8070 con 0x5610075132c0 2024-05-08T18:05:05.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.524+0000 7f25ec0a0640 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f24b803bb00 0x7f24b803dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.524+0000 7f25eeda2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.524+0000 7f25eeda2800 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561007525d70 0x561007528230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.524+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561007528980 con 0x561007525d70 2024-05-08T18:05:05.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.524+0000 7f25ed0a2640 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561007525d70 0x561007528230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.524+0000 7f25ec0a0640 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f24b803bb00 0x7f24b803dfc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f25e000c010 tx=0x7f25e0010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.525+0000 7f25ed0a2640 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561007525d70 0x561007528230 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.525+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f25e400a4a0 con 0x561007525d70 2024-05-08T18:05:05.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.525+0000 7f25eeda2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.525+0000 7f25eeda2800 20 realm 2024-05-08T18:05:05.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.525+0000 7f25eeda2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.525+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561007528d90 con 0x561007525d70 2024-05-08T18:05:05.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.525+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f25e400a4a0 con 0x561007525d70 2024-05-08T18:05:05.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.525+0000 7f25eeda2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.525+0000 7f25eeda2800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:05.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.525+0000 7f25eeda2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.525+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561007529660 con 0x561007525d70 2024-05-08T18:05:05.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.525+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f25e400a4a0 con 0x561007525d70 2024-05-08T18:05:05.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.525+0000 7f25eeda2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.525+0000 7f25eeda2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.525+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x561007525d70 2024-05-08T18:05:05.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.526+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f25e400a4a0 con 0x561007525d70 2024-05-08T18:05:05.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.526+0000 7f25eeda2800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:05.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.526+0000 7f25eeda2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.526+0000 7f25eeda2800 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56100752a4f0 0x56100752c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.526+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56100752d040 con 0x56100752a4f0 2024-05-08T18:05:05.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.526+0000 7f25ec8a1640 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56100752a4f0 0x56100752c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.526+0000 7f25ec8a1640 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56100752a4f0 0x56100752c8e0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.526+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.526+0000 7f25eeda2800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:05.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.526+0000 7f25eeda2800 20 searching for the correct realm 2024-05-08T18:05:05.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.526+0000 7f25eeda2800 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56100752f0f0 0x5610075315b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.526+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007531c80 con 0x56100752f0f0 2024-05-08T18:05:05.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.526+0000 7f25ec0a0640 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56100752f0f0 0x5610075315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.527+0000 7f25ec0a0640 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56100752f0f0 0x5610075315b0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.527+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e0009050 con 0x56100752f0f0 2024-05-08T18:05:05.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.527+0000 7f25ec0a0640 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f25e0037040 0x7f25e0036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.527+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007531c80 con 0x7f25e0037040 2024-05-08T18:05:05.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.527+0000 7f25ed0a2640 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f25e0037040 0x7f25e0036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.527+0000 7f25ed0a2640 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f25e0037040 0x7f25e0036410 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.528+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e400a4a0 con 0x7f25e0037040 2024-05-08T18:05:05.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.528+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752a4f0 2024-05-08T18:05:05.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.528+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.528+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x561007525d70 2024-05-08T18:05:05.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.528+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e400a4a0 con 0x561007525d70 2024-05-08T18:05:05.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.528+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x561007525d70 2024-05-08T18:05:05.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.528+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e400a4a0 con 0x561007525d70 2024-05-08T18:05:05.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.528+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752a4f0 2024-05-08T18:05:05.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.529+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.529+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752a4f0 2024-05-08T18:05:05.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.529+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.529+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x7f25e0037040 2024-05-08T18:05:05.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.529+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e400a4a0 con 0x7f25e0037040 2024-05-08T18:05:05.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.529+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752f0f0 2024-05-08T18:05:05.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.530+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e0009050 con 0x56100752f0f0 2024-05-08T18:05:05.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.530+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x561007525d70 2024-05-08T18:05:05.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.530+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e400a4a0 con 0x561007525d70 2024-05-08T18:05:05.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.530+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752a4f0 2024-05-08T18:05:05.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.530+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.530+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x561007525d70 2024-05-08T18:05:05.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.530+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e400a4a0 con 0x561007525d70 2024-05-08T18:05:05.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.530+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x7f25e0037040 2024-05-08T18:05:05.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.531+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f25e400a4a0 con 0x7f25e0037040 2024-05-08T18:05:05.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.531+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752f0f0 2024-05-08T18:05:05.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.531+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e0009050 con 0x56100752f0f0 2024-05-08T18:05:05.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.531+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752f0f0 2024-05-08T18:05:05.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.531+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e0009050 con 0x56100752f0f0 2024-05-08T18:05:05.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.531+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752a4f0 2024-05-08T18:05:05.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.531+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.531+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752f0f0 2024-05-08T18:05:05.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.532+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e0009050 con 0x56100752f0f0 2024-05-08T18:05:05.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.532+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752f0f0 2024-05-08T18:05:05.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.532+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f25e0009050 con 0x56100752f0f0 2024-05-08T18:05:05.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.532+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x561007525d70 2024-05-08T18:05:05.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.532+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e400a4a0 con 0x561007525d70 2024-05-08T18:05:05.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.532+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x561007525d70 2024-05-08T18:05:05.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.532+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f25e400a4a0 con 0x561007525d70 2024-05-08T18:05:05.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.532+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752a4f0 2024-05-08T18:05:05.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.533+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.533+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752f0f0 2024-05-08T18:05:05.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.533+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e0009050 con 0x56100752f0f0 2024-05-08T18:05:05.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.533+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x7f25e0037040 2024-05-08T18:05:05.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.533+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e400a4a0 con 0x7f25e0037040 2024-05-08T18:05:05.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.533+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x561007525d70 2024-05-08T18:05:05.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.533+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e400a4a0 con 0x561007525d70 2024-05-08T18:05:05.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.533+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752a4f0 2024-05-08T18:05:05.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.534+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.534+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752f0f0 2024-05-08T18:05:05.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.534+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e0009050 con 0x56100752f0f0 2024-05-08T18:05:05.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.534+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752a4f0 2024-05-08T18:05:05.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.534+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.534+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x7f25e0037040 2024-05-08T18:05:05.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.534+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e400a4a0 con 0x7f25e0037040 2024-05-08T18:05:05.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.534+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752f0f0 2024-05-08T18:05:05.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.535+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e0009050 con 0x56100752f0f0 2024-05-08T18:05:05.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.535+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x56100752a4f0 2024-05-08T18:05:05.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.535+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.535+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x561007525d70 2024-05-08T18:05:05.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.535+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f25e400a4a0 con 0x561007525d70 2024-05-08T18:05:05.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.535+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x561007529a00 con 0x7f25e0037040 2024-05-08T18:05:05.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.535+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f25e400a4a0 con 0x7f25e0037040 2024-05-08T18:05:05.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.535+0000 7f25eeda2800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:05.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.535+0000 7f25eeda2800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:05.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.535+0000 7f25eeda2800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:05.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.535+0000 7f25eeda2800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:05.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.535+0000 7f25eeda2800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:05.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.535+0000 7f25eeda2800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:05.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.535+0000 7f25eeda2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.535+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56100752d9f0 con 0x561007525d70 2024-05-08T18:05:05.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f25e400a4a0 con 0x561007525d70 2024-05-08T18:05:05.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25eeda2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25eeda2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56100752d9f0 con 0x561007525d70 2024-05-08T18:05:05.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f25e400a4a0 con 0x561007525d70 2024-05-08T18:05:05.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25eeda2800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:05.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25eeda2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56100752d9c0 con 0x56100752f0f0 2024-05-08T18:05:05.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f25e0009050 con 0x56100752f0f0 2024-05-08T18:05:05.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25eeda2800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:05.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25eeda2800 20 zone default found 2024-05-08T18:05:05.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25eeda2800 4 Realm: () 2024-05-08T18:05:05.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25eeda2800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:05.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25eeda2800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:05.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25eeda2800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:05.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25eeda2800 20 zonegroup default 2024-05-08T18:05:05.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25eeda2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.536+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56100752e290 con 0x56100752a4f0 2024-05-08T18:05:05.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.537+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.537+0000 7f25eeda2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.537+0000 7f25eeda2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.537+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561007533ae0 con 0x56100752f0f0 2024-05-08T18:05:05.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.537+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f25e0009050 con 0x56100752f0f0 2024-05-08T18:05:05.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.537+0000 7f25eeda2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.537+0000 7f25eeda2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.537+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x561007533e80 con 0x56100752f0f0 2024-05-08T18:05:05.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.537+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f25e0009050 con 0x56100752f0f0 2024-05-08T18:05:05.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.537+0000 7f25eeda2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.537+0000 7f25eeda2800 20 started sync module instance, tier type = 2024-05-08T18:05:05.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.537+0000 7f25eeda2800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:05.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.537+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007532f80 con 0x7f25e0037040 2024-05-08T18:05:05.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.538+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25e400a4a0 con 0x7f25e0037040 2024-05-08T18:05:05.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.538+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94626842358992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007532f80 con 0x7f25e0037040 2024-05-08T18:05:05.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.538+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007534900 con 0x561007525d70 2024-05-08T18:05:05.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.539+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94626842358992] v27'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25e4040040 con 0x7f25e0037040 2024-05-08T18:05:05.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.539+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25e4040040 con 0x561007525d70 2024-05-08T18:05:05.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.539+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94626842362864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007532f80 con 0x561007525d70 2024-05-08T18:05:05.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.539+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5610075353a0 con 0x56100752f0f0 2024-05-08T18:05:05.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.540+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94626842362864] v27'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25e4040040 con 0x561007525d70 2024-05-08T18:05:05.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.540+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25e0009050 con 0x56100752f0f0 2024-05-08T18:05:05.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.540+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94626842356432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007532ef0 con 0x56100752f0f0 2024-05-08T18:05:05.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.541+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007536260 con 0x7f25e0037040 2024-05-08T18:05:05.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.542+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25e4040040 con 0x7f25e0037040 2024-05-08T18:05:05.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.542+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94626842356432] v27'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25e0009050 con 0x56100752f0f0 2024-05-08T18:05:05.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.542+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94626842369936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007536430 con 0x7f25e0037040 2024-05-08T18:05:05.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.542+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007536f50 con 0x561007525d70 2024-05-08T18:05:05.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.542+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25e4040040 con 0x561007525d70 2024-05-08T18:05:05.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.543+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94626842372640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007536f10 con 0x561007525d70 2024-05-08T18:05:05.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.543+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007537ec0 con 0x56100752a4f0 2024-05-08T18:05:05.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.543+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94626842369936] v27'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25e4040040 con 0x7f25e0037040 2024-05-08T18:05:05.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.543+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.544+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94626842372640] v27'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25e4040040 con 0x561007525d70 2024-05-08T18:05:05.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.544+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94626842376688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5610075382b0 con 0x56100752a4f0 2024-05-08T18:05:05.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.544+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007537340 con 0x56100752a4f0 2024-05-08T18:05:05.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.545+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94626842376688] v27'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.545+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.545+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94626842367888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007537340 con 0x56100752a4f0 2024-05-08T18:05:05.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.545+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007539120 con 0x56100752a4f0 2024-05-08T18:05:05.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.546+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94626842367888] v27'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.546+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.546+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94626842380224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007539090 con 0x56100752a4f0 2024-05-08T18:05:05.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.546+0000 7f25eeda2800 20 add_watcher() i=0 2024-05-08T18:05:05.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.546+0000 7f25eeda2800 20 add_watcher() i=1 2024-05-08T18:05:05.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.546+0000 7f25eeda2800 20 add_watcher() i=2 2024-05-08T18:05:05.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.546+0000 7f25eeda2800 20 add_watcher() i=3 2024-05-08T18:05:05.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.546+0000 7f25eeda2800 20 add_watcher() i=4 2024-05-08T18:05:05.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.546+0000 7f25eeda2800 20 add_watcher() i=5 2024-05-08T18:05:05.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.546+0000 7f25eeda2800 20 add_watcher() i=6 2024-05-08T18:05:05.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.547+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94626842380224] v27'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.547+0000 7f25eeda2800 20 add_watcher() i=7 2024-05-08T18:05:05.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.547+0000 7f25eeda2800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:05.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.547+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56100753c520 con 0x561007525d70 2024-05-08T18:05:05.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.547+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f25e4040040 con 0x561007525d70 2024-05-08T18:05:05.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.547+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94626842381744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5610075399e0 con 0x561007525d70 2024-05-08T18:05:05.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.548+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94626842381744] v27'74 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f25e4040040 con 0x561007525d70 2024-05-08T18:05:05.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.548+0000 7f24a37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:05.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.548+0000 7f25eeda2800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:05.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.548+0000 7f25eeda2800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:05.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.548+0000 7f25eeda2800 5 note: GC not initialized 2024-05-08T18:05:05.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.548+0000 7f24a27e4640 20 reqs_thread_entry: start 2024-05-08T18:05:05.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.549+0000 7f25eeda2800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:05.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.549+0000 7f25eeda2800 20 Filter name: none 2024-05-08T18:05:05.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.549+0000 7f249bfff640 20 reqs_thread_entry: start 2024-05-08T18:05:05.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.549+0000 7f25eeda2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.549+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56100756f570 con 0x561007525d70 2024-05-08T18:05:05.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.549+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv7 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7f25e4040040 con 0x561007525d70 2024-05-08T18:05:05.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.549+0000 7f25eeda2800 20 rados_obj.operate() r=0 bl.length=556 2024-05-08T18:05:05.557 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:05.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.550+0000 7f24a37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:05.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.550+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94626842381744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56100756e120 con 0x561007525d70 2024-05-08T18:05:05.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.551+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94626842381744] v27'75 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f25e4040040 con 0x561007525d70 2024-05-08T18:05:05.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.551+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94626842358992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007538d80 con 0x7f25e0037040 2024-05-08T18:05:05.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.552+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94626842358992] v27'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25e4040040 con 0x7f25e0037040 2024-05-08T18:05:05.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.552+0000 7f25eeda2800 20 remove_watcher() i=0 2024-05-08T18:05:05.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.552+0000 7f25eeda2800 2 removed watcher, disabling cache 2024-05-08T18:05:05.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.552+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94626842362864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007538d80 con 0x561007525d70 2024-05-08T18:05:05.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.553+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94626842362864] v27'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25e4040040 con 0x561007525d70 2024-05-08T18:05:05.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.553+0000 7f25eeda2800 20 remove_watcher() i=1 2024-05-08T18:05:05.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.554+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94626842356432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5610075343f0 con 0x56100752f0f0 2024-05-08T18:05:05.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.554+0000 7f25ec0a0640 1 -- 172.21.15.50:0/1386954147 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94626842356432] v27'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25e0009050 con 0x56100752f0f0 2024-05-08T18:05:05.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.554+0000 7f25eeda2800 20 remove_watcher() i=2 2024-05-08T18:05:05.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.554+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94626842369936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007532ad0 con 0x7f25e0037040 2024-05-08T18:05:05.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.555+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94626842369936] v27'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25e4040040 con 0x7f25e0037040 2024-05-08T18:05:05.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.555+0000 7f25eeda2800 20 remove_watcher() i=3 2024-05-08T18:05:05.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.555+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94626842372640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007532ad0 con 0x561007525d70 2024-05-08T18:05:05.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.556+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94626842372640] v27'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25e4040040 con 0x561007525d70 2024-05-08T18:05:05.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.556+0000 7f25eeda2800 20 remove_watcher() i=4 2024-05-08T18:05:05.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.556+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94626842376688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007532ad0 con 0x56100752a4f0 2024-05-08T18:05:05.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.557+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94626842376688] v27'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.557+0000 7f25eeda2800 20 remove_watcher() i=5 2024-05-08T18:05:05.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.557+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94626842367888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5610075379f0 con 0x56100752a4f0 2024-05-08T18:05:05.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.558+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94626842367888] v27'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.558+0000 7f25eeda2800 20 remove_watcher() i=6 2024-05-08T18:05:05.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.558+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94626842380224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x561007532ad0 con 0x56100752a4f0 2024-05-08T18:05:05.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.559+0000 7f25ec8a1640 1 -- 172.21.15.50:0/1386954147 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94626842380224] v27'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f25dc05a040 con 0x56100752a4f0 2024-05-08T18:05:05.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.559+0000 7f25eeda2800 20 remove_watcher() i=7 2024-05-08T18:05:05.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.559+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56100752a4f0 msgr2=0x56100752c8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.559+0000 7f25eeda2800 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56100752a4f0 0x56100752c8e0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.559+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561007525d70 msgr2=0x561007528230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.559+0000 7f25eeda2800 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561007525d70 0x561007528230 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.559+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f25e0037040 msgr2=0x7f25e0036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.559+0000 7f25eeda2800 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f25e0037040 0x7f25e0036410 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.559+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56100752f0f0 msgr2=0x5610075315b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.559+0000 7f25eeda2800 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56100752f0f0 0x5610075315b0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.559+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f24b803bb00 msgr2=0x7f24b803dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.559+0000 7f25eeda2800 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f24b803bb00 0x7f24b803dfc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f25e000c010 tx=0x7f25e0010000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.559+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610075132c0 msgr2=0x56100751a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.559+0000 7f25ed0a2640 1 -- 172.21.15.50:0/1386954147 reap_dead start 2024-05-08T18:05:05.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.559+0000 7f25eeda2800 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610075132c0 0x56100751a5e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f25dc07dbc0 tx=0x7f25dc002ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 shutdown_connections 2024-05-08T18:05:05.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 --2- 172.21.15.50:0/1386954147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610075132c0 0x56100751a5e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 >> 172.21.15.50:0/1386954147 conn(0x56100750f590 msgr2=0x56100750cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 shutdown_connections 2024-05-08T18:05:05.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 -- 172.21.15.50:0/1386954147 wait complete. 2024-05-08T18:05:05.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 -- 172.21.15.50:0/2335670654 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f24c403bba0 msgr2=0x7f24c403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 --2- 172.21.15.50:0/2335670654 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f24c403bba0 0x7f24c403e060 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f25e000bf40 tx=0x7f25e000f000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 -- 172.21.15.50:0/2335670654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561007506110 msgr2=0x561007506530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 --2- 172.21.15.50:0/2335670654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561007506110 0x561007506530 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f25dc05b7e0 tx=0x7f25dc05b3e0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 -- 172.21.15.50:0/2335670654 shutdown_connections 2024-05-08T18:05:05.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 --2- 172.21.15.50:0/2335670654 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f24c403bba0 0x7f24c403e060 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 --2- 172.21.15.50:0/2335670654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561007506110 0x561007506530 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 -- 172.21.15.50:0/2335670654 >> 172.21.15.50:0/2335670654 conn(0x5610074fa1b0 msgr2=0x5610074fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 -- 172.21.15.50:0/2335670654 shutdown_connections 2024-05-08T18:05:05.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 -- 172.21.15.50:0/2335670654 wait complete. 2024-05-08T18:05:05.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 -- 172.21.15.50:0/235681224 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5610074f1270 msgr2=0x5610074f3700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5610074f1270 0x5610074f3700 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 -- 172.21.15.50:0/235681224 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5610073ed8b0 msgr2=0x5610074eeb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5610073ed8b0 0x5610074eeb40 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 -- 172.21.15.50:0/235681224 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5610074f52f0 msgr2=0x5610074f76e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.560+0000 7f25eeda2800 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5610074f52f0 0x5610074f76e0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.561+0000 7f25eeda2800 1 -- 172.21.15.50:0/235681224 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f24d403bba0 msgr2=0x7f24d403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.561+0000 7f25eeda2800 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f24d403bba0 0x7f24d403e060 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f25e0001d70 tx=0x7f25e00012d0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.561+0000 7f25eeda2800 1 -- 172.21.15.50:0/235681224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 msgr2=0x5610074e7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.561+0000 7f25eeda2800 1 --2- 172.21.15.50:0/235681224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5610073ec6f0 0x5610074e7900 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f25dc01ce60 tx=0x7f25dc002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.561+0000 7f25ed0a2640 1 -- 172.21.15.50:0/235681224 reap_dead start 2024-05-08T18:05:05.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.561+0000 7f25eeda2800 1 -- 172.21.15.50:0/235681224 shutdown_connections 2024-05-08T18:05:05.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.561+0000 7f25eeda2800 1 -- 172.21.15.50:0/235681224 >> 172.21.15.50:0/235681224 conn(0x5610073e7c70 msgr2=0x561007360cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.561+0000 7f25eeda2800 1 -- 172.21.15.50:0/235681224 shutdown_connections 2024-05-08T18:05:05.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.561+0000 7f25eeda2800 1 -- 172.21.15.50:0/235681224 wait complete. 2024-05-08T18:05:05.569 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:05.569 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-05-08T18:05:05.569 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-05-08T18:05:05.569 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-05-08T18:05:05.614 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:05.615 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:05.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.617+0000 7f7ab2744800 1 Processor -- start 2024-05-08T18:05:05.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.617+0000 7f7ab2744800 1 -- start start 2024-05-08T18:05:05.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.618+0000 7f7ab2744800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x562576cad180 0x562576caf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.618+0000 7f7ab2744800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562576cb35c0 con 0x562576cafab0 2024-05-08T18:05:05.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.618+0000 7f7ab2744800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x562576b4ce20 con 0x562576cad180 2024-05-08T18:05:05.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.618+0000 7f7ab023c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x562576cad180 0x562576caf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.618+0000 7f7ab023c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x562576cad180 0x562576caf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36878/0 (socket says 172.21.15.50:36878) 2024-05-08T18:05:05.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.618+0000 7f7ab023c640 1 -- 172.21.15.50:0/1279916886 learned_addr learned my addr 172.21.15.50:0/1279916886 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.618+0000 7f7aafa3b640 1 --1- 172.21.15.50:0/1279916886 >> v1:172.21.15.46:6789/0 conn(0x562576cafab0 0x562576cb1ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43022/0 (socket says 172.21.15.50:43022) 2024-05-08T18:05:05.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.618+0000 7f7ab023c640 1 -- 172.21.15.50:0/1279916886 >> v1:172.21.15.46:6789/0 conn(0x562576cafab0 legacy=0x562576cb1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.618+0000 7f7ab023c640 1 -- 172.21.15.50:0/1279916886 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562576cb47a0 con 0x562576cad180 2024-05-08T18:05:05.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.620+0000 7f7ab023c640 1 --2- 172.21.15.50:0/1279916886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 0x562576caf570 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000a520 tx=0x7f7aa001d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30849d0a62b2799a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.620+0000 7f7aaf23a640 1 -- 172.21.15.50:0/1279916886 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa0057070 con 0x562576cad180 2024-05-08T18:05:05.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.620+0000 7f7aaf23a640 1 -- 172.21.15.50:0/1279916886 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa0052570 con 0x562576cad180 2024-05-08T18:05:05.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.620+0000 7f7aaf23a640 1 -- 172.21.15.50:0/1279916886 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa004ae60 con 0x562576cad180 2024-05-08T18:05:05.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.621+0000 7f7ab2744800 1 -- 172.21.15.50:0/1279916886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 msgr2=0x562576caf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.621+0000 7f7ab2744800 1 --2- 172.21.15.50:0/1279916886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 0x562576caf570 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000a520 tx=0x7f7aa001d010 comp rx=0 tx=0).stop 2024-05-08T18:05:05.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.621+0000 7f7ab2744800 1 -- 172.21.15.50:0/1279916886 shutdown_connections 2024-05-08T18:05:05.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.621+0000 7f7ab2744800 1 --2- 172.21.15.50:0/1279916886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 0x562576caf570 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.621+0000 7f7ab2744800 1 -- 172.21.15.50:0/1279916886 >> 172.21.15.50:0/1279916886 conn(0x562576ca6ad0 msgr2=0x562576ca8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.621+0000 7f7ab2744800 1 -- 172.21.15.50:0/1279916886 shutdown_connections 2024-05-08T18:05:05.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.621+0000 7f7ab2744800 1 -- 172.21.15.50:0/1279916886 wait complete. 2024-05-08T18:05:05.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.636+0000 7f7ab2744800 1 Processor -- start 2024-05-08T18:05:05.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.636+0000 7f7ab2744800 1 -- start start 2024-05-08T18:05:05.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.636+0000 7f7ab2744800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 0x562576d9db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.636+0000 7f7ab2744800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562576bf7a10 con 0x562576cad180 2024-05-08T18:05:05.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.636+0000 7f7ab023c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 0x562576d9db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.636+0000 7f7ab023c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 0x562576d9db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36888/0 (socket says 172.21.15.50:36888) 2024-05-08T18:05:05.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.636+0000 7f7ab023c640 1 -- 172.21.15.50:0/63749264 learned_addr learned my addr 172.21.15.50:0/63749264 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.637+0000 7f7ab023c640 1 -- 172.21.15.50:0/63749264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562576d9e120 con 0x562576cad180 2024-05-08T18:05:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.637+0000 7f7ab023c640 1 --2- 172.21.15.50:0/63749264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 0x562576d9db50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0055040 tx=0x7f7aa004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4652a3751d896d8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.637+0000 7f799e60c640 1 -- 172.21.15.50:0/63749264 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa0057040 con 0x562576cad180 2024-05-08T18:05:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.637+0000 7f799e60c640 1 -- 172.21.15.50:0/63749264 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa0052720 con 0x562576cad180 2024-05-08T18:05:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.637+0000 7f799e60c640 1 -- 172.21.15.50:0/63749264 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa0052a40 con 0x562576cad180 2024-05-08T18:05:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.638+0000 7f7ab2744800 1 -- 172.21.15.50:0/63749264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 msgr2=0x562576d9db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.638+0000 7f7ab2744800 1 --2- 172.21.15.50:0/63749264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 0x562576d9db50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0055040 tx=0x7f7aa004c000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.638+0000 7f7ab2744800 1 -- 172.21.15.50:0/63749264 shutdown_connections 2024-05-08T18:05:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.638+0000 7f7ab2744800 1 --2- 172.21.15.50:0/63749264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 0x562576d9db50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.638+0000 7f7ab2744800 1 -- 172.21.15.50:0/63749264 >> 172.21.15.50:0/63749264 conn(0x562576ca6ad0 msgr2=0x562576d99470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.638+0000 7f7ab2744800 1 -- 172.21.15.50:0/63749264 shutdown_connections 2024-05-08T18:05:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.638+0000 7f7ab2744800 1 -- 172.21.15.50:0/63749264 wait complete. 2024-05-08T18:05:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.638+0000 7f7ab2744800 1 Processor -- start 2024-05-08T18:05:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.638+0000 7f7ab2744800 1 -- start start 2024-05-08T18:05:05.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.638+0000 7f7ab2744800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 0x562576da5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.638+0000 7f7ab2744800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562576cb3a30 con 0x562576cad180 2024-05-08T18:05:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.638+0000 7f7ab023c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 0x562576da5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.638+0000 7f7ab023c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 0x562576da5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36900/0 (socket says 172.21.15.50:36900) 2024-05-08T18:05:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.638+0000 7f7ab023c640 1 -- 172.21.15.50:0/1402748333 learned_addr learned my addr 172.21.15.50:0/1402748333 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.639+0000 7f7ab023c640 1 -- 172.21.15.50:0/1402748333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562576da5ba0 con 0x562576cad180 2024-05-08T18:05:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.639+0000 7f7ab023c640 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 0x562576da5660 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0052540 tx=0x7f7aa0003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.639+0000 7f799ce09640 1 -- 172.21.15.50:0/1402748333 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa0073070 con 0x562576cad180 2024-05-08T18:05:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.639+0000 7f799ce09640 1 -- 172.21.15.50:0/1402748333 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa005d440 con 0x562576cad180 2024-05-08T18:05:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.639+0000 7f799ce09640 1 -- 172.21.15.50:0/1402748333 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa0057070 con 0x562576cad180 2024-05-08T18:05:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.639+0000 7f7ab2744800 1 -- 172.21.15.50:0/1402748333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562576da5e30 con 0x562576cad180 2024-05-08T18:05:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.639+0000 7f7ab2744800 1 -- 172.21.15.50:0/1402748333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562576da8a90 con 0x562576cad180 2024-05-08T18:05:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.640+0000 7f799ce09640 1 -- 172.21.15.50:0/1402748333 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7aa0076070 con 0x562576cad180 2024-05-08T18:05:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.640+0000 7f799ce09640 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f799803bba0 0x7f799803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.640+0000 7f799ce09640 1 -- 172.21.15.50:0/1402748333 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7aa0093ff0 con 0x562576cad180 2024-05-08T18:05:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.640+0000 7f7aafa3b640 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f799803bba0 0x7f799803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.640+0000 7f7ab2744800 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562576cafab0 0x562576dac850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.640+0000 7f7ab2744800 1 -- 172.21.15.50:0/1402748333 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576dacd90 con 0x562576cafab0 2024-05-08T18:05:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.640+0000 7f7ab0a3d640 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562576cafab0 0x562576dac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.641+0000 7f7aafa3b640 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f799803bba0 0x7f799803e060 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7a98001e30 tx=0x7f7a98001390 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.641+0000 7f7ab0a3d640 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562576cafab0 0x562576dac850 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.641+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/1402748333 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576cafab0 2024-05-08T18:05:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.641+0000 7f7ab2744800 1 -- 172.21.15.50:0/1402748333 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576dad660 con 0x562576cafab0 2024-05-08T18:05:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.641+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/1402748333 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576cafab0 2024-05-08T18:05:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.641+0000 7f7ab2744800 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562576daefb0 0x562576db1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.641+0000 7f7ab2744800 1 -- 172.21.15.50:0/1402748333 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576db1bd0 con 0x562576daefb0 2024-05-08T18:05:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.642+0000 7f7ab023c640 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562576daefb0 0x562576db1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.642+0000 7f7ab023c640 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562576daefb0 0x562576db1440 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.642+0000 7f7ab023c640 1 -- 172.21.15.50:0/1402748333 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7aa0075030 con 0x562576daefb0 2024-05-08T18:05:05.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.642+0000 7f7ab2744800 1 -- 172.21.15.50:0/1402748333 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576dadeb0 con 0x562576cafab0 2024-05-08T18:05:05.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.642+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/1402748333 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576cafab0 2024-05-08T18:05:05.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.643+0000 7f7ab2744800 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562576db3030 0x562576db5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.643+0000 7f7ab2744800 1 -- 172.21.15.50:0/1402748333 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576db5b60 con 0x562576db3030 2024-05-08T18:05:05.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.643+0000 7f7aafa3b640 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562576db3030 0x562576db5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.643+0000 7f7aafa3b640 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562576db3030 0x562576db5420 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.643+0000 7f7aafa3b640 1 -- 172.21.15.50:0/1402748333 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7a9800bd50 con 0x562576db3030 2024-05-08T18:05:05.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.643+0000 7f7ab2744800 1 Processor -- start 2024-05-08T18:05:05.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.644+0000 7f7ab2744800 1 -- start start 2024-05-08T18:05:05.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.644+0000 7f7ab2744800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cacd40 0x562576cab1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.644+0000 7f7ab2744800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562576b9e330 con 0x562576cacd40 2024-05-08T18:05:05.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.644+0000 7f7ab023c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cacd40 0x562576cab1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.644+0000 7f7ab023c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cacd40 0x562576cab1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36914/0 (socket says 172.21.15.50:36914) 2024-05-08T18:05:05.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.644+0000 7f7ab023c640 1 -- 172.21.15.50:0/1152202824 learned_addr learned my addr 172.21.15.50:0/1152202824 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.644+0000 7f7ab023c640 1 -- 172.21.15.50:0/1152202824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562576cab6f0 con 0x562576cacd40 2024-05-08T18:05:05.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.644+0000 7f7ab023c640 1 --2- 172.21.15.50:0/1152202824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cacd40 0x562576cab1b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0092350 tx=0x7f7aa006e6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.644+0000 7f7995ffb640 1 -- 172.21.15.50:0/1152202824 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa0066b70 con 0x562576cacd40 2024-05-08T18:05:05.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.644+0000 7f7995ffb640 1 -- 172.21.15.50:0/1152202824 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa0066d10 con 0x562576cacd40 2024-05-08T18:05:05.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.644+0000 7f7ab2744800 1 -- 172.21.15.50:0/1152202824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562576cab920 con 0x562576cacd40 2024-05-08T18:05:05.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.644+0000 7f7995ffb640 1 -- 172.21.15.50:0/1152202824 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa00774e0 con 0x562576cacd40 2024-05-08T18:05:05.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.645+0000 7f7ab2744800 1 -- 172.21.15.50:0/1152202824 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562576cabe20 con 0x562576cacd40 2024-05-08T18:05:05.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.645+0000 7f7995ffb640 1 -- 172.21.15.50:0/1152202824 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7aa0077680 con 0x562576cacd40 2024-05-08T18:05:05.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.645+0000 7f7995ffb640 1 --2- 172.21.15.50:0/1152202824 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f798803bb50 0x7f798803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.646+0000 7f7995ffb640 1 -- 172.21.15.50:0/1152202824 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7aa008a070 con 0x562576cacd40 2024-05-08T18:05:05.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.646+0000 7f7aafa3b640 1 --2- 172.21.15.50:0/1152202824 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f798803bb50 0x7f798803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.646+0000 7f7ab2744800 1 Processor -- start 2024-05-08T18:05:05.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.646+0000 7f7aafa3b640 1 --2- 172.21.15.50:0/1152202824 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f798803bb50 0x7f798803e010 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x562576da9e60 tx=0x7f7a9800f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.646+0000 7f7ab2744800 1 -- start start 2024-05-08T18:05:05.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.646+0000 7f7ab2744800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576dd3130 0x562576dd3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.646+0000 7f7ab2744800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562576d9e3e0 con 0x562576dd3130 2024-05-08T18:05:05.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.646+0000 7f7ab023c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576dd3130 0x562576dd3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.646+0000 7f7ab023c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576dd3130 0x562576dd3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36928/0 (socket says 172.21.15.50:36928) 2024-05-08T18:05:05.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.646+0000 7f7ab023c640 1 -- 172.21.15.50:0/1183986169 learned_addr learned my addr 172.21.15.50:0/1183986169 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.647+0000 7f7ab023c640 1 -- 172.21.15.50:0/1183986169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562576dd3b20 con 0x562576dd3130 2024-05-08T18:05:05.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.647+0000 7f7ab023c640 1 --2- 172.21.15.50:0/1183986169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576dd3130 0x562576dd3550 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0092fd0 tx=0x7f7aa00779f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37eb07feef08895e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.647+0000 7f798f7fe640 1 -- 172.21.15.50:0/1183986169 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa00a7070 con 0x562576dd3130 2024-05-08T18:05:05.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.647+0000 7f798f7fe640 1 -- 172.21.15.50:0/1183986169 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa008d7a0 con 0x562576dd3130 2024-05-08T18:05:05.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.647+0000 7f798f7fe640 1 -- 172.21.15.50:0/1183986169 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa0079c80 con 0x562576dd3130 2024-05-08T18:05:05.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.647+0000 7f7ab2744800 1 -- 172.21.15.50:0/1183986169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576dd3130 msgr2=0x562576dd3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.647+0000 7f7ab2744800 1 --2- 172.21.15.50:0/1183986169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576dd3130 0x562576dd3550 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0092fd0 tx=0x7f7aa00779f0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.647+0000 7f7ab2744800 1 -- 172.21.15.50:0/1183986169 shutdown_connections 2024-05-08T18:05:05.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.647+0000 7f7ab2744800 1 --2- 172.21.15.50:0/1183986169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576dd3130 0x562576dd3550 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.647+0000 7f7ab2744800 1 -- 172.21.15.50:0/1183986169 >> 172.21.15.50:0/1183986169 conn(0x562576dcf440 msgr2=0x562576dcd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.647+0000 7f7ab2744800 1 -- 172.21.15.50:0/1183986169 shutdown_connections 2024-05-08T18:05:05.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.647+0000 7f7ab2744800 1 -- 172.21.15.50:0/1183986169 wait complete. 2024-05-08T18:05:05.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.648+0000 7f7ab2744800 1 Processor -- start 2024-05-08T18:05:05.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.648+0000 7f7ab2744800 1 -- start start 2024-05-08T18:05:05.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.648+0000 7f7ab2744800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576dd3130 0x562576dda490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.648+0000 7f7ab2744800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562576dd4230 con 0x562576dd3130 2024-05-08T18:05:05.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.648+0000 7f7ab023c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576dd3130 0x562576dda490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.648+0000 7f7ab023c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576dd3130 0x562576dda490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36932/0 (socket says 172.21.15.50:36932) 2024-05-08T18:05:05.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.648+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 learned_addr learned my addr 172.21.15.50:0/837624965 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.648+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562576dda9d0 con 0x562576dd3130 2024-05-08T18:05:05.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.649+0000 7f7ab023c640 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576dd3130 0x562576dda490 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0083cd0 tx=0x7f7aa005dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.649+0000 7f798dffb640 1 -- 172.21.15.50:0/837624965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa00bb070 con 0x562576dd3130 2024-05-08T18:05:05.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.649+0000 7f798dffb640 1 -- 172.21.15.50:0/837624965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa005c900 con 0x562576dd3130 2024-05-08T18:05:05.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.649+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562576dda9d0 con 0x562576dd3130 2024-05-08T18:05:05.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.649+0000 7f798dffb640 1 -- 172.21.15.50:0/837624965 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7aa0084a10 con 0x562576dd3130 2024-05-08T18:05:05.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.649+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562576dda9d0 con 0x562576dd3130 2024-05-08T18:05:05.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.650+0000 7f798dffb640 1 -- 172.21.15.50:0/837624965 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7aa00be070 con 0x562576dd3130 2024-05-08T18:05:05.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.650+0000 7f798dffb640 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f798003bba0 0x7f798003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.650+0000 7f798dffb640 1 -- 172.21.15.50:0/837624965 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7aa00b3050 con 0x562576dd3130 2024-05-08T18:05:05.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.650+0000 7f7aafa3b640 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f798003bba0 0x7f798003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.650+0000 7f7ab2744800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.650+0000 7f7ab2744800 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562576de5b60 0x562576de8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.650+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576de8770 con 0x562576de5b60 2024-05-08T18:05:05.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.650+0000 7f7ab0a3d640 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562576de5b60 0x562576de8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.650+0000 7f7aafa3b640 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f798003bba0 0x7f798003e060 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x562576db5f40 tx=0x7f7a98010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.651+0000 7f7ab0a3d640 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562576de5b60 0x562576de8020 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.651+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576de5b60 2024-05-08T18:05:05.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.651+0000 7f7ab2744800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.651+0000 7f7ab2744800 20 realm 2024-05-08T18:05:05.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.651+0000 7f7ab2744800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.651+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576de8b80 con 0x562576de5b60 2024-05-08T18:05:05.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.651+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576de5b60 2024-05-08T18:05:05.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.651+0000 7f7ab2744800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.651+0000 7f7ab2744800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:05.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.651+0000 7f7ab2744800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.651+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576de9420 con 0x562576de5b60 2024-05-08T18:05:05.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.651+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576de5b60 2024-05-08T18:05:05.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.651+0000 7f7ab2744800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.651+0000 7f7ab2744800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.651+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576de5b60 2024-05-08T18:05:05.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.652+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576de5b60 2024-05-08T18:05:05.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.652+0000 7f7ab2744800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:05.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.652+0000 7f7ab2744800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.652+0000 7f7ab2744800 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562576dea520 0x562576dec910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.652+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576dece50 con 0x562576dea520 2024-05-08T18:05:05.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.652+0000 7f7ab023c640 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562576dea520 0x562576dec910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.652+0000 7f7ab023c640 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562576dea520 0x562576dec910 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.652+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.652+0000 7f7ab2744800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:05.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.652+0000 7f7ab2744800 20 searching for the correct realm 2024-05-08T18:05:05.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.652+0000 7f7ab2744800 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562576deeed0 0x562576df1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.652+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576df1a60 con 0x562576deeed0 2024-05-08T18:05:05.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.653+0000 7f7aafa3b640 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562576deeed0 0x562576df1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.653+0000 7f7aafa3b640 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562576deeed0 0x562576df1390 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.653+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a98037040 con 0x562576deeed0 2024-05-08T18:05:05.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.653+0000 7f7aafa3b640 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7a9800f070 0x7f7a9802ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.653+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576df1a60 con 0x7f7a9800f070 2024-05-08T18:05:05.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.653+0000 7f7ab0a3d640 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7a9800f070 0x7f7a9802ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.653+0000 7f7ab0a3d640 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7a9800f070 0x7f7a9802ea80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.654+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa800a4a0 con 0x7f7a9800f070 2024-05-08T18:05:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.654+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576dea520 2024-05-08T18:05:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.654+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.654+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576de5b60 2024-05-08T18:05:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.654+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576de5b60 2024-05-08T18:05:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.654+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576de5b60 2024-05-08T18:05:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.655+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576de5b60 2024-05-08T18:05:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.655+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576dea520 2024-05-08T18:05:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.655+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.655+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576dea520 2024-05-08T18:05:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.655+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.655+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x7f7a9800f070 2024-05-08T18:05:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.655+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa800a4a0 con 0x7f7a9800f070 2024-05-08T18:05:05.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.655+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576deeed0 2024-05-08T18:05:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.656+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a98037040 con 0x562576deeed0 2024-05-08T18:05:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.656+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576de5b60 2024-05-08T18:05:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.656+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576de5b60 2024-05-08T18:05:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.656+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576dea520 2024-05-08T18:05:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.656+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.656+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576de5b60 2024-05-08T18:05:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.656+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576de5b60 2024-05-08T18:05:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.656+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x7f7a9800f070 2024-05-08T18:05:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.657+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7aa800a4a0 con 0x7f7a9800f070 2024-05-08T18:05:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.657+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576deeed0 2024-05-08T18:05:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.657+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a98037040 con 0x562576deeed0 2024-05-08T18:05:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.657+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576deeed0 2024-05-08T18:05:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.657+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a98037040 con 0x562576deeed0 2024-05-08T18:05:05.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.657+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576dea520 2024-05-08T18:05:05.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.657+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.657+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576deeed0 2024-05-08T18:05:05.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.657+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a98037040 con 0x562576deeed0 2024-05-08T18:05:05.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.657+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576deeed0 2024-05-08T18:05:05.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.658+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7a98037040 con 0x562576deeed0 2024-05-08T18:05:05.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.658+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576de5b60 2024-05-08T18:05:05.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.658+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576de5b60 2024-05-08T18:05:05.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.658+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576de5b60 2024-05-08T18:05:05.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.658+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576de5b60 2024-05-08T18:05:05.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.658+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576dea520 2024-05-08T18:05:05.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.658+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.659+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576deeed0 2024-05-08T18:05:05.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.659+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a98037040 con 0x562576deeed0 2024-05-08T18:05:05.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.659+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x7f7a9800f070 2024-05-08T18:05:05.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.659+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa800a4a0 con 0x7f7a9800f070 2024-05-08T18:05:05.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.659+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576de5b60 2024-05-08T18:05:05.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.659+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576de5b60 2024-05-08T18:05:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.659+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576dea520 2024-05-08T18:05:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.660+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.660+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576deeed0 2024-05-08T18:05:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.660+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a98037040 con 0x562576deeed0 2024-05-08T18:05:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.660+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576dea520 2024-05-08T18:05:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.660+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.660+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x7f7a9800f070 2024-05-08T18:05:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.660+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa800a4a0 con 0x7f7a9800f070 2024-05-08T18:05:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.660+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576deeed0 2024-05-08T18:05:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.661+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a98037040 con 0x562576deeed0 2024-05-08T18:05:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.661+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576dea520 2024-05-08T18:05:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.661+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.661+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x562576de5b60 2024-05-08T18:05:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.661+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576de5b60 2024-05-08T18:05:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.661+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562576de97c0 con 0x7f7a9800f070 2024-05-08T18:05:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.661+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7aa800a4a0 con 0x7f7a9800f070 2024-05-08T18:05:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.661+0000 7f7ab2744800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.661+0000 7f7ab2744800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.661+0000 7f7ab2744800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.661+0000 7f7ab2744800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.661+0000 7f7ab2744800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.661+0000 7f7ab2744800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.661+0000 7f7ab2744800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.661+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576ded7d0 con 0x562576de5b60 2024-05-08T18:05:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576de5b60 2024-05-08T18:05:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab2744800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab2744800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576ded7d0 con 0x562576de5b60 2024-05-08T18:05:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7aa800a4a0 con 0x562576de5b60 2024-05-08T18:05:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab2744800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab2744800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576ded7a0 con 0x562576deeed0 2024-05-08T18:05:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7a98037040 con 0x562576deeed0 2024-05-08T18:05:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab2744800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab2744800 20 zone default found 2024-05-08T18:05:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab2744800 4 Realm: () 2024-05-08T18:05:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab2744800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab2744800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab2744800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab2744800 20 zonegroup default 2024-05-08T18:05:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab2744800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.662+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576dee070 con 0x562576dea520 2024-05-08T18:05:05.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.663+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.663+0000 7f7ab2744800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.663+0000 7f7ab2744800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.663+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576df38c0 con 0x562576deeed0 2024-05-08T18:05:05.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.663+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7a98037040 con 0x562576deeed0 2024-05-08T18:05:05.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.663+0000 7f7ab2744800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.663+0000 7f7ab2744800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.663+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576df3c60 con 0x562576deeed0 2024-05-08T18:05:05.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.663+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7a98037040 con 0x562576deeed0 2024-05-08T18:05:05.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.663+0000 7f7ab2744800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.663+0000 7f7ab2744800 20 started sync module instance, tier type = 2024-05-08T18:05:05.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.663+0000 7f7ab2744800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:05.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.663+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df2d60 con 0x7f7a9800f070 2024-05-08T18:05:05.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.664+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa800a4a0 con 0x7f7a9800f070 2024-05-08T18:05:05.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.664+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94718908117632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df2d60 con 0x7f7a9800f070 2024-05-08T18:05:05.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.664+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df4660 con 0x562576de5b60 2024-05-08T18:05:05.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.665+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94718908117632] v27'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa8040040 con 0x7f7a9800f070 2024-05-08T18:05:05.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.665+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa8040040 con 0x562576de5b60 2024-05-08T18:05:05.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.665+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94718908121456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df2d60 con 0x562576de5b60 2024-05-08T18:05:05.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.666+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df5110 con 0x562576deeed0 2024-05-08T18:05:05.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.666+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94718908121456] v27'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa8040040 con 0x562576de5b60 2024-05-08T18:05:05.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.667+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a98037040 con 0x562576deeed0 2024-05-08T18:05:05.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.667+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94718908115120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df2cd0 con 0x562576deeed0 2024-05-08T18:05:05.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.667+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df5fd0 con 0x7f7a9800f070 2024-05-08T18:05:05.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.668+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94718908115120] v27'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a98037040 con 0x562576deeed0 2024-05-08T18:05:05.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.668+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa8040040 con 0x7f7a9800f070 2024-05-08T18:05:05.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.668+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94718908128032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df5f40 con 0x7f7a9800f070 2024-05-08T18:05:05.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.668+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df6d90 con 0x562576de5b60 2024-05-08T18:05:05.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.669+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa8040040 con 0x562576de5b60 2024-05-08T18:05:05.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.669+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94718908131440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df6d90 con 0x562576de5b60 2024-05-08T18:05:05.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.669+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df7cd0 con 0x562576dea520 2024-05-08T18:05:05.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.669+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94718908128032] v27'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa8040040 con 0x7f7a9800f070 2024-05-08T18:05:05.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.670+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94718908131440] v27'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa8040040 con 0x562576de5b60 2024-05-08T18:05:05.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.670+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.670+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94718908133760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df6390 con 0x562576dea520 2024-05-08T18:05:05.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.670+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df8570 con 0x562576dea520 2024-05-08T18:05:05.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.671+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94718908133760] v27'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.671+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.671+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94718908135328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df8dc0 con 0x562576dea520 2024-05-08T18:05:05.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.671+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df9650 con 0x562576dea520 2024-05-08T18:05:05.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.672+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94718908135328] v27'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.672+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.672+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94718908139792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df98d0 con 0x562576dea520 2024-05-08T18:05:05.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.672+0000 7f7ab2744800 20 add_watcher() i=0 2024-05-08T18:05:05.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.672+0000 7f7ab2744800 20 add_watcher() i=1 2024-05-08T18:05:05.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.672+0000 7f7ab2744800 20 add_watcher() i=2 2024-05-08T18:05:05.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.672+0000 7f7ab2744800 20 add_watcher() i=3 2024-05-08T18:05:05.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.672+0000 7f7ab2744800 20 add_watcher() i=4 2024-05-08T18:05:05.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.672+0000 7f7ab2744800 20 add_watcher() i=5 2024-05-08T18:05:05.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.672+0000 7f7ab2744800 20 add_watcher() i=6 2024-05-08T18:05:05.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.673+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94718908139792] v27'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.673+0000 7f7ab2744800 20 add_watcher() i=7 2024-05-08T18:05:05.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.673+0000 7f7ab2744800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:05.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.673+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576dfce40 con 0x562576de5b60 2024-05-08T18:05:05.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.673+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7aa8040040 con 0x562576de5b60 2024-05-08T18:05:05.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.673+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94718908142224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576dfa300 con 0x562576de5b60 2024-05-08T18:05:05.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.674+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94718908142224] v27'76 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7aa8040040 con 0x562576de5b60 2024-05-08T18:05:05.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.674+0000 7f79667e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:05.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.674+0000 7f7ab2744800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:05.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.674+0000 7f7ab2744800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:05.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.674+0000 7f7ab2744800 5 note: GC not initialized 2024-05-08T18:05:05.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.675+0000 7f79657e2640 20 reqs_thread_entry: start 2024-05-08T18:05:05.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.675+0000 7f7ab2744800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:05.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.675+0000 7f7ab2744800 20 Filter name: none 2024-05-08T18:05:05.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.675+0000 7f78d6ffd640 20 reqs_thread_entry: start 2024-05-08T18:05:05.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.675+0000 7f7ab2744800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.675+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562576e2fee0 con 0x562576de5b60 2024-05-08T18:05:05.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.675+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv7 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7f7aa8040040 con 0x562576de5b60 2024-05-08T18:05:05.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.675+0000 7f7ab2744800 20 rados_obj.operate() r=0 bl.length=556 2024-05-08T18:05:05.684 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:05.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.676+0000 7f79667e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:05.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.676+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94718908142224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576e2eab0 con 0x562576de5b60 2024-05-08T18:05:05.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.677+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94718908142224] v27'77 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7aa8040040 con 0x562576de5b60 2024-05-08T18:05:05.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.677+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94718908117632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df9290 con 0x7f7a9800f070 2024-05-08T18:05:05.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.678+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94718908117632] v27'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa8040040 con 0x7f7a9800f070 2024-05-08T18:05:05.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.678+0000 7f7ab2744800 20 remove_watcher() i=0 2024-05-08T18:05:05.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.678+0000 7f7ab2744800 2 removed watcher, disabling cache 2024-05-08T18:05:05.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.678+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94718908121456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df3260 con 0x562576de5b60 2024-05-08T18:05:05.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.679+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94718908121456] v27'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa8040040 con 0x562576de5b60 2024-05-08T18:05:05.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.679+0000 7f7ab2744800 20 remove_watcher() i=1 2024-05-08T18:05:05.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.679+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94718908115120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df4170 con 0x562576deeed0 2024-05-08T18:05:05.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.680+0000 7f7aafa3b640 1 -- 172.21.15.50:0/837624965 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94718908115120] v27'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a98037040 con 0x562576deeed0 2024-05-08T18:05:05.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.680+0000 7f7ab2744800 20 remove_watcher() i=2 2024-05-08T18:05:05.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.680+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94718908128032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df28b0 con 0x7f7a9800f070 2024-05-08T18:05:05.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.681+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94718908128032] v27'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa8040040 con 0x7f7a9800f070 2024-05-08T18:05:05.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.681+0000 7f7ab2744800 20 remove_watcher() i=3 2024-05-08T18:05:05.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.681+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94718908131440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df28b0 con 0x562576de5b60 2024-05-08T18:05:05.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.682+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94718908131440] v27'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa8040040 con 0x562576de5b60 2024-05-08T18:05:05.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.682+0000 7f7ab2744800 20 remove_watcher() i=4 2024-05-08T18:05:05.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.682+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94718908133760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df28b0 con 0x562576dea520 2024-05-08T18:05:05.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.683+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94718908133760] v27'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.683+0000 7f7ab2744800 20 remove_watcher() i=5 2024-05-08T18:05:05.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.683+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94718908135328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df7180 con 0x562576dea520 2024-05-08T18:05:05.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.684+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94718908135328] v27'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.684+0000 7f7ab2744800 20 remove_watcher() i=6 2024-05-08T18:05:05.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.684+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94718908139792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562576df28b0 con 0x562576dea520 2024-05-08T18:05:05.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.685+0000 7f7ab023c640 1 -- 172.21.15.50:0/837624965 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94718908139792] v27'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7aa00bd030 con 0x562576dea520 2024-05-08T18:05:05.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.685+0000 7f7ab2744800 20 remove_watcher() i=7 2024-05-08T18:05:05.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.685+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562576dea520 msgr2=0x562576dec910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.685+0000 7f7ab2744800 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562576dea520 0x562576dec910 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.685+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562576de5b60 msgr2=0x562576de8020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.685+0000 7f7ab2744800 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562576de5b60 0x562576de8020 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.685+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7a9800f070 msgr2=0x7f7a9802ea80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.685+0000 7f7ab2744800 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7a9800f070 0x7f7a9802ea80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.685+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562576deeed0 msgr2=0x562576df1390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.685+0000 7f7ab2744800 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562576deeed0 0x562576df1390 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.685+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f798003bba0 msgr2=0x7f798003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.685+0000 7f7ab2744800 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f798003bba0 0x7f798003e060 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x562576db5f40 tx=0x7f7a98010000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.685+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576dd3130 msgr2=0x562576dda490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.685+0000 7f7ab2744800 1 --2- 172.21.15.50:0/837624965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576dd3130 0x562576dda490 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0083cd0 tx=0x7f7aa005dc90 comp rx=0 tx=0).stop 2024-05-08T18:05:05.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.685+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 reap_dead start 2024-05-08T18:05:05.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/837624965 reap_dead start 2024-05-08T18:05:05.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 shutdown_connections 2024-05-08T18:05:05.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 >> 172.21.15.50:0/837624965 conn(0x562576dcf440 msgr2=0x562576dccaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 shutdown_connections 2024-05-08T18:05:05.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 -- 172.21.15.50:0/837624965 wait complete. 2024-05-08T18:05:05.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 -- 172.21.15.50:0/1152202824 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f798803bb50 msgr2=0x7f798803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 --2- 172.21.15.50:0/1152202824 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f798803bb50 0x7f798803e010 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x562576da9e60 tx=0x7f7a9800f000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 -- 172.21.15.50:0/1152202824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cacd40 msgr2=0x562576cab1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 --2- 172.21.15.50:0/1152202824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cacd40 0x562576cab1b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0092350 tx=0x7f7aa006e6f0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 -- 172.21.15.50:0/1152202824 shutdown_connections 2024-05-08T18:05:05.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 --2- 172.21.15.50:0/1152202824 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f798803bb50 0x7f798803e010 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 --2- 172.21.15.50:0/1152202824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cacd40 0x562576cab1b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 -- 172.21.15.50:0/1152202824 >> 172.21.15.50:0/1152202824 conn(0x562576db7f30 msgr2=0x562576db8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 -- 172.21.15.50:0/1152202824 shutdown_connections 2024-05-08T18:05:05.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 -- 172.21.15.50:0/1152202824 wait complete. 2024-05-08T18:05:05.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 -- 172.21.15.50:0/1402748333 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562576daefb0 msgr2=0x562576db1440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562576daefb0 0x562576db1440 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 -- 172.21.15.50:0/1402748333 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562576cafab0 msgr2=0x562576dac850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562576cafab0 0x562576dac850 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 -- 172.21.15.50:0/1402748333 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562576db3030 msgr2=0x562576db5420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562576db3030 0x562576db5420 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 -- 172.21.15.50:0/1402748333 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f799803bba0 msgr2=0x7f799803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.686+0000 7f7ab2744800 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f799803bba0 0x7f799803e060 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7a98001e30 tx=0x7f7a98001390 comp rx=0 tx=0).stop 2024-05-08T18:05:05.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.687+0000 7f7ab2744800 1 -- 172.21.15.50:0/1402748333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 msgr2=0x562576da5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.687+0000 7f7ab2744800 1 --2- 172.21.15.50:0/1402748333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562576cad180 0x562576da5660 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0052540 tx=0x7f7aa0003020 comp rx=0 tx=0).stop 2024-05-08T18:05:05.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.687+0000 7f7ab0a3d640 1 -- 172.21.15.50:0/1402748333 reap_dead start 2024-05-08T18:05:05.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.687+0000 7f7ab2744800 1 -- 172.21.15.50:0/1402748333 shutdown_connections 2024-05-08T18:05:05.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.687+0000 7f7ab2744800 1 -- 172.21.15.50:0/1402748333 >> 172.21.15.50:0/1402748333 conn(0x562576ca6ad0 msgr2=0x562576d97c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.687+0000 7f7ab2744800 1 -- 172.21.15.50:0/1402748333 shutdown_connections 2024-05-08T18:05:05.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.687+0000 7f7ab2744800 1 -- 172.21.15.50:0/1402748333 wait complete. 2024-05-08T18:05:05.695 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:05.695 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2024-05-08T18:05:05.695 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2024-05-08T18:05:05.695 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo2 --secret ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy --key-type swift 2024-05-08T18:05:05.745 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:05.745 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:05.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.748+0000 7f4d5132d800 1 Processor -- start 2024-05-08T18:05:05.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.748+0000 7f4d5132d800 1 -- start start 2024-05-08T18:05:05.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.748+0000 7f4d5132d800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ecc05ebcd0 0x55ecc05ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.748+0000 7f4d5132d800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55ecc05ee600 con 0x55ecc05e81c0 2024-05-08T18:05:05.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.748+0000 7f4d5132d800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55ecc0487e20 con 0x55ecc05ebcd0 2024-05-08T18:05:05.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.749+0000 7f4d47fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ecc05ebcd0 0x55ecc05ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.749+0000 7f4d47fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ecc05ebcd0 0x55ecc05ee0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36938/0 (socket says 172.21.15.50:36938) 2024-05-08T18:05:05.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.749+0000 7f4d47fff640 1 -- 172.21.15.50:0/3764820658 learned_addr learned my addr 172.21.15.50:0/3764820658 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.749+0000 7f4d47fff640 1 -- 172.21.15.50:0/3764820658 >> v1:172.21.15.46:6789/0 conn(0x55ecc05e81c0 legacy=0x55ecc05ea5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.749+0000 7f4d47fff640 1 -- 172.21.15.50:0/3764820658 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc05ef7e0 con 0x55ecc05ebcd0 2024-05-08T18:05:05.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.751+0000 7f4d47fff640 1 --2- 172.21.15.50:0/3764820658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05ebcd0 0x55ecc05ee0c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800a520 tx=0x7f4d3801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c00aee991dfd16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.751+0000 7f4d477fe640 1 -- 172.21.15.50:0/3764820658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d38057070 con 0x55ecc05ebcd0 2024-05-08T18:05:05.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.751+0000 7f4d477fe640 1 -- 172.21.15.50:0/3764820658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d38052570 con 0x55ecc05ebcd0 2024-05-08T18:05:05.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.751+0000 7f4d477fe640 1 -- 172.21.15.50:0/3764820658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d3804ae60 con 0x55ecc05ebcd0 2024-05-08T18:05:05.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.751+0000 7f4d5132d800 1 -- 172.21.15.50:0/3764820658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05ebcd0 msgr2=0x55ecc05ee0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.751+0000 7f4d5132d800 1 --2- 172.21.15.50:0/3764820658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05ebcd0 0x55ecc05ee0c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800a520 tx=0x7f4d3801d010 comp rx=0 tx=0).stop 2024-05-08T18:05:05.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.751+0000 7f4d5132d800 1 -- 172.21.15.50:0/3764820658 shutdown_connections 2024-05-08T18:05:05.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.751+0000 7f4d5132d800 1 --2- 172.21.15.50:0/3764820658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05ebcd0 0x55ecc05ee0c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.751+0000 7f4d5132d800 1 -- 172.21.15.50:0/3764820658 >> 172.21.15.50:0/3764820658 conn(0x55ecc05e1af0 msgr2=0x55ecc05e3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.751+0000 7f4d5132d800 1 -- 172.21.15.50:0/3764820658 shutdown_connections 2024-05-08T18:05:05.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.751+0000 7f4d5132d800 1 -- 172.21.15.50:0/3764820658 wait complete. 2024-05-08T18:05:05.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.767+0000 7f4d5132d800 1 Processor -- start 2024-05-08T18:05:05.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.767+0000 7f4d5132d800 1 -- start start 2024-05-08T18:05:05.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.767+0000 7f4d5132d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05e81c0 0x55ecc05e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.767+0000 7f4d5132d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ecc0532a30 con 0x55ecc05e81c0 2024-05-08T18:05:05.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.767+0000 7f4d4cf78640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05e81c0 0x55ecc05e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.767+0000 7f4d4cf78640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05e81c0 0x55ecc05e7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36948/0 (socket says 172.21.15.50:36948) 2024-05-08T18:05:05.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.767+0000 7f4d4cf78640 1 -- 172.21.15.50:0/3426572581 learned_addr learned my addr 172.21.15.50:0/3426572581 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.767+0000 7f4d4cf78640 1 -- 172.21.15.50:0/3426572581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc05e60c0 con 0x55ecc05e81c0 2024-05-08T18:05:05.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.767+0000 7f4d4cf78640 1 --2- 172.21.15.50:0/3426572581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05e81c0 0x55ecc05e7b00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4d40001d70 tx=0x7f4d4000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e8376a4b7e69e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.768+0000 7f4c3ae05640 1 -- 172.21.15.50:0/3426572581 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d4000be60 con 0x55ecc05e81c0 2024-05-08T18:05:05.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.768+0000 7f4c3ae05640 1 -- 172.21.15.50:0/3426572581 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d4000f400 con 0x55ecc05e81c0 2024-05-08T18:05:05.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.768+0000 7f4c3ae05640 1 -- 172.21.15.50:0/3426572581 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d4000f720 con 0x55ecc05e81c0 2024-05-08T18:05:05.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.768+0000 7f4d5132d800 1 -- 172.21.15.50:0/3426572581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05e81c0 msgr2=0x55ecc05e7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.768+0000 7f4d5132d800 1 --2- 172.21.15.50:0/3426572581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05e81c0 0x55ecc05e7b00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4d40001d70 tx=0x7f4d4000b5b0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.768+0000 7f4d5132d800 1 -- 172.21.15.50:0/3426572581 shutdown_connections 2024-05-08T18:05:05.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.768+0000 7f4d5132d800 1 --2- 172.21.15.50:0/3426572581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05e81c0 0x55ecc05e7b00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.768+0000 7f4d5132d800 1 -- 172.21.15.50:0/3426572581 >> 172.21.15.50:0/3426572581 conn(0x55ecc05e1af0 msgr2=0x55ecc06d4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.768+0000 7f4d5132d800 1 -- 172.21.15.50:0/3426572581 shutdown_connections 2024-05-08T18:05:05.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.768+0000 7f4d5132d800 1 -- 172.21.15.50:0/3426572581 wait complete. 2024-05-08T18:05:05.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.769+0000 7f4d5132d800 1 Processor -- start 2024-05-08T18:05:05.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.769+0000 7f4d5132d800 1 -- start start 2024-05-08T18:05:05.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.769+0000 7f4d5132d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05e81c0 0x55ecc06e2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.769+0000 7f4d5132d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ecc05eead0 con 0x55ecc05e81c0 2024-05-08T18:05:05.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.769+0000 7f4d4cf78640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05e81c0 0x55ecc06e2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.769+0000 7f4d4cf78640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05e81c0 0x55ecc06e2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36962/0 (socket says 172.21.15.50:36962) 2024-05-08T18:05:05.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.769+0000 7f4d4cf78640 1 -- 172.21.15.50:0/3495822586 learned_addr learned my addr 172.21.15.50:0/3495822586 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.770+0000 7f4d4cf78640 1 -- 172.21.15.50:0/3495822586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc06e2f90 con 0x55ecc05e81c0 2024-05-08T18:05:05.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.770+0000 7f4d4cf78640 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05e81c0 0x55ecc06e2a50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4d40000c00 tx=0x7f4d40000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.770+0000 7f4c39602640 1 -- 172.21.15.50:0/3495822586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d4001a5b0 con 0x55ecc05e81c0 2024-05-08T18:05:05.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.770+0000 7f4c39602640 1 -- 172.21.15.50:0/3495822586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d4001a750 con 0x55ecc05e81c0 2024-05-08T18:05:05.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.770+0000 7f4d5132d800 1 -- 172.21.15.50:0/3495822586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ecc06e3220 con 0x55ecc05e81c0 2024-05-08T18:05:05.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.770+0000 7f4c39602640 1 -- 172.21.15.50:0/3495822586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d4001aa50 con 0x55ecc05e81c0 2024-05-08T18:05:05.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.770+0000 7f4d5132d800 1 -- 172.21.15.50:0/3495822586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc06e5e80 con 0x55ecc05e81c0 2024-05-08T18:05:05.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.771+0000 7f4c39602640 1 -- 172.21.15.50:0/3495822586 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f4d40017070 con 0x55ecc05e81c0 2024-05-08T18:05:05.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.771+0000 7f4c39602640 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4c3403bba0 0x7f4c3403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.771+0000 7f4c39602640 1 -- 172.21.15.50:0/3495822586 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f4d40015020 con 0x55ecc05e81c0 2024-05-08T18:05:05.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.771+0000 7f4d47fff640 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4c3403bba0 0x7f4c3403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.771+0000 7f4d5132d800 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ecc05ebcd0 0x55ecc06e9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.771+0000 7f4d5132d800 1 -- 172.21.15.50:0/3495822586 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc06ea1d0 con 0x55ecc05ebcd0 2024-05-08T18:05:05.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.771+0000 7f4d4d779640 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ecc05ebcd0 0x55ecc06e9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.772+0000 7f4d47fff640 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4c3403bba0 0x7f4c3403e060 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d38054000 tx=0x7f4d3804e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.772+0000 7f4d4d779640 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ecc05ebcd0 0x55ecc06e9c40 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.772+0000 7f4d4d779640 1 -- 172.21.15.50:0/3495822586 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc05ebcd0 2024-05-08T18:05:05.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.772+0000 7f4d5132d800 1 -- 172.21.15.50:0/3495822586 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc06e7210 con 0x55ecc05ebcd0 2024-05-08T18:05:05.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.772+0000 7f4d4d779640 1 -- 172.21.15.50:0/3495822586 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc05ebcd0 2024-05-08T18:05:05.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.772+0000 7f4d5132d800 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ecc06ec500 0x55ecc06ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.772+0000 7f4d5132d800 1 -- 172.21.15.50:0/3495822586 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc06eef90 con 0x55ecc06ec500 2024-05-08T18:05:05.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.772+0000 7f4d4cf78640 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ecc06ec500 0x55ecc06ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.773+0000 7f4d4cf78640 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ecc06ec500 0x55ecc06ee8f0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.773+0000 7f4d4cf78640 1 -- 172.21.15.50:0/3495822586 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f4d4002e070 con 0x55ecc06ec500 2024-05-08T18:05:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.773+0000 7f4d5132d800 1 -- 172.21.15.50:0/3495822586 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc06eb3a0 con 0x55ecc05ebcd0 2024-05-08T18:05:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.773+0000 7f4d4d779640 1 -- 172.21.15.50:0/3495822586 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc05ebcd0 2024-05-08T18:05:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.773+0000 7f4d5132d800 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ecc06f03f0 0x55ecc06f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.773+0000 7f4d5132d800 1 -- 172.21.15.50:0/3495822586 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc06f2ee0 con 0x55ecc06f03f0 2024-05-08T18:05:05.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.773+0000 7f4d47fff640 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ecc06f03f0 0x55ecc06f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.774+0000 7f4d47fff640 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ecc06f03f0 0x55ecc06f27e0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.774+0000 7f4d47fff640 1 -- 172.21.15.50:0/3495822586 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4d3805a040 con 0x55ecc06f03f0 2024-05-08T18:05:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.774+0000 7f4d5132d800 1 Processor -- start 2024-05-08T18:05:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.774+0000 7f4d5132d800 1 -- start start 2024-05-08T18:05:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.775+0000 7f4d5132d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc0701250 0x55ecc0701670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.775+0000 7f4d5132d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ecc04d9330 con 0x55ecc0701250 2024-05-08T18:05:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.775+0000 7f4d4cf78640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc0701250 0x55ecc0701670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.775+0000 7f4d4cf78640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc0701250 0x55ecc0701670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36978/0 (socket says 172.21.15.50:36978) 2024-05-08T18:05:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.775+0000 7f4d4cf78640 1 -- 172.21.15.50:0/3025338392 learned_addr learned my addr 172.21.15.50:0/3025338392 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.775+0000 7f4d4cf78640 1 -- 172.21.15.50:0/3025338392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc0701bb0 con 0x55ecc0701250 2024-05-08T18:05:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.775+0000 7f4d4cf78640 1 --2- 172.21.15.50:0/3025338392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc0701250 0x55ecc0701670 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000e750 tx=0x7f4d4001cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.775+0000 7f4c327fc640 1 -- 172.21.15.50:0/3025338392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d4001b720 con 0x55ecc0701250 2024-05-08T18:05:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.775+0000 7f4c327fc640 1 -- 172.21.15.50:0/3025338392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d4001b8c0 con 0x55ecc0701250 2024-05-08T18:05:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.775+0000 7f4c327fc640 1 -- 172.21.15.50:0/3025338392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d4001bba0 con 0x55ecc0701250 2024-05-08T18:05:05.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.775+0000 7f4d5132d800 1 -- 172.21.15.50:0/3025338392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ecc0702620 con 0x55ecc0701250 2024-05-08T18:05:05.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.775+0000 7f4d5132d800 1 -- 172.21.15.50:0/3025338392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc0702620 con 0x55ecc0701250 2024-05-08T18:05:05.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.776+0000 7f4c327fc640 1 -- 172.21.15.50:0/3025338392 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f4d4000e070 con 0x55ecc0701250 2024-05-08T18:05:05.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.776+0000 7f4c327fc640 1 --2- 172.21.15.50:0/3025338392 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4c2403bba0 0x7f4c2403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.776+0000 7f4c327fc640 1 -- 172.21.15.50:0/3025338392 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f4d4003cca0 con 0x55ecc0701250 2024-05-08T18:05:05.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.776+0000 7f4d47fff640 1 --2- 172.21.15.50:0/3025338392 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4c2403bba0 0x7f4c2403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.777+0000 7f4d5132d800 1 Processor -- start 2024-05-08T18:05:05.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.777+0000 7f4d5132d800 1 -- start start 2024-05-08T18:05:05.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.777+0000 7f4d47fff640 1 --2- 172.21.15.50:0/3025338392 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4c2403bba0 0x7f4c2403e060 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55ecc06e7130 tx=0x7f4d38005ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.777+0000 7f4d5132d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc070e190 0x55ecc070e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.777+0000 7f4d5132d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ecc05e6440 con 0x55ecc070e190 2024-05-08T18:05:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.777+0000 7f4d4cf78640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc070e190 0x55ecc070e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.777+0000 7f4d4cf78640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc070e190 0x55ecc070e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36982/0 (socket says 172.21.15.50:36982) 2024-05-08T18:05:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.777+0000 7f4d4cf78640 1 -- 172.21.15.50:0/2366707158 learned_addr learned my addr 172.21.15.50:0/2366707158 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.777+0000 7f4d4cf78640 1 -- 172.21.15.50:0/2366707158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc070eb80 con 0x55ecc070e190 2024-05-08T18:05:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.778+0000 7f4d4cf78640 1 --2- 172.21.15.50:0/2366707158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc070e190 0x55ecc070e5b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4d4003cfd0 tx=0x7f4d40027df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1044d0c3eb9a60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.778+0000 7f4c23fff640 1 -- 172.21.15.50:0/2366707158 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d40019510 con 0x55ecc070e190 2024-05-08T18:05:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.778+0000 7f4c23fff640 1 -- 172.21.15.50:0/2366707158 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d40035020 con 0x55ecc070e190 2024-05-08T18:05:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.778+0000 7f4c23fff640 1 -- 172.21.15.50:0/2366707158 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d40027560 con 0x55ecc070e190 2024-05-08T18:05:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.778+0000 7f4d5132d800 1 -- 172.21.15.50:0/2366707158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc070e190 msgr2=0x55ecc070e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.778+0000 7f4d5132d800 1 --2- 172.21.15.50:0/2366707158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc070e190 0x55ecc070e5b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4d4003cfd0 tx=0x7f4d40027df0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.778+0000 7f4d5132d800 1 -- 172.21.15.50:0/2366707158 shutdown_connections 2024-05-08T18:05:05.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.778+0000 7f4d5132d800 1 --2- 172.21.15.50:0/2366707158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc070e190 0x55ecc070e5b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.778+0000 7f4d5132d800 1 -- 172.21.15.50:0/2366707158 >> 172.21.15.50:0/2366707158 conn(0x55ecc070a4a0 msgr2=0x55ecc0708500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.778+0000 7f4d5132d800 1 -- 172.21.15.50:0/2366707158 shutdown_connections 2024-05-08T18:05:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.778+0000 7f4d5132d800 1 -- 172.21.15.50:0/2366707158 wait complete. 2024-05-08T18:05:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.779+0000 7f4d5132d800 1 Processor -- start 2024-05-08T18:05:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.779+0000 7f4d5132d800 1 -- start start 2024-05-08T18:05:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.779+0000 7f4d5132d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc070e190 0x55ecc0715640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.779+0000 7f4d5132d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ecc070f210 con 0x55ecc070e190 2024-05-08T18:05:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.779+0000 7f4d4cf78640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc070e190 0x55ecc0715640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.779+0000 7f4d4cf78640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc070e190 0x55ecc0715640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:36998/0 (socket says 172.21.15.50:36998) 2024-05-08T18:05:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.779+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 learned_addr learned my addr 172.21.15.50:0/848396595 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.779+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc0715b80 con 0x55ecc070e190 2024-05-08T18:05:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.780+0000 7f4d4cf78640 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc070e190 0x55ecc0715640 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4d4003cfd0 tx=0x7f4d40044640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.780+0000 7f4c227fc640 1 -- 172.21.15.50:0/848396595 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d4004a960 con 0x55ecc070e190 2024-05-08T18:05:05.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.780+0000 7f4c227fc640 1 -- 172.21.15.50:0/848396595 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d4004ab00 con 0x55ecc070e190 2024-05-08T18:05:05.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.780+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ecc0715b80 con 0x55ecc070e190 2024-05-08T18:05:05.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.780+0000 7f4c227fc640 1 -- 172.21.15.50:0/848396595 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d400078d0 con 0x55ecc070e190 2024-05-08T18:05:05.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.780+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc0715b80 con 0x55ecc070e190 2024-05-08T18:05:05.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.781+0000 7f4c227fc640 1 -- 172.21.15.50:0/848396595 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f4d4006a070 con 0x55ecc070e190 2024-05-08T18:05:05.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.781+0000 7f4c227fc640 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4c1803bba0 0x7f4c1803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.781+0000 7f4c227fc640 1 -- 172.21.15.50:0/848396595 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f4d4006dcb0 con 0x55ecc070e190 2024-05-08T18:05:05.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.781+0000 7f4d47fff640 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4c1803bba0 0x7f4c1803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.781+0000 7f4d47fff640 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4c1803bba0 0x7f4c1803e060 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4d3801d050 tx=0x7f4d3804f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.782+0000 7f4d5132d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.782+0000 7f4d5132d800 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ecc0720bc0 0x55ecc0723080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.782+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc07237d0 con 0x55ecc0720bc0 2024-05-08T18:05:05.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.782+0000 7f4d4d779640 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ecc0720bc0 0x55ecc0723080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.782+0000 7f4d4d779640 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ecc0720bc0 0x55ecc0723080 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.782+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc0720bc0 2024-05-08T18:05:05.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.782+0000 7f4d5132d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.782+0000 7f4d5132d800 20 realm 2024-05-08T18:05:05.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.782+0000 7f4d5132d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.782+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0723be0 con 0x55ecc0720bc0 2024-05-08T18:05:05.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.783+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc0720bc0 2024-05-08T18:05:05.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.783+0000 7f4d5132d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.783+0000 7f4d5132d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:05.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.783+0000 7f4d5132d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.783+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc07244b0 con 0x55ecc0720bc0 2024-05-08T18:05:05.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.783+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc0720bc0 2024-05-08T18:05:05.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.783+0000 7f4d5132d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.783+0000 7f4d5132d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.783+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0724850 con 0x55ecc0720bc0 2024-05-08T18:05:05.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.783+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc0720bc0 2024-05-08T18:05:05.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.783+0000 7f4d5132d800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:05.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.783+0000 7f4d5132d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.783+0000 7f4d5132d800 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ecc07255b0 0x55ecc07279a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.783+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0727fb0 con 0x55ecc07255b0 2024-05-08T18:05:05.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.783+0000 7f4d4cf78640 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ecc07255b0 0x55ecc07279a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.784+0000 7f4d4cf78640 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ecc07255b0 0x55ecc07279a0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.784+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.784+0000 7f4d5132d800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:05.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.784+0000 7f4d5132d800 20 searching for the correct realm 2024-05-08T18:05:05.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.784+0000 7f4d5132d800 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ecc0729f60 0x55ecc072c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.784+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ecc072caf0 con 0x55ecc0729f60 2024-05-08T18:05:05.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.784+0000 7f4d47fff640 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ecc0729f60 0x55ecc072c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.785+0000 7f4d47fff640 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ecc0729f60 0x55ecc072c420 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.785+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d3807f070 con 0x55ecc0729f60 2024-05-08T18:05:05.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.785+0000 7f4d47fff640 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f4d38074040 0x7f4d38081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.785+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ecc072caf0 con 0x7f4d38074040 2024-05-08T18:05:05.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.785+0000 7f4d4d779640 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f4d38074040 0x7f4d38081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.785+0000 7f4d4d779640 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f4d38074040 0x7f4d38081410 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.785+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d4800a4a0 con 0x7f4d38074040 2024-05-08T18:05:05.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.786+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0724850 con 0x55ecc07255b0 2024-05-08T18:05:05.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.786+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.786+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0724850 con 0x55ecc0720bc0 2024-05-08T18:05:05.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.786+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc0720bc0 2024-05-08T18:05:05.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.786+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0724850 con 0x55ecc0720bc0 2024-05-08T18:05:05.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.786+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc0720bc0 2024-05-08T18:05:05.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.786+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0724850 con 0x55ecc07255b0 2024-05-08T18:05:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.787+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.787+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0724850 con 0x55ecc07255b0 2024-05-08T18:05:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.787+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.787+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0724850 con 0x7f4d38074040 2024-05-08T18:05:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.787+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d4800a4a0 con 0x7f4d38074040 2024-05-08T18:05:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.787+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0724850 con 0x55ecc0729f60 2024-05-08T18:05:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.787+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d3807f070 con 0x55ecc0729f60 2024-05-08T18:05:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.787+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0724850 con 0x55ecc0720bc0 2024-05-08T18:05:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.788+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc0720bc0 2024-05-08T18:05:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.788+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0724850 con 0x55ecc07255b0 2024-05-08T18:05:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.788+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.788+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0724850 con 0x55ecc0720bc0 2024-05-08T18:05:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.788+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc0720bc0 2024-05-08T18:05:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.788+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0724850 con 0x7f4d38074040 2024-05-08T18:05:05.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.788+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f4d4800a4a0 con 0x7f4d38074040 2024-05-08T18:05:05.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.788+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0724850 con 0x55ecc0729f60 2024-05-08T18:05:05.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.789+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d3807f070 con 0x55ecc0729f60 2024-05-08T18:05:05.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.789+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x55ecc0729f60 2024-05-08T18:05:05.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.789+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d3807f070 con 0x55ecc0729f60 2024-05-08T18:05:05.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.789+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x55ecc07255b0 2024-05-08T18:05:05.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.789+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.789+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x55ecc0729f60 2024-05-08T18:05:05.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.789+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d3807f070 con 0x55ecc0729f60 2024-05-08T18:05:05.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.789+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x55ecc0729f60 2024-05-08T18:05:05.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.790+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f4d3807f070 con 0x55ecc0729f60 2024-05-08T18:05:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.790+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x55ecc0720bc0 2024-05-08T18:05:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.790+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc0720bc0 2024-05-08T18:05:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.790+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x55ecc0720bc0 2024-05-08T18:05:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.790+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc0720bc0 2024-05-08T18:05:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.790+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x55ecc07255b0 2024-05-08T18:05:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.790+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.790+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x55ecc0729f60 2024-05-08T18:05:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.791+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d3807f070 con 0x55ecc0729f60 2024-05-08T18:05:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.791+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x7f4d38074040 2024-05-08T18:05:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.791+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d4800a4a0 con 0x7f4d38074040 2024-05-08T18:05:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.791+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x55ecc0720bc0 2024-05-08T18:05:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.791+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc0720bc0 2024-05-08T18:05:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.791+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x55ecc07255b0 2024-05-08T18:05:05.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.791+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.792+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x55ecc0729f60 2024-05-08T18:05:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.792+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d3807f070 con 0x55ecc0729f60 2024-05-08T18:05:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.792+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x55ecc07255b0 2024-05-08T18:05:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.792+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.792+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x7f4d38074040 2024-05-08T18:05:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.792+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d4800a4a0 con 0x7f4d38074040 2024-05-08T18:05:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.792+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x55ecc0729f60 2024-05-08T18:05:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.792+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d3807f070 con 0x55ecc0729f60 2024-05-08T18:05:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.793+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x55ecc07255b0 2024-05-08T18:05:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.793+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.793+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x55ecc0720bc0 2024-05-08T18:05:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.793+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc0720bc0 2024-05-08T18:05:05.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.793+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f4d3808e040 con 0x7f4d38074040 2024-05-08T18:05:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.793+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4d4800a4a0 con 0x7f4d38074040 2024-05-08T18:05:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.793+0000 7f4d5132d800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.793+0000 7f4d5132d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.793+0000 7f4d5132d800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.793+0000 7f4d5132d800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.793+0000 7f4d5132d800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.793+0000 7f4d5132d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.793+0000 7f4d5132d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.793+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0728870 con 0x55ecc0720bc0 2024-05-08T18:05:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc0720bc0 2024-05-08T18:05:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d5132d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d5132d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0728870 con 0x55ecc0720bc0 2024-05-08T18:05:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4d4800a4a0 con 0x55ecc0720bc0 2024-05-08T18:05:05.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d5132d800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d5132d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0728870 con 0x55ecc0729f60 2024-05-08T18:05:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f4d3807f070 con 0x55ecc0729f60 2024-05-08T18:05:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d5132d800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d5132d800 20 zone default found 2024-05-08T18:05:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d5132d800 4 Realm: () 2024-05-08T18:05:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d5132d800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d5132d800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d5132d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d5132d800 20 zonegroup default 2024-05-08T18:05:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d5132d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.794+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0729180 con 0x55ecc07255b0 2024-05-08T18:05:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.795+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.795+0000 7f4d5132d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.795+0000 7f4d5132d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.795+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc072e950 con 0x55ecc0729f60 2024-05-08T18:05:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.795+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4d3807f070 con 0x55ecc0729f60 2024-05-08T18:05:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.795+0000 7f4d5132d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.795+0000 7f4d5132d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.795+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc072edb0 con 0x55ecc0729f60 2024-05-08T18:05:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.795+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4d3807f070 con 0x55ecc0729f60 2024-05-08T18:05:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.795+0000 7f4d5132d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.795+0000 7f4d5132d800 20 started sync module instance, tier type = 2024-05-08T18:05:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.795+0000 7f4d5132d800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.795+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc072ddf0 con 0x7f4d38074040 2024-05-08T18:05:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.796+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d4800a4a0 con 0x7f4d38074040 2024-05-08T18:05:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.796+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94475329397488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc072ddf0 con 0x7f4d38074040 2024-05-08T18:05:05.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.796+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc072f8c0 con 0x55ecc0720bc0 2024-05-08T18:05:05.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.797+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94475329397488] v27'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x7f4d38074040 2024-05-08T18:05:05.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.797+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x55ecc0720bc0 2024-05-08T18:05:05.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.797+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94475329401776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc072ddf0 con 0x55ecc0720bc0 2024-05-08T18:05:05.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.797+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0730370 con 0x55ecc0729f60 2024-05-08T18:05:05.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.798+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94475329401776] v27'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x55ecc0720bc0 2024-05-08T18:05:05.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.799+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d3807f070 con 0x55ecc0729f60 2024-05-08T18:05:05.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.799+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94475329395008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc072dd60 con 0x55ecc0729f60 2024-05-08T18:05:05.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.799+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0731230 con 0x7f4d38074040 2024-05-08T18:05:05.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.799+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94475329395008] v27'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d3807f070 con 0x55ecc0729f60 2024-05-08T18:05:05.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.800+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x7f4d38074040 2024-05-08T18:05:05.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.800+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94475329408384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc07311a0 con 0x7f4d38074040 2024-05-08T18:05:05.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.800+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0731fc0 con 0x55ecc0720bc0 2024-05-08T18:05:05.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.800+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x55ecc0720bc0 2024-05-08T18:05:05.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.801+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94475329411792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0731fc0 con 0x55ecc0720bc0 2024-05-08T18:05:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.801+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0732f20 con 0x55ecc07255b0 2024-05-08T18:05:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.801+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94475329408384] v27'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x7f4d38074040 2024-05-08T18:05:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.801+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94475329411792] v27'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x55ecc0720bc0 2024-05-08T18:05:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.802+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.802+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94475329414064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0732a00 con 0x55ecc07255b0 2024-05-08T18:05:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.802+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc07337c0 con 0x55ecc07255b0 2024-05-08T18:05:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.803+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.803+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94475329414064] v27'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.803+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94475329420128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0734430 con 0x55ecc07255b0 2024-05-08T18:05:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.803+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0731620 con 0x55ecc07255b0 2024-05-08T18:05:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.803+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94475329420128] v27'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.804+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.804+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94475329421184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0734df0 con 0x55ecc07255b0 2024-05-08T18:05:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.804+0000 7f4d5132d800 20 add_watcher() i=0 2024-05-08T18:05:05.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.804+0000 7f4d5132d800 20 add_watcher() i=1 2024-05-08T18:05:05.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.804+0000 7f4d5132d800 20 add_watcher() i=2 2024-05-08T18:05:05.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.804+0000 7f4d5132d800 20 add_watcher() i=3 2024-05-08T18:05:05.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.804+0000 7f4d5132d800 20 add_watcher() i=4 2024-05-08T18:05:05.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.804+0000 7f4d5132d800 20 add_watcher() i=5 2024-05-08T18:05:05.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.804+0000 7f4d5132d800 20 add_watcher() i=6 2024-05-08T18:05:05.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.805+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94475329421184] v27'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.805+0000 7f4d5132d800 20 add_watcher() i=7 2024-05-08T18:05:05.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.805+0000 7f4d5132d800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:05.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.805+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0738040 con 0x55ecc0720bc0 2024-05-08T18:05:05.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.805+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f4d48040040 con 0x55ecc0720bc0 2024-05-08T18:05:05.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.805+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94475329424048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0735380 con 0x55ecc0720bc0 2024-05-08T18:05:05.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.806+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94475329424048] v27'78 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x55ecc0720bc0 2024-05-08T18:05:05.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.806+0000 7f4c02fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:05.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.806+0000 7f4d5132d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.806+0000 7f4d5132d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.806+0000 7f4d5132d800 5 note: GC not initialized 2024-05-08T18:05:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.807+0000 7f4c01fe3640 20 reqs_thread_entry: start 2024-05-08T18:05:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.807+0000 7f4d5132d800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.807+0000 7f4d5132d800 20 Filter name: none 2024-05-08T18:05:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.807+0000 7f4b737fe640 20 reqs_thread_entry: start 2024-05-08T18:05:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.807+0000 7f4d5132d800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.807+0000 7f4d5132d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.807+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc076b5c0 con 0x55ecc0720bc0 2024-05-08T18:05:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.807+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv7 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7f4d48040040 con 0x55ecc0720bc0 2024-05-08T18:05:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.807+0000 7f4d5132d800 20 rados_obj.operate() r=0 bl.length=556 2024-05-08T18:05:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.807+0000 7f4d5132d800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.807+0000 7f4d5132d800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:05.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.807+0000 7f4d5132d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.807+0000 7f4d5132d800 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : miss 2024-05-08T18:05:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.807+0000 7f4d5132d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.807+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:63 8.2 8:45671577:users.swift::foo%3afoo2:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc076c300 con 0x7f4d38074040 2024-05-08T18:05:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.808+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(63 foo:foo2 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 194+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x7f4d38074040 2024-05-08T18:05:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.808+0000 7f4d5132d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.808+0000 7f4d5132d800 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x0 2024-05-08T18:05:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.808+0000 7f4d5132d800 10 adding default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-05-08T18:05:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.808+0000 7f4d5132d800 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : hit (negative entry) 2024-05-08T18:05:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.808+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~673 in=673b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc07720f0 con 0x55ecc0720bc0 2024-05-08T18:05:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.809+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~673] v27'8 uv8 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x55ecc0720bc0 2024-05-08T18:05:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.809+0000 7f4d5132d800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.809+0000 7f4d5132d800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.809+0000 7f4d5132d800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-05-08T18:05:05.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.809+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94475329673968 in=861b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc076ebf0 con 0x55ecc0720bc0 2024-05-08T18:05:05.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.809+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== watch-notify(notify (1) cookie 94475329411792 notify 115964116999 ret 0) ==== 891+0+0 (crc 0 0 0) 0x7f4d48018c30 con 0x55ecc0720bc0 2024-05-08T18:05:05.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.809+0000 7f4c23fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116999 cookie 94475329411792 notifier 4382 bl.length()=849 2024-05-08T18:05:05.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.809+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(65 notify.4 [notify cookie 94475329673968 out=8b] v0'0 uv66 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f4d48040040 con 0x55ecc0720bc0 2024-05-08T18:05:05.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.809+0000 7f4c23fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:05.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.809+0000 7f4c23fff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:05.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.809+0000 7f4c23fff640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f4c10003ea0 con 0x55ecc0720bc0 2024-05-08T18:05:05.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.809+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x55ecc0720bc0 2024-05-08T18:05:05.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.809+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== watch-notify(notify_complete (2) cookie 94475329673968 notify 115964116999 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f4d48018e00 con 0x55ecc0720bc0 2024-05-08T18:05:05.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.810+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 8.2 8:45671577:users.swift::foo%3afoo2:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0771af0 con 0x7f4d38074040 2024-05-08T18:05:05.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.811+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(67 foo:foo2 [delete,create,writefull 0~7] v27'1 uv1 ondisk = 0) ==== 236+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x7f4d38074040 2024-05-08T18:05:05.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.811+0000 7f4d5132d800 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2024-05-08T18:05:05.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.811+0000 7f4d5132d800 10 moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-05-08T18:05:05.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.811+0000 7f4d5132d800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2024-05-08T18:05:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.811+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94475329673968 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ecc076c6d0 con 0x7f4d38074040 2024-05-08T18:05:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.811+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== watch-notify(notify (1) cookie 94475329397488 notify 115964116995 ret 0) ==== 208+0+0 (crc 0 0 0) 0x7f4d48018e00 con 0x7f4d38074040 2024-05-08T18:05:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.811+0000 7f4c23fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116995 cookie 94475329397488 notifier 4382 bl.length()=166 2024-05-08T18:05:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.811+0000 7f4c23fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2024-05-08T18:05:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.811+0000 7f4c23fff640 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-05-08T18:05:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.811+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(68 notify.0 [notify cookie 94475329673968 out=8b] v0'0 uv66 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f4d48040040 con 0x7f4d38074040 2024-05-08T18:05:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.811+0000 7f4c23fff640 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f4c10003ea0 con 0x7f4d38074040 2024-05-08T18:05:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.812+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== watch-notify(notify_complete (2) cookie 94475329673968 notify 115964116995 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f4d48038520 con 0x7f4d38074040 2024-05-08T18:05:05.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.812+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(69 notify.0 [notify-ack] v0'0 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x7f4d38074040 2024-05-08T18:05:05.826 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"},{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"},{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy","active":true,"create_date":"2024-05-08T18:05:05.808771Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:05.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.813+0000 7f4c02fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:05.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.813+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94475329424048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc076a8e0 con 0x55ecc0720bc0 2024-05-08T18:05:05.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.813+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94475329424048] v27'79 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x55ecc0720bc0 2024-05-08T18:05:05.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.814+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94475329397488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc076a8e0 con 0x7f4d38074040 2024-05-08T18:05:05.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.814+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94475329397488] v27'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x7f4d38074040 2024-05-08T18:05:05.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.814+0000 7f4d5132d800 20 remove_watcher() i=0 2024-05-08T18:05:05.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.814+0000 7f4d5132d800 2 removed watcher, disabling cache 2024-05-08T18:05:05.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.814+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94475329401776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc072e2f0 con 0x55ecc0720bc0 2024-05-08T18:05:05.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.815+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94475329401776] v27'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x55ecc0720bc0 2024-05-08T18:05:05.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.815+0000 7f4d5132d800 20 remove_watcher() i=1 2024-05-08T18:05:05.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.815+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94475329395008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc072e2f0 con 0x55ecc0729f60 2024-05-08T18:05:05.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.816+0000 7f4d47fff640 1 -- 172.21.15.50:0/848396595 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94475329395008] v27'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d3807f070 con 0x55ecc0729f60 2024-05-08T18:05:05.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.816+0000 7f4d5132d800 20 remove_watcher() i=2 2024-05-08T18:05:05.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.816+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94475329408384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc072e2f0 con 0x7f4d38074040 2024-05-08T18:05:05.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.817+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94475329408384] v27'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x7f4d38074040 2024-05-08T18:05:05.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.817+0000 7f4d5132d800 20 remove_watcher() i=3 2024-05-08T18:05:05.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.817+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94475329411792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0730d80 con 0x55ecc0720bc0 2024-05-08T18:05:05.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.818+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94475329411792] v27'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d48040040 con 0x55ecc0720bc0 2024-05-08T18:05:05.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.818+0000 7f4d5132d800 20 remove_watcher() i=4 2024-05-08T18:05:05.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.818+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94475329414064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc072e2f0 con 0x55ecc07255b0 2024-05-08T18:05:05.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.819+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94475329414064] v27'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.819+0000 7f4d5132d800 20 remove_watcher() i=5 2024-05-08T18:05:05.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.819+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94475329420128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc0730d80 con 0x55ecc07255b0 2024-05-08T18:05:05.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.820+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94475329420128] v27'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.820+0000 7f4d5132d800 20 remove_watcher() i=6 2024-05-08T18:05:05.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.820+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94475329421184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ecc07323b0 con 0x55ecc07255b0 2024-05-08T18:05:05.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d4cf78640 1 -- 172.21.15.50:0/848396595 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94475329421184] v27'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4d40073070 con 0x55ecc07255b0 2024-05-08T18:05:05.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 20 remove_watcher() i=7 2024-05-08T18:05:05.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ecc07255b0 msgr2=0x55ecc07279a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ecc07255b0 0x55ecc07279a0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ecc0720bc0 msgr2=0x55ecc0723080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ecc0720bc0 0x55ecc0723080 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f4d38074040 msgr2=0x7f4d38081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f4d38074040 0x7f4d38081410 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ecc0729f60 msgr2=0x55ecc072c420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ecc0729f60 0x55ecc072c420 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4c1803bba0 msgr2=0x7f4c1803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4c1803bba0 0x7f4c1803e060 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4d3801d050 tx=0x7f4d3804f000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc070e190 msgr2=0x55ecc0715640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 1 --2- 172.21.15.50:0/848396595 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc070e190 0x55ecc0715640 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4d4003cfd0 tx=0x7f4d40044640 comp rx=0 tx=0).stop 2024-05-08T18:05:05.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 reap_dead start 2024-05-08T18:05:05.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d4d779640 1 -- 172.21.15.50:0/848396595 reap_dead start 2024-05-08T18:05:05.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 shutdown_connections 2024-05-08T18:05:05.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 >> 172.21.15.50:0/848396595 conn(0x55ecc070a4a0 msgr2=0x55ecc0707b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 shutdown_connections 2024-05-08T18:05:05.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.821+0000 7f4d5132d800 1 -- 172.21.15.50:0/848396595 wait complete. 2024-05-08T18:05:05.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 -- 172.21.15.50:0/3025338392 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4c2403bba0 msgr2=0x7f4c2403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 --2- 172.21.15.50:0/3025338392 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4c2403bba0 0x7f4c2403e060 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55ecc06e7130 tx=0x7f4d38005ac0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 -- 172.21.15.50:0/3025338392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc0701250 msgr2=0x55ecc0701670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 --2- 172.21.15.50:0/3025338392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc0701250 0x55ecc0701670 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000e750 tx=0x7f4d4001cbb0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 -- 172.21.15.50:0/3025338392 shutdown_connections 2024-05-08T18:05:05.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 --2- 172.21.15.50:0/3025338392 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4c2403bba0 0x7f4c2403e060 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 --2- 172.21.15.50:0/3025338392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc0701250 0x55ecc0701670 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 -- 172.21.15.50:0/3025338392 >> 172.21.15.50:0/3025338392 conn(0x55ecc06f52f0 msgr2=0x55ecc06f56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 -- 172.21.15.50:0/3025338392 shutdown_connections 2024-05-08T18:05:05.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 -- 172.21.15.50:0/3025338392 wait complete. 2024-05-08T18:05:05.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 -- 172.21.15.50:0/3495822586 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ecc06ec500 msgr2=0x55ecc06ee8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ecc06ec500 0x55ecc06ee8f0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 -- 172.21.15.50:0/3495822586 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ecc05ebcd0 msgr2=0x55ecc06e9c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ecc05ebcd0 0x55ecc06e9c40 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 -- 172.21.15.50:0/3495822586 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ecc06f03f0 msgr2=0x55ecc06f27e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ecc06f03f0 0x55ecc06f27e0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 -- 172.21.15.50:0/3495822586 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4c3403bba0 msgr2=0x7f4c3403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f4c3403bba0 0x7f4c3403e060 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d38054000 tx=0x7f4d3804e000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 -- 172.21.15.50:0/3495822586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05e81c0 msgr2=0x55ecc06e2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 --2- 172.21.15.50:0/3495822586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecc05e81c0 0x55ecc06e2a50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4d40000c00 tx=0x7f4d40000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d4d779640 1 -- 172.21.15.50:0/3495822586 reap_dead start 2024-05-08T18:05:05.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 -- 172.21.15.50:0/3495822586 shutdown_connections 2024-05-08T18:05:05.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 -- 172.21.15.50:0/3495822586 >> 172.21.15.50:0/3495822586 conn(0x55ecc05e1af0 msgr2=0x55ecc05e69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 -- 172.21.15.50:0/3495822586 shutdown_connections 2024-05-08T18:05:05.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.822+0000 7f4d5132d800 1 -- 172.21.15.50:0/3495822586 wait complete. 2024-05-08T18:05:05.837 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True, 'create_date': '2024-05-08T18:05:05.808771Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:05.837 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-05-08T18:05:05.837 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-05-08T18:05:05.838 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-05-08T18:05:05.883 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:05.883 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:05.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.886+0000 7f16c4c19800 1 Processor -- start 2024-05-08T18:05:05.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.886+0000 7f16c4c19800 1 -- start start 2024-05-08T18:05:05.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.886+0000 7f16c4c19800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5604128b66f0 0x5604128b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.886+0000 7f16c4c19800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5604128b7190 con 0x5604128b78b0 2024-05-08T18:05:05.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.886+0000 7f16c4c19800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x560412756e20 con 0x5604128b66f0 2024-05-08T18:05:05.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.887+0000 7f16c2711640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5604128b66f0 0x5604128b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.887+0000 7f16c2711640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5604128b66f0 0x5604128b6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37012/0 (socket says 172.21.15.50:37012) 2024-05-08T18:05:05.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.887+0000 7f16c2711640 1 -- 172.21.15.50:0/4056424619 learned_addr learned my addr 172.21.15.50:0/4056424619 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.887+0000 7f16c1f10640 1 --1- 172.21.15.50:0/4056424619 >> v1:172.21.15.46:6789/0 conn(0x5604128b78b0 0x5604128b7d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43044/0 (socket says 172.21.15.50:43044) 2024-05-08T18:05:05.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.887+0000 7f16c2711640 1 -- 172.21.15.50:0/4056424619 >> v1:172.21.15.46:6789/0 conn(0x5604128b78b0 legacy=0x5604128b7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.887+0000 7f16c2711640 1 -- 172.21.15.50:0/4056424619 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604128b8470 con 0x5604128b66f0 2024-05-08T18:05:05.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.887+0000 7f16c170f640 1 -- 172.21.15.50:0/4056424619 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 816876331 0 0) 0x5604128b7190 con 0x5604128b78b0 2024-05-08T18:05:05.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.889+0000 7f16c2711640 1 --2- 172.21.15.50:0/4056424619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 0x5604128b6b10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f16b400a520 tx=0x7f16b401d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9eb1e13ee5da944 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.889+0000 7f16c170f640 1 -- 172.21.15.50:0/4056424619 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16b4057070 con 0x5604128b66f0 2024-05-08T18:05:05.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.889+0000 7f16c170f640 1 -- 172.21.15.50:0/4056424619 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b4052570 con 0x5604128b66f0 2024-05-08T18:05:05.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.889+0000 7f16c170f640 1 -- 172.21.15.50:0/4056424619 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16b4052870 con 0x5604128b66f0 2024-05-08T18:05:05.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.889+0000 7f16c4c19800 1 -- 172.21.15.50:0/4056424619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 msgr2=0x5604128b6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.889+0000 7f16c4c19800 1 --2- 172.21.15.50:0/4056424619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 0x5604128b6b10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f16b400a520 tx=0x7f16b401d010 comp rx=0 tx=0).stop 2024-05-08T18:05:05.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.889+0000 7f16c4c19800 1 -- 172.21.15.50:0/4056424619 shutdown_connections 2024-05-08T18:05:05.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.889+0000 7f16c4c19800 1 --2- 172.21.15.50:0/4056424619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 0x5604128b6b10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.889+0000 7f16c4c19800 1 -- 172.21.15.50:0/4056424619 >> 172.21.15.50:0/4056424619 conn(0x5604128b1c70 msgr2=0x5604128b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.889+0000 7f16c4c19800 1 -- 172.21.15.50:0/4056424619 shutdown_connections 2024-05-08T18:05:05.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.889+0000 7f16c4c19800 1 -- 172.21.15.50:0/4056424619 wait complete. 2024-05-08T18:05:05.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.905+0000 7f16c4c19800 1 Processor -- start 2024-05-08T18:05:05.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.905+0000 7f16c4c19800 1 -- start start 2024-05-08T18:05:05.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.905+0000 7f16c4c19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 0x56041282bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.905+0000 7f16c4c19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560412801a10 con 0x5604128b66f0 2024-05-08T18:05:05.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.905+0000 7f16c2711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 0x56041282bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.905+0000 7f16c2711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 0x56041282bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37018/0 (socket says 172.21.15.50:37018) 2024-05-08T18:05:05.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.905+0000 7f16c2711640 1 -- 172.21.15.50:0/4101957164 learned_addr learned my addr 172.21.15.50:0/4101957164 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.905+0000 7f16c2711640 1 -- 172.21.15.50:0/4101957164 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56041282a4a0 con 0x5604128b66f0 2024-05-08T18:05:05.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.905+0000 7f16c2711640 1 --2- 172.21.15.50:0/4101957164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 0x56041282bee0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f16b4055040 tx=0x7f16b404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e200c5523935b9a9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.906+0000 7f15b0608640 1 -- 172.21.15.50:0/4101957164 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16b4062070 con 0x5604128b66f0 2024-05-08T18:05:05.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.906+0000 7f15b0608640 1 -- 172.21.15.50:0/4101957164 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b4052720 con 0x5604128b66f0 2024-05-08T18:05:05.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.906+0000 7f15b0608640 1 -- 172.21.15.50:0/4101957164 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16b4057040 con 0x5604128b66f0 2024-05-08T18:05:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.906+0000 7f16c4c19800 1 -- 172.21.15.50:0/4101957164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 msgr2=0x56041282bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.906+0000 7f16c4c19800 1 --2- 172.21.15.50:0/4101957164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 0x56041282bee0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f16b4055040 tx=0x7f16b404c000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.906+0000 7f16c4c19800 1 -- 172.21.15.50:0/4101957164 shutdown_connections 2024-05-08T18:05:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.906+0000 7f16c4c19800 1 --2- 172.21.15.50:0/4101957164 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 0x56041282bee0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.906+0000 7f16c4c19800 1 -- 172.21.15.50:0/4101957164 >> 172.21.15.50:0/4101957164 conn(0x5604128b1c70 msgr2=0x5604129a3310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.906+0000 7f16c4c19800 1 -- 172.21.15.50:0/4101957164 shutdown_connections 2024-05-08T18:05:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.906+0000 7f16c4c19800 1 -- 172.21.15.50:0/4101957164 wait complete. 2024-05-08T18:05:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.907+0000 7f16c4c19800 1 Processor -- start 2024-05-08T18:05:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.907+0000 7f16c4c19800 1 -- start start 2024-05-08T18:05:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.907+0000 7f16c4c19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 0x5604129b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.907+0000 7f16c4c19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5604128bfd80 con 0x5604128b66f0 2024-05-08T18:05:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.907+0000 7f16c2711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 0x5604129b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.907+0000 7f16c2711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 0x5604129b16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37032/0 (socket says 172.21.15.50:37032) 2024-05-08T18:05:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.907+0000 7f16c2711640 1 -- 172.21.15.50:0/955113962 learned_addr learned my addr 172.21.15.50:0/955113962 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.907+0000 7f16c2711640 1 -- 172.21.15.50:0/955113962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604129b1be0 con 0x5604128b66f0 2024-05-08T18:05:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.908+0000 7f16c2711640 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 0x5604129b16a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f16b4000c00 tx=0x7f16b4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.908+0000 7f15aee05640 1 -- 172.21.15.50:0/955113962 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16b4057050 con 0x5604128b66f0 2024-05-08T18:05:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.908+0000 7f15aee05640 1 -- 172.21.15.50:0/955113962 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b4067580 con 0x5604128b66f0 2024-05-08T18:05:05.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.908+0000 7f16c4c19800 1 -- 172.21.15.50:0/955113962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5604129b1e70 con 0x5604128b66f0 2024-05-08T18:05:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.908+0000 7f15aee05640 1 -- 172.21.15.50:0/955113962 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16b4062040 con 0x5604128b66f0 2024-05-08T18:05:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.908+0000 7f16c4c19800 1 -- 172.21.15.50:0/955113962 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604129b4b20 con 0x5604128b66f0 2024-05-08T18:05:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.909+0000 7f15aee05640 1 -- 172.21.15.50:0/955113962 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f16b4067860 con 0x5604128b66f0 2024-05-08T18:05:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.909+0000 7f15aee05640 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f15a803bba0 0x7f15a803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.909+0000 7f15aee05640 1 -- 172.21.15.50:0/955113962 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f16b4094700 con 0x5604128b66f0 2024-05-08T18:05:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.909+0000 7f16c1f10640 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f15a803bba0 0x7f15a803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.909+0000 7f16c4c19800 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604128b78b0 0x5604129b88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.909+0000 7f16c4c19800 1 -- 172.21.15.50:0/955113962 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604129b8e20 con 0x5604128b78b0 2024-05-08T18:05:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.909+0000 7f16c2f12640 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604128b78b0 0x5604129b88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.909+0000 7f16c1f10640 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f15a803bba0 0x7f15a803e060 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f16ac000c00 tx=0x7f16ac0013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.910+0000 7f16c2f12640 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604128b78b0 0x5604129b88e0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.910+0000 7f16c2f12640 1 -- 172.21.15.50:0/955113962 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604128b78b0 2024-05-08T18:05:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.910+0000 7f16c4c19800 1 -- 172.21.15.50:0/955113962 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604129b5e90 con 0x5604128b78b0 2024-05-08T18:05:05.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.910+0000 7f16c2f12640 1 -- 172.21.15.50:0/955113962 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604128b78b0 2024-05-08T18:05:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.910+0000 7f16c4c19800 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604129bb140 0x5604129bd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.910+0000 7f16c4c19800 1 -- 172.21.15.50:0/955113962 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604129bdc20 con 0x5604129bb140 2024-05-08T18:05:05.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.910+0000 7f16c2711640 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604129bb140 0x5604129bd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.911+0000 7f16c2711640 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604129bb140 0x5604129bd530 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.911+0000 7f16c2711640 1 -- 172.21.15.50:0/955113962 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f16b4064070 con 0x5604129bb140 2024-05-08T18:05:05.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.911+0000 7f16c4c19800 1 -- 172.21.15.50:0/955113962 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604129b9fe0 con 0x5604128b78b0 2024-05-08T18:05:05.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.911+0000 7f16c2f12640 1 -- 172.21.15.50:0/955113962 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604128b78b0 2024-05-08T18:05:05.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.912+0000 7f16c4c19800 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604129bf080 0x5604129c1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.912+0000 7f16c4c19800 1 -- 172.21.15.50:0/955113962 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604129c1b70 con 0x5604129bf080 2024-05-08T18:05:05.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.912+0000 7f16c1f10640 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604129bf080 0x5604129c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.912+0000 7f16c1f10640 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604129bf080 0x5604129c1470 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.912+0000 7f16c1f10640 1 -- 172.21.15.50:0/955113962 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f16ac00f670 con 0x5604129bf080 2024-05-08T18:05:05.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.912+0000 7f16c4c19800 1 Processor -- start 2024-05-08T18:05:05.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.913+0000 7f16c4c19800 1 -- start start 2024-05-08T18:05:05.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.913+0000 7f16c4c19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129cfee0 0x5604129d0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.913+0000 7f16c4c19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5604127a8330 con 0x5604129cfee0 2024-05-08T18:05:05.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.913+0000 7f16c2711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129cfee0 0x5604129d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.913+0000 7f16c2711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129cfee0 0x5604129d0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37040/0 (socket says 172.21.15.50:37040) 2024-05-08T18:05:05.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.913+0000 7f16c2711640 1 -- 172.21.15.50:0/3175643156 learned_addr learned my addr 172.21.15.50:0/3175643156 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.913+0000 7f16c2711640 1 -- 172.21.15.50:0/3175643156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604129d0840 con 0x5604129cfee0 2024-05-08T18:05:05.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.913+0000 7f16c2711640 1 --2- 172.21.15.50:0/3175643156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129cfee0 0x5604129d0300 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f16b4094010 tx=0x7f16b4006580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.913+0000 7f15a7fff640 1 -- 172.21.15.50:0/3175643156 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16b405d960 con 0x5604129cfee0 2024-05-08T18:05:05.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.913+0000 7f15a7fff640 1 -- 172.21.15.50:0/3175643156 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b405db00 con 0x5604129cfee0 2024-05-08T18:05:05.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.913+0000 7f16c4c19800 1 -- 172.21.15.50:0/3175643156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5604129d12b0 con 0x5604129cfee0 2024-05-08T18:05:05.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.914+0000 7f15a7fff640 1 -- 172.21.15.50:0/3175643156 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16b40940b0 con 0x5604129cfee0 2024-05-08T18:05:05.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.914+0000 7f16c4c19800 1 -- 172.21.15.50:0/3175643156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604129d17b0 con 0x5604129cfee0 2024-05-08T18:05:05.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.914+0000 7f15a7fff640 1 -- 172.21.15.50:0/3175643156 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f16b4094250 con 0x5604129cfee0 2024-05-08T18:05:05.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.914+0000 7f15a7fff640 1 --2- 172.21.15.50:0/3175643156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f159c03bb50 0x7f159c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.914+0000 7f15a7fff640 1 -- 172.21.15.50:0/3175643156 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f16b407f040 con 0x5604129cfee0 2024-05-08T18:05:05.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.915+0000 7f16c1f10640 1 --2- 172.21.15.50:0/3175643156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f159c03bb50 0x7f159c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.915+0000 7f16c4c19800 1 Processor -- start 2024-05-08T18:05:05.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.915+0000 7f16c4c19800 1 -- start start 2024-05-08T18:05:05.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.915+0000 7f16c1f10640 1 --2- 172.21.15.50:0/3175643156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f159c03bb50 0x7f159c03e010 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x5604129b96f0 tx=0x7f16ac006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.915+0000 7f16c4c19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129dcfe0 0x5604129dd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.915+0000 7f16c4c19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56041282a760 con 0x5604129dcfe0 2024-05-08T18:05:05.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.915+0000 7f16c2711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129dcfe0 0x5604129dd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.915+0000 7f16c2711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129dcfe0 0x5604129dd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37048/0 (socket says 172.21.15.50:37048) 2024-05-08T18:05:05.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.915+0000 7f16c2711640 1 -- 172.21.15.50:0/1142891304 learned_addr learned my addr 172.21.15.50:0/1142891304 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.915+0000 7f16c2711640 1 -- 172.21.15.50:0/1142891304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604129dd9d0 con 0x5604129dcfe0 2024-05-08T18:05:05.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.916+0000 7f16c2711640 1 --2- 172.21.15.50:0/1142891304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129dcfe0 0x5604129dd400 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f16b4097030 tx=0x7f16b408ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543a4cca2e26ed62 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.916+0000 7f15a5ffb640 1 -- 172.21.15.50:0/1142891304 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16b40823e0 con 0x5604129dcfe0 2024-05-08T18:05:05.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.916+0000 7f15a5ffb640 1 -- 172.21.15.50:0/1142891304 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b4083480 con 0x5604129dcfe0 2024-05-08T18:05:05.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.916+0000 7f15a5ffb640 1 -- 172.21.15.50:0/1142891304 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16b406f400 con 0x5604129dcfe0 2024-05-08T18:05:05.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.916+0000 7f16c4c19800 1 -- 172.21.15.50:0/1142891304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129dcfe0 msgr2=0x5604129dd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.916+0000 7f16c4c19800 1 --2- 172.21.15.50:0/1142891304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129dcfe0 0x5604129dd400 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f16b4097030 tx=0x7f16b408ed50 comp rx=0 tx=0).stop 2024-05-08T18:05:05.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.916+0000 7f16c4c19800 1 -- 172.21.15.50:0/1142891304 shutdown_connections 2024-05-08T18:05:05.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.916+0000 7f16c4c19800 1 --2- 172.21.15.50:0/1142891304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129dcfe0 0x5604129dd400 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.916+0000 7f16c4c19800 1 -- 172.21.15.50:0/1142891304 >> 172.21.15.50:0/1142891304 conn(0x5604129d92f0 msgr2=0x5604129d7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.916+0000 7f16c4c19800 1 -- 172.21.15.50:0/1142891304 shutdown_connections 2024-05-08T18:05:05.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.916+0000 7f16c4c19800 1 -- 172.21.15.50:0/1142891304 wait complete. 2024-05-08T18:05:05.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.917+0000 7f16c4c19800 1 Processor -- start 2024-05-08T18:05:05.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.917+0000 7f16c4c19800 1 -- start start 2024-05-08T18:05:05.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.917+0000 7f16c4c19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129dcfe0 0x5604129e4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.917+0000 7f16c4c19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5604129de060 con 0x5604129dcfe0 2024-05-08T18:05:05.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.917+0000 7f16c2711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129dcfe0 0x5604129e4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.917+0000 7f16c2711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129dcfe0 0x5604129e4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37050/0 (socket says 172.21.15.50:37050) 2024-05-08T18:05:05.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.917+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 learned_addr learned my addr 172.21.15.50:0/4079288061 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:05.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.917+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604129e4880 con 0x5604129dcfe0 2024-05-08T18:05:05.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.917+0000 7f16c2711640 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129dcfe0 0x5604129e4340 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f16b408e3d0 tx=0x7f16b4002e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.918+0000 7f1593fff640 1 -- 172.21.15.50:0/4079288061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16b4091380 con 0x5604129dcfe0 2024-05-08T18:05:05.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.918+0000 7f1593fff640 1 -- 172.21.15.50:0/4079288061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b408b360 con 0x5604129dcfe0 2024-05-08T18:05:05.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.918+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5604129e4880 con 0x5604129dcfe0 2024-05-08T18:05:05.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.918+0000 7f1593fff640 1 -- 172.21.15.50:0/4079288061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16b408b510 con 0x5604129dcfe0 2024-05-08T18:05:05.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.918+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604129e4880 con 0x5604129dcfe0 2024-05-08T18:05:05.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.918+0000 7f1593fff640 1 -- 172.21.15.50:0/4079288061 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f16b40ba060 con 0x5604129dcfe0 2024-05-08T18:05:05.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.919+0000 7f1593fff640 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f158c03bba0 0x7f158c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.919+0000 7f1593fff640 1 -- 172.21.15.50:0/4079288061 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f16b40a8020 con 0x5604129dcfe0 2024-05-08T18:05:05.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.919+0000 7f16c1f10640 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f158c03bba0 0x7f158c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.919+0000 7f16c4c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.919+0000 7f16c4c19800 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604129efae0 0x5604129f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.919+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604129f26f0 con 0x5604129efae0 2024-05-08T18:05:05.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.919+0000 7f16c1f10640 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f158c03bba0 0x7f158c03e060 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5604129c1f70 tx=0x7f16ac023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.919+0000 7f16c2f12640 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604129efae0 0x5604129f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.920+0000 7f16c2f12640 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604129efae0 0x5604129f1fa0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.920+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604129efae0 2024-05-08T18:05:05.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.920+0000 7f16c4c19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.920+0000 7f16c4c19800 20 realm 2024-05-08T18:05:05.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.920+0000 7f16c4c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.920+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604129f2b00 con 0x5604129efae0 2024-05-08T18:05:05.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.920+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604129efae0 2024-05-08T18:05:05.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.920+0000 7f16c4c19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.920+0000 7f16c4c19800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:05.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.920+0000 7f16c4c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.920+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604129f33d0 con 0x5604129efae0 2024-05-08T18:05:05.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.920+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604129efae0 2024-05-08T18:05:05.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.920+0000 7f16c4c19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.920+0000 7f16c4c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.920+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129efae0 2024-05-08T18:05:05.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.921+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604129efae0 2024-05-08T18:05:05.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.921+0000 7f16c4c19800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:05.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.921+0000 7f16c4c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.921+0000 7f16c4c19800 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604129f4260 0x5604129f6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.921+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604129f6db0 con 0x5604129f4260 2024-05-08T18:05:05.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.921+0000 7f16c2711640 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604129f4260 0x5604129f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.921+0000 7f16c2711640 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604129f4260 0x5604129f6650 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.921+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.921+0000 7f16c4c19800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:05.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.921+0000 7f16c4c19800 20 searching for the correct realm 2024-05-08T18:05:05.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.921+0000 7f16c4c19800 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604129f8e50 0x5604129fb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.921+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129fb9e0 con 0x5604129f8e50 2024-05-08T18:05:05.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.922+0000 7f16c1f10640 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604129f8e50 0x5604129fb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.922+0000 7f16c1f10640 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604129f8e50 0x5604129fb310 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.922+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16ac037040 con 0x5604129f8e50 2024-05-08T18:05:05.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.922+0000 7f16c1f10640 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f16ac02ea80 0x7f16ac03a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:05.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.922+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129fb9e0 con 0x7f16ac02ea80 2024-05-08T18:05:05.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.922+0000 7f16c2f12640 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f16ac02ea80 0x7f16ac03a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:05.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.923+0000 7f16c2f12640 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f16ac02ea80 0x7f16ac03a3c0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:05.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.923+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc00a4a0 con 0x7f16ac02ea80 2024-05-08T18:05:05.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.923+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f4260 2024-05-08T18:05:05.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.923+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.923+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129efae0 2024-05-08T18:05:05.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.923+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604129efae0 2024-05-08T18:05:05.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.923+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129efae0 2024-05-08T18:05:05.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.923+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604129efae0 2024-05-08T18:05:05.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.924+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f4260 2024-05-08T18:05:05.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.924+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.924+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f4260 2024-05-08T18:05:05.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.924+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.924+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x7f16ac02ea80 2024-05-08T18:05:05.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.924+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc00a4a0 con 0x7f16ac02ea80 2024-05-08T18:05:05.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.924+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f8e50 2024-05-08T18:05:05.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.924+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16ac037040 con 0x5604129f8e50 2024-05-08T18:05:05.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.924+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129efae0 2024-05-08T18:05:05.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.925+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604129efae0 2024-05-08T18:05:05.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.925+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f4260 2024-05-08T18:05:05.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.925+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.925+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129efae0 2024-05-08T18:05:05.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.925+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604129efae0 2024-05-08T18:05:05.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.925+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x7f16ac02ea80 2024-05-08T18:05:05.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.925+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f16bc00a4a0 con 0x7f16ac02ea80 2024-05-08T18:05:05.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.926+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f8e50 2024-05-08T18:05:05.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.926+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16ac037040 con 0x5604129f8e50 2024-05-08T18:05:05.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.926+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f8e50 2024-05-08T18:05:05.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.926+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16ac037040 con 0x5604129f8e50 2024-05-08T18:05:05.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.926+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f4260 2024-05-08T18:05:05.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.926+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.926+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f8e50 2024-05-08T18:05:05.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.926+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16ac037040 con 0x5604129f8e50 2024-05-08T18:05:05.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.926+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f8e50 2024-05-08T18:05:05.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.927+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f16ac037040 con 0x5604129f8e50 2024-05-08T18:05:05.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.927+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129efae0 2024-05-08T18:05:05.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.927+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604129efae0 2024-05-08T18:05:05.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.927+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129efae0 2024-05-08T18:05:05.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.927+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604129efae0 2024-05-08T18:05:05.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.927+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f4260 2024-05-08T18:05:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.927+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.927+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f8e50 2024-05-08T18:05:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.928+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16ac037040 con 0x5604129f8e50 2024-05-08T18:05:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.928+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x7f16ac02ea80 2024-05-08T18:05:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.928+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc00a4a0 con 0x7f16ac02ea80 2024-05-08T18:05:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.928+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129efae0 2024-05-08T18:05:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.928+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604129efae0 2024-05-08T18:05:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.928+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f4260 2024-05-08T18:05:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.929+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.929+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f8e50 2024-05-08T18:05:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.929+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16ac037040 con 0x5604129f8e50 2024-05-08T18:05:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.929+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f4260 2024-05-08T18:05:05.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.929+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.929+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x7f16ac02ea80 2024-05-08T18:05:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.929+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc00a4a0 con 0x7f16ac02ea80 2024-05-08T18:05:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.929+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f8e50 2024-05-08T18:05:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.929+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16ac037040 con 0x5604129f8e50 2024-05-08T18:05:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.930+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129f4260 2024-05-08T18:05:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.930+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.930+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x5604129efae0 2024-05-08T18:05:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.930+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604129efae0 2024-05-08T18:05:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.930+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604129f3770 con 0x7f16ac02ea80 2024-05-08T18:05:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.930+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc00a4a0 con 0x7f16ac02ea80 2024-05-08T18:05:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.930+0000 7f16c4c19800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.930+0000 7f16c4c19800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.930+0000 7f16c4c19800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.930+0000 7f16c4c19800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:05.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.930+0000 7f16c4c19800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:05.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.930+0000 7f16c4c19800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:05.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.930+0000 7f16c4c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.930+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604129f7750 con 0x5604129efae0 2024-05-08T18:05:05.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604129efae0 2024-05-08T18:05:05.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c4c19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c4c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604129f7750 con 0x5604129efae0 2024-05-08T18:05:05.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f16bc00a4a0 con 0x5604129efae0 2024-05-08T18:05:05.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c4c19800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:05.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c4c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604129f7750 con 0x5604129f8e50 2024-05-08T18:05:05.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f16ac037040 con 0x5604129f8e50 2024-05-08T18:05:05.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c4c19800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:05.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c4c19800 20 zone default found 2024-05-08T18:05:05.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c4c19800 4 Realm: () 2024-05-08T18:05:05.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c4c19800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:05.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c4c19800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:05.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c4c19800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:05.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c4c19800 20 zonegroup default 2024-05-08T18:05:05.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c4c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.931+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604129f7ff0 con 0x5604129f4260 2024-05-08T18:05:05.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.932+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.932+0000 7f16c4c19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.932+0000 7f16c4c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.932+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604129fd840 con 0x5604129f8e50 2024-05-08T18:05:05.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.932+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f16ac037040 con 0x5604129f8e50 2024-05-08T18:05:05.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.932+0000 7f16c4c19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.932+0000 7f16c4c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.932+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604129fdbe0 con 0x5604129f8e50 2024-05-08T18:05:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.932+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f16ac037040 con 0x5604129f8e50 2024-05-08T18:05:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.932+0000 7f16c4c19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.932+0000 7f16c4c19800 20 started sync module instance, tier type = 2024-05-08T18:05:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.932+0000 7f16c4c19800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.932+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604129fcce0 con 0x7f16ac02ea80 2024-05-08T18:05:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.933+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc00a4a0 con 0x7f16ac02ea80 2024-05-08T18:05:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.933+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94575492321840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604129fcce0 con 0x7f16ac02ea80 2024-05-08T18:05:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.933+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604129fe640 con 0x5604129efae0 2024-05-08T18:05:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.934+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94575492321840] v27'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc040040 con 0x7f16ac02ea80 2024-05-08T18:05:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.934+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc040040 con 0x5604129efae0 2024-05-08T18:05:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.934+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94575492325712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604129fcce0 con 0x5604129efae0 2024-05-08T18:05:05.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.934+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604129ff100 con 0x5604129f8e50 2024-05-08T18:05:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.935+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94575492325712] v27'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc040040 con 0x5604129efae0 2024-05-08T18:05:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.936+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16ac037040 con 0x5604129f8e50 2024-05-08T18:05:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.936+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94575492319280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604129fcc50 con 0x5604129f8e50 2024-05-08T18:05:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.936+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604129fffc0 con 0x7f16ac02ea80 2024-05-08T18:05:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.936+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94575492319280] v27'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16ac037040 con 0x5604129f8e50 2024-05-08T18:05:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.937+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc040040 con 0x7f16ac02ea80 2024-05-08T18:05:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.937+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94575492332304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604129fff30 con 0x7f16ac02ea80 2024-05-08T18:05:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.937+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560412a00d50 con 0x5604129efae0 2024-05-08T18:05:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.938+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc040040 con 0x5604129efae0 2024-05-08T18:05:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.938+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94575492332304] v27'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc040040 con 0x7f16ac02ea80 2024-05-08T18:05:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.938+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94575492335712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560412a00d50 con 0x5604129efae0 2024-05-08T18:05:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.938+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560412a003f0 con 0x5604129f4260 2024-05-08T18:05:05.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.939+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.939+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94575492335712] v27'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc040040 con 0x5604129efae0 2024-05-08T18:05:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.939+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94575492339600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560412a02060 con 0x5604129f4260 2024-05-08T18:05:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.939+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560412a01140 con 0x5604129f4260 2024-05-08T18:05:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.940+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94575492339600] v27'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.940+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.940+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94575492343152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560412a02e40 con 0x5604129f4260 2024-05-08T18:05:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.940+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560412a036d0 con 0x5604129f4260 2024-05-08T18:05:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.941+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v27'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.941+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94575492345408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560412a03d60 con 0x5604129f4260 2024-05-08T18:05:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.941+0000 7f16c4c19800 20 add_watcher() i=0 2024-05-08T18:05:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.941+0000 7f16c4c19800 20 add_watcher() i=1 2024-05-08T18:05:05.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.941+0000 7f16c4c19800 20 add_watcher() i=2 2024-05-08T18:05:05.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.941+0000 7f16c4c19800 20 add_watcher() i=3 2024-05-08T18:05:05.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.941+0000 7f16c4c19800 20 add_watcher() i=4 2024-05-08T18:05:05.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.941+0000 7f16c4c19800 20 add_watcher() i=5 2024-05-08T18:05:05.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.941+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94575492343152] v27'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.941+0000 7f16c4c19800 20 add_watcher() i=6 2024-05-08T18:05:05.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.942+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94575492345408] v27'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.942+0000 7f16c4c19800 20 add_watcher() i=7 2024-05-08T18:05:05.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.942+0000 7f16c4c19800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:05.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.942+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560412a06c80 con 0x5604129efae0 2024-05-08T18:05:05.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.942+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f16bc040040 con 0x5604129efae0 2024-05-08T18:05:05.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.942+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94575492347808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560412a04070 con 0x5604129efae0 2024-05-08T18:05:05.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.943+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94575492347808] v27'80 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f16bc040040 con 0x5604129efae0 2024-05-08T18:05:05.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.943+0000 7f1578fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.943+0000 7f16c4c19800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.943+0000 7f16c4c19800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.944+0000 7f16c4c19800 5 note: GC not initialized 2024-05-08T18:05:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.944+0000 7f14f37fe640 20 reqs_thread_entry: start 2024-05-08T18:05:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.944+0000 7f16c4c19800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.944+0000 7f16c4c19800 20 Filter name: none 2024-05-08T18:05:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.944+0000 7f14f17fa640 20 reqs_thread_entry: start 2024-05-08T18:05:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.944+0000 7f16c4c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.944+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560412a39b30 con 0x5604129efae0 2024-05-08T18:05:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.944+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~673 out=673b,getxattrs out=75b] v0'0 uv8 ondisk = 0) ==== 231+0+796 (crc 0 0 0) 0x7f16bc040040 con 0x5604129efae0 2024-05-08T18:05:05.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.944+0000 7f16c4c19800 20 rados_obj.operate() r=0 bl.length=673 2024-05-08T18:05:05.953 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"},{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"},{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy","active":true,"create_date":"2024-05-08T18:05:05.808771Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:05.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.945+0000 7f1578fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:05.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.945+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94575492347808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560412a3a070 con 0x5604129efae0 2024-05-08T18:05:05.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.946+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94575492347808] v27'81 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f16bc040040 con 0x5604129efae0 2024-05-08T18:05:05.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.946+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94575492321840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560412a3a070 con 0x7f16ac02ea80 2024-05-08T18:05:05.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.947+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94575492321840] v27'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc040040 con 0x7f16ac02ea80 2024-05-08T18:05:05.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.947+0000 7f16c4c19800 20 remove_watcher() i=0 2024-05-08T18:05:05.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.947+0000 7f16c4c19800 2 removed watcher, disabling cache 2024-05-08T18:05:05.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.947+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94575492325712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604129fd230 con 0x5604129efae0 2024-05-08T18:05:05.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.948+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94575492325712] v27'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc040040 con 0x5604129efae0 2024-05-08T18:05:05.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.948+0000 7f16c4c19800 20 remove_watcher() i=1 2024-05-08T18:05:05.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.948+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94575492319280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604129fd230 con 0x5604129f8e50 2024-05-08T18:05:05.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.949+0000 7f16c1f10640 1 -- 172.21.15.50:0/4079288061 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94575492319280] v27'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16ac037040 con 0x5604129f8e50 2024-05-08T18:05:05.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.949+0000 7f16c4c19800 20 remove_watcher() i=2 2024-05-08T18:05:05.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.949+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94575492332304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604129fc830 con 0x7f16ac02ea80 2024-05-08T18:05:05.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.950+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94575492332304] v27'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc040040 con 0x7f16ac02ea80 2024-05-08T18:05:05.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.950+0000 7f16c4c19800 20 remove_watcher() i=3 2024-05-08T18:05:05.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.950+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94575492335712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604129ffb10 con 0x5604129efae0 2024-05-08T18:05:05.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.951+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94575492335712] v27'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc040040 con 0x5604129efae0 2024-05-08T18:05:05.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.951+0000 7f16c4c19800 20 remove_watcher() i=4 2024-05-08T18:05:05.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.951+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94575492339600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604129fc830 con 0x5604129f4260 2024-05-08T18:05:05.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.952+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94575492339600] v27'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.952+0000 7f16c4c19800 20 remove_watcher() i=5 2024-05-08T18:05:05.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.952+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94575492343152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560412a00860 con 0x5604129f4260 2024-05-08T18:05:05.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.953+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94575492343152] v27'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.953+0000 7f16c4c19800 20 remove_watcher() i=6 2024-05-08T18:05:05.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.953+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94575492345408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604129ffb10 con 0x5604129f4260 2024-05-08T18:05:05.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.954+0000 7f16c2711640 1 -- 172.21.15.50:0/4079288061 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94575492345408] v27'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16b4064070 con 0x5604129f4260 2024-05-08T18:05:05.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.954+0000 7f16c4c19800 20 remove_watcher() i=7 2024-05-08T18:05:05.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604129f4260 msgr2=0x5604129f6650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604129f4260 0x5604129f6650 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604129efae0 msgr2=0x5604129f1fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604129efae0 0x5604129f1fa0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f16ac02ea80 msgr2=0x7f16ac03a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f16ac02ea80 0x7f16ac03a3c0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604129f8e50 msgr2=0x5604129fb310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604129f8e50 0x5604129fb310 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f158c03bba0 msgr2=0x7f158c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f158c03bba0 0x7f158c03e060 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5604129c1f70 tx=0x7f16ac023040 comp rx=0 tx=0).stop 2024-05-08T18:05:05.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129dcfe0 msgr2=0x5604129e4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c2f12640 1 -- 172.21.15.50:0/4079288061 reap_dead start 2024-05-08T18:05:05.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129dcfe0 0x5604129e4340 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f16b408e3d0 tx=0x7f16b4002e30 comp rx=0 tx=0).stop 2024-05-08T18:05:05.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 shutdown_connections 2024-05-08T18:05:05.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 --2- 172.21.15.50:0/4079288061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129dcfe0 0x5604129e4340 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 >> 172.21.15.50:0/4079288061 conn(0x5604129d92f0 msgr2=0x5604129d6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 shutdown_connections 2024-05-08T18:05:05.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 -- 172.21.15.50:0/4079288061 wait complete. 2024-05-08T18:05:05.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 -- 172.21.15.50:0/3175643156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f159c03bb50 msgr2=0x7f159c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 --2- 172.21.15.50:0/3175643156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f159c03bb50 0x7f159c03e010 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x5604129b96f0 tx=0x7f16ac006000 comp rx=0 tx=0).stop 2024-05-08T18:05:05.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 -- 172.21.15.50:0/3175643156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129cfee0 msgr2=0x5604129d0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 --2- 172.21.15.50:0/3175643156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129cfee0 0x5604129d0300 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f16b4094010 tx=0x7f16b4006580 comp rx=0 tx=0).stop 2024-05-08T18:05:05.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 -- 172.21.15.50:0/3175643156 shutdown_connections 2024-05-08T18:05:05.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 --2- 172.21.15.50:0/3175643156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f159c03bb50 0x7f159c03e010 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 --2- 172.21.15.50:0/3175643156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604129cfee0 0x5604129d0300 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 -- 172.21.15.50:0/3175643156 >> 172.21.15.50:0/3175643156 conn(0x5604129c3f80 msgr2=0x5604129c4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 -- 172.21.15.50:0/3175643156 shutdown_connections 2024-05-08T18:05:05.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.955+0000 7f16c4c19800 1 -- 172.21.15.50:0/3175643156 wait complete. 2024-05-08T18:05:05.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.956+0000 7f16c4c19800 1 -- 172.21.15.50:0/955113962 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604129bb140 msgr2=0x5604129bd530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.956+0000 7f16c4c19800 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604129bb140 0x5604129bd530 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.956+0000 7f16c4c19800 1 -- 172.21.15.50:0/955113962 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604128b78b0 msgr2=0x5604129b88e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.956+0000 7f16c4c19800 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604128b78b0 0x5604129b88e0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.956+0000 7f16c4c19800 1 -- 172.21.15.50:0/955113962 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604129bf080 msgr2=0x5604129c1470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.956+0000 7f16c4c19800 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604129bf080 0x5604129c1470 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.956+0000 7f16c4c19800 1 -- 172.21.15.50:0/955113962 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f15a803bba0 msgr2=0x7f15a803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.956+0000 7f16c4c19800 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f15a803bba0 0x7f15a803e060 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f16ac000c00 tx=0x7f16ac0013b0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.956+0000 7f16c4c19800 1 -- 172.21.15.50:0/955113962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 msgr2=0x5604129b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:05.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.956+0000 7f16c4c19800 1 --2- 172.21.15.50:0/955113962 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604128b66f0 0x5604129b16a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f16b4000c00 tx=0x7f16b4002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:05.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.956+0000 7f16c2f12640 1 -- 172.21.15.50:0/955113962 reap_dead start 2024-05-08T18:05:05.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.956+0000 7f16c4c19800 1 -- 172.21.15.50:0/955113962 shutdown_connections 2024-05-08T18:05:05.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.956+0000 7f16c4c19800 1 -- 172.21.15.50:0/955113962 >> 172.21.15.50:0/955113962 conn(0x5604128b1c70 msgr2=0x56041282acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:05.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.956+0000 7f16c4c19800 1 -- 172.21.15.50:0/955113962 shutdown_connections 2024-05-08T18:05:05.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:05.956+0000 7f16c4c19800 1 -- 172.21.15.50:0/955113962 wait complete. 2024-05-08T18:05:05.963 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True, 'create_date': '2024-05-08T18:05:05.808771Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:05.964 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2024-05-08T18:05:05.964 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2024-05-08T18:05:05.964 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --subuser foo:foo1 --key-type swift 2024-05-08T18:05:06.012 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:06.012 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:06.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.015+0000 7ff828575800 1 Processor -- start 2024-05-08T18:05:06.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.015+0000 7ff828575800 1 -- start start 2024-05-08T18:05:06.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.015+0000 7ff828575800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55fee2273a70 0x55fee2273e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.015+0000 7ff828575800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55fee2274510 con 0x55fee22747f0 2024-05-08T18:05:06.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.015+0000 7ff828575800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55fee2114e20 con 0x55fee2273a70 2024-05-08T18:05:06.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.016+0000 7ff82606d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55fee2273a70 0x55fee2273e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.016+0000 7ff82606d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55fee2273a70 0x55fee2273e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37066/0 (socket says 172.21.15.50:37066) 2024-05-08T18:05:06.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.016+0000 7ff82606d640 1 -- 172.21.15.50:0/651769141 learned_addr learned my addr 172.21.15.50:0/651769141 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.016+0000 7ff82586c640 1 --1- 172.21.15.50:0/651769141 >> v1:172.21.15.46:6789/0 conn(0x55fee22747f0 0x55fee2274ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43052/0 (socket says 172.21.15.50:43052) 2024-05-08T18:05:06.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.016+0000 7ff82606d640 1 -- 172.21.15.50:0/651769141 >> v1:172.21.15.46:6789/0 conn(0x55fee22747f0 legacy=0x55fee2274ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.016+0000 7ff82606d640 1 -- 172.21.15.50:0/651769141 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fee22753b0 con 0x55fee2273a70 2024-05-08T18:05:06.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.018+0000 7ff82606d640 1 --2- 172.21.15.50:0/651769141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 0x55fee2273e90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff81800a520 tx=0x7ff81801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75186bda5841bd04 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.018+0000 7ff82506b640 1 -- 172.21.15.50:0/651769141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff818057070 con 0x55fee2273a70 2024-05-08T18:05:06.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.018+0000 7ff82506b640 1 -- 172.21.15.50:0/651769141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff818052570 con 0x55fee2273a70 2024-05-08T18:05:06.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.018+0000 7ff82506b640 1 -- 172.21.15.50:0/651769141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff818052870 con 0x55fee2273a70 2024-05-08T18:05:06.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.018+0000 7ff828575800 1 -- 172.21.15.50:0/651769141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 msgr2=0x55fee2273e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.018+0000 7ff828575800 1 --2- 172.21.15.50:0/651769141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 0x55fee2273e90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff81800a520 tx=0x7ff81801d010 comp rx=0 tx=0).stop 2024-05-08T18:05:06.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.019+0000 7ff828575800 1 -- 172.21.15.50:0/651769141 shutdown_connections 2024-05-08T18:05:06.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.019+0000 7ff828575800 1 --2- 172.21.15.50:0/651769141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 0x55fee2273e90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.019+0000 7ff828575800 1 -- 172.21.15.50:0/651769141 >> 172.21.15.50:0/651769141 conn(0x55fee226e2f0 msgr2=0x55fee2270710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.019+0000 7ff828575800 1 -- 172.21.15.50:0/651769141 shutdown_connections 2024-05-08T18:05:06.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.019+0000 7ff828575800 1 -- 172.21.15.50:0/651769141 wait complete. 2024-05-08T18:05:06.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.034+0000 7ff828575800 1 Processor -- start 2024-05-08T18:05:06.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.034+0000 7ff828575800 1 -- start start 2024-05-08T18:05:06.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.034+0000 7ff828575800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 0x55fee23619b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.034+0000 7ff828575800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fee21bfa30 con 0x55fee2273a70 2024-05-08T18:05:06.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.034+0000 7ff82606d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 0x55fee23619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.034+0000 7ff82606d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 0x55fee23619b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37068/0 (socket says 172.21.15.50:37068) 2024-05-08T18:05:06.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.034+0000 7ff82606d640 1 -- 172.21.15.50:0/3254015569 learned_addr learned my addr 172.21.15.50:0/3254015569 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.035+0000 7ff82606d640 1 -- 172.21.15.50:0/3254015569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fee235ff70 con 0x55fee2273a70 2024-05-08T18:05:06.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.035+0000 7ff82606d640 1 --2- 172.21.15.50:0/3254015569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 0x55fee23619b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff818055040 tx=0x7ff81804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c013897f855828 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.035+0000 7ff714608640 1 -- 172.21.15.50:0/3254015569 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff818057040 con 0x55fee2273a70 2024-05-08T18:05:06.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.035+0000 7ff714608640 1 -- 172.21.15.50:0/3254015569 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff818052870 con 0x55fee2273a70 2024-05-08T18:05:06.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.035+0000 7ff714608640 1 -- 172.21.15.50:0/3254015569 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff818052b90 con 0x55fee2273a70 2024-05-08T18:05:06.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.036+0000 7ff828575800 1 -- 172.21.15.50:0/3254015569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 msgr2=0x55fee23619b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.036+0000 7ff828575800 1 --2- 172.21.15.50:0/3254015569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 0x55fee23619b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff818055040 tx=0x7ff81804c000 comp rx=0 tx=0).stop 2024-05-08T18:05:06.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.036+0000 7ff828575800 1 -- 172.21.15.50:0/3254015569 shutdown_connections 2024-05-08T18:05:06.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.036+0000 7ff828575800 1 --2- 172.21.15.50:0/3254015569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 0x55fee23619b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.036+0000 7ff828575800 1 -- 172.21.15.50:0/3254015569 >> 172.21.15.50:0/3254015569 conn(0x55fee226e2f0 msgr2=0x55fee2364040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.036+0000 7ff828575800 1 -- 172.21.15.50:0/3254015569 shutdown_connections 2024-05-08T18:05:06.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.036+0000 7ff828575800 1 -- 172.21.15.50:0/3254015569 wait complete. 2024-05-08T18:05:06.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.036+0000 7ff828575800 1 Processor -- start 2024-05-08T18:05:06.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.036+0000 7ff828575800 1 -- start start 2024-05-08T18:05:06.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.036+0000 7ff828575800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 0x55fee236f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.036+0000 7ff828575800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fee227ccc0 con 0x55fee2273a70 2024-05-08T18:05:06.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.036+0000 7ff82606d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 0x55fee236f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.036+0000 7ff82606d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 0x55fee236f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37074/0 (socket says 172.21.15.50:37074) 2024-05-08T18:05:06.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.036+0000 7ff82606d640 1 -- 172.21.15.50:0/2427173713 learned_addr learned my addr 172.21.15.50:0/2427173713 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.037+0000 7ff82606d640 1 -- 172.21.15.50:0/2427173713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fee236fe00 con 0x55fee2273a70 2024-05-08T18:05:06.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.037+0000 7ff82606d640 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 0x55fee236f8c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff818052540 tx=0x7ff818003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.037+0000 7ff712e05640 1 -- 172.21.15.50:0/2427173713 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff818073070 con 0x55fee2273a70 2024-05-08T18:05:06.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.037+0000 7ff712e05640 1 -- 172.21.15.50:0/2427173713 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff81805d570 con 0x55fee2273a70 2024-05-08T18:05:06.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.037+0000 7ff828575800 1 -- 172.21.15.50:0/2427173713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fee2370090 con 0x55fee2273a70 2024-05-08T18:05:06.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.037+0000 7ff712e05640 1 -- 172.21.15.50:0/2427173713 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff818057070 con 0x55fee2273a70 2024-05-08T18:05:06.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.037+0000 7ff828575800 1 -- 172.21.15.50:0/2427173713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fee2372d40 con 0x55fee2273a70 2024-05-08T18:05:06.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.038+0000 7ff712e05640 1 -- 172.21.15.50:0/2427173713 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff818076070 con 0x55fee2273a70 2024-05-08T18:05:06.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.038+0000 7ff712e05640 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff70c03bba0 0x7ff70c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.038+0000 7ff712e05640 1 -- 172.21.15.50:0/2427173713 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7ff818094130 con 0x55fee2273a70 2024-05-08T18:05:06.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.038+0000 7ff82586c640 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff70c03bba0 0x7ff70c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.038+0000 7ff828575800 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fee22747f0 0x55fee2376b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.038+0000 7ff828575800 1 -- 172.21.15.50:0/2427173713 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee2377040 con 0x55fee22747f0 2024-05-08T18:05:06.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.039+0000 7ff82686e640 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fee22747f0 0x55fee2376b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.039+0000 7ff82586c640 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff70c03bba0 0x7ff70c03e060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff81c001e80 tx=0x7ff81c0013e0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.039+0000 7ff82686e640 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fee22747f0 0x55fee2376b00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.039+0000 7ff82686e640 1 -- 172.21.15.50:0/2427173713 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee22747f0 2024-05-08T18:05:06.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.039+0000 7ff828575800 1 -- 172.21.15.50:0/2427173713 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23740d0 con 0x55fee22747f0 2024-05-08T18:05:06.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.039+0000 7ff82686e640 1 -- 172.21.15.50:0/2427173713 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee22747f0 2024-05-08T18:05:06.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.040+0000 7ff828575800 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fee2379330 0x55fee237b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.040+0000 7ff828575800 1 -- 172.21.15.50:0/2427173713 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee237bdf0 con 0x55fee2379330 2024-05-08T18:05:06.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.040+0000 7ff82606d640 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fee2379330 0x55fee237b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.040+0000 7ff82606d640 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fee2379330 0x55fee237b720 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.040+0000 7ff82606d640 1 -- 172.21.15.50:0/2427173713 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff818075030 con 0x55fee2379330 2024-05-08T18:05:06.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.040+0000 7ff828575800 1 -- 172.21.15.50:0/2427173713 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23781d0 con 0x55fee22747f0 2024-05-08T18:05:06.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.041+0000 7ff82686e640 1 -- 172.21.15.50:0/2427173713 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee22747f0 2024-05-08T18:05:06.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.041+0000 7ff828575800 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fee237d230 0x55fee237f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.041+0000 7ff828575800 1 -- 172.21.15.50:0/2427173713 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee237fd60 con 0x55fee237d230 2024-05-08T18:05:06.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.041+0000 7ff82586c640 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fee237d230 0x55fee237f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.041+0000 7ff82586c640 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fee237d230 0x55fee237f620 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.041+0000 7ff82586c640 1 -- 172.21.15.50:0/2427173713 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff81c00bda0 con 0x55fee237d230 2024-05-08T18:05:06.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.042+0000 7ff828575800 1 Processor -- start 2024-05-08T18:05:06.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.042+0000 7ff828575800 1 -- start start 2024-05-08T18:05:06.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.042+0000 7ff828575800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee238e0d0 0x55fee238e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.042+0000 7ff828575800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fee2166330 con 0x55fee238e0d0 2024-05-08T18:05:06.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.042+0000 7ff82606d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee238e0d0 0x55fee238e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.042+0000 7ff82606d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee238e0d0 0x55fee238e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37080/0 (socket says 172.21.15.50:37080) 2024-05-08T18:05:06.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.042+0000 7ff82606d640 1 -- 172.21.15.50:0/1886836873 learned_addr learned my addr 172.21.15.50:0/1886836873 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.042+0000 7ff82606d640 1 -- 172.21.15.50:0/1886836873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fee238ea30 con 0x55fee238e0d0 2024-05-08T18:05:06.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.042+0000 7ff82606d640 1 --2- 172.21.15.50:0/1886836873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee238e0d0 0x55fee238e4f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff81806d000 tx=0x7ff81808bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.043+0000 7ff70bfff640 1 -- 172.21.15.50:0/1886836873 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff81808a070 con 0x55fee238e0d0 2024-05-08T18:05:06.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.043+0000 7ff70bfff640 1 -- 172.21.15.50:0/1886836873 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff818092600 con 0x55fee238e0d0 2024-05-08T18:05:06.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.043+0000 7ff828575800 1 -- 172.21.15.50:0/1886836873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fee238f4a0 con 0x55fee238e0d0 2024-05-08T18:05:06.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.043+0000 7ff70bfff640 1 -- 172.21.15.50:0/1886836873 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff818077e10 con 0x55fee238e0d0 2024-05-08T18:05:06.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.043+0000 7ff828575800 1 -- 172.21.15.50:0/1886836873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fee238f9a0 con 0x55fee238e0d0 2024-05-08T18:05:06.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.043+0000 7ff70bfff640 1 -- 172.21.15.50:0/1886836873 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff81805cb30 con 0x55fee238e0d0 2024-05-08T18:05:06.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.044+0000 7ff70bfff640 1 --2- 172.21.15.50:0/1886836873 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff70003bb00 0x7ff70003dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.044+0000 7ff70bfff640 1 -- 172.21.15.50:0/1886836873 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7ff81809aa00 con 0x55fee238e0d0 2024-05-08T18:05:06.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.044+0000 7ff82586c640 1 --2- 172.21.15.50:0/1886836873 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff70003bb00 0x7ff70003dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.044+0000 7ff828575800 1 Processor -- start 2024-05-08T18:05:06.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.044+0000 7ff828575800 1 -- start start 2024-05-08T18:05:06.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.044+0000 7ff82586c640 1 --2- 172.21.15.50:0/1886836873 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff70003bb00 0x7ff70003dfc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55fee23774b0 tx=0x7ff81c00f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.044+0000 7ff828575800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee239b240 0x55fee239b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.044+0000 7ff828575800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fee2360230 con 0x55fee239b240 2024-05-08T18:05:06.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.044+0000 7ff82606d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee239b240 0x55fee239b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.044+0000 7ff82606d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee239b240 0x55fee239b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37090/0 (socket says 172.21.15.50:37090) 2024-05-08T18:05:06.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.044+0000 7ff82606d640 1 -- 172.21.15.50:0/2331776465 learned_addr learned my addr 172.21.15.50:0/2331776465 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.045+0000 7ff82606d640 1 -- 172.21.15.50:0/2331776465 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fee239bc30 con 0x55fee239b240 2024-05-08T18:05:06.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.045+0000 7ff82606d640 1 --2- 172.21.15.50:0/2331776465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee239b240 0x55fee239b660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff81807b040 tx=0x7ff81806c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72f1a68ee241608 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.045+0000 7ff709ffb640 1 -- 172.21.15.50:0/2331776465 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff818096e30 con 0x55fee239b240 2024-05-08T18:05:06.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.045+0000 7ff709ffb640 1 -- 172.21.15.50:0/2331776465 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff818076640 con 0x55fee239b240 2024-05-08T18:05:06.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.045+0000 7ff709ffb640 1 -- 172.21.15.50:0/2331776465 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff818095430 con 0x55fee239b240 2024-05-08T18:05:06.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.045+0000 7ff828575800 1 -- 172.21.15.50:0/2331776465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee239b240 msgr2=0x55fee239b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.045+0000 7ff828575800 1 --2- 172.21.15.50:0/2331776465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee239b240 0x55fee239b660 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff81807b040 tx=0x7ff81806c4c0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.045+0000 7ff828575800 1 -- 172.21.15.50:0/2331776465 shutdown_connections 2024-05-08T18:05:06.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.045+0000 7ff828575800 1 --2- 172.21.15.50:0/2331776465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee239b240 0x55fee239b660 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.045+0000 7ff828575800 1 -- 172.21.15.50:0/2331776465 >> 172.21.15.50:0/2331776465 conn(0x55fee23974c0 msgr2=0x55fee2395520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.045+0000 7ff828575800 1 -- 172.21.15.50:0/2331776465 shutdown_connections 2024-05-08T18:05:06.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.045+0000 7ff828575800 1 -- 172.21.15.50:0/2331776465 wait complete. 2024-05-08T18:05:06.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.046+0000 7ff828575800 1 Processor -- start 2024-05-08T18:05:06.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.046+0000 7ff828575800 1 -- start start 2024-05-08T18:05:06.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.046+0000 7ff828575800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee239b240 0x55fee23a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.046+0000 7ff828575800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fee239c2a0 con 0x55fee239b240 2024-05-08T18:05:06.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.046+0000 7ff82606d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee239b240 0x55fee23a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.046+0000 7ff82606d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee239b240 0x55fee23a25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37106/0 (socket says 172.21.15.50:37106) 2024-05-08T18:05:06.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.046+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 learned_addr learned my addr 172.21.15.50:0/3841935843 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.046+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fee23a2af0 con 0x55fee239b240 2024-05-08T18:05:06.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.047+0000 7ff82606d640 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee239b240 0x55fee23a25b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff8180940a0 tx=0x7ff818062f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.047+0000 7ff6f7fff640 1 -- 172.21.15.50:0/3841935843 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8180b3070 con 0x55fee239b240 2024-05-08T18:05:06.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.047+0000 7ff6f7fff640 1 -- 172.21.15.50:0/3841935843 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff818077710 con 0x55fee239b240 2024-05-08T18:05:06.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.047+0000 7ff6f7fff640 1 -- 172.21.15.50:0/3841935843 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff818077a30 con 0x55fee239b240 2024-05-08T18:05:06.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.047+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fee23a2af0 con 0x55fee239b240 2024-05-08T18:05:06.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.047+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fee23a2af0 con 0x55fee239b240 2024-05-08T18:05:06.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.047+0000 7ff6f7fff640 1 -- 172.21.15.50:0/3841935843 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff8180be060 con 0x55fee239b240 2024-05-08T18:05:06.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.048+0000 7ff6f7fff640 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff6f003bb50 0x7ff6f003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.048+0000 7ff6f7fff640 1 -- 172.21.15.50:0/3841935843 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7ff8180a9020 con 0x55fee239b240 2024-05-08T18:05:06.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.048+0000 7ff82586c640 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff6f003bb50 0x7ff6f003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.048+0000 7ff82586c640 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff6f003bb50 0x7ff6f003e010 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55fee2380160 tx=0x7ff81c010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.048+0000 7ff828575800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.048+0000 7ff828575800 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fee23adbd0 0x55fee23b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.048+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b07e0 con 0x55fee23adbd0 2024-05-08T18:05:06.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.049+0000 7ff82686e640 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fee23adbd0 0x55fee23b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.049+0000 7ff82686e640 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fee23adbd0 0x55fee23b0090 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.049+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee23adbd0 2024-05-08T18:05:06.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.049+0000 7ff828575800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.049+0000 7ff828575800 20 realm 2024-05-08T18:05:06.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.049+0000 7ff828575800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.049+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b0bf0 con 0x55fee23adbd0 2024-05-08T18:05:06.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.049+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee23adbd0 2024-05-08T18:05:06.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.049+0000 7ff828575800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.049+0000 7ff828575800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:06.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.049+0000 7ff828575800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.049+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b14c0 con 0x55fee23adbd0 2024-05-08T18:05:06.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.050+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee23adbd0 2024-05-08T18:05:06.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.050+0000 7ff828575800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.050+0000 7ff828575800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.050+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23adbd0 2024-05-08T18:05:06.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.050+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee23adbd0 2024-05-08T18:05:06.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.050+0000 7ff828575800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:06.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.050+0000 7ff828575800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.050+0000 7ff828575800 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fee23b25c0 0x55fee23b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.050+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b4f70 con 0x55fee23b25c0 2024-05-08T18:05:06.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.050+0000 7ff82606d640 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fee23b25c0 0x55fee23b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.050+0000 7ff82606d640 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fee23b25c0 0x55fee23b49b0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.051+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.051+0000 7ff828575800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:06.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.051+0000 7ff828575800 20 searching for the correct realm 2024-05-08T18:05:06.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.051+0000 7ff828575800 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fee23b6f20 0x55fee23b93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.051+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b9ab0 con 0x55fee23b6f20 2024-05-08T18:05:06.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.051+0000 7ff82586c640 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fee23b6f20 0x55fee23b93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.051+0000 7ff82586c640 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fee23b6f20 0x55fee23b93e0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.051+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff81c037040 con 0x55fee23b6f20 2024-05-08T18:05:06.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.051+0000 7ff82586c640 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff81c00f070 0x7ff81c03a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.051+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b9ab0 con 0x7ff81c00f070 2024-05-08T18:05:06.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.052+0000 7ff82686e640 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff81c00f070 0x7ff81c03a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.052+0000 7ff82686e640 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff81c00f070 0x7ff81c03a3c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.052+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82000a4a0 con 0x7ff81c00f070 2024-05-08T18:05:06.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.052+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b25c0 2024-05-08T18:05:06.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.052+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.052+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23adbd0 2024-05-08T18:05:06.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.053+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee23adbd0 2024-05-08T18:05:06.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.053+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23adbd0 2024-05-08T18:05:06.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.053+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee23adbd0 2024-05-08T18:05:06.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.053+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b25c0 2024-05-08T18:05:06.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.053+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.053+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b25c0 2024-05-08T18:05:06.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.053+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.053+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x7ff81c00f070 2024-05-08T18:05:06.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.054+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82000a4a0 con 0x7ff81c00f070 2024-05-08T18:05:06.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.054+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b6f20 2024-05-08T18:05:06.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.054+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff81c037040 con 0x55fee23b6f20 2024-05-08T18:05:06.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.054+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23adbd0 2024-05-08T18:05:06.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.054+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee23adbd0 2024-05-08T18:05:06.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.054+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b25c0 2024-05-08T18:05:06.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.054+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.054+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23adbd0 2024-05-08T18:05:06.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.054+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee23adbd0 2024-05-08T18:05:06.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.055+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x7ff81c00f070 2024-05-08T18:05:06.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.055+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff82000a4a0 con 0x7ff81c00f070 2024-05-08T18:05:06.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.055+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b6f20 2024-05-08T18:05:06.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.055+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff81c037040 con 0x55fee23b6f20 2024-05-08T18:05:06.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.055+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b6f20 2024-05-08T18:05:06.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.055+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff81c037040 con 0x55fee23b6f20 2024-05-08T18:05:06.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.055+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b25c0 2024-05-08T18:05:06.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.056+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.056+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b6f20 2024-05-08T18:05:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.056+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff81c037040 con 0x55fee23b6f20 2024-05-08T18:05:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.056+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b6f20 2024-05-08T18:05:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.056+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ff81c037040 con 0x55fee23b6f20 2024-05-08T18:05:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.056+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23adbd0 2024-05-08T18:05:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.056+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee23adbd0 2024-05-08T18:05:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.056+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23adbd0 2024-05-08T18:05:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.057+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee23adbd0 2024-05-08T18:05:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.057+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b25c0 2024-05-08T18:05:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.057+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.057+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b6f20 2024-05-08T18:05:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.057+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff81c037040 con 0x55fee23b6f20 2024-05-08T18:05:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.057+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x7ff81c00f070 2024-05-08T18:05:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.057+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82000a4a0 con 0x7ff81c00f070 2024-05-08T18:05:06.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.057+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23adbd0 2024-05-08T18:05:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.058+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee23adbd0 2024-05-08T18:05:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.058+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b25c0 2024-05-08T18:05:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.058+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.058+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b6f20 2024-05-08T18:05:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.058+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff81c037040 con 0x55fee23b6f20 2024-05-08T18:05:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.058+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b25c0 2024-05-08T18:05:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.058+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.059+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x7ff81c00f070 2024-05-08T18:05:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.059+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82000a4a0 con 0x7ff81c00f070 2024-05-08T18:05:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.059+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b6f20 2024-05-08T18:05:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.059+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff81c037040 con 0x55fee23b6f20 2024-05-08T18:05:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.059+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23b25c0 2024-05-08T18:05:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.059+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.059+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x55fee23adbd0 2024-05-08T18:05:06.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee23adbd0 2024-05-08T18:05:06.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b1860 con 0x7ff81c00f070 2024-05-08T18:05:06.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff82000a4a0 con 0x7ff81c00f070 2024-05-08T18:05:06.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff828575800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:06.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff828575800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:06.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff828575800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:06.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff828575800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:06.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff828575800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:06.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff828575800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:06.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff828575800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b5830 con 0x55fee23adbd0 2024-05-08T18:05:06.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee23adbd0 2024-05-08T18:05:06.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff828575800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff828575800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b5830 con 0x55fee23adbd0 2024-05-08T18:05:06.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff82000a4a0 con 0x55fee23adbd0 2024-05-08T18:05:06.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff828575800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:06.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff828575800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.060+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b5830 con 0x55fee23b6f20 2024-05-08T18:05:06.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff81c037040 con 0x55fee23b6f20 2024-05-08T18:05:06.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff828575800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:06.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff828575800 20 zone default found 2024-05-08T18:05:06.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff828575800 4 Realm: () 2024-05-08T18:05:06.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff828575800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:06.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff828575800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:06.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff828575800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:06.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff828575800 20 zonegroup default 2024-05-08T18:05:06.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff828575800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23b61d0 con 0x55fee23b25c0 2024-05-08T18:05:06.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff828575800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff828575800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23bb9a0 con 0x55fee23b6f20 2024-05-08T18:05:06.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff81c037040 con 0x55fee23b6f20 2024-05-08T18:05:06.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff828575800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff828575800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.061+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23bbe70 con 0x55fee23b6f20 2024-05-08T18:05:06.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.062+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff81c037040 con 0x55fee23b6f20 2024-05-08T18:05:06.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.062+0000 7ff828575800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.062+0000 7ff828575800 20 started sync module instance, tier type = 2024-05-08T18:05:06.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.062+0000 7ff828575800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:06.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.062+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23badb0 con 0x7ff81c00f070 2024-05-08T18:05:06.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.063+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff82000a4a0 con 0x7ff81c00f070 2024-05-08T18:05:06.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.063+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94553205617328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23badb0 con 0x7ff81c00f070 2024-05-08T18:05:06.074 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.063+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23bc9a0 con 0x55fee23adbd0 2024-05-08T18:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.064+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94553205617328] v27'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820040040 con 0x7ff81c00f070 2024-05-08T18:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.064+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820040040 con 0x55fee23adbd0 2024-05-08T18:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.064+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94553205621872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23badb0 con 0x55fee23adbd0 2024-05-08T18:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.064+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23bd450 con 0x55fee23b6f20 2024-05-08T18:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.065+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94553205621872] v27'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820040040 con 0x55fee23adbd0 2024-05-08T18:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.065+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff81c037040 con 0x55fee23b6f20 2024-05-08T18:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.065+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94553205614848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23bad20 con 0x55fee23b6f20 2024-05-08T18:05:06.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.065+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23be310 con 0x7ff81c00f070 2024-05-08T18:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.066+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94553205614848] v27'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff81c037040 con 0x55fee23b6f20 2024-05-08T18:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.066+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820040040 con 0x7ff81c00f070 2024-05-08T18:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.066+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94553205628512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23be280 con 0x7ff81c00f070 2024-05-08T18:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.066+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23bf080 con 0x55fee23adbd0 2024-05-08T18:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.067+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820040040 con 0x55fee23adbd0 2024-05-08T18:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.068+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94553205628512] v27'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820040040 con 0x7ff81c00f070 2024-05-08T18:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.068+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94553205631920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23bf080 con 0x55fee23adbd0 2024-05-08T18:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.068+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23be740 con 0x55fee23b25c0 2024-05-08T18:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.068+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.069+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94553205635776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23c0390 con 0x55fee23b25c0 2024-05-08T18:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.069+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23c0d90 con 0x55fee23b25c0 2024-05-08T18:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.069+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94553205631920] v27'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820040040 con 0x55fee23adbd0 2024-05-08T18:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.070+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94553205635776] v27'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.070+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.070+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94553205634160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23c1130 con 0x55fee23b25c0 2024-05-08T18:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.070+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23c19c0 con 0x55fee23b25c0 2024-05-08T18:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.071+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94553205634160] v27'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.071+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.071+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94553205639328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23c1c60 con 0x55fee23b25c0 2024-05-08T18:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.071+0000 7ff828575800 20 add_watcher() i=0 2024-05-08T18:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.071+0000 7ff828575800 20 add_watcher() i=1 2024-05-08T18:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.071+0000 7ff828575800 20 add_watcher() i=2 2024-05-08T18:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.071+0000 7ff828575800 20 add_watcher() i=3 2024-05-08T18:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.071+0000 7ff828575800 20 add_watcher() i=4 2024-05-08T18:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.071+0000 7ff828575800 20 add_watcher() i=5 2024-05-08T18:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.071+0000 7ff828575800 20 add_watcher() i=6 2024-05-08T18:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.072+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94553205639328] v27'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.072+0000 7ff828575800 20 add_watcher() i=7 2024-05-08T18:05:06.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.072+0000 7ff828575800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:06.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.072+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23c51a0 con 0x55fee23adbd0 2024-05-08T18:05:06.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.072+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff820040040 con 0x55fee23adbd0 2024-05-08T18:05:06.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.072+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94553205642512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23c2660 con 0x55fee23adbd0 2024-05-08T18:05:06.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.073+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94553205642512] v27'82 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff820040040 con 0x55fee23adbd0 2024-05-08T18:05:06.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.073+0000 7ff6dcfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:06.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.073+0000 7ff828575800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:06.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.073+0000 7ff828575800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:06.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.074+0000 7ff828575800 5 note: GC not initialized 2024-05-08T18:05:06.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.074+0000 7ff6577fe640 20 reqs_thread_entry: start 2024-05-08T18:05:06.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.074+0000 7ff828575800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:06.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.074+0000 7ff828575800 20 Filter name: none 2024-05-08T18:05:06.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.074+0000 7ff6557fa640 20 reqs_thread_entry: start 2024-05-08T18:05:06.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.074+0000 7ff828575800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:06.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.074+0000 7ff828575800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.074+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23f86b0 con 0x55fee23adbd0 2024-05-08T18:05:06.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.074+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~673 out=673b,getxattrs out=75b] v0'0 uv8 ondisk = 0) ==== 231+0+796 (crc 0 0 0) 0x7ff820040040 con 0x55fee23adbd0 2024-05-08T18:05:06.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.074+0000 7ff828575800 20 rados_obj.operate() r=0 bl.length=673 2024-05-08T18:05:06.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.074+0000 7ff828575800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:06.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.074+0000 7ff828575800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:06.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.074+0000 7ff828575800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:06.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.075+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23ff350 con 0x55fee23adbd0 2024-05-08T18:05:06.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.076+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~582] v27'9 uv9 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7ff820040040 con 0x55fee23adbd0 2024-05-08T18:05:06.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.076+0000 7ff828575800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:06.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.076+0000 7ff828575800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:06.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.076+0000 7ff828575800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-05-08T18:05:06.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.076+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94553205894416 in=770b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23fa9f0 con 0x55fee23adbd0 2024-05-08T18:05:06.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.076+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== watch-notify(notify (1) cookie 94553205631920 notify 115964117000 ret 0) ==== 800+0+0 (crc 0 0 0) 0x7ff820018a30 con 0x55fee23adbd0 2024-05-08T18:05:06.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.076+0000 7ff709ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117000 cookie 94553205631920 notifier 4396 bl.length()=758 2024-05-08T18:05:06.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.076+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(64 notify.4 [notify cookie 94553205894416 out=8b] v0'0 uv72 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff820040040 con 0x55fee23adbd0 2024-05-08T18:05:06.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.076+0000 7ff709ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:06.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.076+0000 7ff709ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:06.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.076+0000 7ff709ffb640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7ff6ec003ea0 con 0x55fee23adbd0 2024-05-08T18:05:06.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.076+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820040040 con 0x55fee23adbd0 2024-05-08T18:05:06.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.076+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== watch-notify(notify_complete (2) cookie 94553205894416 notify 115964117000 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7ff820018c00 con 0x55fee23adbd0 2024-05-08T18:05:06.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.076+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23fed10 con 0x55fee23adbd0 2024-05-08T18:05:06.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.077+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(66 foo:foo1 [delete] v27'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820040040 con 0x55fee23adbd0 2024-05-08T18:05:06.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.077+0000 7ff828575800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2024-05-08T18:05:06.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.077+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94553205894416 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55fee23f9830 con 0x7ff81c00f070 2024-05-08T18:05:06.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.077+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== watch-notify(notify (1) cookie 94553205617328 notify 115964116996 ret 0) ==== 201+0+0 (crc 0 0 0) 0x7ff820018c00 con 0x7ff81c00f070 2024-05-08T18:05:06.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.077+0000 7ff709ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116996 cookie 94553205617328 notifier 4396 bl.length()=159 2024-05-08T18:05:06.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.077+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(67 notify.0 [notify cookie 94553205894416 out=8b] v0'0 uv72 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff820040040 con 0x7ff81c00f070 2024-05-08T18:05:06.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.077+0000 7ff709ffb640 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7ff6ec003ea0 con 0x7ff81c00f070 2024-05-08T18:05:06.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.078+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== watch-notify(notify_complete (2) cookie 94553205894416 notify 115964116996 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7ff820038520 con 0x7ff81c00f070 2024-05-08T18:05:06.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.078+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820040040 con 0x7ff81c00f070 2024-05-08T18:05:06.086 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"},{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy","active":true,"create_date":"2024-05-08T18:05:05.808771Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:06.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.079+0000 7ff6dcfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:06.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.079+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94553205642512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23fadc0 con 0x55fee23adbd0 2024-05-08T18:05:06.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.080+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94553205642512] v27'83 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff820040040 con 0x55fee23adbd0 2024-05-08T18:05:06.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.080+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94553205617328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23c14d0 con 0x7ff81c00f070 2024-05-08T18:05:06.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.081+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94553205617328] v27'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820040040 con 0x7ff81c00f070 2024-05-08T18:05:06.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.081+0000 7ff828575800 20 remove_watcher() i=0 2024-05-08T18:05:06.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.081+0000 7ff828575800 2 removed watcher, disabling cache 2024-05-08T18:05:06.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.081+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94553205621872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23bb2b0 con 0x55fee23adbd0 2024-05-08T18:05:06.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.082+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94553205621872] v27'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820040040 con 0x55fee23adbd0 2024-05-08T18:05:06.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.082+0000 7ff828575800 20 remove_watcher() i=1 2024-05-08T18:05:06.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.082+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94553205614848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23bb2b0 con 0x55fee23b6f20 2024-05-08T18:05:06.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.083+0000 7ff82586c640 1 -- 172.21.15.50:0/3841935843 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94553205614848] v27'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff81c037040 con 0x55fee23b6f20 2024-05-08T18:05:06.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.083+0000 7ff828575800 20 remove_watcher() i=2 2024-05-08T18:05:06.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.083+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94553205628512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23ba900 con 0x7ff81c00f070 2024-05-08T18:05:06.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.084+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94553205628512] v27'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820040040 con 0x7ff81c00f070 2024-05-08T18:05:06.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.084+0000 7ff828575800 20 remove_watcher() i=3 2024-05-08T18:05:06.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.084+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94553205631920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23bde60 con 0x55fee23adbd0 2024-05-08T18:05:06.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.085+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94553205631920] v27'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff820040040 con 0x55fee23adbd0 2024-05-08T18:05:06.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.085+0000 7ff828575800 20 remove_watcher() i=4 2024-05-08T18:05:06.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.085+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94553205635776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23ba900 con 0x55fee23b25c0 2024-05-08T18:05:06.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.086+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94553205635776] v27'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.086+0000 7ff828575800 20 remove_watcher() i=5 2024-05-08T18:05:06.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.086+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94553205634160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23bebb0 con 0x55fee23b25c0 2024-05-08T18:05:06.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.087+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94553205634160] v27'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.087+0000 7ff828575800 20 remove_watcher() i=6 2024-05-08T18:05:06.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.087+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94553205639328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55fee23bde60 con 0x55fee23b25c0 2024-05-08T18:05:06.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff82606d640 1 -- 172.21.15.50:0/3841935843 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94553205639328] v27'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff818083070 con 0x55fee23b25c0 2024-05-08T18:05:06.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff828575800 20 remove_watcher() i=7 2024-05-08T18:05:06.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fee23b25c0 msgr2=0x55fee23b49b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff828575800 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fee23b25c0 0x55fee23b49b0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fee23adbd0 msgr2=0x55fee23b0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff828575800 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fee23adbd0 0x55fee23b0090 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff81c00f070 msgr2=0x7ff81c03a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff828575800 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff81c00f070 0x7ff81c03a3c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fee23b6f20 msgr2=0x55fee23b93e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff828575800 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fee23b6f20 0x55fee23b93e0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff6f003bb50 msgr2=0x7ff6f003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff828575800 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff6f003bb50 0x7ff6f003e010 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55fee2380160 tx=0x7ff81c010000 comp rx=0 tx=0).stop 2024-05-08T18:05:06.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff82686e640 1 -- 172.21.15.50:0/3841935843 reap_dead start 2024-05-08T18:05:06.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee239b240 msgr2=0x55fee23a25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff828575800 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee239b240 0x55fee23a25b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff8180940a0 tx=0x7ff818062f70 comp rx=0 tx=0).stop 2024-05-08T18:05:06.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 shutdown_connections 2024-05-08T18:05:06.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff828575800 1 --2- 172.21.15.50:0/3841935843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee239b240 0x55fee23a25b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.088+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 >> 172.21.15.50:0/3841935843 conn(0x55fee23974c0 msgr2=0x55fee2394b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 shutdown_connections 2024-05-08T18:05:06.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 -- 172.21.15.50:0/3841935843 wait complete. 2024-05-08T18:05:06.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 -- 172.21.15.50:0/1886836873 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff70003bb00 msgr2=0x7ff70003dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 --2- 172.21.15.50:0/1886836873 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff70003bb00 0x7ff70003dfc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55fee23774b0 tx=0x7ff81c00f000 comp rx=0 tx=0).stop 2024-05-08T18:05:06.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 -- 172.21.15.50:0/1886836873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee238e0d0 msgr2=0x55fee238e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 --2- 172.21.15.50:0/1886836873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee238e0d0 0x55fee238e4f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff81806d000 tx=0x7ff81808bd80 comp rx=0 tx=0).stop 2024-05-08T18:05:06.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 -- 172.21.15.50:0/1886836873 shutdown_connections 2024-05-08T18:05:06.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 --2- 172.21.15.50:0/1886836873 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff70003bb00 0x7ff70003dfc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 --2- 172.21.15.50:0/1886836873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee238e0d0 0x55fee238e4f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 -- 172.21.15.50:0/1886836873 >> 172.21.15.50:0/1886836873 conn(0x55fee2382170 msgr2=0x55fee2382570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 -- 172.21.15.50:0/1886836873 shutdown_connections 2024-05-08T18:05:06.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 -- 172.21.15.50:0/1886836873 wait complete. 2024-05-08T18:05:06.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 -- 172.21.15.50:0/2427173713 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fee2379330 msgr2=0x55fee237b720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fee2379330 0x55fee237b720 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 -- 172.21.15.50:0/2427173713 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fee22747f0 msgr2=0x55fee2376b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fee22747f0 0x55fee2376b00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 -- 172.21.15.50:0/2427173713 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fee237d230 msgr2=0x55fee237f620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fee237d230 0x55fee237f620 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 -- 172.21.15.50:0/2427173713 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff70c03bba0 msgr2=0x7ff70c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff70c03bba0 0x7ff70c03e060 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff81c001e80 tx=0x7ff81c0013e0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 -- 172.21.15.50:0/2427173713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 msgr2=0x55fee236f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.089+0000 7ff828575800 1 --2- 172.21.15.50:0/2427173713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fee2273a70 0x55fee236f8c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff818052540 tx=0x7ff818003020 comp rx=0 tx=0).stop 2024-05-08T18:05:06.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.090+0000 7ff82686e640 1 -- 172.21.15.50:0/2427173713 reap_dead start 2024-05-08T18:05:06.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.090+0000 7ff828575800 1 -- 172.21.15.50:0/2427173713 shutdown_connections 2024-05-08T18:05:06.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.090+0000 7ff828575800 1 -- 172.21.15.50:0/2427173713 >> 172.21.15.50:0/2427173713 conn(0x55fee226e2f0 msgr2=0x55fee235f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.090+0000 7ff828575800 1 -- 172.21.15.50:0/2427173713 shutdown_connections 2024-05-08T18:05:06.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.090+0000 7ff828575800 1 -- 172.21.15.50:0/2427173713 wait complete. 2024-05-08T18:05:06.097 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True, 'create_date': '2024-05-08T18:05:05.808771Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:06.097 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo1'] 2024-05-08T18:05:06.097 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo1'] 2024-05-08T18:05:06.098 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo1 2024-05-08T18:05:06.143 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:06.143 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:06.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.146+0000 7f52eb72e800 1 Processor -- start 2024-05-08T18:05:06.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.146+0000 7f52eb72e800 1 -- start start 2024-05-08T18:05:06.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.146+0000 7f52eb72e800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55f9909baae0 0x55f9909bced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.146+0000 7f52eb72e800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55f9909bd4a0 con 0x55f9909b66f0 2024-05-08T18:05:06.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.146+0000 7f52eb72e800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55f990856e20 con 0x55f9909baae0 2024-05-08T18:05:06.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.146+0000 7f52e6c83640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55f9909baae0 0x55f9909bced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.146+0000 7f52e7484640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55f9909b66f0 0x55f9909b6b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43058/0 (socket says 172.21.15.50:43058) 2024-05-08T18:05:06.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.146+0000 7f52e7484640 1 -- 172.21.15.50:0/565757280 learned_addr learned my addr 172.21.15.50:0/565757280 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:06.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.146+0000 7f52e6c83640 1 -- 172.21.15.50:0/565757280 >> v1:172.21.15.46:6789/0 conn(0x55f9909b66f0 legacy=0x55f9909b6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.146+0000 7f52e6c83640 1 -- 172.21.15.50:0/565757280 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9909be6d0 con 0x55f9909baae0 2024-05-08T18:05:06.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.146+0000 7f52e6482640 1 -- 172.21.15.50:0/565757280 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2669099707 0 0) 0x55f9909bd4a0 con 0x55f9909b66f0 2024-05-08T18:05:06.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.148+0000 7f52e6c83640 1 --2- 172.21.15.50:0/565757280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909baae0 0x55f9909bced0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00a470 tx=0x7f52dc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8b1a5c42b5c4e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.148+0000 7f52e6482640 1 -- 172.21.15.50:0/565757280 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52dc057070 con 0x55f9909baae0 2024-05-08T18:05:06.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.148+0000 7f52e6482640 1 -- 172.21.15.50:0/565757280 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52dc052570 con 0x55f9909baae0 2024-05-08T18:05:06.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.148+0000 7f52e6482640 1 -- 172.21.15.50:0/565757280 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52dc052870 con 0x55f9909baae0 2024-05-08T18:05:06.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.149+0000 7f52eb72e800 1 -- 172.21.15.50:0/565757280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909baae0 msgr2=0x55f9909bced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.149+0000 7f52eb72e800 1 --2- 172.21.15.50:0/565757280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909baae0 0x55f9909bced0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f52dc00a470 tx=0x7f52dc01ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:06.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.149+0000 7f52eb72e800 1 -- 172.21.15.50:0/565757280 shutdown_connections 2024-05-08T18:05:06.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.149+0000 7f52eb72e800 1 --2- 172.21.15.50:0/565757280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909baae0 0x55f9909bced0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.149+0000 7f52eb72e800 1 -- 172.21.15.50:0/565757280 >> 172.21.15.50:0/565757280 conn(0x55f9909b1c70 msgr2=0x55f9909b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.149+0000 7f52eb72e800 1 -- 172.21.15.50:0/565757280 shutdown_connections 2024-05-08T18:05:06.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.149+0000 7f52eb72e800 1 -- 172.21.15.50:0/565757280 wait complete. 2024-05-08T18:05:06.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.164+0000 7f52eb72e800 1 Processor -- start 2024-05-08T18:05:06.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.165+0000 7f52eb72e800 1 -- start start 2024-05-08T18:05:06.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.165+0000 7f52eb72e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909b66f0 0x55f990aa79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.165+0000 7f52eb72e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f990901a10 con 0x55f9909b66f0 2024-05-08T18:05:06.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.165+0000 7f52e7484640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909b66f0 0x55f990aa79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.165+0000 7f52e7484640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909b66f0 0x55f990aa79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37128/0 (socket says 172.21.15.50:37128) 2024-05-08T18:05:06.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.165+0000 7f52e7484640 1 -- 172.21.15.50:0/81900162 learned_addr learned my addr 172.21.15.50:0/81900162 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.165+0000 7f52e7484640 1 -- 172.21.15.50:0/81900162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f990aa7f70 con 0x55f9909b66f0 2024-05-08T18:05:06.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.166+0000 7f52e7484640 1 --2- 172.21.15.50:0/81900162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909b66f0 0x55f990aa79a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f52d8001ea0 tx=0x7f52d800b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314e33a345084da3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.166+0000 7f51d560a640 1 -- 172.21.15.50:0/81900162 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52d8008070 con 0x55f9909b66f0 2024-05-08T18:05:06.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.166+0000 7f51d560a640 1 -- 172.21.15.50:0/81900162 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d800d500 con 0x55f9909b66f0 2024-05-08T18:05:06.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.166+0000 7f51d560a640 1 -- 172.21.15.50:0/81900162 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52d800d880 con 0x55f9909b66f0 2024-05-08T18:05:06.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.166+0000 7f52eb72e800 1 -- 172.21.15.50:0/81900162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909b66f0 msgr2=0x55f990aa79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.166+0000 7f52eb72e800 1 --2- 172.21.15.50:0/81900162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909b66f0 0x55f990aa79a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f52d8001ea0 tx=0x7f52d800b6a0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.166+0000 7f52eb72e800 1 -- 172.21.15.50:0/81900162 shutdown_connections 2024-05-08T18:05:06.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.166+0000 7f52eb72e800 1 --2- 172.21.15.50:0/81900162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909b66f0 0x55f990aa79a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.167+0000 7f52eb72e800 1 -- 172.21.15.50:0/81900162 >> 172.21.15.50:0/81900162 conn(0x55f9909b1c70 msgr2=0x55f990aa32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.167+0000 7f52eb72e800 1 -- 172.21.15.50:0/81900162 shutdown_connections 2024-05-08T18:05:06.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.167+0000 7f52eb72e800 1 -- 172.21.15.50:0/81900162 wait complete. 2024-05-08T18:05:06.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.167+0000 7f52eb72e800 1 Processor -- start 2024-05-08T18:05:06.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.167+0000 7f52eb72e800 1 -- start start 2024-05-08T18:05:06.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.167+0000 7f52eb72e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909b66f0 0x55f990ab16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.167+0000 7f52eb72e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f9909bd970 con 0x55f9909b66f0 2024-05-08T18:05:06.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.167+0000 7f52e7484640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909b66f0 0x55f990ab16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.167+0000 7f52e7484640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909b66f0 0x55f990ab16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37144/0 (socket says 172.21.15.50:37144) 2024-05-08T18:05:06.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.167+0000 7f52e7484640 1 -- 172.21.15.50:0/1540520817 learned_addr learned my addr 172.21.15.50:0/1540520817 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.168+0000 7f52e7484640 1 -- 172.21.15.50:0/1540520817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f990ab1c30 con 0x55f9909b66f0 2024-05-08T18:05:06.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.168+0000 7f52e7484640 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909b66f0 0x55f990ab16f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f52d80012f0 tx=0x7f52d8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.168+0000 7f51d3e07640 1 -- 172.21.15.50:0/1540520817 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52d800f9d0 con 0x55f9909b66f0 2024-05-08T18:05:06.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.168+0000 7f51d3e07640 1 -- 172.21.15.50:0/1540520817 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d800fb70 con 0x55f9909b66f0 2024-05-08T18:05:06.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.168+0000 7f51d3e07640 1 -- 172.21.15.50:0/1540520817 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52d800fe50 con 0x55f9909b66f0 2024-05-08T18:05:06.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.168+0000 7f52eb72e800 1 -- 172.21.15.50:0/1540520817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f990ab1ec0 con 0x55f9909b66f0 2024-05-08T18:05:06.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.168+0000 7f52eb72e800 1 -- 172.21.15.50:0/1540520817 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f990ab4b70 con 0x55f9909b66f0 2024-05-08T18:05:06.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.169+0000 7f51d3e07640 1 -- 172.21.15.50:0/1540520817 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f52d8017040 con 0x55f9909b66f0 2024-05-08T18:05:06.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.169+0000 7f51d3e07640 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f51cc03bcf0 0x7f51cc03e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.169+0000 7f51d3e07640 1 -- 172.21.15.50:0/1540520817 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f52d8015020 con 0x55f9909b66f0 2024-05-08T18:05:06.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.169+0000 7f52e6c83640 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f51cc03bcf0 0x7f51cc03e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.169+0000 7f52eb72e800 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f9909baae0 0x55f990ab8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.169+0000 7f52eb72e800 1 -- 172.21.15.50:0/1540520817 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990ab8eb0 con 0x55f9909baae0 2024-05-08T18:05:06.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.169+0000 7f52e7c85640 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f9909baae0 0x55f990ab8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.170+0000 7f52e6c83640 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f51cc03bcf0 0x7f51cc03e1b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f52dc054000 tx=0x7f52dc04e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.170+0000 7f52e7c85640 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f9909baae0 0x55f990ab8930 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.170+0000 7f52e7c85640 1 -- 172.21.15.50:0/1540520817 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f9909baae0 2024-05-08T18:05:06.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.170+0000 7f52eb72e800 1 -- 172.21.15.50:0/1540520817 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990ab5f00 con 0x55f9909baae0 2024-05-08T18:05:06.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.170+0000 7f52e7c85640 1 -- 172.21.15.50:0/1540520817 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f9909baae0 2024-05-08T18:05:06.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.170+0000 7f52eb72e800 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f990abb1e0 0x55f990abd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.170+0000 7f52eb72e800 1 -- 172.21.15.50:0/1540520817 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990abdcc0 con 0x55f990abb1e0 2024-05-08T18:05:06.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.171+0000 7f52e7484640 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f990abb1e0 0x55f990abd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.171+0000 7f52e7484640 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f990abb1e0 0x55f990abd5d0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.171+0000 7f52e7484640 1 -- 172.21.15.50:0/1540520817 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f52d802d070 con 0x55f990abb1e0 2024-05-08T18:05:06.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.171+0000 7f52eb72e800 1 -- 172.21.15.50:0/1540520817 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990aba080 con 0x55f9909baae0 2024-05-08T18:05:06.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.171+0000 7f52e7c85640 1 -- 172.21.15.50:0/1540520817 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f9909baae0 2024-05-08T18:05:06.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.171+0000 7f52eb72e800 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f990abf120 0x55f990ac1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.171+0000 7f52eb72e800 1 -- 172.21.15.50:0/1540520817 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990ac1c10 con 0x55f990abf120 2024-05-08T18:05:06.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.172+0000 7f52e6c83640 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f990abf120 0x55f990ac1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.172+0000 7f52e6c83640 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f990abf120 0x55f990ac1510 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.172+0000 7f52e6c83640 1 -- 172.21.15.50:0/1540520817 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f52dc052940 con 0x55f990abf120 2024-05-08T18:05:06.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.172+0000 7f52eb72e800 1 Processor -- start 2024-05-08T18:05:06.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.173+0000 7f52eb72e800 1 -- start start 2024-05-08T18:05:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.173+0000 7f52eb72e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990acff80 0x55f990ad03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.173+0000 7f52eb72e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f9908a8330 con 0x55f990acff80 2024-05-08T18:05:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.173+0000 7f52e7484640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990acff80 0x55f990ad03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.173+0000 7f52e7484640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990acff80 0x55f990ad03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37150/0 (socket says 172.21.15.50:37150) 2024-05-08T18:05:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.173+0000 7f52e7484640 1 -- 172.21.15.50:0/3792950436 learned_addr learned my addr 172.21.15.50:0/3792950436 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.173+0000 7f52e7484640 1 -- 172.21.15.50:0/3792950436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f990ad08e0 con 0x55f990acff80 2024-05-08T18:05:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.173+0000 7f52e7484640 1 --2- 172.21.15.50:0/3792950436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990acff80 0x55f990ad03a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f52d800b6c0 tx=0x7f52d801ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.173+0000 7f51d1602640 1 -- 172.21.15.50:0/3792950436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52d804a370 con 0x55f990acff80 2024-05-08T18:05:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.173+0000 7f51d1602640 1 -- 172.21.15.50:0/3792950436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d8047070 con 0x55f990acff80 2024-05-08T18:05:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.173+0000 7f51d1602640 1 -- 172.21.15.50:0/3792950436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52d800e840 con 0x55f990acff80 2024-05-08T18:05:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.173+0000 7f52eb72e800 1 -- 172.21.15.50:0/3792950436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f990ad1350 con 0x55f990acff80 2024-05-08T18:05:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.174+0000 7f52eb72e800 1 -- 172.21.15.50:0/3792950436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f990ad1850 con 0x55f990acff80 2024-05-08T18:05:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.174+0000 7f51d1602640 1 -- 172.21.15.50:0/3792950436 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f52d80183d0 con 0x55f990acff80 2024-05-08T18:05:06.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.174+0000 7f51d1602640 1 --2- 172.21.15.50:0/3792950436 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f51c003bb00 0x7f51c003dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.175+0000 7f51d1602640 1 -- 172.21.15.50:0/3792950436 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f52d803bf20 con 0x55f990acff80 2024-05-08T18:05:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.175+0000 7f52e6c83640 1 --2- 172.21.15.50:0/3792950436 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f51c003bb00 0x7f51c003dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.175+0000 7f52eb72e800 1 Processor -- start 2024-05-08T18:05:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.175+0000 7f52eb72e800 1 -- start start 2024-05-08T18:05:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.175+0000 7f52e6c83640 1 --2- 172.21.15.50:0/3792950436 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f51c003bb00 0x7f51c003dfc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55f990ab9790 tx=0x7f52dc005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.175+0000 7f52eb72e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990add220 0x55f990add640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.175+0000 7f52eb72e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f990aa82f0 con 0x55f990add220 2024-05-08T18:05:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.175+0000 7f52e7484640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990add220 0x55f990add640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.175+0000 7f52e7484640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990add220 0x55f990add640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37154/0 (socket says 172.21.15.50:37154) 2024-05-08T18:05:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.175+0000 7f52e7484640 1 -- 172.21.15.50:0/373418633 learned_addr learned my addr 172.21.15.50:0/373418633 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.175+0000 7f52e7484640 1 -- 172.21.15.50:0/373418633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f990addc10 con 0x55f990add220 2024-05-08T18:05:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.176+0000 7f52e7484640 1 --2- 172.21.15.50:0/373418633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990add220 0x55f990add640 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f52d8030fd0 tx=0x7f52d8041680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79d5d6d1a4e920a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.176+0000 7f51caffd640 1 -- 172.21.15.50:0/373418633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52d8035070 con 0x55f990add220 2024-05-08T18:05:06.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.176+0000 7f51caffd640 1 -- 172.21.15.50:0/373418633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d804a630 con 0x55f990add220 2024-05-08T18:05:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.176+0000 7f51caffd640 1 -- 172.21.15.50:0/373418633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52d8030480 con 0x55f990add220 2024-05-08T18:05:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.176+0000 7f52eb72e800 1 -- 172.21.15.50:0/373418633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990add220 msgr2=0x55f990add640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.176+0000 7f52eb72e800 1 --2- 172.21.15.50:0/373418633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990add220 0x55f990add640 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f52d8030fd0 tx=0x7f52d8041680 comp rx=0 tx=0).stop 2024-05-08T18:05:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.176+0000 7f52eb72e800 1 -- 172.21.15.50:0/373418633 shutdown_connections 2024-05-08T18:05:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.176+0000 7f52eb72e800 1 --2- 172.21.15.50:0/373418633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990add220 0x55f990add640 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.176+0000 7f52eb72e800 1 -- 172.21.15.50:0/373418633 >> 172.21.15.50:0/373418633 conn(0x55f990ad9580 msgr2=0x55f990ad75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.176+0000 7f52eb72e800 1 -- 172.21.15.50:0/373418633 shutdown_connections 2024-05-08T18:05:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.176+0000 7f52eb72e800 1 -- 172.21.15.50:0/373418633 wait complete. 2024-05-08T18:05:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.177+0000 7f52eb72e800 1 Processor -- start 2024-05-08T18:05:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.177+0000 7f52eb72e800 1 -- start start 2024-05-08T18:05:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.177+0000 7f52eb72e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990add220 0x55f990ae4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.177+0000 7f52eb72e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f990ade2c0 con 0x55f990add220 2024-05-08T18:05:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.177+0000 7f52e7484640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990add220 0x55f990ae4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.177+0000 7f52e7484640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990add220 0x55f990ae4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37166/0 (socket says 172.21.15.50:37166) 2024-05-08T18:05:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.177+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 learned_addr learned my addr 172.21.15.50:0/3749481852 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.177+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f990ae4b80 con 0x55f990add220 2024-05-08T18:05:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.177+0000 7f52e7484640 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990add220 0x55f990ae4640 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f52d8032000 tx=0x7f52d8004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.178+0000 7f51c97fa640 1 -- 172.21.15.50:0/3749481852 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52d806d070 con 0x55f990add220 2024-05-08T18:05:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.178+0000 7f51c97fa640 1 -- 172.21.15.50:0/3749481852 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d80269c0 con 0x55f990add220 2024-05-08T18:05:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.178+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f990ae4b80 con 0x55f990add220 2024-05-08T18:05:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.178+0000 7f51c97fa640 1 -- 172.21.15.50:0/3749481852 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52d8027040 con 0x55f990add220 2024-05-08T18:05:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.178+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f990ae5080 con 0x55f990add220 2024-05-08T18:05:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.178+0000 7f51c97fa640 1 -- 172.21.15.50:0/3749481852 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f52d8070070 con 0x55f990add220 2024-05-08T18:05:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.179+0000 7f51c97fa640 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f51b003bba0 0x7f51b003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.179+0000 7f51c97fa640 1 -- 172.21.15.50:0/3749481852 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f52d8059070 con 0x55f990add220 2024-05-08T18:05:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.179+0000 7f52e6c83640 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f51b003bba0 0x7f51b003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.179+0000 7f52e6c83640 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f51b003bba0 0x7f51b003e060 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f52dc052fd0 tx=0x7f52dc04f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.179+0000 7f52eb72e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.179+0000 7f52eb72e800 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f990aefd90 0x55f990af2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.179+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990af29a0 con 0x55f990aefd90 2024-05-08T18:05:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.180+0000 7f52e7c85640 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f990aefd90 0x55f990af2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.180+0000 7f52e7c85640 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f990aefd90 0x55f990af2250 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.180+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f990aefd90 2024-05-08T18:05:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.180+0000 7f52eb72e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.180+0000 7f52eb72e800 20 realm 2024-05-08T18:05:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.180+0000 7f52eb72e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.180+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990af2db0 con 0x55f990aefd90 2024-05-08T18:05:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.180+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f990aefd90 2024-05-08T18:05:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.180+0000 7f52eb72e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.180+0000 7f52eb72e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.180+0000 7f52eb72e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.180+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990af3680 con 0x55f990aefd90 2024-05-08T18:05:06.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.181+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f990aefd90 2024-05-08T18:05:06.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.181+0000 7f52eb72e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.181+0000 7f52eb72e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.181+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990af3a20 con 0x55f990aefd90 2024-05-08T18:05:06.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.181+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f990aefd90 2024-05-08T18:05:06.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.181+0000 7f52eb72e800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:06.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.181+0000 7f52eb72e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.181+0000 7f52eb72e800 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f990af4780 0x55f990af6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.181+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990af7180 con 0x55f990af4780 2024-05-08T18:05:06.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.181+0000 7f52e7484640 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f990af4780 0x55f990af6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.181+0000 7f52e7484640 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f990af4780 0x55f990af6b70 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.182+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.182+0000 7f52eb72e800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:06.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.182+0000 7f52eb72e800 20 searching for the correct realm 2024-05-08T18:05:06.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.182+0000 7f52eb72e800 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f990af9130 0x55f990afb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.182+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f990afbcc0 con 0x55f990af9130 2024-05-08T18:05:06.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.182+0000 7f52e6c83640 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f990af9130 0x55f990afb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.182+0000 7f52e6c83640 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f990af9130 0x55f990afb5f0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.182+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52dc081070 con 0x55f990af9130 2024-05-08T18:05:06.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.182+0000 7f52e6c83640 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f52dc076040 0x7f52dc083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.182+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f990afbcc0 con 0x7f52dc076040 2024-05-08T18:05:06.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.182+0000 7f52e7c85640 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f52dc076040 0x7f52dc083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.183+0000 7f52e7c85640 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f52dc076040 0x7f52dc083410 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.183+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52e000a4a0 con 0x7f52dc076040 2024-05-08T18:05:06.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.183+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f990af3a20 con 0x55f990af4780 2024-05-08T18:05:06.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.183+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.183+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f990af3a20 con 0x55f990aefd90 2024-05-08T18:05:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.183+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f990aefd90 2024-05-08T18:05:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.183+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f990af3a20 con 0x55f990aefd90 2024-05-08T18:05:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.184+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f990aefd90 2024-05-08T18:05:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.184+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f990af3a20 con 0x55f990af4780 2024-05-08T18:05:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.184+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.184+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f990af3a20 con 0x55f990af4780 2024-05-08T18:05:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.184+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.184+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f990af3a20 con 0x7f52dc076040 2024-05-08T18:05:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.184+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52e000a4a0 con 0x7f52dc076040 2024-05-08T18:05:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.184+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f990af3a20 con 0x55f990af9130 2024-05-08T18:05:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.185+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52dc081070 con 0x55f990af9130 2024-05-08T18:05:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.185+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f990af3a20 con 0x55f990aefd90 2024-05-08T18:05:06.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.185+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f990aefd90 2024-05-08T18:05:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.185+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f990af3a20 con 0x55f990af4780 2024-05-08T18:05:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.185+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.185+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f990af3a20 con 0x55f990aefd90 2024-05-08T18:05:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.185+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f990aefd90 2024-05-08T18:05:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.185+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f990af3a20 con 0x7f52dc076040 2024-05-08T18:05:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.186+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f52e000a4a0 con 0x7f52dc076040 2024-05-08T18:05:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.186+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f990af3a20 con 0x55f990af9130 2024-05-08T18:05:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.186+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52dc081070 con 0x55f990af9130 2024-05-08T18:05:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.186+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x55f990af9130 2024-05-08T18:05:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.186+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52dc081070 con 0x55f990af9130 2024-05-08T18:05:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.186+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x55f990af4780 2024-05-08T18:05:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.186+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.186+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x55f990af9130 2024-05-08T18:05:06.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.187+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52dc081070 con 0x55f990af9130 2024-05-08T18:05:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.187+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x55f990af9130 2024-05-08T18:05:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.187+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f52dc081070 con 0x55f990af9130 2024-05-08T18:05:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.187+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x55f990aefd90 2024-05-08T18:05:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.187+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f990aefd90 2024-05-08T18:05:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.187+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x55f990aefd90 2024-05-08T18:05:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.187+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f990aefd90 2024-05-08T18:05:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.187+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x55f990af4780 2024-05-08T18:05:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.188+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.188+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x55f990af9130 2024-05-08T18:05:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.188+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52dc081070 con 0x55f990af9130 2024-05-08T18:05:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.188+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x7f52dc076040 2024-05-08T18:05:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.188+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52e000a4a0 con 0x7f52dc076040 2024-05-08T18:05:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.188+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x55f990aefd90 2024-05-08T18:05:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.188+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f990aefd90 2024-05-08T18:05:06.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.188+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x55f990af4780 2024-05-08T18:05:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.189+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.189+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x55f990af9130 2024-05-08T18:05:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.189+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52dc081070 con 0x55f990af9130 2024-05-08T18:05:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.189+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x55f990af4780 2024-05-08T18:05:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.189+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.189+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x7f52dc076040 2024-05-08T18:05:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.189+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52e000a4a0 con 0x7f52dc076040 2024-05-08T18:05:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.189+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x55f990af9130 2024-05-08T18:05:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.190+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52dc081070 con 0x55f990af9130 2024-05-08T18:05:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.190+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x55f990af4780 2024-05-08T18:05:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.190+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.190+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x55f990aefd90 2024-05-08T18:05:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.190+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f990aefd90 2024-05-08T18:05:06.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.190+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f52dc090040 con 0x7f52dc076040 2024-05-08T18:05:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.190+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f52e000a4a0 con 0x7f52dc076040 2024-05-08T18:05:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.190+0000 7f52eb72e800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.190+0000 7f52eb72e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990af7a40 con 0x55f990aefd90 2024-05-08T18:05:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f990aefd90 2024-05-08T18:05:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990af7a40 con 0x55f990aefd90 2024-05-08T18:05:06.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f52e000a4a0 con 0x55f990aefd90 2024-05-08T18:05:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990af7a40 con 0x55f990af9130 2024-05-08T18:05:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f52dc081070 con 0x55f990af9130 2024-05-08T18:05:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 20 zone default found 2024-05-08T18:05:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 4 Realm: () 2024-05-08T18:05:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 20 zonegroup default 2024-05-08T18:05:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.191+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990af8370 con 0x55f990af4780 2024-05-08T18:05:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.192+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.192+0000 7f52eb72e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.192+0000 7f52eb72e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.192+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990afdb80 con 0x55f990af9130 2024-05-08T18:05:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.192+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f52dc081070 con 0x55f990af9130 2024-05-08T18:05:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.192+0000 7f52eb72e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.192+0000 7f52eb72e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.192+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990afe010 con 0x55f990af9130 2024-05-08T18:05:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.192+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f52dc081070 con 0x55f990af9130 2024-05-08T18:05:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.192+0000 7f52eb72e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.192+0000 7f52eb72e800 20 started sync module instance, tier type = 2024-05-08T18:05:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.192+0000 7f52eb72e800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:06.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.192+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990afcfc0 con 0x7f52dc076040 2024-05-08T18:05:06.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.193+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52e000a4a0 con 0x7f52dc076040 2024-05-08T18:05:06.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.193+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94530362660064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990afcfc0 con 0x7f52dc076040 2024-05-08T18:05:06.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.193+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990afead0 con 0x55f990aefd90 2024-05-08T18:05:06.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.194+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52e0040040 con 0x55f990aefd90 2024-05-08T18:05:06.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.194+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94530362660064] v27'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52e0040040 con 0x7f52dc076040 2024-05-08T18:05:06.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.195+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94530362664416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990afea80 con 0x55f990aefd90 2024-05-08T18:05:06.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.195+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990afcb10 con 0x55f990af9130 2024-05-08T18:05:06.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.195+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94530362664416] v27'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52e0040040 con 0x55f990aefd90 2024-05-08T18:05:06.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.196+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52dc081070 con 0x55f990af9130 2024-05-08T18:05:06.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.196+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94530362666688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990affa40 con 0x55f990af9130 2024-05-08T18:05:06.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.196+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990b00400 con 0x7f52dc076040 2024-05-08T18:05:06.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.197+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94530362666688] v27'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52dc081070 con 0x55f990af9130 2024-05-08T18:05:06.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.197+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52e0040040 con 0x7f52dc076040 2024-05-08T18:05:06.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.197+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94530362670928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990b00820 con 0x7f52dc076040 2024-05-08T18:05:06.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.197+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990b011e0 con 0x55f990aefd90 2024-05-08T18:05:06.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.198+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94530362670928] v27'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52e0040040 con 0x7f52dc076040 2024-05-08T18:05:06.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.198+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52e0040040 con 0x55f990aefd90 2024-05-08T18:05:06.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.198+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94530362672464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990b011e0 con 0x55f990aefd90 2024-05-08T18:05:06.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.198+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990b01ba0 con 0x55f990af4780 2024-05-08T18:05:06.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.199+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.199+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94530362672464] v27'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52e0040040 con 0x55f990aefd90 2024-05-08T18:05:06.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.199+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94530362676976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990b02440 con 0x55f990af4780 2024-05-08T18:05:06.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.199+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990b00d70 con 0x55f990af4780 2024-05-08T18:05:06.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.200+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94530362676976] v27'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.200+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.200+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94530362681728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990b03250 con 0x55f990af4780 2024-05-08T18:05:06.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.200+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990b03aa0 con 0x55f990af4780 2024-05-08T18:05:06.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.201+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94530362681728] v27'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.201+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.201+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94530362684912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990b03ec0 con 0x55f990af4780 2024-05-08T18:05:06.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.201+0000 7f52eb72e800 20 add_watcher() i=0 2024-05-08T18:05:06.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.201+0000 7f52eb72e800 20 add_watcher() i=1 2024-05-08T18:05:06.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.201+0000 7f52eb72e800 20 add_watcher() i=2 2024-05-08T18:05:06.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.201+0000 7f52eb72e800 20 add_watcher() i=3 2024-05-08T18:05:06.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.201+0000 7f52eb72e800 20 add_watcher() i=4 2024-05-08T18:05:06.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.201+0000 7f52eb72e800 20 add_watcher() i=5 2024-05-08T18:05:06.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.201+0000 7f52eb72e800 20 add_watcher() i=6 2024-05-08T18:05:06.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.202+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94530362684912] v27'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.202+0000 7f52eb72e800 20 add_watcher() i=7 2024-05-08T18:05:06.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.202+0000 7f52eb72e800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:06.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.202+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990b07220 con 0x55f990aefd90 2024-05-08T18:05:06.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.202+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f52e0040040 con 0x55f990aefd90 2024-05-08T18:05:06.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.202+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94530362698784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990b04030 con 0x55f990aefd90 2024-05-08T18:05:06.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.203+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94530362698784] v27'84 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f52e0040040 con 0x55f990aefd90 2024-05-08T18:05:06.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.203+0000 7f519dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:06.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.203+0000 7f52eb72e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:06.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.203+0000 7f52eb72e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:06.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.203+0000 7f52eb72e800 5 note: GC not initialized 2024-05-08T18:05:06.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.204+0000 7f519cfe1640 20 reqs_thread_entry: start 2024-05-08T18:05:06.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.204+0000 7f52eb72e800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:06.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.204+0000 7f52eb72e800 20 Filter name: none 2024-05-08T18:05:06.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.204+0000 7f51167fc640 20 reqs_thread_entry: start 2024-05-08T18:05:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.204+0000 7f52eb72e800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.204+0000 7f52eb72e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.204+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990b3a7b0 con 0x55f990aefd90 2024-05-08T18:05:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.204+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~582 out=582b,getxattrs out=75b] v0'0 uv9 ondisk = 0) ==== 231+0+705 (crc 0 0 0) 0x7f52e0040040 con 0x55f990aefd90 2024-05-08T18:05:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.204+0000 7f52eb72e800 20 rados_obj.operate() r=0 bl.length=582 2024-05-08T18:05:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.204+0000 7f52eb72e800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.204+0000 7f52eb72e800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.204+0000 7f52eb72e800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.204+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~556 in=556b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990b40ec0 con 0x55f990aefd90 2024-05-08T18:05:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.206+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~556] v27'10 uv10 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f52e0040040 con 0x55f990aefd90 2024-05-08T18:05:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.206+0000 7f52eb72e800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.206+0000 7f52eb72e800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.206+0000 7f52eb72e800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-05-08T18:05:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.206+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94530362935424 in=744b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f990b3c7b0 con 0x55f990aefd90 2024-05-08T18:05:06.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.206+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== watch-notify(notify (1) cookie 94530362672464 notify 115964117001 ret 0) ==== 774+0+0 (crc 0 0 0) 0x7f52e00189d0 con 0x55f990aefd90 2024-05-08T18:05:06.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.206+0000 7f51caffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117001 cookie 94530362672464 notifier 4403 bl.length()=732 2024-05-08T18:05:06.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.206+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(64 notify.4 [notify cookie 94530362935424 out=8b] v0'0 uv75 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f52e0040040 con 0x55f990aefd90 2024-05-08T18:05:06.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.206+0000 7f51caffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:06.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.206+0000 7f51caffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:06.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.206+0000 7f51caffd640 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f51ac003ea0 con 0x55f990aefd90 2024-05-08T18:05:06.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.206+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52e0040040 con 0x55f990aefd90 2024-05-08T18:05:06.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.207+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== watch-notify(notify_complete (2) cookie 94530362935424 notify 115964117001 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f52e0018ba0 con 0x55f990aefd90 2024-05-08T18:05:06.216 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy","active":true,"create_date":"2024-05-08T18:05:05.808771Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:06.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.208+0000 7f519dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:06.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.208+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94530362698784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990b3cf70 con 0x55f990aefd90 2024-05-08T18:05:06.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.208+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94530362698784] v27'85 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f52e0040040 con 0x55f990aefd90 2024-05-08T18:05:06.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.208+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94530362660064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990b3cf70 con 0x7f52dc076040 2024-05-08T18:05:06.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.209+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94530362660064] v27'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52e0040040 con 0x7f52dc076040 2024-05-08T18:05:06.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.209+0000 7f52eb72e800 20 remove_watcher() i=0 2024-05-08T18:05:06.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.209+0000 7f52eb72e800 2 removed watcher, disabling cache 2024-05-08T18:05:06.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.209+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94530362664416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990afd4c0 con 0x55f990aefd90 2024-05-08T18:05:06.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.211+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94530362664416] v27'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52e0040040 con 0x55f990aefd90 2024-05-08T18:05:06.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.211+0000 7f52eb72e800 20 remove_watcher() i=1 2024-05-08T18:05:06.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.211+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94530362666688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990afe5e0 con 0x55f990af9130 2024-05-08T18:05:06.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.212+0000 7f52e6c83640 1 -- 172.21.15.50:0/3749481852 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94530362666688] v27'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52dc081070 con 0x55f990af9130 2024-05-08T18:05:06.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.212+0000 7f52eb72e800 20 remove_watcher() i=2 2024-05-08T18:05:06.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.212+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94530362670928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990afe5e0 con 0x7f52dc076040 2024-05-08T18:05:06.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.213+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94530362670928] v27'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52e0040040 con 0x7f52dc076040 2024-05-08T18:05:06.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.213+0000 7f52eb72e800 20 remove_watcher() i=3 2024-05-08T18:05:06.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.213+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94530362672464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990afe5e0 con 0x55f990aefd90 2024-05-08T18:05:06.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.214+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94530362672464] v27'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52e0040040 con 0x55f990aefd90 2024-05-08T18:05:06.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.214+0000 7f52eb72e800 20 remove_watcher() i=4 2024-05-08T18:05:06.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.214+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94530362676976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990b00550 con 0x55f990af4780 2024-05-08T18:05:06.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.215+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94530362676976] v27'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.215+0000 7f52eb72e800 20 remove_watcher() i=5 2024-05-08T18:05:06.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.215+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94530362681728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990afe5e0 con 0x55f990af4780 2024-05-08T18:05:06.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.215+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94530362681728] v27'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.215+0000 7f52eb72e800 20 remove_watcher() i=6 2024-05-08T18:05:06.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.215+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94530362684912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f990b016f0 con 0x55f990af4780 2024-05-08T18:05:06.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.216+0000 7f52e7484640 1 -- 172.21.15.50:0/3749481852 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94530362684912] v27'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f52d806f030 con 0x55f990af4780 2024-05-08T18:05:06.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.216+0000 7f52eb72e800 20 remove_watcher() i=7 2024-05-08T18:05:06.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f990af4780 msgr2=0x55f990af6b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f990af4780 0x55f990af6b70 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f990aefd90 msgr2=0x55f990af2250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f990aefd90 0x55f990af2250 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f52dc076040 msgr2=0x7f52dc083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f52dc076040 0x7f52dc083410 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f990af9130 msgr2=0x55f990afb5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f990af9130 0x55f990afb5f0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f51b003bba0 msgr2=0x7f51b003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f51b003bba0 0x7f51b003e060 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f52dc052fd0 tx=0x7f52dc04f000 comp rx=0 tx=0).stop 2024-05-08T18:05:06.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990add220 msgr2=0x55f990ae4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52e7c85640 1 -- 172.21.15.50:0/3749481852 reap_dead start 2024-05-08T18:05:06.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990add220 0x55f990ae4640 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f52d8032000 tx=0x7f52d8004d70 comp rx=0 tx=0).stop 2024-05-08T18:05:06.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 shutdown_connections 2024-05-08T18:05:06.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 --2- 172.21.15.50:0/3749481852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990add220 0x55f990ae4640 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 >> 172.21.15.50:0/3749481852 conn(0x55f990ad9580 msgr2=0x55f990ad6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 shutdown_connections 2024-05-08T18:05:06.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 -- 172.21.15.50:0/3749481852 wait complete. 2024-05-08T18:05:06.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 -- 172.21.15.50:0/3792950436 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f51c003bb00 msgr2=0x7f51c003dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 --2- 172.21.15.50:0/3792950436 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f51c003bb00 0x7f51c003dfc0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x55f990ab9790 tx=0x7f52dc005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:06.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 -- 172.21.15.50:0/3792950436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990acff80 msgr2=0x55f990ad03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 --2- 172.21.15.50:0/3792950436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990acff80 0x55f990ad03a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f52d800b6c0 tx=0x7f52d801ab20 comp rx=0 tx=0).stop 2024-05-08T18:05:06.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 -- 172.21.15.50:0/3792950436 shutdown_connections 2024-05-08T18:05:06.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 --2- 172.21.15.50:0/3792950436 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f51c003bb00 0x7f51c003dfc0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 --2- 172.21.15.50:0/3792950436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f990acff80 0x55f990ad03a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 -- 172.21.15.50:0/3792950436 >> 172.21.15.50:0/3792950436 conn(0x55f990ac4020 msgr2=0x55f990ac4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 -- 172.21.15.50:0/3792950436 shutdown_connections 2024-05-08T18:05:06.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.217+0000 7f52eb72e800 1 -- 172.21.15.50:0/3792950436 wait complete. 2024-05-08T18:05:06.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.218+0000 7f52eb72e800 1 -- 172.21.15.50:0/1540520817 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f990abb1e0 msgr2=0x55f990abd5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.218+0000 7f52eb72e800 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f990abb1e0 0x55f990abd5d0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.218+0000 7f52eb72e800 1 -- 172.21.15.50:0/1540520817 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f9909baae0 msgr2=0x55f990ab8930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.218+0000 7f52eb72e800 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f9909baae0 0x55f990ab8930 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.218+0000 7f52eb72e800 1 -- 172.21.15.50:0/1540520817 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f990abf120 msgr2=0x55f990ac1510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.218+0000 7f52eb72e800 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f990abf120 0x55f990ac1510 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.218+0000 7f52eb72e800 1 -- 172.21.15.50:0/1540520817 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f51cc03bcf0 msgr2=0x7f51cc03e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.218+0000 7f52eb72e800 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f51cc03bcf0 0x7f51cc03e1b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f52dc054000 tx=0x7f52dc04e000 comp rx=0 tx=0).stop 2024-05-08T18:05:06.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.218+0000 7f52eb72e800 1 -- 172.21.15.50:0/1540520817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909b66f0 msgr2=0x55f990ab16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.218+0000 7f52eb72e800 1 --2- 172.21.15.50:0/1540520817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9909b66f0 0x55f990ab16f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f52d80012f0 tx=0x7f52d8000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.218+0000 7f52e7c85640 1 -- 172.21.15.50:0/1540520817 reap_dead start 2024-05-08T18:05:06.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.218+0000 7f52eb72e800 1 -- 172.21.15.50:0/1540520817 shutdown_connections 2024-05-08T18:05:06.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.218+0000 7f52eb72e800 1 -- 172.21.15.50:0/1540520817 >> 172.21.15.50:0/1540520817 conn(0x55f9909b1c70 msgr2=0x55f990aa1ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.218+0000 7f52eb72e800 1 -- 172.21.15.50:0/1540520817 shutdown_connections 2024-05-08T18:05:06.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.218+0000 7f52eb72e800 1 -- 172.21.15.50:0/1540520817 wait complete. 2024-05-08T18:05:06.228 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True, 'create_date': '2024-05-08T18:05:05.808771Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:06.228 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2024-05-08T18:05:06.228 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2024-05-08T18:05:06.228 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo2 --key-type swift --purge-keys 2024-05-08T18:05:06.277 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:06.277 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:06.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.279+0000 7f86677af800 1 Processor -- start 2024-05-08T18:05:06.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.280+0000 7f86677af800 1 -- start start 2024-05-08T18:05:06.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.280+0000 7f86677af800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56120fbb0a70 0x56120fbb0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.280+0000 7f86677af800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56120fbb1510 con 0x56120fbb17f0 2024-05-08T18:05:06.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.280+0000 7f86677af800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x56120fa51e20 con 0x56120fbb0a70 2024-05-08T18:05:06.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.280+0000 7f86652a7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56120fbb0a70 0x56120fbb0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.280+0000 7f8664aa6640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x56120fbb17f0 0x56120fbb1ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43062/0 (socket says 172.21.15.50:43062) 2024-05-08T18:05:06.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.280+0000 7f8664aa6640 1 -- 172.21.15.50:0/3562148708 learned_addr learned my addr 172.21.15.50:0/3562148708 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:06.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.280+0000 7f86652a7640 1 --2- 172.21.15.50:0/3562148708 >> v2:172.21.15.46:3300/0 conn(0x56120fbb0a70 0x56120fbb0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37170/0 (socket says 172.21.15.50:37170) 2024-05-08T18:05:06.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.280+0000 7f86652a7640 1 -- 172.21.15.50:0/3562148708 >> v1:172.21.15.46:6789/0 conn(0x56120fbb17f0 legacy=0x56120fbb1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.280+0000 7f8657fff640 1 -- 172.21.15.50:0/3562148708 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2369730372 0 0) 0x56120fbb1510 con 0x56120fbb17f0 2024-05-08T18:05:06.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.280+0000 7f86652a7640 1 -- 172.21.15.50:0/3562148708 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56120fbb23b0 con 0x56120fbb0a70 2024-05-08T18:05:06.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.282+0000 7f86652a7640 1 --2- 172.21.15.50:0/3562148708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 0x56120fbb0e90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f865800a470 tx=0x7f865801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cffabb9f6c893a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.282+0000 7f8657fff640 1 -- 172.21.15.50:0/3562148708 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8658057070 con 0x56120fbb0a70 2024-05-08T18:05:06.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.282+0000 7f8657fff640 1 -- 172.21.15.50:0/3562148708 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865804adc0 con 0x56120fbb0a70 2024-05-08T18:05:06.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.282+0000 7f8657fff640 1 -- 172.21.15.50:0/3562148708 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8658052590 con 0x56120fbb0a70 2024-05-08T18:05:06.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.283+0000 7f86677af800 1 -- 172.21.15.50:0/3562148708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 msgr2=0x56120fbb0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.283+0000 7f86677af800 1 --2- 172.21.15.50:0/3562148708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 0x56120fbb0e90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f865800a470 tx=0x7f865801ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:06.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.283+0000 7f86677af800 1 -- 172.21.15.50:0/3562148708 shutdown_connections 2024-05-08T18:05:06.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.283+0000 7f86677af800 1 --2- 172.21.15.50:0/3562148708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 0x56120fbb0e90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.283+0000 7f86677af800 1 -- 172.21.15.50:0/3562148708 >> 172.21.15.50:0/3562148708 conn(0x56120fbab2f0 msgr2=0x56120fbad710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.283+0000 7f86677af800 1 -- 172.21.15.50:0/3562148708 shutdown_connections 2024-05-08T18:05:06.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.283+0000 7f86677af800 1 -- 172.21.15.50:0/3562148708 wait complete. 2024-05-08T18:05:06.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.298+0000 7f86677af800 1 Processor -- start 2024-05-08T18:05:06.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.298+0000 7f86677af800 1 -- start start 2024-05-08T18:05:06.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.298+0000 7f86677af800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 0x56120fbb0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.298+0000 7f86677af800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56120fafca30 con 0x56120fbb0a70 2024-05-08T18:05:06.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.299+0000 7f86652a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 0x56120fbb0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.299+0000 7f86652a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 0x56120fbb0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37182/0 (socket says 172.21.15.50:37182) 2024-05-08T18:05:06.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.299+0000 7f86652a7640 1 -- 172.21.15.50:0/4039162282 learned_addr learned my addr 172.21.15.50:0/4039162282 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.299+0000 7f86652a7640 1 -- 172.21.15.50:0/4039162282 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56120fca5770 con 0x56120fbb0a70 2024-05-08T18:05:06.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.299+0000 7f86652a7640 1 --2- 172.21.15.50:0/4039162282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 0x56120fbb0170 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8658055040 tx=0x7f865804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b899e0721e1c38 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.299+0000 7f8553606640 1 -- 172.21.15.50:0/4039162282 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8658057040 con 0x56120fbb0a70 2024-05-08T18:05:06.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.299+0000 7f8553606640 1 -- 172.21.15.50:0/4039162282 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8658062070 con 0x56120fbb0a70 2024-05-08T18:05:06.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.299+0000 7f8553606640 1 -- 172.21.15.50:0/4039162282 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f865805d470 con 0x56120fbb0a70 2024-05-08T18:05:06.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.300+0000 7f86677af800 1 -- 172.21.15.50:0/4039162282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 msgr2=0x56120fbb0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.300+0000 7f86677af800 1 --2- 172.21.15.50:0/4039162282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 0x56120fbb0170 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8658055040 tx=0x7f865804c000 comp rx=0 tx=0).stop 2024-05-08T18:05:06.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.300+0000 7f86677af800 1 -- 172.21.15.50:0/4039162282 shutdown_connections 2024-05-08T18:05:06.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.300+0000 7f86677af800 1 --2- 172.21.15.50:0/4039162282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 0x56120fbb0170 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.300+0000 7f86677af800 1 -- 172.21.15.50:0/4039162282 >> 172.21.15.50:0/4039162282 conn(0x56120fbab2f0 msgr2=0x56120fc9d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.300+0000 7f86677af800 1 -- 172.21.15.50:0/4039162282 shutdown_connections 2024-05-08T18:05:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.300+0000 7f86677af800 1 -- 172.21.15.50:0/4039162282 wait complete. 2024-05-08T18:05:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.300+0000 7f86677af800 1 Processor -- start 2024-05-08T18:05:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.300+0000 7f86677af800 1 -- start start 2024-05-08T18:05:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.301+0000 7f86677af800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 0x56120fcaa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.301+0000 7f86677af800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56120fbb9cc0 con 0x56120fbb0a70 2024-05-08T18:05:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.301+0000 7f86652a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 0x56120fcaa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.301+0000 7f86652a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 0x56120fcaa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37190/0 (socket says 172.21.15.50:37190) 2024-05-08T18:05:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.301+0000 7f86652a7640 1 -- 172.21.15.50:0/856739685 learned_addr learned my addr 172.21.15.50:0/856739685 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.301+0000 7f86652a7640 1 -- 172.21.15.50:0/856739685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56120fcaaa30 con 0x56120fbb0a70 2024-05-08T18:05:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.301+0000 7f86652a7640 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 0x56120fcaa4f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f865801ce60 tx=0x7f8658002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.301+0000 7f8551e03640 1 -- 172.21.15.50:0/856739685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f865806b070 con 0x56120fbb0a70 2024-05-08T18:05:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.301+0000 7f8551e03640 1 -- 172.21.15.50:0/856739685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865805d970 con 0x56120fbb0a70 2024-05-08T18:05:06.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.301+0000 7f86677af800 1 -- 172.21.15.50:0/856739685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56120fcaacc0 con 0x56120fbb0a70 2024-05-08T18:05:06.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.301+0000 7f8551e03640 1 -- 172.21.15.50:0/856739685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8658057070 con 0x56120fbb0a70 2024-05-08T18:05:06.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.302+0000 7f86677af800 1 -- 172.21.15.50:0/856739685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56120fcad970 con 0x56120fbb0a70 2024-05-08T18:05:06.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.302+0000 7f8551e03640 1 -- 172.21.15.50:0/856739685 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f865805dc90 con 0x56120fbb0a70 2024-05-08T18:05:06.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.303+0000 7f8551e03640 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f854c03bba0 0x7f854c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.303+0000 7f8551e03640 1 -- 172.21.15.50:0/856739685 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f86580945f0 con 0x56120fbb0a70 2024-05-08T18:05:06.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.303+0000 7f8664aa6640 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f854c03bba0 0x7f854c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.303+0000 7f86677af800 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56120fbb17f0 0x56120fcb1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.303+0000 7f86677af800 1 -- 172.21.15.50:0/856739685 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fcb1c70 con 0x56120fbb17f0 2024-05-08T18:05:06.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.303+0000 7f8665aa8640 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56120fbb17f0 0x56120fcb1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.303+0000 7f8664aa6640 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f854c03bba0 0x7f854c03e060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f865c000c00 tx=0x7f865c0013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.303+0000 7f8665aa8640 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56120fbb17f0 0x56120fcb1730 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.303+0000 7f8665aa8640 1 -- 172.21.15.50:0/856739685 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fbb17f0 2024-05-08T18:05:06.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.303+0000 7f86677af800 1 -- 172.21.15.50:0/856739685 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fcaece0 con 0x56120fbb17f0 2024-05-08T18:05:06.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.304+0000 7f8665aa8640 1 -- 172.21.15.50:0/856739685 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fbb17f0 2024-05-08T18:05:06.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.304+0000 7f86677af800 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56120fcb3f40 0x56120fcb6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.304+0000 7f86677af800 1 -- 172.21.15.50:0/856739685 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fcb6a00 con 0x56120fcb3f40 2024-05-08T18:05:06.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.304+0000 7f86652a7640 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56120fcb3f40 0x56120fcb6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.304+0000 7f86652a7640 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56120fcb3f40 0x56120fcb6330 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.305+0000 7f86652a7640 1 -- 172.21.15.50:0/856739685 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f865805a040 con 0x56120fcb3f40 2024-05-08T18:05:06.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.305+0000 7f86677af800 1 -- 172.21.15.50:0/856739685 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fcb2db0 con 0x56120fbb17f0 2024-05-08T18:05:06.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.305+0000 7f8665aa8640 1 -- 172.21.15.50:0/856739685 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fbb17f0 2024-05-08T18:05:06.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.305+0000 7f86677af800 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56120fcb7e40 0x56120fcba230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.305+0000 7f86677af800 1 -- 172.21.15.50:0/856739685 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fcba970 con 0x56120fcb7e40 2024-05-08T18:05:06.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.305+0000 7f8664aa6640 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56120fcb7e40 0x56120fcba230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.305+0000 7f8664aa6640 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56120fcb7e40 0x56120fcba230 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.306+0000 7f8664aa6640 1 -- 172.21.15.50:0/856739685 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f865c00f670 con 0x56120fcb7e40 2024-05-08T18:05:06.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.306+0000 7f86677af800 1 Processor -- start 2024-05-08T18:05:06.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.306+0000 7f86677af800 1 -- start start 2024-05-08T18:05:06.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.306+0000 7f86677af800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fca3300 0x56120fca1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.306+0000 7f86677af800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56120faa3330 con 0x56120fca3300 2024-05-08T18:05:06.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.306+0000 7f86652a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fca3300 0x56120fca1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.307+0000 7f86652a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fca3300 0x56120fca1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37202/0 (socket says 172.21.15.50:37202) 2024-05-08T18:05:06.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.307+0000 7f86652a7640 1 -- 172.21.15.50:0/452014454 learned_addr learned my addr 172.21.15.50:0/452014454 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.307+0000 7f86652a7640 1 -- 172.21.15.50:0/452014454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56120fca1cb0 con 0x56120fca3300 2024-05-08T18:05:06.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.307+0000 7f86652a7640 1 --2- 172.21.15.50:0/452014454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fca3300 0x56120fca1770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f865805b7e0 tx=0x7f865805b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.307+0000 7f854affd640 1 -- 172.21.15.50:0/452014454 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8658070e40 con 0x56120fca3300 2024-05-08T18:05:06.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.307+0000 7f854affd640 1 -- 172.21.15.50:0/452014454 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8658009410 con 0x56120fca3300 2024-05-08T18:05:06.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.307+0000 7f86677af800 1 -- 172.21.15.50:0/452014454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56120fca1f40 con 0x56120fca3300 2024-05-08T18:05:06.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.307+0000 7f854affd640 1 -- 172.21.15.50:0/452014454 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86580097a0 con 0x56120fca3300 2024-05-08T18:05:06.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.307+0000 7f86677af800 1 -- 172.21.15.50:0/452014454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56120fca2440 con 0x56120fca3300 2024-05-08T18:05:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.308+0000 7f854affd640 1 -- 172.21.15.50:0/452014454 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f8658009940 con 0x56120fca3300 2024-05-08T18:05:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.308+0000 7f854affd640 1 --2- 172.21.15.50:0/452014454 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f853c03bba0 0x7f853c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.308+0000 7f854affd640 1 -- 172.21.15.50:0/452014454 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f865807e030 con 0x56120fca3300 2024-05-08T18:05:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.308+0000 7f8664aa6640 1 --2- 172.21.15.50:0/452014454 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f853c03bba0 0x7f853c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.308+0000 7f86677af800 1 Processor -- start 2024-05-08T18:05:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.309+0000 7f86677af800 1 -- start start 2024-05-08T18:05:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.309+0000 7f8664aa6640 1 --2- 172.21.15.50:0/452014454 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f853c03bba0 0x7f853c03e060 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x56120fcb20e0 tx=0x7f865c006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.309+0000 7f86677af800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fcd7ff0 0x56120fcd8410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.309+0000 7f86677af800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56120fca5a00 con 0x56120fcd7ff0 2024-05-08T18:05:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.309+0000 7f86652a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fcd7ff0 0x56120fcd8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.309+0000 7f86652a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fcd7ff0 0x56120fcd8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37214/0 (socket says 172.21.15.50:37214) 2024-05-08T18:05:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.309+0000 7f86652a7640 1 -- 172.21.15.50:0/707914410 learned_addr learned my addr 172.21.15.50:0/707914410 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.309+0000 7f86652a7640 1 -- 172.21.15.50:0/707914410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56120fcd89e0 con 0x56120fcd7ff0 2024-05-08T18:05:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.309+0000 7f86652a7640 1 --2- 172.21.15.50:0/707914410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fcd7ff0 0x56120fcd8410 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8658009710 tx=0x7f8658094f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42118c6a13f02c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.309+0000 7f8548ff9640 1 -- 172.21.15.50:0/707914410 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8658091580 con 0x56120fcd7ff0 2024-05-08T18:05:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.309+0000 7f8548ff9640 1 -- 172.21.15.50:0/707914410 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86580a2070 con 0x56120fcd7ff0 2024-05-08T18:05:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.309+0000 7f8548ff9640 1 -- 172.21.15.50:0/707914410 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f865807de00 con 0x56120fcd7ff0 2024-05-08T18:05:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.310+0000 7f86677af800 1 -- 172.21.15.50:0/707914410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fcd7ff0 msgr2=0x56120fcd8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.310+0000 7f86677af800 1 --2- 172.21.15.50:0/707914410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fcd7ff0 0x56120fcd8410 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8658009710 tx=0x7f8658094f10 comp rx=0 tx=0).stop 2024-05-08T18:05:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.310+0000 7f86677af800 1 -- 172.21.15.50:0/707914410 shutdown_connections 2024-05-08T18:05:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.310+0000 7f86677af800 1 --2- 172.21.15.50:0/707914410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fcd7ff0 0x56120fcd8410 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.310+0000 7f86677af800 1 -- 172.21.15.50:0/707914410 >> 172.21.15.50:0/707914410 conn(0x56120fcd4300 msgr2=0x56120fcd2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.310+0000 7f86677af800 1 -- 172.21.15.50:0/707914410 shutdown_connections 2024-05-08T18:05:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.310+0000 7f86677af800 1 -- 172.21.15.50:0/707914410 wait complete. 2024-05-08T18:05:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.310+0000 7f86677af800 1 Processor -- start 2024-05-08T18:05:06.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.310+0000 7f86677af800 1 -- start start 2024-05-08T18:05:06.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.310+0000 7f86677af800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fcd7ff0 0x56120fcdf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.310+0000 7f86677af800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56120fcd9070 con 0x56120fcd7ff0 2024-05-08T18:05:06.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.311+0000 7f86652a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fcd7ff0 0x56120fcdf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.311+0000 7f86652a7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fcd7ff0 0x56120fcdf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37216/0 (socket says 172.21.15.50:37216) 2024-05-08T18:05:06.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.311+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 learned_addr learned my addr 172.21.15.50:0/2788515326 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.311+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56120fcdf910 con 0x56120fcd7ff0 2024-05-08T18:05:06.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.311+0000 7f86652a7640 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fcd7ff0 0x56120fcdf3d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f865807dc40 tx=0x7f86580a0070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.311+0000 7f853affd640 1 -- 172.21.15.50:0/2788515326 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f865806e360 con 0x56120fcd7ff0 2024-05-08T18:05:06.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.311+0000 7f853affd640 1 -- 172.21.15.50:0/2788515326 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f865806e500 con 0x56120fcd7ff0 2024-05-08T18:05:06.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.311+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56120fcdf910 con 0x56120fcd7ff0 2024-05-08T18:05:06.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.311+0000 7f853affd640 1 -- 172.21.15.50:0/2788515326 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f865806e6b0 con 0x56120fcd7ff0 2024-05-08T18:05:06.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.311+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56120fcdf910 con 0x56120fcd7ff0 2024-05-08T18:05:06.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.312+0000 7f853affd640 1 -- 172.21.15.50:0/2788515326 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f865808b1c0 con 0x56120fcd7ff0 2024-05-08T18:05:06.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.312+0000 7f853affd640 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f853003bba0 0x7f853003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.312+0000 7f853affd640 1 -- 172.21.15.50:0/2788515326 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f86580b0070 con 0x56120fcd7ff0 2024-05-08T18:05:06.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.312+0000 7f8664aa6640 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f853003bba0 0x7f853003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.313+0000 7f8664aa6640 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f853003bba0 0x7f853003e060 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f865c00ffd0 tx=0x7f865c023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.313+0000 7f86677af800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.313+0000 7f86677af800 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56120fceaaf0 0x56120fcecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.313+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fced700 con 0x56120fceaaf0 2024-05-08T18:05:06.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.313+0000 7f8665aa8640 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56120fceaaf0 0x56120fcecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.313+0000 7f8665aa8640 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56120fceaaf0 0x56120fcecfb0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fceaaf0 2024-05-08T18:05:06.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f86677af800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f86677af800 20 realm 2024-05-08T18:05:06.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f86677af800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fcedb10 con 0x56120fceaaf0 2024-05-08T18:05:06.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fceaaf0 2024-05-08T18:05:06.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f86677af800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f86677af800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:06.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f86677af800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee3e0 con 0x56120fceaaf0 2024-05-08T18:05:06.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fceaaf0 2024-05-08T18:05:06.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f86677af800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f86677af800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fceaaf0 2024-05-08T18:05:06.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fceaaf0 2024-05-08T18:05:06.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f86677af800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:06.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f86677af800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.314+0000 7f86677af800 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56120fcef4e0 0x56120fcf18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.315+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fcf1e90 con 0x56120fcef4e0 2024-05-08T18:05:06.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.315+0000 7f86652a7640 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56120fcef4e0 0x56120fcf18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.315+0000 7f86652a7640 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56120fcef4e0 0x56120fcf18d0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.315+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.315+0000 7f86677af800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:06.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.315+0000 7f86677af800 20 searching for the correct realm 2024-05-08T18:05:06.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.316+0000 7f86677af800 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56120fcf3e40 0x56120fcf6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.316+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcf69d0 con 0x56120fcf3e40 2024-05-08T18:05:06.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.316+0000 7f8664aa6640 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56120fcf3e40 0x56120fcf6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.316+0000 7f8664aa6640 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56120fcf3e40 0x56120fcf6300 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.316+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f865c037040 con 0x56120fcf3e40 2024-05-08T18:05:06.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.316+0000 7f8664aa6640 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f865c02e910 0x7f865c03a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.316+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcf69d0 con 0x7f865c02e910 2024-05-08T18:05:06.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.316+0000 7f8665aa8640 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f865c02e910 0x7f865c03a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.317+0000 7f8665aa8640 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f865c02e910 0x7f865c03a3c0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.317+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f866000a4a0 con 0x7f865c02e910 2024-05-08T18:05:06.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.317+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcef4e0 2024-05-08T18:05:06.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.317+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.317+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fceaaf0 2024-05-08T18:05:06.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.318+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fceaaf0 2024-05-08T18:05:06.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.318+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fceaaf0 2024-05-08T18:05:06.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.318+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fceaaf0 2024-05-08T18:05:06.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.318+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcef4e0 2024-05-08T18:05:06.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.318+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.318+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcef4e0 2024-05-08T18:05:06.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.318+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.318+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x7f865c02e910 2024-05-08T18:05:06.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.319+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f866000a4a0 con 0x7f865c02e910 2024-05-08T18:05:06.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.319+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcf3e40 2024-05-08T18:05:06.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.319+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f865c037040 con 0x56120fcf3e40 2024-05-08T18:05:06.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.319+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fceaaf0 2024-05-08T18:05:06.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.319+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fceaaf0 2024-05-08T18:05:06.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.319+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcef4e0 2024-05-08T18:05:06.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.319+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.319+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fceaaf0 2024-05-08T18:05:06.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.320+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fceaaf0 2024-05-08T18:05:06.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.320+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x7f865c02e910 2024-05-08T18:05:06.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.320+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f866000a4a0 con 0x7f865c02e910 2024-05-08T18:05:06.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.320+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcf3e40 2024-05-08T18:05:06.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.320+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f865c037040 con 0x56120fcf3e40 2024-05-08T18:05:06.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.320+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcf3e40 2024-05-08T18:05:06.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.320+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f865c037040 con 0x56120fcf3e40 2024-05-08T18:05:06.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.320+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcef4e0 2024-05-08T18:05:06.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.321+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.321+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcf3e40 2024-05-08T18:05:06.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.321+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f865c037040 con 0x56120fcf3e40 2024-05-08T18:05:06.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.321+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcf3e40 2024-05-08T18:05:06.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.321+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f865c037040 con 0x56120fcf3e40 2024-05-08T18:05:06.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.321+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fceaaf0 2024-05-08T18:05:06.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.322+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fceaaf0 2024-05-08T18:05:06.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.322+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fceaaf0 2024-05-08T18:05:06.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.322+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fceaaf0 2024-05-08T18:05:06.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.322+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcef4e0 2024-05-08T18:05:06.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.322+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.322+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcf3e40 2024-05-08T18:05:06.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.322+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f865c037040 con 0x56120fcf3e40 2024-05-08T18:05:06.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.322+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x7f865c02e910 2024-05-08T18:05:06.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.323+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f866000a4a0 con 0x7f865c02e910 2024-05-08T18:05:06.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.323+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fceaaf0 2024-05-08T18:05:06.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.323+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fceaaf0 2024-05-08T18:05:06.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.323+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcef4e0 2024-05-08T18:05:06.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.323+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.323+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcf3e40 2024-05-08T18:05:06.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.323+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f865c037040 con 0x56120fcf3e40 2024-05-08T18:05:06.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.323+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcef4e0 2024-05-08T18:05:06.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.324+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.324+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x7f865c02e910 2024-05-08T18:05:06.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.324+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f866000a4a0 con 0x7f865c02e910 2024-05-08T18:05:06.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.324+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcf3e40 2024-05-08T18:05:06.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.324+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f865c037040 con 0x56120fcf3e40 2024-05-08T18:05:06.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.324+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fcef4e0 2024-05-08T18:05:06.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.324+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.324+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x56120fceaaf0 2024-05-08T18:05:06.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fceaaf0 2024-05-08T18:05:06.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56120fcee780 con 0x7f865c02e910 2024-05-08T18:05:06.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f866000a4a0 con 0x7f865c02e910 2024-05-08T18:05:06.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f86677af800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:06.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f86677af800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:06.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f86677af800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:06.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f86677af800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:06.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f86677af800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:06.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f86677af800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:06.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f86677af800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fcf2750 con 0x56120fceaaf0 2024-05-08T18:05:06.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fceaaf0 2024-05-08T18:05:06.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f86677af800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f86677af800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fcf2750 con 0x56120fceaaf0 2024-05-08T18:05:06.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f866000a4a0 con 0x56120fceaaf0 2024-05-08T18:05:06.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f86677af800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:06.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f86677af800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.325+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fcf2750 con 0x56120fcf3e40 2024-05-08T18:05:06.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f865c037040 con 0x56120fcf3e40 2024-05-08T18:05:06.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f86677af800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:06.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f86677af800 20 zone default found 2024-05-08T18:05:06.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f86677af800 4 Realm: () 2024-05-08T18:05:06.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f86677af800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:06.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f86677af800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:06.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f86677af800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:06.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f86677af800 20 zonegroup default 2024-05-08T18:05:06.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f86677af800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fcf30c0 con 0x56120fcef4e0 2024-05-08T18:05:06.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f86677af800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f86677af800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fcf88c0 con 0x56120fcf3e40 2024-05-08T18:05:06.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f865c037040 con 0x56120fcf3e40 2024-05-08T18:05:06.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f86677af800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f86677af800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.326+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fcf8db0 con 0x56120fcf3e40 2024-05-08T18:05:06.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.327+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f865c037040 con 0x56120fcf3e40 2024-05-08T18:05:06.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.327+0000 7f86677af800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.327+0000 7f86677af800 20 started sync module instance, tier type = 2024-05-08T18:05:06.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.327+0000 7f86677af800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:06.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.327+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcf7cd0 con 0x7f865c02e910 2024-05-08T18:05:06.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.328+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f866000a4a0 con 0x7f865c02e910 2024-05-08T18:05:06.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.328+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94635574657488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcf7cd0 con 0x7f865c02e910 2024-05-08T18:05:06.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.328+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcf9870 con 0x56120fceaaf0 2024-05-08T18:05:06.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.329+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94635574657488] v27'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8660040040 con 0x7f865c02e910 2024-05-08T18:05:06.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.329+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8660040040 con 0x56120fceaaf0 2024-05-08T18:05:06.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.329+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94635574662016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcf7cd0 con 0x56120fceaaf0 2024-05-08T18:05:06.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.329+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfa320 con 0x56120fcf3e40 2024-05-08T18:05:06.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.330+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94635574662016] v27'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8660040040 con 0x56120fceaaf0 2024-05-08T18:05:06.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.330+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f865c037040 con 0x56120fcf3e40 2024-05-08T18:05:06.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.330+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94635574655008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcf7c40 con 0x56120fcf3e40 2024-05-08T18:05:06.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.330+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfb1e0 con 0x7f865c02e910 2024-05-08T18:05:06.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.331+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94635574655008] v27'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f865c037040 con 0x56120fcf3e40 2024-05-08T18:05:06.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.331+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8660040040 con 0x7f865c02e910 2024-05-08T18:05:06.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.331+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94635574668592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfb150 con 0x7f865c02e910 2024-05-08T18:05:06.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.331+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfbf50 con 0x56120fceaaf0 2024-05-08T18:05:06.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.332+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8660040040 con 0x56120fceaaf0 2024-05-08T18:05:06.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.332+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94635574668592] v27'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8660040040 con 0x7f865c02e910 2024-05-08T18:05:06.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.332+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94635574672000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfbf50 con 0x56120fceaaf0 2024-05-08T18:05:06.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.332+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfb610 con 0x56120fcef4e0 2024-05-08T18:05:06.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.333+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.333+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94635574675856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfd260 con 0x56120fcef4e0 2024-05-08T18:05:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.333+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfdc60 con 0x56120fcef4e0 2024-05-08T18:05:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.333+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94635574672000] v27'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8660040040 con 0x56120fceaaf0 2024-05-08T18:05:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.334+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94635574675856] v27'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.334+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.334+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94635574676912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfd7f0 con 0x56120fcef4e0 2024-05-08T18:05:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.334+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfe4f0 con 0x56120fcef4e0 2024-05-08T18:05:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.335+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94635574676912] v27'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.335+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.335+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94635574681568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfe4b0 con 0x56120fcef4e0 2024-05-08T18:05:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.335+0000 7f86677af800 20 add_watcher() i=0 2024-05-08T18:05:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.335+0000 7f86677af800 20 add_watcher() i=1 2024-05-08T18:05:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.335+0000 7f86677af800 20 add_watcher() i=2 2024-05-08T18:05:06.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.335+0000 7f86677af800 20 add_watcher() i=3 2024-05-08T18:05:06.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.335+0000 7f86677af800 20 add_watcher() i=4 2024-05-08T18:05:06.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.335+0000 7f86677af800 20 add_watcher() i=5 2024-05-08T18:05:06.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.335+0000 7f86677af800 20 add_watcher() i=6 2024-05-08T18:05:06.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.336+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94635574681568] v27'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.336+0000 7f86677af800 20 add_watcher() i=7 2024-05-08T18:05:06.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.336+0000 7f86677af800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:06.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.336+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fd01970 con 0x56120fceaaf0 2024-05-08T18:05:06.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.337+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f8660040040 con 0x56120fceaaf0 2024-05-08T18:05:06.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.337+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94635574674160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfedf0 con 0x56120fceaaf0 2024-05-08T18:05:06.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.338+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94635574674160] v27'86 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8660040040 con 0x56120fceaaf0 2024-05-08T18:05:06.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.338+0000 7f851b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:06.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.338+0000 7f86677af800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:06.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.338+0000 7f86677af800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:06.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.338+0000 7f86677af800 5 note: GC not initialized 2024-05-08T18:05:06.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.338+0000 7f851a7e4640 20 reqs_thread_entry: start 2024-05-08T18:05:06.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.338+0000 7f86677af800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:06.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.338+0000 7f86677af800 20 Filter name: none 2024-05-08T18:05:06.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.338+0000 7f848bfff640 20 reqs_thread_entry: start 2024-05-08T18:05:06.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.338+0000 7f86677af800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:06.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.338+0000 7f86677af800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.338+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fd34e50 con 0x56120fceaaf0 2024-05-08T18:05:06.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.339+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv10 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7f8660040040 con 0x56120fceaaf0 2024-05-08T18:05:06.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.339+0000 7f86677af800 20 rados_obj.operate() r=0 bl.length=556 2024-05-08T18:05:06.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.339+0000 7f86677af800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:06.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.339+0000 7f86677af800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:06.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.339+0000 7f86677af800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:06.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.339+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fd3b630 con 0x56120fceaaf0 2024-05-08T18:05:06.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.340+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~391] v27'11 uv11 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f8660040040 con 0x56120fceaaf0 2024-05-08T18:05:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.340+0000 7f86677af800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.340+0000 7f86677af800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.340+0000 7f86677af800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-05-08T18:05:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.340+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94635574931440 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fd36cd0 con 0x56120fceaaf0 2024-05-08T18:05:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.340+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== watch-notify(notify (1) cookie 94635574672000 notify 115964117002 ret 0) ==== 609+0+0 (crc 0 0 0) 0x7f8660038450 con 0x56120fceaaf0 2024-05-08T18:05:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.340+0000 7f8548ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117002 cookie 94635574672000 notifier 4410 bl.length()=567 2024-05-08T18:05:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.340+0000 7f8548ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.340+0000 7f8548ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.340+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(64 notify.4 [notify cookie 94635574931440 out=8b] v0'0 uv78 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f8660040040 con 0x56120fceaaf0 2024-05-08T18:05:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.340+0000 7f8548ff9640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f8528003ea0 con 0x56120fceaaf0 2024-05-08T18:05:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.341+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8660040040 con 0x56120fceaaf0 2024-05-08T18:05:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.341+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== watch-notify(notify_complete (2) cookie 94635574931440 notify 115964117002 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f8660038620 con 0x56120fceaaf0 2024-05-08T18:05:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.341+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 8.2 8:45671577:users.swift::foo%3afoo2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fd3aff0 con 0x7f865c02e910 2024-05-08T18:05:06.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.342+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 foo:foo2 [delete] v27'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8660040040 con 0x7f865c02e910 2024-05-08T18:05:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.342+0000 7f86677af800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2024-05-08T18:05:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.342+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94635574931440 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56120fd35b20 con 0x7f865c02e910 2024-05-08T18:05:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.342+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== watch-notify(notify (1) cookie 94635574657488 notify 115964116997 ret 0) ==== 201+0+0 (crc 0 0 0) 0x7f8660038620 con 0x7f865c02e910 2024-05-08T18:05:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.342+0000 7f8548ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116997 cookie 94635574657488 notifier 4410 bl.length()=159 2024-05-08T18:05:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.342+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(67 notify.0 [notify cookie 94635574931440 out=8b] v0'0 uv78 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f8660040040 con 0x7f865c02e910 2024-05-08T18:05:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.342+0000 7f8548ff9640 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f8528003ea0 con 0x7f865c02e910 2024-05-08T18:05:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.342+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8660040040 con 0x7f865c02e910 2024-05-08T18:05:06.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.343+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== watch-notify(notify_complete (2) cookie 94635574931440 notify 115964116997 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f86600388c0 con 0x7f865c02e910 2024-05-08T18:05:06.352 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:06.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.344+0000 7f851b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:06.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.344+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94635574674160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fd34170 con 0x56120fceaaf0 2024-05-08T18:05:06.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.345+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94635574674160] v27'87 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8660040040 con 0x56120fceaaf0 2024-05-08T18:05:06.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.345+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94635574657488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfc2f0 con 0x7f865c02e910 2024-05-08T18:05:06.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.346+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94635574657488] v27'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8660040040 con 0x7f865c02e910 2024-05-08T18:05:06.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.346+0000 7f86677af800 20 remove_watcher() i=0 2024-05-08T18:05:06.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.346+0000 7f86677af800 2 removed watcher, disabling cache 2024-05-08T18:05:06.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.346+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94635574662016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfc2f0 con 0x56120fceaaf0 2024-05-08T18:05:06.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.347+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94635574662016] v27'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8660040040 con 0x56120fceaaf0 2024-05-08T18:05:06.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.347+0000 7f86677af800 20 remove_watcher() i=1 2024-05-08T18:05:06.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.347+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94635574655008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfc2f0 con 0x56120fcf3e40 2024-05-08T18:05:06.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.348+0000 7f8664aa6640 1 -- 172.21.15.50:0/2788515326 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94635574655008] v27'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f865c037040 con 0x56120fcf3e40 2024-05-08T18:05:06.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.348+0000 7f86677af800 20 remove_watcher() i=2 2024-05-08T18:05:06.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.348+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94635574668592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcf7820 con 0x7f865c02e910 2024-05-08T18:05:06.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.349+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94635574668592] v27'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8660040040 con 0x7f865c02e910 2024-05-08T18:05:06.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.349+0000 7f86677af800 20 remove_watcher() i=3 2024-05-08T18:05:06.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.349+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94635574672000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfad30 con 0x56120fceaaf0 2024-05-08T18:05:06.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.350+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94635574672000] v27'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8660040040 con 0x56120fceaaf0 2024-05-08T18:05:06.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.350+0000 7f86677af800 20 remove_watcher() i=4 2024-05-08T18:05:06.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.350+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94635574675856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcf7820 con 0x56120fcef4e0 2024-05-08T18:05:06.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.351+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94635574675856] v27'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.351+0000 7f86677af800 20 remove_watcher() i=5 2024-05-08T18:05:06.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.351+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94635574676912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfba80 con 0x56120fcef4e0 2024-05-08T18:05:06.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.352+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94635574676912] v27'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.352+0000 7f86677af800 20 remove_watcher() i=6 2024-05-08T18:05:06.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.352+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94635574681568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56120fcfad30 con 0x56120fcef4e0 2024-05-08T18:05:06.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86652a7640 1 -- 172.21.15.50:0/2788515326 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94635574681568] v27'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f865805a040 con 0x56120fcef4e0 2024-05-08T18:05:06.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 20 remove_watcher() i=7 2024-05-08T18:05:06.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56120fcef4e0 msgr2=0x56120fcf18d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56120fcef4e0 0x56120fcf18d0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56120fceaaf0 msgr2=0x56120fcecfb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56120fceaaf0 0x56120fcecfb0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f865c02e910 msgr2=0x7f865c03a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f865c02e910 0x7f865c03a3c0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56120fcf3e40 msgr2=0x56120fcf6300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56120fcf3e40 0x56120fcf6300 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f853003bba0 msgr2=0x7f853003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f853003bba0 0x7f853003e060 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f865c00ffd0 tx=0x7f865c023040 comp rx=0 tx=0).stop 2024-05-08T18:05:06.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f8665aa8640 1 -- 172.21.15.50:0/2788515326 reap_dead start 2024-05-08T18:05:06.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fcd7ff0 msgr2=0x56120fcdf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fcd7ff0 0x56120fcdf3d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f865807dc40 tx=0x7f86580a0070 comp rx=0 tx=0).stop 2024-05-08T18:05:06.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 shutdown_connections 2024-05-08T18:05:06.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 --2- 172.21.15.50:0/2788515326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fcd7ff0 0x56120fcdf3d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 >> 172.21.15.50:0/2788515326 conn(0x56120fcd4300 msgr2=0x56120fcd1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 shutdown_connections 2024-05-08T18:05:06.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 -- 172.21.15.50:0/2788515326 wait complete. 2024-05-08T18:05:06.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 -- 172.21.15.50:0/452014454 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f853c03bba0 msgr2=0x7f853c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 --2- 172.21.15.50:0/452014454 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f853c03bba0 0x7f853c03e060 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x56120fcb20e0 tx=0x7f865c006000 comp rx=0 tx=0).stop 2024-05-08T18:05:06.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 -- 172.21.15.50:0/452014454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fca3300 msgr2=0x56120fca1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 --2- 172.21.15.50:0/452014454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fca3300 0x56120fca1770 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f865805b7e0 tx=0x7f865805b3e0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 -- 172.21.15.50:0/452014454 shutdown_connections 2024-05-08T18:05:06.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 --2- 172.21.15.50:0/452014454 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f853c03bba0 0x7f853c03e060 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 --2- 172.21.15.50:0/452014454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fca3300 0x56120fca1770 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.353+0000 7f86677af800 1 -- 172.21.15.50:0/452014454 >> 172.21.15.50:0/452014454 conn(0x56120fcbcd50 msgr2=0x56120fcbd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f86677af800 1 -- 172.21.15.50:0/452014454 shutdown_connections 2024-05-08T18:05:06.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f86677af800 1 -- 172.21.15.50:0/452014454 wait complete. 2024-05-08T18:05:06.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f86677af800 1 -- 172.21.15.50:0/856739685 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56120fcb3f40 msgr2=0x56120fcb6330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f86677af800 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56120fcb3f40 0x56120fcb6330 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f86677af800 1 -- 172.21.15.50:0/856739685 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56120fbb17f0 msgr2=0x56120fcb1730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f86677af800 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56120fbb17f0 0x56120fcb1730 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f86677af800 1 -- 172.21.15.50:0/856739685 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56120fcb7e40 msgr2=0x56120fcba230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f86677af800 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56120fcb7e40 0x56120fcba230 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f86677af800 1 -- 172.21.15.50:0/856739685 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f854c03bba0 msgr2=0x7f854c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f86677af800 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f854c03bba0 0x7f854c03e060 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f865c000c00 tx=0x7f865c0013b0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f86677af800 1 -- 172.21.15.50:0/856739685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 msgr2=0x56120fcaa4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f86677af800 1 --2- 172.21.15.50:0/856739685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56120fbb0a70 0x56120fcaa4f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f865801ce60 tx=0x7f8658002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f8665aa8640 1 -- 172.21.15.50:0/856739685 reap_dead start 2024-05-08T18:05:06.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f86677af800 1 -- 172.21.15.50:0/856739685 shutdown_connections 2024-05-08T18:05:06.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f86677af800 1 -- 172.21.15.50:0/856739685 >> 172.21.15.50:0/856739685 conn(0x56120fbab2f0 msgr2=0x56120fc9ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f86677af800 1 -- 172.21.15.50:0/856739685 shutdown_connections 2024-05-08T18:05:06.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.354+0000 7f86677af800 1 -- 172.21.15.50:0/856739685 wait complete. 2024-05-08T18:05:06.364 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:06.364 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2024-05-08T18:05:06.364 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2024-05-08T18:05:06.364 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2024-05-08T18:05:06.410 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:06.410 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:06.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.413+0000 7f37ff698800 1 Processor -- start 2024-05-08T18:05:06.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.413+0000 7f37ff698800 1 -- start start 2024-05-08T18:05:06.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.413+0000 7f37ff698800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55a89c731c70 0x55a89c734060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.413+0000 7f37ff698800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a89c7345a0 con 0x55a89c72e160 2024-05-08T18:05:06.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.413+0000 7f37ff698800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55a89c5cde20 con 0x55a89c731c70 2024-05-08T18:05:06.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.413+0000 7f37fc996640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55a89c731c70 0x55a89c734060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.413+0000 7f37fd197640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55a89c72e160 0x55a89c730550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43066/0 (socket says 172.21.15.50:43066) 2024-05-08T18:05:06.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.413+0000 7f37fd197640 1 -- 172.21.15.50:0/927704112 learned_addr learned my addr 172.21.15.50:0/927704112 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:06.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.413+0000 7f37fc996640 1 -- 172.21.15.50:0/927704112 >> v1:172.21.15.46:6789/0 conn(0x55a89c72e160 legacy=0x55a89c730550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.413+0000 7f37fc996640 1 -- 172.21.15.50:0/927704112 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a89c735780 con 0x55a89c731c70 2024-05-08T18:05:06.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.413+0000 7f37effff640 1 -- 172.21.15.50:0/927704112 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 692374995 0 0) 0x55a89c7345a0 con 0x55a89c72e160 2024-05-08T18:05:06.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.415+0000 7f37fc996640 1 --2- 172.21.15.50:0/927704112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c731c70 0x55a89c734060 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f37f400a470 tx=0x7f37f401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae88fe6a8dddf90 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.415+0000 7f37effff640 1 -- 172.21.15.50:0/927704112 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f4057070 con 0x55a89c731c70 2024-05-08T18:05:06.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.415+0000 7f37effff640 1 -- 172.21.15.50:0/927704112 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f4052570 con 0x55a89c731c70 2024-05-08T18:05:06.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.415+0000 7f37effff640 1 -- 172.21.15.50:0/927704112 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f4052870 con 0x55a89c731c70 2024-05-08T18:05:06.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.416+0000 7f37ff698800 1 -- 172.21.15.50:0/927704112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c731c70 msgr2=0x55a89c734060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.416+0000 7f37ff698800 1 --2- 172.21.15.50:0/927704112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c731c70 0x55a89c734060 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f37f400a470 tx=0x7f37f401ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:06.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.416+0000 7f37ff698800 1 -- 172.21.15.50:0/927704112 shutdown_connections 2024-05-08T18:05:06.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.416+0000 7f37ff698800 1 --2- 172.21.15.50:0/927704112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c731c70 0x55a89c734060 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.416+0000 7f37ff698800 1 -- 172.21.15.50:0/927704112 >> 172.21.15.50:0/927704112 conn(0x55a89c727b50 msgr2=0x55a89c729f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.416+0000 7f37ff698800 1 -- 172.21.15.50:0/927704112 shutdown_connections 2024-05-08T18:05:06.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.416+0000 7f37ff698800 1 -- 172.21.15.50:0/927704112 wait complete. 2024-05-08T18:05:06.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.431+0000 7f37ff698800 1 Processor -- start 2024-05-08T18:05:06.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.432+0000 7f37ff698800 1 -- start start 2024-05-08T18:05:06.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.432+0000 7f37ff698800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72e160 0x55a89c81e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.432+0000 7f37ff698800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a89c678a10 con 0x55a89c72e160 2024-05-08T18:05:06.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.432+0000 7f37fd197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72e160 0x55a89c81e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.432+0000 7f37fd197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72e160 0x55a89c81e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37246/0 (socket says 172.21.15.50:37246) 2024-05-08T18:05:06.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.432+0000 7f37fd197640 1 -- 172.21.15.50:0/4065968173 learned_addr learned my addr 172.21.15.50:0/4065968173 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.432+0000 7f37fd197640 1 -- 172.21.15.50:0/4065968173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a89c81ef60 con 0x55a89c72e160 2024-05-08T18:05:06.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.432+0000 7f37fd197640 1 --2- 172.21.15.50:0/4065968173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72e160 0x55a89c81e990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f37f0001e70 tx=0x7f37f000b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee8f7c64624f37c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.433+0000 7f36eb606640 1 -- 172.21.15.50:0/4065968173 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f0008070 con 0x55a89c72e160 2024-05-08T18:05:06.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.433+0000 7f36eb606640 1 -- 172.21.15.50:0/4065968173 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f000d500 con 0x55a89c72e160 2024-05-08T18:05:06.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.433+0000 7f36eb606640 1 -- 172.21.15.50:0/4065968173 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f000d8a0 con 0x55a89c72e160 2024-05-08T18:05:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.433+0000 7f37ff698800 1 -- 172.21.15.50:0/4065968173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72e160 msgr2=0x55a89c81e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.433+0000 7f37ff698800 1 --2- 172.21.15.50:0/4065968173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72e160 0x55a89c81e990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f37f0001e70 tx=0x7f37f000b6a0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.433+0000 7f37ff698800 1 -- 172.21.15.50:0/4065968173 shutdown_connections 2024-05-08T18:05:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.433+0000 7f37ff698800 1 --2- 172.21.15.50:0/4065968173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72e160 0x55a89c81e990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.433+0000 7f37ff698800 1 -- 172.21.15.50:0/4065968173 >> 172.21.15.50:0/4065968173 conn(0x55a89c727b50 msgr2=0x55a89c81a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.433+0000 7f37ff698800 1 -- 172.21.15.50:0/4065968173 shutdown_connections 2024-05-08T18:05:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.433+0000 7f37ff698800 1 -- 172.21.15.50:0/4065968173 wait complete. 2024-05-08T18:05:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.434+0000 7f37ff698800 1 Processor -- start 2024-05-08T18:05:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.434+0000 7f37ff698800 1 -- start start 2024-05-08T18:05:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.434+0000 7f37ff698800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72e160 0x55a89c826500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.434+0000 7f37ff698800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a89c734a70 con 0x55a89c72e160 2024-05-08T18:05:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.434+0000 7f37fd197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72e160 0x55a89c826500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.434+0000 7f37fd197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72e160 0x55a89c826500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37250/0 (socket says 172.21.15.50:37250) 2024-05-08T18:05:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.434+0000 7f37fd197640 1 -- 172.21.15.50:0/2850036649 learned_addr learned my addr 172.21.15.50:0/2850036649 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.435+0000 7f37fd197640 1 -- 172.21.15.50:0/2850036649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a89c826a40 con 0x55a89c72e160 2024-05-08T18:05:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.435+0000 7f37fd197640 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72e160 0x55a89c826500 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55a89c81f720 tx=0x7f37f0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.435+0000 7f36e9e03640 1 -- 172.21.15.50:0/2850036649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f000f980 con 0x55a89c72e160 2024-05-08T18:05:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.435+0000 7f36e9e03640 1 -- 172.21.15.50:0/2850036649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f000fb20 con 0x55a89c72e160 2024-05-08T18:05:06.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.435+0000 7f37ff698800 1 -- 172.21.15.50:0/2850036649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a89c826cd0 con 0x55a89c72e160 2024-05-08T18:05:06.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.435+0000 7f36e9e03640 1 -- 172.21.15.50:0/2850036649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f000fe00 con 0x55a89c72e160 2024-05-08T18:05:06.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.435+0000 7f37ff698800 1 -- 172.21.15.50:0/2850036649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a89c829980 con 0x55a89c72e160 2024-05-08T18:05:06.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.436+0000 7f36e9e03640 1 -- 172.21.15.50:0/2850036649 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f37f0017040 con 0x55a89c72e160 2024-05-08T18:05:06.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.436+0000 7f36e9e03640 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f36e403bb50 0x7f36e403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.436+0000 7f36e9e03640 1 -- 172.21.15.50:0/2850036649 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f37f0015020 con 0x55a89c72e160 2024-05-08T18:05:06.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.436+0000 7f37fc996640 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f36e403bb50 0x7f36e403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.436+0000 7f37ff698800 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a89c731c70 0x55a89c82d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.436+0000 7f37ff698800 1 -- 172.21.15.50:0/2850036649 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c82dc80 con 0x55a89c731c70 2024-05-08T18:05:06.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.436+0000 7f37fd998640 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a89c731c70 0x55a89c82d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.437+0000 7f37fc996640 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f36e403bb50 0x7f36e403e010 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f37f4054000 tx=0x7f37f404e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.437+0000 7f37fd998640 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a89c731c70 0x55a89c82d740 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.437+0000 7f37fd998640 1 -- 172.21.15.50:0/2850036649 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c731c70 2024-05-08T18:05:06.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.437+0000 7f37ff698800 1 -- 172.21.15.50:0/2850036649 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c82ad10 con 0x55a89c731c70 2024-05-08T18:05:06.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.437+0000 7f37fd998640 1 -- 172.21.15.50:0/2850036649 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c731c70 2024-05-08T18:05:06.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.437+0000 7f37ff698800 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a89c82ff90 0x55a89c832380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.437+0000 7f37ff698800 1 -- 172.21.15.50:0/2850036649 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c832a20 con 0x55a89c82ff90 2024-05-08T18:05:06.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.438+0000 7f37fd197640 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a89c82ff90 0x55a89c832380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.438+0000 7f37fd197640 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a89c82ff90 0x55a89c832380 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.438+0000 7f37fd197640 1 -- 172.21.15.50:0/2850036649 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f37f002d070 con 0x55a89c82ff90 2024-05-08T18:05:06.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.438+0000 7f37ff698800 1 -- 172.21.15.50:0/2850036649 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c82ee30 con 0x55a89c731c70 2024-05-08T18:05:06.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.438+0000 7f37fd998640 1 -- 172.21.15.50:0/2850036649 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c731c70 2024-05-08T18:05:06.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.439+0000 7f37ff698800 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a89c833e80 0x55a89c836270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.439+0000 7f37ff698800 1 -- 172.21.15.50:0/2850036649 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c8369b0 con 0x55a89c833e80 2024-05-08T18:05:06.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.439+0000 7f37fc996640 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a89c833e80 0x55a89c836270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.439+0000 7f37fc996640 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a89c833e80 0x55a89c836270 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.439+0000 7f37fc996640 1 -- 172.21.15.50:0/2850036649 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f37f4052940 con 0x55a89c833e80 2024-05-08T18:05:06.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.440+0000 7f37ff698800 1 Processor -- start 2024-05-08T18:05:06.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.440+0000 7f37ff698800 1 -- start start 2024-05-08T18:05:06.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.440+0000 7f37ff698800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c844cb0 0x55a89c8450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.440+0000 7f37ff698800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a89c61f330 con 0x55a89c844cb0 2024-05-08T18:05:06.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.440+0000 7f37fd197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c844cb0 0x55a89c8450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.440+0000 7f37fd197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c844cb0 0x55a89c8450d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37254/0 (socket says 172.21.15.50:37254) 2024-05-08T18:05:06.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.440+0000 7f37fd197640 1 -- 172.21.15.50:0/886619743 learned_addr learned my addr 172.21.15.50:0/886619743 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.440+0000 7f37fd197640 1 -- 172.21.15.50:0/886619743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a89c845610 con 0x55a89c844cb0 2024-05-08T18:05:06.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.441+0000 7f37fd197640 1 --2- 172.21.15.50:0/886619743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c844cb0 0x55a89c8450d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f37f000ea70 tx=0x7f37f001ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.441+0000 7f36e2ffd640 1 -- 172.21.15.50:0/886619743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f004a320 con 0x55a89c844cb0 2024-05-08T18:05:06.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.441+0000 7f36e2ffd640 1 -- 172.21.15.50:0/886619743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f00183d0 con 0x55a89c844cb0 2024-05-08T18:05:06.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.441+0000 7f36e2ffd640 1 -- 172.21.15.50:0/886619743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f004a4c0 con 0x55a89c844cb0 2024-05-08T18:05:06.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.441+0000 7f37ff698800 1 -- 172.21.15.50:0/886619743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a89c846080 con 0x55a89c844cb0 2024-05-08T18:05:06.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.441+0000 7f37ff698800 1 -- 172.21.15.50:0/886619743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a89c846580 con 0x55a89c844cb0 2024-05-08T18:05:06.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.441+0000 7f36e2ffd640 1 -- 172.21.15.50:0/886619743 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f37f00186b0 con 0x55a89c844cb0 2024-05-08T18:05:06.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.442+0000 7f36e2ffd640 1 --2- 172.21.15.50:0/886619743 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f36d403bba0 0x7f36d403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.442+0000 7f36e2ffd640 1 -- 172.21.15.50:0/886619743 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f37f004ae00 con 0x55a89c844cb0 2024-05-08T18:05:06.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.442+0000 7f37fc996640 1 --2- 172.21.15.50:0/886619743 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f36d403bba0 0x7f36d403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.442+0000 7f37ff698800 1 Processor -- start 2024-05-08T18:05:06.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.442+0000 7f37ff698800 1 -- start start 2024-05-08T18:05:06.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.442+0000 7f37fc996640 1 --2- 172.21.15.50:0/886619743 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f36d403bba0 0x7f36d403e060 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55a89c82ac30 tx=0x7f37f4005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.442+0000 7f37ff698800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72d750 0x55a89c72db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.442+0000 7f37ff698800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a89c81f2e0 con 0x55a89c72d750 2024-05-08T18:05:06.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.442+0000 7f37fd197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72d750 0x55a89c72db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.443+0000 7f37fd197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72d750 0x55a89c72db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37268/0 (socket says 172.21.15.50:37268) 2024-05-08T18:05:06.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.443+0000 7f37fd197640 1 -- 172.21.15.50:0/3760508806 learned_addr learned my addr 172.21.15.50:0/3760508806 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.443+0000 7f37fd197640 1 -- 172.21.15.50:0/3760508806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a89c72bf50 con 0x55a89c72d750 2024-05-08T18:05:06.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.443+0000 7f37fd197640 1 --2- 172.21.15.50:0/3760508806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72d750 0x55a89c72db70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f37f001cfd0 tx=0x7f37f0051e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992818ff4a646100 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.443+0000 7f36e0ff9640 1 -- 172.21.15.50:0/3760508806 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f003ed10 con 0x55a89c72d750 2024-05-08T18:05:06.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.443+0000 7f36e0ff9640 1 -- 172.21.15.50:0/3760508806 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f004a6a0 con 0x55a89c72d750 2024-05-08T18:05:06.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.443+0000 7f36e0ff9640 1 -- 172.21.15.50:0/3760508806 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f0051910 con 0x55a89c72d750 2024-05-08T18:05:06.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.443+0000 7f37ff698800 1 -- 172.21.15.50:0/3760508806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72d750 msgr2=0x55a89c72db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.443+0000 7f37ff698800 1 --2- 172.21.15.50:0/3760508806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72d750 0x55a89c72db70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f37f001cfd0 tx=0x7f37f0051e30 comp rx=0 tx=0).stop 2024-05-08T18:05:06.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.444+0000 7f37ff698800 1 -- 172.21.15.50:0/3760508806 shutdown_connections 2024-05-08T18:05:06.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.444+0000 7f37ff698800 1 --2- 172.21.15.50:0/3760508806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72d750 0x55a89c72db70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.444+0000 7f37ff698800 1 -- 172.21.15.50:0/3760508806 >> 172.21.15.50:0/3760508806 conn(0x55a89c84e390 msgr2=0x55a89c84c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.444+0000 7f37ff698800 1 -- 172.21.15.50:0/3760508806 shutdown_connections 2024-05-08T18:05:06.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.444+0000 7f37ff698800 1 -- 172.21.15.50:0/3760508806 wait complete. 2024-05-08T18:05:06.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.444+0000 7f37ff698800 1 Processor -- start 2024-05-08T18:05:06.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.444+0000 7f37ff698800 1 -- start start 2024-05-08T18:05:06.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.444+0000 7f37ff698800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72d750 0x55a89c85b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.444+0000 7f37ff698800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a89c72c360 con 0x55a89c72d750 2024-05-08T18:05:06.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.444+0000 7f37fd197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72d750 0x55a89c85b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.444+0000 7f37fd197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72d750 0x55a89c85b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37272/0 (socket says 172.21.15.50:37272) 2024-05-08T18:05:06.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.444+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 learned_addr learned my addr 172.21.15.50:0/1424278071 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.445+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a89c85bb10 con 0x55a89c72d750 2024-05-08T18:05:06.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.445+0000 7f37fd197640 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72d750 0x55a89c85b5d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f37f001cfd0 tx=0x7f37f003ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.445+0000 7f36d2ffd640 1 -- 172.21.15.50:0/1424278071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f006e070 con 0x55a89c72d750 2024-05-08T18:05:06.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.445+0000 7f36d2ffd640 1 -- 172.21.15.50:0/1424278071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f002e560 con 0x55a89c72d750 2024-05-08T18:05:06.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.445+0000 7f36d2ffd640 1 -- 172.21.15.50:0/1424278071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37f002f5a0 con 0x55a89c72d750 2024-05-08T18:05:06.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.445+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a89c85bb10 con 0x55a89c72d750 2024-05-08T18:05:06.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.445+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a89c85bb10 con 0x55a89c72d750 2024-05-08T18:05:06.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.446+0000 7f36d2ffd640 1 -- 172.21.15.50:0/1424278071 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f37f0071070 con 0x55a89c72d750 2024-05-08T18:05:06.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.446+0000 7f36d2ffd640 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f36c803bb50 0x7f36c803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.446+0000 7f36d2ffd640 1 -- 172.21.15.50:0/1424278071 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f37f0059020 con 0x55a89c72d750 2024-05-08T18:05:06.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.446+0000 7f37fc996640 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f36c803bb50 0x7f36c803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.446+0000 7f37ff698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.446+0000 7f37ff698800 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a89c866d10 0x55a89c8691d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.446+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c869920 con 0x55a89c866d10 2024-05-08T18:05:06.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.446+0000 7f37fd998640 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a89c866d10 0x55a89c8691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.447+0000 7f37fc996640 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f36c803bb50 0x7f36c803e010 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f37f4052fd0 tx=0x7f37f404f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.447+0000 7f37fd998640 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a89c866d10 0x55a89c8691d0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.447+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c866d10 2024-05-08T18:05:06.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.447+0000 7f37ff698800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.447+0000 7f37ff698800 20 realm 2024-05-08T18:05:06.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.447+0000 7f37ff698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.447+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c869d30 con 0x55a89c866d10 2024-05-08T18:05:06.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.447+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c866d10 2024-05-08T18:05:06.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.447+0000 7f37ff698800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.447+0000 7f37ff698800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:06.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.447+0000 7f37ff698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.447+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86a600 con 0x55a89c866d10 2024-05-08T18:05:06.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.448+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c866d10 2024-05-08T18:05:06.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.448+0000 7f37ff698800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.448+0000 7f37ff698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.448+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86a9a0 con 0x55a89c866d10 2024-05-08T18:05:06.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.448+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c866d10 2024-05-08T18:05:06.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.448+0000 7f37ff698800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:06.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.448+0000 7f37ff698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.448+0000 7f37ff698800 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a89c86b4d0 0x55a89c86d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.448+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86e000 con 0x55a89c86b4d0 2024-05-08T18:05:06.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.448+0000 7f37fd197640 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a89c86b4d0 0x55a89c86d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.449+0000 7f37fd197640 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a89c86b4d0 0x55a89c86d8c0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.449+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.449+0000 7f37ff698800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:06.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.449+0000 7f37ff698800 20 searching for the correct realm 2024-05-08T18:05:06.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.449+0000 7f37ff698800 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a89c870060 0x55a89c8724f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.449+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a89c872bc0 con 0x55a89c870060 2024-05-08T18:05:06.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.449+0000 7f37fc996640 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a89c870060 0x55a89c8724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.449+0000 7f37fc996640 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a89c870060 0x55a89c8724f0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.450+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f4081070 con 0x55a89c870060 2024-05-08T18:05:06.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.450+0000 7f37fc996640 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f37f4076040 0x7f37f4083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.450+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a89c872bc0 con 0x7f37f4076040 2024-05-08T18:05:06.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.450+0000 7f37fd998640 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f37f4076040 0x7f37f4083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.450+0000 7f37fd998640 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f37f4076040 0x7f37f4083410 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.450+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f800a4a0 con 0x7f37f4076040 2024-05-08T18:05:06.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.450+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86a9a0 con 0x55a89c86b4d0 2024-05-08T18:05:06.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.451+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.451+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86a9a0 con 0x55a89c866d10 2024-05-08T18:05:06.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.451+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c866d10 2024-05-08T18:05:06.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.451+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86a9a0 con 0x55a89c866d10 2024-05-08T18:05:06.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.451+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c866d10 2024-05-08T18:05:06.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.451+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86a9a0 con 0x55a89c86b4d0 2024-05-08T18:05:06.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.452+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.452+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86a9a0 con 0x55a89c86b4d0 2024-05-08T18:05:06.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.452+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.452+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86a9a0 con 0x7f37f4076040 2024-05-08T18:05:06.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.452+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f800a4a0 con 0x7f37f4076040 2024-05-08T18:05:06.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.452+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86a9a0 con 0x55a89c870060 2024-05-08T18:05:06.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.452+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f4081070 con 0x55a89c870060 2024-05-08T18:05:06.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.452+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86a9a0 con 0x55a89c866d10 2024-05-08T18:05:06.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.453+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c866d10 2024-05-08T18:05:06.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.453+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86a9a0 con 0x55a89c86b4d0 2024-05-08T18:05:06.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.453+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.453+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86a9a0 con 0x55a89c866d10 2024-05-08T18:05:06.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.453+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c866d10 2024-05-08T18:05:06.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.453+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86a9a0 con 0x7f37f4076040 2024-05-08T18:05:06.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.453+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f37f800a4a0 con 0x7f37f4076040 2024-05-08T18:05:06.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.453+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86a9a0 con 0x55a89c870060 2024-05-08T18:05:06.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.454+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f4081070 con 0x55a89c870060 2024-05-08T18:05:06.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.454+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x55a89c870060 2024-05-08T18:05:06.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.454+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f4081070 con 0x55a89c870060 2024-05-08T18:05:06.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.454+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x55a89c86b4d0 2024-05-08T18:05:06.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.454+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.454+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x55a89c870060 2024-05-08T18:05:06.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.454+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f4081070 con 0x55a89c870060 2024-05-08T18:05:06.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.455+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x55a89c870060 2024-05-08T18:05:06.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.455+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f37f4081070 con 0x55a89c870060 2024-05-08T18:05:06.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.455+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x55a89c866d10 2024-05-08T18:05:06.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.455+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c866d10 2024-05-08T18:05:06.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.455+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x55a89c866d10 2024-05-08T18:05:06.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.455+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c866d10 2024-05-08T18:05:06.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.455+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x55a89c86b4d0 2024-05-08T18:05:06.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.455+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.456+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x55a89c870060 2024-05-08T18:05:06.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.456+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f4081070 con 0x55a89c870060 2024-05-08T18:05:06.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.456+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x7f37f4076040 2024-05-08T18:05:06.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.456+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f800a4a0 con 0x7f37f4076040 2024-05-08T18:05:06.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.456+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x55a89c866d10 2024-05-08T18:05:06.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.456+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c866d10 2024-05-08T18:05:06.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.456+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x55a89c86b4d0 2024-05-08T18:05:06.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.456+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.457+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x55a89c870060 2024-05-08T18:05:06.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.457+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f4081070 con 0x55a89c870060 2024-05-08T18:05:06.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.457+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x55a89c86b4d0 2024-05-08T18:05:06.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.457+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.457+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x7f37f4076040 2024-05-08T18:05:06.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.457+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f800a4a0 con 0x7f37f4076040 2024-05-08T18:05:06.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.457+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x55a89c870060 2024-05-08T18:05:06.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.457+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f4081070 con 0x55a89c870060 2024-05-08T18:05:06.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.457+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x55a89c86b4d0 2024-05-08T18:05:06.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.458+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.458+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x55a89c866d10 2024-05-08T18:05:06.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.458+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c866d10 2024-05-08T18:05:06.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.458+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f37f4090040 con 0x7f37f4076040 2024-05-08T18:05:06.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.458+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f37f800a4a0 con 0x7f37f4076040 2024-05-08T18:05:06.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.458+0000 7f37ff698800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:06.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.458+0000 7f37ff698800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:06.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.458+0000 7f37ff698800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:06.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.458+0000 7f37ff698800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:06.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.458+0000 7f37ff698800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:06.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.458+0000 7f37ff698800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:06.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.458+0000 7f37ff698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.458+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86e980 con 0x55a89c866d10 2024-05-08T18:05:06.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.458+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c866d10 2024-05-08T18:05:06.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86e980 con 0x55a89c866d10 2024-05-08T18:05:06.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f37f800a4a0 con 0x55a89c866d10 2024-05-08T18:05:06.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:06.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86e950 con 0x55a89c870060 2024-05-08T18:05:06.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f37f4081070 con 0x55a89c870060 2024-05-08T18:05:06.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:06.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 20 zone default found 2024-05-08T18:05:06.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 4 Realm: () 2024-05-08T18:05:06.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:06.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:06.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:06.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 20 zonegroup default 2024-05-08T18:05:06.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c86f220 con 0x55a89c86b4d0 2024-05-08T18:05:06.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.459+0000 7f37ff698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.460+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c874a20 con 0x55a89c870060 2024-05-08T18:05:06.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.460+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f37f4081070 con 0x55a89c870060 2024-05-08T18:05:06.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.460+0000 7f37ff698800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.460+0000 7f37ff698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.460+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c874dc0 con 0x55a89c870060 2024-05-08T18:05:06.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.460+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f37f4081070 con 0x55a89c870060 2024-05-08T18:05:06.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.460+0000 7f37ff698800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.460+0000 7f37ff698800 20 started sync module instance, tier type = 2024-05-08T18:05:06.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.460+0000 7f37ff698800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:06.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.460+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c873ec0 con 0x7f37f4076040 2024-05-08T18:05:06.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.461+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f800a4a0 con 0x7f37f4076040 2024-05-08T18:05:06.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.461+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94182668977088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c873ec0 con 0x7f37f4076040 2024-05-08T18:05:06.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.461+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c875820 con 0x55a89c866d10 2024-05-08T18:05:06.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.462+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94182668977088] v27'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f8040040 con 0x7f37f4076040 2024-05-08T18:05:06.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.462+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f8040040 con 0x55a89c866d10 2024-05-08T18:05:06.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.462+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94182668980992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c873ec0 con 0x55a89c866d10 2024-05-08T18:05:06.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.462+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c876230 con 0x55a89c870060 2024-05-08T18:05:06.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.463+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f4081070 con 0x55a89c870060 2024-05-08T18:05:06.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.463+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94182668983680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c876a80 con 0x55a89c870060 2024-05-08T18:05:06.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.463+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c877580 con 0x7f37f4076040 2024-05-08T18:05:06.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.464+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94182668980992] v27'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f8040040 con 0x55a89c866d10 2024-05-08T18:05:06.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.464+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94182668983680] v27'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f4081070 con 0x55a89c870060 2024-05-08T18:05:06.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.464+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f8040040 con 0x7f37f4076040 2024-05-08T18:05:06.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.465+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94182668985808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c873a10 con 0x7f37f4076040 2024-05-08T18:05:06.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.465+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c877e20 con 0x55a89c866d10 2024-05-08T18:05:06.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.465+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f8040040 con 0x55a89c866d10 2024-05-08T18:05:06.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.465+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94182668990752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c877d40 con 0x55a89c866d10 2024-05-08T18:05:06.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.465+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c878d60 con 0x55a89c86b4d0 2024-05-08T18:05:06.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.466+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94182668985808] v27'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f8040040 con 0x7f37f4076040 2024-05-08T18:05:06.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.466+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.466+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94182668994608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c879150 con 0x55a89c86b4d0 2024-05-08T18:05:06.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.466+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c879b10 con 0x55a89c86b4d0 2024-05-08T18:05:06.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.466+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94182668990752] v27'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f8040040 con 0x55a89c866d10 2024-05-08T18:05:06.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.467+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94182668994608] v27'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.468+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.468+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94182668998240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c879eb0 con 0x55a89c86b4d0 2024-05-08T18:05:06.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.468+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c87a740 con 0x55a89c86b4d0 2024-05-08T18:05:06.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.469+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94182668998240] v27'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.469+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.469+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94182668993040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c87a920 con 0x55a89c86b4d0 2024-05-08T18:05:06.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.469+0000 7f37ff698800 20 add_watcher() i=0 2024-05-08T18:05:06.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.469+0000 7f37ff698800 20 add_watcher() i=1 2024-05-08T18:05:06.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.469+0000 7f37ff698800 20 add_watcher() i=2 2024-05-08T18:05:06.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.469+0000 7f37ff698800 20 add_watcher() i=3 2024-05-08T18:05:06.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.469+0000 7f37ff698800 20 add_watcher() i=4 2024-05-08T18:05:06.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.469+0000 7f37ff698800 20 add_watcher() i=5 2024-05-08T18:05:06.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.469+0000 7f37ff698800 20 add_watcher() i=6 2024-05-08T18:05:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.470+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94182668993040] v27'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.470+0000 7f37ff698800 20 add_watcher() i=7 2024-05-08T18:05:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.470+0000 7f37ff698800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.470+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c87dcf0 con 0x55a89c866d10 2024-05-08T18:05:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.470+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f37f8040040 con 0x55a89c866d10 2024-05-08T18:05:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.470+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94182669002768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c87b0e0 con 0x55a89c866d10 2024-05-08T18:05:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.471+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94182669002768] v27'88 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f37f8040040 con 0x55a89c866d10 2024-05-08T18:05:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.471+0000 7f36b37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.471+0000 7f37ff698800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.471+0000 7f37ff698800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:06.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.471+0000 7f37ff698800 5 note: GC not initialized 2024-05-08T18:05:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.471+0000 7f36b27e4640 20 reqs_thread_entry: start 2024-05-08T18:05:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.472+0000 7f37ff698800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.472+0000 7f37ff698800 20 Filter name: none 2024-05-08T18:05:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.472+0000 7f3623fff640 20 reqs_thread_entry: start 2024-05-08T18:05:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.472+0000 7f37ff698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.472+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c8b0cd0 con 0x55a89c866d10 2024-05-08T18:05:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.472+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f37f8040040 con 0x55a89c866d10 2024-05-08T18:05:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.472+0000 7f37ff698800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.472+0000 7f37ff698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.472+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c8b16c0 con 0x55a89c866d10 2024-05-08T18:05:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.472+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f37f8040040 con 0x55a89c866d10 2024-05-08T18:05:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.472+0000 7f37ff698800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.472+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a89c8b26d0 con 0x7f37f4076040 2024-05-08T18:05:06.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.473+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(64 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 155+0+0 (crc 0 0 0) 0x7f37f8040040 con 0x7f37f4076040 2024-05-08T18:05:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.474+0000 7f36b37e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.474+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94182669002768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c87b950 con 0x55a89c866d10 2024-05-08T18:05:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.475+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94182669002768] v27'89 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f37f8040040 con 0x55a89c866d10 2024-05-08T18:05:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.475+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94182668977088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c87a810 con 0x7f37f4076040 2024-05-08T18:05:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.476+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94182668977088] v27'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f8040040 con 0x7f37f4076040 2024-05-08T18:05:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.476+0000 7f37ff698800 20 remove_watcher() i=0 2024-05-08T18:05:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.476+0000 7f37ff698800 2 removed watcher, disabling cache 2024-05-08T18:05:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.476+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94182668980992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c87a810 con 0x55a89c866d10 2024-05-08T18:05:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.477+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94182668980992] v27'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f8040040 con 0x55a89c866d10 2024-05-08T18:05:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.477+0000 7f37ff698800 20 remove_watcher() i=1 2024-05-08T18:05:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.477+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94182668983680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c875300 con 0x55a89c870060 2024-05-08T18:05:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.478+0000 7f37fc996640 1 -- 172.21.15.50:0/1424278071 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94182668983680] v27'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f4081070 con 0x55a89c870060 2024-05-08T18:05:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.478+0000 7f37ff698800 20 remove_watcher() i=2 2024-05-08T18:05:06.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.478+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94182668985808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c87a810 con 0x7f37f4076040 2024-05-08T18:05:06.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.479+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94182668985808] v27'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f8040040 con 0x7f37f4076040 2024-05-08T18:05:06.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.479+0000 7f37ff698800 20 remove_watcher() i=3 2024-05-08T18:05:06.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.479+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94182668990752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c875300 con 0x55a89c866d10 2024-05-08T18:05:06.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.480+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94182668990752] v27'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f8040040 con 0x55a89c866d10 2024-05-08T18:05:06.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.480+0000 7f37ff698800 20 remove_watcher() i=4 2024-05-08T18:05:06.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.480+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94182668994608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c877920 con 0x55a89c86b4d0 2024-05-08T18:05:06.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.481+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94182668994608] v27'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.481+0000 7f37ff698800 20 remove_watcher() i=5 2024-05-08T18:05:06.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.481+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94182668998240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c875300 con 0x55a89c86b4d0 2024-05-08T18:05:06.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.482+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94182668998240] v27'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.482+0000 7f37ff698800 20 remove_watcher() i=6 2024-05-08T18:05:06.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.482+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94182668993040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a89c877920 con 0x55a89c86b4d0 2024-05-08T18:05:06.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37fd197640 1 -- 172.21.15.50:0/1424278071 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94182668993040] v27'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f37f0070030 con 0x55a89c86b4d0 2024-05-08T18:05:06.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 20 remove_watcher() i=7 2024-05-08T18:05:06.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a89c86b4d0 msgr2=0x55a89c86d8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a89c86b4d0 0x55a89c86d8c0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a89c866d10 msgr2=0x55a89c8691d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a89c866d10 0x55a89c8691d0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f37f4076040 msgr2=0x7f37f4083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f37f4076040 0x7f37f4083410 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a89c870060 msgr2=0x55a89c8724f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a89c870060 0x55a89c8724f0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f36c803bb50 msgr2=0x7f36c803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f36c803bb50 0x7f36c803e010 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f37f4052fd0 tx=0x7f37f404f000 comp rx=0 tx=0).stop 2024-05-08T18:05:06.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72d750 msgr2=0x55a89c85b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37fd998640 1 -- 172.21.15.50:0/1424278071 reap_dead start 2024-05-08T18:05:06.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72d750 0x55a89c85b5d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f37f001cfd0 tx=0x7f37f003ad30 comp rx=0 tx=0).stop 2024-05-08T18:05:06.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 shutdown_connections 2024-05-08T18:05:06.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 --2- 172.21.15.50:0/1424278071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72d750 0x55a89c85b5d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 >> 172.21.15.50:0/1424278071 conn(0x55a89c84e390 msgr2=0x55a89c84b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 shutdown_connections 2024-05-08T18:05:06.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 -- 172.21.15.50:0/1424278071 wait complete. 2024-05-08T18:05:06.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 -- 172.21.15.50:0/886619743 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f36d403bba0 msgr2=0x7f36d403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 --2- 172.21.15.50:0/886619743 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f36d403bba0 0x7f36d403e060 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55a89c82ac30 tx=0x7f37f4005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:06.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 -- 172.21.15.50:0/886619743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c844cb0 msgr2=0x55a89c8450d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 --2- 172.21.15.50:0/886619743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c844cb0 0x55a89c8450d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f37f000ea70 tx=0x7f37f001ab20 comp rx=0 tx=0).stop 2024-05-08T18:05:06.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 -- 172.21.15.50:0/886619743 shutdown_connections 2024-05-08T18:05:06.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 --2- 172.21.15.50:0/886619743 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f36d403bba0 0x7f36d403e060 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 --2- 172.21.15.50:0/886619743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c844cb0 0x55a89c8450d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.483+0000 7f37ff698800 1 -- 172.21.15.50:0/886619743 >> 172.21.15.50:0/886619743 conn(0x55a89c838d50 msgr2=0x55a89c839150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37ff698800 1 -- 172.21.15.50:0/886619743 shutdown_connections 2024-05-08T18:05:06.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37ff698800 1 -- 172.21.15.50:0/886619743 wait complete. 2024-05-08T18:05:06.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37ff698800 1 -- 172.21.15.50:0/2850036649 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a89c82ff90 msgr2=0x55a89c832380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37ff698800 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55a89c82ff90 0x55a89c832380 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37ff698800 1 -- 172.21.15.50:0/2850036649 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a89c731c70 msgr2=0x55a89c82d740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37ff698800 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55a89c731c70 0x55a89c82d740 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37ff698800 1 -- 172.21.15.50:0/2850036649 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a89c833e80 msgr2=0x55a89c836270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37ff698800 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55a89c833e80 0x55a89c836270 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37ff698800 1 -- 172.21.15.50:0/2850036649 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f36e403bb50 msgr2=0x7f36e403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37ff698800 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f36e403bb50 0x7f36e403e010 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f37f4054000 tx=0x7f37f404e000 comp rx=0 tx=0).stop 2024-05-08T18:05:06.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37ff698800 1 -- 172.21.15.50:0/2850036649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72e160 msgr2=0x55a89c826500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37ff698800 1 --2- 172.21.15.50:0/2850036649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a89c72e160 0x55a89c826500 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55a89c81f720 tx=0x7f37f0000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37fd998640 1 -- 172.21.15.50:0/2850036649 reap_dead start 2024-05-08T18:05:06.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37ff698800 1 -- 172.21.15.50:0/2850036649 shutdown_connections 2024-05-08T18:05:06.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37ff698800 1 -- 172.21.15.50:0/2850036649 >> 172.21.15.50:0/2850036649 conn(0x55a89c727b50 msgr2=0x55a89c818ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37ff698800 1 -- 172.21.15.50:0/2850036649 shutdown_connections 2024-05-08T18:05:06.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.484+0000 7f37ff698800 1 -- 172.21.15.50:0/2850036649 wait complete. 2024-05-08T18:05:06.492 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-08T18:05:06.492 DEBUG:tasks.util.rgw: json result: [] 2024-05-08T18:05:06.492 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2024-05-08T18:05:06.492 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2024-05-08T18:05:06.492 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2024-05-08T18:05:06.537 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:06.537 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:06.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.540+0000 7f07077a2800 1 Processor -- start 2024-05-08T18:05:06.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.540+0000 7f07077a2800 1 -- start start 2024-05-08T18:05:06.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.540+0000 7f07077a2800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5604e95346f0 0x5604e9534b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.540+0000 7f07077a2800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5604e9535190 con 0x5604e95358b0 2024-05-08T18:05:06.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.540+0000 7f07077a2800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5604e93d4e20 con 0x5604e95346f0 2024-05-08T18:05:06.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.541+0000 7f070529a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5604e95346f0 0x5604e9534b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.541+0000 7f070529a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5604e95346f0 0x5604e9534b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37284/0 (socket says 172.21.15.50:37284) 2024-05-08T18:05:06.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.541+0000 7f06fca99640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5604e95358b0 0x5604e9535d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43080/0 (socket says 172.21.15.50:43080) 2024-05-08T18:05:06.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.541+0000 7f070529a640 1 -- 172.21.15.50:0/4047933309 learned_addr learned my addr 172.21.15.50:0/4047933309 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.541+0000 7f070529a640 1 -- 172.21.15.50:0/4047933309 >> v1:172.21.15.46:6789/0 conn(0x5604e95358b0 legacy=0x5604e9535d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.541+0000 7f070529a640 1 -- 172.21.15.50:0/4047933309 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604e9536470 con 0x5604e95346f0 2024-05-08T18:05:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.543+0000 7f070529a640 1 --2- 172.21.15.50:0/4047933309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 0x5604e9534b10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f06f800a520 tx=0x7f06f801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c7a5b6e184ef07 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.543+0000 7f0704a99640 1 -- 172.21.15.50:0/4047933309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f8057070 con 0x5604e95346f0 2024-05-08T18:05:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.543+0000 7f0704a99640 1 -- 172.21.15.50:0/4047933309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f8052570 con 0x5604e95346f0 2024-05-08T18:05:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.543+0000 7f0704a99640 1 -- 172.21.15.50:0/4047933309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f8052870 con 0x5604e95346f0 2024-05-08T18:05:06.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.543+0000 7f07077a2800 1 -- 172.21.15.50:0/4047933309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 msgr2=0x5604e9534b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.543+0000 7f07077a2800 1 --2- 172.21.15.50:0/4047933309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 0x5604e9534b10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f06f800a520 tx=0x7f06f801d010 comp rx=0 tx=0).stop 2024-05-08T18:05:06.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.544+0000 7f07077a2800 1 -- 172.21.15.50:0/4047933309 shutdown_connections 2024-05-08T18:05:06.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.544+0000 7f07077a2800 1 --2- 172.21.15.50:0/4047933309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 0x5604e9534b10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.544+0000 7f07077a2800 1 -- 172.21.15.50:0/4047933309 >> 172.21.15.50:0/4047933309 conn(0x5604e952fc70 msgr2=0x5604e95320b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.544+0000 7f07077a2800 1 -- 172.21.15.50:0/4047933309 shutdown_connections 2024-05-08T18:05:06.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.544+0000 7f07077a2800 1 -- 172.21.15.50:0/4047933309 wait complete. 2024-05-08T18:05:06.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.559+0000 7f07077a2800 1 Processor -- start 2024-05-08T18:05:06.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.559+0000 7f07077a2800 1 -- start start 2024-05-08T18:05:06.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.559+0000 7f07077a2800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 0x5604e94a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.559+0000 7f07077a2800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5604e947fa10 con 0x5604e95346f0 2024-05-08T18:05:06.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.559+0000 7f070529a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 0x5604e94a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.559+0000 7f070529a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 0x5604e94a9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37296/0 (socket says 172.21.15.50:37296) 2024-05-08T18:05:06.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.559+0000 7f070529a640 1 -- 172.21.15.50:0/3944652941 learned_addr learned my addr 172.21.15.50:0/3944652941 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.560+0000 7f070529a640 1 -- 172.21.15.50:0/3944652941 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604e94a84a0 con 0x5604e95346f0 2024-05-08T18:05:06.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.560+0000 7f070529a640 1 --2- 172.21.15.50:0/3944652941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 0x5604e94a9ee0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f06f8055040 tx=0x7f06f804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd4e0e4a29f9a18 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.560+0000 7f05f3606640 1 -- 172.21.15.50:0/3944652941 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f8057040 con 0x5604e95346f0 2024-05-08T18:05:06.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.560+0000 7f05f3606640 1 -- 172.21.15.50:0/3944652941 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f8052870 con 0x5604e95346f0 2024-05-08T18:05:06.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.560+0000 7f05f3606640 1 -- 172.21.15.50:0/3944652941 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f8052b90 con 0x5604e95346f0 2024-05-08T18:05:06.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.561+0000 7f07077a2800 1 -- 172.21.15.50:0/3944652941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 msgr2=0x5604e94a9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.561+0000 7f07077a2800 1 --2- 172.21.15.50:0/3944652941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 0x5604e94a9ee0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f06f8055040 tx=0x7f06f804c000 comp rx=0 tx=0).stop 2024-05-08T18:05:06.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.561+0000 7f07077a2800 1 -- 172.21.15.50:0/3944652941 shutdown_connections 2024-05-08T18:05:06.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.561+0000 7f07077a2800 1 --2- 172.21.15.50:0/3944652941 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 0x5604e94a9ee0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.561+0000 7f07077a2800 1 -- 172.21.15.50:0/3944652941 >> 172.21.15.50:0/3944652941 conn(0x5604e952fc70 msgr2=0x5604e9621570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.561+0000 7f07077a2800 1 -- 172.21.15.50:0/3944652941 shutdown_connections 2024-05-08T18:05:06.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.561+0000 7f07077a2800 1 -- 172.21.15.50:0/3944652941 wait complete. 2024-05-08T18:05:06.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.561+0000 7f07077a2800 1 Processor -- start 2024-05-08T18:05:06.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.561+0000 7f07077a2800 1 -- start start 2024-05-08T18:05:06.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.561+0000 7f07077a2800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 0x5604e962f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.561+0000 7f07077a2800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5604e953dd80 con 0x5604e95346f0 2024-05-08T18:05:06.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.562+0000 7f070529a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 0x5604e962f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.562+0000 7f070529a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 0x5604e962f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37300/0 (socket says 172.21.15.50:37300) 2024-05-08T18:05:06.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.562+0000 7f070529a640 1 -- 172.21.15.50:0/3292418204 learned_addr learned my addr 172.21.15.50:0/3292418204 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.562+0000 7f070529a640 1 -- 172.21.15.50:0/3292418204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604e962fe40 con 0x5604e95346f0 2024-05-08T18:05:06.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.562+0000 7f070529a640 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 0x5604e962f900 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f06f8052540 tx=0x7f06f8003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.562+0000 7f05f1e03640 1 -- 172.21.15.50:0/3292418204 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f8073070 con 0x5604e95346f0 2024-05-08T18:05:06.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.562+0000 7f05f1e03640 1 -- 172.21.15.50:0/3292418204 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f805d570 con 0x5604e95346f0 2024-05-08T18:05:06.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.562+0000 7f05f1e03640 1 -- 172.21.15.50:0/3292418204 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f8057070 con 0x5604e95346f0 2024-05-08T18:05:06.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.562+0000 7f07077a2800 1 -- 172.21.15.50:0/3292418204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5604e96300d0 con 0x5604e95346f0 2024-05-08T18:05:06.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.562+0000 7f07077a2800 1 -- 172.21.15.50:0/3292418204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604e9632d80 con 0x5604e95346f0 2024-05-08T18:05:06.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.563+0000 7f05f1e03640 1 -- 172.21.15.50:0/3292418204 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f06f8076070 con 0x5604e95346f0 2024-05-08T18:05:06.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.563+0000 7f05f1e03640 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f05ec03bba0 0x7f05ec03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.563+0000 7f05f1e03640 1 -- 172.21.15.50:0/3292418204 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f06f8094130 con 0x5604e95346f0 2024-05-08T18:05:06.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.563+0000 7f06fca99640 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f05ec03bba0 0x7f05ec03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.564+0000 7f07077a2800 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604e95358b0 0x5604e9636b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.564+0000 7f07077a2800 1 -- 172.21.15.50:0/3292418204 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e9637080 con 0x5604e95358b0 2024-05-08T18:05:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.564+0000 7f0705a9b640 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604e95358b0 0x5604e9636b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.564+0000 7f06fca99640 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f05ec03bba0 0x7f05ec03e060 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f06f0001e30 tx=0x7f06f0001390 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.564+0000 7f0705a9b640 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604e95358b0 0x5604e9636b40 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.564+0000 7f0705a9b640 1 -- 172.21.15.50:0/3292418204 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e95358b0 2024-05-08T18:05:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.564+0000 7f07077a2800 1 -- 172.21.15.50:0/3292418204 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e9634110 con 0x5604e95358b0 2024-05-08T18:05:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.565+0000 7f0705a9b640 1 -- 172.21.15.50:0/3292418204 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e95358b0 2024-05-08T18:05:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.565+0000 7f07077a2800 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604e9639370 0x5604e963b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.565+0000 7f07077a2800 1 -- 172.21.15.50:0/3292418204 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e963be30 con 0x5604e9639370 2024-05-08T18:05:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.565+0000 7f070529a640 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604e9639370 0x5604e963b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.565+0000 7f070529a640 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604e9639370 0x5604e963b760 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.565+0000 7f070529a640 1 -- 172.21.15.50:0/3292418204 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f06f8075030 con 0x5604e9639370 2024-05-08T18:05:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.565+0000 7f07077a2800 1 -- 172.21.15.50:0/3292418204 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e9638210 con 0x5604e95358b0 2024-05-08T18:05:06.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.566+0000 7f0705a9b640 1 -- 172.21.15.50:0/3292418204 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e95358b0 2024-05-08T18:05:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.566+0000 7f07077a2800 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604e963d270 0x5604e963f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.566+0000 7f07077a2800 1 -- 172.21.15.50:0/3292418204 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e963fda0 con 0x5604e963d270 2024-05-08T18:05:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.566+0000 7f06fca99640 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604e963d270 0x5604e963f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.566+0000 7f06fca99640 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604e963d270 0x5604e963f660 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.566+0000 7f06fca99640 1 -- 172.21.15.50:0/3292418204 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f06f000bd50 con 0x5604e963d270 2024-05-08T18:05:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.567+0000 7f07077a2800 1 Processor -- start 2024-05-08T18:05:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.567+0000 7f07077a2800 1 -- start start 2024-05-08T18:05:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.567+0000 7f07077a2800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e964e110 0x5604e964e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.567+0000 7f07077a2800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5604e9426330 con 0x5604e964e110 2024-05-08T18:05:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.567+0000 7f070529a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e964e110 0x5604e964e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.567+0000 7f070529a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e964e110 0x5604e964e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37312/0 (socket says 172.21.15.50:37312) 2024-05-08T18:05:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.567+0000 7f070529a640 1 -- 172.21.15.50:0/1695159305 learned_addr learned my addr 172.21.15.50:0/1695159305 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.567+0000 7f070529a640 1 -- 172.21.15.50:0/1695159305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604e964ea70 con 0x5604e964e110 2024-05-08T18:05:06.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.567+0000 7f070529a640 1 --2- 172.21.15.50:0/1695159305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e964e110 0x5604e964e530 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f06f806d000 tx=0x7f06f808bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.568+0000 7f05eaffd640 1 -- 172.21.15.50:0/1695159305 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f808a070 con 0x5604e964e110 2024-05-08T18:05:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.568+0000 7f05eaffd640 1 -- 172.21.15.50:0/1695159305 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f8092600 con 0x5604e964e110 2024-05-08T18:05:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.568+0000 7f07077a2800 1 -- 172.21.15.50:0/1695159305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5604e964f4e0 con 0x5604e964e110 2024-05-08T18:05:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.568+0000 7f05eaffd640 1 -- 172.21.15.50:0/1695159305 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f8077e10 con 0x5604e964e110 2024-05-08T18:05:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.568+0000 7f07077a2800 1 -- 172.21.15.50:0/1695159305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604e964f9e0 con 0x5604e964e110 2024-05-08T18:05:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.568+0000 7f05eaffd640 1 -- 172.21.15.50:0/1695159305 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f06f805cb30 con 0x5604e964e110 2024-05-08T18:05:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.569+0000 7f05eaffd640 1 --2- 172.21.15.50:0/1695159305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f05e003bab0 0x7f05e003df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.569+0000 7f05eaffd640 1 -- 172.21.15.50:0/1695159305 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f06f809aa00 con 0x5604e964e110 2024-05-08T18:05:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.569+0000 7f06fca99640 1 --2- 172.21.15.50:0/1695159305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f05e003bab0 0x7f05e003df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.569+0000 7f07077a2800 1 Processor -- start 2024-05-08T18:05:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.569+0000 7f07077a2800 1 -- start start 2024-05-08T18:05:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.569+0000 7f06fca99640 1 --2- 172.21.15.50:0/1695159305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f05e003bab0 0x7f05e003df70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x5604e96374f0 tx=0x7f06f000f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.569+0000 7f07077a2800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e965b200 0x5604e965b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.569+0000 7f07077a2800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5604e94a8760 con 0x5604e965b200 2024-05-08T18:05:06.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.569+0000 7f070529a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e965b200 0x5604e965b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.569+0000 7f070529a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e965b200 0x5604e965b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37324/0 (socket says 172.21.15.50:37324) 2024-05-08T18:05:06.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.569+0000 7f070529a640 1 -- 172.21.15.50:0/2679971747 learned_addr learned my addr 172.21.15.50:0/2679971747 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.570+0000 7f070529a640 1 -- 172.21.15.50:0/2679971747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604e965bbf0 con 0x5604e965b200 2024-05-08T18:05:06.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.570+0000 7f070529a640 1 --2- 172.21.15.50:0/2679971747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e965b200 0x5604e965b620 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f06f8077fd0 tx=0x7f06f8095dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37185030de10fcfa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.570+0000 7f05e8ff9640 1 -- 172.21.15.50:0/2679971747 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f80a7070 con 0x5604e965b200 2024-05-08T18:05:06.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.570+0000 7f05e8ff9640 1 -- 172.21.15.50:0/2679971747 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f8085e50 con 0x5604e965b200 2024-05-08T18:05:06.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.570+0000 7f05e8ff9640 1 -- 172.21.15.50:0/2679971747 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f8076800 con 0x5604e965b200 2024-05-08T18:05:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.570+0000 7f07077a2800 1 -- 172.21.15.50:0/2679971747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e965b200 msgr2=0x5604e965b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.570+0000 7f07077a2800 1 --2- 172.21.15.50:0/2679971747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e965b200 0x5604e965b620 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f06f8077fd0 tx=0x7f06f8095dc0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.570+0000 7f07077a2800 1 -- 172.21.15.50:0/2679971747 shutdown_connections 2024-05-08T18:05:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.570+0000 7f07077a2800 1 --2- 172.21.15.50:0/2679971747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e965b200 0x5604e965b620 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.570+0000 7f07077a2800 1 -- 172.21.15.50:0/2679971747 >> 172.21.15.50:0/2679971747 conn(0x5604e9657560 msgr2=0x5604e96555c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.571+0000 7f07077a2800 1 -- 172.21.15.50:0/2679971747 shutdown_connections 2024-05-08T18:05:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.571+0000 7f07077a2800 1 -- 172.21.15.50:0/2679971747 wait complete. 2024-05-08T18:05:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.571+0000 7f07077a2800 1 Processor -- start 2024-05-08T18:05:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.571+0000 7f07077a2800 1 -- start start 2024-05-08T18:05:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.571+0000 7f07077a2800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e965b200 0x5604e9662590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.571+0000 7f07077a2800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5604e965c2a0 con 0x5604e965b200 2024-05-08T18:05:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.571+0000 7f070529a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e965b200 0x5604e9662590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.571+0000 7f070529a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e965b200 0x5604e9662590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37332/0 (socket says 172.21.15.50:37332) 2024-05-08T18:05:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.571+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 learned_addr learned my addr 172.21.15.50:0/282212200 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.572+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604e9662ad0 con 0x5604e965b200 2024-05-08T18:05:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.572+0000 7f070529a640 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e965b200 0x5604e9662590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f06f8077fd0 tx=0x7f06f808b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.572+0000 7f05d6ffd640 1 -- 172.21.15.50:0/282212200 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f808b670 con 0x5604e965b200 2024-05-08T18:05:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.572+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5604e9662ad0 con 0x5604e965b200 2024-05-08T18:05:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.572+0000 7f05d6ffd640 1 -- 172.21.15.50:0/282212200 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f8077710 con 0x5604e965b200 2024-05-08T18:05:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.572+0000 7f05d6ffd640 1 -- 172.21.15.50:0/282212200 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06f80779f0 con 0x5604e965b200 2024-05-08T18:05:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.572+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604e9662ad0 con 0x5604e965b200 2024-05-08T18:05:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.573+0000 7f05d6ffd640 1 -- 172.21.15.50:0/282212200 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f06f80b9020 con 0x5604e965b200 2024-05-08T18:05:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.573+0000 7f05d6ffd640 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f05d003bb50 0x7f05d003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.573+0000 7f05d6ffd640 1 -- 172.21.15.50:0/282212200 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f06f80c29b0 con 0x5604e965b200 2024-05-08T18:05:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.573+0000 7f06fca99640 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f05d003bb50 0x7f05d003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.574+0000 7f07077a2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.574+0000 7f07077a2800 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604e966da10 0x5604e966fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.574+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e9670620 con 0x5604e966da10 2024-05-08T18:05:06.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.574+0000 7f06fca99640 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f05d003bb50 0x7f05d003e010 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x5604e96401a0 tx=0x7f06f0010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.574+0000 7f0705a9b640 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604e966da10 0x5604e966fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.574+0000 7f0705a9b640 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604e966da10 0x5604e966fed0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.574+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e966da10 2024-05-08T18:05:06.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f07077a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f07077a2800 20 realm 2024-05-08T18:05:06.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f07077a2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e9670a30 con 0x5604e966da10 2024-05-08T18:05:06.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e966da10 2024-05-08T18:05:06.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f07077a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f07077a2800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:06.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f07077a2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e9671300 con 0x5604e966da10 2024-05-08T18:05:06.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e966da10 2024-05-08T18:05:06.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f07077a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f07077a2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e966da10 2024-05-08T18:05:06.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e966da10 2024-05-08T18:05:06.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f07077a2800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:06.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f07077a2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f07077a2800 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604e9672190 0x5604e9674580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.575+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e9674cc0 con 0x5604e9672190 2024-05-08T18:05:06.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.576+0000 7f070529a640 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604e9672190 0x5604e9674580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.576+0000 7f070529a640 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604e9672190 0x5604e9674580 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.576+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.576+0000 7f07077a2800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:06.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.576+0000 7f07077a2800 20 searching for the correct realm 2024-05-08T18:05:06.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.576+0000 7f07077a2800 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604e9676d60 0x5604e9679220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.576+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96798f0 con 0x5604e9676d60 2024-05-08T18:05:06.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.576+0000 7f06fca99640 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604e9676d60 0x5604e9679220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.577+0000 7f06fca99640 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604e9676d60 0x5604e9679220 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.577+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06f0037040 con 0x5604e9676d60 2024-05-08T18:05:06.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.577+0000 7f06fca99640 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f06f000f070 0x7f06f002e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.577+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96798f0 con 0x7f06f000f070 2024-05-08T18:05:06.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.577+0000 7f0705a9b640 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f06f000f070 0x7f06f002e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.577+0000 7f0705a9b640 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f06f000f070 0x7f06f002e910 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.578+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f070000a4a0 con 0x7f06f000f070 2024-05-08T18:05:06.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.578+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9672190 2024-05-08T18:05:06.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.578+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.578+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e966da10 2024-05-08T18:05:06.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.578+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e966da10 2024-05-08T18:05:06.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.578+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e966da10 2024-05-08T18:05:06.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.578+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e966da10 2024-05-08T18:05:06.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.578+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9672190 2024-05-08T18:05:06.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.579+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.579+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9672190 2024-05-08T18:05:06.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.579+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.579+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x7f06f000f070 2024-05-08T18:05:06.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.579+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f070000a4a0 con 0x7f06f000f070 2024-05-08T18:05:06.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.579+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9676d60 2024-05-08T18:05:06.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.579+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06f0037040 con 0x5604e9676d60 2024-05-08T18:05:06.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.579+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e966da10 2024-05-08T18:05:06.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.580+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e966da10 2024-05-08T18:05:06.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.580+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9672190 2024-05-08T18:05:06.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.580+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.580+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e966da10 2024-05-08T18:05:06.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.580+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e966da10 2024-05-08T18:05:06.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.580+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x7f06f000f070 2024-05-08T18:05:06.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.580+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f070000a4a0 con 0x7f06f000f070 2024-05-08T18:05:06.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.580+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9676d60 2024-05-08T18:05:06.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.581+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06f0037040 con 0x5604e9676d60 2024-05-08T18:05:06.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.581+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9676d60 2024-05-08T18:05:06.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.581+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06f0037040 con 0x5604e9676d60 2024-05-08T18:05:06.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.581+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9672190 2024-05-08T18:05:06.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.581+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.581+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9676d60 2024-05-08T18:05:06.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.582+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06f0037040 con 0x5604e9676d60 2024-05-08T18:05:06.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.582+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9676d60 2024-05-08T18:05:06.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.582+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f06f0037040 con 0x5604e9676d60 2024-05-08T18:05:06.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.582+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e966da10 2024-05-08T18:05:06.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.582+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e966da10 2024-05-08T18:05:06.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.582+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e966da10 2024-05-08T18:05:06.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.582+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e966da10 2024-05-08T18:05:06.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.582+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9672190 2024-05-08T18:05:06.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.583+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.583+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9676d60 2024-05-08T18:05:06.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.583+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06f0037040 con 0x5604e9676d60 2024-05-08T18:05:06.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.583+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x7f06f000f070 2024-05-08T18:05:06.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.583+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f070000a4a0 con 0x7f06f000f070 2024-05-08T18:05:06.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.583+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e966da10 2024-05-08T18:05:06.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.584+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e966da10 2024-05-08T18:05:06.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.584+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9672190 2024-05-08T18:05:06.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.584+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.584+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9676d60 2024-05-08T18:05:06.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.584+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06f0037040 con 0x5604e9676d60 2024-05-08T18:05:06.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.584+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9672190 2024-05-08T18:05:06.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.584+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.584+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x7f06f000f070 2024-05-08T18:05:06.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.585+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f070000a4a0 con 0x7f06f000f070 2024-05-08T18:05:06.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.585+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9676d60 2024-05-08T18:05:06.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.585+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06f0037040 con 0x5604e9676d60 2024-05-08T18:05:06.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.585+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e9672190 2024-05-08T18:05:06.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.585+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.585+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x5604e966da10 2024-05-08T18:05:06.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.585+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e966da10 2024-05-08T18:05:06.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.585+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5604e96716a0 con 0x7f06f000f070 2024-05-08T18:05:06.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f070000a4a0 con 0x7f06f000f070 2024-05-08T18:05:06.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:06.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:06.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:06.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:06.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:06.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:06.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e9675660 con 0x5604e966da10 2024-05-08T18:05:06.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e966da10 2024-05-08T18:05:06.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e9675660 con 0x5604e966da10 2024-05-08T18:05:06.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f070000a4a0 con 0x5604e966da10 2024-05-08T18:05:06.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:06.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e9675610 con 0x5604e9676d60 2024-05-08T18:05:06.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f06f0037040 con 0x5604e9676d60 2024-05-08T18:05:06.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:06.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 20 zone default found 2024-05-08T18:05:06.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 4 Realm: () 2024-05-08T18:05:06.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:06.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:06.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:06.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 20 zonegroup default 2024-05-08T18:05:06.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.586+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e9675f00 con 0x5604e9672190 2024-05-08T18:05:06.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.587+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.587+0000 7f07077a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.587+0000 7f07077a2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.587+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e967b750 con 0x5604e9676d60 2024-05-08T18:05:06.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.587+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f06f0037040 con 0x5604e9676d60 2024-05-08T18:05:06.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.587+0000 7f07077a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.587+0000 7f07077a2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.587+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e967baf0 con 0x5604e9676d60 2024-05-08T18:05:06.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.587+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f06f0037040 con 0x5604e9676d60 2024-05-08T18:05:06.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.587+0000 7f07077a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.587+0000 7f07077a2800 20 started sync module instance, tier type = 2024-05-08T18:05:06.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.587+0000 7f07077a2800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:06.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.587+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967abf0 con 0x7f06f000f070 2024-05-08T18:05:06.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.588+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f070000a4a0 con 0x7f06f000f070 2024-05-08T18:05:06.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.588+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94579095744784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967abf0 con 0x7f06f000f070 2024-05-08T18:05:06.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.588+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967c5b0 con 0x5604e966da10 2024-05-08T18:05:06.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.589+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94579095744784] v27'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0700040040 con 0x7f06f000f070 2024-05-08T18:05:06.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.589+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0700040040 con 0x5604e966da10 2024-05-08T18:05:06.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.589+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94579095748768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967abf0 con 0x5604e966da10 2024-05-08T18:05:06.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.589+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967d050 con 0x5604e9676d60 2024-05-08T18:05:06.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.590+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94579095748768] v27'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0700040040 con 0x5604e966da10 2024-05-08T18:05:06.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.591+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06f0037040 con 0x5604e9676d60 2024-05-08T18:05:06.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.591+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94579095751488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967d3f0 con 0x5604e9676d60 2024-05-08T18:05:06.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.591+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967deb0 con 0x7f06f000f070 2024-05-08T18:05:06.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.592+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94579095751488] v27'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06f0037040 con 0x5604e9676d60 2024-05-08T18:05:06.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.592+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0700040040 con 0x7f06f000f070 2024-05-08T18:05:06.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.592+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94579095755136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967e2a0 con 0x7f06f000f070 2024-05-08T18:05:06.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.592+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967eca0 con 0x5604e966da10 2024-05-08T18:05:06.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.593+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0700040040 con 0x5604e966da10 2024-05-08T18:05:06.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.593+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94579095758768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967ebd0 con 0x5604e966da10 2024-05-08T18:05:06.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.593+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94579095755136] v27'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0700040040 con 0x7f06f000f070 2024-05-08T18:05:06.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.593+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967a740 con 0x5604e9672190 2024-05-08T18:05:06.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.594+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94579095758768] v27'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0700040040 con 0x5604e966da10 2024-05-08T18:05:06.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.594+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.594+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94579095762656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967fbb0 con 0x5604e9672190 2024-05-08T18:05:06.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.594+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e9680400 con 0x5604e9672190 2024-05-08T18:05:06.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.595+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.595+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94579095767072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e96810f0 con 0x5604e9672190 2024-05-08T18:05:06.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.595+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e9681af0 con 0x5604e9672190 2024-05-08T18:05:06.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.595+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94579095762656] v27'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.595+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94579095767072] v27'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.596+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.596+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94579095761040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e96810f0 con 0x5604e9672190 2024-05-08T18:05:06.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.596+0000 7f07077a2800 20 add_watcher() i=0 2024-05-08T18:05:06.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.596+0000 7f07077a2800 20 add_watcher() i=1 2024-05-08T18:05:06.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.596+0000 7f07077a2800 20 add_watcher() i=2 2024-05-08T18:05:06.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.596+0000 7f07077a2800 20 add_watcher() i=3 2024-05-08T18:05:06.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.596+0000 7f07077a2800 20 add_watcher() i=4 2024-05-08T18:05:06.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.596+0000 7f07077a2800 20 add_watcher() i=5 2024-05-08T18:05:06.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.596+0000 7f07077a2800 20 add_watcher() i=6 2024-05-08T18:05:06.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.597+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94579095761040] v27'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.597+0000 7f07077a2800 20 add_watcher() i=7 2024-05-08T18:05:06.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.597+0000 7f07077a2800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:06.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.597+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e96849b0 con 0x5604e966da10 2024-05-08T18:05:06.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.597+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f0700040040 con 0x5604e966da10 2024-05-08T18:05:06.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.597+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94579095768592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e9681e30 con 0x5604e966da10 2024-05-08T18:05:06.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.598+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94579095768592] v27'90 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0700040040 con 0x5604e966da10 2024-05-08T18:05:06.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.598+0000 7f05bbfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:06.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.598+0000 7f07077a2800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:06.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.598+0000 7f07077a2800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:06.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.598+0000 7f07077a2800 5 note: GC not initialized 2024-05-08T18:05:06.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.599+0000 7f05bafe5640 20 reqs_thread_entry: start 2024-05-08T18:05:06.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.599+0000 7f07077a2800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:06.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.599+0000 7f07077a2800 20 Filter name: none 2024-05-08T18:05:06.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.599+0000 7f05b8fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:06.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.599+0000 7f07077a2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.599+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e96b7a30 con 0x5604e966da10 2024-05-08T18:05:06.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.599+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f0700040040 con 0x5604e966da10 2024-05-08T18:05:06.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.599+0000 7f07077a2800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:06.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.599+0000 7f07077a2800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.599+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e96b8420 con 0x5604e966da10 2024-05-08T18:05:06.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.600+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f0700040040 con 0x5604e966da10 2024-05-08T18:05:06.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.600+0000 7f07077a2800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:06.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.600+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5604e96b9430 con 0x7f06f000f070 2024-05-08T18:05:06.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.600+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(64 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 155+0+0 (crc 0 0 0) 0x7f0700040040 con 0x7f06f000f070 2024-05-08T18:05:06.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.601+0000 7f05bbfe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:06.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.601+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94579095768592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e966ae30 con 0x5604e966da10 2024-05-08T18:05:06.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.602+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94579095768592] v27'91 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0700040040 con 0x5604e966da10 2024-05-08T18:05:06.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.602+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94579095744784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e9680de0 con 0x7f06f000f070 2024-05-08T18:05:06.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.603+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94579095744784] v27'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0700040040 con 0x7f06f000f070 2024-05-08T18:05:06.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.603+0000 7f07077a2800 20 remove_watcher() i=0 2024-05-08T18:05:06.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.603+0000 7f07077a2800 2 removed watcher, disabling cache 2024-05-08T18:05:06.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.603+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94579095748768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967b0f0 con 0x5604e966da10 2024-05-08T18:05:06.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.604+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94579095748768] v27'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0700040040 con 0x5604e966da10 2024-05-08T18:05:06.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.604+0000 7f07077a2800 20 remove_watcher() i=1 2024-05-08T18:05:06.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.604+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94579095751488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967c0a0 con 0x5604e9676d60 2024-05-08T18:05:06.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.605+0000 7f06fca99640 1 -- 172.21.15.50:0/282212200 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94579095751488] v27'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06f0037040 con 0x5604e9676d60 2024-05-08T18:05:06.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.605+0000 7f07077a2800 20 remove_watcher() i=2 2024-05-08T18:05:06.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.605+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94579095755136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967cb40 con 0x7f06f000f070 2024-05-08T18:05:06.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.606+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94579095755136] v27'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0700040040 con 0x7f06f000f070 2024-05-08T18:05:06.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.606+0000 7f07077a2800 20 remove_watcher() i=3 2024-05-08T18:05:06.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.606+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94579095758768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967cb40 con 0x5604e966da10 2024-05-08T18:05:06.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.607+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94579095758768] v27'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0700040040 con 0x5604e966da10 2024-05-08T18:05:06.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.607+0000 7f07077a2800 20 remove_watcher() i=4 2024-05-08T18:05:06.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.607+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94579095762656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967e7b0 con 0x5604e9672190 2024-05-08T18:05:06.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.608+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94579095762656] v27'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.608+0000 7f07077a2800 20 remove_watcher() i=5 2024-05-08T18:05:06.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.608+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94579095767072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967e7b0 con 0x5604e9672190 2024-05-08T18:05:06.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.609+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94579095767072] v27'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.609+0000 7f07077a2800 20 remove_watcher() i=6 2024-05-08T18:05:06.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.609+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94579095761040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5604e967e7b0 con 0x5604e9672190 2024-05-08T18:05:06.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f070529a640 1 -- 172.21.15.50:0/282212200 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94579095761040] v27'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f06f80a9030 con 0x5604e9672190 2024-05-08T18:05:06.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 20 remove_watcher() i=7 2024-05-08T18:05:06.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604e9672190 msgr2=0x5604e9674580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604e9672190 0x5604e9674580 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604e966da10 msgr2=0x5604e966fed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604e966da10 0x5604e966fed0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f06f000f070 msgr2=0x7f06f002e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f06f000f070 0x7f06f002e910 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604e9676d60 msgr2=0x5604e9679220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604e9676d60 0x5604e9679220 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f05d003bb50 msgr2=0x7f05d003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f05d003bb50 0x7f05d003e010 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x5604e96401a0 tx=0x7f06f0010000 comp rx=0 tx=0).stop 2024-05-08T18:05:06.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e965b200 msgr2=0x5604e9662590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 --2- 172.21.15.50:0/282212200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e965b200 0x5604e9662590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f06f8077fd0 tx=0x7f06f808b260 comp rx=0 tx=0).stop 2024-05-08T18:05:06.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 reap_dead start 2024-05-08T18:05:06.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f0705a9b640 1 -- 172.21.15.50:0/282212200 reap_dead start 2024-05-08T18:05:06.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 shutdown_connections 2024-05-08T18:05:06.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 >> 172.21.15.50:0/282212200 conn(0x5604e9657560 msgr2=0x5604e9654bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 shutdown_connections 2024-05-08T18:05:06.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 -- 172.21.15.50:0/282212200 wait complete. 2024-05-08T18:05:06.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 -- 172.21.15.50:0/1695159305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f05e003bab0 msgr2=0x7f05e003df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 --2- 172.21.15.50:0/1695159305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f05e003bab0 0x7f05e003df70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x5604e96374f0 tx=0x7f06f000f000 comp rx=0 tx=0).stop 2024-05-08T18:05:06.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 -- 172.21.15.50:0/1695159305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e964e110 msgr2=0x5604e964e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 --2- 172.21.15.50:0/1695159305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e964e110 0x5604e964e530 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f06f806d000 tx=0x7f06f808bd80 comp rx=0 tx=0).stop 2024-05-08T18:05:06.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.610+0000 7f07077a2800 1 -- 172.21.15.50:0/1695159305 shutdown_connections 2024-05-08T18:05:06.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 --2- 172.21.15.50:0/1695159305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f05e003bab0 0x7f05e003df70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 --2- 172.21.15.50:0/1695159305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e964e110 0x5604e964e530 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 -- 172.21.15.50:0/1695159305 >> 172.21.15.50:0/1695159305 conn(0x5604e96421b0 msgr2=0x5604e96425b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 -- 172.21.15.50:0/1695159305 shutdown_connections 2024-05-08T18:05:06.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 -- 172.21.15.50:0/1695159305 wait complete. 2024-05-08T18:05:06.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 -- 172.21.15.50:0/3292418204 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604e9639370 msgr2=0x5604e963b760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5604e9639370 0x5604e963b760 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 -- 172.21.15.50:0/3292418204 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604e95358b0 msgr2=0x5604e9636b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5604e95358b0 0x5604e9636b40 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 -- 172.21.15.50:0/3292418204 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604e963d270 msgr2=0x5604e963f660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5604e963d270 0x5604e963f660 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 -- 172.21.15.50:0/3292418204 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f05ec03bba0 msgr2=0x7f05ec03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f05ec03bba0 0x7f05ec03e060 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f06f0001e30 tx=0x7f06f0001390 comp rx=0 tx=0).stop 2024-05-08T18:05:06.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 -- 172.21.15.50:0/3292418204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 msgr2=0x5604e962f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 --2- 172.21.15.50:0/3292418204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5604e95346f0 0x5604e962f900 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f06f8052540 tx=0x7f06f8003020 comp rx=0 tx=0).stop 2024-05-08T18:05:06.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f0705a9b640 1 -- 172.21.15.50:0/3292418204 reap_dead start 2024-05-08T18:05:06.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 -- 172.21.15.50:0/3292418204 shutdown_connections 2024-05-08T18:05:06.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 -- 172.21.15.50:0/3292418204 >> 172.21.15.50:0/3292418204 conn(0x5604e952fc70 msgr2=0x5604e94a8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 -- 172.21.15.50:0/3292418204 shutdown_connections 2024-05-08T18:05:06.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.611+0000 7f07077a2800 1 -- 172.21.15.50:0/3292418204 wait complete. 2024-05-08T18:05:06.620 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-08T18:05:06.620 DEBUG:tasks.util.rgw: json result: [] 2024-05-08T18:05:06.710 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:06.711 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2024-05-08T18:05:06.711 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2024-05-08T18:05:06.711 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2024-05-08T18:05:06.757 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:06.757 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:06.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.760+0000 7fb8ff813800 1 Processor -- start 2024-05-08T18:05:06.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.760+0000 7fb8ff813800 1 -- start start 2024-05-08T18:05:06.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.760+0000 7fb8ff813800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5609898a6180 0x5609898a8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.760+0000 7fb8ff813800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5609898ac5c0 con 0x5609898a8ab0 2024-05-08T18:05:06.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.760+0000 7fb8ff813800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x560989745e20 con 0x5609898a6180 2024-05-08T18:05:06.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.760+0000 7fb8fae1a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5609898a6180 0x5609898a8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.760+0000 7fb8fae1a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5609898a6180 0x5609898a8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37346/0 (socket says 172.21.15.50:37346) 2024-05-08T18:05:06.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.760+0000 7fb8f3fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5609898a8ab0 0x5609898aaea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43086/0 (socket says 172.21.15.50:43086) 2024-05-08T18:05:06.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.760+0000 7fb8fae1a640 1 -- 172.21.15.50:0/1371317805 learned_addr learned my addr 172.21.15.50:0/1371317805 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.760+0000 7fb8fae1a640 1 -- 172.21.15.50:0/1371317805 >> v1:172.21.15.46:6789/0 conn(0x5609898a8ab0 legacy=0x5609898aaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.760+0000 7fb8fa619640 1 -- 172.21.15.50:0/1371317805 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3683222754 0 0) 0x5609898ac5c0 con 0x5609898a8ab0 2024-05-08T18:05:06.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.760+0000 7fb8fae1a640 1 -- 172.21.15.50:0/1371317805 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609898ad7a0 con 0x5609898a6180 2024-05-08T18:05:06.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.762+0000 7fb8fae1a640 1 --2- 172.21.15.50:0/1371317805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 0x5609898a8570 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00a520 tx=0x7fb8ec01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48aef328090112b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.762+0000 7fb8fa619640 1 -- 172.21.15.50:0/1371317805 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec057070 con 0x5609898a6180 2024-05-08T18:05:06.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.762+0000 7fb8fa619640 1 -- 172.21.15.50:0/1371317805 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8ec052570 con 0x5609898a6180 2024-05-08T18:05:06.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.762+0000 7fb8fa619640 1 -- 172.21.15.50:0/1371317805 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec04ae60 con 0x5609898a6180 2024-05-08T18:05:06.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.763+0000 7fb8ff813800 1 -- 172.21.15.50:0/1371317805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 msgr2=0x5609898a8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.763+0000 7fb8ff813800 1 --2- 172.21.15.50:0/1371317805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 0x5609898a8570 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00a520 tx=0x7fb8ec01d010 comp rx=0 tx=0).stop 2024-05-08T18:05:06.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.763+0000 7fb8ff813800 1 -- 172.21.15.50:0/1371317805 shutdown_connections 2024-05-08T18:05:06.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.763+0000 7fb8ff813800 1 --2- 172.21.15.50:0/1371317805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 0x5609898a8570 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.763+0000 7fb8ff813800 1 -- 172.21.15.50:0/1371317805 >> 172.21.15.50:0/1371317805 conn(0x56098989fad0 msgr2=0x5609898a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.763+0000 7fb8ff813800 1 -- 172.21.15.50:0/1371317805 shutdown_connections 2024-05-08T18:05:06.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.763+0000 7fb8ff813800 1 -- 172.21.15.50:0/1371317805 wait complete. 2024-05-08T18:05:06.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.778+0000 7fb8ff813800 1 Processor -- start 2024-05-08T18:05:06.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.778+0000 7fb8ff813800 1 -- start start 2024-05-08T18:05:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.779+0000 7fb8ff813800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 0x5609899968e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.779+0000 7fb8ff813800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5609897f0a10 con 0x5609898a6180 2024-05-08T18:05:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.779+0000 7fb8fae1a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 0x5609899968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.779+0000 7fb8fae1a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 0x5609899968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37362/0 (socket says 172.21.15.50:37362) 2024-05-08T18:05:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.779+0000 7fb8fae1a640 1 -- 172.21.15.50:0/789415215 learned_addr learned my addr 172.21.15.50:0/789415215 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.779+0000 7fb8fae1a640 1 -- 172.21.15.50:0/789415215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560989996eb0 con 0x5609898a6180 2024-05-08T18:05:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.779+0000 7fb8fae1a640 1 --2- 172.21.15.50:0/789415215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 0x5609899968e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec055040 tx=0x7fb8ec04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960b57b32d2a99d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.780+0000 7fb7e8e09640 1 -- 172.21.15.50:0/789415215 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec062070 con 0x5609898a6180 2024-05-08T18:05:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.780+0000 7fb7e8e09640 1 -- 172.21.15.50:0/789415215 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8ec052720 con 0x5609898a6180 2024-05-08T18:05:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.780+0000 7fb7e8e09640 1 -- 172.21.15.50:0/789415215 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec057040 con 0x5609898a6180 2024-05-08T18:05:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.780+0000 7fb8ff813800 1 -- 172.21.15.50:0/789415215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 msgr2=0x5609899968e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.780+0000 7fb8ff813800 1 --2- 172.21.15.50:0/789415215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 0x5609899968e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec055040 tx=0x7fb8ec04c000 comp rx=0 tx=0).stop 2024-05-08T18:05:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.780+0000 7fb8ff813800 1 -- 172.21.15.50:0/789415215 shutdown_connections 2024-05-08T18:05:06.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.780+0000 7fb8ff813800 1 --2- 172.21.15.50:0/789415215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 0x5609899968e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.780+0000 7fb8ff813800 1 -- 172.21.15.50:0/789415215 >> 172.21.15.50:0/789415215 conn(0x56098989fad0 msgr2=0x560989992200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.780+0000 7fb8ff813800 1 -- 172.21.15.50:0/789415215 shutdown_connections 2024-05-08T18:05:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.780+0000 7fb8ff813800 1 -- 172.21.15.50:0/789415215 wait complete. 2024-05-08T18:05:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.781+0000 7fb8ff813800 1 Processor -- start 2024-05-08T18:05:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.781+0000 7fb8ff813800 1 -- start start 2024-05-08T18:05:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.781+0000 7fb8ff813800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 0x56098999e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.781+0000 7fb8ff813800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5609898aca90 con 0x5609898a6180 2024-05-08T18:05:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.781+0000 7fb8fae1a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 0x56098999e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.781+0000 7fb8fae1a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 0x56098999e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37366/0 (socket says 172.21.15.50:37366) 2024-05-08T18:05:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.781+0000 7fb8fae1a640 1 -- 172.21.15.50:0/734663816 learned_addr learned my addr 172.21.15.50:0/734663816 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.781+0000 7fb8fae1a640 1 -- 172.21.15.50:0/734663816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56098999e8a0 con 0x5609898a6180 2024-05-08T18:05:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.782+0000 7fb8fae1a640 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 0x56098999e360 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0057e0 tx=0x7fb8ec002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.782+0000 7fb7e7606640 1 -- 172.21.15.50:0/734663816 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec057050 con 0x5609898a6180 2024-05-08T18:05:06.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.782+0000 7fb7e7606640 1 -- 172.21.15.50:0/734663816 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8ec067450 con 0x5609898a6180 2024-05-08T18:05:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.782+0000 7fb8ff813800 1 -- 172.21.15.50:0/734663816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56098999eb30 con 0x5609898a6180 2024-05-08T18:05:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.782+0000 7fb7e7606640 1 -- 172.21.15.50:0/734663816 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec062040 con 0x5609898a6180 2024-05-08T18:05:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.782+0000 7fb8ff813800 1 -- 172.21.15.50:0/734663816 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609899a17e0 con 0x5609898a6180 2024-05-08T18:05:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.783+0000 7fb7e7606640 1 -- 172.21.15.50:0/734663816 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fb8ec067750 con 0x5609898a6180 2024-05-08T18:05:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.783+0000 7fb7e7606640 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb7e003bb50 0x7fb7e003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.783+0000 7fb7e7606640 1 -- 172.21.15.50:0/734663816 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fb8ec094440 con 0x5609898a6180 2024-05-08T18:05:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.783+0000 7fb8f3fff640 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb7e003bb50 0x7fb7e003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.783+0000 7fb8ff813800 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5609898a8ab0 0x5609899a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.783+0000 7fb8ff813800 1 -- 172.21.15.50:0/734663816 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899a5b70 con 0x5609898a8ab0 2024-05-08T18:05:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.783+0000 7fb8fb61b640 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5609898a8ab0 0x5609899a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.783+0000 7fb8f3fff640 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb7e003bb50 0x7fb7e003e010 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4000c00 tx=0x7fb8e40013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.784+0000 7fb8fb61b640 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5609898a8ab0 0x5609899a55a0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.784+0000 7fb8fb61b640 1 -- 172.21.15.50:0/734663816 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609898a8ab0 2024-05-08T18:05:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.784+0000 7fb8ff813800 1 -- 172.21.15.50:0/734663816 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899a2b30 con 0x5609898a8ab0 2024-05-08T18:05:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.784+0000 7fb8fb61b640 1 -- 172.21.15.50:0/734663816 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609898a8ab0 2024-05-08T18:05:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.784+0000 7fb8ff813800 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5609899a7ed0 0x5609899aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.784+0000 7fb8ff813800 1 -- 172.21.15.50:0/734663816 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899aa920 con 0x5609899a7ed0 2024-05-08T18:05:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.784+0000 7fb8fae1a640 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5609899a7ed0 0x5609899aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.785+0000 7fb8fae1a640 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5609899a7ed0 0x5609899aa2c0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.785+0000 7fb8fae1a640 1 -- 172.21.15.50:0/734663816 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fb8ec064070 con 0x5609899a7ed0 2024-05-08T18:05:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.785+0000 7fb8ff813800 1 -- 172.21.15.50:0/734663816 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899a6d70 con 0x5609898a8ab0 2024-05-08T18:05:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.785+0000 7fb8fb61b640 1 -- 172.21.15.50:0/734663816 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609898a8ab0 2024-05-08T18:05:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.785+0000 7fb8ff813800 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5609899abd80 0x5609899ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.785+0000 7fb8ff813800 1 -- 172.21.15.50:0/734663816 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899ae870 con 0x5609899abd80 2024-05-08T18:05:06.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.785+0000 7fb8f3fff640 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5609899abd80 0x5609899ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.786+0000 7fb8f3fff640 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5609899abd80 0x5609899ae170 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.786+0000 7fb8f3fff640 1 -- 172.21.15.50:0/734663816 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fb8e400f670 con 0x5609899abd80 2024-05-08T18:05:06.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.786+0000 7fb8ff813800 1 Processor -- start 2024-05-08T18:05:06.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.786+0000 7fb8ff813800 1 -- start start 2024-05-08T18:05:06.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.787+0000 7fb8ff813800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609899bcbe0 0x5609899bd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.787+0000 7fb8ff813800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560989797330 con 0x5609899bcbe0 2024-05-08T18:05:06.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.787+0000 7fb8fae1a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609899bcbe0 0x5609899bd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.787+0000 7fb8fae1a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609899bcbe0 0x5609899bd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37368/0 (socket says 172.21.15.50:37368) 2024-05-08T18:05:06.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.787+0000 7fb8fae1a640 1 -- 172.21.15.50:0/1798859833 learned_addr learned my addr 172.21.15.50:0/1798859833 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.787+0000 7fb8fae1a640 1 -- 172.21.15.50:0/1798859833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609899bd540 con 0x5609899bcbe0 2024-05-08T18:05:06.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.787+0000 7fb8fae1a640 1 --2- 172.21.15.50:0/1798859833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609899bcbe0 0x5609899bd000 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec05b500 tx=0x7fb8ec090370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.787+0000 7fb7e4e01640 1 -- 172.21.15.50:0/1798859833 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec087020 con 0x5609899bcbe0 2024-05-08T18:05:06.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.787+0000 7fb7e4e01640 1 -- 172.21.15.50:0/1798859833 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8ec05d440 con 0x5609899bcbe0 2024-05-08T18:05:06.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.787+0000 7fb8ff813800 1 -- 172.21.15.50:0/1798859833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5609899bdfb0 con 0x5609899bcbe0 2024-05-08T18:05:06.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.787+0000 7fb7e4e01640 1 -- 172.21.15.50:0/1798859833 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec08be60 con 0x5609899bcbe0 2024-05-08T18:05:06.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.787+0000 7fb8ff813800 1 -- 172.21.15.50:0/1798859833 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609899bdfb0 con 0x5609899bcbe0 2024-05-08T18:05:06.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.788+0000 7fb7e4e01640 1 -- 172.21.15.50:0/1798859833 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fb8ec082070 con 0x5609899bcbe0 2024-05-08T18:05:06.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.788+0000 7fb7e4e01640 1 --2- 172.21.15.50:0/1798859833 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb7d403bb50 0x7fb7d403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.788+0000 7fb7e4e01640 1 -- 172.21.15.50:0/1798859833 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fb8ec095020 con 0x5609899bcbe0 2024-05-08T18:05:06.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.788+0000 7fb8f3fff640 1 --2- 172.21.15.50:0/1798859833 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb7d403bb50 0x7fb7d403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.789+0000 7fb8ff813800 1 Processor -- start 2024-05-08T18:05:06.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.789+0000 7fb8ff813800 1 -- start start 2024-05-08T18:05:06.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.789+0000 7fb8f3fff640 1 --2- 172.21.15.50:0/1798859833 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb7d403bb50 0x7fb7d403e010 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x5609899a2a90 tx=0x7fb8e4006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.789+0000 7fb8ff813800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a57f0 0x5609898a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.789+0000 7fb8ff813800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560989997170 con 0x5609898a57f0 2024-05-08T18:05:06.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.789+0000 7fb8fae1a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a57f0 0x5609898a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.789+0000 7fb8fae1a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a57f0 0x5609898a5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37370/0 (socket says 172.21.15.50:37370) 2024-05-08T18:05:06.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.789+0000 7fb8fae1a640 1 -- 172.21.15.50:0/2547899211 learned_addr learned my addr 172.21.15.50:0/2547899211 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.789+0000 7fb8fae1a640 1 -- 172.21.15.50:0/2547899211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609898a4000 con 0x5609898a57f0 2024-05-08T18:05:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.789+0000 7fb8fae1a640 1 --2- 172.21.15.50:0/2547899211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a57f0 0x5609898a5c10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec051760 tx=0x7fb8ec051850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e67eaea976aea3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.790+0000 7fb7de7fc640 1 -- 172.21.15.50:0/2547899211 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec0a7020 con 0x5609898a57f0 2024-05-08T18:05:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.790+0000 7fb7de7fc640 1 -- 172.21.15.50:0/2547899211 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8ec051920 con 0x5609898a57f0 2024-05-08T18:05:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.790+0000 7fb7de7fc640 1 -- 172.21.15.50:0/2547899211 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec090580 con 0x5609898a57f0 2024-05-08T18:05:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.790+0000 7fb8ff813800 1 -- 172.21.15.50:0/2547899211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a57f0 msgr2=0x5609898a5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.790+0000 7fb8ff813800 1 --2- 172.21.15.50:0/2547899211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a57f0 0x5609898a5c10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec051760 tx=0x7fb8ec051850 comp rx=0 tx=0).stop 2024-05-08T18:05:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.790+0000 7fb8ff813800 1 -- 172.21.15.50:0/2547899211 shutdown_connections 2024-05-08T18:05:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.790+0000 7fb8ff813800 1 --2- 172.21.15.50:0/2547899211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a57f0 0x5609898a5c10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.790+0000 7fb8ff813800 1 -- 172.21.15.50:0/2547899211 >> 172.21.15.50:0/2547899211 conn(0x5609899c5f20 msgr2=0x5609899c3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.790+0000 7fb8ff813800 1 -- 172.21.15.50:0/2547899211 shutdown_connections 2024-05-08T18:05:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.790+0000 7fb8ff813800 1 -- 172.21.15.50:0/2547899211 wait complete. 2024-05-08T18:05:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.790+0000 7fb8ff813800 1 Processor -- start 2024-05-08T18:05:06.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.791+0000 7fb8ff813800 1 -- start start 2024-05-08T18:05:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.791+0000 7fb8ff813800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a57f0 0x5609899d3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.791+0000 7fb8ff813800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5609898a4410 con 0x5609898a57f0 2024-05-08T18:05:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.791+0000 7fb8fae1a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a57f0 0x5609899d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.791+0000 7fb8fae1a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a57f0 0x5609899d3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37372/0 (socket says 172.21.15.50:37372) 2024-05-08T18:05:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.791+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 learned_addr learned my addr 172.21.15.50:0/4110382738 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.791+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609899d36d0 con 0x5609898a57f0 2024-05-08T18:05:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.791+0000 7fb8fae1a640 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a57f0 0x5609899d3190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec071040 tx=0x7fb8ec0915d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.791+0000 7fb7dcff9640 1 -- 172.21.15.50:0/4110382738 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec0bb070 con 0x5609898a57f0 2024-05-08T18:05:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.791+0000 7fb7dcff9640 1 -- 172.21.15.50:0/4110382738 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8ec090580 con 0x5609898a57f0 2024-05-08T18:05:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.791+0000 7fb7dcff9640 1 -- 172.21.15.50:0/4110382738 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb8ec0871c0 con 0x5609898a57f0 2024-05-08T18:05:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.792+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5609899d36d0 con 0x5609898a57f0 2024-05-08T18:05:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.792+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609899d36d0 con 0x5609898a57f0 2024-05-08T18:05:06.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.792+0000 7fb7dcff9640 1 -- 172.21.15.50:0/4110382738 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fb8ec0be070 con 0x5609898a57f0 2024-05-08T18:05:06.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.793+0000 7fb7dcff9640 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb7c403bab0 0x7fb7c403df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.793+0000 7fb7dcff9640 1 -- 172.21.15.50:0/4110382738 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fb8ec0b3050 con 0x5609898a57f0 2024-05-08T18:05:06.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.793+0000 7fb8f3fff640 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb7c403bab0 0x7fb7c403df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.793+0000 7fb8ff813800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.793+0000 7fb8ff813800 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5609899de8b0 0x5609899e0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.793+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899e14c0 con 0x5609899de8b0 2024-05-08T18:05:06.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.793+0000 7fb8f3fff640 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb7c403bab0 0x7fb7c403df70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5609899aec70 tx=0x7fb8e4023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.793+0000 7fb8fb61b640 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5609899de8b0 0x5609899e0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.794+0000 7fb8fb61b640 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5609899de8b0 0x5609899e0d70 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.794+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609899de8b0 2024-05-08T18:05:06.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.794+0000 7fb8ff813800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.794+0000 7fb8ff813800 20 realm 2024-05-08T18:05:06.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.794+0000 7fb8ff813800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.794+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899e18d0 con 0x5609899de8b0 2024-05-08T18:05:06.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.794+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609899de8b0 2024-05-08T18:05:06.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.794+0000 7fb8ff813800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.794+0000 7fb8ff813800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:06.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.794+0000 7fb8ff813800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.794+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899e21a0 con 0x5609899de8b0 2024-05-08T18:05:06.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.794+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609899de8b0 2024-05-08T18:05:06.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.794+0000 7fb8ff813800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.794+0000 7fb8ff813800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.794+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899e2540 con 0x5609899de8b0 2024-05-08T18:05:06.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.795+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609899de8b0 2024-05-08T18:05:06.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.795+0000 7fb8ff813800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:06.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.795+0000 7fb8ff813800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.795+0000 7fb8ff813800 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5609899e3030 0x5609899e5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.795+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899e5b80 con 0x5609899e3030 2024-05-08T18:05:06.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.795+0000 7fb8fae1a640 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5609899e3030 0x5609899e5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.795+0000 7fb8fae1a640 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5609899e3030 0x5609899e5420 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.795+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.795+0000 7fb8ff813800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:06.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.795+0000 7fb8ff813800 20 searching for the correct realm 2024-05-08T18:05:06.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.796+0000 7fb8ff813800 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5609899e7c30 0x5609899ea0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.796+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5609899ea7c0 con 0x5609899e7c30 2024-05-08T18:05:06.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.796+0000 7fb8f3fff640 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5609899e7c30 0x5609899ea0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.796+0000 7fb8f3fff640 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5609899e7c30 0x5609899ea0f0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.796+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8e4037040 con 0x5609899e7c30 2024-05-08T18:05:06.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.796+0000 7fb8f3fff640 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fb8e402ea80 0x7fb8e403a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.796+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5609899ea7c0 con 0x7fb8e402ea80 2024-05-08T18:05:06.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.796+0000 7fb8fb61b640 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fb8e402ea80 0x7fb8e403a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.797+0000 7fb8fb61b640 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fb8e402ea80 0x7fb8e403a3c0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.797+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8f400a4a0 con 0x7fb8e402ea80 2024-05-08T18:05:06.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.797+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5609899e2540 con 0x5609899e3030 2024-05-08T18:05:06.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.797+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.797+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5609899e2540 con 0x5609899de8b0 2024-05-08T18:05:06.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.798+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609899de8b0 2024-05-08T18:05:06.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.798+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5609899e2540 con 0x5609899de8b0 2024-05-08T18:05:06.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.798+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609899de8b0 2024-05-08T18:05:06.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.798+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5609899e2540 con 0x5609899e3030 2024-05-08T18:05:06.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.798+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.798+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899e3030 2024-05-08T18:05:06.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.798+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.798+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x7fb8e402ea80 2024-05-08T18:05:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.798+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8f400a4a0 con 0x7fb8e402ea80 2024-05-08T18:05:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.798+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899e7c30 2024-05-08T18:05:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.799+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8e4037040 con 0x5609899e7c30 2024-05-08T18:05:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.799+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899de8b0 2024-05-08T18:05:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.799+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609899de8b0 2024-05-08T18:05:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.799+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899e3030 2024-05-08T18:05:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.800+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.800+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899de8b0 2024-05-08T18:05:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.800+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609899de8b0 2024-05-08T18:05:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.800+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x7fb8e402ea80 2024-05-08T18:05:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.800+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fb8f400a4a0 con 0x7fb8e402ea80 2024-05-08T18:05:06.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.800+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899e7c30 2024-05-08T18:05:06.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.800+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8e4037040 con 0x5609899e7c30 2024-05-08T18:05:06.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.800+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899e7c30 2024-05-08T18:05:06.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.801+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8e4037040 con 0x5609899e7c30 2024-05-08T18:05:06.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.801+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899e3030 2024-05-08T18:05:06.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.801+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.801+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899e7c30 2024-05-08T18:05:06.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.801+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8e4037040 con 0x5609899e7c30 2024-05-08T18:05:06.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.801+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899e7c30 2024-05-08T18:05:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.802+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fb8e4037040 con 0x5609899e7c30 2024-05-08T18:05:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.802+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899de8b0 2024-05-08T18:05:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.802+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609899de8b0 2024-05-08T18:05:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.802+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899de8b0 2024-05-08T18:05:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.802+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609899de8b0 2024-05-08T18:05:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.802+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899e3030 2024-05-08T18:05:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.802+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.802+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899e7c30 2024-05-08T18:05:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.803+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8e4037040 con 0x5609899e7c30 2024-05-08T18:05:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.803+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x7fb8e402ea80 2024-05-08T18:05:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.803+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8f400a4a0 con 0x7fb8e402ea80 2024-05-08T18:05:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.803+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899de8b0 2024-05-08T18:05:06.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.803+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609899de8b0 2024-05-08T18:05:06.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.803+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899e3030 2024-05-08T18:05:06.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.803+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.803+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899e7c30 2024-05-08T18:05:06.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.804+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8e4037040 con 0x5609899e7c30 2024-05-08T18:05:06.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.804+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899e3030 2024-05-08T18:05:06.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.804+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.804+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x7fb8e402ea80 2024-05-08T18:05:06.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.804+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8f400a4a0 con 0x7fb8e402ea80 2024-05-08T18:05:06.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.804+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899e7c30 2024-05-08T18:05:06.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.804+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8e4037040 con 0x5609899e7c30 2024-05-08T18:05:06.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.805+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899e3030 2024-05-08T18:05:06.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.805+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.805+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x5609899de8b0 2024-05-08T18:05:06.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.805+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609899de8b0 2024-05-08T18:05:06.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.805+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fb8ec0c9040 con 0x7fb8e402ea80 2024-05-08T18:05:06.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.805+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb8f400a4a0 con 0x7fb8e402ea80 2024-05-08T18:05:06.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.805+0000 7fb8ff813800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:06.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.805+0000 7fb8ff813800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:06.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.805+0000 7fb8ff813800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:06.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.805+0000 7fb8ff813800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:06.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.805+0000 7fb8ff813800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:06.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.805+0000 7fb8ff813800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:06.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.805+0000 7fb8ff813800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.805+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899e6530 con 0x5609899de8b0 2024-05-08T18:05:06.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609899de8b0 2024-05-08T18:05:06.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8ff813800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8ff813800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899e6530 con 0x5609899de8b0 2024-05-08T18:05:06.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fb8f400a4a0 con 0x5609899de8b0 2024-05-08T18:05:06.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8ff813800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:06.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8ff813800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899e6500 con 0x5609899e7c30 2024-05-08T18:05:06.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fb8e4037040 con 0x5609899e7c30 2024-05-08T18:05:06.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8ff813800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:06.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8ff813800 20 zone default found 2024-05-08T18:05:06.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8ff813800 4 Realm: () 2024-05-08T18:05:06.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8ff813800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:06.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8ff813800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:06.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8ff813800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:06.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8ff813800 20 zonegroup default 2024-05-08T18:05:06.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8ff813800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.806+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899e6dd0 con 0x5609899e3030 2024-05-08T18:05:06.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.807+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.807+0000 7fb8ff813800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.807+0000 7fb8ff813800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.807+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899ec620 con 0x5609899e7c30 2024-05-08T18:05:06.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.807+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fb8e4037040 con 0x5609899e7c30 2024-05-08T18:05:06.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.807+0000 7fb8ff813800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.807+0000 7fb8ff813800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.807+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899ec9c0 con 0x5609899e7c30 2024-05-08T18:05:06.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.807+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fb8e4037040 con 0x5609899e7c30 2024-05-08T18:05:06.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.807+0000 7fb8ff813800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:06.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.807+0000 7fb8ff813800 20 started sync module instance, tier type = 2024-05-08T18:05:06.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.807+0000 7fb8ff813800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:06.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.807+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899ebac0 con 0x7fb8e402ea80 2024-05-08T18:05:06.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.808+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8f400a4a0 con 0x7fb8e402ea80 2024-05-08T18:05:06.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.808+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94598963576768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899ebac0 con 0x7fb8e402ea80 2024-05-08T18:05:06.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.808+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899ed3e0 con 0x5609899de8b0 2024-05-08T18:05:06.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.809+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94598963576768] v27'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8f4040040 con 0x7fb8e402ea80 2024-05-08T18:05:06.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.809+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8f4040040 con 0x5609899de8b0 2024-05-08T18:05:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.810+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94598963580624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899ebac0 con 0x5609899de8b0 2024-05-08T18:05:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.810+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899ede70 con 0x5609899e7c30 2024-05-08T18:05:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.810+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8e4037040 con 0x5609899e7c30 2024-05-08T18:05:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.810+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94598963583296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899ee710 con 0x5609899e7c30 2024-05-08T18:05:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.810+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899ef1f0 con 0x7fb8e402ea80 2024-05-08T18:05:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.811+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94598963580624] v27'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8f4040040 con 0x5609899de8b0 2024-05-08T18:05:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.812+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94598963583296] v27'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8e4037040 con 0x5609899e7c30 2024-05-08T18:05:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.812+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8f4040040 con 0x7fb8e402ea80 2024-05-08T18:05:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.812+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94598963585632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899eb610 con 0x7fb8e402ea80 2024-05-08T18:05:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.812+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899efa90 con 0x5609899de8b0 2024-05-08T18:05:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.813+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94598963585632] v27'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8f4040040 con 0x7fb8e402ea80 2024-05-08T18:05:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.813+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8f4040040 con 0x5609899de8b0 2024-05-08T18:05:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.813+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94598963588352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899efa40 con 0x5609899de8b0 2024-05-08T18:05:06.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.813+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899f04a0 con 0x5609899e3030 2024-05-08T18:05:06.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.814+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.814+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94598963593200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899f0cf0 con 0x5609899e3030 2024-05-08T18:05:06.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.814+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899f1740 con 0x5609899e3030 2024-05-08T18:05:06.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.814+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94598963588352] v27'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8f4040040 con 0x5609899de8b0 2024-05-08T18:05:06.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.815+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94598963593200] v27'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.815+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.815+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94598963595328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899ef590 con 0x5609899e3030 2024-05-08T18:05:06.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.815+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899f1d20 con 0x5609899e3030 2024-05-08T18:05:06.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.816+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94598963595328] v27'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.816+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.816+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94598963597872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899f1d20 con 0x5609899e3030 2024-05-08T18:05:06.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.816+0000 7fb8ff813800 20 add_watcher() i=0 2024-05-08T18:05:06.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.816+0000 7fb8ff813800 20 add_watcher() i=1 2024-05-08T18:05:06.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.816+0000 7fb8ff813800 20 add_watcher() i=2 2024-05-08T18:05:06.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.816+0000 7fb8ff813800 20 add_watcher() i=3 2024-05-08T18:05:06.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.816+0000 7fb8ff813800 20 add_watcher() i=4 2024-05-08T18:05:06.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.816+0000 7fb8ff813800 20 add_watcher() i=5 2024-05-08T18:05:06.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.816+0000 7fb8ff813800 20 add_watcher() i=6 2024-05-08T18:05:06.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.817+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94598963597872] v27'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.817+0000 7fb8ff813800 20 add_watcher() i=7 2024-05-08T18:05:06.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.817+0000 7fb8ff813800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:06.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.817+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5609899f5030 con 0x5609899de8b0 2024-05-08T18:05:06.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.817+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fb8f4040040 con 0x5609899de8b0 2024-05-08T18:05:06.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.817+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94598963599840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899f22b0 con 0x5609899de8b0 2024-05-08T18:05:06.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.818+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94598963599840] v27'92 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fb8f4040040 con 0x5609899de8b0 2024-05-08T18:05:06.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.818+0000 7fb7b17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:06.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.818+0000 7fb8ff813800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:06.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.818+0000 7fb8ff813800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:06.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.818+0000 7fb8ff813800 5 note: GC not initialized 2024-05-08T18:05:06.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.819+0000 7fb72bfff640 20 reqs_thread_entry: start 2024-05-08T18:05:06.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.819+0000 7fb8ff813800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:06.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.819+0000 7fb8ff813800 20 Filter name: none 2024-05-08T18:05:06.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.819+0000 7fb729ffb640 20 reqs_thread_entry: start 2024-05-08T18:05:06.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.819+0000 7fb8ff813800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.819+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560989a280d0 con 0x5609899de8b0 2024-05-08T18:05:06.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.819+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fb8f4040040 con 0x5609899de8b0 2024-05-08T18:05:06.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.819+0000 7fb8ff813800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:06.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.819+0000 7fb8ff813800 20 rados->read ofs=0 len=0 2024-05-08T18:05:06.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.819+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560989a28ac0 con 0x5609899de8b0 2024-05-08T18:05:06.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.820+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fb8f4040040 con 0x5609899de8b0 2024-05-08T18:05:06.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.820+0000 7fb8ff813800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:06.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.820+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560989a29ad0 con 0x7fb8e402ea80 2024-05-08T18:05:06.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.820+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(64 foo.buckets [call out=183b] v0'0 uv2 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7fb8f4040040 con 0x7fb8e402ea80 2024-05-08T18:05:06.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.821+0000 7fb7b17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:06.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.821+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94598963599840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899f22b0 con 0x5609899de8b0 2024-05-08T18:05:06.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.822+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94598963599840] v27'93 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fb8f4040040 con 0x5609899de8b0 2024-05-08T18:05:06.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.822+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94598963576768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899f1e90 con 0x7fb8e402ea80 2024-05-08T18:05:06.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.823+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94598963576768] v27'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8f4040040 con 0x7fb8e402ea80 2024-05-08T18:05:06.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.823+0000 7fb8ff813800 20 remove_watcher() i=0 2024-05-08T18:05:06.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.823+0000 7fb8ff813800 2 removed watcher, disabling cache 2024-05-08T18:05:06.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.823+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94598963580624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899ebfc0 con 0x5609899de8b0 2024-05-08T18:05:06.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.824+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94598963580624] v27'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8f4040040 con 0x5609899de8b0 2024-05-08T18:05:06.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.824+0000 7fb8ff813800 20 remove_watcher() i=1 2024-05-08T18:05:06.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.824+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94598963583296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899eced0 con 0x5609899e7c30 2024-05-08T18:05:06.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.825+0000 7fb8f3fff640 1 -- 172.21.15.50:0/4110382738 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94598963583296] v27'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8e4037040 con 0x5609899e7c30 2024-05-08T18:05:06.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.825+0000 7fb8ff813800 20 remove_watcher() i=2 2024-05-08T18:05:06.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.825+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94598963585632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899ed940 con 0x7fb8e402ea80 2024-05-08T18:05:06.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.826+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94598963585632] v27'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8f4040040 con 0x7fb8e402ea80 2024-05-08T18:05:06.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.826+0000 7fb8ff813800 20 remove_watcher() i=3 2024-05-08T18:05:06.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.826+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94598963588352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899ee260 con 0x5609899de8b0 2024-05-08T18:05:06.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.827+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94598963588352] v27'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8f4040040 con 0x5609899de8b0 2024-05-08T18:05:06.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.827+0000 7fb8ff813800 20 remove_watcher() i=4 2024-05-08T18:05:06.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.827+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94598963593200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899eed00 con 0x5609899e3030 2024-05-08T18:05:06.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.828+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94598963593200] v27'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.828+0000 7fb8ff813800 20 remove_watcher() i=5 2024-05-08T18:05:06.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.828+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94598963595328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899efff0 con 0x5609899e3030 2024-05-08T18:05:06.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.829+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94598963595328] v27'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.829+0000 7fb8ff813800 20 remove_watcher() i=6 2024-05-08T18:05:06.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.829+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94598963597872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5609899efff0 con 0x5609899e3030 2024-05-08T18:05:06.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8fae1a640 1 -- 172.21.15.50:0/4110382738 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94598963597872] v27'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb8ec0bd030 con 0x5609899e3030 2024-05-08T18:05:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 20 remove_watcher() i=7 2024-05-08T18:05:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5609899e3030 msgr2=0x5609899e5420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5609899e3030 0x5609899e5420 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5609899de8b0 msgr2=0x5609899e0d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5609899de8b0 0x5609899e0d70 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fb8e402ea80 msgr2=0x7fb8e403a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fb8e402ea80 0x7fb8e403a3c0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5609899e7c30 msgr2=0x5609899ea0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5609899e7c30 0x5609899ea0f0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb7c403bab0 msgr2=0x7fb7c403df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb7c403bab0 0x7fb7c403df70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5609899aec70 tx=0x7fb8e4023040 comp rx=0 tx=0).stop 2024-05-08T18:05:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a57f0 msgr2=0x5609899d3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8fb61b640 1 -- 172.21.15.50:0/4110382738 reap_dead start 2024-05-08T18:05:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a57f0 0x5609899d3190 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec071040 tx=0x7fb8ec0915d0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 shutdown_connections 2024-05-08T18:05:06.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 --2- 172.21.15.50:0/4110382738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a57f0 0x5609899d3190 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 >> 172.21.15.50:0/4110382738 conn(0x5609899c5f20 msgr2=0x5609899c3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 shutdown_connections 2024-05-08T18:05:06.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.830+0000 7fb8ff813800 1 -- 172.21.15.50:0/4110382738 wait complete. 2024-05-08T18:05:06.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 -- 172.21.15.50:0/1798859833 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb7d403bb50 msgr2=0x7fb7d403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 --2- 172.21.15.50:0/1798859833 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb7d403bb50 0x7fb7d403e010 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x5609899a2a90 tx=0x7fb8e4006000 comp rx=0 tx=0).stop 2024-05-08T18:05:06.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 -- 172.21.15.50:0/1798859833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609899bcbe0 msgr2=0x5609899bd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 --2- 172.21.15.50:0/1798859833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609899bcbe0 0x5609899bd000 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec05b500 tx=0x7fb8ec090370 comp rx=0 tx=0).stop 2024-05-08T18:05:06.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 -- 172.21.15.50:0/1798859833 shutdown_connections 2024-05-08T18:05:06.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 --2- 172.21.15.50:0/1798859833 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb7d403bb50 0x7fb7d403e010 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 --2- 172.21.15.50:0/1798859833 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609899bcbe0 0x5609899bd000 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 -- 172.21.15.50:0/1798859833 >> 172.21.15.50:0/1798859833 conn(0x5609899b0c80 msgr2=0x5609899b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 -- 172.21.15.50:0/1798859833 shutdown_connections 2024-05-08T18:05:06.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 -- 172.21.15.50:0/1798859833 wait complete. 2024-05-08T18:05:06.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 -- 172.21.15.50:0/734663816 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5609899a7ed0 msgr2=0x5609899aa2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5609899a7ed0 0x5609899aa2c0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 -- 172.21.15.50:0/734663816 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5609898a8ab0 msgr2=0x5609899a55a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5609898a8ab0 0x5609899a55a0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 -- 172.21.15.50:0/734663816 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5609899abd80 msgr2=0x5609899ae170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5609899abd80 0x5609899ae170 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 -- 172.21.15.50:0/734663816 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb7e003bb50 msgr2=0x7fb7e003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb7e003bb50 0x7fb7e003e010 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4000c00 tx=0x7fb8e40013b0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 -- 172.21.15.50:0/734663816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 msgr2=0x56098999e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 --2- 172.21.15.50:0/734663816 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609898a6180 0x56098999e360 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0057e0 tx=0x7fb8ec002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8fb61b640 1 -- 172.21.15.50:0/734663816 reap_dead start 2024-05-08T18:05:06.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 -- 172.21.15.50:0/734663816 shutdown_connections 2024-05-08T18:05:06.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 -- 172.21.15.50:0/734663816 >> 172.21.15.50:0/734663816 conn(0x56098989fad0 msgr2=0x560989990a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 -- 172.21.15.50:0/734663816 shutdown_connections 2024-05-08T18:05:06.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.831+0000 7fb8ff813800 1 -- 172.21.15.50:0/734663816 wait complete. 2024-05-08T18:05:06.843 INFO:teuthology.orchestra.run.smithi050.stdout:["myfoo"] 2024-05-08T18:05:06.844 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2024-05-08T18:05:06.918 INFO:tasks.radosgw_admin:RL: list_buckets foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:06.919 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2024-05-08T18:05:06.919 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2024-05-08T18:05:06.919 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2024-05-08T18:05:06.963 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:06.963 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:06.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.965+0000 7fda5ff28800 1 Processor -- start 2024-05-08T18:05:06.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.966+0000 7fda5ff28800 1 -- start start 2024-05-08T18:05:06.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.966+0000 7fda5ff28800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ac7a254fa0 0x55ac7a2553c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.966+0000 7fda5ff28800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55ac7a1ca120 con 0x55ac7a255900 2024-05-08T18:05:06.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.966+0000 7fda5ff28800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55ac7a0f6e20 con 0x55ac7a254fa0 2024-05-08T18:05:06.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.966+0000 7fda5bc21640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ac7a254fa0 0x55ac7a2553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.966+0000 7fda5bc21640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ac7a254fa0 0x55ac7a2553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37382/0 (socket says 172.21.15.50:37382) 2024-05-08T18:05:06.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.966+0000 7fda5bc21640 1 -- 172.21.15.50:0/312382307 learned_addr learned my addr 172.21.15.50:0/312382307 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.966+0000 7fda5b420640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55ac7a255900 0x55ac7a255db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43100/0 (socket says 172.21.15.50:43100) 2024-05-08T18:05:06.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.966+0000 7fda5bc21640 1 -- 172.21.15.50:0/312382307 >> v1:172.21.15.46:6789/0 conn(0x55ac7a255900 legacy=0x55ac7a255db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.966+0000 7fda5bc21640 1 -- 172.21.15.50:0/312382307 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac7a2564c0 con 0x55ac7a254fa0 2024-05-08T18:05:06.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.968+0000 7fda5bc21640 1 --2- 172.21.15.50:0/312382307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 0x55ac7a2553c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00a520 tx=0x7fda4c01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46180b7910d38a79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.968+0000 7fda5ac1f640 1 -- 172.21.15.50:0/312382307 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda4c057070 con 0x55ac7a254fa0 2024-05-08T18:05:06.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.968+0000 7fda5ac1f640 1 -- 172.21.15.50:0/312382307 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4c052570 con 0x55ac7a254fa0 2024-05-08T18:05:06.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.968+0000 7fda5ac1f640 1 -- 172.21.15.50:0/312382307 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda4c04ae60 con 0x55ac7a254fa0 2024-05-08T18:05:06.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.969+0000 7fda5ff28800 1 -- 172.21.15.50:0/312382307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 msgr2=0x55ac7a2553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.969+0000 7fda5ff28800 1 --2- 172.21.15.50:0/312382307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 0x55ac7a2553c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fda4c00a520 tx=0x7fda4c01d010 comp rx=0 tx=0).stop 2024-05-08T18:05:06.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.969+0000 7fda5ff28800 1 -- 172.21.15.50:0/312382307 shutdown_connections 2024-05-08T18:05:06.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.969+0000 7fda5ff28800 1 --2- 172.21.15.50:0/312382307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 0x55ac7a2553c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.969+0000 7fda5ff28800 1 -- 172.21.15.50:0/312382307 >> 172.21.15.50:0/312382307 conn(0x55ac7a250ac0 msgr2=0x55ac7a252f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.969+0000 7fda5ff28800 1 -- 172.21.15.50:0/312382307 shutdown_connections 2024-05-08T18:05:06.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.969+0000 7fda5ff28800 1 -- 172.21.15.50:0/312382307 wait complete. 2024-05-08T18:05:06.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.984+0000 7fda5ff28800 1 Processor -- start 2024-05-08T18:05:06.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.984+0000 7fda5ff28800 1 -- start start 2024-05-08T18:05:06.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.985+0000 7fda5ff28800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 0x55ac7a349f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.985+0000 7fda5ff28800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ac7a1a1a00 con 0x55ac7a254fa0 2024-05-08T18:05:06.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.985+0000 7fda5bc21640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 0x55ac7a349f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.985+0000 7fda5bc21640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 0x55ac7a349f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37390/0 (socket says 172.21.15.50:37390) 2024-05-08T18:05:06.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.985+0000 7fda5bc21640 1 -- 172.21.15.50:0/4141024686 learned_addr learned my addr 172.21.15.50:0/4141024686 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.985+0000 7fda5bc21640 1 -- 172.21.15.50:0/4141024686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac7a34a540 con 0x55ac7a254fa0 2024-05-08T18:05:06.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.985+0000 7fda5bc21640 1 --2- 172.21.15.50:0/4141024686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 0x55ac7a349f70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fda4c055040 tx=0x7fda4c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cab9899f8b44d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.986+0000 7fd949e0b640 1 -- 172.21.15.50:0/4141024686 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda4c057040 con 0x55ac7a254fa0 2024-05-08T18:05:06.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.986+0000 7fd949e0b640 1 -- 172.21.15.50:0/4141024686 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4c052720 con 0x55ac7a254fa0 2024-05-08T18:05:06.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.986+0000 7fd949e0b640 1 -- 172.21.15.50:0/4141024686 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda4c052a40 con 0x55ac7a254fa0 2024-05-08T18:05:06.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.986+0000 7fda5ff28800 1 -- 172.21.15.50:0/4141024686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 msgr2=0x55ac7a349f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:06.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.986+0000 7fda5ff28800 1 --2- 172.21.15.50:0/4141024686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 0x55ac7a349f70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fda4c055040 tx=0x7fda4c04c000 comp rx=0 tx=0).stop 2024-05-08T18:05:06.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.986+0000 7fda5ff28800 1 -- 172.21.15.50:0/4141024686 shutdown_connections 2024-05-08T18:05:06.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.986+0000 7fda5ff28800 1 --2- 172.21.15.50:0/4141024686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 0x55ac7a349f70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:06.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.986+0000 7fda5ff28800 1 -- 172.21.15.50:0/4141024686 >> 172.21.15.50:0/4141024686 conn(0x55ac7a250ac0 msgr2=0x55ac7a345890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:06.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.986+0000 7fda5ff28800 1 -- 172.21.15.50:0/4141024686 shutdown_connections 2024-05-08T18:05:06.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.986+0000 7fda5ff28800 1 -- 172.21.15.50:0/4141024686 wait complete. 2024-05-08T18:05:06.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.987+0000 7fda5ff28800 1 Processor -- start 2024-05-08T18:05:06.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.987+0000 7fda5ff28800 1 -- start start 2024-05-08T18:05:06.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.987+0000 7fda5ff28800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 0x55ac7a351a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.987+0000 7fda5ff28800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ac7a2569f0 con 0x55ac7a254fa0 2024-05-08T18:05:06.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.987+0000 7fda5bc21640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 0x55ac7a351a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.987+0000 7fda5bc21640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 0x55ac7a351a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37406/0 (socket says 172.21.15.50:37406) 2024-05-08T18:05:06.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.987+0000 7fda5bc21640 1 -- 172.21.15.50:0/3851252184 learned_addr learned my addr 172.21.15.50:0/3851252184 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:06.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.988+0000 7fda5bc21640 1 -- 172.21.15.50:0/3851252184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac7a351fc0 con 0x55ac7a254fa0 2024-05-08T18:05:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.988+0000 7fda5bc21640 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 0x55ac7a351a80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fda4c052540 tx=0x7fda4c003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.988+0000 7fd943fff640 1 -- 172.21.15.50:0/3851252184 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda4c073070 con 0x55ac7a254fa0 2024-05-08T18:05:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.988+0000 7fd943fff640 1 -- 172.21.15.50:0/3851252184 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4c05d440 con 0x55ac7a254fa0 2024-05-08T18:05:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.988+0000 7fda5ff28800 1 -- 172.21.15.50:0/3851252184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ac7a352250 con 0x55ac7a254fa0 2024-05-08T18:05:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.988+0000 7fd943fff640 1 -- 172.21.15.50:0/3851252184 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda4c057070 con 0x55ac7a254fa0 2024-05-08T18:05:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.988+0000 7fda5ff28800 1 -- 172.21.15.50:0/3851252184 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac7a354eb0 con 0x55ac7a254fa0 2024-05-08T18:05:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.989+0000 7fd943fff640 1 -- 172.21.15.50:0/3851252184 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fda4c076070 con 0x55ac7a254fa0 2024-05-08T18:05:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.989+0000 7fd943fff640 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd94403bba0 0x7fd94403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.989+0000 7fd943fff640 1 -- 172.21.15.50:0/3851252184 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fda4c093ff0 con 0x55ac7a254fa0 2024-05-08T18:05:06.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.989+0000 7fda5b420640 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd94403bba0 0x7fd94403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.989+0000 7fda5ff28800 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ac7a255900 0x55ac7a358c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.989+0000 7fda5ff28800 1 -- 172.21.15.50:0/3851252184 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3591b0 con 0x55ac7a255900 2024-05-08T18:05:06.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.989+0000 7fda5c422640 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ac7a255900 0x55ac7a358c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.990+0000 7fda5b420640 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd94403bba0 0x7fd94403e060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda44001e30 tx=0x7fda44001390 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.990+0000 7fda5c422640 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ac7a255900 0x55ac7a358c70 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.990+0000 7fda5c422640 1 -- 172.21.15.50:0/3851252184 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a255900 2024-05-08T18:05:06.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.990+0000 7fda5ff28800 1 -- 172.21.15.50:0/3851252184 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a356240 con 0x55ac7a255900 2024-05-08T18:05:06.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.990+0000 7fda5c422640 1 -- 172.21.15.50:0/3851252184 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a255900 2024-05-08T18:05:06.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.990+0000 7fda5ff28800 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ac7a35b4a0 0x55ac7a35d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.990+0000 7fda5ff28800 1 -- 172.21.15.50:0/3851252184 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a35df60 con 0x55ac7a35b4a0 2024-05-08T18:05:06.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.991+0000 7fda5bc21640 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ac7a35b4a0 0x55ac7a35d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.991+0000 7fda5bc21640 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ac7a35b4a0 0x55ac7a35d890 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.991+0000 7fda5bc21640 1 -- 172.21.15.50:0/3851252184 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fda4c075030 con 0x55ac7a35b4a0 2024-05-08T18:05:06.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.991+0000 7fda5ff28800 1 -- 172.21.15.50:0/3851252184 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a35a340 con 0x55ac7a255900 2024-05-08T18:05:06.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.991+0000 7fda5c422640 1 -- 172.21.15.50:0/3851252184 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a255900 2024-05-08T18:05:06.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.991+0000 7fda5ff28800 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ac7a35f3a0 0x55ac7a361790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:06.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.991+0000 7fda5ff28800 1 -- 172.21.15.50:0/3851252184 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a361f00 con 0x55ac7a35f3a0 2024-05-08T18:05:06.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.992+0000 7fda5b420640 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ac7a35f3a0 0x55ac7a361790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:06.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.992+0000 7fda5b420640 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ac7a35f3a0 0x55ac7a361790 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:06.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.992+0000 7fda5b420640 1 -- 172.21.15.50:0/3851252184 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fda4400bd50 con 0x55ac7a35f3a0 2024-05-08T18:05:07.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.992+0000 7fda5ff28800 1 Processor -- start 2024-05-08T18:05:07.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.993+0000 7fda5ff28800 1 -- start start 2024-05-08T18:05:07.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.993+0000 7fda5ff28800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a370230 0x55ac7a370650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.993+0000 7fda5ff28800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ac7a148330 con 0x55ac7a370230 2024-05-08T18:05:07.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.993+0000 7fda5bc21640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a370230 0x55ac7a370650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.993+0000 7fda5bc21640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a370230 0x55ac7a370650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37410/0 (socket says 172.21.15.50:37410) 2024-05-08T18:05:07.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.993+0000 7fda5bc21640 1 -- 172.21.15.50:0/597819776 learned_addr learned my addr 172.21.15.50:0/597819776 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.993+0000 7fda5bc21640 1 -- 172.21.15.50:0/597819776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac7a370b90 con 0x55ac7a370230 2024-05-08T18:05:07.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.993+0000 7fda5bc21640 1 --2- 172.21.15.50:0/597819776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a370230 0x55ac7a370650 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fda4c092350 tx=0x7fda4c06e6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.993+0000 7fd9417fa640 1 -- 172.21.15.50:0/597819776 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda4c066b70 con 0x55ac7a370230 2024-05-08T18:05:07.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.993+0000 7fd9417fa640 1 -- 172.21.15.50:0/597819776 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4c066d10 con 0x55ac7a370230 2024-05-08T18:05:07.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.993+0000 7fda5ff28800 1 -- 172.21.15.50:0/597819776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ac7a371600 con 0x55ac7a370230 2024-05-08T18:05:07.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.993+0000 7fd9417fa640 1 -- 172.21.15.50:0/597819776 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda4c0774e0 con 0x55ac7a370230 2024-05-08T18:05:07.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.994+0000 7fda5ff28800 1 -- 172.21.15.50:0/597819776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac7a371b00 con 0x55ac7a370230 2024-05-08T18:05:07.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.994+0000 7fd9417fa640 1 -- 172.21.15.50:0/597819776 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fda4c095070 con 0x55ac7a370230 2024-05-08T18:05:07.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.994+0000 7fd9417fa640 1 --2- 172.21.15.50:0/597819776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd93403bb50 0x7fd93403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.994+0000 7fd9417fa640 1 -- 172.21.15.50:0/597819776 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fda4c08a070 con 0x55ac7a370230 2024-05-08T18:05:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.995+0000 7fda5b420640 1 --2- 172.21.15.50:0/597819776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd93403bb50 0x7fd93403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.995+0000 7fda5ff28800 1 Processor -- start 2024-05-08T18:05:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.995+0000 7fda5ff28800 1 -- start start 2024-05-08T18:05:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.995+0000 7fda5b420640 1 --2- 172.21.15.50:0/597819776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd93403bb50 0x7fd93403e010 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55ac7a362300 tx=0x7fda4400f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.995+0000 7fda5ff28800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a37d370 0x55ac7a37d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.995+0000 7fda5ff28800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ac7a34a800 con 0x55ac7a37d370 2024-05-08T18:05:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.995+0000 7fda5bc21640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a37d370 0x55ac7a37d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.995+0000 7fda5bc21640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a37d370 0x55ac7a37d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37416/0 (socket says 172.21.15.50:37416) 2024-05-08T18:05:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.995+0000 7fda5bc21640 1 -- 172.21.15.50:0/4134787912 learned_addr learned my addr 172.21.15.50:0/4134787912 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.996+0000 7fda5bc21640 1 -- 172.21.15.50:0/4134787912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac7a37dd60 con 0x55ac7a37d370 2024-05-08T18:05:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.996+0000 7fda5bc21640 1 --2- 172.21.15.50:0/4134787912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a37d370 0x55ac7a37d790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fda4c07f010 tx=0x7fda4c08d400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61342e13cce25c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.996+0000 7fd93affd640 1 -- 172.21.15.50:0/4134787912 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda4c079e80 con 0x55ac7a37d370 2024-05-08T18:05:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.996+0000 7fd93affd640 1 -- 172.21.15.50:0/4134787912 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4c0a9070 con 0x55ac7a37d370 2024-05-08T18:05:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.996+0000 7fd93affd640 1 -- 172.21.15.50:0/4134787912 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda4c05ca30 con 0x55ac7a37d370 2024-05-08T18:05:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.996+0000 7fda5ff28800 1 -- 172.21.15.50:0/4134787912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a37d370 msgr2=0x55ac7a37d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.996+0000 7fda5ff28800 1 --2- 172.21.15.50:0/4134787912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a37d370 0x55ac7a37d790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fda4c07f010 tx=0x7fda4c08d400 comp rx=0 tx=0).stop 2024-05-08T18:05:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.996+0000 7fda5ff28800 1 -- 172.21.15.50:0/4134787912 shutdown_connections 2024-05-08T18:05:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.996+0000 7fda5ff28800 1 --2- 172.21.15.50:0/4134787912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a37d370 0x55ac7a37d790 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.996+0000 7fda5ff28800 1 -- 172.21.15.50:0/4134787912 >> 172.21.15.50:0/4134787912 conn(0x55ac7a379640 msgr2=0x55ac7a3776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.996+0000 7fda5ff28800 1 -- 172.21.15.50:0/4134787912 shutdown_connections 2024-05-08T18:05:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.996+0000 7fda5ff28800 1 -- 172.21.15.50:0/4134787912 wait complete. 2024-05-08T18:05:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.997+0000 7fda5ff28800 1 Processor -- start 2024-05-08T18:05:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.997+0000 7fda5ff28800 1 -- start start 2024-05-08T18:05:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.997+0000 7fda5ff28800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a37d370 0x55ac7a3846b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.997+0000 7fda5ff28800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ac7a37e3c0 con 0x55ac7a37d370 2024-05-08T18:05:07.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.997+0000 7fda5bc21640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a37d370 0x55ac7a3846b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.997+0000 7fda5bc21640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a37d370 0x55ac7a3846b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37424/0 (socket says 172.21.15.50:37424) 2024-05-08T18:05:07.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.997+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 learned_addr learned my addr 172.21.15.50:0/880567344 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.997+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac7a384bf0 con 0x55ac7a37d370 2024-05-08T18:05:07.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.997+0000 7fda5bc21640 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a37d370 0x55ac7a3846b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fda4c083990 tx=0x7fda4c097e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.998+0000 7fd9397fa640 1 -- 172.21.15.50:0/880567344 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda4c0bc070 con 0x55ac7a37d370 2024-05-08T18:05:07.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.998+0000 7fd9397fa640 1 -- 172.21.15.50:0/880567344 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4c05c900 con 0x55ac7a37d370 2024-05-08T18:05:07.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.998+0000 7fd9397fa640 1 -- 172.21.15.50:0/880567344 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fda4c0870c0 con 0x55ac7a37d370 2024-05-08T18:05:07.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.998+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ac7a384bf0 con 0x55ac7a37d370 2024-05-08T18:05:07.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.998+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac7a384bf0 con 0x55ac7a37d370 2024-05-08T18:05:07.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.998+0000 7fd9397fa640 1 -- 172.21.15.50:0/880567344 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fda4c0bf070 con 0x55ac7a37d370 2024-05-08T18:05:07.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.999+0000 7fd9397fa640 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd92403bb50 0x7fd92403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.999+0000 7fd9397fa640 1 -- 172.21.15.50:0/880567344 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fda4c083040 con 0x55ac7a37d370 2024-05-08T18:05:07.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.999+0000 7fda5b420640 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd92403bb50 0x7fd92403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.999+0000 7fda5ff28800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.999+0000 7fda5ff28800 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ac7a38fc80 0x55ac7a392140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.999+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a392890 con 0x55ac7a38fc80 2024-05-08T18:05:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.999+0000 7fda5c422640 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ac7a38fc80 0x55ac7a392140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:06.999+0000 7fda5b420640 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd92403bb50 0x7fd92403e010 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fda4400ffd0 tx=0x7fda44010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.000+0000 7fda5c422640 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ac7a38fc80 0x55ac7a392140 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.000+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a38fc80 2024-05-08T18:05:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.000+0000 7fda5ff28800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.000+0000 7fda5ff28800 20 realm 2024-05-08T18:05:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.000+0000 7fda5ff28800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.000+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a392ca0 con 0x55ac7a38fc80 2024-05-08T18:05:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.000+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a38fc80 2024-05-08T18:05:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.000+0000 7fda5ff28800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.000+0000 7fda5ff28800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.000+0000 7fda5ff28800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.000+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393570 con 0x55ac7a38fc80 2024-05-08T18:05:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.000+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a38fc80 2024-05-08T18:05:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.000+0000 7fda5ff28800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.000+0000 7fda5ff28800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.000+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a38fc80 2024-05-08T18:05:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.001+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a38fc80 2024-05-08T18:05:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.001+0000 7fda5ff28800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.001+0000 7fda5ff28800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.001+0000 7fda5ff28800 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ac7a394400 0x55ac7a3967f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.001+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a396f30 con 0x55ac7a394400 2024-05-08T18:05:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.001+0000 7fda5bc21640 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ac7a394400 0x55ac7a3967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.001+0000 7fda5bc21640 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ac7a394400 0x55ac7a3967f0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.001+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.001+0000 7fda5ff28800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:07.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.002+0000 7fda5ff28800 20 searching for the correct realm 2024-05-08T18:05:07.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.002+0000 7fda5ff28800 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ac7a398fd0 0x55ac7a39b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.002+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a39bb60 con 0x55ac7a398fd0 2024-05-08T18:05:07.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.002+0000 7fda5b420640 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ac7a398fd0 0x55ac7a39b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.002+0000 7fda5b420640 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ac7a398fd0 0x55ac7a39b490 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.002+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.002+0000 7fda5b420640 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fda4400f070 0x7fda4403a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.002+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a39bb60 con 0x7fda4400f070 2024-05-08T18:05:07.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.002+0000 7fda5c422640 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fda4400f070 0x7fda4403a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.003+0000 7fda5c422640 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fda4400f070 0x7fda4403a3c0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.003+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda5400a4a0 con 0x7fda4400f070 2024-05-08T18:05:07.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.003+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a394400 2024-05-08T18:05:07.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.003+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.003+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a38fc80 2024-05-08T18:05:07.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.004+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a38fc80 2024-05-08T18:05:07.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.004+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a38fc80 2024-05-08T18:05:07.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.004+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a38fc80 2024-05-08T18:05:07.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.004+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a394400 2024-05-08T18:05:07.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.004+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.004+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a394400 2024-05-08T18:05:07.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.004+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.004+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x7fda4400f070 2024-05-08T18:05:07.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.005+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda5400a4a0 con 0x7fda4400f070 2024-05-08T18:05:07.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.005+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a398fd0 2024-05-08T18:05:07.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.005+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.005+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a38fc80 2024-05-08T18:05:07.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.005+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a38fc80 2024-05-08T18:05:07.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.005+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a394400 2024-05-08T18:05:07.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.005+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.005+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a38fc80 2024-05-08T18:05:07.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.006+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a38fc80 2024-05-08T18:05:07.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.006+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x7fda4400f070 2024-05-08T18:05:07.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.006+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fda5400a4a0 con 0x7fda4400f070 2024-05-08T18:05:07.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.006+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a398fd0 2024-05-08T18:05:07.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.006+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.006+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a398fd0 2024-05-08T18:05:07.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.006+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.006+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a394400 2024-05-08T18:05:07.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.007+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.007+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a398fd0 2024-05-08T18:05:07.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.007+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.007+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a398fd0 2024-05-08T18:05:07.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.007+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.007+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a38fc80 2024-05-08T18:05:07.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.007+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a38fc80 2024-05-08T18:05:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.007+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a38fc80 2024-05-08T18:05:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.007+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a38fc80 2024-05-08T18:05:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.007+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a394400 2024-05-08T18:05:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.008+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.008+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a398fd0 2024-05-08T18:05:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.008+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.008+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x7fda4400f070 2024-05-08T18:05:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.008+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda5400a4a0 con 0x7fda4400f070 2024-05-08T18:05:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.008+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a38fc80 2024-05-08T18:05:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.008+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a38fc80 2024-05-08T18:05:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.009+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a394400 2024-05-08T18:05:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.009+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.009+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a398fd0 2024-05-08T18:05:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.009+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.009+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a394400 2024-05-08T18:05:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.009+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.009+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x7fda4400f070 2024-05-08T18:05:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.010+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda5400a4a0 con 0x7fda4400f070 2024-05-08T18:05:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.010+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a398fd0 2024-05-08T18:05:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.010+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.010+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a394400 2024-05-08T18:05:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.010+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.010+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a38fc80 2024-05-08T18:05:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.010+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a38fc80 2024-05-08T18:05:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.010+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x7fda4400f070 2024-05-08T18:05:07.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda5400a4a0 con 0x7fda4400f070 2024-05-08T18:05:07.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:07.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:07.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:07.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:07.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:07.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:07.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3978d0 con 0x55ac7a38fc80 2024-05-08T18:05:07.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a38fc80 2024-05-08T18:05:07.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3978d0 con 0x55ac7a38fc80 2024-05-08T18:05:07.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fda5400a4a0 con 0x55ac7a38fc80 2024-05-08T18:05:07.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3978d0 con 0x55ac7a398fd0 2024-05-08T18:05:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 20 zone default found 2024-05-08T18:05:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 4 Realm: () 2024-05-08T18:05:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 20 zonegroup default 2024-05-08T18:05:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.011+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a398170 con 0x55ac7a394400 2024-05-08T18:05:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.012+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.012+0000 7fda5ff28800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.012+0000 7fda5ff28800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.012+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a39d9c0 con 0x55ac7a398fd0 2024-05-08T18:05:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.012+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.012+0000 7fda5ff28800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.012+0000 7fda5ff28800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.012+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a39dd60 con 0x55ac7a398fd0 2024-05-08T18:05:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.012+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.012+0000 7fda5ff28800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.012+0000 7fda5ff28800 20 started sync module instance, tier type = 2024-05-08T18:05:07.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.012+0000 7fda5ff28800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.012+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a39ce60 con 0x7fda4400f070 2024-05-08T18:05:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.013+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda5400a4a0 con 0x7fda4400f070 2024-05-08T18:05:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.013+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94199273345888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a39ce60 con 0x7fda4400f070 2024-05-08T18:05:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.013+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a39e780 con 0x55ac7a38fc80 2024-05-08T18:05:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.014+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94199273345888] v27'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda54040040 con 0x7fda4400f070 2024-05-08T18:05:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.015+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.015+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94199273349744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a39ce60 con 0x55ac7a38fc80 2024-05-08T18:05:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.015+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a39f210 con 0x55ac7a398fd0 2024-05-08T18:05:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.016+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94199273349744] v27'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.016+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.016+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94199273343408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a39cdd0 con 0x55ac7a398fd0 2024-05-08T18:05:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.016+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a00d0 con 0x7fda4400f070 2024-05-08T18:05:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.017+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94199273343408] v27'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.017+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda54040040 con 0x7fda4400f070 2024-05-08T18:05:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.017+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94199273356320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a0040 con 0x7fda4400f070 2024-05-08T18:05:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.017+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a0e60 con 0x55ac7a38fc80 2024-05-08T18:05:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.018+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.018+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94199273359728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a0e60 con 0x55ac7a38fc80 2024-05-08T18:05:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.018+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a1e00 con 0x55ac7a394400 2024-05-08T18:05:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.018+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94199273356320] v27'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda54040040 con 0x7fda4400f070 2024-05-08T18:05:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.019+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94199273359728] v27'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.019+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.019+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94199273363664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a1250 con 0x55ac7a394400 2024-05-08T18:05:07.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.019+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a26a0 con 0x55ac7a394400 2024-05-08T18:05:07.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.020+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94199273363664] v27'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.020+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.020+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94199273358528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a394400 2024-05-08T18:05:07.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.020+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a37a0 con 0x55ac7a394400 2024-05-08T18:05:07.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.021+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94199273358528] v27'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.021+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.021+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94199273370256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a3a20 con 0x55ac7a394400 2024-05-08T18:05:07.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.021+0000 7fda5ff28800 20 add_watcher() i=0 2024-05-08T18:05:07.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.021+0000 7fda5ff28800 20 add_watcher() i=1 2024-05-08T18:05:07.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.021+0000 7fda5ff28800 20 add_watcher() i=2 2024-05-08T18:05:07.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.021+0000 7fda5ff28800 20 add_watcher() i=3 2024-05-08T18:05:07.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.021+0000 7fda5ff28800 20 add_watcher() i=4 2024-05-08T18:05:07.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.021+0000 7fda5ff28800 20 add_watcher() i=5 2024-05-08T18:05:07.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.021+0000 7fda5ff28800 20 add_watcher() i=6 2024-05-08T18:05:07.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.022+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94199273370256] v27'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.022+0000 7fda5ff28800 20 add_watcher() i=7 2024-05-08T18:05:07.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.022+0000 7fda5ff28800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:07.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.022+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a6f90 con 0x55ac7a38fc80 2024-05-08T18:05:07.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.023+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.023+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94199273368128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a4480 con 0x55ac7a38fc80 2024-05-08T18:05:07.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.024+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94199273368128] v27'94 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.024+0000 7fd911fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:07.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.024+0000 7fda5ff28800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:07.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.024+0000 7fda5ff28800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:07.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.024+0000 7fda5ff28800 5 note: GC not initialized 2024-05-08T18:05:07.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.024+0000 7fd910fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:07.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.024+0000 7fda5ff28800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:07.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.024+0000 7fda5ff28800 20 Filter name: none 2024-05-08T18:05:07.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.024+0000 7fd88a7fc640 20 reqs_thread_entry: start 2024-05-08T18:05:07.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.024+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:62 8.0 8:00000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3d9c60 con 0x55ac7a398fd0 2024-05-08T18:05:07.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.025+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(62 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.025+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:63 8.10 8:08000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3d9c60 con 0x55ac7a398fd0 2024-05-08T18:05:07.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.025+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(63 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.025+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 8.8 8:10000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3d9c60 con 0x55ac7a38fc80 2024-05-08T18:05:07.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.025+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(64 [pgnls start_epoch 27 out=49b] v27'2 uv2 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.025+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 8.18 8:18000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a38fc80 2024-05-08T18:05:07.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.025+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 [pgnls start_epoch 27 out=49b] v27'11 uv11 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.025+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 8.4 8:20000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x7fda4400f070 2024-05-08T18:05:07.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.026+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda54040040 con 0x7fda4400f070 2024-05-08T18:05:07.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.026+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 8.14 8:28000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a398fd0 2024-05-08T18:05:07.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.026+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(67 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.026+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 8.c 8:30000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a38fc80 2024-05-08T18:05:07.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.026+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.026+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 8.1c 8:38000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a398fd0 2024-05-08T18:05:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.026+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(69 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.026+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 8.2 8:40000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x7fda4400f070 2024-05-08T18:05:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.027+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 [pgnls start_epoch 27 out=49b] v27'2 uv2 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda54040040 con 0x7fda4400f070 2024-05-08T18:05:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.027+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 8.12 8:48000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a398fd0 2024-05-08T18:05:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.027+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(71 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.027+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 8.a 8:50000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a38fc80 2024-05-08T18:05:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.027+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(72 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.027+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 8.1a 8:58000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a394400 2024-05-08T18:05:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.027+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(73 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.027+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 8.6 8:60000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a394400 2024-05-08T18:05:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.028+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(74 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.028+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:75 8.16 8:68000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a398fd0 2024-05-08T18:05:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.028+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(75 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.028+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 8.e 8:70000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a394400 2024-05-08T18:05:07.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.028+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(76 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.028+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 8.1e 8:78000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a394400 2024-05-08T18:05:07.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.029+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(77 [pgnls start_epoch 27 out=127b] v27'1 uv1 ondisk = 1) ==== 144+0+127 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.029+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 8.1 8:80000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a394400 2024-05-08T18:05:07.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.029+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(78 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.029+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 8.11 8:88000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a398fd0 2024-05-08T18:05:07.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.029+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(79 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.029+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:80 8.9 8:90000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x7fda4400f070 2024-05-08T18:05:07.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.029+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(80 [pgnls start_epoch 27 out=70b] v27'2 uv2 ondisk = 1) ==== 144+0+70 (crc 0 0 0) 0x7fda54040040 con 0x7fda4400f070 2024-05-08T18:05:07.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.029+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 8.19 8:98000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a38fc80 2024-05-08T18:05:07.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.030+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(81 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.030+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:82 8.5 8:a0000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a38fc80 2024-05-08T18:05:07.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.030+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(82 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.030+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:83 8.15 8:a8000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a398fd0 2024-05-08T18:05:07.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.030+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(83 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.030+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:84 8.d 8:b0000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x7fda4400f070 2024-05-08T18:05:07.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.030+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(84 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda54040040 con 0x7fda4400f070 2024-05-08T18:05:07.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.030+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:85 8.1d 8:b8000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a394400 2024-05-08T18:05:07.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.031+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(85 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.031+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:86 8.3 8:c0000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a38fc80 2024-05-08T18:05:07.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.031+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(86 [pgnls start_epoch 27 out=49b] v27'1 uv1 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.031+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:87 8.13 8:c8000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a394400 2024-05-08T18:05:07.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.031+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(87 [pgnls start_epoch 27 out=49b] v27'2 uv2 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.031+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:88 8.b 8:d0000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a398fd0 2024-05-08T18:05:07.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.031+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(88 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.031+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:89 8.1b 8:d8000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x7fda4400f070 2024-05-08T18:05:07.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.032+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(89 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda54040040 con 0x7fda4400f070 2024-05-08T18:05:07.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.032+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 8.7 8:e0000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a394400 2024-05-08T18:05:07.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.032+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(90 [pgnls start_epoch 27 out=49b] v27'1 uv1 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.032+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:91 8.17 8:e8000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a38fc80 2024-05-08T18:05:07.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.032+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(91 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.032+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:92 8.f 8:f0000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x55ac7a398fd0 2024-05-08T18:05:07.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.032+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(92 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.032+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 8.1f 8:f8000000:root:::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a3a2ef0 con 0x7fda4400f070 2024-05-08T18:05:07.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.033+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(93 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fda54040040 con 0x7fda4400f070 2024-05-08T18:05:07.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.033+0000 7fda5ff28800 20 RGWRados::pool_iterate: got .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 2024-05-08T18:05:07.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.033+0000 7fda5ff28800 20 RGWRados::pool_iterate: got myfoo 2024-05-08T18:05:07.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.033+0000 7fd911fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.033+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94199273368128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a38fc80 2024-05-08T18:05:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.034+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94199273368128] v27'95 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.034+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94199273345888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x7fda4400f070 2024-05-08T18:05:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.035+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94199273345888] v27'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda54040040 con 0x7fda4400f070 2024-05-08T18:05:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.035+0000 7fda5ff28800 20 remove_watcher() i=0 2024-05-08T18:05:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.035+0000 7fda5ff28800 2 removed watcher, disabling cache 2024-05-08T18:05:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.035+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94199273349744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a38fc80 2024-05-08T18:05:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.037+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94199273349744] v27'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.037+0000 7fda5ff28800 20 remove_watcher() i=1 2024-05-08T18:05:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.037+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94199273343408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a393910 con 0x55ac7a398fd0 2024-05-08T18:05:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.038+0000 7fda5b420640 1 -- 172.21.15.50:0/880567344 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94199273343408] v27'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda44037040 con 0x55ac7a398fd0 2024-05-08T18:05:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.038+0000 7fda5ff28800 20 remove_watcher() i=2 2024-05-08T18:05:07.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.038+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94199273356320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a39e270 con 0x7fda4400f070 2024-05-08T18:05:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.039+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94199273356320] v27'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda54040040 con 0x7fda4400f070 2024-05-08T18:05:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.039+0000 7fda5ff28800 20 remove_watcher() i=3 2024-05-08T18:05:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.039+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94199273359728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a39c9b0 con 0x55ac7a38fc80 2024-05-08T18:05:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.040+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94199273359728] v27'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda54040040 con 0x55ac7a38fc80 2024-05-08T18:05:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.040+0000 7fda5ff28800 20 remove_watcher() i=4 2024-05-08T18:05:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.040+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94199273363664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a39fc20 con 0x55ac7a394400 2024-05-08T18:05:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.041+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94199273363664] v27'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.041+0000 7fda5ff28800 20 remove_watcher() i=5 2024-05-08T18:05:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.041+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94199273358528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a39e270 con 0x55ac7a394400 2024-05-08T18:05:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.042+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94199273358528] v27'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.042+0000 7fda5ff28800 20 remove_watcher() i=6 2024-05-08T18:05:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.042+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94199273370256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ac7a39c9b0 con 0x55ac7a394400 2024-05-08T18:05:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5bc21640 1 -- 172.21.15.50:0/880567344 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94199273370256] v27'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fda4c0be030 con 0x55ac7a394400 2024-05-08T18:05:07.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5ff28800 20 remove_watcher() i=7 2024-05-08T18:05:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ac7a394400 msgr2=0x55ac7a3967f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5ff28800 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ac7a394400 0x55ac7a3967f0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ac7a38fc80 msgr2=0x55ac7a392140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5ff28800 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ac7a38fc80 0x55ac7a392140 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fda4400f070 msgr2=0x7fda4403a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5ff28800 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fda4400f070 0x7fda4403a3c0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ac7a398fd0 msgr2=0x55ac7a39b490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5ff28800 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ac7a398fd0 0x55ac7a39b490 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd92403bb50 msgr2=0x7fd92403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5ff28800 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd92403bb50 0x7fd92403e010 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fda4400ffd0 tx=0x7fda44010000 comp rx=0 tx=0).stop 2024-05-08T18:05:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5c422640 1 -- 172.21.15.50:0/880567344 reap_dead start 2024-05-08T18:05:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a37d370 msgr2=0x55ac7a3846b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5ff28800 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a37d370 0x55ac7a3846b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fda4c083990 tx=0x7fda4c097e80 comp rx=0 tx=0).stop 2024-05-08T18:05:07.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 shutdown_connections 2024-05-08T18:05:07.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5ff28800 1 --2- 172.21.15.50:0/880567344 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a37d370 0x55ac7a3846b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.043+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 >> 172.21.15.50:0/880567344 conn(0x55ac7a379640 msgr2=0x55ac7a376ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 shutdown_connections 2024-05-08T18:05:07.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 -- 172.21.15.50:0/880567344 wait complete. 2024-05-08T18:05:07.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 -- 172.21.15.50:0/597819776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd93403bb50 msgr2=0x7fd93403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 --2- 172.21.15.50:0/597819776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd93403bb50 0x7fd93403e010 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x55ac7a362300 tx=0x7fda4400f000 comp rx=0 tx=0).stop 2024-05-08T18:05:07.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 -- 172.21.15.50:0/597819776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a370230 msgr2=0x55ac7a370650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 --2- 172.21.15.50:0/597819776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a370230 0x55ac7a370650 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fda4c092350 tx=0x7fda4c06e6f0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 -- 172.21.15.50:0/597819776 shutdown_connections 2024-05-08T18:05:07.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 --2- 172.21.15.50:0/597819776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd93403bb50 0x7fd93403e010 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 --2- 172.21.15.50:0/597819776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a370230 0x55ac7a370650 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 -- 172.21.15.50:0/597819776 >> 172.21.15.50:0/597819776 conn(0x55ac7a3642d0 msgr2=0x55ac7a3646d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 -- 172.21.15.50:0/597819776 shutdown_connections 2024-05-08T18:05:07.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 -- 172.21.15.50:0/597819776 wait complete. 2024-05-08T18:05:07.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 -- 172.21.15.50:0/3851252184 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ac7a35b4a0 msgr2=0x55ac7a35d890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ac7a35b4a0 0x55ac7a35d890 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 -- 172.21.15.50:0/3851252184 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ac7a255900 msgr2=0x55ac7a358c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ac7a255900 0x55ac7a358c70 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 -- 172.21.15.50:0/3851252184 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ac7a35f3a0 msgr2=0x55ac7a361790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ac7a35f3a0 0x55ac7a361790 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 -- 172.21.15.50:0/3851252184 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd94403bba0 msgr2=0x7fd94403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd94403bba0 0x7fd94403e060 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fda44001e30 tx=0x7fda44001390 comp rx=0 tx=0).stop 2024-05-08T18:05:07.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 -- 172.21.15.50:0/3851252184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 msgr2=0x55ac7a351a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5ff28800 1 --2- 172.21.15.50:0/3851252184 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac7a254fa0 0x55ac7a351a80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fda4c052540 tx=0x7fda4c003020 comp rx=0 tx=0).stop 2024-05-08T18:05:07.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.044+0000 7fda5c422640 1 -- 172.21.15.50:0/3851252184 reap_dead start 2024-05-08T18:05:07.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.045+0000 7fda5ff28800 1 -- 172.21.15.50:0/3851252184 shutdown_connections 2024-05-08T18:05:07.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.045+0000 7fda5ff28800 1 -- 172.21.15.50:0/3851252184 >> 172.21.15.50:0/3851252184 conn(0x55ac7a250ac0 msgr2=0x55ac7a344090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.045+0000 7fda5ff28800 1 -- 172.21.15.50:0/3851252184 shutdown_connections 2024-05-08T18:05:07.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.045+0000 7fda5ff28800 1 -- 172.21.15.50:0/3851252184 wait complete. 2024-05-08T18:05:07.057 INFO:teuthology.orchestra.run.smithi050.stdout:["myfoo"] 2024-05-08T18:05:07.057 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2024-05-08T18:05:07.067 INFO:tasks.radosgw_admin:RL: create_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:07.076 INFO:tasks.radosgw_admin:RL: create_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:07.086 INFO:tasks.radosgw_admin:RL: create_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:07.135 INFO:tasks.radosgw_admin:RL: create_bucket myfoo5 foo bytes_out=0 bytes_in=237 failed=True 2024-05-08T18:05:07.162 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:07.189 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:07.215 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:07.215 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-05-08T18:05:07.215 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-05-08T18:05:07.215 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-05-08T18:05:07.261 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:07.261 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:07.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.264+0000 7fd7a0213800 1 Processor -- start 2024-05-08T18:05:07.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.264+0000 7fd7a0213800 1 -- start start 2024-05-08T18:05:07.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.264+0000 7fd7a0213800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5600a4478c70 0x5600a447b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.264+0000 7fd7a0213800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5600a447b5a0 con 0x5600a4475160 2024-05-08T18:05:07.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.264+0000 7fd7a0213800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5600a4314e20 con 0x5600a4478c70 2024-05-08T18:05:07.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.264+0000 7fd79b066640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5600a4478c70 0x5600a447b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.264+0000 7fd79b867640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5600a4475160 0x5600a4477550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43114/0 (socket says 172.21.15.50:43114) 2024-05-08T18:05:07.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.264+0000 7fd79b066640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5600a4478c70 0x5600a447b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37426/0 (socket says 172.21.15.50:37426) 2024-05-08T18:05:07.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.264+0000 7fd79b867640 1 -- 172.21.15.50:0/2550951439 learned_addr learned my addr 172.21.15.50:0/2550951439 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:07.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.265+0000 7fd79b066640 1 -- 172.21.15.50:0/2550951439 >> v1:172.21.15.46:6789/0 conn(0x5600a4475160 legacy=0x5600a4477550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.265+0000 7fd79b066640 1 -- 172.21.15.50:0/2550951439 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600a447c780 con 0x5600a4478c70 2024-05-08T18:05:07.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.267+0000 7fd79b066640 1 --2- 172.21.15.50:0/2550951439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4478c70 0x5600a447b060 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd79000a470 tx=0x7fd79001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87661d1064662eb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.267+0000 7fd79a865640 1 -- 172.21.15.50:0/2550951439 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd790057070 con 0x5600a4478c70 2024-05-08T18:05:07.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.267+0000 7fd79a865640 1 -- 172.21.15.50:0/2550951439 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd790052570 con 0x5600a4478c70 2024-05-08T18:05:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.267+0000 7fd79a865640 1 -- 172.21.15.50:0/2550951439 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd790052870 con 0x5600a4478c70 2024-05-08T18:05:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.267+0000 7fd7a0213800 1 -- 172.21.15.50:0/2550951439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4478c70 msgr2=0x5600a447b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.267+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2550951439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4478c70 0x5600a447b060 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd79000a470 tx=0x7fd79001ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.267+0000 7fd7a0213800 1 -- 172.21.15.50:0/2550951439 shutdown_connections 2024-05-08T18:05:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.267+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2550951439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4478c70 0x5600a447b060 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.267+0000 7fd7a0213800 1 -- 172.21.15.50:0/2550951439 >> 172.21.15.50:0/2550951439 conn(0x5600a446eb50 msgr2=0x5600a4470f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.267+0000 7fd7a0213800 1 -- 172.21.15.50:0/2550951439 shutdown_connections 2024-05-08T18:05:07.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.267+0000 7fd7a0213800 1 -- 172.21.15.50:0/2550951439 wait complete. 2024-05-08T18:05:07.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.283+0000 7fd7a0213800 1 Processor -- start 2024-05-08T18:05:07.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.283+0000 7fd7a0213800 1 -- start start 2024-05-08T18:05:07.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.283+0000 7fd7a0213800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4475160 0x5600a4565b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.283+0000 7fd7a0213800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5600a43bfa10 con 0x5600a4475160 2024-05-08T18:05:07.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.283+0000 7fd79b867640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4475160 0x5600a4565b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.283+0000 7fd79b867640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4475160 0x5600a4565b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37436/0 (socket says 172.21.15.50:37436) 2024-05-08T18:05:07.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.283+0000 7fd79b867640 1 -- 172.21.15.50:0/839675008 learned_addr learned my addr 172.21.15.50:0/839675008 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.283+0000 7fd79b867640 1 -- 172.21.15.50:0/839675008 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600a4566110 con 0x5600a4475160 2024-05-08T18:05:07.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.284+0000 7fd79b867640 1 --2- 172.21.15.50:0/839675008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4475160 0x5600a4565b40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd78c001e10 tx=0x7fd78c00b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0df17c069a773d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.284+0000 7fd689e0b640 1 -- 172.21.15.50:0/839675008 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd78c012020 con 0x5600a4475160 2024-05-08T18:05:07.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.284+0000 7fd689e0b640 1 -- 172.21.15.50:0/839675008 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd78c00e5a0 con 0x5600a4475160 2024-05-08T18:05:07.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.284+0000 7fd689e0b640 1 -- 172.21.15.50:0/839675008 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd78c00e8a0 con 0x5600a4475160 2024-05-08T18:05:07.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.285+0000 7fd7a0213800 1 -- 172.21.15.50:0/839675008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4475160 msgr2=0x5600a4565b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.285+0000 7fd7a0213800 1 --2- 172.21.15.50:0/839675008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4475160 0x5600a4565b40 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd78c001e10 tx=0x7fd78c00b780 comp rx=0 tx=0).stop 2024-05-08T18:05:07.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.285+0000 7fd7a0213800 1 -- 172.21.15.50:0/839675008 shutdown_connections 2024-05-08T18:05:07.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.285+0000 7fd7a0213800 1 --2- 172.21.15.50:0/839675008 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4475160 0x5600a4565b40 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.285+0000 7fd7a0213800 1 -- 172.21.15.50:0/839675008 >> 172.21.15.50:0/839675008 conn(0x5600a446eb50 msgr2=0x5600a4561460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.285+0000 7fd7a0213800 1 -- 172.21.15.50:0/839675008 shutdown_connections 2024-05-08T18:05:07.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.285+0000 7fd7a0213800 1 -- 172.21.15.50:0/839675008 wait complete. 2024-05-08T18:05:07.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.285+0000 7fd7a0213800 1 Processor -- start 2024-05-08T18:05:07.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.285+0000 7fd7a0213800 1 -- start start 2024-05-08T18:05:07.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.285+0000 7fd7a0213800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4475160 0x5600a456d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.285+0000 7fd7a0213800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5600a447ba70 con 0x5600a4475160 2024-05-08T18:05:07.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.285+0000 7fd79b867640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4475160 0x5600a456d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.285+0000 7fd79b867640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4475160 0x5600a456d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37452/0 (socket says 172.21.15.50:37452) 2024-05-08T18:05:07.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.285+0000 7fd79b867640 1 -- 172.21.15.50:0/2964563807 learned_addr learned my addr 172.21.15.50:0/2964563807 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.286+0000 7fd79b867640 1 -- 172.21.15.50:0/2964563807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600a456dc50 con 0x5600a4475160 2024-05-08T18:05:07.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.286+0000 7fd79b867640 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4475160 0x5600a456d710 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x5600a45668d0 tx=0x7fd78c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.286+0000 7fd683fff640 1 -- 172.21.15.50:0/2964563807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd78c012030 con 0x5600a4475160 2024-05-08T18:05:07.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.286+0000 7fd683fff640 1 -- 172.21.15.50:0/2964563807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd78c001470 con 0x5600a4475160 2024-05-08T18:05:07.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.286+0000 7fd7a0213800 1 -- 172.21.15.50:0/2964563807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5600a456dee0 con 0x5600a4475160 2024-05-08T18:05:07.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.286+0000 7fd683fff640 1 -- 172.21.15.50:0/2964563807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd78c01a7a0 con 0x5600a4475160 2024-05-08T18:05:07.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.286+0000 7fd7a0213800 1 -- 172.21.15.50:0/2964563807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600a4570b90 con 0x5600a4475160 2024-05-08T18:05:07.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.287+0000 7fd683fff640 1 -- 172.21.15.50:0/2964563807 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fd78c015020 con 0x5600a4475160 2024-05-08T18:05:07.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.287+0000 7fd683fff640 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd68403bbf0 0x7fd68403e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.287+0000 7fd683fff640 1 -- 172.21.15.50:0/2964563807 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fd78c020030 con 0x5600a4475160 2024-05-08T18:05:07.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.287+0000 7fd79b066640 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd68403bbf0 0x7fd68403e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.287+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5600a4478c70 0x5600a4574950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.287+0000 7fd7a0213800 1 -- 172.21.15.50:0/2964563807 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a4574e90 con 0x5600a4478c70 2024-05-08T18:05:07.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.287+0000 7fd79c068640 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5600a4478c70 0x5600a4574950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.288+0000 7fd79b066640 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd68403bbf0 0x7fd68403e0b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd790054000 tx=0x7fd79004e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.288+0000 7fd79c068640 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5600a4478c70 0x5600a4574950 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.288+0000 7fd79c068640 1 -- 172.21.15.50:0/2964563807 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a4478c70 2024-05-08T18:05:07.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.288+0000 7fd7a0213800 1 -- 172.21.15.50:0/2964563807 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a4571f00 con 0x5600a4478c70 2024-05-08T18:05:07.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.288+0000 7fd79c068640 1 -- 172.21.15.50:0/2964563807 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a4478c70 2024-05-08T18:05:07.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.288+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5600a45771b0 0x5600a45795a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.288+0000 7fd7a0213800 1 -- 172.21.15.50:0/2964563807 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a4579c90 con 0x5600a45771b0 2024-05-08T18:05:07.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.289+0000 7fd79b867640 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5600a45771b0 0x5600a45795a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.289+0000 7fd79b867640 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5600a45771b0 0x5600a45795a0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.289+0000 7fd79b867640 1 -- 172.21.15.50:0/2964563807 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45771b0 2024-05-08T18:05:07.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.289+0000 7fd7a0213800 1 -- 172.21.15.50:0/2964563807 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a4576050 con 0x5600a4478c70 2024-05-08T18:05:07.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.289+0000 7fd79c068640 1 -- 172.21.15.50:0/2964563807 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a4478c70 2024-05-08T18:05:07.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.289+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5600a457b0f0 0x5600a457d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.290+0000 7fd7a0213800 1 -- 172.21.15.50:0/2964563807 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a457dbe0 con 0x5600a457b0f0 2024-05-08T18:05:07.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.290+0000 7fd79b066640 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5600a457b0f0 0x5600a457d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.290+0000 7fd79b066640 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5600a457b0f0 0x5600a457d4e0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.290+0000 7fd79b066640 1 -- 172.21.15.50:0/2964563807 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fd790052a40 con 0x5600a457b0f0 2024-05-08T18:05:07.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.291+0000 7fd7a0213800 1 Processor -- start 2024-05-08T18:05:07.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.291+0000 7fd7a0213800 1 -- start start 2024-05-08T18:05:07.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.291+0000 7fd7a0213800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a458be90 0x5600a458c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.291+0000 7fd7a0213800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5600a4366330 con 0x5600a458be90 2024-05-08T18:05:07.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.291+0000 7fd79b867640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a458be90 0x5600a458c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.291+0000 7fd79b867640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a458be90 0x5600a458c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37464/0 (socket says 172.21.15.50:37464) 2024-05-08T18:05:07.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.291+0000 7fd79b867640 1 -- 172.21.15.50:0/3148821231 learned_addr learned my addr 172.21.15.50:0/3148821231 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.291+0000 7fd79b867640 1 -- 172.21.15.50:0/3148821231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600a458c7f0 con 0x5600a458be90 2024-05-08T18:05:07.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.291+0000 7fd79b867640 1 --2- 172.21.15.50:0/3148821231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a458be90 0x5600a458c2b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd78c050f60 tx=0x7fd78c04a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.292+0000 7fd6817fa640 1 -- 172.21.15.50:0/3148821231 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd78c00ccc0 con 0x5600a458be90 2024-05-08T18:05:07.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.292+0000 7fd6817fa640 1 -- 172.21.15.50:0/3148821231 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd78c048070 con 0x5600a458be90 2024-05-08T18:05:07.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.292+0000 7fd7a0213800 1 -- 172.21.15.50:0/3148821231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5600a458d260 con 0x5600a458be90 2024-05-08T18:05:07.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.292+0000 7fd6817fa640 1 -- 172.21.15.50:0/3148821231 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd78c0163b0 con 0x5600a458be90 2024-05-08T18:05:07.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.292+0000 7fd7a0213800 1 -- 172.21.15.50:0/3148821231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600a458d760 con 0x5600a458be90 2024-05-08T18:05:07.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.292+0000 7fd6817fa640 1 -- 172.21.15.50:0/3148821231 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fd78c016780 con 0x5600a458be90 2024-05-08T18:05:07.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.293+0000 7fd6817fa640 1 --2- 172.21.15.50:0/3148821231 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd67403bba0 0x7fd67403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.293+0000 7fd6817fa640 1 -- 172.21.15.50:0/3148821231 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fd78c040490 con 0x5600a458be90 2024-05-08T18:05:07.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.293+0000 7fd79b066640 1 --2- 172.21.15.50:0/3148821231 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd67403bba0 0x7fd67403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.293+0000 7fd7a0213800 1 Processor -- start 2024-05-08T18:05:07.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.293+0000 7fd7a0213800 1 -- start start 2024-05-08T18:05:07.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.293+0000 7fd79b066640 1 --2- 172.21.15.50:0/3148821231 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd67403bba0 0x7fd67403e060 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x5600a4575760 tx=0x7fd790005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.293+0000 7fd7a0213800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4599060 0x5600a4599480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.293+0000 7fd7a0213800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5600a4566490 con 0x5600a4599060 2024-05-08T18:05:07.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.293+0000 7fd79b867640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4599060 0x5600a4599480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.293+0000 7fd79b867640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4599060 0x5600a4599480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37470/0 (socket says 172.21.15.50:37470) 2024-05-08T18:05:07.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.293+0000 7fd79b867640 1 -- 172.21.15.50:0/3210657477 learned_addr learned my addr 172.21.15.50:0/3210657477 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.294+0000 7fd79b867640 1 -- 172.21.15.50:0/3210657477 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600a4599a50 con 0x5600a4599060 2024-05-08T18:05:07.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.294+0000 7fd79b867640 1 --2- 172.21.15.50:0/3210657477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4599060 0x5600a4599480 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd78c045f90 tx=0x7fd78c044f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f4bb2e8088021 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.294+0000 7fd67affd640 1 -- 172.21.15.50:0/3210657477 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd78c0380b0 con 0x5600a4599060 2024-05-08T18:05:07.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.294+0000 7fd67affd640 1 -- 172.21.15.50:0/3210657477 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd78c0245e0 con 0x5600a4599060 2024-05-08T18:05:07.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.294+0000 7fd67affd640 1 -- 172.21.15.50:0/3210657477 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd78c00dac0 con 0x5600a4599060 2024-05-08T18:05:07.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.294+0000 7fd7a0213800 1 -- 172.21.15.50:0/3210657477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4599060 msgr2=0x5600a4599480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.294+0000 7fd7a0213800 1 --2- 172.21.15.50:0/3210657477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4599060 0x5600a4599480 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd78c045f90 tx=0x7fd78c044f70 comp rx=0 tx=0).stop 2024-05-08T18:05:07.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.294+0000 7fd7a0213800 1 -- 172.21.15.50:0/3210657477 shutdown_connections 2024-05-08T18:05:07.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.294+0000 7fd7a0213800 1 --2- 172.21.15.50:0/3210657477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4599060 0x5600a4599480 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.294+0000 7fd7a0213800 1 -- 172.21.15.50:0/3210657477 >> 172.21.15.50:0/3210657477 conn(0x5600a4595330 msgr2=0x5600a4593390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.294+0000 7fd7a0213800 1 -- 172.21.15.50:0/3210657477 shutdown_connections 2024-05-08T18:05:07.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.294+0000 7fd7a0213800 1 -- 172.21.15.50:0/3210657477 wait complete. 2024-05-08T18:05:07.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.295+0000 7fd7a0213800 1 Processor -- start 2024-05-08T18:05:07.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.295+0000 7fd7a0213800 1 -- start start 2024-05-08T18:05:07.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.295+0000 7fd7a0213800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4599060 0x5600a45a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.295+0000 7fd7a0213800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5600a459a120 con 0x5600a4599060 2024-05-08T18:05:07.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.295+0000 7fd79b867640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4599060 0x5600a45a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.295+0000 7fd79b867640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4599060 0x5600a45a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37486/0 (socket says 172.21.15.50:37486) 2024-05-08T18:05:07.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.295+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 learned_addr learned my addr 172.21.15.50:0/2361505135 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.295+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600a45a08d0 con 0x5600a4599060 2024-05-08T18:05:07.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.296+0000 7fd79b867640 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4599060 0x5600a45a0390 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0247d0 tx=0x7fd78c040130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.296+0000 7fd6797fa640 1 -- 172.21.15.50:0/2361505135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd78c0380b0 con 0x5600a4599060 2024-05-08T18:05:07.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.296+0000 7fd6797fa640 1 -- 172.21.15.50:0/2361505135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd78c024a70 con 0x5600a4599060 2024-05-08T18:05:07.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.296+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5600a45a08d0 con 0x5600a4599060 2024-05-08T18:05:07.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.296+0000 7fd6797fa640 1 -- 172.21.15.50:0/2361505135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd78c024d50 con 0x5600a4599060 2024-05-08T18:05:07.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.296+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600a45a08d0 con 0x5600a4599060 2024-05-08T18:05:07.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.296+0000 7fd6797fa640 1 -- 172.21.15.50:0/2361505135 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fd78c05c030 con 0x5600a4599060 2024-05-08T18:05:07.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.297+0000 7fd6797fa640 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd66403bba0 0x7fd66403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.297+0000 7fd6797fa640 1 -- 172.21.15.50:0/2361505135 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fd78c066070 con 0x5600a4599060 2024-05-08T18:05:07.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.297+0000 7fd79b066640 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd66403bba0 0x7fd66403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.297+0000 7fd7a0213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.297+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5600a45abaa0 0x5600a45adf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.297+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45ae6b0 con 0x5600a45abaa0 2024-05-08T18:05:07.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.297+0000 7fd79c068640 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5600a45abaa0 0x5600a45adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.297+0000 7fd79b066640 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd66403bba0 0x7fd66403e060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd790052fd0 tx=0x7fd79004f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.298+0000 7fd79c068640 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5600a45abaa0 0x5600a45adf60 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.298+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a45abaa0 2024-05-08T18:05:07.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.298+0000 7fd7a0213800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.298+0000 7fd7a0213800 20 realm 2024-05-08T18:05:07.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.298+0000 7fd7a0213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.298+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45aeac0 con 0x5600a45abaa0 2024-05-08T18:05:07.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.298+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a45abaa0 2024-05-08T18:05:07.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.298+0000 7fd7a0213800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.298+0000 7fd7a0213800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:07.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.298+0000 7fd7a0213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.298+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45af390 con 0x5600a45abaa0 2024-05-08T18:05:07.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.298+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a45abaa0 2024-05-08T18:05:07.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.298+0000 7fd7a0213800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.298+0000 7fd7a0213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.298+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45af730 con 0x5600a45abaa0 2024-05-08T18:05:07.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.298+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a45abaa0 2024-05-08T18:05:07.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.299+0000 7fd7a0213800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:07.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.299+0000 7fd7a0213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.299+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5600a45b0220 0x5600a45b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.299+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45b2d70 con 0x5600a45b0220 2024-05-08T18:05:07.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.299+0000 7fd79b867640 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5600a45b0220 0x5600a45b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.299+0000 7fd79b867640 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5600a45b0220 0x5600a45b2610 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.299+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.299+0000 7fd7a0213800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:07.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.299+0000 7fd7a0213800 20 searching for the correct realm 2024-05-08T18:05:07.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.299+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5600a45b4e20 0x5600a45b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.299+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5600a45b79b0 con 0x5600a45b4e20 2024-05-08T18:05:07.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.300+0000 7fd79b066640 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5600a45b4e20 0x5600a45b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.300+0000 7fd79b066640 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5600a45b4e20 0x5600a45b72e0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.300+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.300+0000 7fd79b066640 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fd790076040 0x7fd790083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.300+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5600a45b79b0 con 0x7fd790076040 2024-05-08T18:05:07.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.300+0000 7fd79c068640 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fd790076040 0x7fd790083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.301+0000 7fd79c068640 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fd790076040 0x7fd790083410 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.301+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd79400a4a0 con 0x7fd790076040 2024-05-08T18:05:07.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.301+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5600a45af730 con 0x5600a45b0220 2024-05-08T18:05:07.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.301+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.301+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5600a45af730 con 0x5600a45abaa0 2024-05-08T18:05:07.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.301+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a45abaa0 2024-05-08T18:05:07.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.301+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5600a45af730 con 0x5600a45abaa0 2024-05-08T18:05:07.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.302+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a45abaa0 2024-05-08T18:05:07.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.302+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5600a45af730 con 0x5600a45b0220 2024-05-08T18:05:07.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.302+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.302+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5600a45af730 con 0x5600a45b0220 2024-05-08T18:05:07.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.302+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.302+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5600a45af730 con 0x7fd790076040 2024-05-08T18:05:07.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.303+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd79400a4a0 con 0x7fd790076040 2024-05-08T18:05:07.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.303+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5600a45af730 con 0x5600a45b4e20 2024-05-08T18:05:07.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.303+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.303+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5600a45af730 con 0x5600a45abaa0 2024-05-08T18:05:07.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.303+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a45abaa0 2024-05-08T18:05:07.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.303+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5600a45af730 con 0x5600a45b0220 2024-05-08T18:05:07.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.304+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.304+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5600a45af730 con 0x5600a45abaa0 2024-05-08T18:05:07.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.304+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a45abaa0 2024-05-08T18:05:07.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.304+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5600a45af730 con 0x7fd790076040 2024-05-08T18:05:07.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.304+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fd79400a4a0 con 0x7fd790076040 2024-05-08T18:05:07.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.304+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5600a45af730 con 0x5600a45b4e20 2024-05-08T18:05:07.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.304+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.305+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x5600a45b4e20 2024-05-08T18:05:07.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.305+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.305+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x5600a45b0220 2024-05-08T18:05:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.305+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.305+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x5600a45b4e20 2024-05-08T18:05:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.305+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.305+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x5600a45b4e20 2024-05-08T18:05:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.305+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.305+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x5600a45abaa0 2024-05-08T18:05:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.306+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a45abaa0 2024-05-08T18:05:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.306+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x5600a45abaa0 2024-05-08T18:05:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.306+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a45abaa0 2024-05-08T18:05:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.306+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x5600a45b0220 2024-05-08T18:05:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.306+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.306+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x5600a45b4e20 2024-05-08T18:05:07.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.307+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.307+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x7fd790076040 2024-05-08T18:05:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.307+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd79400a4a0 con 0x7fd790076040 2024-05-08T18:05:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.307+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x5600a45abaa0 2024-05-08T18:05:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.307+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a45abaa0 2024-05-08T18:05:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.307+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x5600a45b0220 2024-05-08T18:05:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.307+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.307+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x5600a45b4e20 2024-05-08T18:05:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.308+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.308+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x5600a45b0220 2024-05-08T18:05:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.308+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.308+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x7fd790076040 2024-05-08T18:05:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.308+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd79400a4a0 con 0x7fd790076040 2024-05-08T18:05:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.308+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x5600a45b4e20 2024-05-08T18:05:07.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.308+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.308+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x5600a45b0220 2024-05-08T18:05:07.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x5600a45abaa0 2024-05-08T18:05:07.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a45abaa0 2024-05-08T18:05:07.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fd790090040 con 0x7fd790076040 2024-05-08T18:05:07.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd79400a4a0 con 0x7fd790076040 2024-05-08T18:05:07.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd7a0213800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:07.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd7a0213800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:07.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd7a0213800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:07.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd7a0213800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:07.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd7a0213800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:07.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd7a0213800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:07.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd7a0213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45b3720 con 0x5600a45abaa0 2024-05-08T18:05:07.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a45abaa0 2024-05-08T18:05:07.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd7a0213800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd7a0213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.309+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45b3720 con 0x5600a45abaa0 2024-05-08T18:05:07.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd79400a4a0 con 0x5600a45abaa0 2024-05-08T18:05:07.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd7a0213800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:07.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd7a0213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45b36f0 con 0x5600a45b4e20 2024-05-08T18:05:07.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd7a0213800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:07.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd7a0213800 20 zone default found 2024-05-08T18:05:07.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd7a0213800 4 Realm: () 2024-05-08T18:05:07.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd7a0213800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:07.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd7a0213800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:07.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd7a0213800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:07.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd7a0213800 20 zonegroup default 2024-05-08T18:05:07.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd7a0213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45b3fc0 con 0x5600a45b0220 2024-05-08T18:05:07.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd7a0213800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd7a0213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45b9810 con 0x5600a45b4e20 2024-05-08T18:05:07.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.310+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.311+0000 7fd7a0213800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.311+0000 7fd7a0213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.311+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45b9bb0 con 0x5600a45b4e20 2024-05-08T18:05:07.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.311+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.311+0000 7fd7a0213800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.311+0000 7fd7a0213800 20 started sync module instance, tier type = 2024-05-08T18:05:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.311+0000 7fd7a0213800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.311+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45b8cb0 con 0x7fd790076040 2024-05-08T18:05:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.312+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd79400a4a0 con 0x7fd790076040 2024-05-08T18:05:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.312+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94560757453280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45b8cb0 con 0x7fd790076040 2024-05-08T18:05:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.312+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45ba5b0 con 0x5600a45abaa0 2024-05-08T18:05:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.313+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94560757453280] v27'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd794040040 con 0x7fd790076040 2024-05-08T18:05:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.313+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd794040040 con 0x5600a45abaa0 2024-05-08T18:05:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.313+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94560757457088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45b8cb0 con 0x5600a45abaa0 2024-05-08T18:05:07.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.313+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45bb080 con 0x5600a45b4e20 2024-05-08T18:05:07.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.314+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.314+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94560757459824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45bb920 con 0x5600a45b4e20 2024-05-08T18:05:07.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.314+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45bc3e0 con 0x7fd790076040 2024-05-08T18:05:07.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.315+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94560757457088] v27'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd794040040 con 0x5600a45abaa0 2024-05-08T18:05:07.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.315+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94560757459824] v27'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.315+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd794040040 con 0x7fd790076040 2024-05-08T18:05:07.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.315+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94560757462128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45b8800 con 0x7fd790076040 2024-05-08T18:05:07.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.315+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45bccb0 con 0x5600a45abaa0 2024-05-08T18:05:07.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.317+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94560757462128] v27'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd794040040 con 0x7fd790076040 2024-05-08T18:05:07.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.317+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd794040040 con 0x5600a45abaa0 2024-05-08T18:05:07.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.317+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94560757464848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45bcc30 con 0x5600a45abaa0 2024-05-08T18:05:07.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.317+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45bd630 con 0x5600a45b0220 2024-05-08T18:05:07.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.318+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94560757464848] v27'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd794040040 con 0x5600a45abaa0 2024-05-08T18:05:07.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.318+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.318+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94560757467008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45bcba0 con 0x5600a45b0220 2024-05-08T18:05:07.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.318+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45be3a0 con 0x5600a45b0220 2024-05-08T18:05:07.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.319+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.319+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94560757475216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45bf060 con 0x5600a45b0220 2024-05-08T18:05:07.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.319+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94560757467008] v27'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.319+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45bf570 con 0x5600a45b0220 2024-05-08T18:05:07.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.320+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94560757475216] v27'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.320+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.320+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94560757471776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45bf750 con 0x5600a45b0220 2024-05-08T18:05:07.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.320+0000 7fd7a0213800 20 add_watcher() i=0 2024-05-08T18:05:07.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.320+0000 7fd7a0213800 20 add_watcher() i=1 2024-05-08T18:05:07.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.320+0000 7fd7a0213800 20 add_watcher() i=2 2024-05-08T18:05:07.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.320+0000 7fd7a0213800 20 add_watcher() i=3 2024-05-08T18:05:07.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.320+0000 7fd7a0213800 20 add_watcher() i=4 2024-05-08T18:05:07.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.320+0000 7fd7a0213800 20 add_watcher() i=5 2024-05-08T18:05:07.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.320+0000 7fd7a0213800 20 add_watcher() i=6 2024-05-08T18:05:07.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.321+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94560757471776] v27'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.321+0000 7fd7a0213800 20 add_watcher() i=7 2024-05-08T18:05:07.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.321+0000 7fd7a0213800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:07.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.321+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45c2b20 con 0x5600a45abaa0 2024-05-08T18:05:07.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.321+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fd794040040 con 0x5600a45abaa0 2024-05-08T18:05:07.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.321+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94560757478976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45bff10 con 0x5600a45abaa0 2024-05-08T18:05:07.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.322+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94560757478976] v27'96 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd794040040 con 0x5600a45abaa0 2024-05-08T18:05:07.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.322+0000 7fd651fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:07.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.322+0000 7fd7a0213800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:07.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.322+0000 7fd7a0213800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:07.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.322+0000 7fd7a0213800 5 note: GC not initialized 2024-05-08T18:05:07.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.323+0000 7fd650fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:07.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.323+0000 7fd7a0213800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:07.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.323+0000 7fd7a0213800 20 Filter name: none 2024-05-08T18:05:07.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.323+0000 7fd5ca7fc640 20 reqs_thread_entry: start 2024-05-08T18:05:07.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.323+0000 7fd7a0213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.323+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45f5f40 con 0x7fd790076040 2024-05-08T18:05:07.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.323+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fd794040040 con 0x7fd790076040 2024-05-08T18:05:07.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.323+0000 7fd7a0213800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:07.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.323+0000 7fd7a0213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.323+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45f67d0 con 0x5600a45b0220 2024-05-08T18:05:07.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.324+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.324+0000 7fd7a0213800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:07.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.324+0000 7fd7a0213800 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-08T18:05:07.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.324+0000 7fd7a0213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.324+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45f6980 con 0x7fd790076040 2024-05-08T18:05:07.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.324+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fd794040040 con 0x7fd790076040 2024-05-08T18:05:07.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.324+0000 7fd7a0213800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:07.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.324+0000 7fd7a0213800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.324+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:65 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45f52e0 con 0x5600a45b0220 2024-05-08T18:05:07.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(65 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd7a0213800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:07.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd7a0213800 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-08T18:05:07.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:66 4.37 4:eecadb7d:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45f92c0 con 0x5600a45b0220 2024-05-08T18:05:07.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 4.22 4:46cfe895:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45fbc10 con 0x5600a45abaa0 2024-05-08T18:05:07.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 4.0 4:018ce329:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45fe6f0 con 0x5600a45b4e20 2024-05-08T18:05:07.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 4.12 4:4afe9c7f:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a4601140 con 0x5600a45abaa0 2024-05-08T18:05:07.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:70 4.d 4:b2c4ac36:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a45fe6f0 con 0x5600a45b4e20 2024-05-08T18:05:07.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 4.33 4:cc412025:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a46065a0 con 0x5600a45b0220 2024-05-08T18:05:07.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(66 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 4.3 4:c17a4869:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a46065a0 con 0x5600a45abaa0 2024-05-08T18:05:07.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(71 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.5 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(67 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fd794040040 con 0x5600a45abaa0 2024-05-08T18:05:07.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(70 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:73 4.6 4:61ff1ea9:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a4605f70 con 0x5600a45b4e20 2024-05-08T18:05:07.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 4.1c 4:3ae7d9ef:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a4604560 con 0x5600a45b0220 2024-05-08T18:05:07.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(68 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(69 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fd794040040 con 0x5600a45abaa0 2024-05-08T18:05:07.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 4.f 4:f2293886:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a4600b10 con 0x5600a45abaa0 2024-05-08T18:05:07.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.325+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 4.21 4:87ec338a:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5600a4604560 con 0x5600a45b0220 2024-05-08T18:05:07.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.326+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(74 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.326+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(72 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.6 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fd794040040 con 0x5600a45abaa0 2024-05-08T18:05:07.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.326+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(76 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.326+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(73 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.326+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(75 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fd794040040 con 0x5600a45abaa0 2024-05-08T18:05:07.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.327+0000 7fd651fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:07.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.327+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94560757478976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a4600b10 con 0x5600a45abaa0 2024-05-08T18:05:07.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.328+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94560757478976] v27'97 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd794040040 con 0x5600a45abaa0 2024-05-08T18:05:07.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.328+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94560757453280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a4605f70 con 0x7fd790076040 2024-05-08T18:05:07.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.329+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94560757453280] v27'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd794040040 con 0x7fd790076040 2024-05-08T18:05:07.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.329+0000 7fd7a0213800 20 remove_watcher() i=0 2024-05-08T18:05:07.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.329+0000 7fd7a0213800 2 removed watcher, disabling cache 2024-05-08T18:05:07.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.329+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94560757457088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45bf640 con 0x5600a45abaa0 2024-05-08T18:05:07.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.330+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94560757457088] v27'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd794040040 con 0x5600a45abaa0 2024-05-08T18:05:07.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.330+0000 7fd7a0213800 20 remove_watcher() i=1 2024-05-08T18:05:07.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.330+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94560757459824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45ba0c0 con 0x5600a45b4e20 2024-05-08T18:05:07.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.331+0000 7fd79b066640 1 -- 172.21.15.50:0/2361505135 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94560757459824] v27'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd790081070 con 0x5600a45b4e20 2024-05-08T18:05:07.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.331+0000 7fd7a0213800 20 remove_watcher() i=2 2024-05-08T18:05:07.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.331+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94560757462128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45bf640 con 0x7fd790076040 2024-05-08T18:05:07.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.332+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94560757462128] v27'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd794040040 con 0x7fd790076040 2024-05-08T18:05:07.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.332+0000 7fd7a0213800 20 remove_watcher() i=3 2024-05-08T18:05:07.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.332+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94560757464848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a45ba0c0 con 0x5600a45abaa0 2024-05-08T18:05:07.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.333+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94560757464848] v27'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd794040040 con 0x5600a45abaa0 2024-05-08T18:05:07.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.333+0000 7fd7a0213800 20 remove_watcher() i=4 2024-05-08T18:05:07.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.333+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94560757467008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a46049a0 con 0x5600a45b0220 2024-05-08T18:05:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.334+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94560757467008] v27'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd78c0430c0 con 0x5600a45b0220 2024-05-08T18:05:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.334+0000 7fd7a0213800 20 remove_watcher() i=5 2024-05-08T18:05:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.334+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94560757475216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a46049a0 con 0x5600a45b0220 2024-05-08T18:05:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.335+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94560757475216] v27'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd78c090040 con 0x5600a45b0220 2024-05-08T18:05:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.335+0000 7fd7a0213800 20 remove_watcher() i=6 2024-05-08T18:05:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.335+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94560757471776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5600a46049a0 con 0x5600a45b0220 2024-05-08T18:05:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd79b867640 1 -- 172.21.15.50:0/2361505135 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94560757471776] v27'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd78c090040 con 0x5600a45b0220 2024-05-08T18:05:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 20 remove_watcher() i=7 2024-05-08T18:05:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5600a45b0220 msgr2=0x5600a45b2610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5600a45b0220 0x5600a45b2610 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5600a45abaa0 msgr2=0x5600a45adf60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5600a45abaa0 0x5600a45adf60 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fd790076040 msgr2=0x7fd790083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fd790076040 0x7fd790083410 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5600a45b4e20 msgr2=0x5600a45b72e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5600a45b4e20 0x5600a45b72e0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd66403bba0 msgr2=0x7fd66403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd66403bba0 0x7fd66403e060 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd790052fd0 tx=0x7fd79004f000 comp rx=0 tx=0).stop 2024-05-08T18:05:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4599060 msgr2=0x5600a45a0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2361505135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4599060 0x5600a45a0390 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd78c0247d0 tx=0x7fd78c040130 comp rx=0 tx=0).stop 2024-05-08T18:05:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 reap_dead start 2024-05-08T18:05:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd79c068640 1 -- 172.21.15.50:0/2361505135 reap_dead start 2024-05-08T18:05:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 shutdown_connections 2024-05-08T18:05:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 >> 172.21.15.50:0/2361505135 conn(0x5600a4595330 msgr2=0x5600a4592990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 shutdown_connections 2024-05-08T18:05:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.336+0000 7fd7a0213800 1 -- 172.21.15.50:0/2361505135 wait complete. 2024-05-08T18:05:07.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 -- 172.21.15.50:0/3148821231 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd67403bba0 msgr2=0x7fd67403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 --2- 172.21.15.50:0/3148821231 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd67403bba0 0x7fd67403e060 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x5600a4575760 tx=0x7fd790005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 -- 172.21.15.50:0/3148821231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a458be90 msgr2=0x5600a458c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 --2- 172.21.15.50:0/3148821231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a458be90 0x5600a458c2b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd78c050f60 tx=0x7fd78c04a1a0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 -- 172.21.15.50:0/3148821231 shutdown_connections 2024-05-08T18:05:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 --2- 172.21.15.50:0/3148821231 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd67403bba0 0x7fd67403e060 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 --2- 172.21.15.50:0/3148821231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a458be90 0x5600a458c2b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 -- 172.21.15.50:0/3148821231 >> 172.21.15.50:0/3148821231 conn(0x5600a457ff30 msgr2=0x5600a4580330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 -- 172.21.15.50:0/3148821231 shutdown_connections 2024-05-08T18:05:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 -- 172.21.15.50:0/3148821231 wait complete. 2024-05-08T18:05:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 -- 172.21.15.50:0/2964563807 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5600a45771b0 msgr2=0x5600a45795a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5600a45771b0 0x5600a45795a0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 -- 172.21.15.50:0/2964563807 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5600a4478c70 msgr2=0x5600a4574950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5600a4478c70 0x5600a4574950 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 -- 172.21.15.50:0/2964563807 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5600a457b0f0 msgr2=0x5600a457d4e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5600a457b0f0 0x5600a457d4e0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 -- 172.21.15.50:0/2964563807 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd68403bbf0 msgr2=0x7fd68403e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd68403bbf0 0x7fd68403e0b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd790054000 tx=0x7fd79004e000 comp rx=0 tx=0).stop 2024-05-08T18:05:07.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 -- 172.21.15.50:0/2964563807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4475160 msgr2=0x5600a456d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 --2- 172.21.15.50:0/2964563807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5600a4475160 0x5600a456d710 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x5600a45668d0 tx=0x7fd78c000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd79c068640 1 -- 172.21.15.50:0/2964563807 reap_dead start 2024-05-08T18:05:07.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 -- 172.21.15.50:0/2964563807 shutdown_connections 2024-05-08T18:05:07.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 -- 172.21.15.50:0/2964563807 >> 172.21.15.50:0/2964563807 conn(0x5600a446eb50 msgr2=0x5600a455fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 -- 172.21.15.50:0/2964563807 shutdown_connections 2024-05-08T18:05:07.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.337+0000 7fd7a0213800 1 -- 172.21.15.50:0/2964563807 wait complete. 2024-05-08T18:05:07.347 INFO:teuthology.orchestra.run.smithi050.stdout:{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"36133b23-abaa-446a-bf67-ceb1d1559ceb","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1","marker":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1","index_type":"Normal","index_generation":0,"num_shards":11,"object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-05-08T18:05:06.708772Z","creation_time":"2024-05-08T18:05:06.706377Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}} 2024-05-08T18:05:07.347 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': '36133b23-abaa-446a-bf67-ceb1d1559ceb', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1', 'marker': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-05-08T18:05:06.708772Z', 'creation_time': '2024-05-08T18:05:06.706377Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-05-08T18:05:07.347 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2024-05-08T18:05:07.348 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2024-05-08T18:05:07.348 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2024-05-08T18:05:07.393 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:07.393 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:07.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.396+0000 7fe935b33800 1 Processor -- start 2024-05-08T18:05:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.396+0000 7fe935b33800 1 -- start start 2024-05-08T18:05:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.396+0000 7fe935b33800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x559d033d6b60 0x559d033d8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.396+0000 7fe935b33800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559d033d9520 con 0x559d033d0940 2024-05-08T18:05:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.396+0000 7fe935b33800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x559d03272e20 con 0x559d033d6b60 2024-05-08T18:05:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.396+0000 7fe930fc9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x559d033d6b60 0x559d033d8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.396+0000 7fe930fc9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x559d033d6b60 0x559d033d8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37500/0 (socket says 172.21.15.50:37500) 2024-05-08T18:05:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.396+0000 7fe930fc9640 1 -- 172.21.15.50:0/3771855302 learned_addr learned my addr 172.21.15.50:0/3771855302 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.396+0000 7fe9317ca640 1 --1- 172.21.15.50:0/3771855302 >> v1:172.21.15.46:6789/0 conn(0x559d033d0940 0x559d033d0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43118/0 (socket says 172.21.15.50:43118) 2024-05-08T18:05:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.397+0000 7fe930fc9640 1 -- 172.21.15.50:0/3771855302 >> v1:172.21.15.46:6789/0 conn(0x559d033d0940 legacy=0x559d033d0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.397+0000 7fe930fc9640 1 -- 172.21.15.50:0/3771855302 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d033da700 con 0x559d033d6b60 2024-05-08T18:05:07.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.398+0000 7fe930fc9640 1 --2- 172.21.15.50:0/3771855302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d6b60 0x559d033d8f50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe92800a520 tx=0x7fe92801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d671307e688c2c85 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.399+0000 7fe923fff640 1 -- 172.21.15.50:0/3771855302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe928057070 con 0x559d033d6b60 2024-05-08T18:05:07.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.399+0000 7fe923fff640 1 -- 172.21.15.50:0/3771855302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe928052570 con 0x559d033d6b60 2024-05-08T18:05:07.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.399+0000 7fe923fff640 1 -- 172.21.15.50:0/3771855302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe92804ae60 con 0x559d033d6b60 2024-05-08T18:05:07.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.399+0000 7fe935b33800 1 -- 172.21.15.50:0/3771855302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d6b60 msgr2=0x559d033d8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.399+0000 7fe935b33800 1 --2- 172.21.15.50:0/3771855302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d6b60 0x559d033d8f50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe92800a520 tx=0x7fe92801d010 comp rx=0 tx=0).stop 2024-05-08T18:05:07.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.399+0000 7fe935b33800 1 -- 172.21.15.50:0/3771855302 shutdown_connections 2024-05-08T18:05:07.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.399+0000 7fe935b33800 1 --2- 172.21.15.50:0/3771855302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d6b60 0x559d033d8f50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.399+0000 7fe935b33800 1 -- 172.21.15.50:0/3771855302 >> 172.21.15.50:0/3771855302 conn(0x559d033cc2d0 msgr2=0x559d033ce6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.399+0000 7fe935b33800 1 -- 172.21.15.50:0/3771855302 shutdown_connections 2024-05-08T18:05:07.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.399+0000 7fe935b33800 1 -- 172.21.15.50:0/3771855302 wait complete. 2024-05-08T18:05:07.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.414+0000 7fe935b33800 1 Processor -- start 2024-05-08T18:05:07.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.414+0000 7fe935b33800 1 -- start start 2024-05-08T18:05:07.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.415+0000 7fe935b33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d0940 0x559d034c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.415+0000 7fe935b33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559d0331da10 con 0x559d033d0940 2024-05-08T18:05:07.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.415+0000 7fe9317ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d0940 0x559d034c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.415+0000 7fe9317ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d0940 0x559d034c3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37512/0 (socket says 172.21.15.50:37512) 2024-05-08T18:05:07.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.415+0000 7fe9317ca640 1 -- 172.21.15.50:0/3690092165 learned_addr learned my addr 172.21.15.50:0/3690092165 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.415+0000 7fe9317ca640 1 -- 172.21.15.50:0/3690092165 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d034c40d0 con 0x559d033d0940 2024-05-08T18:05:07.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.415+0000 7fe9317ca640 1 --2- 172.21.15.50:0/3690092165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d0940 0x559d034c3b00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe92400b260 tx=0x7fe92400b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4998172419cf6e7e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.415+0000 7fe81f606640 1 -- 172.21.15.50:0/3690092165 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe92400e5c0 con 0x559d033d0940 2024-05-08T18:05:07.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.416+0000 7fe81f606640 1 -- 172.21.15.50:0/3690092165 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe924006070 con 0x559d033d0940 2024-05-08T18:05:07.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.416+0000 7fe81f606640 1 -- 172.21.15.50:0/3690092165 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe92400e890 con 0x559d033d0940 2024-05-08T18:05:07.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.416+0000 7fe935b33800 1 -- 172.21.15.50:0/3690092165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d0940 msgr2=0x559d034c3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.416+0000 7fe935b33800 1 --2- 172.21.15.50:0/3690092165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d0940 0x559d034c3b00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe92400b260 tx=0x7fe92400b720 comp rx=0 tx=0).stop 2024-05-08T18:05:07.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.416+0000 7fe935b33800 1 -- 172.21.15.50:0/3690092165 shutdown_connections 2024-05-08T18:05:07.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.416+0000 7fe935b33800 1 --2- 172.21.15.50:0/3690092165 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d0940 0x559d034c3b00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.416+0000 7fe935b33800 1 -- 172.21.15.50:0/3690092165 >> 172.21.15.50:0/3690092165 conn(0x559d033cc2d0 msgr2=0x559d034bf420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.416+0000 7fe935b33800 1 -- 172.21.15.50:0/3690092165 shutdown_connections 2024-05-08T18:05:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.416+0000 7fe935b33800 1 -- 172.21.15.50:0/3690092165 wait complete. 2024-05-08T18:05:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.417+0000 7fe935b33800 1 Processor -- start 2024-05-08T18:05:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.417+0000 7fe935b33800 1 -- start start 2024-05-08T18:05:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.417+0000 7fe935b33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d0940 0x559d034cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.417+0000 7fe935b33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559d033d99c0 con 0x559d033d0940 2024-05-08T18:05:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.417+0000 7fe9317ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d0940 0x559d034cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.417+0000 7fe9317ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d0940 0x559d034cb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37520/0 (socket says 172.21.15.50:37520) 2024-05-08T18:05:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.417+0000 7fe9317ca640 1 -- 172.21.15.50:0/4289333904 learned_addr learned my addr 172.21.15.50:0/4289333904 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.418+0000 7fe9317ca640 1 -- 172.21.15.50:0/4289333904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d034cbc40 con 0x559d033d0940 2024-05-08T18:05:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.418+0000 7fe9317ca640 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d0940 0x559d034cb700 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe924000f60 tx=0x7fe924000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.418+0000 7fe81de03640 1 -- 172.21.15.50:0/4289333904 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe92401a7c0 con 0x559d033d0940 2024-05-08T18:05:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.418+0000 7fe81de03640 1 -- 172.21.15.50:0/4289333904 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe92401a960 con 0x559d033d0940 2024-05-08T18:05:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.418+0000 7fe935b33800 1 -- 172.21.15.50:0/4289333904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559d034cbed0 con 0x559d033d0940 2024-05-08T18:05:07.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.418+0000 7fe81de03640 1 -- 172.21.15.50:0/4289333904 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe92401ac40 con 0x559d033d0940 2024-05-08T18:05:07.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.418+0000 7fe935b33800 1 -- 172.21.15.50:0/4289333904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d034ceb30 con 0x559d033d0940 2024-05-08T18:05:07.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.419+0000 7fe81de03640 1 -- 172.21.15.50:0/4289333904 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fe924017070 con 0x559d033d0940 2024-05-08T18:05:07.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.419+0000 7fe81de03640 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe81803bba0 0x7fe81803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.419+0000 7fe81de03640 1 -- 172.21.15.50:0/4289333904 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fe924015020 con 0x559d033d0940 2024-05-08T18:05:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.419+0000 7fe930fc9640 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe81803bba0 0x7fe81803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.419+0000 7fe935b33800 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559d033d6b60 0x559d034d28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.419+0000 7fe935b33800 1 -- 172.21.15.50:0/4289333904 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d034d2e30 con 0x559d033d6b60 2024-05-08T18:05:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.419+0000 7fe931fcb640 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559d033d6b60 0x559d034d28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.420+0000 7fe930fc9640 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe81803bba0 0x7fe81803e060 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe928054000 tx=0x7fe92804e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.420+0000 7fe931fcb640 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559d033d6b60 0x559d034d28f0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.420+0000 7fe931fcb640 1 -- 172.21.15.50:0/4289333904 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d033d6b60 2024-05-08T18:05:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.420+0000 7fe935b33800 1 -- 172.21.15.50:0/4289333904 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d034cfec0 con 0x559d033d6b60 2024-05-08T18:05:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.420+0000 7fe931fcb640 1 -- 172.21.15.50:0/4289333904 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d033d6b60 2024-05-08T18:05:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.420+0000 7fe935b33800 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559d034d5180 0x559d034d7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.420+0000 7fe935b33800 1 -- 172.21.15.50:0/4289333904 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d034d7c10 con 0x559d034d5180 2024-05-08T18:05:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.420+0000 7fe9317ca640 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559d034d5180 0x559d034d7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.421+0000 7fe9317ca640 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559d034d5180 0x559d034d7570 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.421+0000 7fe9317ca640 1 -- 172.21.15.50:0/4289333904 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe92402e070 con 0x559d034d5180 2024-05-08T18:05:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.421+0000 7fe935b33800 1 -- 172.21.15.50:0/4289333904 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d034d4020 con 0x559d033d6b60 2024-05-08T18:05:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.421+0000 7fe931fcb640 1 -- 172.21.15.50:0/4289333904 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d033d6b60 2024-05-08T18:05:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.421+0000 7fe935b33800 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559d034d9070 0x559d034db460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.421+0000 7fe935b33800 1 -- 172.21.15.50:0/4289333904 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d034dbb60 con 0x559d034d9070 2024-05-08T18:05:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.421+0000 7fe930fc9640 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559d034d9070 0x559d034db460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.422+0000 7fe930fc9640 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559d034d9070 0x559d034db460 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.422+0000 7fe930fc9640 1 -- 172.21.15.50:0/4289333904 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fe92805a040 con 0x559d034d9070 2024-05-08T18:05:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.422+0000 7fe935b33800 1 Processor -- start 2024-05-08T18:05:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.422+0000 7fe935b33800 1 -- start start 2024-05-08T18:05:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.423+0000 7fe935b33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d034e9ed0 0x559d034ea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.423+0000 7fe935b33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559d032c4330 con 0x559d034e9ed0 2024-05-08T18:05:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.423+0000 7fe9317ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d034e9ed0 0x559d034ea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.423+0000 7fe9317ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d034e9ed0 0x559d034ea2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37526/0 (socket says 172.21.15.50:37526) 2024-05-08T18:05:07.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.423+0000 7fe9317ca640 1 -- 172.21.15.50:0/2721256399 learned_addr learned my addr 172.21.15.50:0/2721256399 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.423+0000 7fe9317ca640 1 -- 172.21.15.50:0/2721256399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d034ea830 con 0x559d034e9ed0 2024-05-08T18:05:07.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.423+0000 7fe9317ca640 1 --2- 172.21.15.50:0/2721256399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d034e9ed0 0x559d034ea2f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe924017b00 tx=0x7fe924017f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.423+0000 7fe816ffd640 1 -- 172.21.15.50:0/2721256399 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe92404a2e0 con 0x559d034e9ed0 2024-05-08T18:05:07.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.423+0000 7fe816ffd640 1 -- 172.21.15.50:0/2721256399 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe92404a480 con 0x559d034e9ed0 2024-05-08T18:05:07.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.423+0000 7fe935b33800 1 -- 172.21.15.50:0/2721256399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559d034eb2a0 con 0x559d034e9ed0 2024-05-08T18:05:07.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.423+0000 7fe816ffd640 1 -- 172.21.15.50:0/2721256399 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe92401dd50 con 0x559d034e9ed0 2024-05-08T18:05:07.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.423+0000 7fe935b33800 1 -- 172.21.15.50:0/2721256399 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d034eb7a0 con 0x559d034e9ed0 2024-05-08T18:05:07.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.424+0000 7fe816ffd640 1 -- 172.21.15.50:0/2721256399 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fe9240518e0 con 0x559d034e9ed0 2024-05-08T18:05:07.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.424+0000 7fe816ffd640 1 --2- 172.21.15.50:0/2721256399 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe80803bb00 0x7fe80803dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.424+0000 7fe816ffd640 1 -- 172.21.15.50:0/2721256399 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fe92403d910 con 0x559d034e9ed0 2024-05-08T18:05:07.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.424+0000 7fe930fc9640 1 --2- 172.21.15.50:0/2721256399 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe80803bb00 0x7fe80803dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.425+0000 7fe935b33800 1 Processor -- start 2024-05-08T18:05:07.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.425+0000 7fe935b33800 1 -- start start 2024-05-08T18:05:07.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.425+0000 7fe930fc9640 1 --2- 172.21.15.50:0/2721256399 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe80803bb00 0x7fe80803dfc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x559d034cfde0 tx=0x7fe928005ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.425+0000 7fe935b33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d2f20 0x559d033d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.425+0000 7fe935b33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559d034c4450 con 0x559d033d2f20 2024-05-08T18:05:07.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.425+0000 7fe9317ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d2f20 0x559d033d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.425+0000 7fe9317ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d2f20 0x559d033d3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37536/0 (socket says 172.21.15.50:37536) 2024-05-08T18:05:07.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.425+0000 7fe9317ca640 1 -- 172.21.15.50:0/3399159984 learned_addr learned my addr 172.21.15.50:0/3399159984 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.425+0000 7fe9317ca640 1 -- 172.21.15.50:0/3399159984 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d033d1720 con 0x559d033d2f20 2024-05-08T18:05:07.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.425+0000 7fe9317ca640 1 --2- 172.21.15.50:0/3399159984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d2f20 0x559d033d3340 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe924022ff0 tx=0x7fe924043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8e718eaf4bcff7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.425+0000 7fe814ff9640 1 -- 172.21.15.50:0/3399159984 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe924027430 con 0x559d033d2f20 2024-05-08T18:05:07.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.426+0000 7fe814ff9640 1 -- 172.21.15.50:0/3399159984 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe924051db0 con 0x559d033d2f20 2024-05-08T18:05:07.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.426+0000 7fe814ff9640 1 -- 172.21.15.50:0/3399159984 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe924019990 con 0x559d033d2f20 2024-05-08T18:05:07.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.426+0000 7fe935b33800 1 -- 172.21.15.50:0/3399159984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d2f20 msgr2=0x559d033d3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.426+0000 7fe935b33800 1 --2- 172.21.15.50:0/3399159984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d2f20 0x559d033d3340 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe924022ff0 tx=0x7fe924043040 comp rx=0 tx=0).stop 2024-05-08T18:05:07.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.426+0000 7fe935b33800 1 -- 172.21.15.50:0/3399159984 shutdown_connections 2024-05-08T18:05:07.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.426+0000 7fe935b33800 1 --2- 172.21.15.50:0/3399159984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d2f20 0x559d033d3340 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.426+0000 7fe935b33800 1 -- 172.21.15.50:0/3399159984 >> 172.21.15.50:0/3399159984 conn(0x559d034f32c0 msgr2=0x559d034f1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.426+0000 7fe935b33800 1 -- 172.21.15.50:0/3399159984 shutdown_connections 2024-05-08T18:05:07.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.426+0000 7fe935b33800 1 -- 172.21.15.50:0/3399159984 wait complete. 2024-05-08T18:05:07.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.426+0000 7fe935b33800 1 Processor -- start 2024-05-08T18:05:07.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.426+0000 7fe935b33800 1 -- start start 2024-05-08T18:05:07.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.427+0000 7fe935b33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d2f20 0x559d035005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.427+0000 7fe935b33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559d033d1b30 con 0x559d033d2f20 2024-05-08T18:05:07.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.427+0000 7fe9317ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d2f20 0x559d035005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.427+0000 7fe9317ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d2f20 0x559d035005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37550/0 (socket says 172.21.15.50:37550) 2024-05-08T18:05:07.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.427+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 learned_addr learned my addr 172.21.15.50:0/3513214583 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.427+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d03500b30 con 0x559d033d2f20 2024-05-08T18:05:07.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.427+0000 7fe9317ca640 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d2f20 0x559d035005f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe92401afa0 tx=0x7fe924042f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.427+0000 7fe806ffd640 1 -- 172.21.15.50:0/3513214583 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe92406d070 con 0x559d033d2f20 2024-05-08T18:05:07.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.427+0000 7fe806ffd640 1 -- 172.21.15.50:0/3513214583 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe924032390 con 0x559d033d2f20 2024-05-08T18:05:07.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.427+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559d03500b30 con 0x559d033d2f20 2024-05-08T18:05:07.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.427+0000 7fe806ffd640 1 -- 172.21.15.50:0/3513214583 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9240328d0 con 0x559d033d2f20 2024-05-08T18:05:07.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.427+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d03500b30 con 0x559d033d2f20 2024-05-08T18:05:07.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.428+0000 7fe806ffd640 1 -- 172.21.15.50:0/3513214583 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fe924070070 con 0x559d033d2f20 2024-05-08T18:05:07.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.428+0000 7fe806ffd640 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7fc03bba0 0x7fe7fc03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.428+0000 7fe806ffd640 1 -- 172.21.15.50:0/3513214583 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fe924059030 con 0x559d033d2f20 2024-05-08T18:05:07.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.429+0000 7fe930fc9640 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7fc03bba0 0x7fe7fc03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.429+0000 7fe935b33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.429+0000 7fe935b33800 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559d0350bca0 0x559d0350e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.429+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d0350e8b0 con 0x559d0350bca0 2024-05-08T18:05:07.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.429+0000 7fe931fcb640 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559d0350bca0 0x559d0350e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.429+0000 7fe930fc9640 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7fc03bba0 0x7fe7fc03e060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x559d034dbf60 tx=0x7fe92804f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.429+0000 7fe931fcb640 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559d0350bca0 0x559d0350e160 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.429+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d0350bca0 2024-05-08T18:05:07.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe935b33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe935b33800 20 realm 2024-05-08T18:05:07.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe935b33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d0350ecc0 con 0x559d0350bca0 2024-05-08T18:05:07.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d0350bca0 2024-05-08T18:05:07.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe935b33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe935b33800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:07.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe935b33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d0350f590 con 0x559d0350bca0 2024-05-08T18:05:07.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d0350bca0 2024-05-08T18:05:07.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe935b33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe935b33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d0350f930 con 0x559d0350bca0 2024-05-08T18:05:07.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d0350bca0 2024-05-08T18:05:07.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe935b33800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:07.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe935b33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe935b33800 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559d03510420 0x559d03512810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.430+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d03512f70 con 0x559d03510420 2024-05-08T18:05:07.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.431+0000 7fe9317ca640 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559d03510420 0x559d03512810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.431+0000 7fe9317ca640 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559d03510420 0x559d03512810 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.431+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.431+0000 7fe935b33800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:07.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.431+0000 7fe935b33800 20 searching for the correct realm 2024-05-08T18:05:07.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.431+0000 7fe935b33800 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559d03515010 0x559d035174d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.431+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559d03517ba0 con 0x559d03515010 2024-05-08T18:05:07.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.431+0000 7fe930fc9640 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559d03515010 0x559d035174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.432+0000 7fe930fc9640 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559d03515010 0x559d035174d0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.432+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.432+0000 7fe930fc9640 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fe928074040 0x7fe928081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.432+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559d03517ba0 con 0x7fe928074040 2024-05-08T18:05:07.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.432+0000 7fe931fcb640 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fe928074040 0x7fe928081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.432+0000 7fe931fcb640 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fe928074040 0x7fe928081410 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.433+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92c00a4a0 con 0x7fe928074040 2024-05-08T18:05:07.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.433+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559d0350f930 con 0x559d03510420 2024-05-08T18:05:07.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.433+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.433+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559d0350f930 con 0x559d0350bca0 2024-05-08T18:05:07.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.433+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d0350bca0 2024-05-08T18:05:07.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.433+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559d0350f930 con 0x559d0350bca0 2024-05-08T18:05:07.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.433+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d0350bca0 2024-05-08T18:05:07.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.433+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559d0350f930 con 0x559d03510420 2024-05-08T18:05:07.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.434+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.434+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559d0350f930 con 0x559d03510420 2024-05-08T18:05:07.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.434+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.434+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559d0350f930 con 0x7fe928074040 2024-05-08T18:05:07.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.434+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92c00a4a0 con 0x7fe928074040 2024-05-08T18:05:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.434+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559d0350f930 con 0x559d03515010 2024-05-08T18:05:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.434+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.434+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559d0350f930 con 0x559d0350bca0 2024-05-08T18:05:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.435+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d0350bca0 2024-05-08T18:05:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.435+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559d0350f930 con 0x559d03510420 2024-05-08T18:05:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.435+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.435+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559d0350f930 con 0x559d0350bca0 2024-05-08T18:05:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.435+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d0350bca0 2024-05-08T18:05:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.435+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559d0350f930 con 0x7fe928074040 2024-05-08T18:05:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.435+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe92c00a4a0 con 0x7fe928074040 2024-05-08T18:05:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.435+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559d0350f930 con 0x559d03515010 2024-05-08T18:05:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.436+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.436+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x559d03515010 2024-05-08T18:05:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.436+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.436+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x559d03510420 2024-05-08T18:05:07.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.436+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.436+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x559d03515010 2024-05-08T18:05:07.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.436+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.436+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x559d03515010 2024-05-08T18:05:07.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.437+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.437+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x559d0350bca0 2024-05-08T18:05:07.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.437+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d0350bca0 2024-05-08T18:05:07.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.437+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x559d0350bca0 2024-05-08T18:05:07.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.437+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d0350bca0 2024-05-08T18:05:07.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.437+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x559d03510420 2024-05-08T18:05:07.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.437+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.437+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x559d03515010 2024-05-08T18:05:07.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.438+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.438+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x7fe928074040 2024-05-08T18:05:07.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.438+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92c00a4a0 con 0x7fe928074040 2024-05-08T18:05:07.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.438+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x559d0350bca0 2024-05-08T18:05:07.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.438+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d0350bca0 2024-05-08T18:05:07.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.438+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x559d03510420 2024-05-08T18:05:07.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.438+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.438+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x559d03515010 2024-05-08T18:05:07.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.439+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.439+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x559d03510420 2024-05-08T18:05:07.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.439+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.439+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x7fe928074040 2024-05-08T18:05:07.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.439+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92c00a4a0 con 0x7fe928074040 2024-05-08T18:05:07.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.439+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x559d03515010 2024-05-08T18:05:07.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.439+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.439+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x559d03510420 2024-05-08T18:05:07.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.440+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.440+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x559d0350bca0 2024-05-08T18:05:07.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.440+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d0350bca0 2024-05-08T18:05:07.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.440+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fe92808e040 con 0x7fe928074040 2024-05-08T18:05:07.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.440+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe92c00a4a0 con 0x7fe928074040 2024-05-08T18:05:07.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.440+0000 7fe935b33800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:07.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.440+0000 7fe935b33800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:07.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.440+0000 7fe935b33800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:07.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.440+0000 7fe935b33800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:07.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.440+0000 7fe935b33800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:07.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.440+0000 7fe935b33800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:07.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.440+0000 7fe935b33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.440+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d03513910 con 0x559d0350bca0 2024-05-08T18:05:07.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d0350bca0 2024-05-08T18:05:07.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d03513910 con 0x559d0350bca0 2024-05-08T18:05:07.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe92c00a4a0 con 0x559d0350bca0 2024-05-08T18:05:07.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:07.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d03513910 con 0x559d03515010 2024-05-08T18:05:07.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:07.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 20 zone default found 2024-05-08T18:05:07.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 4 Realm: () 2024-05-08T18:05:07.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:07.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:07.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:07.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 20 zonegroup default 2024-05-08T18:05:07.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d035141b0 con 0x559d03510420 2024-05-08T18:05:07.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.441+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d03519a00 con 0x559d03515010 2024-05-08T18:05:07.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.442+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.442+0000 7fe935b33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.442+0000 7fe935b33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.442+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d03519da0 con 0x559d03515010 2024-05-08T18:05:07.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.442+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.442+0000 7fe935b33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.442+0000 7fe935b33800 20 started sync module instance, tier type = 2024-05-08T18:05:07.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.442+0000 7fe935b33800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:07.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.442+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d03518ea0 con 0x7fe928074040 2024-05-08T18:05:07.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.443+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92c00a4a0 con 0x7fe928074040 2024-05-08T18:05:07.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.443+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94132853904368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d03518ea0 con 0x7fe928074040 2024-05-08T18:05:07.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.443+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d0351a7a0 con 0x559d0350bca0 2024-05-08T18:05:07.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.444+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94132853904368] v27'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92c040040 con 0x7fe928074040 2024-05-08T18:05:07.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.444+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92c040040 con 0x559d0350bca0 2024-05-08T18:05:07.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.444+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94132853908144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d03518ea0 con 0x559d0350bca0 2024-05-08T18:05:07.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.444+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d0351b1f0 con 0x559d03515010 2024-05-08T18:05:07.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.445+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94132853908144] v27'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92c040040 con 0x559d0350bca0 2024-05-08T18:05:07.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.446+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.446+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94132853901808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d03518e10 con 0x559d03515010 2024-05-08T18:05:07.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.446+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d0351c0b0 con 0x7fe928074040 2024-05-08T18:05:07.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.446+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94132853901808] v27'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.447+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92c040040 con 0x7fe928074040 2024-05-08T18:05:07.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.447+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94132853914624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d0351c4d0 con 0x7fe928074040 2024-05-08T18:05:07.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.447+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d0351ce90 con 0x559d0350bca0 2024-05-08T18:05:07.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.448+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94132853914624] v27'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92c040040 con 0x7fe928074040 2024-05-08T18:05:07.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.448+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92c040040 con 0x559d0350bca0 2024-05-08T18:05:07.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.448+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94132853916160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d0351ce90 con 0x559d0350bca0 2024-05-08T18:05:07.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.448+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d0351d780 con 0x559d03510420 2024-05-08T18:05:07.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.449+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94132853916160] v27'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92c040040 con 0x559d0350bca0 2024-05-08T18:05:07.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.449+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.449+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94132853918656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d0351db70 con 0x559d03510420 2024-05-08T18:05:07.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.449+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d0351e530 con 0x559d03510420 2024-05-08T18:05:07.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.450+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.450+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94132853926176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d0351f1f0 con 0x559d03510420 2024-05-08T18:05:07.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.450+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d0351fc10 con 0x559d03510420 2024-05-08T18:05:07.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.450+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94132853918656] v27'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.451+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94132853926176] v27'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.451+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.451+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94132853927232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d0351d020 con 0x559d03510420 2024-05-08T18:05:07.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.451+0000 7fe935b33800 20 add_watcher() i=0 2024-05-08T18:05:07.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.451+0000 7fe935b33800 20 add_watcher() i=1 2024-05-08T18:05:07.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.451+0000 7fe935b33800 20 add_watcher() i=2 2024-05-08T18:05:07.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.451+0000 7fe935b33800 20 add_watcher() i=3 2024-05-08T18:05:07.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.451+0000 7fe935b33800 20 add_watcher() i=4 2024-05-08T18:05:07.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.451+0000 7fe935b33800 20 add_watcher() i=5 2024-05-08T18:05:07.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.451+0000 7fe935b33800 20 add_watcher() i=6 2024-05-08T18:05:07.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.452+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94132853927232] v27'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.452+0000 7fe935b33800 20 add_watcher() i=7 2024-05-08T18:05:07.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.452+0000 7fe935b33800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:07.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.452+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d03522eb0 con 0x559d0350bca0 2024-05-08T18:05:07.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.452+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fe92c040040 con 0x559d0350bca0 2024-05-08T18:05:07.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.452+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94132853929728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d035203d0 con 0x559d0350bca0 2024-05-08T18:05:07.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.453+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94132853929728] v27'98 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe92c040040 con 0x559d0350bca0 2024-05-08T18:05:07.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.453+0000 7fe7e7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:07.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.453+0000 7fe935b33800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:07.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.453+0000 7fe935b33800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:07.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.453+0000 7fe935b33800 5 note: GC not initialized 2024-05-08T18:05:07.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.454+0000 7fe7e6fe5640 20 reqs_thread_entry: start 2024-05-08T18:05:07.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.454+0000 7fe935b33800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:07.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.454+0000 7fe935b33800 20 Filter name: none 2024-05-08T18:05:07.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.454+0000 7fe7e4fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:07.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.454+0000 7fe935b33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.454+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d03555ed0 con 0x559d0350bca0 2024-05-08T18:05:07.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.454+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fe92c040040 con 0x559d0350bca0 2024-05-08T18:05:07.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.454+0000 7fe935b33800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:07.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.454+0000 7fe935b33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.454+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d035568c0 con 0x559d0350bca0 2024-05-08T18:05:07.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.455+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fe92c040040 con 0x559d0350bca0 2024-05-08T18:05:07.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.455+0000 7fe935b33800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:07.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.455+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d035578d0 con 0x7fe928074040 2024-05-08T18:05:07.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.455+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(64 foo.buckets [call out=183b] v0'0 uv16 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7fe92c040040 con 0x7fe928074040 2024-05-08T18:05:07.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.455+0000 7fe935b33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.455+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:65 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d035551a0 con 0x7fe928074040 2024-05-08T18:05:07.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.455+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(65 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fe92c040040 con 0x7fe928074040 2024-05-08T18:05:07.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.455+0000 7fe935b33800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:07.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.455+0000 7fe935b33800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.455+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:66 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d035578b0 con 0x559d03510420 2024-05-08T18:05:07.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.456+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(66 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.456+0000 7fe935b33800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:07.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.456+0000 7fe935b33800 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-08T18:05:07.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.456+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:67 4.37 4:eecadb7d:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d035591a0 con 0x559d03510420 2024-05-08T18:05:07.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.456+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 4.22 4:46cfe895:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d0355bca0 con 0x559d0350bca0 2024-05-08T18:05:07.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.456+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 4.0 4:018ce329:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d0355e600 con 0x559d03515010 2024-05-08T18:05:07.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.456+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 4.12 4:4afe9c7f:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d03560fe0 con 0x559d0350bca0 2024-05-08T18:05:07.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.456+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 4.d 4:b2c4ac36:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d035639a0 con 0x559d03515010 2024-05-08T18:05:07.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.456+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 4.33 4:cc412025:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d03566210 con 0x559d03510420 2024-05-08T18:05:07.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.456+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:73 4.3 4:c17a4869:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d03568b00 con 0x559d0350bca0 2024-05-08T18:05:07.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.456+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:74 4.6 4:61ff1ea9:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d0356b920 con 0x559d03515010 2024-05-08T18:05:07.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.457+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 4.1c 4:3ae7d9ef:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d03568b00 con 0x559d03510420 2024-05-08T18:05:07.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.457+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe92c040040 con 0x559d0350bca0 2024-05-08T18:05:07.473 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.457+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(69 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.457+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(67 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.457+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(70 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe92c040040 con 0x559d0350bca0 2024-05-08T18:05:07.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.457+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(71 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.457+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(74 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.457+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 4.f 4:f2293886:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d035634b0 con 0x559d0350bca0 2024-05-08T18:05:07.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.457+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(72 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.5 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.457+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 4.21 4:87ec338a:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559d0355b650 con 0x559d03510420 2024-05-08T18:05:07.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.457+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(73 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.6 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe92c040040 con 0x559d0350bca0 2024-05-08T18:05:07.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.457+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(75 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.457+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(76 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fe92c040040 con 0x559d0350bca0 2024-05-08T18:05:07.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.457+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(77 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.458+0000 7fe7e7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:07.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.458+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:78 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94132853929728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d03568b00 con 0x559d0350bca0 2024-05-08T18:05:07.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.459+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(78 data_loggenerations_metadata [watch unwatch cookie 94132853929728] v27'99 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe92c040040 con 0x559d0350bca0 2024-05-08T18:05:07.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.459+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:79 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94132853904368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d03568b00 con 0x7fe928074040 2024-05-08T18:05:07.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.460+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(79 notify.0 [watch unwatch cookie 94132853904368] v27'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92c040040 con 0x7fe928074040 2024-05-08T18:05:07.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.460+0000 7fe935b33800 20 remove_watcher() i=0 2024-05-08T18:05:07.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.460+0000 7fe935b33800 2 removed watcher, disabling cache 2024-05-08T18:05:07.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.460+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94132853908144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d03568b00 con 0x559d0350bca0 2024-05-08T18:05:07.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.461+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(80 notify.1 [watch unwatch cookie 94132853908144] v27'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92c054040 con 0x559d0350bca0 2024-05-08T18:05:07.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.461+0000 7fe935b33800 20 remove_watcher() i=1 2024-05-08T18:05:07.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.461+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:81 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94132853901808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d03568b00 con 0x559d03515010 2024-05-08T18:05:07.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.462+0000 7fe930fc9640 1 -- 172.21.15.50:0/3513214583 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(81 notify.2 [watch unwatch cookie 94132853901808] v27'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92807f070 con 0x559d03515010 2024-05-08T18:05:07.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.462+0000 7fe935b33800 20 remove_watcher() i=2 2024-05-08T18:05:07.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.462+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:82 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94132853914624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d03568b00 con 0x7fe928074040 2024-05-08T18:05:07.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.463+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(82 notify.3 [watch unwatch cookie 94132853914624] v27'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92c054040 con 0x7fe928074040 2024-05-08T18:05:07.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.463+0000 7fe935b33800 20 remove_watcher() i=3 2024-05-08T18:05:07.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.463+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:83 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94132853916160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d03568b00 con 0x559d0350bca0 2024-05-08T18:05:07.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.464+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(83 notify.4 [watch unwatch cookie 94132853916160] v27'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92c054040 con 0x559d0350bca0 2024-05-08T18:05:07.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.464+0000 7fe935b33800 20 remove_watcher() i=4 2024-05-08T18:05:07.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.464+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:84 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94132853918656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d03568b00 con 0x559d03510420 2024-05-08T18:05:07.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.465+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(84 notify.5 [watch unwatch cookie 94132853918656] v27'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.465+0000 7fe935b33800 20 remove_watcher() i=5 2024-05-08T18:05:07.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.465+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:85 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94132853926176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d03568b00 con 0x559d03510420 2024-05-08T18:05:07.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.466+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(85 notify.6 [watch unwatch cookie 94132853926176] v27'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.466+0000 7fe935b33800 20 remove_watcher() i=6 2024-05-08T18:05:07.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.466+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:86 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94132853927232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559d03568b00 con 0x559d03510420 2024-05-08T18:05:07.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.467+0000 7fe9317ca640 1 -- 172.21.15.50:0/3513214583 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(86 notify.7 [watch unwatch cookie 94132853927232] v27'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe92406f030 con 0x559d03510420 2024-05-08T18:05:07.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.467+0000 7fe935b33800 20 remove_watcher() i=7 2024-05-08T18:05:07.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.467+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559d03510420 msgr2=0x559d03512810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.467+0000 7fe935b33800 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559d03510420 0x559d03512810 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.467+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559d0350bca0 msgr2=0x559d0350e160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.467+0000 7fe935b33800 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559d0350bca0 0x559d0350e160 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.467+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fe928074040 msgr2=0x7fe928081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.467+0000 7fe935b33800 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fe928074040 0x7fe928081410 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.467+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559d03515010 msgr2=0x559d035174d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.467+0000 7fe935b33800 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559d03515010 0x559d035174d0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7fc03bba0 msgr2=0x7fe7fc03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe7fc03bba0 0x7fe7fc03e060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x559d034dbf60 tx=0x7fe92804f000 comp rx=0 tx=0).stop 2024-05-08T18:05:07.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d2f20 msgr2=0x559d035005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe931fcb640 1 -- 172.21.15.50:0/3513214583 reap_dead start 2024-05-08T18:05:07.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d2f20 0x559d035005f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe92401afa0 tx=0x7fe924042f40 comp rx=0 tx=0).stop 2024-05-08T18:05:07.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 shutdown_connections 2024-05-08T18:05:07.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 --2- 172.21.15.50:0/3513214583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d2f20 0x559d035005f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 >> 172.21.15.50:0/3513214583 conn(0x559d034f32c0 msgr2=0x559d034f0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 shutdown_connections 2024-05-08T18:05:07.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 -- 172.21.15.50:0/3513214583 wait complete. 2024-05-08T18:05:07.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 -- 172.21.15.50:0/2721256399 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe80803bb00 msgr2=0x7fe80803dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 --2- 172.21.15.50:0/2721256399 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe80803bb00 0x7fe80803dfc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x559d034cfde0 tx=0x7fe928005ac0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 -- 172.21.15.50:0/2721256399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d034e9ed0 msgr2=0x559d034ea2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 --2- 172.21.15.50:0/2721256399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d034e9ed0 0x559d034ea2f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe924017b00 tx=0x7fe924017f40 comp rx=0 tx=0).stop 2024-05-08T18:05:07.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 -- 172.21.15.50:0/2721256399 shutdown_connections 2024-05-08T18:05:07.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 --2- 172.21.15.50:0/2721256399 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe80803bb00 0x7fe80803dfc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 --2- 172.21.15.50:0/2721256399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d034e9ed0 0x559d034ea2f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 -- 172.21.15.50:0/2721256399 >> 172.21.15.50:0/2721256399 conn(0x559d034ddf70 msgr2=0x559d034de370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 -- 172.21.15.50:0/2721256399 shutdown_connections 2024-05-08T18:05:07.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.468+0000 7fe935b33800 1 -- 172.21.15.50:0/2721256399 wait complete. 2024-05-08T18:05:07.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.469+0000 7fe935b33800 1 -- 172.21.15.50:0/4289333904 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559d034d5180 msgr2=0x559d034d7570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.469+0000 7fe935b33800 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559d034d5180 0x559d034d7570 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.469+0000 7fe935b33800 1 -- 172.21.15.50:0/4289333904 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559d033d6b60 msgr2=0x559d034d28f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.469+0000 7fe935b33800 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559d033d6b60 0x559d034d28f0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.469+0000 7fe935b33800 1 -- 172.21.15.50:0/4289333904 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559d034d9070 msgr2=0x559d034db460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.469+0000 7fe935b33800 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559d034d9070 0x559d034db460 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.469+0000 7fe935b33800 1 -- 172.21.15.50:0/4289333904 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe81803bba0 msgr2=0x7fe81803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.469+0000 7fe935b33800 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe81803bba0 0x7fe81803e060 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe928054000 tx=0x7fe92804e000 comp rx=0 tx=0).stop 2024-05-08T18:05:07.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.469+0000 7fe935b33800 1 -- 172.21.15.50:0/4289333904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d0940 msgr2=0x559d034cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.469+0000 7fe935b33800 1 --2- 172.21.15.50:0/4289333904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559d033d0940 0x559d034cb700 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe924000f60 tx=0x7fe924000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.469+0000 7fe931fcb640 1 -- 172.21.15.50:0/4289333904 reap_dead start 2024-05-08T18:05:07.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.469+0000 7fe935b33800 1 -- 172.21.15.50:0/4289333904 shutdown_connections 2024-05-08T18:05:07.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.469+0000 7fe935b33800 1 -- 172.21.15.50:0/4289333904 >> 172.21.15.50:0/4289333904 conn(0x559d033cc2d0 msgr2=0x559d034bdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.469+0000 7fe935b33800 1 -- 172.21.15.50:0/4289333904 shutdown_connections 2024-05-08T18:05:07.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.469+0000 7fe935b33800 1 -- 172.21.15.50:0/4289333904 wait complete. 2024-05-08T18:05:07.480 INFO:teuthology.orchestra.run.smithi050.stdout:[{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"36133b23-abaa-446a-bf67-ceb1d1559ceb","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1","marker":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1","index_type":"Normal","index_generation":0,"num_shards":11,"object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-05-08T18:05:06.708772Z","creation_time":"2024-05-08T18:05:06.706377Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}}] 2024-05-08T18:05:07.480 DEBUG:tasks.util.rgw: json result: [{'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': '36133b23-abaa-446a-bf67-ceb1d1559ceb', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1', 'marker': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-05-08T18:05:06.708772Z', 'creation_time': '2024-05-08T18:05:06.706377Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}] 2024-05-08T18:05:07.487 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-05-08T18:05:07.487 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-05-08T18:05:07.487 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-05-08T18:05:07.488 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-05-08T18:05:07.533 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:07.533 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:07.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.536+0000 7ff31463f800 1 Processor -- start 2024-05-08T18:05:07.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.536+0000 7ff31463f800 1 -- start start 2024-05-08T18:05:07.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.536+0000 7ff31463f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ba6a9afaf0 0x55ba6a9b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.536+0000 7ff31463f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55ba6a922ff0 con 0x55ba6a9adfb0 2024-05-08T18:05:07.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.536+0000 7ff31463f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55ba6a84fe20 con 0x55ba6a9afaf0 2024-05-08T18:05:07.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.536+0000 7ff311936640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ba6a9afaf0 0x55ba6a9b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.536+0000 7ff312137640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55ba6a9adfb0 0x55ba6a9ae3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43130/0 (socket says 172.21.15.50:43130) 2024-05-08T18:05:07.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.536+0000 7ff311936640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ba6a9afaf0 0x55ba6a9b85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37556/0 (socket says 172.21.15.50:37556) 2024-05-08T18:05:07.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.536+0000 7ff312137640 1 -- 172.21.15.50:0/660349575 learned_addr learned my addr 172.21.15.50:0/660349575 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:07.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.536+0000 7ff311936640 1 -- 172.21.15.50:0/660349575 >> v1:172.21.15.46:6789/0 conn(0x55ba6a9adfb0 legacy=0x55ba6a9ae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.535 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.536+0000 7ff311936640 1 -- 172.21.15.50:0/660349575 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba6a9231c0 con 0x55ba6a9afaf0 2024-05-08T18:05:07.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.538+0000 7ff311936640 1 --2- 172.21.15.50:0/660349575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9afaf0 0x55ba6a9b85f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff30800a470 tx=0x7ff30801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75ac87814d6b7aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.538+0000 7ff311135640 1 -- 172.21.15.50:0/660349575 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff308057070 con 0x55ba6a9afaf0 2024-05-08T18:05:07.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.538+0000 7ff311135640 1 -- 172.21.15.50:0/660349575 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff308052570 con 0x55ba6a9afaf0 2024-05-08T18:05:07.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.538+0000 7ff311135640 1 -- 172.21.15.50:0/660349575 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff308052870 con 0x55ba6a9afaf0 2024-05-08T18:05:07.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.539+0000 7ff31463f800 1 -- 172.21.15.50:0/660349575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9afaf0 msgr2=0x55ba6a9b85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.539+0000 7ff31463f800 1 --2- 172.21.15.50:0/660349575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9afaf0 0x55ba6a9b85f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff30800a470 tx=0x7ff30801ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:07.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.539+0000 7ff31463f800 1 -- 172.21.15.50:0/660349575 shutdown_connections 2024-05-08T18:05:07.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.539+0000 7ff31463f800 1 --2- 172.21.15.50:0/660349575 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9afaf0 0x55ba6a9b85f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.539+0000 7ff31463f800 1 -- 172.21.15.50:0/660349575 >> 172.21.15.50:0/660349575 conn(0x55ba6a9a9ad0 msgr2=0x55ba6a9abf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.539+0000 7ff31463f800 1 -- 172.21.15.50:0/660349575 shutdown_connections 2024-05-08T18:05:07.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.539+0000 7ff31463f800 1 -- 172.21.15.50:0/660349575 wait complete. 2024-05-08T18:05:07.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.555+0000 7ff31463f800 1 Processor -- start 2024-05-08T18:05:07.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.555+0000 7ff31463f800 1 -- start start 2024-05-08T18:05:07.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.555+0000 7ff31463f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9adfb0 0x55ba6aaa2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.555+0000 7ff31463f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ba6a8faa10 con 0x55ba6a9adfb0 2024-05-08T18:05:07.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.555+0000 7ff312137640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9adfb0 0x55ba6aaa2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.555+0000 7ff312137640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9adfb0 0x55ba6aaa2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37570/0 (socket says 172.21.15.50:37570) 2024-05-08T18:05:07.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.555+0000 7ff312137640 1 -- 172.21.15.50:0/3963387932 learned_addr learned my addr 172.21.15.50:0/3963387932 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.555+0000 7ff312137640 1 -- 172.21.15.50:0/3963387932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba6aaa3320 con 0x55ba6a9adfb0 2024-05-08T18:05:07.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.556+0000 7ff312137640 1 --2- 172.21.15.50:0/3963387932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9adfb0 0x55ba6aaa2d50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff304001e10 tx=0x7ff30400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cb7034f33a3656 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.556+0000 7ff200608640 1 -- 172.21.15.50:0/3963387932 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff304012020 con 0x55ba6a9adfb0 2024-05-08T18:05:07.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.556+0000 7ff200608640 1 -- 172.21.15.50:0/3963387932 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30400e5a0 con 0x55ba6a9adfb0 2024-05-08T18:05:07.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.556+0000 7ff200608640 1 -- 172.21.15.50:0/3963387932 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff30400e8a0 con 0x55ba6a9adfb0 2024-05-08T18:05:07.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.556+0000 7ff31463f800 1 -- 172.21.15.50:0/3963387932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9adfb0 msgr2=0x55ba6aaa2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.556+0000 7ff31463f800 1 --2- 172.21.15.50:0/3963387932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9adfb0 0x55ba6aaa2d50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff304001e10 tx=0x7ff30400b780 comp rx=0 tx=0).stop 2024-05-08T18:05:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.556+0000 7ff31463f800 1 -- 172.21.15.50:0/3963387932 shutdown_connections 2024-05-08T18:05:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.556+0000 7ff31463f800 1 --2- 172.21.15.50:0/3963387932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9adfb0 0x55ba6aaa2d50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.556+0000 7ff31463f800 1 -- 172.21.15.50:0/3963387932 >> 172.21.15.50:0/3963387932 conn(0x55ba6a9a9ad0 msgr2=0x55ba6aa9e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.557+0000 7ff31463f800 1 -- 172.21.15.50:0/3963387932 shutdown_connections 2024-05-08T18:05:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.557+0000 7ff31463f800 1 -- 172.21.15.50:0/3963387932 wait complete. 2024-05-08T18:05:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.557+0000 7ff31463f800 1 Processor -- start 2024-05-08T18:05:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.557+0000 7ff31463f800 1 -- start start 2024-05-08T18:05:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.557+0000 7ff31463f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9adfb0 0x55ba6aaaa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.557+0000 7ff31463f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ba6a9b8e00 con 0x55ba6a9adfb0 2024-05-08T18:05:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.557+0000 7ff312137640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9adfb0 0x55ba6aaaa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.557+0000 7ff312137640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9adfb0 0x55ba6aaaa8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37582/0 (socket says 172.21.15.50:37582) 2024-05-08T18:05:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.557+0000 7ff312137640 1 -- 172.21.15.50:0/2835419633 learned_addr learned my addr 172.21.15.50:0/2835419633 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.558+0000 7ff312137640 1 -- 172.21.15.50:0/2835419633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba6aaaae30 con 0x55ba6a9adfb0 2024-05-08T18:05:07.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.558+0000 7ff312137640 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9adfb0 0x55ba6aaaa8f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55ba6aaa3ae0 tx=0x7ff304000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.558+0000 7ff1fee05640 1 -- 172.21.15.50:0/2835419633 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff304012030 con 0x55ba6a9adfb0 2024-05-08T18:05:07.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.558+0000 7ff1fee05640 1 -- 172.21.15.50:0/2835419633 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff304001470 con 0x55ba6a9adfb0 2024-05-08T18:05:07.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.558+0000 7ff31463f800 1 -- 172.21.15.50:0/2835419633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ba6aaab0c0 con 0x55ba6a9adfb0 2024-05-08T18:05:07.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.558+0000 7ff1fee05640 1 -- 172.21.15.50:0/2835419633 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff30401a7a0 con 0x55ba6a9adfb0 2024-05-08T18:05:07.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.558+0000 7ff31463f800 1 -- 172.21.15.50:0/2835419633 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba6aaadd70 con 0x55ba6a9adfb0 2024-05-08T18:05:07.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.559+0000 7ff1fee05640 1 -- 172.21.15.50:0/2835419633 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff304015020 con 0x55ba6a9adfb0 2024-05-08T18:05:07.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.559+0000 7ff1fee05640 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff1f803bba0 0x7ff1f803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.559+0000 7ff1fee05640 1 -- 172.21.15.50:0/2835419633 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7ff304020030 con 0x55ba6a9adfb0 2024-05-08T18:05:07.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.559+0000 7ff311936640 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff1f803bba0 0x7ff1f803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.559+0000 7ff31463f800 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ba6a9afaf0 0x55ba6aab1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.559+0000 7ff31463f800 1 -- 172.21.15.50:0/2835419633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aab2070 con 0x55ba6a9afaf0 2024-05-08T18:05:07.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.560+0000 7ff312938640 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ba6a9afaf0 0x55ba6aab1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.560+0000 7ff311936640 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff1f803bba0 0x7ff1f803e060 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff308054000 tx=0x7ff30804e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.560+0000 7ff312938640 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ba6a9afaf0 0x55ba6aab1b30 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.560+0000 7ff312938640 1 -- 172.21.15.50:0/2835419633 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6a9afaf0 2024-05-08T18:05:07.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.560+0000 7ff31463f800 1 -- 172.21.15.50:0/2835419633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaaf0e0 con 0x55ba6a9afaf0 2024-05-08T18:05:07.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.561+0000 7ff312938640 1 -- 172.21.15.50:0/2835419633 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6a9afaf0 2024-05-08T18:05:07.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.561+0000 7ff31463f800 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ba6aab4390 0x55ba6aab6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.561+0000 7ff31463f800 1 -- 172.21.15.50:0/2835419633 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aab6e70 con 0x55ba6aab4390 2024-05-08T18:05:07.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.561+0000 7ff312137640 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ba6aab4390 0x55ba6aab6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.561+0000 7ff312137640 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ba6aab4390 0x55ba6aab6780 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.561+0000 7ff312137640 1 -- 172.21.15.50:0/2835419633 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff3040430c0 con 0x55ba6aab4390 2024-05-08T18:05:07.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.561+0000 7ff31463f800 1 -- 172.21.15.50:0/2835419633 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aab3230 con 0x55ba6a9afaf0 2024-05-08T18:05:07.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.562+0000 7ff312938640 1 -- 172.21.15.50:0/2835419633 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6a9afaf0 2024-05-08T18:05:07.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.562+0000 7ff31463f800 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ba6aab82d0 0x55ba6aaba6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.562+0000 7ff31463f800 1 -- 172.21.15.50:0/2835419633 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aabadc0 con 0x55ba6aab82d0 2024-05-08T18:05:07.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.562+0000 7ff311936640 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ba6aab82d0 0x55ba6aaba6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.562+0000 7ff311936640 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ba6aab82d0 0x55ba6aaba6c0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.562+0000 7ff311936640 1 -- 172.21.15.50:0/2835419633 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff308052a40 con 0x55ba6aab82d0 2024-05-08T18:05:07.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.563+0000 7ff31463f800 1 Processor -- start 2024-05-08T18:05:07.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.563+0000 7ff31463f800 1 -- start start 2024-05-08T18:05:07.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.563+0000 7ff31463f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aac9070 0x55ba6aac9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.563+0000 7ff31463f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ba6a8a1330 con 0x55ba6aac9070 2024-05-08T18:05:07.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.563+0000 7ff312137640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aac9070 0x55ba6aac9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.563+0000 7ff312137640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aac9070 0x55ba6aac9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37584/0 (socket says 172.21.15.50:37584) 2024-05-08T18:05:07.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.563+0000 7ff312137640 1 -- 172.21.15.50:0/3902956914 learned_addr learned my addr 172.21.15.50:0/3902956914 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.563+0000 7ff312137640 1 -- 172.21.15.50:0/3902956914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba6aac99d0 con 0x55ba6aac9070 2024-05-08T18:05:07.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.564+0000 7ff312137640 1 --2- 172.21.15.50:0/3902956914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aac9070 0x55ba6aac9490 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff304050f60 tx=0x7ff30403fef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.564+0000 7ff1f7fff640 1 -- 172.21.15.50:0/3902956914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff304052590 con 0x55ba6aac9070 2024-05-08T18:05:07.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.564+0000 7ff1f7fff640 1 -- 172.21.15.50:0/3902956914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff304048070 con 0x55ba6aac9070 2024-05-08T18:05:07.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.564+0000 7ff31463f800 1 -- 172.21.15.50:0/3902956914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ba6aaca440 con 0x55ba6aac9070 2024-05-08T18:05:07.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.564+0000 7ff1f7fff640 1 -- 172.21.15.50:0/3902956914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3040528c0 con 0x55ba6aac9070 2024-05-08T18:05:07.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.564+0000 7ff31463f800 1 -- 172.21.15.50:0/3902956914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba6aaca940 con 0x55ba6aac9070 2024-05-08T18:05:07.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.564+0000 7ff1f7fff640 1 -- 172.21.15.50:0/3902956914 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff30400c790 con 0x55ba6aac9070 2024-05-08T18:05:07.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.565+0000 7ff1f7fff640 1 --2- 172.21.15.50:0/3902956914 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff1ec03bb50 0x7ff1ec03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.565+0000 7ff1f7fff640 1 -- 172.21.15.50:0/3902956914 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7ff3040030d0 con 0x55ba6aac9070 2024-05-08T18:05:07.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.565+0000 7ff311936640 1 --2- 172.21.15.50:0/3902956914 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff1ec03bb50 0x7ff1ec03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.565+0000 7ff31463f800 1 Processor -- start 2024-05-08T18:05:07.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.565+0000 7ff31463f800 1 -- start start 2024-05-08T18:05:07.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.565+0000 7ff311936640 1 --2- 172.21.15.50:0/3902956914 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff1ec03bb50 0x7ff1ec03e010 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55ba6aab2940 tx=0x7ff308005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.565+0000 7ff31463f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aad62c0 0x55ba6aad66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.565+0000 7ff31463f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ba6aaa36a0 con 0x55ba6aad62c0 2024-05-08T18:05:07.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.565+0000 7ff312137640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aad62c0 0x55ba6aad66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.565+0000 7ff312137640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aad62c0 0x55ba6aad66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37590/0 (socket says 172.21.15.50:37590) 2024-05-08T18:05:07.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.565+0000 7ff312137640 1 -- 172.21.15.50:0/916153972 learned_addr learned my addr 172.21.15.50:0/916153972 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.566+0000 7ff312137640 1 -- 172.21.15.50:0/916153972 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba6aad6cb0 con 0x55ba6aad62c0 2024-05-08T18:05:07.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.566+0000 7ff312137640 1 --2- 172.21.15.50:0/916153972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aad62c0 0x55ba6aad66e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff30401afd0 tx=0x7ff304024550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929c0e4cbc6feb30 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.566+0000 7ff1f5ffb640 1 -- 172.21.15.50:0/916153972 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3040168d0 con 0x55ba6aad62c0 2024-05-08T18:05:07.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.566+0000 7ff1f5ffb640 1 -- 172.21.15.50:0/916153972 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff30404bcd0 con 0x55ba6aad62c0 2024-05-08T18:05:07.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.566+0000 7ff1f5ffb640 1 -- 172.21.15.50:0/916153972 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff30400d5f0 con 0x55ba6aad62c0 2024-05-08T18:05:07.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.566+0000 7ff31463f800 1 -- 172.21.15.50:0/916153972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aad62c0 msgr2=0x55ba6aad66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.566+0000 7ff31463f800 1 --2- 172.21.15.50:0/916153972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aad62c0 0x55ba6aad66e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff30401afd0 tx=0x7ff304024550 comp rx=0 tx=0).stop 2024-05-08T18:05:07.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.566+0000 7ff31463f800 1 -- 172.21.15.50:0/916153972 shutdown_connections 2024-05-08T18:05:07.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.567+0000 7ff31463f800 1 --2- 172.21.15.50:0/916153972 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aad62c0 0x55ba6aad66e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.567+0000 7ff31463f800 1 -- 172.21.15.50:0/916153972 >> 172.21.15.50:0/916153972 conn(0x55ba6aad2590 msgr2=0x55ba6aad05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.567+0000 7ff31463f800 1 -- 172.21.15.50:0/916153972 shutdown_connections 2024-05-08T18:05:07.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.567+0000 7ff31463f800 1 -- 172.21.15.50:0/916153972 wait complete. 2024-05-08T18:05:07.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.567+0000 7ff31463f800 1 Processor -- start 2024-05-08T18:05:07.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.567+0000 7ff31463f800 1 -- start start 2024-05-08T18:05:07.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.567+0000 7ff31463f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aad62c0 0x55ba6aadd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.567+0000 7ff31463f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ba6aad7380 con 0x55ba6aad62c0 2024-05-08T18:05:07.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.567+0000 7ff312137640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aad62c0 0x55ba6aadd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.567+0000 7ff312137640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aad62c0 0x55ba6aadd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37602/0 (socket says 172.21.15.50:37602) 2024-05-08T18:05:07.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.567+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 learned_addr learned my addr 172.21.15.50:0/1416668156 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.568+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba6aaddac0 con 0x55ba6aad62c0 2024-05-08T18:05:07.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.568+0000 7ff312137640 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aad62c0 0x55ba6aadd580 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff30401afd0 tx=0x7ff304039ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.568+0000 7ff1e3fff640 1 -- 172.21.15.50:0/1416668156 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff30406d020 con 0x55ba6aad62c0 2024-05-08T18:05:07.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.568+0000 7ff1e3fff640 1 -- 172.21.15.50:0/1416668156 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff304064040 con 0x55ba6aad62c0 2024-05-08T18:05:07.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.568+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ba6aaddac0 con 0x55ba6aad62c0 2024-05-08T18:05:07.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.568+0000 7ff1e3fff640 1 -- 172.21.15.50:0/1416668156 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff30404ffd0 con 0x55ba6aad62c0 2024-05-08T18:05:07.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.568+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba6aaddac0 con 0x55ba6aad62c0 2024-05-08T18:05:07.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.569+0000 7ff1e3fff640 1 -- 172.21.15.50:0/1416668156 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff304070020 con 0x55ba6aad62c0 2024-05-08T18:05:07.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.569+0000 7ff1e3fff640 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff1dc03bb00 0x7ff1dc03dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.569+0000 7ff1e3fff640 1 -- 172.21.15.50:0/1416668156 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7ff304073c60 con 0x55ba6aad62c0 2024-05-08T18:05:07.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.569+0000 7ff311936640 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff1dc03bb00 0x7ff1dc03dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.569+0000 7ff31463f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.570+0000 7ff31463f800 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ba6aae8d90 0x55ba6aaeb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.570+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaeb9a0 con 0x55ba6aae8d90 2024-05-08T18:05:07.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.570+0000 7ff312938640 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ba6aae8d90 0x55ba6aaeb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.570+0000 7ff311936640 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff1dc03bb00 0x7ff1dc03dfc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff308052fd0 tx=0x7ff30804f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.570+0000 7ff312938640 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ba6aae8d90 0x55ba6aaeb250 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.570+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6aae8d90 2024-05-08T18:05:07.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.570+0000 7ff31463f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.570+0000 7ff31463f800 20 realm 2024-05-08T18:05:07.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.570+0000 7ff31463f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.570+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaebdb0 con 0x55ba6aae8d90 2024-05-08T18:05:07.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.571+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6aae8d90 2024-05-08T18:05:07.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.571+0000 7ff31463f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.571+0000 7ff31463f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:07.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.571+0000 7ff31463f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.571+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaec650 con 0x55ba6aae8d90 2024-05-08T18:05:07.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.571+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6aae8d90 2024-05-08T18:05:07.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.571+0000 7ff31463f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.571+0000 7ff31463f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.571+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaec9f0 con 0x55ba6aae8d90 2024-05-08T18:05:07.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.571+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6aae8d90 2024-05-08T18:05:07.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.571+0000 7ff31463f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:07.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.571+0000 7ff31463f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.571+0000 7ff31463f800 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ba6aaed750 0x55ba6aaefb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.571+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf00c0 con 0x55ba6aaed750 2024-05-08T18:05:07.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.571+0000 7ff312137640 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ba6aaed750 0x55ba6aaefb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.572+0000 7ff312137640 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ba6aaed750 0x55ba6aaefb40 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.572+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.572+0000 7ff31463f800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.572+0000 7ff31463f800 20 searching for the correct realm 2024-05-08T18:05:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.572+0000 7ff31463f800 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ba6aaf20f0 0x55ba6aaf45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.572+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf4c80 con 0x55ba6aaf20f0 2024-05-08T18:05:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.572+0000 7ff311936640 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ba6aaf20f0 0x55ba6aaf45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.573+0000 7ff311936640 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ba6aaf20f0 0x55ba6aaf45b0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.573+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.573+0000 7ff311936640 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff308076040 0x7ff308083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.573+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf4c80 con 0x7ff308076040 2024-05-08T18:05:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.573+0000 7ff312938640 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff308076040 0x7ff308083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.573+0000 7ff312938640 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff308076040 0x7ff308083410 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.573+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30c00a4a0 con 0x7ff308076040 2024-05-08T18:05:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.573+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaec9f0 con 0x55ba6aaed750 2024-05-08T18:05:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.574+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.574+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaec9f0 con 0x55ba6aae8d90 2024-05-08T18:05:07.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.574+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6aae8d90 2024-05-08T18:05:07.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.574+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaec9f0 con 0x55ba6aae8d90 2024-05-08T18:05:07.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.574+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6aae8d90 2024-05-08T18:05:07.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.574+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaec9f0 con 0x55ba6aaed750 2024-05-08T18:05:07.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.575+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.575+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaec9f0 con 0x55ba6aaed750 2024-05-08T18:05:07.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.575+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.575+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaec9f0 con 0x7ff308076040 2024-05-08T18:05:07.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.575+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30c00a4a0 con 0x7ff308076040 2024-05-08T18:05:07.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.575+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaec9f0 con 0x55ba6aaf20f0 2024-05-08T18:05:07.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.576+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.576+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaec9f0 con 0x55ba6aae8d90 2024-05-08T18:05:07.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.576+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6aae8d90 2024-05-08T18:05:07.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.576+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaec9f0 con 0x55ba6aaed750 2024-05-08T18:05:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.576+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.576+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaec9f0 con 0x55ba6aae8d90 2024-05-08T18:05:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.576+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6aae8d90 2024-05-08T18:05:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.576+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaec9f0 con 0x7ff308076040 2024-05-08T18:05:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.577+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff30c00a4a0 con 0x7ff308076040 2024-05-08T18:05:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.577+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaec9f0 con 0x55ba6aaf20f0 2024-05-08T18:05:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.577+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.577+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x55ba6aaf20f0 2024-05-08T18:05:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.577+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.577+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x55ba6aaed750 2024-05-08T18:05:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.577+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.577+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x55ba6aaf20f0 2024-05-08T18:05:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.578+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.578+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x55ba6aaf20f0 2024-05-08T18:05:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.578+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.578+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x55ba6aae8d90 2024-05-08T18:05:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.578+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6aae8d90 2024-05-08T18:05:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.578+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x55ba6aae8d90 2024-05-08T18:05:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.578+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6aae8d90 2024-05-08T18:05:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.578+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x55ba6aaed750 2024-05-08T18:05:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.579+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.579+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x55ba6aaf20f0 2024-05-08T18:05:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.579+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.579+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x7ff308076040 2024-05-08T18:05:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.579+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30c00a4a0 con 0x7ff308076040 2024-05-08T18:05:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.579+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x55ba6aae8d90 2024-05-08T18:05:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.579+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6aae8d90 2024-05-08T18:05:07.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.579+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x55ba6aaed750 2024-05-08T18:05:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.580+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.580+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x55ba6aaf20f0 2024-05-08T18:05:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.580+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.580+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x55ba6aaed750 2024-05-08T18:05:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.580+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.581+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x7ff308076040 2024-05-08T18:05:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.581+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30c00a4a0 con 0x7ff308076040 2024-05-08T18:05:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.581+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x55ba6aaf20f0 2024-05-08T18:05:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.581+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.581+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x55ba6aaed750 2024-05-08T18:05:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x55ba6aae8d90 2024-05-08T18:05:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6aae8d90 2024-05-08T18:05:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7ff308090040 con 0x7ff308076040 2024-05-08T18:05:07.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff30c00a4a0 con 0x7ff308076040 2024-05-08T18:05:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff31463f800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff31463f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff31463f800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff31463f800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff31463f800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff31463f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff31463f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf0a10 con 0x55ba6aae8d90 2024-05-08T18:05:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6aae8d90 2024-05-08T18:05:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff31463f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff31463f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.582+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf0a10 con 0x55ba6aae8d90 2024-05-08T18:05:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.583+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff30c00a4a0 con 0x55ba6aae8d90 2024-05-08T18:05:07.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.583+0000 7ff31463f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.583+0000 7ff31463f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.583+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf0a10 con 0x55ba6aaf20f0 2024-05-08T18:05:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.583+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.583+0000 7ff31463f800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.583+0000 7ff31463f800 20 zone default found 2024-05-08T18:05:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.583+0000 7ff31463f800 4 Realm: () 2024-05-08T18:05:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.583+0000 7ff31463f800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.583+0000 7ff31463f800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.583+0000 7ff31463f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.583+0000 7ff31463f800 20 zonegroup default 2024-05-08T18:05:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.583+0000 7ff31463f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.583+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf12b0 con 0x55ba6aaed750 2024-05-08T18:05:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.584+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.584+0000 7ff31463f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.584+0000 7ff31463f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.584+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf6ae0 con 0x55ba6aaf20f0 2024-05-08T18:05:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.584+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.584+0000 7ff31463f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.584+0000 7ff31463f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.584+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf6e80 con 0x55ba6aaf20f0 2024-05-08T18:05:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.584+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.584+0000 7ff31463f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.584+0000 7ff31463f800 20 started sync module instance, tier type = 2024-05-08T18:05:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.584+0000 7ff31463f800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.584+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf5f80 con 0x7ff308076040 2024-05-08T18:05:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.585+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30c00a4a0 con 0x7ff308076040 2024-05-08T18:05:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.585+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94259142157488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf5f80 con 0x7ff308076040 2024-05-08T18:05:07.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.585+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf7880 con 0x55ba6aae8d90 2024-05-08T18:05:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.586+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94259142157488] v27'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30c040040 con 0x7ff308076040 2024-05-08T18:05:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.587+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30c040040 con 0x55ba6aae8d90 2024-05-08T18:05:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.587+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94259142161296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf5f80 con 0x55ba6aae8d90 2024-05-08T18:05:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.587+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf8320 con 0x55ba6aaf20f0 2024-05-08T18:05:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.588+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94259142161296] v27'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30c040040 con 0x55ba6aae8d90 2024-05-08T18:05:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.588+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.588+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94259142154960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf5ef0 con 0x55ba6aaf20f0 2024-05-08T18:05:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.588+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf91e0 con 0x7ff308076040 2024-05-08T18:05:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.589+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94259142154960] v27'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.589+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30c040040 con 0x7ff308076040 2024-05-08T18:05:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.589+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94259142167856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf9600 con 0x7ff308076040 2024-05-08T18:05:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.589+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf9fc0 con 0x55ba6aae8d90 2024-05-08T18:05:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.590+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94259142167856] v27'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30c040040 con 0x7ff308076040 2024-05-08T18:05:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.590+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30c040040 con 0x55ba6aae8d90 2024-05-08T18:05:07.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.590+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94259142169392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf9fc0 con 0x55ba6aae8d90 2024-05-08T18:05:07.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.590+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aafa980 con 0x55ba6aaed750 2024-05-08T18:05:07.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.591+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94259142169392] v27'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30c040040 con 0x55ba6aae8d90 2024-05-08T18:05:07.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.591+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.591+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94259142171408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aaf9f30 con 0x55ba6aaed750 2024-05-08T18:05:07.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.591+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aafb700 con 0x55ba6aaed750 2024-05-08T18:05:07.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.592+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.592+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94259142179488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aafc370 con 0x55ba6aaed750 2024-05-08T18:05:07.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.592+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aafcde0 con 0x55ba6aaed750 2024-05-08T18:05:07.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.592+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94259142171408] v27'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.593+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94259142179488] v27'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.593+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.593+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94259142180544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aafad20 con 0x55ba6aaed750 2024-05-08T18:05:07.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.593+0000 7ff31463f800 20 add_watcher() i=0 2024-05-08T18:05:07.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.593+0000 7ff31463f800 20 add_watcher() i=1 2024-05-08T18:05:07.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.593+0000 7ff31463f800 20 add_watcher() i=2 2024-05-08T18:05:07.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.593+0000 7ff31463f800 20 add_watcher() i=3 2024-05-08T18:05:07.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.593+0000 7ff31463f800 20 add_watcher() i=4 2024-05-08T18:05:07.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.593+0000 7ff31463f800 20 add_watcher() i=5 2024-05-08T18:05:07.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.593+0000 7ff31463f800 20 add_watcher() i=6 2024-05-08T18:05:07.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.594+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94259142180544] v27'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.594+0000 7ff31463f800 20 add_watcher() i=7 2024-05-08T18:05:07.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.594+0000 7ff31463f800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:07.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.594+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab00200 con 0x55ba6aae8d90 2024-05-08T18:05:07.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.594+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff30c040040 con 0x55ba6aae8d90 2024-05-08T18:05:07.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.594+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94259142183088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6aafd680 con 0x55ba6aae8d90 2024-05-08T18:05:07.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.595+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94259142183088] v27'100 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff30c040040 con 0x55ba6aae8d90 2024-05-08T18:05:07.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.595+0000 7ff1c8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:07.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.595+0000 7ff31463f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:07.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.595+0000 7ff31463f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:07.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.595+0000 7ff31463f800 5 note: GC not initialized 2024-05-08T18:05:07.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.596+0000 7ff1437fe640 20 reqs_thread_entry: start 2024-05-08T18:05:07.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.596+0000 7ff31463f800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:07.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.596+0000 7ff31463f800 20 Filter name: none 2024-05-08T18:05:07.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.596+0000 7ff1417fa640 20 reqs_thread_entry: start 2024-05-08T18:05:07.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.596+0000 7ff31463f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.596+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab335c0 con 0x7ff308076040 2024-05-08T18:05:07.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.596+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7ff30c040040 con 0x7ff308076040 2024-05-08T18:05:07.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.596+0000 7ff31463f800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:07.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.596+0000 7ff31463f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.596+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab33e50 con 0x55ba6aaed750 2024-05-08T18:05:07.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.597+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.597+0000 7ff31463f800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:07.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.597+0000 7ff31463f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-08T18:05:07.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.597+0000 7ff31463f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.597+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab34000 con 0x7ff308076040 2024-05-08T18:05:07.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.597+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7ff30c040040 con 0x7ff308076040 2024-05-08T18:05:07.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.597+0000 7ff31463f800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:07.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.597+0000 7ff31463f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.597+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:65 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab32980 con 0x55ba6aaed750 2024-05-08T18:05:07.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(65 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff31463f800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:07.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff31463f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-08T18:05:07.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:66 4.37 4:eecadb7d:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab36920 con 0x55ba6aaed750 2024-05-08T18:05:07.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 4.22 4:46cfe895:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab392b0 con 0x55ba6aae8d90 2024-05-08T18:05:07.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 4.0 4:018ce329:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab3bed0 con 0x55ba6aaf20f0 2024-05-08T18:05:07.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 4.12 4:4afe9c7f:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab3e840 con 0x55ba6aae8d90 2024-05-08T18:05:07.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:70 4.d 4:b2c4ac36:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab41680 con 0x55ba6aaf20f0 2024-05-08T18:05:07.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 4.33 4:cc412025:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab43f70 con 0x55ba6aaed750 2024-05-08T18:05:07.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 4.3 4:c17a4869:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab3e840 con 0x55ba6aae8d90 2024-05-08T18:05:07.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:73 4.6 4:61ff1ea9:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab48cb0 con 0x55ba6aaf20f0 2024-05-08T18:05:07.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 4.1c 4:3ae7d9ef:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab4bab0 con 0x55ba6aaed750 2024-05-08T18:05:07.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 4.f 4:f2293886:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab4e4d0 con 0x55ba6aae8d90 2024-05-08T18:05:07.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 4.21 4:87ec338a:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab51210 con 0x55ba6aaed750 2024-05-08T18:05:07.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(66 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(71 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.5 [call out=124b] v0'0 uv3 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(68 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(70 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.598+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(74 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.599+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(76 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.599+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(67 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7ff30c040040 con 0x55ba6aae8d90 2024-05-08T18:05:07.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.599+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(69 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7ff30c040040 con 0x55ba6aae8d90 2024-05-08T18:05:07.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.599+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(72 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.6 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7ff30c040040 con 0x55ba6aae8d90 2024-05-08T18:05:07.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.599+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(75 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7ff30c040040 con 0x55ba6aae8d90 2024-05-08T18:05:07.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.599+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(73 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.600+0000 7ff1c8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:07.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.600+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94259142183088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab1bb40 con 0x55ba6aae8d90 2024-05-08T18:05:07.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.601+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94259142183088] v27'101 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff30c040040 con 0x55ba6aae8d90 2024-05-08T18:05:07.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.601+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94259142157488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab1bb40 con 0x7ff308076040 2024-05-08T18:05:07.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.602+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94259142157488] v27'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30c040040 con 0x7ff308076040 2024-05-08T18:05:07.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.602+0000 7ff31463f800 20 remove_watcher() i=0 2024-05-08T18:05:07.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.602+0000 7ff31463f800 2 removed watcher, disabling cache 2024-05-08T18:05:07.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.602+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94259142161296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab1bb40 con 0x55ba6aae8d90 2024-05-08T18:05:07.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.602+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94259142161296] v27'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30c040040 con 0x55ba6aae8d90 2024-05-08T18:05:07.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.603+0000 7ff31463f800 20 remove_watcher() i=1 2024-05-08T18:05:07.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.603+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94259142154960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab1bb40 con 0x55ba6aaf20f0 2024-05-08T18:05:07.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.603+0000 7ff311936640 1 -- 172.21.15.50:0/1416668156 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94259142154960] v27'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff308081070 con 0x55ba6aaf20f0 2024-05-08T18:05:07.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.603+0000 7ff31463f800 20 remove_watcher() i=2 2024-05-08T18:05:07.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.603+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94259142167856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab1bb40 con 0x7ff308076040 2024-05-08T18:05:07.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.604+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94259142167856] v27'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30c040040 con 0x7ff308076040 2024-05-08T18:05:07.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.604+0000 7ff31463f800 20 remove_watcher() i=3 2024-05-08T18:05:07.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.604+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94259142169392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab1bb40 con 0x55ba6aae8d90 2024-05-08T18:05:07.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.605+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94259142169392] v27'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30c040040 con 0x55ba6aae8d90 2024-05-08T18:05:07.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.605+0000 7ff31463f800 20 remove_watcher() i=4 2024-05-08T18:05:07.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.605+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94259142171408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab1bb40 con 0x55ba6aaed750 2024-05-08T18:05:07.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.606+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94259142171408] v27'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.606+0000 7ff31463f800 20 remove_watcher() i=5 2024-05-08T18:05:07.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.606+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94259142179488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab1bb40 con 0x55ba6aaed750 2024-05-08T18:05:07.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.607+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94259142179488] v27'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.607+0000 7ff31463f800 20 remove_watcher() i=6 2024-05-08T18:05:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.607+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94259142180544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ba6ab1bb40 con 0x55ba6aaed750 2024-05-08T18:05:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.608+0000 7ff312137640 1 -- 172.21.15.50:0/1416668156 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94259142180544] v27'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff30405b030 con 0x55ba6aaed750 2024-05-08T18:05:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.608+0000 7ff31463f800 20 remove_watcher() i=7 2024-05-08T18:05:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.608+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ba6aaed750 msgr2=0x55ba6aaefb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.608+0000 7ff31463f800 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ba6aaed750 0x55ba6aaefb40 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.608+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ba6aae8d90 msgr2=0x55ba6aaeb250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.608+0000 7ff31463f800 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ba6aae8d90 0x55ba6aaeb250 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.608+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff308076040 msgr2=0x7ff308083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.608+0000 7ff31463f800 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff308076040 0x7ff308083410 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.608+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ba6aaf20f0 msgr2=0x55ba6aaf45b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.608+0000 7ff31463f800 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ba6aaf20f0 0x55ba6aaf45b0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.608+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff1dc03bb00 msgr2=0x7ff1dc03dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.608+0000 7ff31463f800 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff1dc03bb00 0x7ff1dc03dfc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff308052fd0 tx=0x7ff30804f000 comp rx=0 tx=0).stop 2024-05-08T18:05:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.608+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aad62c0 msgr2=0x55ba6aadd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.608+0000 7ff31463f800 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aad62c0 0x55ba6aadd580 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff30401afd0 tx=0x7ff304039ee0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.608+0000 7ff312938640 1 -- 172.21.15.50:0/1416668156 reap_dead start 2024-05-08T18:05:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 shutdown_connections 2024-05-08T18:05:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 --2- 172.21.15.50:0/1416668156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aad62c0 0x55ba6aadd580 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 >> 172.21.15.50:0/1416668156 conn(0x55ba6aad2590 msgr2=0x55ba6aacfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 shutdown_connections 2024-05-08T18:05:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 -- 172.21.15.50:0/1416668156 wait complete. 2024-05-08T18:05:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 -- 172.21.15.50:0/3902956914 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff1ec03bb50 msgr2=0x7ff1ec03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 --2- 172.21.15.50:0/3902956914 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff1ec03bb50 0x7ff1ec03e010 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x55ba6aab2940 tx=0x7ff308005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 -- 172.21.15.50:0/3902956914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aac9070 msgr2=0x55ba6aac9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 --2- 172.21.15.50:0/3902956914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aac9070 0x55ba6aac9490 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff304050f60 tx=0x7ff30403fef0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 -- 172.21.15.50:0/3902956914 shutdown_connections 2024-05-08T18:05:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 --2- 172.21.15.50:0/3902956914 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff1ec03bb50 0x7ff1ec03e010 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 --2- 172.21.15.50:0/3902956914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6aac9070 0x55ba6aac9490 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 -- 172.21.15.50:0/3902956914 >> 172.21.15.50:0/3902956914 conn(0x55ba6aabd110 msgr2=0x55ba6aabd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 -- 172.21.15.50:0/3902956914 shutdown_connections 2024-05-08T18:05:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 -- 172.21.15.50:0/3902956914 wait complete. 2024-05-08T18:05:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 -- 172.21.15.50:0/2835419633 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ba6aab4390 msgr2=0x55ba6aab6780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ba6aab4390 0x55ba6aab6780 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 -- 172.21.15.50:0/2835419633 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ba6a9afaf0 msgr2=0x55ba6aab1b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ba6a9afaf0 0x55ba6aab1b30 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 -- 172.21.15.50:0/2835419633 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ba6aab82d0 msgr2=0x55ba6aaba6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.609+0000 7ff31463f800 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ba6aab82d0 0x55ba6aaba6c0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.610+0000 7ff31463f800 1 -- 172.21.15.50:0/2835419633 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff1f803bba0 msgr2=0x7ff1f803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.610+0000 7ff31463f800 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff1f803bba0 0x7ff1f803e060 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff308054000 tx=0x7ff30804e000 comp rx=0 tx=0).stop 2024-05-08T18:05:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.610+0000 7ff31463f800 1 -- 172.21.15.50:0/2835419633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9adfb0 msgr2=0x55ba6aaaa8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.610+0000 7ff31463f800 1 --2- 172.21.15.50:0/2835419633 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ba6a9adfb0 0x55ba6aaaa8f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55ba6aaa3ae0 tx=0x7ff304000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.610+0000 7ff312938640 1 -- 172.21.15.50:0/2835419633 reap_dead start 2024-05-08T18:05:07.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.610+0000 7ff31463f800 1 -- 172.21.15.50:0/2835419633 shutdown_connections 2024-05-08T18:05:07.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.610+0000 7ff31463f800 1 -- 172.21.15.50:0/2835419633 >> 172.21.15.50:0/2835419633 conn(0x55ba6a9a9ad0 msgr2=0x55ba6aa9ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.610+0000 7ff31463f800 1 -- 172.21.15.50:0/2835419633 shutdown_connections 2024-05-08T18:05:07.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.610+0000 7ff31463f800 1 -- 172.21.15.50:0/2835419633 wait complete. 2024-05-08T18:05:07.623 INFO:teuthology.orchestra.run.smithi050.stdout:{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"36133b23-abaa-446a-bf67-ceb1d1559ceb","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1","marker":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1","index_type":"Normal","index_generation":0,"num_shards":11,"object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-05-08T18:05:06.708772Z","creation_time":"2024-05-08T18:05:06.706377Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{"rgw.main":{"size":3,"size_actual":4096,"size_utilized":3,"size_kb":1,"size_kb_actual":4,"size_kb_utilized":1,"num_objects":1}},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}} 2024-05-08T18:05:07.623 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': '36133b23-abaa-446a-bf67-ceb1d1559ceb', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1', 'marker': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-05-08T18:05:06.708772Z', 'creation_time': '2024-05-08T18:05:06.706377Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-05-08T18:05:07.623 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'stats', '--uid', 'foo', '--sync-stats'] 2024-05-08T18:05:07.623 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'stats', '--uid', 'foo', '--sync-stats'] 2024-05-08T18:05:07.624 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user stats --uid foo --sync-stats 2024-05-08T18:05:07.670 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:07.670 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:07.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.673+0000 7f16cff2e800 1 Processor -- start 2024-05-08T18:05:07.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.673+0000 7f16cff2e800 1 -- start start 2024-05-08T18:05:07.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.673+0000 7f16cff2e800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x557fd12f6fd0 0x557fd12f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.673+0000 7f16cff2e800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x557fd126c150 con 0x557fd12f7930 2024-05-08T18:05:07.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.673+0000 7f16cff2e800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x557fd1198e20 con 0x557fd12f6fd0 2024-05-08T18:05:07.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.673+0000 7f16cbc60640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x557fd12f6fd0 0x557fd12f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.673+0000 7f16cbc60640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x557fd12f6fd0 0x557fd12f73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37618/0 (socket says 172.21.15.50:37618) 2024-05-08T18:05:07.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.673+0000 7f16cbc60640 1 -- 172.21.15.50:0/3649989678 learned_addr learned my addr 172.21.15.50:0/3649989678 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.673+0000 7f16cb45f640 1 --1- 172.21.15.50:0/3649989678 >> v1:172.21.15.46:6789/0 conn(0x557fd12f7930 0x557fd12f7de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43140/0 (socket says 172.21.15.50:43140) 2024-05-08T18:05:07.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.674+0000 7f16cbc60640 1 -- 172.21.15.50:0/3649989678 >> v1:172.21.15.46:6789/0 conn(0x557fd12f7930 legacy=0x557fd12f7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.674+0000 7f16cac5e640 1 -- 172.21.15.50:0/3649989678 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3352432212 0 0) 0x557fd126c150 con 0x557fd12f7930 2024-05-08T18:05:07.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.674+0000 7f16cbc60640 1 -- 172.21.15.50:0/3649989678 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fd12f84f0 con 0x557fd12f6fd0 2024-05-08T18:05:07.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.675+0000 7f16cbc60640 1 --2- 172.21.15.50:0/3649989678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 0x557fd12f73f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f16bc005fd0 tx=0x7f16bc01cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8409aaf51d491ccd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.676+0000 7f16cac5e640 1 -- 172.21.15.50:0/3649989678 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16bc052540 con 0x557fd12f6fd0 2024-05-08T18:05:07.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.676+0000 7f16cac5e640 1 -- 172.21.15.50:0/3649989678 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16bc04e070 con 0x557fd12f6fd0 2024-05-08T18:05:07.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.676+0000 7f16cac5e640 1 -- 172.21.15.50:0/3649989678 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16bc052850 con 0x557fd12f6fd0 2024-05-08T18:05:07.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.676+0000 7f16cff2e800 1 -- 172.21.15.50:0/3649989678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 msgr2=0x557fd12f73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.676+0000 7f16cff2e800 1 --2- 172.21.15.50:0/3649989678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 0x557fd12f73f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f16bc005fd0 tx=0x7f16bc01cd80 comp rx=0 tx=0).stop 2024-05-08T18:05:07.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.676+0000 7f16cff2e800 1 -- 172.21.15.50:0/3649989678 shutdown_connections 2024-05-08T18:05:07.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.676+0000 7f16cff2e800 1 --2- 172.21.15.50:0/3649989678 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 0x557fd12f73f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.676+0000 7f16cff2e800 1 -- 172.21.15.50:0/3649989678 >> 172.21.15.50:0/3649989678 conn(0x557fd12f2af0 msgr2=0x557fd12f4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.676+0000 7f16cff2e800 1 -- 172.21.15.50:0/3649989678 shutdown_connections 2024-05-08T18:05:07.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.676+0000 7f16cff2e800 1 -- 172.21.15.50:0/3649989678 wait complete. 2024-05-08T18:05:07.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.691+0000 7f16cff2e800 1 Processor -- start 2024-05-08T18:05:07.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.691+0000 7f16cff2e800 1 -- start start 2024-05-08T18:05:07.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.692+0000 7f16cff2e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 0x557fd13ebda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.692+0000 7f16cff2e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557fd1243a30 con 0x557fd12f6fd0 2024-05-08T18:05:07.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.692+0000 7f16cbc60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 0x557fd13ebda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.692+0000 7f16cbc60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 0x557fd13ebda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37626/0 (socket says 172.21.15.50:37626) 2024-05-08T18:05:07.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.692+0000 7f16cbc60640 1 -- 172.21.15.50:0/2405404284 learned_addr learned my addr 172.21.15.50:0/2405404284 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.692+0000 7f16cbc60640 1 -- 172.21.15.50:0/2405404284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fd13ec370 con 0x557fd12f6fd0 2024-05-08T18:05:07.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.692+0000 7f16cbc60640 1 --2- 172.21.15.50:0/2405404284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 0x557fd13ebda0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f16bc007000 tx=0x7f16bc04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413b26429fae603b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.693+0000 7f15b9e0b640 1 -- 172.21.15.50:0/2405404284 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16bc052860 con 0x557fd12f6fd0 2024-05-08T18:05:07.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.693+0000 7f15b9e0b640 1 -- 172.21.15.50:0/2405404284 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16bc058040 con 0x557fd12f6fd0 2024-05-08T18:05:07.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.693+0000 7f15b9e0b640 1 -- 172.21.15.50:0/2405404284 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16bc052b50 con 0x557fd12f6fd0 2024-05-08T18:05:07.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.693+0000 7f16cff2e800 1 -- 172.21.15.50:0/2405404284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 msgr2=0x557fd13ebda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.693+0000 7f16cff2e800 1 --2- 172.21.15.50:0/2405404284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 0x557fd13ebda0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f16bc007000 tx=0x7f16bc04c000 comp rx=0 tx=0).stop 2024-05-08T18:05:07.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.693+0000 7f16cff2e800 1 -- 172.21.15.50:0/2405404284 shutdown_connections 2024-05-08T18:05:07.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.693+0000 7f16cff2e800 1 --2- 172.21.15.50:0/2405404284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 0x557fd13ebda0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.693+0000 7f16cff2e800 1 -- 172.21.15.50:0/2405404284 >> 172.21.15.50:0/2405404284 conn(0x557fd12f2af0 msgr2=0x557fd13e76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.693+0000 7f16cff2e800 1 -- 172.21.15.50:0/2405404284 shutdown_connections 2024-05-08T18:05:07.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.693+0000 7f16cff2e800 1 -- 172.21.15.50:0/2405404284 wait complete. 2024-05-08T18:05:07.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.694+0000 7f16cff2e800 1 Processor -- start 2024-05-08T18:05:07.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.694+0000 7f16cff2e800 1 -- start start 2024-05-08T18:05:07.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.694+0000 7f16cff2e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 0x557fd13f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.694+0000 7f16cff2e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557fd12f8a80 con 0x557fd12f6fd0 2024-05-08T18:05:07.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.694+0000 7f16cbc60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 0x557fd13f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.694+0000 7f16cbc60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 0x557fd13f38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37628/0 (socket says 172.21.15.50:37628) 2024-05-08T18:05:07.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.694+0000 7f16cbc60640 1 -- 172.21.15.50:0/1475435903 learned_addr learned my addr 172.21.15.50:0/1475435903 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.694+0000 7f16cbc60640 1 -- 172.21.15.50:0/1475435903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fd13f3e30 con 0x557fd12f6fd0 2024-05-08T18:05:07.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.694+0000 7f16cbc60640 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 0x557fd13f38f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f16bc007000 tx=0x7f16bc003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.695+0000 7f15b3fff640 1 -- 172.21.15.50:0/1475435903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16bc05d630 con 0x557fd12f6fd0 2024-05-08T18:05:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.695+0000 7f15b3fff640 1 -- 172.21.15.50:0/1475435903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16bc05d7d0 con 0x557fd12f6fd0 2024-05-08T18:05:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.695+0000 7f15b3fff640 1 -- 172.21.15.50:0/1475435903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16bc05dad0 con 0x557fd12f6fd0 2024-05-08T18:05:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.695+0000 7f16cff2e800 1 -- 172.21.15.50:0/1475435903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557fd13f40c0 con 0x557fd12f6fd0 2024-05-08T18:05:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.695+0000 7f16cff2e800 1 -- 172.21.15.50:0/1475435903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fd13f6cd0 con 0x557fd12f6fd0 2024-05-08T18:05:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.695+0000 7f15b3fff640 1 -- 172.21.15.50:0/1475435903 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f16bc050070 con 0x557fd12f6fd0 2024-05-08T18:05:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.696+0000 7f15b3fff640 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f15b403bb50 0x7f15b403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.696+0000 7f15b3fff640 1 -- 172.21.15.50:0/1475435903 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f16bc062070 con 0x557fd12f6fd0 2024-05-08T18:05:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.696+0000 7f16cb45f640 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f15b403bb50 0x7f15b403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.696+0000 7f16cff2e800 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557fd12f7930 0x557fd13faa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.696+0000 7f16cff2e800 1 -- 172.21.15.50:0/1475435903 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd13fafd0 con 0x557fd12f7930 2024-05-08T18:05:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.696+0000 7f16cc461640 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557fd12f7930 0x557fd13faa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.696+0000 7f16cb45f640 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f15b403bb50 0x7f15b403e010 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f16b4000c00 tx=0x7f16b40013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.696+0000 7f16cc461640 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557fd12f7930 0x557fd13faa90 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.697+0000 7f16cc461640 1 -- 172.21.15.50:0/1475435903 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f16c40083e0 con 0x557fd12f7930 2024-05-08T18:05:07.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.697+0000 7f16cff2e800 1 -- 172.21.15.50:0/1475435903 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd13f8020 con 0x557fd12f7930 2024-05-08T18:05:07.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.697+0000 7f16cc461640 1 -- 172.21.15.50:0/1475435903 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f16c40083e0 con 0x557fd12f7930 2024-05-08T18:05:07.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.697+0000 7f16cff2e800 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557fd13fd2a0 0x557fd13ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.697+0000 7f16cff2e800 1 -- 172.21.15.50:0/1475435903 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd13ffd60 con 0x557fd13fd2a0 2024-05-08T18:05:07.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.697+0000 7f16cbc60640 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557fd13fd2a0 0x557fd13ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.698+0000 7f16cbc60640 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557fd13fd2a0 0x557fd13ff690 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.698+0000 7f16cbc60640 1 -- 172.21.15.50:0/1475435903 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f16bc078070 con 0x557fd13fd2a0 2024-05-08T18:05:07.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.698+0000 7f16cff2e800 1 -- 172.21.15.50:0/1475435903 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd13fc140 con 0x557fd12f7930 2024-05-08T18:05:07.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.698+0000 7f16cc461640 1 -- 172.21.15.50:0/1475435903 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f16c40083e0 con 0x557fd12f7930 2024-05-08T18:05:07.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.698+0000 7f16cff2e800 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557fd14011a0 0x557fd1403590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.698+0000 7f16cff2e800 1 -- 172.21.15.50:0/1475435903 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd1403d20 con 0x557fd14011a0 2024-05-08T18:05:07.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.698+0000 7f16cb45f640 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557fd14011a0 0x557fd1403590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.699+0000 7f16cb45f640 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557fd14011a0 0x557fd1403590 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.699+0000 7f16cb45f640 1 -- 172.21.15.50:0/1475435903 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f16b400f670 con 0x557fd14011a0 2024-05-08T18:05:07.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.699+0000 7f16cff2e800 1 Processor -- start 2024-05-08T18:05:07.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.699+0000 7f16cff2e800 1 -- start start 2024-05-08T18:05:07.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.700+0000 7f16cff2e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd1412050 0x557fd1412470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.700+0000 7f16cff2e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557fd11ea330 con 0x557fd1412050 2024-05-08T18:05:07.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.700+0000 7f16cbc60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd1412050 0x557fd1412470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.700+0000 7f16cbc60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd1412050 0x557fd1412470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37638/0 (socket says 172.21.15.50:37638) 2024-05-08T18:05:07.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.700+0000 7f16cbc60640 1 -- 172.21.15.50:0/1138928611 learned_addr learned my addr 172.21.15.50:0/1138928611 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.700+0000 7f16cbc60640 1 -- 172.21.15.50:0/1138928611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fd14129b0 con 0x557fd1412050 2024-05-08T18:05:07.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.700+0000 7f16cbc60640 1 --2- 172.21.15.50:0/1138928611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd1412050 0x557fd1412470 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f16bc092d50 tx=0x7f16bc068ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.700+0000 7f15b17fa640 1 -- 172.21.15.50:0/1138928611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16bc064380 con 0x557fd1412050 2024-05-08T18:05:07.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.700+0000 7f15b17fa640 1 -- 172.21.15.50:0/1138928611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16bc064520 con 0x557fd1412050 2024-05-08T18:05:07.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.700+0000 7f16cff2e800 1 -- 172.21.15.50:0/1138928611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557fd1413420 con 0x557fd1412050 2024-05-08T18:05:07.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.700+0000 7f15b17fa640 1 -- 172.21.15.50:0/1138928611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16bc064840 con 0x557fd1412050 2024-05-08T18:05:07.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.700+0000 7f16cff2e800 1 -- 172.21.15.50:0/1138928611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fd1413920 con 0x557fd1412050 2024-05-08T18:05:07.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.701+0000 7f15b17fa640 1 -- 172.21.15.50:0/1138928611 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f16bc05c7e0 con 0x557fd1412050 2024-05-08T18:05:07.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.701+0000 7f15b17fa640 1 --2- 172.21.15.50:0/1138928611 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f15a403bba0 0x7f15a403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.701+0000 7f15b17fa640 1 -- 172.21.15.50:0/1138928611 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f16bc087070 con 0x557fd1412050 2024-05-08T18:05:07.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.701+0000 7f16cb45f640 1 --2- 172.21.15.50:0/1138928611 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f15a403bba0 0x7f15a403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.702+0000 7f16cff2e800 1 Processor -- start 2024-05-08T18:05:07.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.702+0000 7f16cff2e800 1 -- start start 2024-05-08T18:05:07.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.702+0000 7f16cb45f640 1 --2- 172.21.15.50:0/1138928611 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f15a403bba0 0x7f15a403e060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x557fd1404120 tx=0x7f16b4006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.702+0000 7f16cff2e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd141f170 0x557fd141f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.702+0000 7f16cff2e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557fd13ec630 con 0x557fd141f170 2024-05-08T18:05:07.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.702+0000 7f16cbc60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd141f170 0x557fd141f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.702+0000 7f16cbc60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd141f170 0x557fd141f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37644/0 (socket says 172.21.15.50:37644) 2024-05-08T18:05:07.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.702+0000 7f16cbc60640 1 -- 172.21.15.50:0/1107550615 learned_addr learned my addr 172.21.15.50:0/1107550615 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.703+0000 7f16cbc60640 1 -- 172.21.15.50:0/1107550615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fd141fb60 con 0x557fd141f170 2024-05-08T18:05:07.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.703+0000 7f16cbc60640 1 --2- 172.21.15.50:0/1107550615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd141f170 0x557fd141f590 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0945e0 tx=0x7f16bc07b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691bb51e8fb2aa60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.703+0000 7f15aaffd640 1 -- 172.21.15.50:0/1107550615 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16bc071d40 con 0x557fd141f170 2024-05-08T18:05:07.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.703+0000 7f15aaffd640 1 -- 172.21.15.50:0/1107550615 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16bc051950 con 0x557fd141f170 2024-05-08T18:05:07.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.703+0000 7f15aaffd640 1 -- 172.21.15.50:0/1107550615 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16bc071420 con 0x557fd141f170 2024-05-08T18:05:07.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.703+0000 7f16cff2e800 1 -- 172.21.15.50:0/1107550615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd141f170 msgr2=0x557fd141f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.703+0000 7f16cff2e800 1 --2- 172.21.15.50:0/1107550615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd141f170 0x557fd141f590 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f16bc0945e0 tx=0x7f16bc07b7b0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.704+0000 7f16cff2e800 1 -- 172.21.15.50:0/1107550615 shutdown_connections 2024-05-08T18:05:07.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.704+0000 7f16cff2e800 1 --2- 172.21.15.50:0/1107550615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd141f170 0x557fd141f590 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.704+0000 7f16cff2e800 1 -- 172.21.15.50:0/1107550615 >> 172.21.15.50:0/1107550615 conn(0x557fd141b440 msgr2=0x557fd14194a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.704+0000 7f16cff2e800 1 -- 172.21.15.50:0/1107550615 shutdown_connections 2024-05-08T18:05:07.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.704+0000 7f16cff2e800 1 -- 172.21.15.50:0/1107550615 wait complete. 2024-05-08T18:05:07.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.704+0000 7f16cff2e800 1 Processor -- start 2024-05-08T18:05:07.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.704+0000 7f16cff2e800 1 -- start start 2024-05-08T18:05:07.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.704+0000 7f16cff2e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd141f170 0x557fd14264b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.704+0000 7f16cff2e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557fd14201c0 con 0x557fd141f170 2024-05-08T18:05:07.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.704+0000 7f16cbc60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd141f170 0x557fd14264b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.704+0000 7f16cbc60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd141f170 0x557fd14264b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37654/0 (socket says 172.21.15.50:37654) 2024-05-08T18:05:07.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.704+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 learned_addr learned my addr 172.21.15.50:0/3781182410 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.705+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fd14269f0 con 0x557fd141f170 2024-05-08T18:05:07.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.705+0000 7f16cbc60640 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd141f170 0x557fd14264b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f16bc051740 tx=0x7f16bc066e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.705+0000 7f15a97fa640 1 -- 172.21.15.50:0/3781182410 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16bc094b10 con 0x557fd141f170 2024-05-08T18:05:07.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.705+0000 7f15a97fa640 1 -- 172.21.15.50:0/3781182410 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16bc0ad070 con 0x557fd141f170 2024-05-08T18:05:07.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.705+0000 7f15a97fa640 1 -- 172.21.15.50:0/3781182410 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16bc094e20 con 0x557fd141f170 2024-05-08T18:05:07.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.705+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557fd14269f0 con 0x557fd141f170 2024-05-08T18:05:07.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.705+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fd14269f0 con 0x557fd141f170 2024-05-08T18:05:07.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.706+0000 7f15a97fa640 1 -- 172.21.15.50:0/3781182410 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f16bc09d070 con 0x557fd141f170 2024-05-08T18:05:07.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.706+0000 7f15a97fa640 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f159403bba0 0x7f159403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.706+0000 7f15a97fa640 1 -- 172.21.15.50:0/3781182410 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f16bc0a3070 con 0x557fd141f170 2024-05-08T18:05:07.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.706+0000 7f16cb45f640 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f159403bba0 0x7f159403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.706+0000 7f16cff2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.706+0000 7f16cff2e800 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557fd1431a60 0x557fd1433f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.706+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd1434670 con 0x557fd1431a60 2024-05-08T18:05:07.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.707+0000 7f16cc461640 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557fd1431a60 0x557fd1433f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.707+0000 7f16cb45f640 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f159403bba0 0x7f159403e060 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f16b400ffd0 tx=0x7f16b4023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.707+0000 7f16cc461640 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557fd1431a60 0x557fd1433f20 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.707+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f16c40083e0 con 0x557fd1431a60 2024-05-08T18:05:07.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.707+0000 7f16cff2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.707+0000 7f16cff2e800 20 realm 2024-05-08T18:05:07.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.707+0000 7f16cff2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.707+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd1434a80 con 0x557fd1431a60 2024-05-08T18:05:07.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.707+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f16c40083e0 con 0x557fd1431a60 2024-05-08T18:05:07.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.707+0000 7f16cff2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.707+0000 7f16cff2e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:07.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.707+0000 7f16cff2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.707+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1431a60 2024-05-08T18:05:07.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.708+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f16c40083e0 con 0x557fd1431a60 2024-05-08T18:05:07.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.708+0000 7f16cff2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.708+0000 7f16cff2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.708+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd14356f0 con 0x557fd1431a60 2024-05-08T18:05:07.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.708+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f16c40083e0 con 0x557fd1431a60 2024-05-08T18:05:07.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.708+0000 7f16cff2e800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:07.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.708+0000 7f16cff2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.708+0000 7f16cff2e800 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557fd1435e00 0x557fd14381f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.708+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd1438980 con 0x557fd1435e00 2024-05-08T18:05:07.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.708+0000 7f16cbc60640 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557fd1435e00 0x557fd14381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.708+0000 7f16cbc60640 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557fd1435e00 0x557fd14381f0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.709+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.709+0000 7f16cff2e800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:07.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.709+0000 7f16cff2e800 20 searching for the correct realm 2024-05-08T18:05:07.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.709+0000 7f16cff2e800 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557fd143aa70 0x557fd143cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.709+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd143d600 con 0x557fd143aa70 2024-05-08T18:05:07.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.709+0000 7f16cb45f640 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557fd143aa70 0x557fd143cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.709+0000 7f16cb45f640 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557fd143aa70 0x557fd143cf30 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.710+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.710+0000 7f16cb45f640 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f16b402e910 0x7f16b403a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.710+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd143d600 con 0x7f16b402e910 2024-05-08T18:05:07.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.710+0000 7f16cc461640 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f16b402e910 0x7f16b403a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.710+0000 7f16cc461640 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f16b402e910 0x7f16b403a3c0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.710+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16c40083e0 con 0x7f16b402e910 2024-05-08T18:05:07.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.710+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1435e00 2024-05-08T18:05:07.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.711+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.711+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1431a60 2024-05-08T18:05:07.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.711+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16c40083e0 con 0x557fd1431a60 2024-05-08T18:05:07.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.711+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1431a60 2024-05-08T18:05:07.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.711+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.711+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1435e00 2024-05-08T18:05:07.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.711+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.711+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1435e00 2024-05-08T18:05:07.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.712+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.712+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x7f16b402e910 2024-05-08T18:05:07.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.712+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16c4026040 con 0x7f16b402e910 2024-05-08T18:05:07.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.712+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd143aa70 2024-05-08T18:05:07.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.712+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.712+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1431a60 2024-05-08T18:05:07.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.712+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.712+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1435e00 2024-05-08T18:05:07.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.713+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.713+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1431a60 2024-05-08T18:05:07.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.713+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.713+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x7f16b402e910 2024-05-08T18:05:07.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.713+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f16c4026040 con 0x7f16b402e910 2024-05-08T18:05:07.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.713+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd143aa70 2024-05-08T18:05:07.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.713+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.713+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd143aa70 2024-05-08T18:05:07.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.714+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.714+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1435e00 2024-05-08T18:05:07.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.714+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.714+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd143aa70 2024-05-08T18:05:07.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.714+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.714+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd143aa70 2024-05-08T18:05:07.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.714+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.714+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1431a60 2024-05-08T18:05:07.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.715+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.715+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1431a60 2024-05-08T18:05:07.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.715+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.715+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1435e00 2024-05-08T18:05:07.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.715+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.715+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd143aa70 2024-05-08T18:05:07.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.715+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.716+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x7f16b402e910 2024-05-08T18:05:07.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.716+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16c4026040 con 0x7f16b402e910 2024-05-08T18:05:07.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.716+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1431a60 2024-05-08T18:05:07.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.716+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.716+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1435e00 2024-05-08T18:05:07.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.716+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.716+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd143aa70 2024-05-08T18:05:07.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.716+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.717+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1435e00 2024-05-08T18:05:07.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.717+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.717+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x7f16b402e910 2024-05-08T18:05:07.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.717+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16c4026040 con 0x7f16b402e910 2024-05-08T18:05:07.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.717+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd143aa70 2024-05-08T18:05:07.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.717+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.717+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1435e00 2024-05-08T18:05:07.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x557fd1431a60 2024-05-08T18:05:07.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557fd1435350 con 0x7f16b402e910 2024-05-08T18:05:07.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f16c4026040 con 0x7f16b402e910 2024-05-08T18:05:07.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cff2e800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:07.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cff2e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:07.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cff2e800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:07.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cff2e800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:07.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cff2e800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:07.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cff2e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:07.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cff2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd1439330 con 0x557fd1431a60 2024-05-08T18:05:07.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cff2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cff2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.718+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd1439330 con 0x557fd1431a60 2024-05-08T18:05:07.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cff2e800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:07.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cff2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd1439300 con 0x557fd143aa70 2024-05-08T18:05:07.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cff2e800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:07.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cff2e800 20 zone default found 2024-05-08T18:05:07.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cff2e800 4 Realm: () 2024-05-08T18:05:07.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cff2e800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:07.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cff2e800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:07.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cff2e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:07.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cff2e800 20 zonegroup default 2024-05-08T18:05:07.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cff2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd1439bd0 con 0x557fd1435e00 2024-05-08T18:05:07.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cff2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cff2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd143f460 con 0x557fd143aa70 2024-05-08T18:05:07.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.719+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.720+0000 7f16cff2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.720+0000 7f16cff2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.720+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd143f800 con 0x557fd143aa70 2024-05-08T18:05:07.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.720+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.720+0000 7f16cff2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.720+0000 7f16cff2e800 20 started sync module instance, tier type = 2024-05-08T18:05:07.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.720+0000 7f16cff2e800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:07.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.720+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd143e900 con 0x7f16b402e910 2024-05-08T18:05:07.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.721+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x7f16b402e910 2024-05-08T18:05:07.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.721+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94007460097568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd143e900 con 0x7f16b402e910 2024-05-08T18:05:07.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.721+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd14401c0 con 0x557fd1431a60 2024-05-08T18:05:07.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.722+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94007460097568] v27'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x7f16b402e910 2024-05-08T18:05:07.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.722+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.722+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94007460101872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd143e900 con 0x557fd1431a60 2024-05-08T18:05:07.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.722+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd1440c10 con 0x557fd143aa70 2024-05-08T18:05:07.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.723+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94007460101872] v27'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.723+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.723+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94007460095056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd143e870 con 0x557fd143aa70 2024-05-08T18:05:07.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.723+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd1441b00 con 0x7f16b402e910 2024-05-08T18:05:07.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.724+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94007460095056] v27'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.725+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x7f16b402e910 2024-05-08T18:05:07.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.725+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94007460107856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd1441a70 con 0x7f16b402e910 2024-05-08T18:05:07.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.725+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd14429c0 con 0x557fd1431a60 2024-05-08T18:05:07.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.726+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.726+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94007460111632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd1442de0 con 0x557fd1431a60 2024-05-08T18:05:07.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.726+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94007460107856] v27'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x7f16b402e910 2024-05-08T18:05:07.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.726+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd14438d0 con 0x557fd1435e00 2024-05-08T18:05:07.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.727+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94007460111632] v27'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.727+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.727+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94007460115360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd1442de0 con 0x557fd1435e00 2024-05-08T18:05:07.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.727+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd1444120 con 0x557fd1435e00 2024-05-08T18:05:07.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.727+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94007460115360] v27'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.728+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.728+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94007460118128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd1442de0 con 0x557fd1435e00 2024-05-08T18:05:07.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.728+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd1444b70 con 0x557fd1435e00 2024-05-08T18:05:07.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.729+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94007460118128] v27'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.729+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.729+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94007460120192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd1444d50 con 0x557fd1435e00 2024-05-08T18:05:07.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.729+0000 7f16cff2e800 20 add_watcher() i=0 2024-05-08T18:05:07.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.729+0000 7f16cff2e800 20 add_watcher() i=1 2024-05-08T18:05:07.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.729+0000 7f16cff2e800 20 add_watcher() i=2 2024-05-08T18:05:07.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.729+0000 7f16cff2e800 20 add_watcher() i=3 2024-05-08T18:05:07.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.729+0000 7f16cff2e800 20 add_watcher() i=4 2024-05-08T18:05:07.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.729+0000 7f16cff2e800 20 add_watcher() i=5 2024-05-08T18:05:07.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.729+0000 7f16cff2e800 20 add_watcher() i=6 2024-05-08T18:05:07.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.729+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94007460120192] v27'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.729+0000 7f16cff2e800 20 add_watcher() i=7 2024-05-08T18:05:07.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.729+0000 7f16cff2e800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:07.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.730+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd1448120 con 0x557fd1431a60 2024-05-08T18:05:07.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.730+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.730+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94007460121664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd1445510 con 0x557fd1431a60 2024-05-08T18:05:07.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.731+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94007460121664] v27'102 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.731+0000 7f1581fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:07.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.731+0000 7f16cff2e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:07.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.731+0000 7f16cff2e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:07.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.731+0000 7f16cff2e800 5 note: GC not initialized 2024-05-08T18:05:07.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.731+0000 7f1580fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:07.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.732+0000 7f16cff2e800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:07.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.732+0000 7f16cff2e800 20 Filter name: none 2024-05-08T18:05:07.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.732+0000 7f14fa7fc640 20 reqs_thread_entry: start 2024-05-08T18:05:07.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.732+0000 7f16cff2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.732+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd147af00 con 0x557fd1431a60 2024-05-08T18:05:07.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.732+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.732+0000 7f16cff2e800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:07.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.732+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd147c450 con 0x7f16b402e910 2024-05-08T18:05:07.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.732+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv16 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f16c4026040 con 0x7f16b402e910 2024-05-08T18:05:07.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.732+0000 7f16cff2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.732+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:64 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd147c450 con 0x557fd1435e00 2024-05-08T18:05:07.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.733+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(64 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.733+0000 7f16cff2e800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:07.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.733+0000 7f16cff2e800 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-08T18:05:07.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.733+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:65 4.37 4:eecadb7d:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd147db20 con 0x557fd1435e00 2024-05-08T18:05:07.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.733+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 4.22 4:46cfe895:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd1480790 con 0x557fd1431a60 2024-05-08T18:05:07.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.733+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 4.0 4:018ce329:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd14833a0 con 0x557fd143aa70 2024-05-08T18:05:07.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.733+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 4.12 4:4afe9c7f:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd1485cb0 con 0x557fd1431a60 2024-05-08T18:05:07.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.733+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 4.d 4:b2c4ac36:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd14885c0 con 0x557fd143aa70 2024-05-08T18:05:07.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.733+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 4.33 4:cc412025:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd148af00 con 0x557fd1435e00 2024-05-08T18:05:07.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.733+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 4.3 4:c17a4869:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd148d830 con 0x557fd1431a60 2024-05-08T18:05:07.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.733+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:72 4.6 4:61ff1ea9:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd14905b0 con 0x557fd143aa70 2024-05-08T18:05:07.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.733+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 4.1c 4:3ae7d9ef:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd1492ea0 con 0x557fd1435e00 2024-05-08T18:05:07.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.733+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:74 4.f 4:f2293886:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd1495810 con 0x557fd1431a60 2024-05-08T18:05:07.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.733+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 4.21 4:87ec338a:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd1498200 con 0x557fd1435e00 2024-05-08T18:05:07.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.733+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(65 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.733+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(70 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.5 [call out=124b] v0'0 uv3 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.734+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(67 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.734+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(69 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.734+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(73 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.734+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(66 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.734+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.734+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(72 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.734+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(75 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.734+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(71 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.6 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.734+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(74 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.734+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:76 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd148ff80 con 0x7f16b402e910 2024-05-08T18:05:07.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.735+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(76 foo.buckets [call] v27'17 uv17 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x7f16b402e910 2024-05-08T18:05:07.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.735+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:77 8.9 8:96d98480:users.uid::foo.buckets:head [call user.complete_stats_sync in=37b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd148ff80 con 0x7f16b402e910 2024-05-08T18:05:07.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.736+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(77 foo.buckets [call] v27'18 uv18 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x7f16b402e910 2024-05-08T18:05:07.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.736+0000 7f16cff2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.736+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:78 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd148ff80 con 0x557fd1431a60 2024-05-08T18:05:07.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.736+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(78 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.736+0000 7f16cff2e800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:07.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.736+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:79 8.9 8:96d98480:users.uid::foo.buckets:head [call user.get_header in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557fd148ff80 con 0x7f16b402e910 2024-05-08T18:05:07.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.737+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(79 foo.buckets [call out=58b] v0'0 uv18 ondisk = 0) ==== 155+0+58 (crc 0 0 0) 0x7f16c4026040 con 0x7f16b402e910 2024-05-08T18:05:07.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.737+0000 7f1581fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:07.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.738+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94007460121664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd148ff80 con 0x557fd1431a60 2024-05-08T18:05:07.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.739+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(80 data_loggenerations_metadata [watch unwatch cookie 94007460121664] v27'103 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.739+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:81 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94007460097568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd148ff80 con 0x7f16b402e910 2024-05-08T18:05:07.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.740+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(81 notify.0 [watch unwatch cookie 94007460097568] v27'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x7f16b402e910 2024-05-08T18:05:07.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.740+0000 7f16cff2e800 20 remove_watcher() i=0 2024-05-08T18:05:07.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.740+0000 7f16cff2e800 2 removed watcher, disabling cache 2024-05-08T18:05:07.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.740+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:82 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94007460101872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd148ff80 con 0x557fd1431a60 2024-05-08T18:05:07.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.741+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(82 notify.1 [watch unwatch cookie 94007460101872] v27'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.741+0000 7f16cff2e800 20 remove_watcher() i=1 2024-05-08T18:05:07.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.741+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:83 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94007460095056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd148ff80 con 0x557fd143aa70 2024-05-08T18:05:07.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.742+0000 7f16cb45f640 1 -- 172.21.15.50:0/3781182410 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(83 notify.2 [watch unwatch cookie 94007460095056] v27'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16b4037040 con 0x557fd143aa70 2024-05-08T18:05:07.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.742+0000 7f16cff2e800 20 remove_watcher() i=2 2024-05-08T18:05:07.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.742+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:84 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94007460107856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd1463a60 con 0x7f16b402e910 2024-05-08T18:05:07.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.743+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(84 notify.3 [watch unwatch cookie 94007460107856] v27'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x7f16b402e910 2024-05-08T18:05:07.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.743+0000 7f16cff2e800 20 remove_watcher() i=3 2024-05-08T18:05:07.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.743+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:85 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94007460111632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd1463a60 con 0x557fd1431a60 2024-05-08T18:05:07.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.744+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(85 notify.4 [watch unwatch cookie 94007460111632] v27'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16c4026040 con 0x557fd1431a60 2024-05-08T18:05:07.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.744+0000 7f16cff2e800 20 remove_watcher() i=4 2024-05-08T18:05:07.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.745+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:86 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94007460115360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd1463a60 con 0x557fd1435e00 2024-05-08T18:05:07.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.746+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(86 notify.5 [watch unwatch cookie 94007460115360] v27'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.746+0000 7f16cff2e800 20 remove_watcher() i=5 2024-05-08T18:05:07.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.746+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:87 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94007460118128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd1463a60 con 0x557fd1435e00 2024-05-08T18:05:07.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.746+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(87 notify.6 [watch unwatch cookie 94007460118128] v27'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.746+0000 7f16cff2e800 20 remove_watcher() i=6 2024-05-08T18:05:07.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.746+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94007460120192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557fd1463a60 con 0x557fd1435e00 2024-05-08T18:05:07.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cbc60640 1 -- 172.21.15.50:0/3781182410 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(88 notify.7 [watch unwatch cookie 94007460120192] v27'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f16bc078070 con 0x557fd1435e00 2024-05-08T18:05:07.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 20 remove_watcher() i=7 2024-05-08T18:05:07.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557fd1435e00 msgr2=0x557fd14381f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557fd1435e00 0x557fd14381f0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557fd1431a60 msgr2=0x557fd1433f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557fd1431a60 0x557fd1433f20 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f16b402e910 msgr2=0x7f16b403a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f16b402e910 0x7f16b403a3c0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557fd143aa70 msgr2=0x557fd143cf30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557fd143aa70 0x557fd143cf30 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f159403bba0 msgr2=0x7f159403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f159403bba0 0x7f159403e060 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f16b400ffd0 tx=0x7f16b4023040 comp rx=0 tx=0).stop 2024-05-08T18:05:07.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd141f170 msgr2=0x557fd14264b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 --2- 172.21.15.50:0/3781182410 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd141f170 0x557fd14264b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f16bc051740 tx=0x7f16bc066e70 comp rx=0 tx=0).stop 2024-05-08T18:05:07.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 reap_dead start 2024-05-08T18:05:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cc461640 1 -- 172.21.15.50:0/3781182410 reap_dead start 2024-05-08T18:05:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 shutdown_connections 2024-05-08T18:05:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 >> 172.21.15.50:0/3781182410 conn(0x557fd141b440 msgr2=0x557fd1418aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 shutdown_connections 2024-05-08T18:05:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 -- 172.21.15.50:0/3781182410 wait complete. 2024-05-08T18:05:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 -- 172.21.15.50:0/1138928611 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f15a403bba0 msgr2=0x7f15a403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 --2- 172.21.15.50:0/1138928611 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f15a403bba0 0x7f15a403e060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x557fd1404120 tx=0x7f16b4006000 comp rx=0 tx=0).stop 2024-05-08T18:05:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 -- 172.21.15.50:0/1138928611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd1412050 msgr2=0x557fd1412470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 --2- 172.21.15.50:0/1138928611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd1412050 0x557fd1412470 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f16bc092d50 tx=0x7f16bc068ce0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 -- 172.21.15.50:0/1138928611 shutdown_connections 2024-05-08T18:05:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 --2- 172.21.15.50:0/1138928611 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f15a403bba0 0x7f15a403e060 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 --2- 172.21.15.50:0/1138928611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd1412050 0x557fd1412470 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 -- 172.21.15.50:0/1138928611 >> 172.21.15.50:0/1138928611 conn(0x557fd14060f0 msgr2=0x557fd14064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 -- 172.21.15.50:0/1138928611 shutdown_connections 2024-05-08T18:05:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.748+0000 7f16cff2e800 1 -- 172.21.15.50:0/1138928611 wait complete. 2024-05-08T18:05:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.749+0000 7f16cff2e800 1 -- 172.21.15.50:0/1475435903 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557fd13fd2a0 msgr2=0x557fd13ff690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.749+0000 7f16cff2e800 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557fd13fd2a0 0x557fd13ff690 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.749+0000 7f16cff2e800 1 -- 172.21.15.50:0/1475435903 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557fd12f7930 msgr2=0x557fd13faa90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.749+0000 7f16cff2e800 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557fd12f7930 0x557fd13faa90 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.749+0000 7f16cff2e800 1 -- 172.21.15.50:0/1475435903 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557fd14011a0 msgr2=0x557fd1403590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.749+0000 7f16cff2e800 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557fd14011a0 0x557fd1403590 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.749+0000 7f16cff2e800 1 -- 172.21.15.50:0/1475435903 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f15b403bb50 msgr2=0x7f15b403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.749+0000 7f16cff2e800 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f15b403bb50 0x7f15b403e010 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f16b4000c00 tx=0x7f16b40013b0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.749+0000 7f16cff2e800 1 -- 172.21.15.50:0/1475435903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 msgr2=0x557fd13f38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.749+0000 7f16cff2e800 1 --2- 172.21.15.50:0/1475435903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fd12f6fd0 0x557fd13f38f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f16bc007000 tx=0x7f16bc003330 comp rx=0 tx=0).stop 2024-05-08T18:05:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.749+0000 7f16cc461640 1 -- 172.21.15.50:0/1475435903 reap_dead start 2024-05-08T18:05:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.749+0000 7f16cff2e800 1 -- 172.21.15.50:0/1475435903 shutdown_connections 2024-05-08T18:05:07.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.749+0000 7f16cff2e800 1 -- 172.21.15.50:0/1475435903 >> 172.21.15.50:0/1475435903 conn(0x557fd12f2af0 msgr2=0x557fd13e5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.749+0000 7f16cff2e800 1 -- 172.21.15.50:0/1475435903 shutdown_connections 2024-05-08T18:05:07.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.749+0000 7f16cff2e800 1 -- 172.21.15.50:0/1475435903 wait complete. 2024-05-08T18:05:07.756 INFO:teuthology.orchestra.run.smithi050.stdout:{"stats":{"size":3,"size_actual":4096,"size_kb":1,"size_kb_actual":4,"num_objects":1},"last_stats_sync":"2024-05-08T18:05:07.736203Z","last_stats_update":"2024-05-08T18:05:07.735213Z"} 2024-05-08T18:05:07.756 DEBUG:tasks.util.rgw: json result: {'stats': {'size': 3, 'size_actual': 4096, 'size_kb': 1, 'size_kb_actual': 4, 'num_objects': 1}, 'last_stats_sync': '2024-05-08T18:05:07.736203Z', 'last_stats_update': '2024-05-08T18:05:07.735213Z'} 2024-05-08T18:05:07.761 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:07.761 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2024-05-08T18:05:07.761 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2024-05-08T18:05:07.762 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid foo --bucket myfoo 2024-05-08T18:05:07.809 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:07.809 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:07.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.812+0000 7f759632d800 1 Processor -- start 2024-05-08T18:05:07.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.812+0000 7f759632d800 1 -- start start 2024-05-08T18:05:07.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.812+0000 7f759632d800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5623f78421a0 0x5623f7844590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.812+0000 7f759632d800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5623f78485e0 con 0x5623f7844ad0 2024-05-08T18:05:07.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.812+0000 7f759632d800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5623f76e1e20 con 0x5623f78421a0 2024-05-08T18:05:07.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.813+0000 7f7591f71640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5623f78421a0 0x5623f7844590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.813+0000 7f7591f71640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5623f78421a0 0x5623f7844590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37666/0 (socket says 172.21.15.50:37666) 2024-05-08T18:05:07.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.813+0000 7f7591f71640 1 -- 172.21.15.50:0/2160254648 learned_addr learned my addr 172.21.15.50:0/2160254648 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.813+0000 7f7591770640 1 --1- 172.21.15.50:0/2160254648 >> v1:172.21.15.46:6789/0 conn(0x5623f7844ad0 0x5623f7846ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43146/0 (socket says 172.21.15.50:43146) 2024-05-08T18:05:07.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.813+0000 7f7591f71640 1 -- 172.21.15.50:0/2160254648 >> v1:172.21.15.46:6789/0 conn(0x5623f7844ad0 legacy=0x5623f7846ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.813+0000 7f7591f71640 1 -- 172.21.15.50:0/2160254648 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623f78497c0 con 0x5623f78421a0 2024-05-08T18:05:07.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.815+0000 7f7591f71640 1 --2- 172.21.15.50:0/2160254648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 0x5623f7844590 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f758400a520 tx=0x7f758401d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563d034fa6560289 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.815+0000 7f7590f6f640 1 -- 172.21.15.50:0/2160254648 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7584057070 con 0x5623f78421a0 2024-05-08T18:05:07.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.815+0000 7f7590f6f640 1 -- 172.21.15.50:0/2160254648 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7584052570 con 0x5623f78421a0 2024-05-08T18:05:07.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.815+0000 7f7590f6f640 1 -- 172.21.15.50:0/2160254648 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f758404ae60 con 0x5623f78421a0 2024-05-08T18:05:07.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.815+0000 7f759632d800 1 -- 172.21.15.50:0/2160254648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 msgr2=0x5623f7844590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.815+0000 7f759632d800 1 --2- 172.21.15.50:0/2160254648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 0x5623f7844590 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f758400a520 tx=0x7f758401d010 comp rx=0 tx=0).stop 2024-05-08T18:05:07.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.816+0000 7f759632d800 1 -- 172.21.15.50:0/2160254648 shutdown_connections 2024-05-08T18:05:07.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.816+0000 7f759632d800 1 --2- 172.21.15.50:0/2160254648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 0x5623f7844590 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.816+0000 7f759632d800 1 -- 172.21.15.50:0/2160254648 >> 172.21.15.50:0/2160254648 conn(0x5623f783bb70 msgr2=0x5623f783df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.816+0000 7f759632d800 1 -- 172.21.15.50:0/2160254648 shutdown_connections 2024-05-08T18:05:07.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.816+0000 7f759632d800 1 -- 172.21.15.50:0/2160254648 wait complete. 2024-05-08T18:05:07.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.832+0000 7f759632d800 1 Processor -- start 2024-05-08T18:05:07.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.832+0000 7f759632d800 1 -- start start 2024-05-08T18:05:07.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.832+0000 7f759632d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 0x5623f7841ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.832+0000 7f759632d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5623f778ca30 con 0x5623f78421a0 2024-05-08T18:05:07.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.832+0000 7f7591f71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 0x5623f7841ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.832+0000 7f7591f71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 0x5623f7841ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37672/0 (socket says 172.21.15.50:37672) 2024-05-08T18:05:07.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.832+0000 7f7591f71640 1 -- 172.21.15.50:0/77568914 learned_addr learned my addr 172.21.15.50:0/77568914 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.832+0000 7f7591f71640 1 -- 172.21.15.50:0/77568914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623f78400a0 con 0x5623f78421a0 2024-05-08T18:05:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.833+0000 7f7591f71640 1 --2- 172.21.15.50:0/77568914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 0x5623f7841ae0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7584055040 tx=0x7f758404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a548f5a39ccc627e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.833+0000 7f747fe07640 1 -- 172.21.15.50:0/77568914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7584057040 con 0x5623f78421a0 2024-05-08T18:05:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.833+0000 7f747fe07640 1 -- 172.21.15.50:0/77568914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7584052720 con 0x5623f78421a0 2024-05-08T18:05:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.833+0000 7f747fe07640 1 -- 172.21.15.50:0/77568914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7584052a40 con 0x5623f78421a0 2024-05-08T18:05:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.834+0000 7f759632d800 1 -- 172.21.15.50:0/77568914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 msgr2=0x5623f7841ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.834+0000 7f759632d800 1 --2- 172.21.15.50:0/77568914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 0x5623f7841ae0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7584055040 tx=0x7f758404c000 comp rx=0 tx=0).stop 2024-05-08T18:05:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.834+0000 7f759632d800 1 -- 172.21.15.50:0/77568914 shutdown_connections 2024-05-08T18:05:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.834+0000 7f759632d800 1 --2- 172.21.15.50:0/77568914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 0x5623f7841ae0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.834+0000 7f759632d800 1 -- 172.21.15.50:0/77568914 >> 172.21.15.50:0/77568914 conn(0x5623f783bb70 msgr2=0x5623f792e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.834+0000 7f759632d800 1 -- 172.21.15.50:0/77568914 shutdown_connections 2024-05-08T18:05:07.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.834+0000 7f759632d800 1 -- 172.21.15.50:0/77568914 wait complete. 2024-05-08T18:05:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.835+0000 7f759632d800 1 Processor -- start 2024-05-08T18:05:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.835+0000 7f759632d800 1 -- start start 2024-05-08T18:05:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.835+0000 7f759632d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 0x5623f793c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.835+0000 7f759632d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5623f7848a50 con 0x5623f78421a0 2024-05-08T18:05:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.835+0000 7f7591f71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 0x5623f793c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.835+0000 7f7591f71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 0x5623f793c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37678/0 (socket says 172.21.15.50:37678) 2024-05-08T18:05:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.835+0000 7f7591f71640 1 -- 172.21.15.50:0/976763460 learned_addr learned my addr 172.21.15.50:0/976763460 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.835+0000 7f7591f71640 1 -- 172.21.15.50:0/976763460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623f793cd60 con 0x5623f78421a0 2024-05-08T18:05:07.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.836+0000 7f7591f71640 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 0x5623f793c820 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7584052540 tx=0x7f7584003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.837+0000 7f747e604640 1 -- 172.21.15.50:0/976763460 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7584073070 con 0x5623f78421a0 2024-05-08T18:05:07.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.837+0000 7f747e604640 1 -- 172.21.15.50:0/976763460 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f758405d440 con 0x5623f78421a0 2024-05-08T18:05:07.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.837+0000 7f759632d800 1 -- 172.21.15.50:0/976763460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5623f793cff0 con 0x5623f78421a0 2024-05-08T18:05:07.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.837+0000 7f747e604640 1 -- 172.21.15.50:0/976763460 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7584057070 con 0x5623f78421a0 2024-05-08T18:05:07.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.838+0000 7f759632d800 1 -- 172.21.15.50:0/976763460 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623f793fc50 con 0x5623f78421a0 2024-05-08T18:05:07.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.839+0000 7f747e604640 1 -- 172.21.15.50:0/976763460 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7584076070 con 0x5623f78421a0 2024-05-08T18:05:07.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.839+0000 7f747e604640 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f747803bbf0 0x7f747803e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.839+0000 7f747e604640 1 -- 172.21.15.50:0/976763460 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7584093e90 con 0x5623f78421a0 2024-05-08T18:05:07.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.840+0000 7f7591770640 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f747803bbf0 0x7f747803e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.840+0000 7f759632d800 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5623f7844ad0 0x5623f7943a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.840+0000 7f759632d800 1 -- 172.21.15.50:0/976763460 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f7943f50 con 0x5623f7844ad0 2024-05-08T18:05:07.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.840+0000 7f7591770640 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f747803bbf0 0x7f747803e0b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7588001e30 tx=0x7f7588001390 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.840+0000 7f7592772640 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5623f7844ad0 0x5623f7943a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.841+0000 7f7592772640 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5623f7844ad0 0x5623f7943a10 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.841+0000 7f7592772640 1 -- 172.21.15.50:0/976763460 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f7844ad0 2024-05-08T18:05:07.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.841+0000 7f759632d800 1 -- 172.21.15.50:0/976763460 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f7940fc0 con 0x5623f7844ad0 2024-05-08T18:05:07.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.841+0000 7f7592772640 1 -- 172.21.15.50:0/976763460 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f7844ad0 2024-05-08T18:05:07.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.841+0000 7f759632d800 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5623f7946220 0x5623f7948610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.841+0000 7f759632d800 1 -- 172.21.15.50:0/976763460 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f7948ce0 con 0x5623f7946220 2024-05-08T18:05:07.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.842+0000 7f7591f71640 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5623f7946220 0x5623f7948610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.842+0000 7f7591f71640 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5623f7946220 0x5623f7948610 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.843+0000 7f7591f71640 1 -- 172.21.15.50:0/976763460 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7584075030 con 0x5623f7946220 2024-05-08T18:05:07.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.843+0000 7f759632d800 1 -- 172.21.15.50:0/976763460 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f79450c0 con 0x5623f7844ad0 2024-05-08T18:05:07.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.843+0000 7f7592772640 1 -- 172.21.15.50:0/976763460 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f7844ad0 2024-05-08T18:05:07.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.843+0000 7f759632d800 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5623f794a120 0x5623f794c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.844+0000 7f759632d800 1 -- 172.21.15.50:0/976763460 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f794cc50 con 0x5623f794a120 2024-05-08T18:05:07.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.844+0000 7f7591770640 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5623f794a120 0x5623f794c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.845+0000 7f7591770640 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5623f794a120 0x5623f794c510 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.845+0000 7f7591770640 1 -- 172.21.15.50:0/976763460 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f758800bd50 con 0x5623f794a120 2024-05-08T18:05:07.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.846+0000 7f759632d800 1 Processor -- start 2024-05-08T18:05:07.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.846+0000 7f759632d800 1 -- start start 2024-05-08T18:05:07.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.846+0000 7f759632d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f795afc0 0x5623f795b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.846+0000 7f759632d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5623f7733330 con 0x5623f795afc0 2024-05-08T18:05:07.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.847+0000 7f7591f71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f795afc0 0x5623f795b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.847+0000 7f7591f71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f795afc0 0x5623f795b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37690/0 (socket says 172.21.15.50:37690) 2024-05-08T18:05:07.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.847+0000 7f7591f71640 1 -- 172.21.15.50:0/693982846 learned_addr learned my addr 172.21.15.50:0/693982846 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.847+0000 7f7591f71640 1 -- 172.21.15.50:0/693982846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623f795b920 con 0x5623f795afc0 2024-05-08T18:05:07.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.847+0000 7f7591f71640 1 --2- 172.21.15.50:0/693982846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f795afc0 0x5623f795b3e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7584006e20 tx=0x7f7584076d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.847+0000 7f74777fe640 1 -- 172.21.15.50:0/693982846 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f758405cb60 con 0x5623f795afc0 2024-05-08T18:05:07.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.847+0000 7f74777fe640 1 -- 172.21.15.50:0/693982846 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f758405cd00 con 0x5623f795afc0 2024-05-08T18:05:07.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.847+0000 7f759632d800 1 -- 172.21.15.50:0/693982846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5623f795c390 con 0x5623f795afc0 2024-05-08T18:05:07.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.847+0000 7f74777fe640 1 -- 172.21.15.50:0/693982846 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7584076360 con 0x5623f795afc0 2024-05-08T18:05:07.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.847+0000 7f759632d800 1 -- 172.21.15.50:0/693982846 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623f795c890 con 0x5623f795afc0 2024-05-08T18:05:07.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.848+0000 7f74777fe640 1 -- 172.21.15.50:0/693982846 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7584076500 con 0x5623f795afc0 2024-05-08T18:05:07.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.848+0000 7f74777fe640 1 --2- 172.21.15.50:0/693982846 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f746c03bb50 0x7f746c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.848+0000 7f74777fe640 1 -- 172.21.15.50:0/693982846 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7584083040 con 0x5623f795afc0 2024-05-08T18:05:07.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.849+0000 7f7591770640 1 --2- 172.21.15.50:0/693982846 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f746c03bb50 0x7f746c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.849+0000 7f759632d800 1 Processor -- start 2024-05-08T18:05:07.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.849+0000 7f759632d800 1 -- start start 2024-05-08T18:05:07.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.849+0000 7f7591770640 1 --2- 172.21.15.50:0/693982846 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f746c03bb50 0x7f746c03e010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x5623f79443c0 tx=0x7f758800f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.849+0000 7f759632d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f7968100 0x5623f7968520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.849+0000 7f759632d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5623f7840360 con 0x5623f7968100 2024-05-08T18:05:07.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.849+0000 7f7591f71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f7968100 0x5623f7968520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.849+0000 7f7591f71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f7968100 0x5623f7968520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37696/0 (socket says 172.21.15.50:37696) 2024-05-08T18:05:07.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.849+0000 7f7591f71640 1 -- 172.21.15.50:0/797106450 learned_addr learned my addr 172.21.15.50:0/797106450 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.850+0000 7f7591f71640 1 -- 172.21.15.50:0/797106450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623f7968af0 con 0x5623f7968100 2024-05-08T18:05:07.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.850+0000 7f7591f71640 1 --2- 172.21.15.50:0/797106450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f7968100 0x5623f7968520 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7584002f10 tx=0x7f7584093da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc8ddd8db98fa90 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.850+0000 7f74757fa640 1 -- 172.21.15.50:0/797106450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f758406cbd0 con 0x5623f7968100 2024-05-08T18:05:07.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.850+0000 7f74757fa640 1 -- 172.21.15.50:0/797106450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7584078d60 con 0x5623f7968100 2024-05-08T18:05:07.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.850+0000 7f74757fa640 1 -- 172.21.15.50:0/797106450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f758406c300 con 0x5623f7968100 2024-05-08T18:05:07.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.850+0000 7f759632d800 1 -- 172.21.15.50:0/797106450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f7968100 msgr2=0x5623f7968520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.850+0000 7f759632d800 1 --2- 172.21.15.50:0/797106450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f7968100 0x5623f7968520 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f7584002f10 tx=0x7f7584093da0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.850+0000 7f759632d800 1 -- 172.21.15.50:0/797106450 shutdown_connections 2024-05-08T18:05:07.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.850+0000 7f759632d800 1 --2- 172.21.15.50:0/797106450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f7968100 0x5623f7968520 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.850+0000 7f759632d800 1 -- 172.21.15.50:0/797106450 >> 172.21.15.50:0/797106450 conn(0x5623f7964460 msgr2=0x5623f79624c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.850+0000 7f759632d800 1 -- 172.21.15.50:0/797106450 shutdown_connections 2024-05-08T18:05:07.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.850+0000 7f759632d800 1 -- 172.21.15.50:0/797106450 wait complete. 2024-05-08T18:05:07.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.851+0000 7f759632d800 1 Processor -- start 2024-05-08T18:05:07.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.851+0000 7f759632d800 1 -- start start 2024-05-08T18:05:07.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.851+0000 7f759632d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f7968100 0x5623f796f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.851+0000 7f759632d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5623f79691a0 con 0x5623f7968100 2024-05-08T18:05:07.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.851+0000 7f7591f71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f7968100 0x5623f796f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.851+0000 7f7591f71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f7968100 0x5623f796f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37706/0 (socket says 172.21.15.50:37706) 2024-05-08T18:05:07.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.851+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 learned_addr learned my addr 172.21.15.50:0/2620754314 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.851+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623f796fa60 con 0x5623f7968100 2024-05-08T18:05:07.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.851+0000 7f7591f71640 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f7968100 0x5623f796f520 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f758409e040 tx=0x7f7584002ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.852+0000 7f74637fe640 1 -- 172.21.15.50:0/2620754314 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75840be070 con 0x5623f7968100 2024-05-08T18:05:07.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.852+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5623f796fa60 con 0x5623f7968100 2024-05-08T18:05:07.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.852+0000 7f74637fe640 1 -- 172.21.15.50:0/2620754314 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f758404c400 con 0x5623f7968100 2024-05-08T18:05:07.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.852+0000 7f74637fe640 1 -- 172.21.15.50:0/2620754314 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f758406c9f0 con 0x5623f7968100 2024-05-08T18:05:07.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.852+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623f796fa60 con 0x5623f7968100 2024-05-08T18:05:07.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.852+0000 7f74637fe640 1 -- 172.21.15.50:0/2620754314 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f75840c1070 con 0x5623f7968100 2024-05-08T18:05:07.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.853+0000 7f74637fe640 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f745c03bba0 0x7f745c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.853+0000 7f74637fe640 1 -- 172.21.15.50:0/2620754314 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f75840bacd0 con 0x5623f7968100 2024-05-08T18:05:07.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.853+0000 7f7591770640 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f745c03bba0 0x7f745c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.853+0000 7f7591770640 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f745c03bba0 0x7f745c03e060 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5623f794d050 tx=0x7f7588010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.853+0000 7f759632d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.854+0000 7f759632d800 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5623f797acd0 0x5623f797d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.854+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f797d8e0 con 0x5623f797acd0 2024-05-08T18:05:07.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.854+0000 7f7592772640 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5623f797acd0 0x5623f797d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.854+0000 7f7592772640 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5623f797acd0 0x5623f797d190 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.854+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f797acd0 2024-05-08T18:05:07.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.854+0000 7f759632d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.854+0000 7f759632d800 20 realm 2024-05-08T18:05:07.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.854+0000 7f759632d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.854+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f797dcf0 con 0x5623f797acd0 2024-05-08T18:05:07.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.855+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f797acd0 2024-05-08T18:05:07.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.855+0000 7f759632d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.855+0000 7f759632d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:07.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.855+0000 7f759632d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.855+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e5c0 con 0x5623f797acd0 2024-05-08T18:05:07.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.855+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f797acd0 2024-05-08T18:05:07.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.855+0000 7f759632d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.855+0000 7f759632d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.855+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797acd0 2024-05-08T18:05:07.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.855+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f797acd0 2024-05-08T18:05:07.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.855+0000 7f759632d800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:07.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.855+0000 7f759632d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.855+0000 7f759632d800 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5623f797f6c0 0x5623f7981ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.855+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f79820c0 con 0x5623f797f6c0 2024-05-08T18:05:07.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.855+0000 7f7591f71640 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5623f797f6c0 0x5623f7981ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.856+0000 7f7591f71640 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5623f797f6c0 0x5623f7981ab0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.856+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.856+0000 7f759632d800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:07.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.856+0000 7f759632d800 20 searching for the correct realm 2024-05-08T18:05:07.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.856+0000 7f759632d800 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5623f7984070 0x5623f7986530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.856+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f7986c00 con 0x5623f7984070 2024-05-08T18:05:07.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.856+0000 7f7591770640 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5623f7984070 0x5623f7986530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.856+0000 7f7591770640 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5623f7984070 0x5623f7986530 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.857+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7588037040 con 0x5623f7984070 2024-05-08T18:05:07.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.857+0000 7f7591770640 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f758800f070 0x7f758802e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.857+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f7986c00 con 0x7f758800f070 2024-05-08T18:05:07.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.857+0000 7f7592772640 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f758800f070 0x7f758802e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.857+0000 7f7592772640 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f758800f070 0x7f758802e910 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.857+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f758c00a4a0 con 0x7f758800f070 2024-05-08T18:05:07.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.857+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797f6c0 2024-05-08T18:05:07.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.858+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.858+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797acd0 2024-05-08T18:05:07.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.858+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f797acd0 2024-05-08T18:05:07.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.858+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797acd0 2024-05-08T18:05:07.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.858+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f797acd0 2024-05-08T18:05:07.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.858+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797f6c0 2024-05-08T18:05:07.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.859+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.859+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797f6c0 2024-05-08T18:05:07.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.859+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.859+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x7f758800f070 2024-05-08T18:05:07.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.859+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f758c00a4a0 con 0x7f758800f070 2024-05-08T18:05:07.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.859+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f7984070 2024-05-08T18:05:07.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.859+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7588037040 con 0x5623f7984070 2024-05-08T18:05:07.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.859+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797acd0 2024-05-08T18:05:07.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.860+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f797acd0 2024-05-08T18:05:07.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.860+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797f6c0 2024-05-08T18:05:07.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.860+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.860+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797acd0 2024-05-08T18:05:07.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.860+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f797acd0 2024-05-08T18:05:07.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.860+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x7f758800f070 2024-05-08T18:05:07.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.860+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f758c00a4a0 con 0x7f758800f070 2024-05-08T18:05:07.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.860+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f7984070 2024-05-08T18:05:07.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.861+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7588037040 con 0x5623f7984070 2024-05-08T18:05:07.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.861+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f7984070 2024-05-08T18:05:07.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.861+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7588037040 con 0x5623f7984070 2024-05-08T18:05:07.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.861+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797f6c0 2024-05-08T18:05:07.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.861+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.861+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f7984070 2024-05-08T18:05:07.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.861+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7588037040 con 0x5623f7984070 2024-05-08T18:05:07.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.861+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f7984070 2024-05-08T18:05:07.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.862+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7588037040 con 0x5623f7984070 2024-05-08T18:05:07.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.862+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797acd0 2024-05-08T18:05:07.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.862+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f797acd0 2024-05-08T18:05:07.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.862+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797acd0 2024-05-08T18:05:07.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.862+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f797acd0 2024-05-08T18:05:07.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.862+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797f6c0 2024-05-08T18:05:07.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.862+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.862+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f7984070 2024-05-08T18:05:07.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.863+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7588037040 con 0x5623f7984070 2024-05-08T18:05:07.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.863+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x7f758800f070 2024-05-08T18:05:07.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.863+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f758c00a4a0 con 0x7f758800f070 2024-05-08T18:05:07.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.863+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797acd0 2024-05-08T18:05:07.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.863+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f797acd0 2024-05-08T18:05:07.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.863+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797f6c0 2024-05-08T18:05:07.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.863+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.863+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f7984070 2024-05-08T18:05:07.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.864+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7588037040 con 0x5623f7984070 2024-05-08T18:05:07.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.864+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797f6c0 2024-05-08T18:05:07.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.864+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.864+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x7f758800f070 2024-05-08T18:05:07.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.864+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f758c00a4a0 con 0x7f758800f070 2024-05-08T18:05:07.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.864+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f7984070 2024-05-08T18:05:07.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.864+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7588037040 con 0x5623f7984070 2024-05-08T18:05:07.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.864+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797f6c0 2024-05-08T18:05:07.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x5623f797acd0 2024-05-08T18:05:07.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f797acd0 2024-05-08T18:05:07.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5623f797e960 con 0x7f758800f070 2024-05-08T18:05:07.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f758c00a4a0 con 0x7f758800f070 2024-05-08T18:05:07.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f759632d800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:07.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f759632d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:07.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f759632d800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:07.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f759632d800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:07.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f759632d800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:07.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f759632d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:07.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f759632d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f7982980 con 0x5623f797acd0 2024-05-08T18:05:07.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f797acd0 2024-05-08T18:05:07.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f759632d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f759632d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.865+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f7982980 con 0x5623f797acd0 2024-05-08T18:05:07.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f758c00a4a0 con 0x5623f797acd0 2024-05-08T18:05:07.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f759632d800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:07.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f759632d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f7982980 con 0x5623f7984070 2024-05-08T18:05:07.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7588037040 con 0x5623f7984070 2024-05-08T18:05:07.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f759632d800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:07.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f759632d800 20 zone default found 2024-05-08T18:05:07.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f759632d800 4 Realm: () 2024-05-08T18:05:07.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f759632d800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:07.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f759632d800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:07.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f759632d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:07.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f759632d800 20 zonegroup default 2024-05-08T18:05:07.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f759632d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f79832c0 con 0x5623f797f6c0 2024-05-08T18:05:07.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f759632d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f759632d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.866+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f7988a90 con 0x5623f7984070 2024-05-08T18:05:07.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.867+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7588037040 con 0x5623f7984070 2024-05-08T18:05:07.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.867+0000 7f759632d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.867+0000 7f759632d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.867+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f7988f80 con 0x5623f7984070 2024-05-08T18:05:07.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.867+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7588037040 con 0x5623f7984070 2024-05-08T18:05:07.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.867+0000 7f759632d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:07.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.867+0000 7f759632d800 20 started sync module instance, tier type = 2024-05-08T18:05:07.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.867+0000 7f759632d800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:07.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.867+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f7987f00 con 0x7f758800f070 2024-05-08T18:05:07.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.868+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f758c00a4a0 con 0x7f758800f070 2024-05-08T18:05:07.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.868+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94712477811712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f7987f00 con 0x7f758800f070 2024-05-08T18:05:07.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.868+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f7989a80 con 0x5623f797acd0 2024-05-08T18:05:07.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.869+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94712477811712] v27'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f758c040040 con 0x7f758800f070 2024-05-08T18:05:07.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.869+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f758c040040 con 0x5623f797acd0 2024-05-08T18:05:07.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.869+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94712477816144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f7987f00 con 0x5623f797acd0 2024-05-08T18:05:07.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.869+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f798a500 con 0x5623f7984070 2024-05-08T18:05:07.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.870+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7588037040 con 0x5623f7984070 2024-05-08T18:05:07.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.870+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94712477818896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f798add0 con 0x5623f7984070 2024-05-08T18:05:07.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.870+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94712477816144] v27'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f758c040040 con 0x5623f797acd0 2024-05-08T18:05:07.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.870+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f7987a50 con 0x7f758800f070 2024-05-08T18:05:07.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.871+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f758c040040 con 0x7f758800f070 2024-05-08T18:05:07.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.871+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94712477823792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f798bc00 con 0x7f758800f070 2024-05-08T18:05:07.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.871+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94712477818896] v27'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7588037040 con 0x5623f7984070 2024-05-08T18:05:07.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.871+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f798c260 con 0x5623f797acd0 2024-05-08T18:05:07.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.872+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94712477823792] v27'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f758c040040 con 0x7f758800f070 2024-05-08T18:05:07.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.873+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f758c040040 con 0x5623f797acd0 2024-05-08T18:05:07.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.873+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94712477824848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f798bd70 con 0x5623f797acd0 2024-05-08T18:05:07.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.873+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f798cb30 con 0x5623f797f6c0 2024-05-08T18:05:07.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.874+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94712477824848] v27'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f758c040040 con 0x5623f797acd0 2024-05-08T18:05:07.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.874+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.874+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94712477828736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f798cf20 con 0x5623f797f6c0 2024-05-08T18:05:07.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.874+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f798d8e0 con 0x5623f797f6c0 2024-05-08T18:05:07.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.875+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94712477828736] v27'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.875+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.875+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94712477832240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f798dd00 con 0x5623f797f6c0 2024-05-08T18:05:07.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.875+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f798e6c0 con 0x5623f797f6c0 2024-05-08T18:05:07.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.876+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94712477832240] v27'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.876+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.876+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94712477833776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f798e6c0 con 0x5623f797f6c0 2024-05-08T18:05:07.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.876+0000 7f759632d800 20 add_watcher() i=0 2024-05-08T18:05:07.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.876+0000 7f759632d800 20 add_watcher() i=1 2024-05-08T18:05:07.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.876+0000 7f759632d800 20 add_watcher() i=2 2024-05-08T18:05:07.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.876+0000 7f759632d800 20 add_watcher() i=3 2024-05-08T18:05:07.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.876+0000 7f759632d800 20 add_watcher() i=4 2024-05-08T18:05:07.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.876+0000 7f759632d800 20 add_watcher() i=5 2024-05-08T18:05:07.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.876+0000 7f759632d800 20 add_watcher() i=6 2024-05-08T18:05:07.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.877+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94712477833776] v27'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.877+0000 7f759632d800 20 add_watcher() i=7 2024-05-08T18:05:07.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.877+0000 7f759632d800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:07.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.877+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f7991a30 con 0x5623f797acd0 2024-05-08T18:05:07.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.877+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f758c040040 con 0x5623f797acd0 2024-05-08T18:05:07.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.877+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94712477836160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f798ec50 con 0x5623f797acd0 2024-05-08T18:05:07.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.878+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94712477836160] v27'104 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f758c040040 con 0x5623f797acd0 2024-05-08T18:05:07.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.878+0000 7f74437fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:07.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.879+0000 7f759632d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:07.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.879+0000 7f759632d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:07.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.879+0000 7f759632d800 5 note: GC not initialized 2024-05-08T18:05:07.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.879+0000 7f74427fc640 20 reqs_thread_entry: start 2024-05-08T18:05:07.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.879+0000 7f759632d800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:07.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.879+0000 7f759632d800 20 Filter name: none 2024-05-08T18:05:07.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.879+0000 7f73b3fff640 20 reqs_thread_entry: start 2024-05-08T18:05:07.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.879+0000 7f759632d800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:07.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.879+0000 7f759632d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.879+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f79c4fb0 con 0x5623f797acd0 2024-05-08T18:05:07.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.880+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f758c040040 con 0x5623f797acd0 2024-05-08T18:05:07.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.880+0000 7f759632d800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:07.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.880+0000 7f759632d800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:07.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.880+0000 7f759632d800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:07.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.880+0000 7f759632d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:07.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.880+0000 7f759632d800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-05-08T18:05:07.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.880+0000 7f759632d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.880+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f79c7090 con 0x7f758800f070 2024-05-08T18:05:07.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.880+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f758c040040 con 0x7f758800f070 2024-05-08T18:05:07.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.880+0000 7f759632d800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:07.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.880+0000 7f759632d800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-05-08T18:05:07.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.880+0000 7f759632d800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:07.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.880+0000 7f759632d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 : miss 2024-05-08T18:05:07.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.880+0000 7f759632d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.880+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:64 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f79c8870 con 0x5623f797f6c0 2024-05-08T18:05:07.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.881+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(64 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.881+0000 7f759632d800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:07.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.881+0000 7f759632d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:07.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.881+0000 7f759632d800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:07.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.881+0000 7f759632d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:07.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.881+0000 7f759632d800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:07.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.881+0000 7f759632d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 2024-05-08T18:05:07.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.881+0000 7f759632d800 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-05-08T18:05:07.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.881+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f79c9b90 con 0x7f758800f070 2024-05-08T18:05:07.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.882+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(65 foo.buckets [call] v27'19 uv19 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f758c040040 con 0x7f758800f070 2024-05-08T18:05:07.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.882+0000 7f759632d800 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x11) 2024-05-08T18:05:07.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.882+0000 7f759632d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.882+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f79c96e0 con 0x7f758800f070 2024-05-08T18:05:07.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.882+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(66 myfoo [call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 233+0+288 (crc 0 0 0) 0x7f758c040040 con 0x7f758800f070 2024-05-08T18:05:07.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.882+0000 7f759632d800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:07.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.882+0000 7f759632d800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2024-05-08T18:05:07.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.882+0000 7f759632d800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:07.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.882+0000 7f759632d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:07.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.882+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f79cfbc0 con 0x7f758800f070 2024-05-08T18:05:07.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.883+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(67 myfoo [delete,create,call,call,writefull 0~165] v27'20 uv20 ondisk = 0) ==== 317+0+0 (crc 0 0 0) 0x7f758c040040 con 0x7f758800f070 2024-05-08T18:05:07.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.883+0000 7f759632d800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-05-08T18:05:07.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.883+0000 7f759632d800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:07.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.883+0000 7f759632d800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-05-08T18:05:07.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.883+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94712478086864 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5623f79cd0f0 con 0x5623f797f6c0 2024-05-08T18:05:07.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.884+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== watch-notify(notify (1) cookie 94712477832240 notify 115964116994 ret 0) ==== 380+0+0 (crc 0 0 0) 0x5623f79c7d90 con 0x5623f797f6c0 2024-05-08T18:05:07.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.884+0000 7f74757fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116994 cookie 94712477832240 notifier 4473 bl.length()=338 2024-05-08T18:05:07.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.884+0000 7f74757fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-05-08T18:05:07.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.884+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(68 notify.6 [notify cookie 94712478086864 out=8b] v0'0 uv105 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.884+0000 7f74757fa640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:07.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.884+0000 7f74757fa640 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f7454003ea0 con 0x5623f797f6c0 2024-05-08T18:05:07.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.884+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.884+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== watch-notify(notify_complete (2) cookie 94712478086864 notify 115964116994 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5623f795b920 con 0x5623f797f6c0 2024-05-08T18:05:07.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.885+0000 7f74437fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:07.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.885+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94712477836160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f79ad8d0 con 0x5623f797acd0 2024-05-08T18:05:07.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.886+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94712477836160] v27'105 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f758c040040 con 0x5623f797acd0 2024-05-08T18:05:07.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.886+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94712477811712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f79ad8d0 con 0x7f758800f070 2024-05-08T18:05:07.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.887+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94712477811712] v27'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f758c040040 con 0x7f758800f070 2024-05-08T18:05:07.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.887+0000 7f759632d800 20 remove_watcher() i=0 2024-05-08T18:05:07.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.887+0000 7f759632d800 2 removed watcher, disabling cache 2024-05-08T18:05:07.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.887+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94712477816144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f79ad8d0 con 0x5623f797acd0 2024-05-08T18:05:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.888+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94712477816144] v27'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f758c040040 con 0x5623f797acd0 2024-05-08T18:05:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.888+0000 7f759632d800 20 remove_watcher() i=1 2024-05-08T18:05:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.888+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94712477818896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f79ad8d0 con 0x5623f7984070 2024-05-08T18:05:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.889+0000 7f7591770640 1 -- 172.21.15.50:0/2620754314 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94712477818896] v27'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7588037040 con 0x5623f7984070 2024-05-08T18:05:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.889+0000 7f759632d800 20 remove_watcher() i=2 2024-05-08T18:05:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.889+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94712477823792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f7988400 con 0x7f758800f070 2024-05-08T18:05:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.890+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94712477823792] v27'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f758c040040 con 0x7f758800f070 2024-05-08T18:05:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.890+0000 7f759632d800 20 remove_watcher() i=3 2024-05-08T18:05:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.890+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94712477824848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f7989550 con 0x5623f797acd0 2024-05-08T18:05:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.891+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94712477824848] v27'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f758c040040 con 0x5623f797acd0 2024-05-08T18:05:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.891+0000 7f759632d800 20 remove_watcher() i=4 2024-05-08T18:05:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.891+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94712477828736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f798a010 con 0x5623f797f6c0 2024-05-08T18:05:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.892+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94712477828736] v27'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.892+0000 7f759632d800 20 remove_watcher() i=5 2024-05-08T18:05:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.892+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94712477832240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f7988400 con 0x5623f797f6c0 2024-05-08T18:05:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.893+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94712477832240] v27'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.893+0000 7f759632d800 20 remove_watcher() i=6 2024-05-08T18:05:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.893+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94712477833776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5623f7989550 con 0x5623f797f6c0 2024-05-08T18:05:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f7591f71640 1 -- 172.21.15.50:0/2620754314 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94712477833776] v27'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75840c0030 con 0x5623f797f6c0 2024-05-08T18:05:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 20 remove_watcher() i=7 2024-05-08T18:05:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5623f797f6c0 msgr2=0x5623f7981ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5623f797f6c0 0x5623f7981ab0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5623f797acd0 msgr2=0x5623f797d190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5623f797acd0 0x5623f797d190 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f758800f070 msgr2=0x7f758802e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f758800f070 0x7f758802e910 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5623f7984070 msgr2=0x5623f7986530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5623f7984070 0x5623f7986530 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f745c03bba0 msgr2=0x7f745c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f745c03bba0 0x7f745c03e060 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x5623f794d050 tx=0x7f7588010000 comp rx=0 tx=0).stop 2024-05-08T18:05:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f7968100 msgr2=0x5623f796f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 1 --2- 172.21.15.50:0/2620754314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f7968100 0x5623f796f520 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f758409e040 tx=0x7f7584002ea0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 reap_dead start 2024-05-08T18:05:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f7592772640 1 -- 172.21.15.50:0/2620754314 reap_dead start 2024-05-08T18:05:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 shutdown_connections 2024-05-08T18:05:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 >> 172.21.15.50:0/2620754314 conn(0x5623f7964460 msgr2=0x5623f7961ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 shutdown_connections 2024-05-08T18:05:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.894+0000 7f759632d800 1 -- 172.21.15.50:0/2620754314 wait complete. 2024-05-08T18:05:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 -- 172.21.15.50:0/693982846 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f746c03bb50 msgr2=0x7f746c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 --2- 172.21.15.50:0/693982846 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f746c03bb50 0x7f746c03e010 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x5623f79443c0 tx=0x7f758800f000 comp rx=0 tx=0).stop 2024-05-08T18:05:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 -- 172.21.15.50:0/693982846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f795afc0 msgr2=0x5623f795b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 --2- 172.21.15.50:0/693982846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f795afc0 0x5623f795b3e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7584006e20 tx=0x7f7584076d60 comp rx=0 tx=0).stop 2024-05-08T18:05:07.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 -- 172.21.15.50:0/693982846 shutdown_connections 2024-05-08T18:05:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 --2- 172.21.15.50:0/693982846 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f746c03bb50 0x7f746c03e010 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 --2- 172.21.15.50:0/693982846 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f795afc0 0x5623f795b3e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 -- 172.21.15.50:0/693982846 >> 172.21.15.50:0/693982846 conn(0x5623f794f060 msgr2=0x5623f794f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 -- 172.21.15.50:0/693982846 shutdown_connections 2024-05-08T18:05:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 -- 172.21.15.50:0/693982846 wait complete. 2024-05-08T18:05:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 -- 172.21.15.50:0/976763460 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5623f7946220 msgr2=0x5623f7948610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5623f7946220 0x5623f7948610 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 -- 172.21.15.50:0/976763460 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5623f7844ad0 msgr2=0x5623f7943a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5623f7844ad0 0x5623f7943a10 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 -- 172.21.15.50:0/976763460 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5623f794a120 msgr2=0x5623f794c510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5623f794a120 0x5623f794c510 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 -- 172.21.15.50:0/976763460 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f747803bbf0 msgr2=0x7f747803e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f747803bbf0 0x7f747803e0b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7588001e30 tx=0x7f7588001390 comp rx=0 tx=0).stop 2024-05-08T18:05:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 -- 172.21.15.50:0/976763460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 msgr2=0x5623f793c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 --2- 172.21.15.50:0/976763460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5623f78421a0 0x5623f793c820 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7584052540 tx=0x7f7584003020 comp rx=0 tx=0).stop 2024-05-08T18:05:07.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f7592772640 1 -- 172.21.15.50:0/976763460 reap_dead start 2024-05-08T18:05:07.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 -- 172.21.15.50:0/976763460 shutdown_connections 2024-05-08T18:05:07.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 -- 172.21.15.50:0/976763460 >> 172.21.15.50:0/976763460 conn(0x5623f783bb70 msgr2=0x5623f78408c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 -- 172.21.15.50:0/976763460 shutdown_connections 2024-05-08T18:05:07.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.895+0000 7f759632d800 1 -- 172.21.15.50:0/976763460 wait complete. 2024-05-08T18:05:07.903 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-05-08T18:05:07.903 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-05-08T18:05:07.903 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2024-05-08T18:05:07.956 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:07.956 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:07.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.959+0000 7f3f0331f800 1 Processor -- start 2024-05-08T18:05:07.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.959+0000 7f3f0331f800 1 -- start start 2024-05-08T18:05:07.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.959+0000 7f3f0331f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x555a1553ac30 0x555a1553d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.959+0000 7f3f0331f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x555a1553d560 con 0x555a15537120 2024-05-08T18:05:07.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.959+0000 7f3f0331f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x555a153d6e20 con 0x555a1553ac30 2024-05-08T18:05:07.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.960+0000 7f3efe7df640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x555a1553ac30 0x555a1553d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.960+0000 7f3efe7df640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x555a1553ac30 0x555a1553d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37720/0 (socket says 172.21.15.50:37720) 2024-05-08T18:05:07.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.960+0000 7f3efe7df640 1 -- 172.21.15.50:0/3103778123 learned_addr learned my addr 172.21.15.50:0/3103778123 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.960+0000 7f3efefe0640 1 --1- 172.21.15.50:0/3103778123 >> v1:172.21.15.46:6789/0 conn(0x555a15537120 0x555a15539510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43158/0 (socket says 172.21.15.50:43158) 2024-05-08T18:05:07.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.960+0000 7f3efe7df640 1 -- 172.21.15.50:0/3103778123 >> v1:172.21.15.46:6789/0 conn(0x555a15537120 legacy=0x555a15539510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.960+0000 7f3efe7df640 1 -- 172.21.15.50:0/3103778123 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1553e740 con 0x555a1553ac30 2024-05-08T18:05:07.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.962+0000 7f3efe7df640 1 --2- 172.21.15.50:0/3103778123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1553ac30 0x555a1553d020 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400a520 tx=0x7f3ef401d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ec3ce79177793c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.962+0000 7f3efdfde640 1 -- 172.21.15.50:0/3103778123 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef4057070 con 0x555a1553ac30 2024-05-08T18:05:07.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.962+0000 7f3efdfde640 1 -- 172.21.15.50:0/3103778123 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef4052570 con 0x555a1553ac30 2024-05-08T18:05:07.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.962+0000 7f3efdfde640 1 -- 172.21.15.50:0/3103778123 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef404ae60 con 0x555a1553ac30 2024-05-08T18:05:07.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.962+0000 7f3f0331f800 1 -- 172.21.15.50:0/3103778123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1553ac30 msgr2=0x555a1553d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.962+0000 7f3f0331f800 1 --2- 172.21.15.50:0/3103778123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1553ac30 0x555a1553d020 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400a520 tx=0x7f3ef401d010 comp rx=0 tx=0).stop 2024-05-08T18:05:07.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.962+0000 7f3f0331f800 1 -- 172.21.15.50:0/3103778123 shutdown_connections 2024-05-08T18:05:07.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.962+0000 7f3f0331f800 1 --2- 172.21.15.50:0/3103778123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1553ac30 0x555a1553d020 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.962+0000 7f3f0331f800 1 -- 172.21.15.50:0/3103778123 >> 172.21.15.50:0/3103778123 conn(0x555a15530b90 msgr2=0x555a15532fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.963+0000 7f3f0331f800 1 -- 172.21.15.50:0/3103778123 shutdown_connections 2024-05-08T18:05:07.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.963+0000 7f3f0331f800 1 -- 172.21.15.50:0/3103778123 wait complete. 2024-05-08T18:05:07.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.978+0000 7f3f0331f800 1 Processor -- start 2024-05-08T18:05:07.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.978+0000 7f3f0331f800 1 -- start start 2024-05-08T18:05:07.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.978+0000 7f3f0331f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a15537120 0x555a15627c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.978+0000 7f3f0331f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555a15481a50 con 0x555a15537120 2024-05-08T18:05:07.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.978+0000 7f3efefe0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a15537120 0x555a15627c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.978+0000 7f3efefe0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a15537120 0x555a15627c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37730/0 (socket says 172.21.15.50:37730) 2024-05-08T18:05:07.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.978+0000 7f3efefe0640 1 -- 172.21.15.50:0/2288151327 learned_addr learned my addr 172.21.15.50:0/2288151327 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.979+0000 7f3efefe0640 1 -- 172.21.15.50:0/2288151327 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a15628200 con 0x555a15537120 2024-05-08T18:05:07.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.979+0000 7f3efefe0640 1 --2- 172.21.15.50:0/2288151327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a15537120 0x555a15627c30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00016e0 tx=0x7f3ef0001ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d5b86f2b28ca45 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.979+0000 7f3dece09640 1 -- 172.21.15.50:0/2288151327 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef0013420 con 0x555a15537120 2024-05-08T18:05:07.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.979+0000 7f3dece09640 1 -- 172.21.15.50:0/2288151327 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef00135c0 con 0x555a15537120 2024-05-08T18:05:07.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.979+0000 7f3dece09640 1 -- 172.21.15.50:0/2288151327 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef00138e0 con 0x555a15537120 2024-05-08T18:05:07.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.980+0000 7f3f0331f800 1 -- 172.21.15.50:0/2288151327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a15537120 msgr2=0x555a15627c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.980+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2288151327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a15537120 0x555a15627c30 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00016e0 tx=0x7f3ef0001ba0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.980+0000 7f3f0331f800 1 -- 172.21.15.50:0/2288151327 shutdown_connections 2024-05-08T18:05:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.980+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2288151327 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a15537120 0x555a15627c30 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.980+0000 7f3f0331f800 1 -- 172.21.15.50:0/2288151327 >> 172.21.15.50:0/2288151327 conn(0x555a15530b90 msgr2=0x555a15623550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.980+0000 7f3f0331f800 1 -- 172.21.15.50:0/2288151327 shutdown_connections 2024-05-08T18:05:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.980+0000 7f3f0331f800 1 -- 172.21.15.50:0/2288151327 wait complete. 2024-05-08T18:05:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.980+0000 7f3f0331f800 1 Processor -- start 2024-05-08T18:05:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.980+0000 7f3f0331f800 1 -- start start 2024-05-08T18:05:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.980+0000 7f3f0331f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a15537120 0x555a1562f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.980+0000 7f3f0331f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555a1553da30 con 0x555a15537120 2024-05-08T18:05:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.980+0000 7f3efefe0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a15537120 0x555a1562f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.980+0000 7f3efefe0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a15537120 0x555a1562f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37736/0 (socket says 172.21.15.50:37736) 2024-05-08T18:05:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.980+0000 7f3efefe0640 1 -- 172.21.15.50:0/2866349511 learned_addr learned my addr 172.21.15.50:0/2866349511 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.981+0000 7f3efefe0640 1 -- 172.21.15.50:0/2866349511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1562fd70 con 0x555a15537120 2024-05-08T18:05:07.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.981+0000 7f3efefe0640 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a15537120 0x555a1562f830 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00065f0 tx=0x7f3ef0001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.981+0000 7f3deb606640 1 -- 172.21.15.50:0/2866349511 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef0022020 con 0x555a15537120 2024-05-08T18:05:07.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.981+0000 7f3deb606640 1 -- 172.21.15.50:0/2866349511 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef00067e0 con 0x555a15537120 2024-05-08T18:05:07.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.981+0000 7f3f0331f800 1 -- 172.21.15.50:0/2866349511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555a15630000 con 0x555a15537120 2024-05-08T18:05:07.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.981+0000 7f3deb606640 1 -- 172.21.15.50:0/2866349511 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef0004bb0 con 0x555a15537120 2024-05-08T18:05:07.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.981+0000 7f3f0331f800 1 -- 172.21.15.50:0/2866349511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a15632c60 con 0x555a15537120 2024-05-08T18:05:07.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.982+0000 7f3deb606640 1 -- 172.21.15.50:0/2866349511 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3ef0025020 con 0x555a15537120 2024-05-08T18:05:07.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.982+0000 7f3deb606640 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3de403bb50 0x7f3de403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.982+0000 7f3deb606640 1 -- 172.21.15.50:0/2866349511 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f3ef0018030 con 0x555a15537120 2024-05-08T18:05:07.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.982+0000 7f3efe7df640 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3de403bb50 0x7f3de403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.982+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555a1553ac30 0x555a15636a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.982+0000 7f3f0331f800 1 -- 172.21.15.50:0/2866349511 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a15636fb0 con 0x555a1553ac30 2024-05-08T18:05:07.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.983+0000 7f3eff7e1640 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555a1553ac30 0x555a15636a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.983+0000 7f3efe7df640 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3de403bb50 0x7f3de403e010 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4054000 tx=0x7f3ef404e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.983+0000 7f3eff7e1640 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555a1553ac30 0x555a15636a20 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.983+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2866349511 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3ef80083e0 con 0x555a1553ac30 2024-05-08T18:05:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.983+0000 7f3f0331f800 1 -- 172.21.15.50:0/2866349511 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a15633fd0 con 0x555a1553ac30 2024-05-08T18:05:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.983+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2866349511 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3ef80083e0 con 0x555a1553ac30 2024-05-08T18:05:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.984+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555a156392e0 0x555a1563b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.984+0000 7f3f0331f800 1 -- 172.21.15.50:0/2866349511 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a1563bdc0 con 0x555a156392e0 2024-05-08T18:05:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.984+0000 7f3efefe0640 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555a156392e0 0x555a1563b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.984+0000 7f3efefe0640 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555a156392e0 0x555a1563b6d0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.984+0000 7f3efefe0640 1 -- 172.21.15.50:0/2866349511 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f3ef001b070 con 0x555a156392e0 2024-05-08T18:05:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.984+0000 7f3f0331f800 1 -- 172.21.15.50:0/2866349511 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a15638150 con 0x555a1553ac30 2024-05-08T18:05:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.985+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2866349511 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3ef80083e0 con 0x555a1553ac30 2024-05-08T18:05:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.985+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555a1563d220 0x555a1563f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.985+0000 7f3f0331f800 1 -- 172.21.15.50:0/2866349511 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a1563fd00 con 0x555a1563d220 2024-05-08T18:05:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.985+0000 7f3efe7df640 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555a1563d220 0x555a1563f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.985+0000 7f3efe7df640 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555a1563d220 0x555a1563f610 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.985+0000 7f3efe7df640 1 -- 172.21.15.50:0/2866349511 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3ef405a040 con 0x555a1563d220 2024-05-08T18:05:07.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.986+0000 7f3f0331f800 1 Processor -- start 2024-05-08T18:05:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.986+0000 7f3f0331f800 1 -- start start 2024-05-08T18:05:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.986+0000 7f3f0331f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1564e070 0x555a1564e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.986+0000 7f3f0331f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555a15428330 con 0x555a1564e070 2024-05-08T18:05:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.986+0000 7f3efefe0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1564e070 0x555a1564e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.986+0000 7f3efefe0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1564e070 0x555a1564e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37750/0 (socket says 172.21.15.50:37750) 2024-05-08T18:05:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.986+0000 7f3efefe0640 1 -- 172.21.15.50:0/3377460367 learned_addr learned my addr 172.21.15.50:0/3377460367 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.986+0000 7f3efefe0640 1 -- 172.21.15.50:0/3377460367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1564e9d0 con 0x555a1564e070 2024-05-08T18:05:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.987+0000 7f3efefe0640 1 --2- 172.21.15.50:0/3377460367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1564e070 0x555a1564e490 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00510a0 tx=0x7f3ef0010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.987+0000 7f3de8e01640 1 -- 172.21.15.50:0/3377460367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef003d990 con 0x555a1564e070 2024-05-08T18:05:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.987+0000 7f3de8e01640 1 -- 172.21.15.50:0/3377460367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef003db30 con 0x555a1564e070 2024-05-08T18:05:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.987+0000 7f3de8e01640 1 -- 172.21.15.50:0/3377460367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef003de10 con 0x555a1564e070 2024-05-08T18:05:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.987+0000 7f3f0331f800 1 -- 172.21.15.50:0/3377460367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555a1564f440 con 0x555a1564e070 2024-05-08T18:05:07.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.987+0000 7f3f0331f800 1 -- 172.21.15.50:0/3377460367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a1564f940 con 0x555a1564e070 2024-05-08T18:05:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.987+0000 7f3de8e01640 1 -- 172.21.15.50:0/3377460367 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3ef0010070 con 0x555a1564e070 2024-05-08T18:05:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.988+0000 7f3de8e01640 1 --2- 172.21.15.50:0/3377460367 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dd803bb50 0x7f3dd803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.988+0000 7f3de8e01640 1 -- 172.21.15.50:0/3377460367 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f3ef0031c50 con 0x555a1564e070 2024-05-08T18:05:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.988+0000 7f3efe7df640 1 --2- 172.21.15.50:0/3377460367 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dd803bb50 0x7f3dd803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.988+0000 7f3f0331f800 1 Processor -- start 2024-05-08T18:05:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.988+0000 7f3f0331f800 1 -- start start 2024-05-08T18:05:07.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.988+0000 7f3efe7df640 1 --2- 172.21.15.50:0/3377460367 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dd803bb50 0x7f3dd803e010 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x555a15637480 tx=0x7f3ef4005ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.988+0000 7f3f0331f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1565b1f0 0x555a1565b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.988+0000 7f3f0331f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555a15628580 con 0x555a1565b1f0 2024-05-08T18:05:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.989+0000 7f3efefe0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1565b1f0 0x555a1565b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.989+0000 7f3efefe0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1565b1f0 0x555a1565b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37756/0 (socket says 172.21.15.50:37756) 2024-05-08T18:05:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.989+0000 7f3efefe0640 1 -- 172.21.15.50:0/3786330635 learned_addr learned my addr 172.21.15.50:0/3786330635 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.989+0000 7f3efefe0640 1 -- 172.21.15.50:0/3786330635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a1565bbe0 con 0x555a1565b1f0 2024-05-08T18:05:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.989+0000 7f3efefe0640 1 --2- 172.21.15.50:0/3786330635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1565b1f0 0x555a1565b610 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0004700 tx=0x7f3ef0039800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55378d840f7ad22a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.989+0000 7f3de27fc640 1 -- 172.21.15.50:0/3786330635 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef0051450 con 0x555a1565b1f0 2024-05-08T18:05:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.989+0000 7f3de27fc640 1 -- 172.21.15.50:0/3786330635 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef00399c0 con 0x555a1565b1f0 2024-05-08T18:05:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.989+0000 7f3de27fc640 1 -- 172.21.15.50:0/3786330635 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef00275e0 con 0x555a1565b1f0 2024-05-08T18:05:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.990+0000 7f3f0331f800 1 -- 172.21.15.50:0/3786330635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1565b1f0 msgr2=0x555a1565b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.990+0000 7f3f0331f800 1 --2- 172.21.15.50:0/3786330635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1565b1f0 0x555a1565b610 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0004700 tx=0x7f3ef0039800 comp rx=0 tx=0).stop 2024-05-08T18:05:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.990+0000 7f3f0331f800 1 -- 172.21.15.50:0/3786330635 shutdown_connections 2024-05-08T18:05:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.990+0000 7f3f0331f800 1 --2- 172.21.15.50:0/3786330635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1565b1f0 0x555a1565b610 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:07.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.990+0000 7f3f0331f800 1 -- 172.21.15.50:0/3786330635 >> 172.21.15.50:0/3786330635 conn(0x555a156574c0 msgr2=0x555a15655520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.990+0000 7f3f0331f800 1 -- 172.21.15.50:0/3786330635 shutdown_connections 2024-05-08T18:05:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.990+0000 7f3f0331f800 1 -- 172.21.15.50:0/3786330635 wait complete. 2024-05-08T18:05:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.990+0000 7f3f0331f800 1 Processor -- start 2024-05-08T18:05:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.990+0000 7f3f0331f800 1 -- start start 2024-05-08T18:05:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.990+0000 7f3f0331f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1565b1f0 0x555a15662530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.990+0000 7f3f0331f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555a1565c240 con 0x555a1565b1f0 2024-05-08T18:05:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.990+0000 7f3efefe0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1565b1f0 0x555a15662530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.990+0000 7f3efefe0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1565b1f0 0x555a15662530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37758/0 (socket says 172.21.15.50:37758) 2024-05-08T18:05:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.990+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 learned_addr learned my addr 172.21.15.50:0/2280670418 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.991+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a15662a70 con 0x555a1565b1f0 2024-05-08T18:05:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.991+0000 7f3efefe0640 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1565b1f0 0x555a15662530 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3ef002efd0 tx=0x7f3ef0034f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.991+0000 7f3de0ff9640 1 -- 172.21.15.50:0/2280670418 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef0043510 con 0x555a1565b1f0 2024-05-08T18:05:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.991+0000 7f3de0ff9640 1 -- 172.21.15.50:0/2280670418 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef0003450 con 0x555a1565b1f0 2024-05-08T18:05:07.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.991+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555a15662a70 con 0x555a1565b1f0 2024-05-08T18:05:07.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.991+0000 7f3de0ff9640 1 -- 172.21.15.50:0/2280670418 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3ef0051ba0 con 0x555a1565b1f0 2024-05-08T18:05:07.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.991+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a15662a70 con 0x555a1565b1f0 2024-05-08T18:05:07.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.992+0000 7f3de0ff9640 1 -- 172.21.15.50:0/2280670418 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3ef006a070 con 0x555a1565b1f0 2024-05-08T18:05:07.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.992+0000 7f3de0ff9640 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dc803bba0 0x7f3dc803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.992+0000 7f3de0ff9640 1 -- 172.21.15.50:0/2280670418 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f3ef005a070 con 0x555a1565b1f0 2024-05-08T18:05:07.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.992+0000 7f3efe7df640 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dc803bba0 0x7f3dc803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.993+0000 7f3f0331f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:07.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.993+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555a1566dbf0 0x555a156700b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:07.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.993+0000 7f3efe7df640 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dc803bba0 0x7f3dc803e060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x555a15640100 tx=0x7f3ef404f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.993+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a15670800 con 0x555a1566dbf0 2024-05-08T18:05:07.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.993+0000 7f3eff7e1640 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555a1566dbf0 0x555a156700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:07.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.993+0000 7f3eff7e1640 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555a1566dbf0 0x555a156700b0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:07.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3ef80083e0 con 0x555a1566dbf0 2024-05-08T18:05:07.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3f0331f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3f0331f800 20 realm 2024-05-08T18:05:08.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3f0331f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a15670c10 con 0x555a1566dbf0 2024-05-08T18:05:08.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3ef80083e0 con 0x555a1566dbf0 2024-05-08T18:05:08.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3f0331f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3f0331f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:08.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3f0331f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a156714e0 con 0x555a1566dbf0 2024-05-08T18:05:08.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3ef80083e0 con 0x555a1566dbf0 2024-05-08T18:05:08.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3f0331f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3f0331f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a15671880 con 0x555a1566dbf0 2024-05-08T18:05:08.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3ef80083e0 con 0x555a1566dbf0 2024-05-08T18:05:08.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3f0331f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:08.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3f0331f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555a15672240 0x555a15674630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.994+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a15674c80 con 0x555a15672240 2024-05-08T18:05:08.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.995+0000 7f3efefe0640 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555a15672240 0x555a15674630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.995+0000 7f3efefe0640 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555a15672240 0x555a15674630 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.995+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.995+0000 7f3f0331f800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:08.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.995+0000 7f3f0331f800 20 searching for the correct realm 2024-05-08T18:05:08.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.995+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555a15676c10 0x555a156790d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.996+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555a156797a0 con 0x555a15676c10 2024-05-08T18:05:08.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.996+0000 7f3efe7df640 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555a15676c10 0x555a156790d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.996+0000 7f3efe7df640 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555a15676c10 0x555a156790d0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.996+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.996+0000 7f3efe7df640 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3ef4074040 0x7f3ef4081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.996+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555a156797a0 con 0x7f3ef4074040 2024-05-08T18:05:08.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.996+0000 7f3eff7e1640 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3ef4074040 0x7f3ef4081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.997+0000 7f3eff7e1640 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3ef4074040 0x7f3ef4081410 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.997+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef80083e0 con 0x7f3ef4074040 2024-05-08T18:05:08.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.997+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555a156714e0 con 0x555a15672240 2024-05-08T18:05:08.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.997+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.997+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555a156714e0 con 0x555a1566dbf0 2024-05-08T18:05:08.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.997+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef80083e0 con 0x555a1566dbf0 2024-05-08T18:05:08.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.997+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555a156714e0 con 0x555a1566dbf0 2024-05-08T18:05:08.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.998+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.998+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555a156714e0 con 0x555a15672240 2024-05-08T18:05:08.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.998+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.998+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555a156714e0 con 0x555a15672240 2024-05-08T18:05:08.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.998+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.998+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555a156714e0 con 0x7f3ef4074040 2024-05-08T18:05:08.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.998+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef8026040 con 0x7f3ef4074040 2024-05-08T18:05:08.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.998+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555a156714e0 con 0x555a15676c10 2024-05-08T18:05:08.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.999+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.999+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555a156714e0 con 0x555a1566dbf0 2024-05-08T18:05:08.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.999+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.999+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555a156714e0 con 0x555a15672240 2024-05-08T18:05:08.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.999+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.999+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555a156714e0 con 0x555a1566dbf0 2024-05-08T18:05:08.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.999+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:07.999+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555a156714e0 con 0x7f3ef4074040 2024-05-08T18:05:08.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.000+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f3ef8026040 con 0x7f3ef4074040 2024-05-08T18:05:08.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.000+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555a156714e0 con 0x555a15676c10 2024-05-08T18:05:08.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.000+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.000+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x555a15676c10 2024-05-08T18:05:08.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.000+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.000+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x555a15672240 2024-05-08T18:05:08.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.000+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.000+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x555a15676c10 2024-05-08T18:05:08.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.001+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.001+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x555a15676c10 2024-05-08T18:05:08.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.001+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.001+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x555a1566dbf0 2024-05-08T18:05:08.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.001+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.001+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x555a1566dbf0 2024-05-08T18:05:08.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.001+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.001+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x555a15672240 2024-05-08T18:05:08.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.002+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.002+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x555a15676c10 2024-05-08T18:05:08.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.002+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.002+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x7f3ef4074040 2024-05-08T18:05:08.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.002+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef8026040 con 0x7f3ef4074040 2024-05-08T18:05:08.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.002+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x555a1566dbf0 2024-05-08T18:05:08.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.002+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.002+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x555a15672240 2024-05-08T18:05:08.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.003+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.003+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x555a15676c10 2024-05-08T18:05:08.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.003+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.003+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x555a15672240 2024-05-08T18:05:08.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.003+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.003+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x7f3ef4074040 2024-05-08T18:05:08.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.004+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef8026040 con 0x7f3ef4074040 2024-05-08T18:05:08.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.004+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x555a15676c10 2024-05-08T18:05:08.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.004+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.004+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x555a15672240 2024-05-08T18:05:08.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.004+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.004+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x555a1566dbf0 2024-05-08T18:05:08.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.004+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.004+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3ef408e040 con 0x7f3ef4074040 2024-05-08T18:05:08.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ef8026040 con 0x7f3ef4074040 2024-05-08T18:05:08.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:08.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:08.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:08.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:08.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:08.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:08.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a15675020 con 0x555a1566dbf0 2024-05-08T18:05:08.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a15675020 con 0x555a1566dbf0 2024-05-08T18:05:08.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:08.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a15675020 con 0x555a15676c10 2024-05-08T18:05:08.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:08.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 20 zone default found 2024-05-08T18:05:08.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 4 Realm: () 2024-05-08T18:05:08.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:08.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:08.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:08.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 20 zonegroup default 2024-05-08T18:05:08.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.005+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a15675920 con 0x555a15672240 2024-05-08T18:05:08.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.006+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.006+0000 7f3f0331f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.006+0000 7f3f0331f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.006+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a1567b630 con 0x555a15676c10 2024-05-08T18:05:08.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.006+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.006+0000 7f3f0331f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.006+0000 7f3f0331f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.006+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a1567bac0 con 0x555a15676c10 2024-05-08T18:05:08.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.006+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.006+0000 7f3f0331f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.006+0000 7f3f0331f800 20 started sync module instance, tier type = 2024-05-08T18:05:08.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.006+0000 7f3f0331f800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:08.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.006+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a1567aaa0 con 0x7f3ef4074040 2024-05-08T18:05:08.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.007+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef8026040 con 0x7f3ef4074040 2024-05-08T18:05:08.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.007+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93845394534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a1567aaa0 con 0x7f3ef4074040 2024-05-08T18:05:08.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.007+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a1567c540 con 0x555a1566dbf0 2024-05-08T18:05:08.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.008+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93845394534336] v27'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef8026040 con 0x7f3ef4074040 2024-05-08T18:05:08.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.008+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.009+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93845394539120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a1567aaa0 con 0x555a1566dbf0 2024-05-08T18:05:08.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.009+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a1567cfc0 con 0x555a15676c10 2024-05-08T18:05:08.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.009+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93845394539120] v27'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.010+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.010+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93845394531824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a1567aa10 con 0x555a15676c10 2024-05-08T18:05:08.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.010+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a1567deb0 con 0x7f3ef4074040 2024-05-08T18:05:08.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.011+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93845394531824] v27'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.011+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef8026040 con 0x7f3ef4074040 2024-05-08T18:05:08.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.011+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93845394545152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a1567e2d0 con 0x7f3ef4074040 2024-05-08T18:05:08.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.011+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a1567ec90 con 0x555a1566dbf0 2024-05-08T18:05:08.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.012+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.012+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93845394545152] v27'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef8026040 con 0x7f3ef4074040 2024-05-08T18:05:08.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.012+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93845394548704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a1567f0b0 con 0x555a1566dbf0 2024-05-08T18:05:08.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.012+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a1567f5f0 con 0x555a15672240 2024-05-08T18:05:08.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.013+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.013+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93845394548704] v27'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.013+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93845394549760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a1567f9e0 con 0x555a15672240 2024-05-08T18:05:08.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.013+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a156803a0 con 0x555a15672240 2024-05-08T18:05:08.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.014+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93845394549760] v27'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.014+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.014+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93845394554608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a156807c0 con 0x555a15672240 2024-05-08T18:05:08.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.014+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a15681180 con 0x555a15672240 2024-05-08T18:05:08.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.015+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v27'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.015+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93845394558160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a156815e0 con 0x555a15672240 2024-05-08T18:05:08.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.015+0000 7f3f0331f800 20 add_watcher() i=0 2024-05-08T18:05:08.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.015+0000 7f3f0331f800 20 add_watcher() i=1 2024-05-08T18:05:08.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.015+0000 7f3f0331f800 20 add_watcher() i=2 2024-05-08T18:05:08.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.015+0000 7f3f0331f800 20 add_watcher() i=3 2024-05-08T18:05:08.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.015+0000 7f3f0331f800 20 add_watcher() i=4 2024-05-08T18:05:08.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.015+0000 7f3f0331f800 20 add_watcher() i=5 2024-05-08T18:05:08.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.015+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 93845394554608] v27'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.015+0000 7f3f0331f800 20 add_watcher() i=6 2024-05-08T18:05:08.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.016+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93845394558160] v27'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.016+0000 7f3f0331f800 20 add_watcher() i=7 2024-05-08T18:05:08.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.016+0000 7f3f0331f800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:08.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.016+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a15680310 con 0x555a1566dbf0 2024-05-08T18:05:08.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.016+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.016+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93845394559216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a15680310 con 0x555a1566dbf0 2024-05-08T18:05:08.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.017+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93845394559216] v27'106 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.017+0000 7f3db57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:08.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.017+0000 7f3f0331f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:08.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.017+0000 7f3f0331f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:08.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.018+0000 7f3f0331f800 5 note: GC not initialized 2024-05-08T18:05:08.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.018+0000 7f3d2ffff640 20 reqs_thread_entry: start 2024-05-08T18:05:08.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.018+0000 7f3f0331f800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:08.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.018+0000 7f3f0331f800 20 Filter name: none 2024-05-08T18:05:08.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.018+0000 7f3d2dffb640 20 reqs_thread_entry: start 2024-05-08T18:05:08.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.018+0000 7f3f0331f800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-05-08T18:05:08.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.018+0000 7f3f0331f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.018+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a156b7cb0 con 0x555a15672240 2024-05-08T18:05:08.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.019+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.019+0000 7f3f0331f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.019+0000 7f3f0331f800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-05-08T18:05:08.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.019+0000 7f3f0331f800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-05-08T18:05:08.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.019+0000 7f3f0331f800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-05-08T18:05:08.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.019+0000 7f3f0331f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.019+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a156b6450 con 0x555a15672240 2024-05-08T18:05:08.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.019+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.019+0000 7f3f0331f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.019+0000 7f3f0331f800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-05-08T18:05:08.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.019+0000 7f3f0331f800 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-05-08T18:05:08.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.019+0000 7f3f0331f800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-05-08T18:05:08.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.019+0000 7f3f0331f800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-05-08T18:05:08.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.019+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~380 in=380b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a156b6d40 con 0x555a15672240 2024-05-08T18:05:08.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.020+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~380] v27'1 uv1 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.020+0000 7f3f0331f800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-05-08T18:05:08.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.020+0000 7f3f0331f800 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-05-08T18:05:08.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.020+0000 7f3f0331f800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-05-08T18:05:08.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.020+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 93845394778256 in=568b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a156bb2a0 con 0x555a15672240 2024-05-08T18:05:08.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.020+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== watch-notify(notify (1) cookie 93845394549760 notify 115964116995 ret 0) ==== 598+0+0 (crc 0 0 0) 0x555a156b6640 con 0x555a15672240 2024-05-08T18:05:08.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.020+0000 7f3de27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116995 cookie 93845394549760 notifier 4480 bl.length()=556 2024-05-08T18:05:08.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.020+0000 7f3de27fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-05-08T18:05:08.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.020+0000 7f3de27fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-05-08T18:05:08.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.020+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(65 notify.5 [notify cookie 93845394778256 out=8b] v0'0 uv108 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.020+0000 7f3de27fc640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f3dc4003ea0 con 0x555a15672240 2024-05-08T18:05:08.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.021+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.021+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== watch-notify(notify_complete (2) cookie 93845394778256 notify 115964116995 ret 0) ==== 42+0+48 (crc 0 0 0) 0x555a1564e9d0 con 0x555a15672240 2024-05-08T18:05:08.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.021+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a156b6cc0 con 0x555a15672240 2024-05-08T18:05:08.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.022+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v27'3 uv3 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.022+0000 7f3f0331f800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-05-08T18:05:08.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.022+0000 7f3f0331f800 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-05-08T18:05:08.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.022+0000 7f3f0331f800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-05-08T18:05:08.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.022+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 93845394778128 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555a156b6c30 con 0x555a15676c10 2024-05-08T18:05:08.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.022+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== watch-notify(notify (1) cookie 93845394531824 notify 115964116994 ret 0) ==== 219+0+0 (crc 0 0 0) 0x7f3ef4088040 con 0x555a15676c10 2024-05-08T18:05:08.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.022+0000 7f3de27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116994 cookie 93845394531824 notifier 4480 bl.length()=177 2024-05-08T18:05:08.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.022+0000 7f3de27fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-05-08T18:05:08.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.022+0000 7f3de27fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-05-08T18:05:08.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.022+0000 7f3de27fc640 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f3dc4003ea0 con 0x555a15676c10 2024-05-08T18:05:08.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.022+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(68 notify.2 [notify cookie 93845394778128 out=8b] v0'0 uv108 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.022+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.022+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== watch-notify(notify_complete (2) cookie 93845394778128 notify 115964116994 ret 0) ==== 42+0+48 (crc 0 0 0) 0x555a1553e740 con 0x555a15676c10 2024-05-08T18:05:08.032 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"fud","display_name":"Fud","email":"","suspended":0,"max_buckets":1,"subusers":[],"keys":[{"user":"fud","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-05-08T18:05:08.020139Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:08.020126Z","tags":[],"group_ids":[]} 2024-05-08T18:05:08.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.023+0000 7f3db57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:08.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.023+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93845394559216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a1566a800 con 0x555a1566dbf0 2024-05-08T18:05:08.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.024+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 93845394559216] v27'107 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.024+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93845394534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a156810f0 con 0x7f3ef4074040 2024-05-08T18:05:08.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.025+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 93845394534336] v27'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef8026040 con 0x7f3ef4074040 2024-05-08T18:05:08.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.025+0000 7f3f0331f800 20 remove_watcher() i=0 2024-05-08T18:05:08.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.025+0000 7f3f0331f800 2 removed watcher, disabling cache 2024-05-08T18:05:08.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.025+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93845394539120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a156810f0 con 0x555a1566dbf0 2024-05-08T18:05:08.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.026+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 93845394539120] v27'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.026+0000 7f3f0331f800 20 remove_watcher() i=1 2024-05-08T18:05:08.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.026+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93845394531824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a156810f0 con 0x555a15676c10 2024-05-08T18:05:08.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.027+0000 7f3efe7df640 1 -- 172.21.15.50:0/2280670418 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 93845394531824] v27'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef407f070 con 0x555a15676c10 2024-05-08T18:05:08.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.027+0000 7f3f0331f800 20 remove_watcher() i=2 2024-05-08T18:05:08.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.027+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93845394545152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a1567a5f0 con 0x7f3ef4074040 2024-05-08T18:05:08.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.028+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 93845394545152] v27'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef8026040 con 0x7f3ef4074040 2024-05-08T18:05:08.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.028+0000 7f3f0331f800 20 remove_watcher() i=3 2024-05-08T18:05:08.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.028+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93845394548704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a156810f0 con 0x555a1566dbf0 2024-05-08T18:05:08.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.029+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 93845394548704] v27'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef8026040 con 0x555a1566dbf0 2024-05-08T18:05:08.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.029+0000 7f3f0331f800 20 remove_watcher() i=4 2024-05-08T18:05:08.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.029+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93845394549760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a1567da00 con 0x555a15672240 2024-05-08T18:05:08.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.030+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 93845394549760] v27'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.030+0000 7f3f0331f800 20 remove_watcher() i=5 2024-05-08T18:05:08.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.030+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93845394554608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a1567a5f0 con 0x555a15672240 2024-05-08T18:05:08.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.031+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 93845394554608] v27'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.031+0000 7f3f0331f800 20 remove_watcher() i=6 2024-05-08T18:05:08.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.031+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93845394558160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555a156810f0 con 0x555a15672240 2024-05-08T18:05:08.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.032+0000 7f3efefe0640 1 -- 172.21.15.50:0/2280670418 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 93845394558160] v27'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ef0073020 con 0x555a15672240 2024-05-08T18:05:08.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.032+0000 7f3f0331f800 20 remove_watcher() i=7 2024-05-08T18:05:08.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555a15672240 msgr2=0x555a15674630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555a15672240 0x555a15674630 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555a1566dbf0 msgr2=0x555a156700b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555a1566dbf0 0x555a156700b0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3ef4074040 msgr2=0x7f3ef4081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3ef4074040 0x7f3ef4081410 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555a15676c10 msgr2=0x555a156790d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555a15676c10 0x555a156790d0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dc803bba0 msgr2=0x7f3dc803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dc803bba0 0x7f3dc803e060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x555a15640100 tx=0x7f3ef404f000 comp rx=0 tx=0).stop 2024-05-08T18:05:08.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1565b1f0 msgr2=0x555a15662530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2280670418 reap_dead start 2024-05-08T18:05:08.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1565b1f0 0x555a15662530 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3ef002efd0 tx=0x7f3ef0034f00 comp rx=0 tx=0).stop 2024-05-08T18:05:08.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 shutdown_connections 2024-05-08T18:05:08.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2280670418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1565b1f0 0x555a15662530 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 >> 172.21.15.50:0/2280670418 conn(0x555a156574c0 msgr2=0x555a15654b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 shutdown_connections 2024-05-08T18:05:08.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 -- 172.21.15.50:0/2280670418 wait complete. 2024-05-08T18:05:08.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 -- 172.21.15.50:0/3377460367 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dd803bb50 msgr2=0x7f3dd803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 --2- 172.21.15.50:0/3377460367 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dd803bb50 0x7f3dd803e010 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x555a15637480 tx=0x7f3ef4005ac0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 -- 172.21.15.50:0/3377460367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1564e070 msgr2=0x555a1564e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 --2- 172.21.15.50:0/3377460367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1564e070 0x555a1564e490 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00510a0 tx=0x7f3ef0010f60 comp rx=0 tx=0).stop 2024-05-08T18:05:08.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 -- 172.21.15.50:0/3377460367 shutdown_connections 2024-05-08T18:05:08.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 --2- 172.21.15.50:0/3377460367 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dd803bb50 0x7f3dd803e010 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 --2- 172.21.15.50:0/3377460367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a1564e070 0x555a1564e490 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 -- 172.21.15.50:0/3377460367 >> 172.21.15.50:0/3377460367 conn(0x555a15642110 msgr2=0x555a15642510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 -- 172.21.15.50:0/3377460367 shutdown_connections 2024-05-08T18:05:08.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.033+0000 7f3f0331f800 1 -- 172.21.15.50:0/3377460367 wait complete. 2024-05-08T18:05:08.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.034+0000 7f3f0331f800 1 -- 172.21.15.50:0/2866349511 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555a156392e0 msgr2=0x555a1563b6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.034+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555a156392e0 0x555a1563b6d0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.034+0000 7f3f0331f800 1 -- 172.21.15.50:0/2866349511 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555a1553ac30 msgr2=0x555a15636a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.034+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555a1553ac30 0x555a15636a20 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.034+0000 7f3f0331f800 1 -- 172.21.15.50:0/2866349511 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555a1563d220 msgr2=0x555a1563f610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.034+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555a1563d220 0x555a1563f610 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.034+0000 7f3f0331f800 1 -- 172.21.15.50:0/2866349511 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3de403bb50 msgr2=0x7f3de403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.034+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3de403bb50 0x7f3de403e010 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4054000 tx=0x7f3ef404e000 comp rx=0 tx=0).stop 2024-05-08T18:05:08.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.034+0000 7f3f0331f800 1 -- 172.21.15.50:0/2866349511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a15537120 msgr2=0x555a1562f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.034+0000 7f3f0331f800 1 --2- 172.21.15.50:0/2866349511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555a15537120 0x555a1562f830 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00065f0 tx=0x7f3ef0001250 comp rx=0 tx=0).stop 2024-05-08T18:05:08.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.034+0000 7f3eff7e1640 1 -- 172.21.15.50:0/2866349511 reap_dead start 2024-05-08T18:05:08.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.034+0000 7f3f0331f800 1 -- 172.21.15.50:0/2866349511 shutdown_connections 2024-05-08T18:05:08.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.034+0000 7f3f0331f800 1 -- 172.21.15.50:0/2866349511 >> 172.21.15.50:0/2866349511 conn(0x555a15530b90 msgr2=0x555a15621d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.034+0000 7f3f0331f800 1 -- 172.21.15.50:0/2866349511 shutdown_connections 2024-05-08T18:05:08.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.034+0000 7f3f0331f800 1 -- 172.21.15.50:0/2866349511 wait complete. 2024-05-08T18:05:08.042 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-05-08T18:05:08.020139Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:08.020126Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:08.048 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-05-08T18:05:08.052 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:08.052 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:myfoo'] 2024-05-08T18:05:08.052 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:myfoo'] 2024-05-08T18:05:08.053 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:myfoo 2024-05-08T18:05:08.097 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:08.097 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:08.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.100+0000 7fc365b51800 1 Processor -- start 2024-05-08T18:05:08.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.100+0000 7fc365b51800 1 -- start start 2024-05-08T18:05:08.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.100+0000 7fc365b51800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x559b2b6fcad0 0x559b2b7055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.100+0000 7fc365b51800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559b2b670010 con 0x559b2b6faf90 2024-05-08T18:05:08.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.100+0000 7fc365b51800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x559b2b59ce20 con 0x559b2b6fcad0 2024-05-08T18:05:08.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.100+0000 7fc362e4f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x559b2b6fcad0 0x559b2b7055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.100+0000 7fc363650640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x559b2b6faf90 0x559b2b6fb3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43162/0 (socket says 172.21.15.50:43162) 2024-05-08T18:05:08.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.100+0000 7fc363650640 1 -- 172.21.15.50:0/4232185361 learned_addr learned my addr 172.21.15.50:0/4232185361 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:08.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.101+0000 7fc362e4f640 1 -- 172.21.15.50:0/4232185361 >> v1:172.21.15.46:6789/0 conn(0x559b2b6faf90 legacy=0x559b2b6fb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.101+0000 7fc362e4f640 1 -- 172.21.15.50:0/4232185361 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b2b706b20 con 0x559b2b6fcad0 2024-05-08T18:05:08.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.103+0000 7fc362e4f640 1 --2- 172.21.15.50:0/4232185361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6fcad0 0x559b2b7055d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00a470 tx=0x7fc34c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73625ba52f55ab4b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.103+0000 7fc36264e640 1 -- 172.21.15.50:0/4232185361 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc34c057070 con 0x559b2b6fcad0 2024-05-08T18:05:08.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.103+0000 7fc36264e640 1 -- 172.21.15.50:0/4232185361 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc34c04adc0 con 0x559b2b6fcad0 2024-05-08T18:05:08.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.103+0000 7fc36264e640 1 -- 172.21.15.50:0/4232185361 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc34c052590 con 0x559b2b6fcad0 2024-05-08T18:05:08.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.103+0000 7fc365b51800 1 -- 172.21.15.50:0/4232185361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6fcad0 msgr2=0x559b2b7055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.103+0000 7fc365b51800 1 --2- 172.21.15.50:0/4232185361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6fcad0 0x559b2b7055d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00a470 tx=0x7fc34c01ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:08.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.103+0000 7fc365b51800 1 -- 172.21.15.50:0/4232185361 shutdown_connections 2024-05-08T18:05:08.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.103+0000 7fc365b51800 1 --2- 172.21.15.50:0/4232185361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6fcad0 0x559b2b7055d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.103+0000 7fc365b51800 1 -- 172.21.15.50:0/4232185361 >> 172.21.15.50:0/4232185361 conn(0x559b2b6f6ad0 msgr2=0x559b2b6f8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.103+0000 7fc365b51800 1 -- 172.21.15.50:0/4232185361 shutdown_connections 2024-05-08T18:05:08.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.103+0000 7fc365b51800 1 -- 172.21.15.50:0/4232185361 wait complete. 2024-05-08T18:05:08.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.119+0000 7fc365b51800 1 Processor -- start 2024-05-08T18:05:08.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.119+0000 7fc365b51800 1 -- start start 2024-05-08T18:05:08.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.119+0000 7fc365b51800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6faf90 0x559b2b7efea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.119+0000 7fc365b51800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559b2b647a10 con 0x559b2b6faf90 2024-05-08T18:05:08.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.119+0000 7fc363650640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6faf90 0x559b2b7efea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.119+0000 7fc363650640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6faf90 0x559b2b7efea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37768/0 (socket says 172.21.15.50:37768) 2024-05-08T18:05:08.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.119+0000 7fc363650640 1 -- 172.21.15.50:0/2047471424 learned_addr learned my addr 172.21.15.50:0/2047471424 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.120+0000 7fc363650640 1 -- 172.21.15.50:0/2047471424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b2b7f0470 con 0x559b2b6faf90 2024-05-08T18:05:08.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.120+0000 7fc363650640 1 --2- 172.21.15.50:0/2047471424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6faf90 0x559b2b7efea0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc354001e10 tx=0x7fc35400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bafb51daa7a7bca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.120+0000 7fc25160a640 1 -- 172.21.15.50:0/2047471424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc354012020 con 0x559b2b6faf90 2024-05-08T18:05:08.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.120+0000 7fc25160a640 1 -- 172.21.15.50:0/2047471424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc35400e5a0 con 0x559b2b6faf90 2024-05-08T18:05:08.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.120+0000 7fc25160a640 1 -- 172.21.15.50:0/2047471424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc35400e8a0 con 0x559b2b6faf90 2024-05-08T18:05:08.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.121+0000 7fc365b51800 1 -- 172.21.15.50:0/2047471424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6faf90 msgr2=0x559b2b7efea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.121+0000 7fc365b51800 1 --2- 172.21.15.50:0/2047471424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6faf90 0x559b2b7efea0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc354001e10 tx=0x7fc35400b780 comp rx=0 tx=0).stop 2024-05-08T18:05:08.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.121+0000 7fc365b51800 1 -- 172.21.15.50:0/2047471424 shutdown_connections 2024-05-08T18:05:08.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.121+0000 7fc365b51800 1 --2- 172.21.15.50:0/2047471424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6faf90 0x559b2b7efea0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.121+0000 7fc365b51800 1 -- 172.21.15.50:0/2047471424 >> 172.21.15.50:0/2047471424 conn(0x559b2b6f6ad0 msgr2=0x559b2b7eb7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.121+0000 7fc365b51800 1 -- 172.21.15.50:0/2047471424 shutdown_connections 2024-05-08T18:05:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.121+0000 7fc365b51800 1 -- 172.21.15.50:0/2047471424 wait complete. 2024-05-08T18:05:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.121+0000 7fc365b51800 1 Processor -- start 2024-05-08T18:05:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.121+0000 7fc365b51800 1 -- start start 2024-05-08T18:05:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.121+0000 7fc365b51800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6faf90 0x559b2b7f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.121+0000 7fc365b51800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559b2b705de0 con 0x559b2b6faf90 2024-05-08T18:05:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.121+0000 7fc363650640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6faf90 0x559b2b7f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.121+0000 7fc363650640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6faf90 0x559b2b7f7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37776/0 (socket says 172.21.15.50:37776) 2024-05-08T18:05:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.121+0000 7fc363650640 1 -- 172.21.15.50:0/3757725993 learned_addr learned my addr 172.21.15.50:0/3757725993 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.122+0000 7fc363650640 1 -- 172.21.15.50:0/3757725993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b2b7f7f80 con 0x559b2b6faf90 2024-05-08T18:05:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.122+0000 7fc363650640 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6faf90 0x559b2b7f7a40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc354000f60 tx=0x7fc354000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.122+0000 7fc24fe07640 1 -- 172.21.15.50:0/3757725993 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc354012030 con 0x559b2b6faf90 2024-05-08T18:05:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.122+0000 7fc24fe07640 1 -- 172.21.15.50:0/3757725993 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc354001470 con 0x559b2b6faf90 2024-05-08T18:05:08.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.122+0000 7fc365b51800 1 -- 172.21.15.50:0/3757725993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559b2b7f8210 con 0x559b2b6faf90 2024-05-08T18:05:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.122+0000 7fc24fe07640 1 -- 172.21.15.50:0/3757725993 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc35401a870 con 0x559b2b6faf90 2024-05-08T18:05:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.122+0000 7fc365b51800 1 -- 172.21.15.50:0/3757725993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b2b7fae70 con 0x559b2b6faf90 2024-05-08T18:05:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.123+0000 7fc24fe07640 1 -- 172.21.15.50:0/3757725993 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fc354015020 con 0x559b2b6faf90 2024-05-08T18:05:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.123+0000 7fc24fe07640 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc24803bb50 0x7fc24803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.123+0000 7fc24fe07640 1 -- 172.21.15.50:0/3757725993 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fc354020030 con 0x559b2b6faf90 2024-05-08T18:05:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.123+0000 7fc362e4f640 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc24803bb50 0x7fc24803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.123+0000 7fc365b51800 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559b2b6fcad0 0x559b2b7fec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.123+0000 7fc365b51800 1 -- 172.21.15.50:0/3757725993 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b7ff1c0 con 0x559b2b6fcad0 2024-05-08T18:05:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.123+0000 7fc363e51640 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559b2b6fcad0 0x559b2b7fec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.124+0000 7fc362e4f640 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc24803bb50 0x7fc24803e010 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc34c054000 tx=0x7fc34c04e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.124+0000 7fc363e51640 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559b2b6fcad0 0x559b2b7fec30 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.124+0000 7fc363e51640 1 -- 172.21.15.50:0/3757725993 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b6fcad0 2024-05-08T18:05:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.124+0000 7fc365b51800 1 -- 172.21.15.50:0/3757725993 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b7fc1e0 con 0x559b2b6fcad0 2024-05-08T18:05:08.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.124+0000 7fc363e51640 1 -- 172.21.15.50:0/3757725993 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b6fcad0 2024-05-08T18:05:08.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.124+0000 7fc365b51800 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559b2b8014f0 0x559b2b8038e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.124+0000 7fc365b51800 1 -- 172.21.15.50:0/3757725993 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b803f80 con 0x559b2b8014f0 2024-05-08T18:05:08.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.125+0000 7fc363650640 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559b2b8014f0 0x559b2b8038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.125+0000 7fc363650640 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559b2b8014f0 0x559b2b8038e0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.125+0000 7fc363650640 1 -- 172.21.15.50:0/3757725993 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fc3540189b0 con 0x559b2b8014f0 2024-05-08T18:05:08.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.125+0000 7fc365b51800 1 -- 172.21.15.50:0/3757725993 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b800390 con 0x559b2b6fcad0 2024-05-08T18:05:08.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.125+0000 7fc363e51640 1 -- 172.21.15.50:0/3757725993 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b6fcad0 2024-05-08T18:05:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.125+0000 7fc365b51800 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559b2b8053e0 0x559b2b8077d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.125+0000 7fc365b51800 1 -- 172.21.15.50:0/3757725993 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b807ed0 con 0x559b2b8053e0 2024-05-08T18:05:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.126+0000 7fc362e4f640 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559b2b8053e0 0x559b2b8077d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.126+0000 7fc362e4f640 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559b2b8053e0 0x559b2b8077d0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.126+0000 7fc362e4f640 1 -- 172.21.15.50:0/3757725993 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fc34c05a040 con 0x559b2b8053e0 2024-05-08T18:05:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.126+0000 7fc365b51800 1 Processor -- start 2024-05-08T18:05:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.127+0000 7fc365b51800 1 -- start start 2024-05-08T18:05:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.127+0000 7fc365b51800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b816200 0x559b2b816620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.127+0000 7fc365b51800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559b2b5ee330 con 0x559b2b816200 2024-05-08T18:05:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.127+0000 7fc363650640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b816200 0x559b2b816620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.127+0000 7fc363650640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b816200 0x559b2b816620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37788/0 (socket says 172.21.15.50:37788) 2024-05-08T18:05:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.127+0000 7fc363650640 1 -- 172.21.15.50:0/799248142 learned_addr learned my addr 172.21.15.50:0/799248142 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.127+0000 7fc363650640 1 -- 172.21.15.50:0/799248142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b2b816b60 con 0x559b2b816200 2024-05-08T18:05:08.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.127+0000 7fc363650640 1 --2- 172.21.15.50:0/799248142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b816200 0x559b2b816620 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc354018c60 tx=0x7fc354023670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.127+0000 7fc24d602640 1 -- 172.21.15.50:0/799248142 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc354016b00 con 0x559b2b816200 2024-05-08T18:05:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.128+0000 7fc24d602640 1 -- 172.21.15.50:0/799248142 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc354047020 con 0x559b2b816200 2024-05-08T18:05:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.128+0000 7fc24d602640 1 -- 172.21.15.50:0/799248142 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc354016dd0 con 0x559b2b816200 2024-05-08T18:05:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.128+0000 7fc365b51800 1 -- 172.21.15.50:0/799248142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559b2b8175d0 con 0x559b2b816200 2024-05-08T18:05:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.128+0000 7fc365b51800 1 -- 172.21.15.50:0/799248142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b2b8175d0 con 0x559b2b816200 2024-05-08T18:05:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.128+0000 7fc24d602640 1 -- 172.21.15.50:0/799248142 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fc35400cb40 con 0x559b2b816200 2024-05-08T18:05:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.128+0000 7fc24d602640 1 --2- 172.21.15.50:0/799248142 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc23c03bbf0 0x7fc23c03e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.129+0000 7fc24d602640 1 -- 172.21.15.50:0/799248142 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fc354048030 con 0x559b2b816200 2024-05-08T18:05:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.129+0000 7fc362e4f640 1 --2- 172.21.15.50:0/799248142 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc23c03bbf0 0x7fc23c03e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.129+0000 7fc365b51800 1 Processor -- start 2024-05-08T18:05:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.129+0000 7fc365b51800 1 -- start start 2024-05-08T18:05:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.129+0000 7fc362e4f640 1 --2- 172.21.15.50:0/799248142 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc23c03bbf0 0x7fc23c03e0b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc34c055040 tx=0x7fc34c005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.129+0000 7fc365b51800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b823210 0x559b2b823630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.129+0000 7fc365b51800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559b2b7f07f0 con 0x559b2b823210 2024-05-08T18:05:08.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.129+0000 7fc363650640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b823210 0x559b2b823630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.129+0000 7fc363650640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b823210 0x559b2b823630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37804/0 (socket says 172.21.15.50:37804) 2024-05-08T18:05:08.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.129+0000 7fc363650640 1 -- 172.21.15.50:0/2874366122 learned_addr learned my addr 172.21.15.50:0/2874366122 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.130+0000 7fc363650640 1 -- 172.21.15.50:0/2874366122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b2b823c00 con 0x559b2b823210 2024-05-08T18:05:08.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.130+0000 7fc363650640 1 --2- 172.21.15.50:0/2874366122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b823210 0x559b2b823630 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc354008fd0 tx=0x7fc354043880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10d628c87502b82 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.130+0000 7fc246ffd640 1 -- 172.21.15.50:0/2874366122 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc354023e90 con 0x559b2b823210 2024-05-08T18:05:08.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.130+0000 7fc246ffd640 1 -- 172.21.15.50:0/2874366122 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc354043620 con 0x559b2b823210 2024-05-08T18:05:08.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.130+0000 7fc246ffd640 1 -- 172.21.15.50:0/2874366122 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc35402ecf0 con 0x559b2b823210 2024-05-08T18:05:08.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.130+0000 7fc365b51800 1 -- 172.21.15.50:0/2874366122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b823210 msgr2=0x559b2b823630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.130+0000 7fc365b51800 1 --2- 172.21.15.50:0/2874366122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b823210 0x559b2b823630 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc354008fd0 tx=0x7fc354043880 comp rx=0 tx=0).stop 2024-05-08T18:05:08.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.130+0000 7fc365b51800 1 -- 172.21.15.50:0/2874366122 shutdown_connections 2024-05-08T18:05:08.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.130+0000 7fc365b51800 1 --2- 172.21.15.50:0/2874366122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b823210 0x559b2b823630 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.130+0000 7fc365b51800 1 -- 172.21.15.50:0/2874366122 >> 172.21.15.50:0/2874366122 conn(0x559b2b81f490 msgr2=0x559b2b81d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.130+0000 7fc365b51800 1 -- 172.21.15.50:0/2874366122 shutdown_connections 2024-05-08T18:05:08.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.130+0000 7fc365b51800 1 -- 172.21.15.50:0/2874366122 wait complete. 2024-05-08T18:05:08.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.131+0000 7fc365b51800 1 Processor -- start 2024-05-08T18:05:08.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.131+0000 7fc365b51800 1 -- start start 2024-05-08T18:05:08.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.131+0000 7fc365b51800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b823210 0x559b2b82a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.131+0000 7fc365b51800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559b2b824270 con 0x559b2b823210 2024-05-08T18:05:08.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.131+0000 7fc363650640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b823210 0x559b2b82a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.131+0000 7fc363650640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b823210 0x559b2b82a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37814/0 (socket says 172.21.15.50:37814) 2024-05-08T18:05:08.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.131+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 learned_addr learned my addr 172.21.15.50:0/877546953 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.131+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b2b82ab20 con 0x559b2b823210 2024-05-08T18:05:08.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.131+0000 7fc363650640 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b823210 0x559b2b82a5e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc354008fd0 tx=0x7fc354016f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.132+0000 7fc2457fa640 1 -- 172.21.15.50:0/877546953 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc35403d4c0 con 0x559b2b823210 2024-05-08T18:05:08.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.132+0000 7fc2457fa640 1 -- 172.21.15.50:0/877546953 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc354043ab0 con 0x559b2b823210 2024-05-08T18:05:08.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.132+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559b2b82ab20 con 0x559b2b823210 2024-05-08T18:05:08.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.132+0000 7fc2457fa640 1 -- 172.21.15.50:0/877546953 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc354043d90 con 0x559b2b823210 2024-05-08T18:05:08.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.132+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b2b82ab20 con 0x559b2b823210 2024-05-08T18:05:08.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.132+0000 7fc2457fa640 1 -- 172.21.15.50:0/877546953 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fc35406a070 con 0x559b2b823210 2024-05-08T18:05:08.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.133+0000 7fc2457fa640 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc22c03bba0 0x7fc22c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.133+0000 7fc2457fa640 1 -- 172.21.15.50:0/877546953 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fc35406ec80 con 0x559b2b823210 2024-05-08T18:05:08.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.133+0000 7fc362e4f640 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc22c03bba0 0x7fc22c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.133+0000 7fc365b51800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.133+0000 7fc365b51800 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559b2b835c40 0x559b2b838100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.133+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b838850 con 0x559b2b835c40 2024-05-08T18:05:08.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.133+0000 7fc363e51640 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559b2b835c40 0x559b2b838100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.133+0000 7fc362e4f640 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc22c03bba0 0x7fc22c03e060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x559b2b8082d0 tx=0x7fc34c04f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.133+0000 7fc363e51640 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559b2b835c40 0x559b2b838100 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b835c40 2024-05-08T18:05:08.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc365b51800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc365b51800 20 realm 2024-05-08T18:05:08.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc365b51800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b838c60 con 0x559b2b835c40 2024-05-08T18:05:08.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b835c40 2024-05-08T18:05:08.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc365b51800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc365b51800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:08.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc365b51800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b839530 con 0x559b2b835c40 2024-05-08T18:05:08.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b835c40 2024-05-08T18:05:08.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc365b51800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc365b51800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b835c40 2024-05-08T18:05:08.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b835c40 2024-05-08T18:05:08.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc365b51800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:08.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc365b51800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.134+0000 7fc365b51800 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559b2b83a3c0 0x559b2b83c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.135+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b83cef0 con 0x559b2b83a3c0 2024-05-08T18:05:08.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.135+0000 7fc363650640 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559b2b83a3c0 0x559b2b83c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.135+0000 7fc363650640 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559b2b83a3c0 0x559b2b83c7b0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.135+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fc354073020 con 0x559b2b83a3c0 2024-05-08T18:05:08.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.135+0000 7fc365b51800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:08.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.135+0000 7fc365b51800 20 searching for the correct realm 2024-05-08T18:05:08.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.135+0000 7fc365b51800 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559b2b83ef90 0x559b2b841450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.135+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b841b20 con 0x559b2b83ef90 2024-05-08T18:05:08.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.135+0000 7fc362e4f640 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559b2b83ef90 0x559b2b841450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.136+0000 7fc362e4f640 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559b2b83ef90 0x559b2b841450 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.136+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc34c05a040 con 0x559b2b83ef90 2024-05-08T18:05:08.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.136+0000 7fc362e4f640 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fc34c05c410 0x7fc34c05c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.136+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b841b20 con 0x7fc34c05c410 2024-05-08T18:05:08.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.136+0000 7fc363e51640 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fc34c05c410 0x7fc34c05c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.137+0000 7fc363e51640 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fc34c05c410 0x7fc34c05c850 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.137+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc35c00a4a0 con 0x7fc34c05c410 2024-05-08T18:05:08.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.137+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83a3c0 2024-05-08T18:05:08.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.137+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc354073020 con 0x559b2b83a3c0 2024-05-08T18:05:08.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.137+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b835c40 2024-05-08T18:05:08.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.137+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b835c40 2024-05-08T18:05:08.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.137+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b835c40 2024-05-08T18:05:08.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.138+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b835c40 2024-05-08T18:05:08.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.138+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83a3c0 2024-05-08T18:05:08.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.138+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc354073020 con 0x559b2b83a3c0 2024-05-08T18:05:08.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.138+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83a3c0 2024-05-08T18:05:08.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.138+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fc354073020 con 0x559b2b83a3c0 2024-05-08T18:05:08.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.138+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x7fc34c05c410 2024-05-08T18:05:08.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.138+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc35c00a4a0 con 0x7fc34c05c410 2024-05-08T18:05:08.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.138+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83ef90 2024-05-08T18:05:08.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.139+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc34c05a040 con 0x559b2b83ef90 2024-05-08T18:05:08.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.139+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b835c40 2024-05-08T18:05:08.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.139+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b835c40 2024-05-08T18:05:08.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.139+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83a3c0 2024-05-08T18:05:08.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.139+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc354073020 con 0x559b2b83a3c0 2024-05-08T18:05:08.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.139+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b835c40 2024-05-08T18:05:08.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.139+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b835c40 2024-05-08T18:05:08.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.139+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x7fc34c05c410 2024-05-08T18:05:08.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.140+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fc35c00a4a0 con 0x7fc34c05c410 2024-05-08T18:05:08.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.140+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83ef90 2024-05-08T18:05:08.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.140+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc34c05a040 con 0x559b2b83ef90 2024-05-08T18:05:08.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.140+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83ef90 2024-05-08T18:05:08.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.140+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc34c05a040 con 0x559b2b83ef90 2024-05-08T18:05:08.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.140+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83a3c0 2024-05-08T18:05:08.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.140+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc354085040 con 0x559b2b83a3c0 2024-05-08T18:05:08.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.140+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83ef90 2024-05-08T18:05:08.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.141+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc34c05a040 con 0x559b2b83ef90 2024-05-08T18:05:08.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.141+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83ef90 2024-05-08T18:05:08.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.141+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fc34c05a040 con 0x559b2b83ef90 2024-05-08T18:05:08.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.141+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b835c40 2024-05-08T18:05:08.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.141+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b835c40 2024-05-08T18:05:08.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.141+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b835c40 2024-05-08T18:05:08.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.141+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b835c40 2024-05-08T18:05:08.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.141+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83a3c0 2024-05-08T18:05:08.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.142+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc354085040 con 0x559b2b83a3c0 2024-05-08T18:05:08.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.142+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83ef90 2024-05-08T18:05:08.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.142+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc34c05a040 con 0x559b2b83ef90 2024-05-08T18:05:08.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.142+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x7fc34c05c410 2024-05-08T18:05:08.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.142+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc35c00a4a0 con 0x7fc34c05c410 2024-05-08T18:05:08.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.142+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b835c40 2024-05-08T18:05:08.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.142+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b835c40 2024-05-08T18:05:08.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.142+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83a3c0 2024-05-08T18:05:08.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.143+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc354085040 con 0x559b2b83a3c0 2024-05-08T18:05:08.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.143+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83ef90 2024-05-08T18:05:08.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.143+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc34c05a040 con 0x559b2b83ef90 2024-05-08T18:05:08.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.143+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83a3c0 2024-05-08T18:05:08.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.143+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc354085040 con 0x559b2b83a3c0 2024-05-08T18:05:08.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.143+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x7fc34c05c410 2024-05-08T18:05:08.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.144+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc35c00a4a0 con 0x7fc34c05c410 2024-05-08T18:05:08.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.144+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83ef90 2024-05-08T18:05:08.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.144+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc34c05a040 con 0x559b2b83ef90 2024-05-08T18:05:08.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.144+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b83a3c0 2024-05-08T18:05:08.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.144+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc354085040 con 0x559b2b83a3c0 2024-05-08T18:05:08.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.144+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x559b2b835c40 2024-05-08T18:05:08.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.144+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b835c40 2024-05-08T18:05:08.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.144+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8398d0 con 0x7fc34c05c410 2024-05-08T18:05:08.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc35c00a4a0 con 0x7fc34c05c410 2024-05-08T18:05:08.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:08.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:08.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:08.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:08.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:08.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:08.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b83d890 con 0x559b2b835c40 2024-05-08T18:05:08.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b835c40 2024-05-08T18:05:08.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b83d890 con 0x559b2b835c40 2024-05-08T18:05:08.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc35c00a4a0 con 0x559b2b835c40 2024-05-08T18:05:08.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:08.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b83d840 con 0x559b2b83ef90 2024-05-08T18:05:08.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fc34c05a040 con 0x559b2b83ef90 2024-05-08T18:05:08.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:08.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 20 zone default found 2024-05-08T18:05:08.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 4 Realm: () 2024-05-08T18:05:08.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:08.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:08.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:08.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 20 zonegroup default 2024-05-08T18:05:08.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.145+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b83e130 con 0x559b2b83a3c0 2024-05-08T18:05:08.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.146+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fc354085040 con 0x559b2b83a3c0 2024-05-08T18:05:08.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.146+0000 7fc365b51800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.146+0000 7fc365b51800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.146+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b843980 con 0x559b2b83ef90 2024-05-08T18:05:08.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.146+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc34c05a040 con 0x559b2b83ef90 2024-05-08T18:05:08.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.146+0000 7fc365b51800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.146+0000 7fc365b51800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.146+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b843d20 con 0x559b2b83ef90 2024-05-08T18:05:08.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.146+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc34c05a040 con 0x559b2b83ef90 2024-05-08T18:05:08.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.146+0000 7fc365b51800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.146+0000 7fc365b51800 20 started sync module instance, tier type = 2024-05-08T18:05:08.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.146+0000 7fc365b51800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:08.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.146+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b842e20 con 0x7fc34c05c410 2024-05-08T18:05:08.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.147+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc35c00a4a0 con 0x7fc34c05c410 2024-05-08T18:05:08.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.147+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94124938376016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b842e20 con 0x7fc34c05c410 2024-05-08T18:05:08.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.147+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b844740 con 0x559b2b835c40 2024-05-08T18:05:08.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.148+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94124938376016] v27'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc35c040040 con 0x7fc34c05c410 2024-05-08T18:05:08.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.148+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc35c040040 con 0x559b2b835c40 2024-05-08T18:05:08.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.149+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94124938379824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b842e20 con 0x559b2b835c40 2024-05-08T18:05:08.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.149+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8451e0 con 0x559b2b83ef90 2024-05-08T18:05:08.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.149+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94124938379824] v27'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc35c040040 con 0x559b2b835c40 2024-05-08T18:05:08.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.150+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc34c05a040 con 0x559b2b83ef90 2024-05-08T18:05:08.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.150+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94124938373488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b842d90 con 0x559b2b83ef90 2024-05-08T18:05:08.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.150+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8460a0 con 0x7fc34c05c410 2024-05-08T18:05:08.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.150+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94124938373488] v27'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc34c05a040 con 0x559b2b83ef90 2024-05-08T18:05:08.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.151+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc35c040040 con 0x7fc34c05c410 2024-05-08T18:05:08.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.151+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94124938386416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8464c0 con 0x7fc34c05c410 2024-05-08T18:05:08.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.151+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b846e80 con 0x559b2b835c40 2024-05-08T18:05:08.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.152+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94124938386416] v27'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc35c040040 con 0x7fc34c05c410 2024-05-08T18:05:08.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.152+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc35c040040 con 0x559b2b835c40 2024-05-08T18:05:08.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.152+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94124938387952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b846e80 con 0x559b2b835c40 2024-05-08T18:05:08.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.152+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b847840 con 0x559b2b83a3c0 2024-05-08T18:05:08.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.153+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94124938387952] v27'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc35c040040 con 0x559b2b835c40 2024-05-08T18:05:08.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.153+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc354085040 con 0x559b2b83a3c0 2024-05-08T18:05:08.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.153+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94124938389968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b846df0 con 0x559b2b83a3c0 2024-05-08T18:05:08.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.153+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b848610 con 0x559b2b83a3c0 2024-05-08T18:05:08.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.154+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94124938389968] v27'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc354085040 con 0x559b2b83a3c0 2024-05-08T18:05:08.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.154+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc354085040 con 0x559b2b83a3c0 2024-05-08T18:05:08.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.154+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94124938394672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8481d0 con 0x559b2b83a3c0 2024-05-08T18:05:08.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.154+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8493d0 con 0x559b2b83a3c0 2024-05-08T18:05:08.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.155+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v27'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc354085040 con 0x559b2b83a3c0 2024-05-08T18:05:08.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.155+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94124938394672] v27'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc354085040 con 0x559b2b83a3c0 2024-05-08T18:05:08.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.155+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94124938399424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b849390 con 0x559b2b83a3c0 2024-05-08T18:05:08.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.155+0000 7fc365b51800 20 add_watcher() i=0 2024-05-08T18:05:08.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.155+0000 7fc365b51800 20 add_watcher() i=1 2024-05-08T18:05:08.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.155+0000 7fc365b51800 20 add_watcher() i=2 2024-05-08T18:05:08.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.155+0000 7fc365b51800 20 add_watcher() i=3 2024-05-08T18:05:08.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.155+0000 7fc365b51800 20 add_watcher() i=4 2024-05-08T18:05:08.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.155+0000 7fc365b51800 20 add_watcher() i=5 2024-05-08T18:05:08.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.155+0000 7fc365b51800 20 add_watcher() i=6 2024-05-08T18:05:08.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.156+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94124938399424] v27'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc354085040 con 0x559b2b83a3c0 2024-05-08T18:05:08.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.156+0000 7fc365b51800 20 add_watcher() i=7 2024-05-08T18:05:08.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.156+0000 7fc365b51800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:08.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.156+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b84c880 con 0x559b2b835c40 2024-05-08T18:05:08.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.156+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fc35c040040 con 0x559b2b835c40 2024-05-08T18:05:08.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.156+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94124938397680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b849dd0 con 0x559b2b835c40 2024-05-08T18:05:08.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.157+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94124938397680] v27'108 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc35c040040 con 0x559b2b835c40 2024-05-08T18:05:08.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.157+0000 7fc219fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:08.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.157+0000 7fc365b51800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:08.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.157+0000 7fc365b51800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:08.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.157+0000 7fc365b51800 5 note: GC not initialized 2024-05-08T18:05:08.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.158+0000 7fc218fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:08.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.158+0000 7fc365b51800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:08.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.158+0000 7fc365b51800 20 Filter name: none 2024-05-08T18:05:08.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.158+0000 7fc1927fc640 20 reqs_thread_entry: start 2024-05-08T18:05:08.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.158+0000 7fc365b51800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.158+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559b2b87f7e0 con 0x7fc34c05c410 2024-05-08T18:05:08.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.158+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(62 myfoo [call out=48b,stat out=16b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv20 ondisk = 0) ==== 275+0+304 (crc 0 0 0) 0x7fc35c040040 con 0x7fc34c05c410 2024-05-08T18:05:08.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.158+0000 7fc365b51800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:08.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.159+0000 7fc219fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:08.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.159+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94124938397680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b87d780 con 0x559b2b835c40 2024-05-08T18:05:08.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.160+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94124938397680] v27'109 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc35c040040 con 0x559b2b835c40 2024-05-08T18:05:08.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.160+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94124938376016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b8487f0 con 0x7fc34c05c410 2024-05-08T18:05:08.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.161+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94124938376016] v27'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc35c040040 con 0x7fc34c05c410 2024-05-08T18:05:08.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.161+0000 7fc365b51800 20 remove_watcher() i=0 2024-05-08T18:05:08.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.161+0000 7fc365b51800 2 removed watcher, disabling cache 2024-05-08T18:05:08.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.161+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94124938379824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b843350 con 0x559b2b835c40 2024-05-08T18:05:08.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.162+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94124938379824] v27'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc35c040040 con 0x559b2b835c40 2024-05-08T18:05:08.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.163+0000 7fc365b51800 20 remove_watcher() i=1 2024-05-08T18:05:08.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.163+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94124938373488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b843350 con 0x559b2b83ef90 2024-05-08T18:05:08.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.163+0000 7fc362e4f640 1 -- 172.21.15.50:0/877546953 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94124938373488] v27'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc34c05a040 con 0x559b2b83ef90 2024-05-08T18:05:08.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.163+0000 7fc365b51800 20 remove_watcher() i=2 2024-05-08T18:05:08.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.163+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94124938386416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b842970 con 0x7fc34c05c410 2024-05-08T18:05:08.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.164+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94124938386416] v27'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc35c040040 con 0x7fc34c05c410 2024-05-08T18:05:08.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.164+0000 7fc365b51800 20 remove_watcher() i=3 2024-05-08T18:05:08.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.164+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94124938387952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b843350 con 0x559b2b835c40 2024-05-08T18:05:08.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.165+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94124938387952] v27'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc35c040040 con 0x559b2b835c40 2024-05-08T18:05:08.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.165+0000 7fc365b51800 20 remove_watcher() i=4 2024-05-08T18:05:08.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.165+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94124938389968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b845bf0 con 0x559b2b83a3c0 2024-05-08T18:05:08.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.166+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94124938389968] v27'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc354085040 con 0x559b2b83a3c0 2024-05-08T18:05:08.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.166+0000 7fc365b51800 20 remove_watcher() i=5 2024-05-08T18:05:08.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.166+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94124938394672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b842970 con 0x559b2b83a3c0 2024-05-08T18:05:08.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.167+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94124938394672] v27'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc354085040 con 0x559b2b83a3c0 2024-05-08T18:05:08.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.167+0000 7fc365b51800 20 remove_watcher() i=6 2024-05-08T18:05:08.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.167+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94124938399424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559b2b843350 con 0x559b2b83a3c0 2024-05-08T18:05:08.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.168+0000 7fc363650640 1 -- 172.21.15.50:0/877546953 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94124938399424] v27'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc354085040 con 0x559b2b83a3c0 2024-05-08T18:05:08.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.168+0000 7fc365b51800 20 remove_watcher() i=7 2024-05-08T18:05:08.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559b2b83a3c0 msgr2=0x559b2b83c7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559b2b83a3c0 0x559b2b83c7b0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559b2b835c40 msgr2=0x559b2b838100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559b2b835c40 0x559b2b838100 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fc34c05c410 msgr2=0x7fc34c05c850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fc34c05c410 0x7fc34c05c850 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559b2b83ef90 msgr2=0x559b2b841450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559b2b83ef90 0x559b2b841450 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc22c03bba0 msgr2=0x7fc22c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc22c03bba0 0x7fc22c03e060 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x559b2b8082d0 tx=0x7fc34c04f000 comp rx=0 tx=0).stop 2024-05-08T18:05:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b823210 msgr2=0x559b2b82a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 --2- 172.21.15.50:0/877546953 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b823210 0x559b2b82a5e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc354008fd0 tx=0x7fc354016f70 comp rx=0 tx=0).stop 2024-05-08T18:05:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 reap_dead start 2024-05-08T18:05:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc363e51640 1 -- 172.21.15.50:0/877546953 reap_dead start 2024-05-08T18:05:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 shutdown_connections 2024-05-08T18:05:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 >> 172.21.15.50:0/877546953 conn(0x559b2b81f490 msgr2=0x559b2b81caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 shutdown_connections 2024-05-08T18:05:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 -- 172.21.15.50:0/877546953 wait complete. 2024-05-08T18:05:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 -- 172.21.15.50:0/799248142 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc23c03bbf0 msgr2=0x7fc23c03e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 --2- 172.21.15.50:0/799248142 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc23c03bbf0 0x7fc23c03e0b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc34c055040 tx=0x7fc34c005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 -- 172.21.15.50:0/799248142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b816200 msgr2=0x559b2b816620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 --2- 172.21.15.50:0/799248142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b816200 0x559b2b816620 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc354018c60 tx=0x7fc354023670 comp rx=0 tx=0).stop 2024-05-08T18:05:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 -- 172.21.15.50:0/799248142 shutdown_connections 2024-05-08T18:05:08.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 --2- 172.21.15.50:0/799248142 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc23c03bbf0 0x7fc23c03e0b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 --2- 172.21.15.50:0/799248142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b816200 0x559b2b816620 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 -- 172.21.15.50:0/799248142 >> 172.21.15.50:0/799248142 conn(0x559b2b80a2a0 msgr2=0x559b2b80a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 -- 172.21.15.50:0/799248142 shutdown_connections 2024-05-08T18:05:08.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.169+0000 7fc365b51800 1 -- 172.21.15.50:0/799248142 wait complete. 2024-05-08T18:05:08.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.170+0000 7fc365b51800 1 -- 172.21.15.50:0/3757725993 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559b2b8014f0 msgr2=0x559b2b8038e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.170+0000 7fc365b51800 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559b2b8014f0 0x559b2b8038e0 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.170+0000 7fc365b51800 1 -- 172.21.15.50:0/3757725993 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559b2b6fcad0 msgr2=0x559b2b7fec30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.170+0000 7fc365b51800 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559b2b6fcad0 0x559b2b7fec30 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.170+0000 7fc365b51800 1 -- 172.21.15.50:0/3757725993 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559b2b8053e0 msgr2=0x559b2b8077d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.170+0000 7fc365b51800 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559b2b8053e0 0x559b2b8077d0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.170+0000 7fc365b51800 1 -- 172.21.15.50:0/3757725993 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc24803bb50 msgr2=0x7fc24803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.170+0000 7fc365b51800 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fc24803bb50 0x7fc24803e010 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fc34c054000 tx=0x7fc34c04e000 comp rx=0 tx=0).stop 2024-05-08T18:05:08.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.170+0000 7fc365b51800 1 -- 172.21.15.50:0/3757725993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6faf90 msgr2=0x559b2b7f7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.170+0000 7fc365b51800 1 --2- 172.21.15.50:0/3757725993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559b2b6faf90 0x559b2b7f7a40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc354000f60 tx=0x7fc354000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.170+0000 7fc363e51640 1 -- 172.21.15.50:0/3757725993 reap_dead start 2024-05-08T18:05:08.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.170+0000 7fc365b51800 1 -- 172.21.15.50:0/3757725993 shutdown_connections 2024-05-08T18:05:08.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.170+0000 7fc365b51800 1 -- 172.21.15.50:0/3757725993 >> 172.21.15.50:0/3757725993 conn(0x559b2b6f6ad0 msgr2=0x559b2b7e9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.170+0000 7fc365b51800 1 -- 172.21.15.50:0/3757725993 shutdown_connections 2024-05-08T18:05:08.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.170+0000 7fc365b51800 1 -- 172.21.15.50:0/3757725993 wait complete. 2024-05-08T18:05:08.177 INFO:teuthology.orchestra.run.smithi050.stdout:{"key":"bucket:myfoo","ver":{"tag":"_8NIcHujJvsHHL9dZkArjSuH","ver":2},"mtime":"2024-05-08T18:05:07.883289Z","data":{"bucket":{"name":"myfoo","marker":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1","bucket_id":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1","tenant":"","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""}},"owner":"foo","creation_time":"2024-05-08T18:05:06.706377Z","linked":false,"has_bucket_info":false}} 2024-05-08T18:05:08.177 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:myfoo', 'ver': {'tag': '_8NIcHujJvsHHL9dZkArjSuH', 'ver': 2}, 'mtime': '2024-05-08T18:05:07.883289Z', 'data': {'bucket': {'name': 'myfoo', 'marker': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1', 'bucket_id': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1', 'tenant': '', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'foo', 'creation_time': '2024-05-08T18:05:06.706377Z', 'linked': False, 'has_bucket_info': False}} 2024-05-08T18:05:08.177 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1'] 2024-05-08T18:05:08.178 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1'] 2024-05-08T18:05:08.178 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid fud --bucket myfoo --bucket-id 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 2024-05-08T18:05:08.227 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:08.228 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:08.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.230+0000 7f7f0281f800 1 Processor -- start 2024-05-08T18:05:08.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.230+0000 7f7f0281f800 1 -- start start 2024-05-08T18:05:08.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.230+0000 7f7f0281f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x559f7a180a20 0x559f7a182e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.231+0000 7f7f0281f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559f7a1833e0 con 0x559f7a17ca70 2024-05-08T18:05:08.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.231+0000 7f7f0281f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x559f7a01de20 con 0x559f7a180a20 2024-05-08T18:05:08.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.231+0000 7f7ef7b16640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x559f7a180a20 0x559f7a182e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.231+0000 7f7f00317640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x559f7a17ca70 0x559f7a17ce90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43174/0 (socket says 172.21.15.50:43174) 2024-05-08T18:05:08.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.231+0000 7f7f00317640 1 -- 172.21.15.50:0/3601697377 learned_addr learned my addr 172.21.15.50:0/3601697377 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:08.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.231+0000 7f7ef7b16640 1 -- 172.21.15.50:0/3601697377 >> v1:172.21.15.46:6789/0 conn(0x559f7a17ca70 legacy=0x559f7a17ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.231+0000 7f7ef7b16640 1 -- 172.21.15.50:0/3601697377 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7a1845c0 con 0x559f7a180a20 2024-05-08T18:05:08.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.231+0000 7f7effb16640 1 -- 172.21.15.50:0/3601697377 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3148788029 0 0) 0x559f7a1833e0 con 0x559f7a17ca70 2024-05-08T18:05:08.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.233+0000 7f7ef7b16640 1 --2- 172.21.15.50:0/3601697377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a180a20 0x559f7a182e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800a470 tx=0x7f7ee801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8317d8c91cc3da server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.233+0000 7f7effb16640 1 -- 172.21.15.50:0/3601697377 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ee8057070 con 0x559f7a180a20 2024-05-08T18:05:08.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.233+0000 7f7effb16640 1 -- 172.21.15.50:0/3601697377 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee8052570 con 0x559f7a180a20 2024-05-08T18:05:08.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.233+0000 7f7effb16640 1 -- 172.21.15.50:0/3601697377 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ee8052870 con 0x559f7a180a20 2024-05-08T18:05:08.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.234+0000 7f7f0281f800 1 -- 172.21.15.50:0/3601697377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a180a20 msgr2=0x559f7a182e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.234+0000 7f7f0281f800 1 --2- 172.21.15.50:0/3601697377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a180a20 0x559f7a182e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800a470 tx=0x7f7ee801ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:08.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.234+0000 7f7f0281f800 1 -- 172.21.15.50:0/3601697377 shutdown_connections 2024-05-08T18:05:08.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.234+0000 7f7f0281f800 1 --2- 172.21.15.50:0/3601697377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a180a20 0x559f7a182e10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.234+0000 7f7f0281f800 1 -- 172.21.15.50:0/3601697377 >> 172.21.15.50:0/3601697377 conn(0x559f7a1772f0 msgr2=0x559f7a179710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.234+0000 7f7f0281f800 1 -- 172.21.15.50:0/3601697377 shutdown_connections 2024-05-08T18:05:08.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.234+0000 7f7f0281f800 1 -- 172.21.15.50:0/3601697377 wait complete. 2024-05-08T18:05:08.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.249+0000 7f7f0281f800 1 Processor -- start 2024-05-08T18:05:08.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.249+0000 7f7f0281f800 1 -- start start 2024-05-08T18:05:08.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.249+0000 7f7f0281f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a17ca70 0x559f7a26d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.249+0000 7f7f0281f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559f7a0c8a30 con 0x559f7a17ca70 2024-05-08T18:05:08.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.250+0000 7f7f00317640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a17ca70 0x559f7a26d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.250+0000 7f7f00317640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a17ca70 0x559f7a26d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37840/0 (socket says 172.21.15.50:37840) 2024-05-08T18:05:08.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.250+0000 7f7f00317640 1 -- 172.21.15.50:0/3715737469 learned_addr learned my addr 172.21.15.50:0/3715737469 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.250+0000 7f7f00317640 1 -- 172.21.15.50:0/3715737469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7a26df30 con 0x559f7a17ca70 2024-05-08T18:05:08.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.250+0000 7f7f00317640 1 --2- 172.21.15.50:0/3715737469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a17ca70 0x559f7a26d960 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0001e70 tx=0x7f7ef000b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f596c715257cf1e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.250+0000 7f7dee60c640 1 -- 172.21.15.50:0/3715737469 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef0008070 con 0x559f7a17ca70 2024-05-08T18:05:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.250+0000 7f7dee60c640 1 -- 172.21.15.50:0/3715737469 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef000d500 con 0x559f7a17ca70 2024-05-08T18:05:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.250+0000 7f7dee60c640 1 -- 172.21.15.50:0/3715737469 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef000d8a0 con 0x559f7a17ca70 2024-05-08T18:05:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.251+0000 7f7f0281f800 1 -- 172.21.15.50:0/3715737469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a17ca70 msgr2=0x559f7a26d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.251+0000 7f7f0281f800 1 --2- 172.21.15.50:0/3715737469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a17ca70 0x559f7a26d960 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0001e70 tx=0x7f7ef000b6a0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.251+0000 7f7f0281f800 1 -- 172.21.15.50:0/3715737469 shutdown_connections 2024-05-08T18:05:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.251+0000 7f7f0281f800 1 --2- 172.21.15.50:0/3715737469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a17ca70 0x559f7a26d960 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.251+0000 7f7f0281f800 1 -- 172.21.15.50:0/3715737469 >> 172.21.15.50:0/3715737469 conn(0x559f7a1772f0 msgr2=0x559f7a269280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.251+0000 7f7f0281f800 1 -- 172.21.15.50:0/3715737469 shutdown_connections 2024-05-08T18:05:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.251+0000 7f7f0281f800 1 -- 172.21.15.50:0/3715737469 wait complete. 2024-05-08T18:05:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.251+0000 7f7f0281f800 1 Processor -- start 2024-05-08T18:05:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.251+0000 7f7f0281f800 1 -- start start 2024-05-08T18:05:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.251+0000 7f7f0281f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a17ca70 0x559f7a2787b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.251+0000 7f7f0281f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559f7a1838b0 con 0x559f7a17ca70 2024-05-08T18:05:08.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.252+0000 7f7f00317640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a17ca70 0x559f7a2787b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.252+0000 7f7f00317640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a17ca70 0x559f7a2787b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37852/0 (socket says 172.21.15.50:37852) 2024-05-08T18:05:08.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.252+0000 7f7f00317640 1 -- 172.21.15.50:0/3030227668 learned_addr learned my addr 172.21.15.50:0/3030227668 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.252+0000 7f7f00317640 1 -- 172.21.15.50:0/3030227668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7a278cf0 con 0x559f7a17ca70 2024-05-08T18:05:08.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.252+0000 7f7f00317640 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a17ca70 0x559f7a2787b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x559f7a26e6f0 tx=0x7f7ef0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.252+0000 7f7dece09640 1 -- 172.21.15.50:0/3030227668 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef000f980 con 0x559f7a17ca70 2024-05-08T18:05:08.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.252+0000 7f7dece09640 1 -- 172.21.15.50:0/3030227668 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef000fb20 con 0x559f7a17ca70 2024-05-08T18:05:08.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.252+0000 7f7f0281f800 1 -- 172.21.15.50:0/3030227668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f7a278f80 con 0x559f7a17ca70 2024-05-08T18:05:08.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.252+0000 7f7dece09640 1 -- 172.21.15.50:0/3030227668 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef000fe00 con 0x559f7a17ca70 2024-05-08T18:05:08.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.252+0000 7f7f0281f800 1 -- 172.21.15.50:0/3030227668 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7a27bc30 con 0x559f7a17ca70 2024-05-08T18:05:08.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.253+0000 7f7dece09640 1 -- 172.21.15.50:0/3030227668 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7ef0017040 con 0x559f7a17ca70 2024-05-08T18:05:08.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.253+0000 7f7dece09640 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7de803bba0 0x7f7de803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.253+0000 7f7dece09640 1 -- 172.21.15.50:0/3030227668 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7ef0015020 con 0x559f7a17ca70 2024-05-08T18:05:08.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.254+0000 7f7ef7b16640 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7de803bba0 0x7f7de803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.254+0000 7f7f0281f800 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559f7a180a20 0x559f7a27f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.254+0000 7f7f0281f800 1 -- 172.21.15.50:0/3030227668 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a27ff30 con 0x559f7a180a20 2024-05-08T18:05:08.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.254+0000 7f7f00b18640 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559f7a180a20 0x559f7a27f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.254+0000 7f7ef7b16640 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7de803bba0 0x7f7de803e060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8054000 tx=0x7f7ee804e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.254+0000 7f7f00b18640 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559f7a180a20 0x559f7a27f9f0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.254+0000 7f7f00b18640 1 -- 172.21.15.50:0/3030227668 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a180a20 2024-05-08T18:05:08.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.254+0000 7f7f0281f800 1 -- 172.21.15.50:0/3030227668 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a27cfc0 con 0x559f7a180a20 2024-05-08T18:05:08.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.255+0000 7f7f00b18640 1 -- 172.21.15.50:0/3030227668 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a180a20 2024-05-08T18:05:08.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.255+0000 7f7f0281f800 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559f7a282240 0x559f7a284630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.255+0000 7f7f0281f800 1 -- 172.21.15.50:0/3030227668 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a284d20 con 0x559f7a282240 2024-05-08T18:05:08.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.255+0000 7f7f00317640 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559f7a282240 0x559f7a284630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.255+0000 7f7f00317640 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559f7a282240 0x559f7a284630 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.256+0000 7f7f00317640 1 -- 172.21.15.50:0/3030227668 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7ef002d070 con 0x559f7a282240 2024-05-08T18:05:08.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.256+0000 7f7f0281f800 1 -- 172.21.15.50:0/3030227668 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2810e0 con 0x559f7a180a20 2024-05-08T18:05:08.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.256+0000 7f7f00b18640 1 -- 172.21.15.50:0/3030227668 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a180a20 2024-05-08T18:05:08.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.256+0000 7f7f0281f800 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559f7a286180 0x559f7a288570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.256+0000 7f7f0281f800 1 -- 172.21.15.50:0/3030227668 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a288c70 con 0x559f7a286180 2024-05-08T18:05:08.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.256+0000 7f7ef7b16640 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559f7a286180 0x559f7a288570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.256+0000 7f7ef7b16640 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559f7a286180 0x559f7a288570 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.257+0000 7f7ef7b16640 1 -- 172.21.15.50:0/3030227668 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7ee8052940 con 0x559f7a286180 2024-05-08T18:05:08.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.257+0000 7f7f0281f800 1 Processor -- start 2024-05-08T18:05:08.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.257+0000 7f7f0281f800 1 -- start start 2024-05-08T18:05:08.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.257+0000 7f7f0281f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a296fe0 0x559f7a297400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.257+0000 7f7f0281f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559f7a06f330 con 0x559f7a296fe0 2024-05-08T18:05:08.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.257+0000 7f7f00317640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a296fe0 0x559f7a297400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.257+0000 7f7f00317640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a296fe0 0x559f7a297400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37856/0 (socket says 172.21.15.50:37856) 2024-05-08T18:05:08.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.257+0000 7f7f00317640 1 -- 172.21.15.50:0/148958808 learned_addr learned my addr 172.21.15.50:0/148958808 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.258+0000 7f7f00317640 1 -- 172.21.15.50:0/148958808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7a297940 con 0x559f7a296fe0 2024-05-08T18:05:08.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.258+0000 7f7f00317640 1 --2- 172.21.15.50:0/148958808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a296fe0 0x559f7a297400 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000ecf0 tx=0x7f7ef001ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.258+0000 7f7de5ffb640 1 -- 172.21.15.50:0/148958808 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef004a350 con 0x559f7a296fe0 2024-05-08T18:05:08.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.258+0000 7f7de5ffb640 1 -- 172.21.15.50:0/148958808 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef001a3c0 con 0x559f7a296fe0 2024-05-08T18:05:08.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.258+0000 7f7de5ffb640 1 -- 172.21.15.50:0/148958808 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef00501b0 con 0x559f7a296fe0 2024-05-08T18:05:08.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.258+0000 7f7f0281f800 1 -- 172.21.15.50:0/148958808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f7a2983b0 con 0x559f7a296fe0 2024-05-08T18:05:08.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.258+0000 7f7f0281f800 1 -- 172.21.15.50:0/148958808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7a2988b0 con 0x559f7a296fe0 2024-05-08T18:05:08.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.259+0000 7f7de5ffb640 1 -- 172.21.15.50:0/148958808 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7ef0051b70 con 0x559f7a296fe0 2024-05-08T18:05:08.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.259+0000 7f7de5ffb640 1 --2- 172.21.15.50:0/148958808 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7dd803bba0 0x7f7dd803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.259+0000 7f7de5ffb640 1 -- 172.21.15.50:0/148958808 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7ef003c9e0 con 0x559f7a296fe0 2024-05-08T18:05:08.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.259+0000 7f7ef7b16640 1 --2- 172.21.15.50:0/148958808 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7dd803bba0 0x7f7dd803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.259+0000 7f7f0281f800 1 Processor -- start 2024-05-08T18:05:08.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.259+0000 7f7f0281f800 1 -- start start 2024-05-08T18:05:08.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.260+0000 7f7ef7b16640 1 --2- 172.21.15.50:0/148958808 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7dd803bba0 0x7f7dd803e060 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x559f7a2807f0 tx=0x7f7ee8005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.260+0000 7f7f0281f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a2a42a0 0x559f7a2a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.260+0000 7f7f0281f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559f7a26e280 con 0x559f7a2a42a0 2024-05-08T18:05:08.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.260+0000 7f7f00317640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a2a42a0 0x559f7a2a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.260+0000 7f7f00317640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a2a42a0 0x559f7a2a46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37872/0 (socket says 172.21.15.50:37872) 2024-05-08T18:05:08.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.260+0000 7f7f00317640 1 -- 172.21.15.50:0/2769736159 learned_addr learned my addr 172.21.15.50:0/2769736159 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.260+0000 7f7f00317640 1 -- 172.21.15.50:0/2769736159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7a2a4c90 con 0x559f7a2a42a0 2024-05-08T18:05:08.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.260+0000 7f7f00317640 1 --2- 172.21.15.50:0/2769736159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a2a42a0 0x559f7a2a46c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7ef001ac70 tx=0x7f7ef002f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999c4327dd8b4aac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.260+0000 7f7dd77fe640 1 -- 172.21.15.50:0/2769736159 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef002f5a0 con 0x559f7a2a42a0 2024-05-08T18:05:08.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.260+0000 7f7dd77fe640 1 -- 172.21.15.50:0/2769736159 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef003c070 con 0x559f7a2a42a0 2024-05-08T18:05:08.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.260+0000 7f7dd77fe640 1 -- 172.21.15.50:0/2769736159 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef0050350 con 0x559f7a2a42a0 2024-05-08T18:05:08.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.261+0000 7f7f0281f800 1 -- 172.21.15.50:0/2769736159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a2a42a0 msgr2=0x559f7a2a46c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.261+0000 7f7f0281f800 1 --2- 172.21.15.50:0/2769736159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a2a42a0 0x559f7a2a46c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f7ef001ac70 tx=0x7f7ef002f8c0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.261+0000 7f7f0281f800 1 -- 172.21.15.50:0/2769736159 shutdown_connections 2024-05-08T18:05:08.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.261+0000 7f7f0281f800 1 --2- 172.21.15.50:0/2769736159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a2a42a0 0x559f7a2a46c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.261+0000 7f7f0281f800 1 -- 172.21.15.50:0/2769736159 >> 172.21.15.50:0/2769736159 conn(0x559f7a2a05c0 msgr2=0x559f7a29e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.261+0000 7f7f0281f800 1 -- 172.21.15.50:0/2769736159 shutdown_connections 2024-05-08T18:05:08.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.261+0000 7f7f0281f800 1 -- 172.21.15.50:0/2769736159 wait complete. 2024-05-08T18:05:08.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.261+0000 7f7f0281f800 1 Processor -- start 2024-05-08T18:05:08.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.261+0000 7f7f0281f800 1 -- start start 2024-05-08T18:05:08.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.261+0000 7f7f0281f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a2a42a0 0x559f7a2ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.261+0000 7f7f0281f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559f7a2a53b0 con 0x559f7a2a42a0 2024-05-08T18:05:08.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.262+0000 7f7f00317640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a2a42a0 0x559f7a2ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.262+0000 7f7f00317640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a2a42a0 0x559f7a2ab6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37878/0 (socket says 172.21.15.50:37878) 2024-05-08T18:05:08.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.262+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 learned_addr learned my addr 172.21.15.50:0/1180284538 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.262+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7a2abbf0 con 0x559f7a2a42a0 2024-05-08T18:05:08.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.262+0000 7f7f00317640 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a2a42a0 0x559f7a2ab6b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000bbf0 tx=0x7f7ef003ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.262+0000 7f7dd5ffb640 1 -- 172.21.15.50:0/1180284538 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef00120d0 con 0x559f7a2a42a0 2024-05-08T18:05:08.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.262+0000 7f7dd5ffb640 1 -- 172.21.15.50:0/1180284538 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef0042400 con 0x559f7a2a42a0 2024-05-08T18:05:08.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.262+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f7a2abbf0 con 0x559f7a2a42a0 2024-05-08T18:05:08.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.262+0000 7f7dd5ffb640 1 -- 172.21.15.50:0/1180284538 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ef0019470 con 0x559f7a2a42a0 2024-05-08T18:05:08.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.262+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7a2abbf0 con 0x559f7a2a42a0 2024-05-08T18:05:08.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.263+0000 7f7dd5ffb640 1 -- 172.21.15.50:0/1180284538 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7ef0019610 con 0x559f7a2a42a0 2024-05-08T18:05:08.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.264+0000 7f7dd5ffb640 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7dc803bba0 0x7f7dc803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.264+0000 7f7dd5ffb640 1 -- 172.21.15.50:0/1180284538 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7ef0066070 con 0x559f7a2a42a0 2024-05-08T18:05:08.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.264+0000 7f7ef7b16640 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7dc803bba0 0x7f7dc803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.264+0000 7f7ef7b16640 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7dc803bba0 0x7f7dc803e060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8052fd0 tx=0x7f7ee804f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.264+0000 7f7f0281f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.264+0000 7f7f0281f800 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559f7a2b6d60 0x559f7a2b9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.264+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2b9970 con 0x559f7a2b6d60 2024-05-08T18:05:08.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.264+0000 7f7f00b18640 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559f7a2b6d60 0x559f7a2b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.265+0000 7f7f00b18640 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559f7a2b6d60 0x559f7a2b9220 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.265+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a2b6d60 2024-05-08T18:05:08.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.265+0000 7f7f0281f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.265+0000 7f7f0281f800 20 realm 2024-05-08T18:05:08.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.265+0000 7f7f0281f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.265+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2b9d80 con 0x559f7a2b6d60 2024-05-08T18:05:08.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.265+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a2b6d60 2024-05-08T18:05:08.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.265+0000 7f7f0281f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.265+0000 7f7f0281f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:08.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.265+0000 7f7f0281f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.265+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2ba650 con 0x559f7a2b6d60 2024-05-08T18:05:08.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.265+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a2b6d60 2024-05-08T18:05:08.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.265+0000 7f7f0281f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.265+0000 7f7f0281f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.265+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2ba9f0 con 0x559f7a2b6d60 2024-05-08T18:05:08.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.266+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a2b6d60 2024-05-08T18:05:08.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.266+0000 7f7f0281f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:08.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.266+0000 7f7f0281f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.266+0000 7f7f0281f800 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559f7a2bb750 0x559f7a2bdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.266+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2be150 con 0x559f7a2bb750 2024-05-08T18:05:08.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.266+0000 7f7f00317640 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559f7a2bb750 0x559f7a2bdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.266+0000 7f7f00317640 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559f7a2bb750 0x559f7a2bdb40 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.267+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.267+0000 7f7f0281f800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:08.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.267+0000 7f7f0281f800 20 searching for the correct realm 2024-05-08T18:05:08.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.267+0000 7f7f0281f800 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559f7a2c0100 0x559f7a2c25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.267+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c2c90 con 0x559f7a2c0100 2024-05-08T18:05:08.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.267+0000 7f7ef7b16640 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559f7a2c0100 0x559f7a2c25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.267+0000 7f7ef7b16640 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559f7a2c0100 0x559f7a2c25c0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.267+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ee8081070 con 0x559f7a2c0100 2024-05-08T18:05:08.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.267+0000 7f7ef7b16640 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7ee8076040 0x7f7ee8083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.267+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c2c90 con 0x7f7ee8076040 2024-05-08T18:05:08.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.268+0000 7f7f00b18640 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7ee8076040 0x7f7ee8083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.268+0000 7f7f00b18640 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7ee8076040 0x7f7ee8083410 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.268+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef800a4a0 con 0x7f7ee8076040 2024-05-08T18:05:08.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.268+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2ba9f0 con 0x559f7a2bb750 2024-05-08T18:05:08.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.268+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.268+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2ba9f0 con 0x559f7a2b6d60 2024-05-08T18:05:08.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.269+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a2b6d60 2024-05-08T18:05:08.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.269+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2ba9f0 con 0x559f7a2b6d60 2024-05-08T18:05:08.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.269+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a2b6d60 2024-05-08T18:05:08.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.269+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2ba9f0 con 0x559f7a2bb750 2024-05-08T18:05:08.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.269+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.269+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2ba9f0 con 0x559f7a2bb750 2024-05-08T18:05:08.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.269+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.269+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ef007d080 con 0x7f7ee8076040 2024-05-08T18:05:08.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.270+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef800a4a0 con 0x7f7ee8076040 2024-05-08T18:05:08.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.270+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ef007d080 con 0x559f7a2c0100 2024-05-08T18:05:08.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.270+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ee8081070 con 0x559f7a2c0100 2024-05-08T18:05:08.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.270+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ef007d080 con 0x559f7a2b6d60 2024-05-08T18:05:08.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.270+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a2b6d60 2024-05-08T18:05:08.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.270+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ef007d080 con 0x559f7a2bb750 2024-05-08T18:05:08.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.270+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.270+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ef007d080 con 0x559f7a2b6d60 2024-05-08T18:05:08.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.270+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a2b6d60 2024-05-08T18:05:08.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.271+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ef007d080 con 0x7f7ee8076040 2024-05-08T18:05:08.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.271+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7ef800a4a0 con 0x7f7ee8076040 2024-05-08T18:05:08.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.271+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ef007d080 con 0x559f7a2c0100 2024-05-08T18:05:08.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.271+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ee8081070 con 0x559f7a2c0100 2024-05-08T18:05:08.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.271+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x559f7a2c0100 2024-05-08T18:05:08.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.271+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ee8081070 con 0x559f7a2c0100 2024-05-08T18:05:08.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.271+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x559f7a2bb750 2024-05-08T18:05:08.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.272+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.272+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x559f7a2c0100 2024-05-08T18:05:08.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.272+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ee8081070 con 0x559f7a2c0100 2024-05-08T18:05:08.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.272+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x559f7a2c0100 2024-05-08T18:05:08.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.272+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7ee8081070 con 0x559f7a2c0100 2024-05-08T18:05:08.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.272+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x559f7a2b6d60 2024-05-08T18:05:08.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.272+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a2b6d60 2024-05-08T18:05:08.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.272+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x559f7a2b6d60 2024-05-08T18:05:08.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.272+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a2b6d60 2024-05-08T18:05:08.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.272+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x559f7a2bb750 2024-05-08T18:05:08.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.273+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.273+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x559f7a2c0100 2024-05-08T18:05:08.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.273+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ee8081070 con 0x559f7a2c0100 2024-05-08T18:05:08.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.273+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x7f7ee8076040 2024-05-08T18:05:08.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.273+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef800a4a0 con 0x7f7ee8076040 2024-05-08T18:05:08.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.273+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x559f7a2b6d60 2024-05-08T18:05:08.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.273+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a2b6d60 2024-05-08T18:05:08.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.273+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x559f7a2bb750 2024-05-08T18:05:08.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.274+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.274+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x559f7a2c0100 2024-05-08T18:05:08.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.274+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ee8081070 con 0x559f7a2c0100 2024-05-08T18:05:08.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.274+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x559f7a2bb750 2024-05-08T18:05:08.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.274+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.274+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x7f7ee8076040 2024-05-08T18:05:08.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.274+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef800a4a0 con 0x7f7ee8076040 2024-05-08T18:05:08.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.274+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x559f7a2c0100 2024-05-08T18:05:08.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.275+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ee8081070 con 0x559f7a2c0100 2024-05-08T18:05:08.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.275+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x559f7a2bb750 2024-05-08T18:05:08.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.275+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.275+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x559f7a2b6d60 2024-05-08T18:05:08.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.275+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a2b6d60 2024-05-08T18:05:08.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.275+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7ee8090040 con 0x7f7ee8076040 2024-05-08T18:05:08.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.275+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ef800a4a0 con 0x7f7ee8076040 2024-05-08T18:05:08.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.275+0000 7f7f0281f800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:08.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.275+0000 7f7f0281f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:08.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.275+0000 7f7f0281f800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:08.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.275+0000 7f7f0281f800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:08.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.275+0000 7f7f0281f800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:08.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.275+0000 7f7f0281f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:08.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.275+0000 7f7f0281f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.275+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2bea10 con 0x559f7a2b6d60 2024-05-08T18:05:08.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a2b6d60 2024-05-08T18:05:08.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f0281f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f0281f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2bea10 con 0x559f7a2b6d60 2024-05-08T18:05:08.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7ef800a4a0 con 0x559f7a2b6d60 2024-05-08T18:05:08.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f0281f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:08.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f0281f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2bea10 con 0x559f7a2c0100 2024-05-08T18:05:08.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7ee8081070 con 0x559f7a2c0100 2024-05-08T18:05:08.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f0281f800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:08.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f0281f800 20 zone default found 2024-05-08T18:05:08.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f0281f800 4 Realm: () 2024-05-08T18:05:08.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f0281f800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:08.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f0281f800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:08.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f0281f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:08.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f0281f800 20 zonegroup default 2024-05-08T18:05:08.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f0281f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2bf340 con 0x559f7a2bb750 2024-05-08T18:05:08.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.276+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.277+0000 7f7f0281f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.277+0000 7f7f0281f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.277+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c4b50 con 0x559f7a2c0100 2024-05-08T18:05:08.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.277+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7ee8081070 con 0x559f7a2c0100 2024-05-08T18:05:08.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.277+0000 7f7f0281f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.277+0000 7f7f0281f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.277+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c4fb0 con 0x559f7a2c0100 2024-05-08T18:05:08.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.277+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7ee8081070 con 0x559f7a2c0100 2024-05-08T18:05:08.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.277+0000 7f7f0281f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.277+0000 7f7f0281f800 20 started sync module instance, tier type = 2024-05-08T18:05:08.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.277+0000 7f7f0281f800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:08.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.277+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c3f90 con 0x7f7ee8076040 2024-05-08T18:05:08.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.278+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef800a4a0 con 0x7f7ee8076040 2024-05-08T18:05:08.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.278+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94143437882544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c3f90 con 0x7f7ee8076040 2024-05-08T18:05:08.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.278+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c5a70 con 0x559f7a2b6d60 2024-05-08T18:05:08.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.279+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94143437882544] v27'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef8040040 con 0x7f7ee8076040 2024-05-08T18:05:08.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.279+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef8040040 con 0x559f7a2b6d60 2024-05-08T18:05:08.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.279+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94143437886848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c3f90 con 0x559f7a2b6d60 2024-05-08T18:05:08.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.279+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c6500 con 0x559f7a2c0100 2024-05-08T18:05:08.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.280+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94143437886848] v27'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef8040040 con 0x559f7a2b6d60 2024-05-08T18:05:08.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.280+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ee8081070 con 0x559f7a2c0100 2024-05-08T18:05:08.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.280+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94143437880032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c3f00 con 0x559f7a2c0100 2024-05-08T18:05:08.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.280+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c73f0 con 0x7f7ee8076040 2024-05-08T18:05:08.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.281+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94143437880032] v27'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ee8081070 con 0x559f7a2c0100 2024-05-08T18:05:08.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.282+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef8040040 con 0x7f7ee8076040 2024-05-08T18:05:08.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.282+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94143437893440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c7810 con 0x7f7ee8076040 2024-05-08T18:05:08.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.282+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c81d0 con 0x559f7a2b6d60 2024-05-08T18:05:08.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.283+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef8040040 con 0x559f7a2b6d60 2024-05-08T18:05:08.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.283+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94143437896992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c8140 con 0x559f7a2b6d60 2024-05-08T18:05:08.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.283+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c9170 con 0x559f7a2bb750 2024-05-08T18:05:08.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.283+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94143437893440] v27'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef8040040 con 0x7f7ee8076040 2024-05-08T18:05:08.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.284+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.284+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94143437900864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c9560 con 0x559f7a2bb750 2024-05-08T18:05:08.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.284+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94143437896992] v27'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef8040040 con 0x559f7a2b6d60 2024-05-08T18:05:08.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.284+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c85c0 con 0x559f7a2bb750 2024-05-08T18:05:08.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.285+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94143437900864] v27'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.285+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.285+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94143437894496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2c9f20 con 0x559f7a2bb750 2024-05-08T18:05:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.285+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2ca7d0 con 0x559f7a2bb750 2024-05-08T18:05:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.286+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94143437894496] v27'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.286+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.286+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94143437904496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2ca340 con 0x559f7a2bb750 2024-05-08T18:05:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.286+0000 7f7f0281f800 20 add_watcher() i=0 2024-05-08T18:05:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.286+0000 7f7f0281f800 20 add_watcher() i=1 2024-05-08T18:05:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.286+0000 7f7f0281f800 20 add_watcher() i=2 2024-05-08T18:05:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.286+0000 7f7f0281f800 20 add_watcher() i=3 2024-05-08T18:05:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.286+0000 7f7f0281f800 20 add_watcher() i=4 2024-05-08T18:05:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.286+0000 7f7f0281f800 20 add_watcher() i=5 2024-05-08T18:05:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.286+0000 7f7f0281f800 20 add_watcher() i=6 2024-05-08T18:05:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.287+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94143437904496] v27'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.287+0000 7f7f0281f800 20 add_watcher() i=7 2024-05-08T18:05:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.287+0000 7f7f0281f800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:08.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.287+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2cdc80 con 0x559f7a2b6d60 2024-05-08T18:05:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.287+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7ef8040040 con 0x559f7a2b6d60 2024-05-08T18:05:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.287+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94143437906016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a2cb100 con 0x559f7a2b6d60 2024-05-08T18:05:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.288+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94143437906016] v27'110 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7ef8040040 con 0x559f7a2b6d60 2024-05-08T18:05:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.288+0000 7f7db67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.288+0000 7f7f0281f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.288+0000 7f7f0281f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.288+0000 7f7f0281f800 5 note: GC not initialized 2024-05-08T18:05:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.288+0000 7f7db57e2640 20 reqs_thread_entry: start 2024-05-08T18:05:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.289+0000 7f7f0281f800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.289+0000 7f7f0281f800 20 Filter name: none 2024-05-08T18:05:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.289+0000 7f7d26ffd640 20 reqs_thread_entry: start 2024-05-08T18:05:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.289+0000 7f7f0281f800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-05-08T18:05:08.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.289+0000 7f7f0281f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.289+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a301190 con 0x559f7a2bb750 2024-05-08T18:05:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.289+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(62 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.289+0000 7f7f0281f800 20 rados_obj.operate() r=0 bl.length=380 2024-05-08T18:05:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.289+0000 7f7f0281f800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-05-08T18:05:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.289+0000 7f7f0281f800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-05-08T18:05:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.289+0000 7f7f0281f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.289+0000 7f7f0281f800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-05-08T18:05:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.289+0000 7f7f0281f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.289+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a303270 con 0x7f7ee8076040 2024-05-08T18:05:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.290+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv20 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f7ef8040040 con 0x7f7ee8076040 2024-05-08T18:05:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.290+0000 7f7f0281f800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.290+0000 7f7f0281f800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-05-08T18:05:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.290+0000 7f7f0281f800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:08.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.290+0000 7f7f0281f800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 : miss 2024-05-08T18:05:08.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.290+0000 7f7f0281f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.290+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:64 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a304d00 con 0x559f7a2bb750 2024-05-08T18:05:08.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.290+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(64 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.290+0000 7f7f0281f800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:08.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.290+0000 7f7f0281f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:08.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.290+0000 7f7f0281f800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:08.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.290+0000 7f7f0281f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:08.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.290+0000 7f7f0281f800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:08.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.290+0000 7f7f0281f800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 2024-05-08T18:05:08.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.290+0000 7f7f0281f800 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x13) 2024-05-08T18:05:08.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.290+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a308f00 con 0x7f7ee8076040 2024-05-08T18:05:08.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.291+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(65 foo.buckets [call] v27'21 uv19 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f7ef8040040 con 0x7f7ee8076040 2024-05-08T18:05:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.291+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:66 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~379 in=379b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a30ee20 con 0x559f7a2bb750 2024-05-08T18:05:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(66 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [delete,create,call,call,writefull 0~379,setxattr (123)] v27'2 uv2 ondisk = 0) ==== 416+0+0 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7f0281f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7f0281f800 10 moving default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7f0281f800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7f0281f800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1, ofs0, ns] 2024-05-08T18:05:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:67 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94143438146672 in=764b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a309330 con 0x559f7a2bb750 2024-05-08T18:05:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== watch-notify(notify (1) cookie 94143437894496 notify 115964116996 ret 0) ==== 794+0+0 (crc 0 0 0) 0x559f7a304270 con 0x559f7a2bb750 2024-05-08T18:05:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(67 notify.6 [notify cookie 94143438146672 out=8b] v0'0 uv114 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7dd77fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116996 cookie 94143437894496 notifier 4494 bl.length()=752 2024-05-08T18:05:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7dd77fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7dd77fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7dd77fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7dd77fe640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f7dc4003ea0 con 0x559f7a2bb750 2024-05-08T18:05:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(68 notify.6 [notify-ack] v0'0 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== watch-notify(notify_complete (2) cookie 94143438146672 notify 115964116996 ret 0) ==== 42+0+48 (crc 0 0 0) 0x559f7a2983b0 con 0x559f7a2bb750 2024-05-08T18:05:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7f0281f800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): orig_sources= new_sources= 2024-05-08T18:05:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7f0281f800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): potential sources added=[] removed=[] 2024-05-08T18:05:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7f0281f800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): orig_dests= new_dests= 2024-05-08T18:05:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7f0281f800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): potential dests added=[] removed=[] 2024-05-08T18:05:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.293+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a30d620 con 0x559f7a2bb750 2024-05-08T18:05:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.294+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(69 fud.buckets [call] v27'1 uv1 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.294+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a304c20 con 0x7f7ee8076040 2024-05-08T18:05:08.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.296+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v27'22 uv22 ondisk = 0) ==== 275+0+0 (crc 0 0 0) 0x7f7ef8040040 con 0x7f7ee8076040 2024-05-08T18:05:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.296+0000 7f7f0281f800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-05-08T18:05:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.296+0000 7f7f0281f800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.296+0000 7f7f0281f800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-05-08T18:05:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.296+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94143438134704 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559f7a304c20 con 0x559f7a2bb750 2024-05-08T18:05:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.296+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== watch-notify(notify (1) cookie 94143437894496 notify 115964116997 ret 0) ==== 356+0+0 (crc 0 0 0) 0x559f7a297940 con 0x559f7a2bb750 2024-05-08T18:05:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.296+0000 7f7dd77fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116997 cookie 94143437894496 notifier 4494 bl.length()=314 2024-05-08T18:05:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.296+0000 7f7dd77fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-05-08T18:05:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.296+0000 7f7dd77fe640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.296+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(71 notify.6 [notify cookie 94143438134704 out=8b] v0'0 uv114 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.296+0000 7f7dd77fe640 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f7dc4003ea0 con 0x559f7a2bb750 2024-05-08T18:05:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.296+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== watch-notify(notify_complete (2) cookie 94143438134704 notify 115964116997 ret 0) ==== 42+0+48 (crc 0 0 0) 0x559f7a27bc30 con 0x559f7a2bb750 2024-05-08T18:05:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.296+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.297+0000 7f7db67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:08.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.297+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94143437906016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a304bf0 con 0x559f7a2b6d60 2024-05-08T18:05:08.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.298+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94143437906016] v27'111 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7ef8040040 con 0x559f7a2b6d60 2024-05-08T18:05:08.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.298+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94143437882544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a304bf0 con 0x7f7ee8076040 2024-05-08T18:05:08.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.299+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94143437882544] v27'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef8040040 con 0x7f7ee8076040 2024-05-08T18:05:08.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.299+0000 7f7f0281f800 20 remove_watcher() i=0 2024-05-08T18:05:08.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.299+0000 7f7f0281f800 2 removed watcher, disabling cache 2024-05-08T18:05:08.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.299+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94143437886848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a304bc0 con 0x559f7a2b6d60 2024-05-08T18:05:08.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.301+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94143437886848] v27'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef8040040 con 0x559f7a2b6d60 2024-05-08T18:05:08.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.301+0000 7f7f0281f800 20 remove_watcher() i=1 2024-05-08T18:05:08.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.301+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94143437880032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a304bc0 con 0x559f7a2c0100 2024-05-08T18:05:08.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.302+0000 7f7ef7b16640 1 -- 172.21.15.50:0/1180284538 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94143437880032] v27'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ee8081070 con 0x559f7a2c0100 2024-05-08T18:05:08.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.302+0000 7f7f0281f800 20 remove_watcher() i=2 2024-05-08T18:05:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.302+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94143437893440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a304bc0 con 0x7f7ee8076040 2024-05-08T18:05:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.302+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94143437893440] v27'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef8040040 con 0x7f7ee8076040 2024-05-08T18:05:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.302+0000 7f7f0281f800 20 remove_watcher() i=3 2024-05-08T18:05:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.302+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94143437896992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a304bc0 con 0x559f7a2b6d60 2024-05-08T18:05:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.303+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94143437896992] v27'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef8040040 con 0x559f7a2b6d60 2024-05-08T18:05:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.303+0000 7f7f0281f800 20 remove_watcher() i=4 2024-05-08T18:05:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.303+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94143437900864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a304bc0 con 0x559f7a2bb750 2024-05-08T18:05:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.304+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94143437900864] v27'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.304+0000 7f7f0281f800 20 remove_watcher() i=5 2024-05-08T18:05:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.304+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94143437894496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a304bc0 con 0x559f7a2bb750 2024-05-08T18:05:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.305+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 31 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94143437894496] v27'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.305+0000 7f7f0281f800 20 remove_watcher() i=6 2024-05-08T18:05:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.305+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94143437904496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559f7a304bc0 con 0x559f7a2bb750 2024-05-08T18:05:08.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.306+0000 7f7f00317640 1 -- 172.21.15.50:0/1180284538 <== osd.0 v2:172.21.15.46:6800/337143975 32 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94143437904496] v27'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ef0049020 con 0x559f7a2bb750 2024-05-08T18:05:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.306+0000 7f7f0281f800 20 remove_watcher() i=7 2024-05-08T18:05:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.306+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559f7a2bb750 msgr2=0x559f7a2bdb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.306+0000 7f7f0281f800 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559f7a2bb750 0x559f7a2bdb40 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.306+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559f7a2b6d60 msgr2=0x559f7a2b9220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.306+0000 7f7f0281f800 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559f7a2b6d60 0x559f7a2b9220 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.306+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7ee8076040 msgr2=0x7f7ee8083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.306+0000 7f7f0281f800 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7ee8076040 0x7f7ee8083410 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.306+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559f7a2c0100 msgr2=0x559f7a2c25c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.306+0000 7f7f0281f800 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559f7a2c0100 0x559f7a2c25c0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7dc803bba0 msgr2=0x7f7dc803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7dc803bba0 0x7f7dc803e060 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8052fd0 tx=0x7f7ee804f000 comp rx=0 tx=0).stop 2024-05-08T18:05:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a2a42a0 msgr2=0x559f7a2ab6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f00b18640 1 -- 172.21.15.50:0/1180284538 reap_dead start 2024-05-08T18:05:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a2a42a0 0x559f7a2ab6b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000bbf0 tx=0x7f7ef003ad00 comp rx=0 tx=0).stop 2024-05-08T18:05:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 shutdown_connections 2024-05-08T18:05:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 --2- 172.21.15.50:0/1180284538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a2a42a0 0x559f7a2ab6b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 >> 172.21.15.50:0/1180284538 conn(0x559f7a2a05c0 msgr2=0x559f7a29dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 shutdown_connections 2024-05-08T18:05:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 -- 172.21.15.50:0/1180284538 wait complete. 2024-05-08T18:05:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 -- 172.21.15.50:0/148958808 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7dd803bba0 msgr2=0x7f7dd803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 --2- 172.21.15.50:0/148958808 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7dd803bba0 0x7f7dd803e060 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x559f7a2807f0 tx=0x7f7ee8005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 -- 172.21.15.50:0/148958808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a296fe0 msgr2=0x559f7a297400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 --2- 172.21.15.50:0/148958808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a296fe0 0x559f7a297400 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7ef000ecf0 tx=0x7f7ef001ab20 comp rx=0 tx=0).stop 2024-05-08T18:05:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 -- 172.21.15.50:0/148958808 shutdown_connections 2024-05-08T18:05:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 --2- 172.21.15.50:0/148958808 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7dd803bba0 0x7f7dd803e060 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 --2- 172.21.15.50:0/148958808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a296fe0 0x559f7a297400 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 -- 172.21.15.50:0/148958808 >> 172.21.15.50:0/148958808 conn(0x559f7a28b080 msgr2=0x559f7a28b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 -- 172.21.15.50:0/148958808 shutdown_connections 2024-05-08T18:05:08.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.307+0000 7f7f0281f800 1 -- 172.21.15.50:0/148958808 wait complete. 2024-05-08T18:05:08.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.308+0000 7f7f0281f800 1 -- 172.21.15.50:0/3030227668 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559f7a282240 msgr2=0x559f7a284630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.308+0000 7f7f0281f800 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559f7a282240 0x559f7a284630 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.308+0000 7f7f0281f800 1 -- 172.21.15.50:0/3030227668 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559f7a180a20 msgr2=0x559f7a27f9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.308+0000 7f7f0281f800 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559f7a180a20 0x559f7a27f9f0 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.308+0000 7f7f0281f800 1 -- 172.21.15.50:0/3030227668 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559f7a286180 msgr2=0x559f7a288570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.308+0000 7f7f0281f800 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559f7a286180 0x559f7a288570 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.308+0000 7f7f0281f800 1 -- 172.21.15.50:0/3030227668 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7de803bba0 msgr2=0x7f7de803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.308+0000 7f7f0281f800 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7de803bba0 0x7f7de803e060 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8054000 tx=0x7f7ee804e000 comp rx=0 tx=0).stop 2024-05-08T18:05:08.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.308+0000 7f7f0281f800 1 -- 172.21.15.50:0/3030227668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a17ca70 msgr2=0x559f7a2787b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.308+0000 7f7f0281f800 1 --2- 172.21.15.50:0/3030227668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559f7a17ca70 0x559f7a2787b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x559f7a26e6f0 tx=0x7f7ef0000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.308+0000 7f7f00b18640 1 -- 172.21.15.50:0/3030227668 reap_dead start 2024-05-08T18:05:08.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.308+0000 7f7f0281f800 1 -- 172.21.15.50:0/3030227668 shutdown_connections 2024-05-08T18:05:08.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.308+0000 7f7f0281f800 1 -- 172.21.15.50:0/3030227668 >> 172.21.15.50:0/3030227668 conn(0x559f7a1772f0 msgr2=0x559f7a17b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.308+0000 7f7f0281f800 1 -- 172.21.15.50:0/3030227668 shutdown_connections 2024-05-08T18:05:08.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.308+0000 7f7f0281f800 1 -- 172.21.15.50:0/3030227668 wait complete. 2024-05-08T18:05:08.317 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-05-08T18:05:08.317 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-05-08T18:05:08.317 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-05-08T18:05:08.362 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:08.363 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.365+0000 7f7fc9331800 1 Processor -- start 2024-05-08T18:05:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.365+0000 7f7fc9331800 1 -- start start 2024-05-08T18:05:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.365+0000 7f7fc9331800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x555d28a06550 0x555d28a06970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.365+0000 7f7fc9331800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x555d28a06ff0 con 0x555d28a077b0 2024-05-08T18:05:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.365+0000 7f7fc9331800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x555d288a7e20 con 0x555d28a06550 2024-05-08T18:05:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.366+0000 7f7fc5028640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x555d28a06550 0x555d28a06970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.366+0000 7f7fc4827640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x555d28a077b0 0x555d28a07c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43178/0 (socket says 172.21.15.50:43178) 2024-05-08T18:05:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.366+0000 7f7fc4827640 1 -- 172.21.15.50:0/2862910506 learned_addr learned my addr 172.21.15.50:0/2862910506 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:08.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.366+0000 7f7fc5028640 1 -- 172.21.15.50:0/2862910506 >> v1:172.21.15.46:6789/0 conn(0x555d28a077b0 legacy=0x555d28a07c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.366+0000 7f7fb7fff640 1 -- 172.21.15.50:0/2862910506 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2713270542 0 0) 0x555d28a06ff0 con 0x555d28a077b0 2024-05-08T18:05:08.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.366+0000 7f7fc5028640 1 -- 172.21.15.50:0/2862910506 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d28a08370 con 0x555d28a06550 2024-05-08T18:05:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.368+0000 7f7fc5028640 1 --2- 172.21.15.50:0/2862910506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 0x555d28a06970 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800a470 tx=0x7f7fb801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e426f219e139b0e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.368+0000 7f7fb7fff640 1 -- 172.21.15.50:0/2862910506 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb8057070 con 0x555d28a06550 2024-05-08T18:05:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.368+0000 7f7fb7fff640 1 -- 172.21.15.50:0/2862910506 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb804adc0 con 0x555d28a06550 2024-05-08T18:05:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.368+0000 7f7fb7fff640 1 -- 172.21.15.50:0/2862910506 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb8052590 con 0x555d28a06550 2024-05-08T18:05:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.368+0000 7f7fc9331800 1 -- 172.21.15.50:0/2862910506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 msgr2=0x555d28a06970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.368+0000 7f7fc9331800 1 --2- 172.21.15.50:0/2862910506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 0x555d28a06970 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7fb800a470 tx=0x7f7fb801ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.368+0000 7f7fc9331800 1 -- 172.21.15.50:0/2862910506 shutdown_connections 2024-05-08T18:05:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.368+0000 7f7fc9331800 1 --2- 172.21.15.50:0/2862910506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 0x555d28a06970 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.369+0000 7f7fc9331800 1 -- 172.21.15.50:0/2862910506 >> 172.21.15.50:0/2862910506 conn(0x555d28a01b50 msgr2=0x555d28a03f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.369+0000 7f7fc9331800 1 -- 172.21.15.50:0/2862910506 shutdown_connections 2024-05-08T18:05:08.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.369+0000 7f7fc9331800 1 -- 172.21.15.50:0/2862910506 wait complete. 2024-05-08T18:05:08.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.384+0000 7f7fc9331800 1 Processor -- start 2024-05-08T18:05:08.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.384+0000 7f7fc9331800 1 -- start start 2024-05-08T18:05:08.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.384+0000 7f7fc9331800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 0x555d28af7970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.384+0000 7f7fc9331800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555d28952a10 con 0x555d28a06550 2024-05-08T18:05:08.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.384+0000 7f7fc5028640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 0x555d28af7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.384+0000 7f7fc5028640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 0x555d28af7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37898/0 (socket says 172.21.15.50:37898) 2024-05-08T18:05:08.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.384+0000 7f7fc5028640 1 -- 172.21.15.50:0/1649361231 learned_addr learned my addr 172.21.15.50:0/1649361231 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.385+0000 7f7fc5028640 1 -- 172.21.15.50:0/1649361231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d28af7f40 con 0x555d28a06550 2024-05-08T18:05:08.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.385+0000 7f7fc5028640 1 --2- 172.21.15.50:0/1649361231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 0x555d28af7970 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8055040 tx=0x7f7fb804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4482c7ea3a1bd483 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.385+0000 7f7eb3606640 1 -- 172.21.15.50:0/1649361231 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb800a2b0 con 0x555d28a06550 2024-05-08T18:05:08.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.385+0000 7f7eb3606640 1 -- 172.21.15.50:0/1649361231 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb8063480 con 0x555d28a06550 2024-05-08T18:05:08.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.385+0000 7f7eb3606640 1 -- 172.21.15.50:0/1649361231 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb8063780 con 0x555d28a06550 2024-05-08T18:05:08.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.386+0000 7f7fc9331800 1 -- 172.21.15.50:0/1649361231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 msgr2=0x555d28af7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.386+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1649361231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 0x555d28af7970 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8055040 tx=0x7f7fb804c000 comp rx=0 tx=0).stop 2024-05-08T18:05:08.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.386+0000 7f7fc9331800 1 -- 172.21.15.50:0/1649361231 shutdown_connections 2024-05-08T18:05:08.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.386+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1649361231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 0x555d28af7970 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.386+0000 7f7fc9331800 1 -- 172.21.15.50:0/1649361231 >> 172.21.15.50:0/1649361231 conn(0x555d28a01b50 msgr2=0x555d28af3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.386+0000 7f7fc9331800 1 -- 172.21.15.50:0/1649361231 shutdown_connections 2024-05-08T18:05:08.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.386+0000 7f7fc9331800 1 -- 172.21.15.50:0/1649361231 wait complete. 2024-05-08T18:05:08.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.386+0000 7f7fc9331800 1 Processor -- start 2024-05-08T18:05:08.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.386+0000 7f7fc9331800 1 -- start start 2024-05-08T18:05:08.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.386+0000 7f7fc9331800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 0x555d28afe330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.386+0000 7f7fc9331800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555d28a0fc80 con 0x555d28a06550 2024-05-08T18:05:08.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.386+0000 7f7fc5028640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 0x555d28afe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.386+0000 7f7fc5028640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 0x555d28afe330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37902/0 (socket says 172.21.15.50:37902) 2024-05-08T18:05:08.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.386+0000 7f7fc5028640 1 -- 172.21.15.50:0/1310279215 learned_addr learned my addr 172.21.15.50:0/1310279215 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.387+0000 7f7fc5028640 1 -- 172.21.15.50:0/1310279215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d28afe870 con 0x555d28a06550 2024-05-08T18:05:08.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.387+0000 7f7fc5028640 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 0x555d28afe330 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7fb801ce60 tx=0x7f7fb8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.387+0000 7f7eb1e03640 1 -- 172.21.15.50:0/1310279215 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb8063ca0 con 0x555d28a06550 2024-05-08T18:05:08.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.387+0000 7f7eb1e03640 1 -- 172.21.15.50:0/1310279215 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb8071070 con 0x555d28a06550 2024-05-08T18:05:08.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.387+0000 7f7fc9331800 1 -- 172.21.15.50:0/1310279215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555d28afeb00 con 0x555d28a06550 2024-05-08T18:05:08.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.387+0000 7f7eb1e03640 1 -- 172.21.15.50:0/1310279215 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb8052440 con 0x555d28a06550 2024-05-08T18:05:08.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.387+0000 7f7fc9331800 1 -- 172.21.15.50:0/1310279215 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d28b01760 con 0x555d28a06550 2024-05-08T18:05:08.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.388+0000 7f7eb1e03640 1 -- 172.21.15.50:0/1310279215 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7fb806b070 con 0x555d28a06550 2024-05-08T18:05:08.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.388+0000 7f7eb1e03640 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7eac03bba0 0x7f7eac03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.388+0000 7f7eb1e03640 1 -- 172.21.15.50:0/1310279215 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7fb8095f50 con 0x555d28a06550 2024-05-08T18:05:08.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.388+0000 7f7fc4827640 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7eac03bba0 0x7f7eac03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.388+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555d28a077b0 0x555d28b05520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.388+0000 7f7fc9331800 1 -- 172.21.15.50:0/1310279215 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b05ab0 con 0x555d28a077b0 2024-05-08T18:05:08.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.388+0000 7f7fc5829640 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555d28a077b0 0x555d28b05520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.389+0000 7f7fc4827640 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7eac03bba0 0x7f7eac03e060 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc000c00 tx=0x7f7fbc0013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.389+0000 7f7fc5829640 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555d28a077b0 0x555d28b05520 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.389+0000 7f7fc5829640 1 -- 172.21.15.50:0/1310279215 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28a077b0 2024-05-08T18:05:08.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.389+0000 7f7fc9331800 1 -- 172.21.15.50:0/1310279215 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b02ad0 con 0x555d28a077b0 2024-05-08T18:05:08.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.389+0000 7f7fc5829640 1 -- 172.21.15.50:0/1310279215 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28a077b0 2024-05-08T18:05:08.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.389+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555d28b07da0 0x555d28b0a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.389+0000 7f7fc9331800 1 -- 172.21.15.50:0/1310279215 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b0a880 con 0x555d28b07da0 2024-05-08T18:05:08.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.390+0000 7f7fc5028640 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555d28b07da0 0x555d28b0a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.390+0000 7f7fc5028640 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555d28b07da0 0x555d28b0a190 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.390+0000 7f7fc5028640 1 -- 172.21.15.50:0/1310279215 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b07da0 2024-05-08T18:05:08.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.390+0000 7f7fc9331800 1 -- 172.21.15.50:0/1310279215 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b06c40 con 0x555d28a077b0 2024-05-08T18:05:08.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.391+0000 7f7fc5829640 1 -- 172.21.15.50:0/1310279215 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28a077b0 2024-05-08T18:05:08.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.391+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555d28b0bce0 0x555d28b0e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.391+0000 7f7fc9331800 1 -- 172.21.15.50:0/1310279215 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b0e7d0 con 0x555d28b0bce0 2024-05-08T18:05:08.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.391+0000 7f7fc4827640 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555d28b0bce0 0x555d28b0e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.391+0000 7f7fc4827640 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555d28b0bce0 0x555d28b0e0d0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.391+0000 7f7fc4827640 1 -- 172.21.15.50:0/1310279215 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7fbc00f670 con 0x555d28b0bce0 2024-05-08T18:05:08.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.392+0000 7f7fc9331800 1 Processor -- start 2024-05-08T18:05:08.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.392+0000 7f7fc9331800 1 -- start start 2024-05-08T18:05:08.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.392+0000 7f7fc9331800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b1cb40 0x555d28b1cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.392+0000 7f7fc9331800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555d288f9330 con 0x555d28b1cb40 2024-05-08T18:05:08.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.392+0000 7f7fc5028640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b1cb40 0x555d28b1cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.392+0000 7f7fc5028640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b1cb40 0x555d28b1cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37904/0 (socket says 172.21.15.50:37904) 2024-05-08T18:05:08.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.392+0000 7f7fc5028640 1 -- 172.21.15.50:0/1893862145 learned_addr learned my addr 172.21.15.50:0/1893862145 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.392+0000 7f7fc5028640 1 -- 172.21.15.50:0/1893862145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d28b1d4a0 con 0x555d28b1cb40 2024-05-08T18:05:08.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.392+0000 7f7fc5028640 1 --2- 172.21.15.50:0/1893862145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b1cb40 0x555d28b1cf60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7fb806c770 tx=0x7f7fb8052dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.393+0000 7f7eaaffd640 1 -- 172.21.15.50:0/1893862145 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb8076730 con 0x555d28b1cb40 2024-05-08T18:05:08.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.393+0000 7f7eaaffd640 1 -- 172.21.15.50:0/1893862145 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb8069070 con 0x555d28b1cb40 2024-05-08T18:05:08.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.393+0000 7f7fc9331800 1 -- 172.21.15.50:0/1893862145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555d28b1df10 con 0x555d28b1cb40 2024-05-08T18:05:08.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.393+0000 7f7eaaffd640 1 -- 172.21.15.50:0/1893862145 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb8009c50 con 0x555d28b1cb40 2024-05-08T18:05:08.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.393+0000 7f7fc9331800 1 -- 172.21.15.50:0/1893862145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d28b1e410 con 0x555d28b1cb40 2024-05-08T18:05:08.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.393+0000 7f7eaaffd640 1 -- 172.21.15.50:0/1893862145 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7fb806c990 con 0x555d28b1cb40 2024-05-08T18:05:08.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.394+0000 7f7eaaffd640 1 --2- 172.21.15.50:0/1893862145 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e9c03bba0 0x7f7e9c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.394+0000 7f7eaaffd640 1 -- 172.21.15.50:0/1893862145 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7fb8085070 con 0x555d28b1cb40 2024-05-08T18:05:08.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.394+0000 7f7fc4827640 1 --2- 172.21.15.50:0/1893862145 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e9c03bba0 0x7f7e9c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.394+0000 7f7fc9331800 1 Processor -- start 2024-05-08T18:05:08.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.394+0000 7f7fc4827640 1 --2- 172.21.15.50:0/1893862145 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e9c03bba0 0x7f7e9c03e060 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x555d28b06350 tx=0x7f7fbc006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.394+0000 7f7fc9331800 1 -- start start 2024-05-08T18:05:08.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.394+0000 7f7fc9331800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b2e110 0x555d28b2e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.394+0000 7f7fc9331800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555d28af81d0 con 0x555d28b2e110 2024-05-08T18:05:08.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.394+0000 7f7fc5028640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b2e110 0x555d28b2e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.394+0000 7f7fc5028640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b2e110 0x555d28b2e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37918/0 (socket says 172.21.15.50:37918) 2024-05-08T18:05:08.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.394+0000 7f7fc5028640 1 -- 172.21.15.50:0/507299889 learned_addr learned my addr 172.21.15.50:0/507299889 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.395+0000 7f7fc5028640 1 -- 172.21.15.50:0/507299889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d28b2eb00 con 0x555d28b2e110 2024-05-08T18:05:08.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.395+0000 7f7fc5028640 1 --2- 172.21.15.50:0/507299889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b2e110 0x555d28b2e530 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7fb808d840 tx=0x7f7fb809b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382d9f6d15c5996e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.395+0000 7f7ea8ff9640 1 -- 172.21.15.50:0/507299889 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb8057070 con 0x555d28b2e110 2024-05-08T18:05:08.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.395+0000 7f7ea8ff9640 1 -- 172.21.15.50:0/507299889 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb8058040 con 0x555d28b2e110 2024-05-08T18:05:08.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.395+0000 7f7ea8ff9640 1 -- 172.21.15.50:0/507299889 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb8009410 con 0x555d28b2e110 2024-05-08T18:05:08.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.395+0000 7f7fc9331800 1 -- 172.21.15.50:0/507299889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b2e110 msgr2=0x555d28b2e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.395+0000 7f7fc9331800 1 --2- 172.21.15.50:0/507299889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b2e110 0x555d28b2e530 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7fb808d840 tx=0x7f7fb809b580 comp rx=0 tx=0).stop 2024-05-08T18:05:08.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.395+0000 7f7fc9331800 1 -- 172.21.15.50:0/507299889 shutdown_connections 2024-05-08T18:05:08.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.395+0000 7f7fc9331800 1 --2- 172.21.15.50:0/507299889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b2e110 0x555d28b2e530 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.395+0000 7f7fc9331800 1 -- 172.21.15.50:0/507299889 >> 172.21.15.50:0/507299889 conn(0x555d28b26010 msgr2=0x555d28b24070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.395+0000 7f7fc9331800 1 -- 172.21.15.50:0/507299889 shutdown_connections 2024-05-08T18:05:08.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.395+0000 7f7fc9331800 1 -- 172.21.15.50:0/507299889 wait complete. 2024-05-08T18:05:08.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.396+0000 7f7fc9331800 1 Processor -- start 2024-05-08T18:05:08.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.396+0000 7f7fc9331800 1 -- start start 2024-05-08T18:05:08.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.396+0000 7f7fc9331800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b2e110 0x555d28b35490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.396+0000 7f7fc9331800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555d28b2f190 con 0x555d28b2e110 2024-05-08T18:05:08.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.396+0000 7f7fc5028640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b2e110 0x555d28b35490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.396+0000 7f7fc5028640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b2e110 0x555d28b35490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37930/0 (socket says 172.21.15.50:37930) 2024-05-08T18:05:08.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.396+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 learned_addr learned my addr 172.21.15.50:0/1596159664 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.396+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d28b359d0 con 0x555d28b2e110 2024-05-08T18:05:08.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.397+0000 7f7fc5028640 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b2e110 0x555d28b35490 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7fb809bba0 tx=0x7f7fb808d7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.397+0000 7f7e9affd640 1 -- 172.21.15.50:0/1596159664 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb80820c0 con 0x555d28b2e110 2024-05-08T18:05:08.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.397+0000 7f7e9affd640 1 -- 172.21.15.50:0/1596159664 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fb8082260 con 0x555d28b2e110 2024-05-08T18:05:08.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.397+0000 7f7e9affd640 1 -- 172.21.15.50:0/1596159664 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7fb80614b0 con 0x555d28b2e110 2024-05-08T18:05:08.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.397+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555d28b359d0 con 0x555d28b2e110 2024-05-08T18:05:08.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.397+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d28b359d0 con 0x555d28b2e110 2024-05-08T18:05:08.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.398+0000 7f7e9affd640 1 -- 172.21.15.50:0/1596159664 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7fb80b5020 con 0x555d28b2e110 2024-05-08T18:05:08.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.398+0000 7f7e9affd640 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e9003bba0 0x7f7e9003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.398+0000 7f7e9affd640 1 -- 172.21.15.50:0/1596159664 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7fb80b8c70 con 0x555d28b2e110 2024-05-08T18:05:08.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.398+0000 7f7fc4827640 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e9003bba0 0x7f7e9003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.398+0000 7f7fc4827640 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e9003bba0 0x7f7e9003e060 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00ffd0 tx=0x7f7fbc023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.398+0000 7f7fc9331800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.399+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555d28b40c30 0x555d28b430f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.399+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b43840 con 0x555d28b40c30 2024-05-08T18:05:08.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.399+0000 7f7fc5829640 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555d28b40c30 0x555d28b430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.399+0000 7f7fc5829640 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555d28b40c30 0x555d28b430f0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.399+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28b40c30 2024-05-08T18:05:08.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.399+0000 7f7fc9331800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.399+0000 7f7fc9331800 20 realm 2024-05-08T18:05:08.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.399+0000 7f7fc9331800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.399+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b43c50 con 0x555d28b40c30 2024-05-08T18:05:08.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.400+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28b40c30 2024-05-08T18:05:08.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.400+0000 7f7fc9331800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.400+0000 7f7fc9331800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:08.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.400+0000 7f7fc9331800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.400+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b44520 con 0x555d28b40c30 2024-05-08T18:05:08.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.400+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28b40c30 2024-05-08T18:05:08.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.400+0000 7f7fc9331800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.400+0000 7f7fc9331800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.400+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b40c30 2024-05-08T18:05:08.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.400+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28b40c30 2024-05-08T18:05:08.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.400+0000 7f7fc9331800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:08.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.400+0000 7f7fc9331800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.400+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555d28b45620 0x555d28b47a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.400+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b48020 con 0x555d28b45620 2024-05-08T18:05:08.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.400+0000 7f7fc5028640 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555d28b45620 0x555d28b47a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.401+0000 7f7fc5028640 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555d28b45620 0x555d28b47a10 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.401+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.401+0000 7f7fc9331800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:08.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.401+0000 7f7fc9331800 20 searching for the correct realm 2024-05-08T18:05:08.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.401+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555d28b49fd0 0x555d28b4c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.401+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b4cb60 con 0x555d28b49fd0 2024-05-08T18:05:08.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.401+0000 7f7fc4827640 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555d28b49fd0 0x555d28b4c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.401+0000 7f7fc4827640 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555d28b49fd0 0x555d28b4c490 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.402+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.402+0000 7f7fc4827640 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7fbc02e910 0x7f7fbc03a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.402+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b4cb60 con 0x7f7fbc02e910 2024-05-08T18:05:08.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.402+0000 7f7fc5829640 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7fbc02e910 0x7f7fbc03a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.402+0000 7f7fc5829640 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7fbc02e910 0x7f7fbc03a3c0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.402+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fc000a4a0 con 0x7f7fbc02e910 2024-05-08T18:05:08.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.402+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b45620 2024-05-08T18:05:08.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.403+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.403+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b40c30 2024-05-08T18:05:08.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.403+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28b40c30 2024-05-08T18:05:08.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.403+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b40c30 2024-05-08T18:05:08.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.403+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28b40c30 2024-05-08T18:05:08.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.403+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b45620 2024-05-08T18:05:08.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.404+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.404+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b45620 2024-05-08T18:05:08.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.404+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.404+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x7f7fbc02e910 2024-05-08T18:05:08.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.404+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fc000a4a0 con 0x7f7fbc02e910 2024-05-08T18:05:08.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.404+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b49fd0 2024-05-08T18:05:08.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.405+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.405+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b40c30 2024-05-08T18:05:08.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.405+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28b40c30 2024-05-08T18:05:08.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.405+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b45620 2024-05-08T18:05:08.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.405+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.405+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b40c30 2024-05-08T18:05:08.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.405+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28b40c30 2024-05-08T18:05:08.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.405+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x7f7fbc02e910 2024-05-08T18:05:08.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.406+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7fc000a4a0 con 0x7f7fbc02e910 2024-05-08T18:05:08.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.406+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b49fd0 2024-05-08T18:05:08.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.406+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.406+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b49fd0 2024-05-08T18:05:08.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.406+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.406+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b45620 2024-05-08T18:05:08.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.406+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.406+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b49fd0 2024-05-08T18:05:08.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.407+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.407+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b49fd0 2024-05-08T18:05:08.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.407+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.407+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b40c30 2024-05-08T18:05:08.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.407+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28b40c30 2024-05-08T18:05:08.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.407+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b40c30 2024-05-08T18:05:08.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.407+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28b40c30 2024-05-08T18:05:08.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.407+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b45620 2024-05-08T18:05:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.408+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.408+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b49fd0 2024-05-08T18:05:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.408+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.408+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x7f7fbc02e910 2024-05-08T18:05:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.408+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fc000a4a0 con 0x7f7fbc02e910 2024-05-08T18:05:08.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.408+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b40c30 2024-05-08T18:05:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.409+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28b40c30 2024-05-08T18:05:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.409+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b45620 2024-05-08T18:05:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.409+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.409+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b49fd0 2024-05-08T18:05:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.409+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.409+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b45620 2024-05-08T18:05:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.409+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.409+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x7f7fbc02e910 2024-05-08T18:05:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.410+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fc000a4a0 con 0x7f7fbc02e910 2024-05-08T18:05:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.410+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b49fd0 2024-05-08T18:05:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.410+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.410+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b45620 2024-05-08T18:05:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.410+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.410+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x555d28b40c30 2024-05-08T18:05:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28b40c30 2024-05-08T18:05:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555d28b448c0 con 0x7f7fbc02e910 2024-05-08T18:05:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7fc000a4a0 con 0x7f7fbc02e910 2024-05-08T18:05:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc9331800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc9331800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc9331800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc9331800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc9331800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc9331800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc9331800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b488e0 con 0x555d28b40c30 2024-05-08T18:05:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28b40c30 2024-05-08T18:05:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc9331800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc9331800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b488e0 con 0x555d28b40c30 2024-05-08T18:05:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7fc000a4a0 con 0x555d28b40c30 2024-05-08T18:05:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc9331800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc9331800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.411+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b488e0 con 0x555d28b49fd0 2024-05-08T18:05:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 20 zone default found 2024-05-08T18:05:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 4 Realm: () 2024-05-08T18:05:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 20 zonegroup default 2024-05-08T18:05:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b492a0 con 0x555d28b45620 2024-05-08T18:05:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b4ea50 con 0x555d28b49fd0 2024-05-08T18:05:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b4ef20 con 0x555d28b49fd0 2024-05-08T18:05:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 20 started sync module instance, tier type = 2024-05-08T18:05:08.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.412+0000 7f7fc9331800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:08.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.413+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b4de60 con 0x7f7fbc02e910 2024-05-08T18:05:08.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.414+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fc000a4a0 con 0x7f7fbc02e910 2024-05-08T18:05:08.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.414+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93858603262816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b4de60 con 0x7f7fbc02e910 2024-05-08T18:05:08.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.414+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b4fa20 con 0x555d28b40c30 2024-05-08T18:05:08.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.415+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93858603262816] v27'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x7f7fbc02e910 2024-05-08T18:05:08.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.415+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x555d28b40c30 2024-05-08T18:05:08.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.415+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93858603267312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b4de60 con 0x555d28b40c30 2024-05-08T18:05:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.415+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b504b0 con 0x555d28b49fd0 2024-05-08T18:05:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.416+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93858603267312] v27'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x555d28b40c30 2024-05-08T18:05:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.416+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.416+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93858603260336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b4ddd0 con 0x555d28b49fd0 2024-05-08T18:05:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.416+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b51370 con 0x7f7fbc02e910 2024-05-08T18:05:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.417+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93858603260336] v27'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.417+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x7f7fbc02e910 2024-05-08T18:05:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.417+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93858603273920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b512e0 con 0x7f7fbc02e910 2024-05-08T18:05:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.417+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b52230 con 0x555d28b40c30 2024-05-08T18:05:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.418+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x555d28b40c30 2024-05-08T18:05:08.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.418+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93858603277696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b52650 con 0x555d28b40c30 2024-05-08T18:05:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.418+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b53100 con 0x555d28b45620 2024-05-08T18:05:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.418+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93858603273920] v27'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x7f7fbc02e910 2024-05-08T18:05:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.419+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.419+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93858603276128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b534f0 con 0x555d28b45620 2024-05-08T18:05:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.419+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b53ef0 con 0x555d28b45620 2024-05-08T18:05:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.419+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93858603277696] v27'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x555d28b40c30 2024-05-08T18:05:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.420+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93858603276128] v27'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.420+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.420+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93858603278752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b53a00 con 0x555d28b45620 2024-05-08T18:05:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.420+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b54790 con 0x555d28b45620 2024-05-08T18:05:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.421+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93858603278752] v27'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.421+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.421+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93858603281376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b54790 con 0x555d28b45620 2024-05-08T18:05:08.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.421+0000 7f7fc9331800 20 add_watcher() i=0 2024-05-08T18:05:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.421+0000 7f7fc9331800 20 add_watcher() i=1 2024-05-08T18:05:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.421+0000 7f7fc9331800 20 add_watcher() i=2 2024-05-08T18:05:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.421+0000 7f7fc9331800 20 add_watcher() i=3 2024-05-08T18:05:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.421+0000 7f7fc9331800 20 add_watcher() i=4 2024-05-08T18:05:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.421+0000 7f7fc9331800 20 add_watcher() i=5 2024-05-08T18:05:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.421+0000 7f7fc9331800 20 add_watcher() i=6 2024-05-08T18:05:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.422+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93858603281376] v27'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.422+0000 7f7fc9331800 20 add_watcher() i=7 2024-05-08T18:05:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.422+0000 7f7fc9331800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.422+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b57af0 con 0x555d28b40c30 2024-05-08T18:05:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.422+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7fc0040040 con 0x555d28b40c30 2024-05-08T18:05:08.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.422+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93858603287632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b54d20 con 0x555d28b40c30 2024-05-08T18:05:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.423+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93858603287632] v27'112 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x555d28b40c30 2024-05-08T18:05:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.423+0000 7f7e7b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.423+0000 7f7fc9331800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.423+0000 7f7fc9331800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.423+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b57280 con 0x555d28b40c30 2024-05-08T18:05:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.424+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'20 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x555d28b40c30 2024-05-08T18:05:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.424+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b60c00 con 0x555d28b45620 2024-05-08T18:05:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.425+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'15 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.425+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b60c00 con 0x555d28b49fd0 2024-05-08T18:05:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.426+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.426+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b616a0 con 0x555d28b40c30 2024-05-08T18:05:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.427+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'113 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x555d28b40c30 2024-05-08T18:05:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.427+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b602e0 con 0x7f7fbc02e910 2024-05-08T18:05:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.428+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'32 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x7f7fbc02e910 2024-05-08T18:05:08.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.428+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b602e0 con 0x555d28b49fd0 2024-05-08T18:05:08.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.429+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'21 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.429+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b623d0 con 0x555d28b40c30 2024-05-08T18:05:08.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.430+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x555d28b40c30 2024-05-08T18:05:08.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.430+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62460 con 0x555d28b45620 2024-05-08T18:05:08.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.431+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'21 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.431+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62530 con 0x7f7fbc02e910 2024-05-08T18:05:08.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.432+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x7f7fbc02e910 2024-05-08T18:05:08.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.432+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b625c0 con 0x555d28b49fd0 2024-05-08T18:05:08.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.433+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'20 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.433+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62be0 con 0x7f7fbc02e910 2024-05-08T18:05:08.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.434+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'33 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x7f7fbc02e910 2024-05-08T18:05:08.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.434+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62be0 con 0x555d28b45620 2024-05-08T18:05:08.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.435+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'16 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.435+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62be0 con 0x7f7fbc02e910 2024-05-08T18:05:08.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.436+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x7f7fbc02e910 2024-05-08T18:05:08.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.436+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62be0 con 0x555d28b45620 2024-05-08T18:05:08.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.437+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'27 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.437+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62be0 con 0x555d28b40c30 2024-05-08T18:05:08.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.438+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'114 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x555d28b40c30 2024-05-08T18:05:08.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.438+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62be0 con 0x555d28b49fd0 2024-05-08T18:05:08.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.439+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'27 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.439+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b631d0 con 0x7f7fbc02e910 2024-05-08T18:05:08.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.440+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'34 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x7f7fbc02e910 2024-05-08T18:05:08.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.440+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62ec0 con 0x555d28b49fd0 2024-05-08T18:05:08.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.441+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'14 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.441+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62ec0 con 0x555d28b40c30 2024-05-08T18:05:08.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.442+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'22 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x555d28b40c30 2024-05-08T18:05:08.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.442+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62ec0 con 0x555d28b40c30 2024-05-08T18:05:08.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.443+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'115 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x555d28b40c30 2024-05-08T18:05:08.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.443+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62ec0 con 0x555d28b49fd0 2024-05-08T18:05:08.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.444+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'28 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.444+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62ec0 con 0x7f7fbc02e910 2024-05-08T18:05:08.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.445+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'24 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x7f7fbc02e910 2024-05-08T18:05:08.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.445+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62ec0 con 0x555d28b49fd0 2024-05-08T18:05:08.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.446+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.446+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62ec0 con 0x555d28b49fd0 2024-05-08T18:05:08.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.447+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'20 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.447+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62ec0 con 0x555d28b49fd0 2024-05-08T18:05:08.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.448+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'10 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.448+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62ec0 con 0x555d28b49fd0 2024-05-08T18:05:08.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.449+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'22 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.449+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62ec0 con 0x555d28b45620 2024-05-08T18:05:08.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.450+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.450+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62ec0 con 0x555d28b49fd0 2024-05-08T18:05:08.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.451+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'14 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.451+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62ec0 con 0x555d28b45620 2024-05-08T18:05:08.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.452+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'22 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.452+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62ec0 con 0x555d28b49fd0 2024-05-08T18:05:08.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.453+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'29 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.453+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62ec0 con 0x555d28b45620 2024-05-08T18:05:08.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.454+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'28 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.454+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b62ec0 con 0x7f7fbc02e910 2024-05-08T18:05:08.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.455+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'20 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x7f7fbc02e910 2024-05-08T18:05:08.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.455+0000 7f7e7a7e4640 20 reqs_thread_entry: start 2024-05-08T18:05:08.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.455+0000 7f7fc9331800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:08.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.455+0000 7f7fc9331800 20 Filter name: none 2024-05-08T18:05:08.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.455+0000 7f7debfff640 20 reqs_thread_entry: start 2024-05-08T18:05:08.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.455+0000 7f7fc9331800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-05-08T18:05:08.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.455+0000 7f7fc9331800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.455+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b8dec0 con 0x555d28b45620 2024-05-08T18:05:08.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.456+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(94 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.456+0000 7f7fc9331800 20 rados_obj.operate() r=0 bl.length=380 2024-05-08T18:05:08.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.456+0000 7f7fc9331800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-05-08T18:05:08.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.456+0000 7f7fc9331800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-05-08T18:05:08.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.456+0000 7f7fc9331800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:08.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.456+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:95 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555d28b90950 con 0x555d28b45620 2024-05-08T18:05:08.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.456+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(95 fud.buckets [call out=183b] v0'0 uv1 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.457 INFO:teuthology.orchestra.run.smithi050.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-05-08T18:05:08.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.457+0000 7f7e7b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:08.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.457+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:96 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93858603287632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b616a0 con 0x555d28b40c30 2024-05-08T18:05:08.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.458+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(96 data_loggenerations_metadata [watch unwatch cookie 93858603287632] v27'116 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x555d28b40c30 2024-05-08T18:05:08.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.458+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:97 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93858603262816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b54450 con 0x7f7fbc02e910 2024-05-08T18:05:08.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.459+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(97 notify.0 [watch unwatch cookie 93858603262816] v27'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x7f7fbc02e910 2024-05-08T18:05:08.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.459+0000 7f7fc9331800 20 remove_watcher() i=0 2024-05-08T18:05:08.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.459+0000 7f7fc9331800 2 removed watcher, disabling cache 2024-05-08T18:05:08.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.459+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93858603267312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b54450 con 0x555d28b40c30 2024-05-08T18:05:08.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.460+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(98 notify.1 [watch unwatch cookie 93858603267312] v27'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x555d28b40c30 2024-05-08T18:05:08.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.460+0000 7f7fc9331800 20 remove_watcher() i=1 2024-05-08T18:05:08.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.460+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:99 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93858603260336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b54450 con 0x555d28b49fd0 2024-05-08T18:05:08.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.461+0000 7f7fc4827640 1 -- 172.21.15.50:0/1596159664 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(99 notify.2 [watch unwatch cookie 93858603260336] v27'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fbc037040 con 0x555d28b49fd0 2024-05-08T18:05:08.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.461+0000 7f7fc9331800 20 remove_watcher() i=2 2024-05-08T18:05:08.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.461+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:100 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93858603273920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b4d9b0 con 0x7f7fbc02e910 2024-05-08T18:05:08.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.462+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(100 notify.3 [watch unwatch cookie 93858603273920] v27'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x7f7fbc02e910 2024-05-08T18:05:08.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.462+0000 7f7fc9331800 20 remove_watcher() i=3 2024-05-08T18:05:08.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.462+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93858603277696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b54450 con 0x555d28b40c30 2024-05-08T18:05:08.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.463+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(101 notify.4 [watch unwatch cookie 93858603277696] v27'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fc0040040 con 0x555d28b40c30 2024-05-08T18:05:08.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.463+0000 7f7fc9331800 20 remove_watcher() i=4 2024-05-08T18:05:08.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.463+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93858603276128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b50ec0 con 0x555d28b45620 2024-05-08T18:05:08.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.464+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(102 notify.5 [watch unwatch cookie 93858603276128] v27'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.464+0000 7f7fc9331800 20 remove_watcher() i=5 2024-05-08T18:05:08.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.464+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:103 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93858603278752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b4d9b0 con 0x555d28b45620 2024-05-08T18:05:08.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.465+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(103 notify.6 [watch unwatch cookie 93858603278752] v27'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.465+0000 7f7fc9331800 20 remove_watcher() i=6 2024-05-08T18:05:08.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.465+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:104 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93858603281376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555d28b54450 con 0x555d28b45620 2024-05-08T18:05:08.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.466+0000 7f7fc5028640 1 -- 172.21.15.50:0/1596159664 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(104 notify.7 [watch unwatch cookie 93858603281376] v27'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7fb80527c0 con 0x555d28b45620 2024-05-08T18:05:08.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.466+0000 7f7fc9331800 20 remove_watcher() i=7 2024-05-08T18:05:08.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.466+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555d28b45620 msgr2=0x555d28b47a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.466+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555d28b45620 0x555d28b47a10 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.466+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555d28b40c30 msgr2=0x555d28b430f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.466+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555d28b40c30 0x555d28b430f0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7fbc02e910 msgr2=0x7f7fbc03a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7fbc02e910 0x7f7fbc03a3c0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555d28b49fd0 msgr2=0x555d28b4c490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555d28b49fd0 0x555d28b4c490 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e9003bba0 msgr2=0x7f7e9003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e9003bba0 0x7f7e9003e060 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00ffd0 tx=0x7f7fbc023040 comp rx=0 tx=0).stop 2024-05-08T18:05:08.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b2e110 msgr2=0x555d28b35490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1596159664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b2e110 0x555d28b35490 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7fb809bba0 tx=0x7f7fb808d7d0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 reap_dead start 2024-05-08T18:05:08.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc5829640 1 -- 172.21.15.50:0/1596159664 reap_dead start 2024-05-08T18:05:08.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 shutdown_connections 2024-05-08T18:05:08.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 >> 172.21.15.50:0/1596159664 conn(0x555d28b26010 msgr2=0x555d28b23670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 shutdown_connections 2024-05-08T18:05:08.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 -- 172.21.15.50:0/1596159664 wait complete. 2024-05-08T18:05:08.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 -- 172.21.15.50:0/1893862145 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e9c03bba0 msgr2=0x7f7e9c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1893862145 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e9c03bba0 0x7f7e9c03e060 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x555d28b06350 tx=0x7f7fbc006000 comp rx=0 tx=0).stop 2024-05-08T18:05:08.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 -- 172.21.15.50:0/1893862145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b1cb40 msgr2=0x555d28b1cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1893862145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b1cb40 0x555d28b1cf60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7fb806c770 tx=0x7f7fb8052dc0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 -- 172.21.15.50:0/1893862145 shutdown_connections 2024-05-08T18:05:08.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1893862145 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7e9c03bba0 0x7f7e9c03e060 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1893862145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28b1cb40 0x555d28b1cf60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 -- 172.21.15.50:0/1893862145 >> 172.21.15.50:0/1893862145 conn(0x555d28b10be0 msgr2=0x555d28b10fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 -- 172.21.15.50:0/1893862145 shutdown_connections 2024-05-08T18:05:08.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.467+0000 7f7fc9331800 1 -- 172.21.15.50:0/1893862145 wait complete. 2024-05-08T18:05:08.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.468+0000 7f7fc9331800 1 -- 172.21.15.50:0/1310279215 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555d28b07da0 msgr2=0x555d28b0a190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.468+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555d28b07da0 0x555d28b0a190 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.468+0000 7f7fc9331800 1 -- 172.21.15.50:0/1310279215 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555d28a077b0 msgr2=0x555d28b05520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.468+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555d28a077b0 0x555d28b05520 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.468+0000 7f7fc9331800 1 -- 172.21.15.50:0/1310279215 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555d28b0bce0 msgr2=0x555d28b0e0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.468+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555d28b0bce0 0x555d28b0e0d0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.468+0000 7f7fc9331800 1 -- 172.21.15.50:0/1310279215 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7eac03bba0 msgr2=0x7f7eac03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.468+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7eac03bba0 0x7f7eac03e060 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc000c00 tx=0x7f7fbc0013b0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.468+0000 7f7fc9331800 1 -- 172.21.15.50:0/1310279215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 msgr2=0x555d28afe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.468+0000 7f7fc9331800 1 --2- 172.21.15.50:0/1310279215 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555d28a06550 0x555d28afe330 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7fb801ce60 tx=0x7f7fb8002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.468+0000 7f7fc5829640 1 -- 172.21.15.50:0/1310279215 reap_dead start 2024-05-08T18:05:08.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.468+0000 7f7fc9331800 1 -- 172.21.15.50:0/1310279215 shutdown_connections 2024-05-08T18:05:08.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.468+0000 7f7fc9331800 1 -- 172.21.15.50:0/1310279215 >> 172.21.15.50:0/1310279215 conn(0x555d28a01b50 msgr2=0x555d28979b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.468+0000 7f7fc9331800 1 -- 172.21.15.50:0/1310279215 shutdown_connections 2024-05-08T18:05:08.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.468+0000 7f7fc9331800 1 -- 172.21.15.50:0/1310279215 wait complete. 2024-05-08T18:05:08.475 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-05-08T18:05:08.475 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2024-05-08T18:05:08.475 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2024-05-08T18:05:08.475 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid fud --bucket myfoo 2024-05-08T18:05:08.522 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:08.523 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:08.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.525+0000 7fde94328800 1 Processor -- start 2024-05-08T18:05:08.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.525+0000 7fde94328800 1 -- start start 2024-05-08T18:05:08.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.526+0000 7fde94328800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x557c2a576710 0x557c2a576b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.526+0000 7fde94328800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x557c2a5771b0 con 0x557c2a5778d0 2024-05-08T18:05:08.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.526+0000 7fde94328800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x557c2a416e20 con 0x557c2a576710 2024-05-08T18:05:08.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.526+0000 7fde8ffe7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x557c2a576710 0x557c2a576b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.526+0000 7fde8f7e6640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x557c2a5778d0 0x557c2a577d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43192/0 (socket says 172.21.15.50:43192) 2024-05-08T18:05:08.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.526+0000 7fde8ffe7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x557c2a576710 0x557c2a576b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37936/0 (socket says 172.21.15.50:37936) 2024-05-08T18:05:08.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.526+0000 7fde8f7e6640 1 -- 172.21.15.50:0/1439910374 learned_addr learned my addr 172.21.15.50:0/1439910374 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:08.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.526+0000 7fde8ffe7640 1 -- 172.21.15.50:0/1439910374 >> v1:172.21.15.46:6789/0 conn(0x557c2a5778d0 legacy=0x557c2a577d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.526+0000 7fde8ffe7640 1 -- 172.21.15.50:0/1439910374 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c2a578490 con 0x557c2a576710 2024-05-08T18:05:08.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.528+0000 7fde8ffe7640 1 --2- 172.21.15.50:0/1439910374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 0x557c2a576b30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fde8000a470 tx=0x7fde8001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a82643802fa4f17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.528+0000 7fde8efe5640 1 -- 172.21.15.50:0/1439910374 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde80057070 con 0x557c2a576710 2024-05-08T18:05:08.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.528+0000 7fde8efe5640 1 -- 172.21.15.50:0/1439910374 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde8004adc0 con 0x557c2a576710 2024-05-08T18:05:08.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.528+0000 7fde8efe5640 1 -- 172.21.15.50:0/1439910374 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde80052590 con 0x557c2a576710 2024-05-08T18:05:08.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.529+0000 7fde94328800 1 -- 172.21.15.50:0/1439910374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 msgr2=0x557c2a576b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.529+0000 7fde94328800 1 --2- 172.21.15.50:0/1439910374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 0x557c2a576b30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fde8000a470 tx=0x7fde8001ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:08.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.529+0000 7fde94328800 1 -- 172.21.15.50:0/1439910374 shutdown_connections 2024-05-08T18:05:08.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.529+0000 7fde94328800 1 --2- 172.21.15.50:0/1439910374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 0x557c2a576b30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.529+0000 7fde94328800 1 -- 172.21.15.50:0/1439910374 >> 172.21.15.50:0/1439910374 conn(0x557c2a571c90 msgr2=0x557c2a5740d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.529+0000 7fde94328800 1 -- 172.21.15.50:0/1439910374 shutdown_connections 2024-05-08T18:05:08.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.529+0000 7fde94328800 1 -- 172.21.15.50:0/1439910374 wait complete. 2024-05-08T18:05:08.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.544+0000 7fde94328800 1 Processor -- start 2024-05-08T18:05:08.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.544+0000 7fde94328800 1 -- start start 2024-05-08T18:05:08.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.544+0000 7fde94328800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 0x557c2a4ebf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.544+0000 7fde94328800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557c2a4c1a30 con 0x557c2a576710 2024-05-08T18:05:08.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.545+0000 7fde8ffe7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 0x557c2a4ebf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.545+0000 7fde8ffe7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 0x557c2a4ebf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37950/0 (socket says 172.21.15.50:37950) 2024-05-08T18:05:08.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.545+0000 7fde8ffe7640 1 -- 172.21.15.50:0/1217687986 learned_addr learned my addr 172.21.15.50:0/1217687986 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.545+0000 7fde8ffe7640 1 -- 172.21.15.50:0/1217687986 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c2a4ea4c0 con 0x557c2a576710 2024-05-08T18:05:08.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.545+0000 7fde8ffe7640 1 --2- 172.21.15.50:0/1217687986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 0x557c2a4ebf00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fde80055040 tx=0x7fde8004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405d777c6f2a4676 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.545+0000 7fdd7de0b640 1 -- 172.21.15.50:0/1217687986 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde80057040 con 0x557c2a576710 2024-05-08T18:05:08.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.545+0000 7fdd7de0b640 1 -- 172.21.15.50:0/1217687986 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde80062070 con 0x557c2a576710 2024-05-08T18:05:08.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.545+0000 7fdd7de0b640 1 -- 172.21.15.50:0/1217687986 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde8005d470 con 0x557c2a576710 2024-05-08T18:05:08.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.546+0000 7fde94328800 1 -- 172.21.15.50:0/1217687986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 msgr2=0x557c2a4ebf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.546+0000 7fde94328800 1 --2- 172.21.15.50:0/1217687986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 0x557c2a4ebf00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fde80055040 tx=0x7fde8004c000 comp rx=0 tx=0).stop 2024-05-08T18:05:08.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.546+0000 7fde94328800 1 -- 172.21.15.50:0/1217687986 shutdown_connections 2024-05-08T18:05:08.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.546+0000 7fde94328800 1 --2- 172.21.15.50:0/1217687986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 0x557c2a4ebf00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.546+0000 7fde94328800 1 -- 172.21.15.50:0/1217687986 >> 172.21.15.50:0/1217687986 conn(0x557c2a571c90 msgr2=0x557c2a663590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.546+0000 7fde94328800 1 -- 172.21.15.50:0/1217687986 shutdown_connections 2024-05-08T18:05:08.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.546+0000 7fde94328800 1 -- 172.21.15.50:0/1217687986 wait complete. 2024-05-08T18:05:08.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.546+0000 7fde94328800 1 Processor -- start 2024-05-08T18:05:08.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.547+0000 7fde94328800 1 -- start start 2024-05-08T18:05:08.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.547+0000 7fde94328800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 0x557c2a6718d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.547+0000 7fde94328800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557c2a57fda0 con 0x557c2a576710 2024-05-08T18:05:08.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.547+0000 7fde8ffe7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 0x557c2a6718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.547+0000 7fde8ffe7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 0x557c2a6718d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37956/0 (socket says 172.21.15.50:37956) 2024-05-08T18:05:08.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.547+0000 7fde8ffe7640 1 -- 172.21.15.50:0/941667054 learned_addr learned my addr 172.21.15.50:0/941667054 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.547+0000 7fde8ffe7640 1 -- 172.21.15.50:0/941667054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c2a671e10 con 0x557c2a576710 2024-05-08T18:05:08.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.547+0000 7fde8ffe7640 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 0x557c2a6718d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fde800057d0 tx=0x7fde80002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.547+0000 7fdd77fff640 1 -- 172.21.15.50:0/941667054 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde8006b070 con 0x557c2a576710 2024-05-08T18:05:08.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.547+0000 7fdd77fff640 1 -- 172.21.15.50:0/941667054 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde8005d9a0 con 0x557c2a576710 2024-05-08T18:05:08.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.547+0000 7fdd77fff640 1 -- 172.21.15.50:0/941667054 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde80057070 con 0x557c2a576710 2024-05-08T18:05:08.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.547+0000 7fde94328800 1 -- 172.21.15.50:0/941667054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557c2a6720a0 con 0x557c2a576710 2024-05-08T18:05:08.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.548+0000 7fde94328800 1 -- 172.21.15.50:0/941667054 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c2a674d50 con 0x557c2a576710 2024-05-08T18:05:08.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.548+0000 7fdd77fff640 1 -- 172.21.15.50:0/941667054 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fde8005dc80 con 0x557c2a576710 2024-05-08T18:05:08.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.549+0000 7fdd77fff640 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdd7803bb50 0x7fdd7803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.549+0000 7fdd77fff640 1 -- 172.21.15.50:0/941667054 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fde800946d0 con 0x557c2a576710 2024-05-08T18:05:08.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.549+0000 7fde8f7e6640 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdd7803bb50 0x7fdd7803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.549+0000 7fde94328800 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557c2a5778d0 0x557c2a678b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.549+0000 7fde94328800 1 -- 172.21.15.50:0/941667054 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a679090 con 0x557c2a5778d0 2024-05-08T18:05:08.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.549+0000 7fde907e8640 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557c2a5778d0 0x557c2a678b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.549+0000 7fde8f7e6640 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdd7803bb50 0x7fdd7803e010 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde78001d70 tx=0x7fde780012d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.549+0000 7fde907e8640 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557c2a5778d0 0x557c2a678b10 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.549+0000 7fde907e8640 1 -- 172.21.15.50:0/941667054 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a5778d0 2024-05-08T18:05:08.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.550+0000 7fde94328800 1 -- 172.21.15.50:0/941667054 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a679960 con 0x557c2a5778d0 2024-05-08T18:05:08.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.550+0000 7fde907e8640 1 -- 172.21.15.50:0/941667054 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a5778d0 2024-05-08T18:05:08.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.550+0000 7fde94328800 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557c2a67b270 0x557c2a67d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.550+0000 7fde94328800 1 -- 172.21.15.50:0/941667054 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a67de90 con 0x557c2a67b270 2024-05-08T18:05:08.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.550+0000 7fde8ffe7640 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557c2a67b270 0x557c2a67d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.552 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.550+0000 7fde8ffe7640 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557c2a67b270 0x557c2a67d700 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.551+0000 7fde8ffe7640 1 -- 172.21.15.50:0/941667054 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fde8005a040 con 0x557c2a67b270 2024-05-08T18:05:08.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.551+0000 7fde94328800 1 -- 172.21.15.50:0/941667054 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a67a1b0 con 0x557c2a5778d0 2024-05-08T18:05:08.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.551+0000 7fde907e8640 1 -- 172.21.15.50:0/941667054 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a5778d0 2024-05-08T18:05:08.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.551+0000 7fde94328800 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557c2a67f2f0 0x557c2a6816e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.551+0000 7fde94328800 1 -- 172.21.15.50:0/941667054 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a681da0 con 0x557c2a67f2f0 2024-05-08T18:05:08.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.551+0000 7fde8f7e6640 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557c2a67f2f0 0x557c2a6816e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.551+0000 7fde8f7e6640 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557c2a67f2f0 0x557c2a6816e0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.552+0000 7fde8f7e6640 1 -- 172.21.15.50:0/941667054 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fde78009050 con 0x557c2a67f2f0 2024-05-08T18:05:08.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.552+0000 7fde94328800 1 Processor -- start 2024-05-08T18:05:08.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.552+0000 7fde94328800 1 -- start start 2024-05-08T18:05:08.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.552+0000 7fde94328800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a690110 0x557c2a690530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.552+0000 7fde94328800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557c2a468330 con 0x557c2a690110 2024-05-08T18:05:08.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.552+0000 7fde8ffe7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a690110 0x557c2a690530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.552+0000 7fde8ffe7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a690110 0x557c2a690530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37970/0 (socket says 172.21.15.50:37970) 2024-05-08T18:05:08.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.552+0000 7fde8ffe7640 1 -- 172.21.15.50:0/915814190 learned_addr learned my addr 172.21.15.50:0/915814190 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.553+0000 7fde8ffe7640 1 -- 172.21.15.50:0/915814190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c2a690a70 con 0x557c2a690110 2024-05-08T18:05:08.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.553+0000 7fde8ffe7640 1 --2- 172.21.15.50:0/915814190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a690110 0x557c2a690530 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fde80092a60 tx=0x7fde80078ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.553+0000 7fdd757fa640 1 -- 172.21.15.50:0/915814190 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde80087070 con 0x557c2a690110 2024-05-08T18:05:08.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.553+0000 7fdd757fa640 1 -- 172.21.15.50:0/915814190 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde80094260 con 0x557c2a690110 2024-05-08T18:05:08.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.553+0000 7fde94328800 1 -- 172.21.15.50:0/915814190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557c2a6914e0 con 0x557c2a690110 2024-05-08T18:05:08.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.553+0000 7fdd757fa640 1 -- 172.21.15.50:0/915814190 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde80079980 con 0x557c2a690110 2024-05-08T18:05:08.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.553+0000 7fde94328800 1 -- 172.21.15.50:0/915814190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c2a6919e0 con 0x557c2a690110 2024-05-08T18:05:08.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.554+0000 7fdd757fa640 1 -- 172.21.15.50:0/915814190 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fde8008a070 con 0x557c2a690110 2024-05-08T18:05:08.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.554+0000 7fdd757fa640 1 --2- 172.21.15.50:0/915814190 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdd6803bba0 0x7fdd6803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.554+0000 7fdd757fa640 1 -- 172.21.15.50:0/915814190 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fde8008d890 con 0x557c2a690110 2024-05-08T18:05:08.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.554+0000 7fde8f7e6640 1 --2- 172.21.15.50:0/915814190 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdd6803bba0 0x7fdd6803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.554+0000 7fde94328800 1 Processor -- start 2024-05-08T18:05:08.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.554+0000 7fde94328800 1 -- start start 2024-05-08T18:05:08.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.555+0000 7fde8f7e6640 1 --2- 172.21.15.50:0/915814190 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdd6803bba0 0x7fdd6803e060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fde780016b0 tx=0x7fde7800f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.555+0000 7fde94328800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a69d1f0 0x557c2a69d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.555+0000 7fde94328800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557c2a4ea780 con 0x557c2a69d1f0 2024-05-08T18:05:08.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.555+0000 7fde8ffe7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a69d1f0 0x557c2a69d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.555+0000 7fde8ffe7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a69d1f0 0x557c2a69d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37980/0 (socket says 172.21.15.50:37980) 2024-05-08T18:05:08.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.555+0000 7fde8ffe7640 1 -- 172.21.15.50:0/1132279807 learned_addr learned my addr 172.21.15.50:0/1132279807 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.555+0000 7fde8ffe7640 1 -- 172.21.15.50:0/1132279807 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c2a69dbe0 con 0x557c2a69d1f0 2024-05-08T18:05:08.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.555+0000 7fde8ffe7640 1 --2- 172.21.15.50:0/1132279807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a69d1f0 0x557c2a69d610 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fde80078ff0 tx=0x7fde800097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a212c61474cc3137 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.555+0000 7fdd6effd640 1 -- 172.21.15.50:0/1132279807 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde800a2070 con 0x557c2a69d1f0 2024-05-08T18:05:08.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.555+0000 7fdd6effd640 1 -- 172.21.15.50:0/1132279807 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde80009cc0 con 0x557c2a69d1f0 2024-05-08T18:05:08.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.555+0000 7fdd6effd640 1 -- 172.21.15.50:0/1132279807 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde80009920 con 0x557c2a69d1f0 2024-05-08T18:05:08.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.556+0000 7fde94328800 1 -- 172.21.15.50:0/1132279807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a69d1f0 msgr2=0x557c2a69d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.556+0000 7fde94328800 1 --2- 172.21.15.50:0/1132279807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a69d1f0 0x557c2a69d610 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fde80078ff0 tx=0x7fde800097e0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.556+0000 7fde94328800 1 -- 172.21.15.50:0/1132279807 shutdown_connections 2024-05-08T18:05:08.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.556+0000 7fde94328800 1 --2- 172.21.15.50:0/1132279807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a69d1f0 0x557c2a69d610 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.556+0000 7fde94328800 1 -- 172.21.15.50:0/1132279807 >> 172.21.15.50:0/1132279807 conn(0x557c2a699500 msgr2=0x557c2a697560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.556+0000 7fde94328800 1 -- 172.21.15.50:0/1132279807 shutdown_connections 2024-05-08T18:05:08.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.556+0000 7fde94328800 1 -- 172.21.15.50:0/1132279807 wait complete. 2024-05-08T18:05:08.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.556+0000 7fde94328800 1 Processor -- start 2024-05-08T18:05:08.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.556+0000 7fde94328800 1 -- start start 2024-05-08T18:05:08.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.556+0000 7fde94328800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a69d1f0 0x557c2a6a4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.556+0000 7fde94328800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557c2a69e270 con 0x557c2a69d1f0 2024-05-08T18:05:08.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.556+0000 7fde8ffe7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a69d1f0 0x557c2a6a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.557+0000 7fde8ffe7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a69d1f0 0x557c2a6a4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37986/0 (socket says 172.21.15.50:37986) 2024-05-08T18:05:08.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.557+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 learned_addr learned my addr 172.21.15.50:0/3875543408 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.557+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c2a6a4a90 con 0x557c2a69d1f0 2024-05-08T18:05:08.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.557+0000 7fde8ffe7640 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a69d1f0 0x557c2a6a4550 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fde800098b0 tx=0x7fde8005b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.557+0000 7fdd6d7fa640 1 -- 172.21.15.50:0/3875543408 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde80009b00 con 0x557c2a69d1f0 2024-05-08T18:05:08.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.557+0000 7fdd6d7fa640 1 -- 172.21.15.50:0/3875543408 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde80052440 con 0x557c2a69d1f0 2024-05-08T18:05:08.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.557+0000 7fdd6d7fa640 1 -- 172.21.15.50:0/3875543408 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde80083360 con 0x557c2a69d1f0 2024-05-08T18:05:08.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.557+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557c2a6a4a90 con 0x557c2a69d1f0 2024-05-08T18:05:08.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.557+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c2a6a4f30 con 0x557c2a69d1f0 2024-05-08T18:05:08.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.558+0000 7fdd6d7fa640 1 -- 172.21.15.50:0/3875543408 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fde800b7030 con 0x557c2a69d1f0 2024-05-08T18:05:08.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.558+0000 7fdd6d7fa640 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdd5803bba0 0x7fdd5803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.558+0000 7fdd6d7fa640 1 -- 172.21.15.50:0/3875543408 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fde800b5070 con 0x557c2a69d1f0 2024-05-08T18:05:08.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.558+0000 7fde8f7e6640 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdd5803bba0 0x7fdd5803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.559+0000 7fde8f7e6640 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdd5803bba0 0x7fdd5803e060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fde7800efd0 tx=0x7fde78010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.559+0000 7fde94328800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.559+0000 7fde94328800 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557c2a6afcf0 0x557c2a6b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.559+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b2900 con 0x557c2a6afcf0 2024-05-08T18:05:08.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.559+0000 7fde907e8640 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557c2a6afcf0 0x557c2a6b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.559+0000 7fde907e8640 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557c2a6afcf0 0x557c2a6b21b0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a6afcf0 2024-05-08T18:05:08.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde94328800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde94328800 20 realm 2024-05-08T18:05:08.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde94328800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b2d10 con 0x557c2a6afcf0 2024-05-08T18:05:08.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a6afcf0 2024-05-08T18:05:08.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde94328800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde94328800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:08.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde94328800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b35e0 con 0x557c2a6afcf0 2024-05-08T18:05:08.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a6afcf0 2024-05-08T18:05:08.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde94328800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde94328800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6afcf0 2024-05-08T18:05:08.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a6afcf0 2024-05-08T18:05:08.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde94328800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde94328800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde94328800 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557c2a6b46e0 0x557c2a6b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.560+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b70e0 con 0x557c2a6b46e0 2024-05-08T18:05:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.561+0000 7fde8ffe7640 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557c2a6b46e0 0x557c2a6b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.561+0000 7fde8ffe7640 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557c2a6b46e0 0x557c2a6b6ad0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.561+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.561+0000 7fde94328800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.561+0000 7fde94328800 20 searching for the correct realm 2024-05-08T18:05:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.561+0000 7fde94328800 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557c2a6b9090 0x557c2a6bb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.561+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6bbc20 con 0x557c2a6b9090 2024-05-08T18:05:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.561+0000 7fde8f7e6640 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557c2a6b9090 0x557c2a6bb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.562+0000 7fde8f7e6640 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557c2a6b9090 0x557c2a6bb550 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.562+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde78009050 con 0x557c2a6b9090 2024-05-08T18:05:08.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.562+0000 7fde8f7e6640 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fde78037040 0x7fde78036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.562+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6bbc20 con 0x7fde78037040 2024-05-08T18:05:08.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.562+0000 7fde907e8640 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fde78037040 0x7fde78036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.562+0000 7fde907e8640 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fde78037040 0x7fde78036410 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.563+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde8800a4a0 con 0x7fde78037040 2024-05-08T18:05:08.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.563+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b46e0 2024-05-08T18:05:08.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.563+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.563+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6afcf0 2024-05-08T18:05:08.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.563+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a6afcf0 2024-05-08T18:05:08.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.563+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6afcf0 2024-05-08T18:05:08.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.563+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a6afcf0 2024-05-08T18:05:08.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.563+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b46e0 2024-05-08T18:05:08.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.564+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.564+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b46e0 2024-05-08T18:05:08.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.564+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.564+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x7fde78037040 2024-05-08T18:05:08.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.564+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde8800a4a0 con 0x7fde78037040 2024-05-08T18:05:08.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.564+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b9090 2024-05-08T18:05:08.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.564+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde78009050 con 0x557c2a6b9090 2024-05-08T18:05:08.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.564+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6afcf0 2024-05-08T18:05:08.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.565+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a6afcf0 2024-05-08T18:05:08.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.565+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b46e0 2024-05-08T18:05:08.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.565+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.565+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6afcf0 2024-05-08T18:05:08.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.565+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a6afcf0 2024-05-08T18:05:08.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.565+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x7fde78037040 2024-05-08T18:05:08.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.565+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fde8800a4a0 con 0x7fde78037040 2024-05-08T18:05:08.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.565+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b9090 2024-05-08T18:05:08.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.566+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde78009050 con 0x557c2a6b9090 2024-05-08T18:05:08.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.566+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b9090 2024-05-08T18:05:08.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.566+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde78009050 con 0x557c2a6b9090 2024-05-08T18:05:08.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.566+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b46e0 2024-05-08T18:05:08.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.566+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.566+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b9090 2024-05-08T18:05:08.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.566+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde78009050 con 0x557c2a6b9090 2024-05-08T18:05:08.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.566+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b9090 2024-05-08T18:05:08.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.567+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fde78009050 con 0x557c2a6b9090 2024-05-08T18:05:08.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.567+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6afcf0 2024-05-08T18:05:08.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.567+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a6afcf0 2024-05-08T18:05:08.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.567+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6afcf0 2024-05-08T18:05:08.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.567+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a6afcf0 2024-05-08T18:05:08.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.567+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b46e0 2024-05-08T18:05:08.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.567+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.567+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b9090 2024-05-08T18:05:08.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.568+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde78009050 con 0x557c2a6b9090 2024-05-08T18:05:08.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.568+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x7fde78037040 2024-05-08T18:05:08.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.568+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde8800a4a0 con 0x7fde78037040 2024-05-08T18:05:08.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.568+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6afcf0 2024-05-08T18:05:08.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.568+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a6afcf0 2024-05-08T18:05:08.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.568+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b46e0 2024-05-08T18:05:08.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.568+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.568+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b9090 2024-05-08T18:05:08.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.569+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde78009050 con 0x557c2a6b9090 2024-05-08T18:05:08.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.569+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b46e0 2024-05-08T18:05:08.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.569+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.569+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x7fde78037040 2024-05-08T18:05:08.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.569+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde8800a4a0 con 0x7fde78037040 2024-05-08T18:05:08.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.569+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b9090 2024-05-08T18:05:08.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.569+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde78009050 con 0x557c2a6b9090 2024-05-08T18:05:08.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.569+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6b46e0 2024-05-08T18:05:08.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.570+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.570+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x557c2a6afcf0 2024-05-08T18:05:08.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.570+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a6afcf0 2024-05-08T18:05:08.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.570+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b3980 con 0x7fde78037040 2024-05-08T18:05:08.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.570+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fde8800a4a0 con 0x7fde78037040 2024-05-08T18:05:08.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.570+0000 7fde94328800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:08.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.570+0000 7fde94328800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:08.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.570+0000 7fde94328800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.570+0000 7fde94328800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.570+0000 7fde94328800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.570+0000 7fde94328800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.570+0000 7fde94328800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.570+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b79a0 con 0x557c2a6afcf0 2024-05-08T18:05:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a6afcf0 2024-05-08T18:05:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde94328800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde94328800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b79a0 con 0x557c2a6afcf0 2024-05-08T18:05:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fde8800a4a0 con 0x557c2a6afcf0 2024-05-08T18:05:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde94328800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde94328800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b79a0 con 0x557c2a6b9090 2024-05-08T18:05:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fde78009050 con 0x557c2a6b9090 2024-05-08T18:05:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde94328800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde94328800 20 zone default found 2024-05-08T18:05:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde94328800 4 Realm: () 2024-05-08T18:05:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde94328800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde94328800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde94328800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde94328800 20 zonegroup default 2024-05-08T18:05:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde94328800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6b82e0 con 0x557c2a6b46e0 2024-05-08T18:05:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.571+0000 7fde94328800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.572+0000 7fde94328800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.572+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6bdab0 con 0x557c2a6b9090 2024-05-08T18:05:08.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.572+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fde78009050 con 0x557c2a6b9090 2024-05-08T18:05:08.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.572+0000 7fde94328800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.572+0000 7fde94328800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.572+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6bdef0 con 0x557c2a6b9090 2024-05-08T18:05:08.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.572+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fde78009050 con 0x557c2a6b9090 2024-05-08T18:05:08.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.572+0000 7fde94328800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.572+0000 7fde94328800 20 started sync module instance, tier type = 2024-05-08T18:05:08.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.572+0000 7fde94328800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:08.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.572+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6bcf20 con 0x7fde78037040 2024-05-08T18:05:08.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.573+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde8800a4a0 con 0x7fde78037040 2024-05-08T18:05:08.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.573+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93991776015424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6bcf20 con 0x7fde78037040 2024-05-08T18:05:08.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.573+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6bea10 con 0x557c2a6afcf0 2024-05-08T18:05:08.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.574+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93991776015424] v27'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde88040040 con 0x7fde78037040 2024-05-08T18:05:08.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.574+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde88040040 con 0x557c2a6afcf0 2024-05-08T18:05:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.574+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93991776019744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6bcf20 con 0x557c2a6afcf0 2024-05-08T18:05:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.574+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6bf4d0 con 0x557c2a6b9090 2024-05-08T18:05:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.575+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93991776019744] v27'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde88040040 con 0x557c2a6afcf0 2024-05-08T18:05:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.575+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde78009050 con 0x557c2a6b9090 2024-05-08T18:05:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.576+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93991776012912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6bce90 con 0x557c2a6b9090 2024-05-08T18:05:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.576+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6c0390 con 0x7fde78037040 2024-05-08T18:05:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.576+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93991776012912] v27'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde78009050 con 0x557c2a6b9090 2024-05-08T18:05:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.577+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde88040040 con 0x7fde78037040 2024-05-08T18:05:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.577+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93991776026336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6c0300 con 0x7fde78037040 2024-05-08T18:05:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.577+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6c1120 con 0x557c2a6afcf0 2024-05-08T18:05:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.578+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde88040040 con 0x557c2a6afcf0 2024-05-08T18:05:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.578+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93991776029744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6c1120 con 0x557c2a6afcf0 2024-05-08T18:05:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.578+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6c2080 con 0x557c2a6b46e0 2024-05-08T18:05:08.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.578+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93991776026336] v27'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde88040040 con 0x7fde78037040 2024-05-08T18:05:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.579+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93991776029744] v27'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde88040040 con 0x557c2a6afcf0 2024-05-08T18:05:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.579+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.579+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93991776032016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6c1b60 con 0x557c2a6b46e0 2024-05-08T18:05:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.579+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6c2920 con 0x557c2a6b46e0 2024-05-08T18:05:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.580+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93991776032016] v27'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.580+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.580+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93991776028544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6c31c0 con 0x557c2a6b46e0 2024-05-08T18:05:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.580+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6c3a70 con 0x557c2a6b46e0 2024-05-08T18:05:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.581+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93991776028544] v27'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.581+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.581+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93991776038160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6c3c50 con 0x557c2a6b46e0 2024-05-08T18:05:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.581+0000 7fde94328800 20 add_watcher() i=0 2024-05-08T18:05:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.581+0000 7fde94328800 20 add_watcher() i=1 2024-05-08T18:05:08.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.581+0000 7fde94328800 20 add_watcher() i=2 2024-05-08T18:05:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.581+0000 7fde94328800 20 add_watcher() i=3 2024-05-08T18:05:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.581+0000 7fde94328800 20 add_watcher() i=4 2024-05-08T18:05:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.581+0000 7fde94328800 20 add_watcher() i=5 2024-05-08T18:05:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.581+0000 7fde94328800 20 add_watcher() i=6 2024-05-08T18:05:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.582+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93991776038160] v27'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.582+0000 7fde94328800 20 add_watcher() i=7 2024-05-08T18:05:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.582+0000 7fde94328800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.582+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6c7020 con 0x557c2a6afcf0 2024-05-08T18:05:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.582+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fde88040040 con 0x557c2a6afcf0 2024-05-08T18:05:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.582+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93991776041792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6c4410 con 0x557c2a6afcf0 2024-05-08T18:05:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.583+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93991776041792] v27'117 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fde88040040 con 0x557c2a6afcf0 2024-05-08T18:05:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.583+0000 7fdd45fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.583+0000 7fde94328800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.583+0000 7fde94328800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:08.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.583+0000 7fde94328800 5 note: GC not initialized 2024-05-08T18:05:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.584+0000 7fdd44fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.584+0000 7fde94328800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.584+0000 7fde94328800 20 Filter name: none 2024-05-08T18:05:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.584+0000 7fdcbe7fc640 20 reqs_thread_entry: start 2024-05-08T18:05:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.584+0000 7fde94328800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-05-08T18:05:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.584+0000 7fde94328800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.584+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6fa3d0 con 0x557c2a6b46e0 2024-05-08T18:05:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.584+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(62 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.584+0000 7fde94328800 20 rados_obj.operate() r=0 bl.length=380 2024-05-08T18:05:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.584+0000 7fde94328800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-05-08T18:05:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.584+0000 7fde94328800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-05-08T18:05:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.584+0000 7fde94328800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.584+0000 7fde94328800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-05-08T18:05:08.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.584+0000 7fde94328800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.585+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6fc4b0 con 0x7fde78037040 2024-05-08T18:05:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.585+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv22 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fde88040040 con 0x7fde78037040 2024-05-08T18:05:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.585+0000 7fde94328800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.585+0000 7fde94328800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-05-08T18:05:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.585+0000 7fde94328800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.585+0000 7fde94328800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 : miss 2024-05-08T18:05:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.585+0000 7fde94328800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.585+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:64 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6fdef0 con 0x557c2a6b46e0 2024-05-08T18:05:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.586+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(64 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv2 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.586+0000 7fde94328800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.586+0000 7fde94328800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.586+0000 7fde94328800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.586+0000 7fde94328800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.586+0000 7fde94328800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:08.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.586+0000 7fde94328800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 2024-05-08T18:05:08.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.586+0000 7fde94328800 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x13) 2024-05-08T18:05:08.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.586+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:65 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a7010e0 con 0x557c2a6b46e0 2024-05-08T18:05:08.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.587+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(65 fud.buckets [call] v27'2 uv2 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.587+0000 7fde94328800 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x11) 2024-05-08T18:05:08.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.587+0000 7fde94328800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.587+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a7000b0 con 0x7fde78037040 2024-05-08T18:05:08.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.587+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(66 myfoo [call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv22 ondisk = 0) ==== 233+0+288 (crc 0 0 0) 0x7fde88040040 con 0x7fde78037040 2024-05-08T18:05:08.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.587+0000 7fde94328800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:08.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.587+0000 7fde94328800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2024-05-08T18:05:08.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.587+0000 7fde94328800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:08.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.587+0000 7fde94328800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:08.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.587+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a700ae0 con 0x7fde78037040 2024-05-08T18:05:08.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.588+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(67 myfoo [delete,create,call,call,writefull 0~165] v27'23 uv23 ondisk = 0) ==== 317+0+0 (crc 0 0 0) 0x7fde88040040 con 0x7fde78037040 2024-05-08T18:05:08.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.588+0000 7fde94328800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-05-08T18:05:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.588+0000 7fde94328800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.588+0000 7fde94328800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-05-08T18:05:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.588+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 93991776290224 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557c2a7029a0 con 0x557c2a6b46e0 2024-05-08T18:05:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.589+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== watch-notify(notify (1) cookie 93991776028544 notify 115964116998 ret 0) ==== 380+0+0 (crc 0 0 0) 0x557c2a6fd430 con 0x557c2a6b46e0 2024-05-08T18:05:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.589+0000 7fdd6effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116998 cookie 93991776028544 notifier 4508 bl.length()=338 2024-05-08T18:05:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.589+0000 7fdd6effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-05-08T18:05:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.589+0000 7fdd6effd640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.589+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(68 notify.6 [notify cookie 93991776290224 out=8b] v0'0 uv120 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.589+0000 7fdd6effd640 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fdd54003ea0 con 0x557c2a6b46e0 2024-05-08T18:05:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.589+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.589+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== watch-notify(notify_complete (2) cookie 93991776290224 notify 115964116998 ret 0) ==== 42+0+48 (crc 0 0 0) 0x557c2a690a70 con 0x557c2a6b46e0 2024-05-08T18:05:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.590+0000 7fdd45fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:08.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.590+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93991776041792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a7005b0 con 0x557c2a6afcf0 2024-05-08T18:05:08.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.591+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 93991776041792] v27'118 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fde88040040 con 0x557c2a6afcf0 2024-05-08T18:05:08.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.591+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93991776015424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a7005b0 con 0x7fde78037040 2024-05-08T18:05:08.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.592+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 93991776015424] v27'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde88040040 con 0x7fde78037040 2024-05-08T18:05:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.592+0000 7fde94328800 20 remove_watcher() i=0 2024-05-08T18:05:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.592+0000 7fde94328800 2 removed watcher, disabling cache 2024-05-08T18:05:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.592+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93991776019744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a7005b0 con 0x557c2a6afcf0 2024-05-08T18:05:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.593+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 93991776019744] v27'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde88040040 con 0x557c2a6afcf0 2024-05-08T18:05:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.593+0000 7fde94328800 20 remove_watcher() i=1 2024-05-08T18:05:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.593+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93991776012912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6be520 con 0x557c2a6b9090 2024-05-08T18:05:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.594+0000 7fde8f7e6640 1 -- 172.21.15.50:0/3875543408 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 93991776012912] v27'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde78009050 con 0x557c2a6b9090 2024-05-08T18:05:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.594+0000 7fde94328800 20 remove_watcher() i=2 2024-05-08T18:05:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.594+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93991776026336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a7005b0 con 0x7fde78037040 2024-05-08T18:05:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.595+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 93991776026336] v27'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde88040040 con 0x7fde78037040 2024-05-08T18:05:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.595+0000 7fde94328800 20 remove_watcher() i=3 2024-05-08T18:05:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.595+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93991776029744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6bca70 con 0x557c2a6afcf0 2024-05-08T18:05:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.596+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 93991776029744] v27'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde88040040 con 0x557c2a6afcf0 2024-05-08T18:05:08.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.596+0000 7fde94328800 20 remove_watcher() i=4 2024-05-08T18:05:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.596+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93991776032016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6bfee0 con 0x557c2a6b46e0 2024-05-08T18:05:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.597+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 93991776032016] v27'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.597+0000 7fde94328800 20 remove_watcher() i=5 2024-05-08T18:05:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.597+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93991776028544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a7005b0 con 0x557c2a6b46e0 2024-05-08T18:05:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.598+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 93991776028544] v27'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.598+0000 7fde94328800 20 remove_watcher() i=6 2024-05-08T18:05:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.598+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93991776038160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557c2a6bca70 con 0x557c2a6b46e0 2024-05-08T18:05:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde8ffe7640 1 -- 172.21.15.50:0/3875543408 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 93991776038160] v27'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fde80089030 con 0x557c2a6b46e0 2024-05-08T18:05:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 20 remove_watcher() i=7 2024-05-08T18:05:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557c2a6b46e0 msgr2=0x557c2a6b6ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557c2a6b46e0 0x557c2a6b6ad0 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557c2a6afcf0 msgr2=0x557c2a6b21b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557c2a6afcf0 0x557c2a6b21b0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fde78037040 msgr2=0x7fde78036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fde78037040 0x7fde78036410 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557c2a6b9090 msgr2=0x557c2a6bb550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557c2a6b9090 0x557c2a6bb550 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdd5803bba0 msgr2=0x7fdd5803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdd5803bba0 0x7fdd5803e060 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fde7800efd0 tx=0x7fde78010000 comp rx=0 tx=0).stop 2024-05-08T18:05:08.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a69d1f0 msgr2=0x557c2a6a4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 --2- 172.21.15.50:0/3875543408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a69d1f0 0x557c2a6a4550 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fde800098b0 tx=0x7fde8005b900 comp rx=0 tx=0).stop 2024-05-08T18:05:08.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 reap_dead start 2024-05-08T18:05:08.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde907e8640 1 -- 172.21.15.50:0/3875543408 reap_dead start 2024-05-08T18:05:08.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 shutdown_connections 2024-05-08T18:05:08.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 >> 172.21.15.50:0/3875543408 conn(0x557c2a699500 msgr2=0x557c2a696b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 shutdown_connections 2024-05-08T18:05:08.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 -- 172.21.15.50:0/3875543408 wait complete. 2024-05-08T18:05:08.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 -- 172.21.15.50:0/915814190 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdd6803bba0 msgr2=0x7fdd6803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.599+0000 7fde94328800 1 --2- 172.21.15.50:0/915814190 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdd6803bba0 0x7fdd6803e060 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fde780016b0 tx=0x7fde7800f000 comp rx=0 tx=0).stop 2024-05-08T18:05:08.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 -- 172.21.15.50:0/915814190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a690110 msgr2=0x557c2a690530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 --2- 172.21.15.50:0/915814190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a690110 0x557c2a690530 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fde80092a60 tx=0x7fde80078ce0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 -- 172.21.15.50:0/915814190 shutdown_connections 2024-05-08T18:05:08.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 --2- 172.21.15.50:0/915814190 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdd6803bba0 0x7fdd6803e060 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 --2- 172.21.15.50:0/915814190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a690110 0x557c2a690530 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 -- 172.21.15.50:0/915814190 >> 172.21.15.50:0/915814190 conn(0x557c2a6841b0 msgr2=0x557c2a6845b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 -- 172.21.15.50:0/915814190 shutdown_connections 2024-05-08T18:05:08.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 -- 172.21.15.50:0/915814190 wait complete. 2024-05-08T18:05:08.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 -- 172.21.15.50:0/941667054 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557c2a67b270 msgr2=0x557c2a67d700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557c2a67b270 0x557c2a67d700 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 -- 172.21.15.50:0/941667054 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557c2a5778d0 msgr2=0x557c2a678b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557c2a5778d0 0x557c2a678b10 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 -- 172.21.15.50:0/941667054 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557c2a67f2f0 msgr2=0x557c2a6816e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557c2a67f2f0 0x557c2a6816e0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 -- 172.21.15.50:0/941667054 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdd7803bb50 msgr2=0x7fdd7803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fdd7803bb50 0x7fdd7803e010 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fde78001d70 tx=0x7fde780012d0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 -- 172.21.15.50:0/941667054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 msgr2=0x557c2a6718d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 --2- 172.21.15.50:0/941667054 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557c2a576710 0x557c2a6718d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fde800057d0 tx=0x7fde80002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde907e8640 1 -- 172.21.15.50:0/941667054 reap_dead start 2024-05-08T18:05:08.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 -- 172.21.15.50:0/941667054 shutdown_connections 2024-05-08T18:05:08.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 -- 172.21.15.50:0/941667054 >> 172.21.15.50:0/941667054 conn(0x557c2a571c90 msgr2=0x557c2a4eace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 -- 172.21.15.50:0/941667054 shutdown_connections 2024-05-08T18:05:08.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.600+0000 7fde94328800 1 -- 172.21.15.50:0/941667054 wait complete. 2024-05-08T18:05:08.610 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1'] 2024-05-08T18:05:08.610 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1'] 2024-05-08T18:05:08.611 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid foo --bucket myfoo --bucket-id 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 2024-05-08T18:05:08.661 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:08.662 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:08.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.663+0000 7fc03551f800 1 Processor -- start 2024-05-08T18:05:08.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.663+0000 7fc03551f800 1 -- start start 2024-05-08T18:05:08.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.663+0000 7fc03551f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55e6bfce2b70 0x55e6bfce2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.663+0000 7fc03551f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55e6bfce35c0 con 0x55e6bfce1880 2024-05-08T18:05:08.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.663+0000 7fc03551f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55e6bfb82e20 con 0x55e6bfce2b70 2024-05-08T18:05:08.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.663+0000 7fc031252640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55e6bfce2b70 0x55e6bfce2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.663+0000 7fc030a51640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55e6bfce1880 0x55e6bfce1d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43204/0 (socket says 172.21.15.50:43204) 2024-05-08T18:05:08.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.663+0000 7fc030a51640 1 -- 172.21.15.50:0/2839302527 learned_addr learned my addr 172.21.15.50:0/2839302527 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:08.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.664+0000 7fc031252640 1 -- 172.21.15.50:0/2839302527 >> v1:172.21.15.46:6789/0 conn(0x55e6bfce1880 legacy=0x55e6bfce1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.664+0000 7fc031252640 1 -- 172.21.15.50:0/2839302527 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6bfce2440 con 0x55e6bfce2b70 2024-05-08T18:05:08.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.664+0000 7fc023fff640 1 -- 172.21.15.50:0/2839302527 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1584104996 0 0) 0x55e6bfce35c0 con 0x55e6bfce1880 2024-05-08T18:05:08.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.665+0000 7fc031252640 1 --2- 172.21.15.50:0/2839302527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce2b70 0x55e6bfce2f90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc02400a470 tx=0x7fc02401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5817d105aaaa64 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.666+0000 7fc023fff640 1 -- 172.21.15.50:0/2839302527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc024057070 con 0x55e6bfce2b70 2024-05-08T18:05:08.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.666+0000 7fc023fff640 1 -- 172.21.15.50:0/2839302527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02404adc0 con 0x55e6bfce2b70 2024-05-08T18:05:08.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.666+0000 7fc023fff640 1 -- 172.21.15.50:0/2839302527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc024052590 con 0x55e6bfce2b70 2024-05-08T18:05:08.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.666+0000 7fc03551f800 1 -- 172.21.15.50:0/2839302527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce2b70 msgr2=0x55e6bfce2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.666+0000 7fc03551f800 1 --2- 172.21.15.50:0/2839302527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce2b70 0x55e6bfce2f90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc02400a470 tx=0x7fc02401ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:08.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.666+0000 7fc03551f800 1 -- 172.21.15.50:0/2839302527 shutdown_connections 2024-05-08T18:05:08.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.666+0000 7fc03551f800 1 --2- 172.21.15.50:0/2839302527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce2b70 0x55e6bfce2f90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.666+0000 7fc03551f800 1 -- 172.21.15.50:0/2839302527 >> 172.21.15.50:0/2839302527 conn(0x55e6bfcdc2f0 msgr2=0x55e6bfcde710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.666+0000 7fc03551f800 1 -- 172.21.15.50:0/2839302527 shutdown_connections 2024-05-08T18:05:08.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.666+0000 7fc03551f800 1 -- 172.21.15.50:0/2839302527 wait complete. 2024-05-08T18:05:08.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.681+0000 7fc03551f800 1 Processor -- start 2024-05-08T18:05:08.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.681+0000 7fc03551f800 1 -- start start 2024-05-08T18:05:08.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.682+0000 7fc03551f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce1880 0x55e6bfdd1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.682+0000 7fc03551f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e6bfc2da30 con 0x55e6bfce1880 2024-05-08T18:05:08.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.682+0000 7fc031252640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce1880 0x55e6bfdd1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.682+0000 7fc031252640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce1880 0x55e6bfdd1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:37996/0 (socket says 172.21.15.50:37996) 2024-05-08T18:05:08.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.682+0000 7fc031252640 1 -- 172.21.15.50:0/454829273 learned_addr learned my addr 172.21.15.50:0/454829273 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.682+0000 7fc031252640 1 -- 172.21.15.50:0/454829273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6bfdd1f20 con 0x55e6bfce1880 2024-05-08T18:05:08.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.682+0000 7fc031252640 1 --2- 172.21.15.50:0/454829273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce1880 0x55e6bfdd1950 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc024000c00 tx=0x7fc02404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456d6aa38f33cb93 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.682+0000 7fbf1f606640 1 -- 172.21.15.50:0/454829273 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc024057040 con 0x55e6bfce1880 2024-05-08T18:05:08.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.682+0000 7fbf1f606640 1 -- 172.21.15.50:0/454829273 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc024063070 con 0x55e6bfce1880 2024-05-08T18:05:08.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.682+0000 7fbf1f606640 1 -- 172.21.15.50:0/454829273 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc02405d470 con 0x55e6bfce1880 2024-05-08T18:05:08.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.683+0000 7fc03551f800 1 -- 172.21.15.50:0/454829273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce1880 msgr2=0x55e6bfdd1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.683+0000 7fc03551f800 1 --2- 172.21.15.50:0/454829273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce1880 0x55e6bfdd1950 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc024000c00 tx=0x7fc02404c000 comp rx=0 tx=0).stop 2024-05-08T18:05:08.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.683+0000 7fc03551f800 1 -- 172.21.15.50:0/454829273 shutdown_connections 2024-05-08T18:05:08.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.683+0000 7fc03551f800 1 --2- 172.21.15.50:0/454829273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce1880 0x55e6bfdd1950 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.683+0000 7fc03551f800 1 -- 172.21.15.50:0/454829273 >> 172.21.15.50:0/454829273 conn(0x55e6bfcdc2f0 msgr2=0x55e6bfdcd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.683+0000 7fc03551f800 1 -- 172.21.15.50:0/454829273 shutdown_connections 2024-05-08T18:05:08.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.683+0000 7fc03551f800 1 -- 172.21.15.50:0/454829273 wait complete. 2024-05-08T18:05:08.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.683+0000 7fc03551f800 1 Processor -- start 2024-05-08T18:05:08.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.684+0000 7fc03551f800 1 -- start start 2024-05-08T18:05:08.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.684+0000 7fc03551f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce1880 0x55e6bfdd93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.684+0000 7fc03551f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e6bfce09b0 con 0x55e6bfce1880 2024-05-08T18:05:08.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.684+0000 7fc031252640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce1880 0x55e6bfdd93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.684+0000 7fc031252640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce1880 0x55e6bfdd93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38010/0 (socket says 172.21.15.50:38010) 2024-05-08T18:05:08.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.684+0000 7fc031252640 1 -- 172.21.15.50:0/3316475235 learned_addr learned my addr 172.21.15.50:0/3316475235 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.684+0000 7fc031252640 1 -- 172.21.15.50:0/3316475235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6bfdd9900 con 0x55e6bfce1880 2024-05-08T18:05:08.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.684+0000 7fc031252640 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce1880 0x55e6bfdd93c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc024000c00 tx=0x7fc024002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.684+0000 7fbf1de03640 1 -- 172.21.15.50:0/3316475235 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc02406c070 con 0x55e6bfce1880 2024-05-08T18:05:08.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.684+0000 7fbf1de03640 1 -- 172.21.15.50:0/3316475235 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02405d8f0 con 0x55e6bfce1880 2024-05-08T18:05:08.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.684+0000 7fc03551f800 1 -- 172.21.15.50:0/3316475235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e6bfdd9b90 con 0x55e6bfce1880 2024-05-08T18:05:08.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.684+0000 7fbf1de03640 1 -- 172.21.15.50:0/3316475235 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc024057070 con 0x55e6bfce1880 2024-05-08T18:05:08.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.685+0000 7fc03551f800 1 -- 172.21.15.50:0/3316475235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6bfddc7a0 con 0x55e6bfce1880 2024-05-08T18:05:08.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.685+0000 7fbf1de03640 1 -- 172.21.15.50:0/3316475235 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fc02405dc10 con 0x55e6bfce1880 2024-05-08T18:05:08.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.686+0000 7fbf1de03640 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf1803bba0 0x7fbf1803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.686+0000 7fbf1de03640 1 -- 172.21.15.50:0/3316475235 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fc024094aa0 con 0x55e6bfce1880 2024-05-08T18:05:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.686+0000 7fc030a51640 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf1803bba0 0x7fbf1803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.686+0000 7fc03551f800 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e6bfce2b70 0x55e6bfde0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.686+0000 7fc03551f800 1 -- 172.21.15.50:0/3316475235 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfde0aa0 con 0x55e6bfce2b70 2024-05-08T18:05:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.686+0000 7fc031a53640 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e6bfce2b70 0x55e6bfde0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.686+0000 7fc030a51640 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf1803bba0 0x7fbf1803e060 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc028000c00 tx=0x7fc0280013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.686+0000 7fc031a53640 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e6bfce2b70 0x55e6bfde0560 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.687+0000 7fc031a53640 1 -- 172.21.15.50:0/3316475235 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfce2b70 2024-05-08T18:05:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.687+0000 7fc03551f800 1 -- 172.21.15.50:0/3316475235 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfdddaf0 con 0x55e6bfce2b70 2024-05-08T18:05:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.687+0000 7fc031a53640 1 -- 172.21.15.50:0/3316475235 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfce2b70 2024-05-08T18:05:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.687+0000 7fc03551f800 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e6bfde2d40 0x55e6bfde5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.687+0000 7fc03551f800 1 -- 172.21.15.50:0/3316475235 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfde57d0 con 0x55e6bfde2d40 2024-05-08T18:05:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.687+0000 7fc031252640 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e6bfde2d40 0x55e6bfde5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.688+0000 7fc031252640 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e6bfde2d40 0x55e6bfde5130 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.688+0000 7fc031252640 1 -- 172.21.15.50:0/3316475235 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfde2d40 2024-05-08T18:05:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.688+0000 7fc03551f800 1 -- 172.21.15.50:0/3316475235 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfde1be0 con 0x55e6bfce2b70 2024-05-08T18:05:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.688+0000 7fc031a53640 1 -- 172.21.15.50:0/3316475235 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfce2b70 2024-05-08T18:05:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.688+0000 7fc03551f800 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e6bfde6c30 0x55e6bfde9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.688+0000 7fc03551f800 1 -- 172.21.15.50:0/3316475235 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfde9760 con 0x55e6bfde6c30 2024-05-08T18:05:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.688+0000 7fc030a51640 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e6bfde6c30 0x55e6bfde9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.689+0000 7fc030a51640 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e6bfde6c30 0x55e6bfde9020 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.689+0000 7fc030a51640 1 -- 172.21.15.50:0/3316475235 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fc02800f670 con 0x55e6bfde6c30 2024-05-08T18:05:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.689+0000 7fc03551f800 1 Processor -- start 2024-05-08T18:05:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.689+0000 7fc03551f800 1 -- start start 2024-05-08T18:05:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.689+0000 7fc03551f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfdfbee0 0x55e6bfdfc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.689+0000 7fc03551f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e6bfbd4330 con 0x55e6bfdfbee0 2024-05-08T18:05:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.690+0000 7fc031252640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfdfbee0 0x55e6bfdfc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.690+0000 7fc031252640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfdfbee0 0x55e6bfdfc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38026/0 (socket says 172.21.15.50:38026) 2024-05-08T18:05:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.690+0000 7fc031252640 1 -- 172.21.15.50:0/3015793718 learned_addr learned my addr 172.21.15.50:0/3015793718 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.690+0000 7fc031252640 1 -- 172.21.15.50:0/3015793718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6bfdfc840 con 0x55e6bfdfbee0 2024-05-08T18:05:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.690+0000 7fc031252640 1 --2- 172.21.15.50:0/3015793718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfdfbee0 0x55e6bfdfc300 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc024092e90 tx=0x7fc02407d1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.690+0000 7fbf16ffd640 1 -- 172.21.15.50:0/3015793718 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc024009410 con 0x55e6bfdfbee0 2024-05-08T18:05:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.690+0000 7fbf16ffd640 1 -- 172.21.15.50:0/3015793718 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02407a3f0 con 0x55e6bfdfbee0 2024-05-08T18:05:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.690+0000 7fc03551f800 1 -- 172.21.15.50:0/3015793718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e6bfdfd2b0 con 0x55e6bfdfbee0 2024-05-08T18:05:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.690+0000 7fbf16ffd640 1 -- 172.21.15.50:0/3015793718 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0240095b0 con 0x55e6bfdfbee0 2024-05-08T18:05:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.690+0000 7fc03551f800 1 -- 172.21.15.50:0/3015793718 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6bfdfd7b0 con 0x55e6bfdfbee0 2024-05-08T18:05:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.691+0000 7fbf16ffd640 1 -- 172.21.15.50:0/3015793718 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fc024068ab0 con 0x55e6bfdfbee0 2024-05-08T18:05:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.691+0000 7fbf16ffd640 1 --2- 172.21.15.50:0/3015793718 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf0803bb50 0x7fbf0803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.691+0000 7fbf16ffd640 1 -- 172.21.15.50:0/3015793718 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fc024085070 con 0x55e6bfdfbee0 2024-05-08T18:05:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.691+0000 7fc030a51640 1 --2- 172.21.15.50:0/3015793718 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf0803bb50 0x7fbf0803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.692+0000 7fc03551f800 1 Processor -- start 2024-05-08T18:05:08.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.692+0000 7fc030a51640 1 --2- 172.21.15.50:0/3015793718 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf0803bb50 0x7fbf0803e010 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55e6bfde0ea0 tx=0x7fc028006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.692+0000 7fc03551f800 1 -- start start 2024-05-08T18:05:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.692+0000 7fc03551f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfe08fe0 0x55e6bfe09400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.692+0000 7fc03551f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e6bfdd21e0 con 0x55e6bfe08fe0 2024-05-08T18:05:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.692+0000 7fc031252640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfe08fe0 0x55e6bfe09400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.692+0000 7fc031252640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfe08fe0 0x55e6bfe09400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38032/0 (socket says 172.21.15.50:38032) 2024-05-08T18:05:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.692+0000 7fc031252640 1 -- 172.21.15.50:0/3069860625 learned_addr learned my addr 172.21.15.50:0/3069860625 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.692+0000 7fc031252640 1 -- 172.21.15.50:0/3069860625 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6bfe099d0 con 0x55e6bfe08fe0 2024-05-08T18:05:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.693+0000 7fc031252640 1 --2- 172.21.15.50:0/3069860625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfe08fe0 0x55e6bfe09400 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc02407efd0 tx=0x7fc02407cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f10376cc70f3b36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.693+0000 7fbf14ff9640 1 -- 172.21.15.50:0/3069860625 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc02406ebc0 con 0x55e6bfe08fe0 2024-05-08T18:05:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.693+0000 7fbf14ff9640 1 -- 172.21.15.50:0/3069860625 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc024068400 con 0x55e6bfe08fe0 2024-05-08T18:05:08.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.693+0000 7fbf14ff9640 1 -- 172.21.15.50:0/3069860625 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc02407d500 con 0x55e6bfe08fe0 2024-05-08T18:05:08.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.693+0000 7fc03551f800 1 -- 172.21.15.50:0/3069860625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfe08fe0 msgr2=0x55e6bfe09400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.693+0000 7fc03551f800 1 --2- 172.21.15.50:0/3069860625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfe08fe0 0x55e6bfe09400 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc02407efd0 tx=0x7fc02407cf90 comp rx=0 tx=0).stop 2024-05-08T18:05:08.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.693+0000 7fc03551f800 1 -- 172.21.15.50:0/3069860625 shutdown_connections 2024-05-08T18:05:08.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.693+0000 7fc03551f800 1 --2- 172.21.15.50:0/3069860625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfe08fe0 0x55e6bfe09400 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.693+0000 7fc03551f800 1 -- 172.21.15.50:0/3069860625 >> 172.21.15.50:0/3069860625 conn(0x55e6bfe052f0 msgr2=0x55e6bfe03350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.693+0000 7fc03551f800 1 -- 172.21.15.50:0/3069860625 shutdown_connections 2024-05-08T18:05:08.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.693+0000 7fc03551f800 1 -- 172.21.15.50:0/3069860625 wait complete. 2024-05-08T18:05:08.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.693+0000 7fc03551f800 1 Processor -- start 2024-05-08T18:05:08.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.694+0000 7fc03551f800 1 -- start start 2024-05-08T18:05:08.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.694+0000 7fc03551f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfe08fe0 0x55e6bfe10350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.694+0000 7fc03551f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e6bfe0a060 con 0x55e6bfe08fe0 2024-05-08T18:05:08.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.694+0000 7fc031252640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfe08fe0 0x55e6bfe10350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.694+0000 7fc031252640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfe08fe0 0x55e6bfe10350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38036/0 (socket says 172.21.15.50:38036) 2024-05-08T18:05:08.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.694+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 learned_addr learned my addr 172.21.15.50:0/671928827 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.694+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6bfe10890 con 0x55e6bfe08fe0 2024-05-08T18:05:08.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.694+0000 7fc031252640 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfe08fe0 0x55e6bfe10350 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc024091c00 tx=0x7fc024067f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.694+0000 7fbf06ffd640 1 -- 172.21.15.50:0/671928827 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0240b6020 con 0x55e6bfe08fe0 2024-05-08T18:05:08.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.694+0000 7fbf06ffd640 1 -- 172.21.15.50:0/671928827 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02405e7b0 con 0x55e6bfe08fe0 2024-05-08T18:05:08.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.694+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e6bfe10890 con 0x55e6bfe08fe0 2024-05-08T18:05:08.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.694+0000 7fbf06ffd640 1 -- 172.21.15.50:0/671928827 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc02407d500 con 0x55e6bfe08fe0 2024-05-08T18:05:08.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.695+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6bfe10d90 con 0x55e6bfe08fe0 2024-05-08T18:05:08.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.695+0000 7fbf06ffd640 1 -- 172.21.15.50:0/671928827 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fc0240b9020 con 0x55e6bfe08fe0 2024-05-08T18:05:08.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.695+0000 7fbf06ffd640 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbefc03bb50 0x7fbefc03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.696+0000 7fbf06ffd640 1 -- 172.21.15.50:0/671928827 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fc0240ac050 con 0x55e6bfe08fe0 2024-05-08T18:05:08.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.696+0000 7fc030a51640 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbefc03bb50 0x7fbefc03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.696+0000 7fc030a51640 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbefc03bb50 0x7fbefc03e010 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc02800ffd0 tx=0x7fc028023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.696+0000 7fc03551f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.696+0000 7fc03551f800 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e6bfe1bb40 0x55e6bfe1e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.696+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1e750 con 0x55e6bfe1bb40 2024-05-08T18:05:08.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.696+0000 7fc031a53640 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e6bfe1bb40 0x55e6bfe1e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.697+0000 7fc031a53640 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e6bfe1bb40 0x55e6bfe1e000 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.697+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.697+0000 7fc03551f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.697+0000 7fc03551f800 20 realm 2024-05-08T18:05:08.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.697+0000 7fc03551f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.697+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1eb60 con 0x55e6bfe1bb40 2024-05-08T18:05:08.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.697+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.697+0000 7fc03551f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.697+0000 7fc03551f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:08.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.697+0000 7fc03551f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.697+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f430 con 0x55e6bfe1bb40 2024-05-08T18:05:08.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.697+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.697+0000 7fc03551f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.697+0000 7fc03551f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.697+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.698+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.698+0000 7fc03551f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:08.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.698+0000 7fc03551f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.698+0000 7fc03551f800 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e6bfe20530 0x55e6bfe22920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.698+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe22ee0 con 0x55e6bfe20530 2024-05-08T18:05:08.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.698+0000 7fc031252640 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e6bfe20530 0x55e6bfe22920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.698+0000 7fc031252640 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e6bfe20530 0x55e6bfe22920 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.698+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.698+0000 7fc03551f800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:08.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.698+0000 7fc03551f800 20 searching for the correct realm 2024-05-08T18:05:08.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.698+0000 7fc03551f800 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e6bfe24e90 0x55e6bfe27350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.699+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe27a20 con 0x55e6bfe24e90 2024-05-08T18:05:08.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.699+0000 7fc030a51640 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e6bfe24e90 0x55e6bfe27350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.699+0000 7fc030a51640 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e6bfe24e90 0x55e6bfe27350 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.699+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc028037040 con 0x55e6bfe24e90 2024-05-08T18:05:08.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.699+0000 7fc030a51640 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fc02802e910 0x7fc02803a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.699+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe27a20 con 0x7fc02802e910 2024-05-08T18:05:08.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.699+0000 7fc031a53640 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fc02802e910 0x7fc02803a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.700+0000 7fc031a53640 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fc02802e910 0x7fc02803a3c0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.700+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02c00a4a0 con 0x7fc02802e910 2024-05-08T18:05:08.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.700+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe20530 2024-05-08T18:05:08.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.701+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.701+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.701+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.701+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.701+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.701+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe20530 2024-05-08T18:05:08.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.701+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.701+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe20530 2024-05-08T18:05:08.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.702+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.702+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x7fc02802e910 2024-05-08T18:05:08.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.702+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02c00a4a0 con 0x7fc02802e910 2024-05-08T18:05:08.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.702+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe24e90 2024-05-08T18:05:08.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.702+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc028037040 con 0x55e6bfe24e90 2024-05-08T18:05:08.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.702+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.702+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.702+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe20530 2024-05-08T18:05:08.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.703+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.703+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.703+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.703+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x7fc02802e910 2024-05-08T18:05:08.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.703+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fc02c00a4a0 con 0x7fc02802e910 2024-05-08T18:05:08.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.703+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe24e90 2024-05-08T18:05:08.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.703+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc028037040 con 0x55e6bfe24e90 2024-05-08T18:05:08.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.703+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe24e90 2024-05-08T18:05:08.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.704+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc028037040 con 0x55e6bfe24e90 2024-05-08T18:05:08.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.704+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe20530 2024-05-08T18:05:08.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.704+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.704+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe24e90 2024-05-08T18:05:08.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.704+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc028037040 con 0x55e6bfe24e90 2024-05-08T18:05:08.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.704+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe24e90 2024-05-08T18:05:08.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.704+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fc028037040 con 0x55e6bfe24e90 2024-05-08T18:05:08.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.704+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.705+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.705+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.705+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.705+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe20530 2024-05-08T18:05:08.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.705+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.705+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe24e90 2024-05-08T18:05:08.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.705+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc028037040 con 0x55e6bfe24e90 2024-05-08T18:05:08.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.705+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x7fc02802e910 2024-05-08T18:05:08.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.706+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02c00a4a0 con 0x7fc02802e910 2024-05-08T18:05:08.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.706+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.706+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.706+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe20530 2024-05-08T18:05:08.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.706+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.706+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe24e90 2024-05-08T18:05:08.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.706+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc028037040 con 0x55e6bfe24e90 2024-05-08T18:05:08.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.706+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe20530 2024-05-08T18:05:08.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.707+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.707+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x7fc02802e910 2024-05-08T18:05:08.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.707+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02c00a4a0 con 0x7fc02802e910 2024-05-08T18:05:08.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.707+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe24e90 2024-05-08T18:05:08.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.707+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc028037040 con 0x55e6bfe24e90 2024-05-08T18:05:08.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.707+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe20530 2024-05-08T18:05:08.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.707+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.707+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.708+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.708+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe1f7d0 con 0x7fc02802e910 2024-05-08T18:05:08.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.708+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc02c00a4a0 con 0x7fc02802e910 2024-05-08T18:05:08.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.708+0000 7fc03551f800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:08.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.708+0000 7fc03551f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:08.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.708+0000 7fc03551f800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:08.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.708+0000 7fc03551f800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:08.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.708+0000 7fc03551f800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:08.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.708+0000 7fc03551f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:08.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.708+0000 7fc03551f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.708+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe237a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.708+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.708+0000 7fc03551f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.708+0000 7fc03551f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.708+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe237a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc02c00a4a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:08.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe237a0 con 0x55e6bfe24e90 2024-05-08T18:05:08.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fc028037040 con 0x55e6bfe24e90 2024-05-08T18:05:08.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:08.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 20 zone default found 2024-05-08T18:05:08.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 4 Realm: () 2024-05-08T18:05:08.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:08.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:08.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:08.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 20 zonegroup default 2024-05-08T18:05:08.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe24140 con 0x55e6bfe20530 2024-05-08T18:05:08.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe29910 con 0x55e6bfe24e90 2024-05-08T18:05:08.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc028037040 con 0x55e6bfe24e90 2024-05-08T18:05:08.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.709+0000 7fc03551f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.710+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe29da0 con 0x55e6bfe24e90 2024-05-08T18:05:08.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.710+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc028037040 con 0x55e6bfe24e90 2024-05-08T18:05:08.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.710+0000 7fc03551f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.710+0000 7fc03551f800 20 started sync module instance, tier type = 2024-05-08T18:05:08.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.710+0000 7fc03551f800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:08.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.710+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe28d20 con 0x7fc02802e910 2024-05-08T18:05:08.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.711+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02c00a4a0 con 0x7fc02802e910 2024-05-08T18:05:08.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.711+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94449550135872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe28d20 con 0x7fc02802e910 2024-05-08T18:05:08.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.711+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2a8b0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.712+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94449550135872] v27'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02c040040 con 0x7fc02802e910 2024-05-08T18:05:08.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.712+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02c040040 con 0x55e6bfe1bb40 2024-05-08T18:05:08.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.712+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94449550140320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe28d20 con 0x55e6bfe1bb40 2024-05-08T18:05:08.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.712+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2b340 con 0x55e6bfe24e90 2024-05-08T18:05:08.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.713+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc028037040 con 0x55e6bfe24e90 2024-05-08T18:05:08.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.713+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94449550142992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2bbe0 con 0x55e6bfe24e90 2024-05-08T18:05:08.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.713+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2c6c0 con 0x7fc02802e910 2024-05-08T18:05:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.713+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94449550140320] v27'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02c040040 con 0x55e6bfe1bb40 2024-05-08T18:05:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.714+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94449550142992] v27'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc028037040 con 0x55e6bfe24e90 2024-05-08T18:05:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.714+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02c040040 con 0x7fc02802e910 2024-05-08T18:05:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.714+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94449550145328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe28870 con 0x7fc02802e910 2024-05-08T18:05:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.714+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2cf60 con 0x55e6bfe1bb40 2024-05-08T18:05:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.715+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02c040040 con 0x55e6bfe1bb40 2024-05-08T18:05:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.715+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94449550150240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2ce80 con 0x55e6bfe1bb40 2024-05-08T18:05:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.715+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2de60 con 0x55e6bfe20530 2024-05-08T18:05:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.715+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94449550145328] v27'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02c040040 con 0x7fc02802e910 2024-05-08T18:05:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.716+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94449550150240] v27'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02c040040 con 0x55e6bfe1bb40 2024-05-08T18:05:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.716+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.716+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94449550152528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2d970 con 0x55e6bfe20530 2024-05-08T18:05:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.716+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2e700 con 0x55e6bfe20530 2024-05-08T18:05:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.717+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94449550152528] v27'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.717+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.717+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94449550156288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2e620 con 0x55e6bfe20530 2024-05-08T18:05:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.717+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2f470 con 0x55e6bfe20530 2024-05-08T18:05:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.718+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94449550156288] v27'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.718+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.718+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94449550158496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2f470 con 0x55e6bfe20530 2024-05-08T18:05:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.718+0000 7fc03551f800 20 add_watcher() i=0 2024-05-08T18:05:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.718+0000 7fc03551f800 20 add_watcher() i=1 2024-05-08T18:05:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.718+0000 7fc03551f800 20 add_watcher() i=2 2024-05-08T18:05:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.718+0000 7fc03551f800 20 add_watcher() i=3 2024-05-08T18:05:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.718+0000 7fc03551f800 20 add_watcher() i=4 2024-05-08T18:05:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.718+0000 7fc03551f800 20 add_watcher() i=5 2024-05-08T18:05:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.718+0000 7fc03551f800 20 add_watcher() i=6 2024-05-08T18:05:08.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.719+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94449550158496] v27'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.719+0000 7fc03551f800 20 add_watcher() i=7 2024-05-08T18:05:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.719+0000 7fc03551f800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.719+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe32900 con 0x55e6bfe1bb40 2024-05-08T18:05:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.719+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fc02c040040 con 0x55e6bfe1bb40 2024-05-08T18:05:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.720+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94449550159552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2fdf0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.721+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94449550159552] v27'119 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc02c040040 con 0x55e6bfe1bb40 2024-05-08T18:05:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.721+0000 7fbee7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.721+0000 7fc03551f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.721+0000 7fc03551f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.721+0000 7fc03551f800 5 note: GC not initialized 2024-05-08T18:05:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.721+0000 7fbee6fe5640 20 reqs_thread_entry: start 2024-05-08T18:05:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.721+0000 7fc03551f800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:08.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.721+0000 7fc03551f800 20 Filter name: none 2024-05-08T18:05:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.721+0000 7fbee4fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.721+0000 7fc03551f800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.721+0000 7fc03551f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.721+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe65e80 con 0x55e6bfe1bb40 2024-05-08T18:05:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.722+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fc02c040040 con 0x55e6bfe1bb40 2024-05-08T18:05:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.722+0000 7fc03551f800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.722+0000 7fc03551f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.722+0000 7fc03551f800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.722+0000 7fc03551f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.722+0000 7fc03551f800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-05-08T18:05:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.722+0000 7fc03551f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.722+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe67f60 con 0x7fc02802e910 2024-05-08T18:05:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.722+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv23 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fc02c040040 con 0x7fc02802e910 2024-05-08T18:05:08.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.722+0000 7fc03551f800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.722+0000 7fc03551f800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-05-08T18:05:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.722+0000 7fc03551f800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.722+0000 7fc03551f800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 : miss 2024-05-08T18:05:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.722+0000 7fc03551f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.722+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:64 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe69750 con 0x55e6bfe20530 2024-05-08T18:05:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.723+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(64 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv2 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.723+0000 7fc03551f800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.723+0000 7fc03551f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.723+0000 7fc03551f800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.723+0000 7fc03551f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.723+0000 7fc03551f800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.723+0000 7fc03551f800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 2024-05-08T18:05:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.723+0000 7fc03551f800 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-05-08T18:05:08.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.723+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:65 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe6db70 con 0x55e6bfe20530 2024-05-08T18:05:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.724+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(65 fud.buckets [call] v27'3 uv2 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.724+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:66 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~379 in=379b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe73ab0 con 0x55e6bfe20530 2024-05-08T18:05:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.725+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(66 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [delete,create,call,call,writefull 0~379,setxattr (123)] v27'3 uv3 ondisk = 0) ==== 416+0+0 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.725+0000 7fc03551f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.725+0000 7fc03551f800 10 moving default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.725+0000 7fc03551f800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.725+0000 7fc03551f800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1, ofs0, ns] 2024-05-08T18:05:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.725+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:67 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94449550397408 in=764b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe6dfa0 con 0x55e6bfe20530 2024-05-08T18:05:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.725+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== watch-notify(notify (1) cookie 94449550156288 notify 115964116999 ret 0) ==== 794+0+0 (crc 0 0 0) 0x55e6bfe68c80 con 0x55e6bfe20530 2024-05-08T18:05:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.725+0000 7fbf14ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116999 cookie 94449550156288 notifier 4515 bl.length()=752 2024-05-08T18:05:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.725+0000 7fbf14ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.725+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(67 notify.6 [notify cookie 94449550397408 out=8b] v0'0 uv123 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.725+0000 7fbf14ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:08.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.725+0000 7fbf14ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.725+0000 7fbf14ff9640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fbef4003ea0 con 0x55e6bfe20530 2024-05-08T18:05:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.726+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(68 notify.6 [notify-ack] v0'0 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.726+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== watch-notify(notify_complete (2) cookie 94449550397408 notify 115964116999 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55e6bfdfc840 con 0x55e6bfe20530 2024-05-08T18:05:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.726+0000 7fc03551f800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): orig_sources= new_sources= 2024-05-08T18:05:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.726+0000 7fc03551f800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): potential sources added=[] removed=[] 2024-05-08T18:05:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.726+0000 7fc03551f800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): orig_dests= new_dests= 2024-05-08T18:05:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.726+0000 7fc03551f800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): potential dests added=[] removed=[] 2024-05-08T18:05:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.726+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:69 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe72290 con 0x7fc02802e910 2024-05-08T18:05:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.727+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(69 foo.buckets [call] v27'24 uv24 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fc02c040040 con 0x7fc02802e910 2024-05-08T18:05:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.727+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe732a0 con 0x7fc02802e910 2024-05-08T18:05:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.728+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v27'25 uv25 ondisk = 0) ==== 275+0+0 (crc 0 0 0) 0x7fc02c040040 con 0x7fc02802e910 2024-05-08T18:05:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.728+0000 7fc03551f800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-05-08T18:05:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.728+0000 7fc03551f800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:08.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.728+0000 7fc03551f800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-05-08T18:05:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.728+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94449550397408 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe72740 con 0x55e6bfe20530 2024-05-08T18:05:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.728+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== watch-notify(notify (1) cookie 94449550156288 notify 115964117000 ret 0) ==== 356+0+0 (crc 0 0 0) 0x55e6bfddc7a0 con 0x55e6bfe20530 2024-05-08T18:05:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.728+0000 7fbf14ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117000 cookie 94449550156288 notifier 4515 bl.length()=314 2024-05-08T18:05:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.728+0000 7fbf14ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-05-08T18:05:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.728+0000 7fbf14ff9640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.728+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(71 notify.6 [notify cookie 94449550397408 out=8b] v0'0 uv123 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.728+0000 7fbf14ff9640 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fbef4003ea0 con 0x55e6bfe20530 2024-05-08T18:05:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.729+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== watch-notify(notify_complete (2) cookie 94449550397408 notify 115964117000 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55e6bfdd9b90 con 0x55e6bfe20530 2024-05-08T18:05:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.729+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.730+0000 7fbee7fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.730+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94449550159552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe32020 con 0x55e6bfe1bb40 2024-05-08T18:05:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.731+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94449550159552] v27'120 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc02c040040 con 0x55e6bfe1bb40 2024-05-08T18:05:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.731+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94449550135872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2eec0 con 0x7fc02802e910 2024-05-08T18:05:08.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.732+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94449550135872] v27'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02c040040 con 0x7fc02802e910 2024-05-08T18:05:08.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.732+0000 7fc03551f800 20 remove_watcher() i=0 2024-05-08T18:05:08.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.732+0000 7fc03551f800 2 removed watcher, disabling cache 2024-05-08T18:05:08.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.732+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94449550140320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2eec0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.732+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94449550140320] v27'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02c040040 con 0x55e6bfe1bb40 2024-05-08T18:05:08.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.732+0000 7fc03551f800 20 remove_watcher() i=1 2024-05-08T18:05:08.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.732+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94449550142992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2eec0 con 0x55e6bfe24e90 2024-05-08T18:05:08.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.733+0000 7fc030a51640 1 -- 172.21.15.50:0/671928827 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94449550142992] v27'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc028037040 con 0x55e6bfe24e90 2024-05-08T18:05:08.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.733+0000 7fc03551f800 20 remove_watcher() i=2 2024-05-08T18:05:08.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.733+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94449550145328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2a3a0 con 0x7fc02802e910 2024-05-08T18:05:08.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.734+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94449550145328] v27'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02c040040 con 0x7fc02802e910 2024-05-08T18:05:08.744 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.734+0000 7fc03551f800 20 remove_watcher() i=3 2024-05-08T18:05:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.734+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94449550150240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2a3a0 con 0x55e6bfe1bb40 2024-05-08T18:05:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.735+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94449550150240] v27'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02c040040 con 0x55e6bfe1bb40 2024-05-08T18:05:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.735+0000 7fc03551f800 20 remove_watcher() i=4 2024-05-08T18:05:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.735+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94449550152528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2ca60 con 0x55e6bfe20530 2024-05-08T18:05:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.736+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94449550152528] v27'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.736+0000 7fc03551f800 20 remove_watcher() i=5 2024-05-08T18:05:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.736+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94449550156288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2a3a0 con 0x55e6bfe20530 2024-05-08T18:05:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.737+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94449550156288] v27'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.737+0000 7fc03551f800 20 remove_watcher() i=6 2024-05-08T18:05:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.737+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94449550158496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6bfe2d350 con 0x55e6bfe20530 2024-05-08T18:05:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc031252640 1 -- 172.21.15.50:0/671928827 <== osd.0 v2:172.21.15.46:6800/337143975 31 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94449550158496] v27'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc02405a040 con 0x55e6bfe20530 2024-05-08T18:05:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 20 remove_watcher() i=7 2024-05-08T18:05:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e6bfe20530 msgr2=0x55e6bfe22920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e6bfe20530 0x55e6bfe22920 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e6bfe1bb40 msgr2=0x55e6bfe1e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e6bfe1bb40 0x55e6bfe1e000 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fc02802e910 msgr2=0x7fc02803a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fc02802e910 0x7fc02803a3c0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e6bfe24e90 msgr2=0x55e6bfe27350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e6bfe24e90 0x55e6bfe27350 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbefc03bb50 msgr2=0x7fbefc03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbefc03bb50 0x7fbefc03e010 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc02800ffd0 tx=0x7fc028023040 comp rx=0 tx=0).stop 2024-05-08T18:05:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfe08fe0 msgr2=0x55e6bfe10350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc031a53640 1 -- 172.21.15.50:0/671928827 reap_dead start 2024-05-08T18:05:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfe08fe0 0x55e6bfe10350 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc024091c00 tx=0x7fc024067f30 comp rx=0 tx=0).stop 2024-05-08T18:05:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 shutdown_connections 2024-05-08T18:05:08.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 --2- 172.21.15.50:0/671928827 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfe08fe0 0x55e6bfe10350 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 >> 172.21.15.50:0/671928827 conn(0x55e6bfe052f0 msgr2=0x55e6bfe02950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 shutdown_connections 2024-05-08T18:05:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 -- 172.21.15.50:0/671928827 wait complete. 2024-05-08T18:05:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 -- 172.21.15.50:0/3015793718 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf0803bb50 msgr2=0x7fbf0803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 --2- 172.21.15.50:0/3015793718 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf0803bb50 0x7fbf0803e010 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x55e6bfde0ea0 tx=0x7fc028006000 comp rx=0 tx=0).stop 2024-05-08T18:05:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 -- 172.21.15.50:0/3015793718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfdfbee0 msgr2=0x55e6bfdfc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 --2- 172.21.15.50:0/3015793718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfdfbee0 0x55e6bfdfc300 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc024092e90 tx=0x7fc02407d1e0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 -- 172.21.15.50:0/3015793718 shutdown_connections 2024-05-08T18:05:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 --2- 172.21.15.50:0/3015793718 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf0803bb50 0x7fbf0803e010 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 --2- 172.21.15.50:0/3015793718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfdfbee0 0x55e6bfdfc300 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.738+0000 7fc03551f800 1 -- 172.21.15.50:0/3015793718 >> 172.21.15.50:0/3015793718 conn(0x55e6bfdebb70 msgr2=0x55e6bfdebf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc03551f800 1 -- 172.21.15.50:0/3015793718 shutdown_connections 2024-05-08T18:05:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc03551f800 1 -- 172.21.15.50:0/3015793718 wait complete. 2024-05-08T18:05:08.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc03551f800 1 -- 172.21.15.50:0/3316475235 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e6bfde2d40 msgr2=0x55e6bfde5130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc03551f800 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e6bfde2d40 0x55e6bfde5130 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc03551f800 1 -- 172.21.15.50:0/3316475235 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e6bfce2b70 msgr2=0x55e6bfde0560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc03551f800 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e6bfce2b70 0x55e6bfde0560 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc03551f800 1 -- 172.21.15.50:0/3316475235 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e6bfde6c30 msgr2=0x55e6bfde9020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc03551f800 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e6bfde6c30 0x55e6bfde9020 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc03551f800 1 -- 172.21.15.50:0/3316475235 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf1803bba0 msgr2=0x7fbf1803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc03551f800 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf1803bba0 0x7fbf1803e060 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc028000c00 tx=0x7fc0280013b0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc03551f800 1 -- 172.21.15.50:0/3316475235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce1880 msgr2=0x55e6bfdd93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc03551f800 1 --2- 172.21.15.50:0/3316475235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e6bfce1880 0x55e6bfdd93c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc024000c00 tx=0x7fc024002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc031a53640 1 -- 172.21.15.50:0/3316475235 reap_dead start 2024-05-08T18:05:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc03551f800 1 -- 172.21.15.50:0/3316475235 shutdown_connections 2024-05-08T18:05:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc03551f800 1 -- 172.21.15.50:0/3316475235 >> 172.21.15.50:0/3316475235 conn(0x55e6bfcdc2f0 msgr2=0x55e6bfdcba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc03551f800 1 -- 172.21.15.50:0/3316475235 shutdown_connections 2024-05-08T18:05:08.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.739+0000 7fc03551f800 1 -- 172.21.15.50:0/3316475235 wait complete. 2024-05-08T18:05:08.749 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-05-08T18:05:08.750 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-05-08T18:05:08.750 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-05-08T18:05:08.795 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:08.795 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:08.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.797+0000 7f9992c0f800 1 Processor -- start 2024-05-08T18:05:08.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.797+0000 7f9992c0f800 1 -- start start 2024-05-08T18:05:08.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.797+0000 7f9992c0f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55c56938eaf0 0x55c5693975f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.797+0000 7f9992c0f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55c569301ff0 con 0x55c56938cfb0 2024-05-08T18:05:08.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.797+0000 7f9992c0f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55c56922ee20 con 0x55c56938eaf0 2024-05-08T18:05:08.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.797+0000 7f998da0c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55c56938eaf0 0x55c5693975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.798+0000 7f998e20d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55c56938cfb0 0x55c56938d3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43214/0 (socket says 172.21.15.50:43214) 2024-05-08T18:05:08.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.798+0000 7f998e20d640 1 -- 172.21.15.50:0/1434704901 learned_addr learned my addr 172.21.15.50:0/1434704901 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:08.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.798+0000 7f998da0c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55c56938eaf0 0x55c5693975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38040/0 (socket says 172.21.15.50:38040) 2024-05-08T18:05:08.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.798+0000 7f998da0c640 1 -- 172.21.15.50:0/1434704901 >> v1:172.21.15.46:6789/0 conn(0x55c56938cfb0 legacy=0x55c56938d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.798+0000 7f998da0c640 1 -- 172.21.15.50:0/1434704901 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5693021c0 con 0x55c56938eaf0 2024-05-08T18:05:08.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.800+0000 7f998da0c640 1 --2- 172.21.15.50:0/1434704901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938eaf0 0x55c5693975f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f998400a470 tx=0x7f998401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea16a87342ce536 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.800+0000 7f998d20b640 1 -- 172.21.15.50:0/1434704901 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9984057070 con 0x55c56938eaf0 2024-05-08T18:05:08.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.800+0000 7f998d20b640 1 -- 172.21.15.50:0/1434704901 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9984052570 con 0x55c56938eaf0 2024-05-08T18:05:08.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.800+0000 7f998d20b640 1 -- 172.21.15.50:0/1434704901 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9984052870 con 0x55c56938eaf0 2024-05-08T18:05:08.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.800+0000 7f9992c0f800 1 -- 172.21.15.50:0/1434704901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938eaf0 msgr2=0x55c5693975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.800+0000 7f9992c0f800 1 --2- 172.21.15.50:0/1434704901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938eaf0 0x55c5693975f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f998400a470 tx=0x7f998401ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:08.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.800+0000 7f9992c0f800 1 -- 172.21.15.50:0/1434704901 shutdown_connections 2024-05-08T18:05:08.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.800+0000 7f9992c0f800 1 --2- 172.21.15.50:0/1434704901 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938eaf0 0x55c5693975f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.800+0000 7f9992c0f800 1 -- 172.21.15.50:0/1434704901 >> 172.21.15.50:0/1434704901 conn(0x55c569388ad0 msgr2=0x55c56938af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.800+0000 7f9992c0f800 1 -- 172.21.15.50:0/1434704901 shutdown_connections 2024-05-08T18:05:08.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.800+0000 7f9992c0f800 1 -- 172.21.15.50:0/1434704901 wait complete. 2024-05-08T18:05:08.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.815+0000 7f9992c0f800 1 Processor -- start 2024-05-08T18:05:08.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.816+0000 7f9992c0f800 1 -- start start 2024-05-08T18:05:08.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.816+0000 7f9992c0f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938cfb0 0x55c569481d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.816+0000 7f9992c0f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c5692d9a10 con 0x55c56938cfb0 2024-05-08T18:05:08.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.816+0000 7f998e20d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938cfb0 0x55c569481d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.816+0000 7f998e20d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938cfb0 0x55c569481d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38056/0 (socket says 172.21.15.50:38056) 2024-05-08T18:05:08.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.816+0000 7f998e20d640 1 -- 172.21.15.50:0/3523630732 learned_addr learned my addr 172.21.15.50:0/3523630732 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.816+0000 7f998e20d640 1 -- 172.21.15.50:0/3523630732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c569482320 con 0x55c56938cfb0 2024-05-08T18:05:08.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.816+0000 7f998e20d640 1 --2- 172.21.15.50:0/3523630732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938cfb0 0x55c569481d50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9980001e10 tx=0x7f998000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da56d3c28b792e2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.817+0000 7f987c608640 1 -- 172.21.15.50:0/3523630732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9980012020 con 0x55c56938cfb0 2024-05-08T18:05:08.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.817+0000 7f987c608640 1 -- 172.21.15.50:0/3523630732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f998000e5a0 con 0x55c56938cfb0 2024-05-08T18:05:08.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.817+0000 7f987c608640 1 -- 172.21.15.50:0/3523630732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f998000e8a0 con 0x55c56938cfb0 2024-05-08T18:05:08.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.817+0000 7f9992c0f800 1 -- 172.21.15.50:0/3523630732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938cfb0 msgr2=0x55c569481d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.817+0000 7f9992c0f800 1 --2- 172.21.15.50:0/3523630732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938cfb0 0x55c569481d50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9980001e10 tx=0x7f998000b780 comp rx=0 tx=0).stop 2024-05-08T18:05:08.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.817+0000 7f9992c0f800 1 -- 172.21.15.50:0/3523630732 shutdown_connections 2024-05-08T18:05:08.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.817+0000 7f9992c0f800 1 --2- 172.21.15.50:0/3523630732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938cfb0 0x55c569481d50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.817+0000 7f9992c0f800 1 -- 172.21.15.50:0/3523630732 >> 172.21.15.50:0/3523630732 conn(0x55c569388ad0 msgr2=0x55c56947d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.817+0000 7f9992c0f800 1 -- 172.21.15.50:0/3523630732 shutdown_connections 2024-05-08T18:05:08.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.817+0000 7f9992c0f800 1 -- 172.21.15.50:0/3523630732 wait complete. 2024-05-08T18:05:08.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.818+0000 7f9992c0f800 1 Processor -- start 2024-05-08T18:05:08.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.818+0000 7f9992c0f800 1 -- start start 2024-05-08T18:05:08.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.818+0000 7f9992c0f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938cfb0 0x55c5694898f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.818+0000 7f9992c0f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c569397e00 con 0x55c56938cfb0 2024-05-08T18:05:08.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.818+0000 7f998e20d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938cfb0 0x55c5694898f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.818+0000 7f998e20d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938cfb0 0x55c5694898f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38058/0 (socket says 172.21.15.50:38058) 2024-05-08T18:05:08.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.818+0000 7f998e20d640 1 -- 172.21.15.50:0/986759349 learned_addr learned my addr 172.21.15.50:0/986759349 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.819+0000 7f998e20d640 1 -- 172.21.15.50:0/986759349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c569489e30 con 0x55c56938cfb0 2024-05-08T18:05:08.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.819+0000 7f998e20d640 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938cfb0 0x55c5694898f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55c569482ae0 tx=0x7f9980000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.819+0000 7f987ae05640 1 -- 172.21.15.50:0/986759349 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9980012030 con 0x55c56938cfb0 2024-05-08T18:05:08.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.819+0000 7f987ae05640 1 -- 172.21.15.50:0/986759349 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9980001470 con 0x55c56938cfb0 2024-05-08T18:05:08.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.819+0000 7f9992c0f800 1 -- 172.21.15.50:0/986759349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c56948a0c0 con 0x55c56938cfb0 2024-05-08T18:05:08.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.819+0000 7f987ae05640 1 -- 172.21.15.50:0/986759349 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f998001a7a0 con 0x55c56938cfb0 2024-05-08T18:05:08.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.819+0000 7f9992c0f800 1 -- 172.21.15.50:0/986759349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c56948cd70 con 0x55c56938cfb0 2024-05-08T18:05:08.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.820+0000 7f987ae05640 1 -- 172.21.15.50:0/986759349 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9980015020 con 0x55c56938cfb0 2024-05-08T18:05:08.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.820+0000 7f987ae05640 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f987403bb50 0x7f987403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.820+0000 7f987ae05640 1 -- 172.21.15.50:0/986759349 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f9980020030 con 0x55c56938cfb0 2024-05-08T18:05:08.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.820+0000 7f998da0c640 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f987403bb50 0x7f987403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.820+0000 7f9992c0f800 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c56938eaf0 0x55c569490b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.820+0000 7f9992c0f800 1 -- 172.21.15.50:0/986759349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c569491070 con 0x55c56938eaf0 2024-05-08T18:05:08.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.820+0000 7f998ea0e640 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c56938eaf0 0x55c569490b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.821+0000 7f998da0c640 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f987403bb50 0x7f987403e010 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9984054000 tx=0x7f998404e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.821+0000 7f998ea0e640 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c56938eaf0 0x55c569490b30 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.821+0000 7f998ea0e640 1 -- 172.21.15.50:0/986759349 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f998800a4a0 con 0x55c56938eaf0 2024-05-08T18:05:08.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.821+0000 7f9992c0f800 1 -- 172.21.15.50:0/986759349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c56948e0e0 con 0x55c56938eaf0 2024-05-08T18:05:08.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.821+0000 7f998ea0e640 1 -- 172.21.15.50:0/986759349 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f998800a4a0 con 0x55c56938eaf0 2024-05-08T18:05:08.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.821+0000 7f9992c0f800 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c5694933f0 0x55c5694957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.821+0000 7f9992c0f800 1 -- 172.21.15.50:0/986759349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c569495e80 con 0x55c5694933f0 2024-05-08T18:05:08.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.822+0000 7f998e20d640 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c5694933f0 0x55c5694957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.822+0000 7f998e20d640 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c5694933f0 0x55c5694957e0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.822+0000 7f998e20d640 1 -- 172.21.15.50:0/986759349 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f9980018490 con 0x55c5694933f0 2024-05-08T18:05:08.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.822+0000 7f9992c0f800 1 -- 172.21.15.50:0/986759349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c569492290 con 0x55c56938eaf0 2024-05-08T18:05:08.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.823+0000 7f998ea0e640 1 -- 172.21.15.50:0/986759349 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f998800a4a0 con 0x55c56938eaf0 2024-05-08T18:05:08.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.823+0000 7f9992c0f800 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c5694972e0 0x55c5694996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.823+0000 7f9992c0f800 1 -- 172.21.15.50:0/986759349 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c569499dd0 con 0x55c5694972e0 2024-05-08T18:05:08.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.823+0000 7f998da0c640 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c5694972e0 0x55c5694996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.823+0000 7f998da0c640 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c5694972e0 0x55c5694996d0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.823+0000 7f998da0c640 1 -- 172.21.15.50:0/986759349 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9984052a40 con 0x55c5694972e0 2024-05-08T18:05:08.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.824+0000 7f9992c0f800 1 Processor -- start 2024-05-08T18:05:08.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.824+0000 7f9992c0f800 1 -- start start 2024-05-08T18:05:08.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.824+0000 7f9992c0f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694a8080 0x55c5694a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.824+0000 7f9992c0f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c569280330 con 0x55c5694a8080 2024-05-08T18:05:08.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.824+0000 7f998e20d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694a8080 0x55c5694a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.824+0000 7f998e20d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694a8080 0x55c5694a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38070/0 (socket says 172.21.15.50:38070) 2024-05-08T18:05:08.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.824+0000 7f998e20d640 1 -- 172.21.15.50:0/1065208596 learned_addr learned my addr 172.21.15.50:0/1065208596 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.825+0000 7f998e20d640 1 -- 172.21.15.50:0/1065208596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5694a89e0 con 0x55c5694a8080 2024-05-08T18:05:08.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.825+0000 7f998e20d640 1 --2- 172.21.15.50:0/1065208596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694a8080 0x55c5694a84a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9980018740 tx=0x7f998001af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.825+0000 7f9873fff640 1 -- 172.21.15.50:0/1065208596 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f998004f070 con 0x55c5694a8080 2024-05-08T18:05:08.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.825+0000 7f9873fff640 1 -- 172.21.15.50:0/1065208596 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9980047070 con 0x55c5694a8080 2024-05-08T18:05:08.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.825+0000 7f9992c0f800 1 -- 172.21.15.50:0/1065208596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c5694a9450 con 0x55c5694a8080 2024-05-08T18:05:08.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.825+0000 7f9873fff640 1 -- 172.21.15.50:0/1065208596 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9980016570 con 0x55c5694a8080 2024-05-08T18:05:08.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.825+0000 7f9992c0f800 1 -- 172.21.15.50:0/1065208596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5694a9950 con 0x55c5694a8080 2024-05-08T18:05:08.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.826+0000 7f9873fff640 1 -- 172.21.15.50:0/1065208596 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9980016710 con 0x55c5694a8080 2024-05-08T18:05:08.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.826+0000 7f9873fff640 1 --2- 172.21.15.50:0/1065208596 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f986803bb50 0x7f986803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.826+0000 7f9873fff640 1 -- 172.21.15.50:0/1065208596 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f998003a910 con 0x55c5694a8080 2024-05-08T18:05:08.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.826+0000 7f998da0c640 1 --2- 172.21.15.50:0/1065208596 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f986803bb50 0x7f986803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.826+0000 7f9992c0f800 1 Processor -- start 2024-05-08T18:05:08.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.826+0000 7f9992c0f800 1 -- start start 2024-05-08T18:05:08.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.826+0000 7f998da0c640 1 --2- 172.21.15.50:0/1065208596 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f986803bb50 0x7f986803e010 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55c56948e020 tx=0x7f9984005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.826+0000 7f9992c0f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694b5160 0x55c5694b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.826+0000 7f9992c0f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c5694826a0 con 0x55c5694b5160 2024-05-08T18:05:08.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.827+0000 7f998e20d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694b5160 0x55c5694b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.827+0000 7f998e20d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694b5160 0x55c5694b5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38074/0 (socket says 172.21.15.50:38074) 2024-05-08T18:05:08.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.827+0000 7f998e20d640 1 -- 172.21.15.50:0/4025003566 learned_addr learned my addr 172.21.15.50:0/4025003566 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.827+0000 7f998e20d640 1 -- 172.21.15.50:0/4025003566 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5694b5b50 con 0x55c5694b5160 2024-05-08T18:05:08.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.827+0000 7f998e20d640 1 --2- 172.21.15.50:0/4025003566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694b5160 0x55c5694b5580 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f998001a6f0 tx=0x7f9980049f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67d9902059e0f8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.827+0000 7f9871ffb640 1 -- 172.21.15.50:0/4025003566 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99800495b0 con 0x55c5694b5160 2024-05-08T18:05:08.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.827+0000 7f9871ffb640 1 -- 172.21.15.50:0/4025003566 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9980049bd0 con 0x55c5694b5160 2024-05-08T18:05:08.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.827+0000 7f9871ffb640 1 -- 172.21.15.50:0/4025003566 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f998004fb30 con 0x55c5694b5160 2024-05-08T18:05:08.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.828+0000 7f9992c0f800 1 -- 172.21.15.50:0/4025003566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694b5160 msgr2=0x55c5694b5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.828+0000 7f9992c0f800 1 --2- 172.21.15.50:0/4025003566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694b5160 0x55c5694b5580 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f998001a6f0 tx=0x7f9980049f00 comp rx=0 tx=0).stop 2024-05-08T18:05:08.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.828+0000 7f9992c0f800 1 -- 172.21.15.50:0/4025003566 shutdown_connections 2024-05-08T18:05:08.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.828+0000 7f9992c0f800 1 --2- 172.21.15.50:0/4025003566 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694b5160 0x55c5694b5580 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.828+0000 7f9992c0f800 1 -- 172.21.15.50:0/4025003566 >> 172.21.15.50:0/4025003566 conn(0x55c5694b1430 msgr2=0x55c5694af490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.828+0000 7f9992c0f800 1 -- 172.21.15.50:0/4025003566 shutdown_connections 2024-05-08T18:05:08.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.828+0000 7f9992c0f800 1 -- 172.21.15.50:0/4025003566 wait complete. 2024-05-08T18:05:08.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.828+0000 7f9992c0f800 1 Processor -- start 2024-05-08T18:05:08.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.828+0000 7f9992c0f800 1 -- start start 2024-05-08T18:05:08.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.828+0000 7f9992c0f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694b5160 0x55c5694bc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.828+0000 7f9992c0f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c5694b6220 con 0x55c5694b5160 2024-05-08T18:05:08.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.828+0000 7f998e20d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694b5160 0x55c5694bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.828+0000 7f998e20d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694b5160 0x55c5694bc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38078/0 (socket says 172.21.15.50:38078) 2024-05-08T18:05:08.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.828+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 learned_addr learned my addr 172.21.15.50:0/25617652 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.829+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5694bcac0 con 0x55c5694b5160 2024-05-08T18:05:08.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.829+0000 7f998e20d640 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694b5160 0x55c5694bc580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9980049930 tx=0x7f9980020f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.829+0000 7f985ffff640 1 -- 172.21.15.50:0/25617652 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99800491f0 con 0x55c5694b5160 2024-05-08T18:05:08.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.829+0000 7f985ffff640 1 -- 172.21.15.50:0/25617652 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f998003d170 con 0x55c5694b5160 2024-05-08T18:05:08.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.829+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c5694bcac0 con 0x55c5694b5160 2024-05-08T18:05:08.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.829+0000 7f985ffff640 1 -- 172.21.15.50:0/25617652 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99800493d0 con 0x55c5694b5160 2024-05-08T18:05:08.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.829+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5694bcac0 con 0x55c5694b5160 2024-05-08T18:05:08.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.830+0000 7f985ffff640 1 -- 172.21.15.50:0/25617652 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9980049fd0 con 0x55c5694b5160 2024-05-08T18:05:08.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.830+0000 7f985ffff640 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f985803bba0 0x7f985803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.830+0000 7f985ffff640 1 -- 172.21.15.50:0/25617652 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f9980065070 con 0x55c5694b5160 2024-05-08T18:05:08.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.830+0000 7f998da0c640 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f985803bba0 0x7f985803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.831+0000 7f998da0c640 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f985803bba0 0x7f985803e060 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9984052fd0 tx=0x7f998404f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.831+0000 7f9992c0f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.831+0000 7f9992c0f800 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c5694c7c60 0x55c5694ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.831+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c5694ca870 con 0x55c5694c7c60 2024-05-08T18:05:08.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.831+0000 7f998ea0e640 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c5694c7c60 0x55c5694ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.831+0000 7f998ea0e640 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c5694c7c60 0x55c5694ca120 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f998800a4a0 con 0x55c5694c7c60 2024-05-08T18:05:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f9992c0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f9992c0f800 20 realm 2024-05-08T18:05:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f9992c0f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cac80 con 0x55c5694c7c60 2024-05-08T18:05:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f998800a4a0 con 0x55c5694c7c60 2024-05-08T18:05:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f9992c0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f9992c0f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f9992c0f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cb550 con 0x55c5694c7c60 2024-05-08T18:05:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f998800a4a0 con 0x55c5694c7c60 2024-05-08T18:05:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f9992c0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f9992c0f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cb8f0 con 0x55c5694c7c60 2024-05-08T18:05:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f998800a4a0 con 0x55c5694c7c60 2024-05-08T18:05:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f9992c0f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f9992c0f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f9992c0f800 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c5694cc650 0x55c5694cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.832+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cf050 con 0x55c5694cc650 2024-05-08T18:05:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.833+0000 7f998e20d640 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c5694cc650 0x55c5694cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.833+0000 7f998e20d640 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c5694cc650 0x55c5694cea40 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.833+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.833+0000 7f9992c0f800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.833+0000 7f9992c0f800 20 searching for the correct realm 2024-05-08T18:05:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.833+0000 7f9992c0f800 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c5694d1000 0x55c5694d34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.833+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d3b90 con 0x55c5694d1000 2024-05-08T18:05:08.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.834+0000 7f998da0c640 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c5694d1000 0x55c5694d34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.834+0000 7f998da0c640 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c5694d1000 0x55c5694d34c0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.834+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.834+0000 7f998da0c640 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9984076040 0x7f9984083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.834+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d3b90 con 0x7f9984076040 2024-05-08T18:05:08.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.834+0000 7f998ea0e640 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9984076040 0x7f9984083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.834+0000 7f998ea0e640 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9984076040 0x7f9984083410 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.835+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f998800a4a0 con 0x7f9984076040 2024-05-08T18:05:08.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.835+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cb8f0 con 0x55c5694cc650 2024-05-08T18:05:08.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.835+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.835+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cb8f0 con 0x55c5694c7c60 2024-05-08T18:05:08.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.835+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f998800a4a0 con 0x55c5694c7c60 2024-05-08T18:05:08.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.835+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cb8f0 con 0x55c5694c7c60 2024-05-08T18:05:08.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.836+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f998800a4a0 con 0x55c5694c7c60 2024-05-08T18:05:08.849 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.836+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cb8f0 con 0x55c5694cc650 2024-05-08T18:05:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.836+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.836+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cb8f0 con 0x55c5694cc650 2024-05-08T18:05:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.836+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.836+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cb8f0 con 0x7f9984076040 2024-05-08T18:05:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.836+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f998800a4a0 con 0x7f9984076040 2024-05-08T18:05:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.836+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cb8f0 con 0x55c5694d1000 2024-05-08T18:05:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.836+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.837+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cb8f0 con 0x55c5694c7c60 2024-05-08T18:05:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.837+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f998800a4a0 con 0x55c5694c7c60 2024-05-08T18:05:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.837+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cb8f0 con 0x55c5694cc650 2024-05-08T18:05:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.837+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.837+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cb8f0 con 0x55c5694c7c60 2024-05-08T18:05:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.837+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f998800a4a0 con 0x55c5694c7c60 2024-05-08T18:05:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.837+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cb8f0 con 0x7f9984076040 2024-05-08T18:05:08.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.837+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f998800a4a0 con 0x7f9984076040 2024-05-08T18:05:08.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.837+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cb8f0 con 0x55c5694d1000 2024-05-08T18:05:08.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.838+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.838+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x55c5694d1000 2024-05-08T18:05:08.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.838+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.838+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x55c5694cc650 2024-05-08T18:05:08.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.838+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.838+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x55c5694d1000 2024-05-08T18:05:08.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.838+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.838+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x55c5694d1000 2024-05-08T18:05:08.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.839+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.839+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x55c5694c7c60 2024-05-08T18:05:08.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.839+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f998800a4a0 con 0x55c5694c7c60 2024-05-08T18:05:08.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.839+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x55c5694c7c60 2024-05-08T18:05:08.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.839+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f998800a4a0 con 0x55c5694c7c60 2024-05-08T18:05:08.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.839+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x55c5694cc650 2024-05-08T18:05:08.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.839+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.839+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x55c5694d1000 2024-05-08T18:05:08.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.840+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.840+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x7f9984076040 2024-05-08T18:05:08.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.840+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f998800a4a0 con 0x7f9984076040 2024-05-08T18:05:08.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.840+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x55c5694c7c60 2024-05-08T18:05:08.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.840+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f998800a4a0 con 0x55c5694c7c60 2024-05-08T18:05:08.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.840+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x55c5694cc650 2024-05-08T18:05:08.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.840+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.840+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x55c5694d1000 2024-05-08T18:05:08.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.841+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.841+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x55c5694cc650 2024-05-08T18:05:08.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.841+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.841+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x7f9984076040 2024-05-08T18:05:08.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.841+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f998800a4a0 con 0x7f9984076040 2024-05-08T18:05:08.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.841+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x55c5694d1000 2024-05-08T18:05:08.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.841+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.841+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x55c5694cc650 2024-05-08T18:05:08.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x55c5694c7c60 2024-05-08T18:05:08.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f998800a4a0 con 0x55c5694c7c60 2024-05-08T18:05:08.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f9984090040 con 0x7f9984076040 2024-05-08T18:05:08.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f998800a4a0 con 0x7f9984076040 2024-05-08T18:05:08.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f9992c0f800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:08.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f9992c0f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:08.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f9992c0f800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:08.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f9992c0f800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:08.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f9992c0f800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:08.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f9992c0f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:08.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f9992c0f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cf910 con 0x55c5694c7c60 2024-05-08T18:05:08.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f998800a4a0 con 0x55c5694c7c60 2024-05-08T18:05:08.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f9992c0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f9992c0f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.842+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cf910 con 0x55c5694c7c60 2024-05-08T18:05:08.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.843+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f998800a4a0 con 0x55c5694c7c60 2024-05-08T18:05:08.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.843+0000 7f9992c0f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:08.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.843+0000 7f9992c0f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.843+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cf910 con 0x55c5694d1000 2024-05-08T18:05:08.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.843+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.843+0000 7f9992c0f800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:08.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.843+0000 7f9992c0f800 20 zone default found 2024-05-08T18:05:08.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.843+0000 7f9992c0f800 4 Realm: () 2024-05-08T18:05:08.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.843+0000 7f9992c0f800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:08.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.843+0000 7f9992c0f800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:08.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.843+0000 7f9992c0f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:08.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.843+0000 7f9992c0f800 20 zonegroup default 2024-05-08T18:05:08.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.843+0000 7f9992c0f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.843+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d02a0 con 0x55c5694cc650 2024-05-08T18:05:08.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.844+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.844+0000 7f9992c0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.844+0000 7f9992c0f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.844+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c5694cfe50 con 0x55c5694d1000 2024-05-08T18:05:08.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.844+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.844+0000 7f9992c0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.844+0000 7f9992c0f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.844+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d5f90 con 0x55c5694d1000 2024-05-08T18:05:08.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.844+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.844+0000 7f9992c0f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:08.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.844+0000 7f9992c0f800 20 started sync module instance, tier type = 2024-05-08T18:05:08.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.844+0000 7f9992c0f800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:08.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.844+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d5330 con 0x7f9984076040 2024-05-08T18:05:08.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.845+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f998800a4a0 con 0x7f9984076040 2024-05-08T18:05:08.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.845+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94306363594800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d5330 con 0x7f9984076040 2024-05-08T18:05:08.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.845+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d6a10 con 0x55c5694c7c60 2024-05-08T18:05:08.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.846+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94306363594800] v27'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9988040040 con 0x7f9984076040 2024-05-08T18:05:08.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.846+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9988040040 con 0x55c5694c7c60 2024-05-08T18:05:08.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.846+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94306363592800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d6a10 con 0x55c5694c7c60 2024-05-08T18:05:08.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.846+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d7430 con 0x55c5694d1000 2024-05-08T18:05:08.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.848+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94306363592800] v27'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9988040040 con 0x55c5694c7c60 2024-05-08T18:05:08.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.848+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.848+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94306363598176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d6980 con 0x55c5694d1000 2024-05-08T18:05:08.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.848+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d82f0 con 0x7f9984076040 2024-05-08T18:05:08.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.849+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9988040040 con 0x7f9984076040 2024-05-08T18:05:08.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.849+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94306363605024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d8a70 con 0x7f9984076040 2024-05-08T18:05:08.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.849+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d9570 con 0x55c5694c7c60 2024-05-08T18:05:08.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.849+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94306363598176] v27'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.850+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9988040040 con 0x55c5694c7c60 2024-05-08T18:05:08.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.850+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94306363609184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d9530 con 0x55c5694c7c60 2024-05-08T18:05:08.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.850+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d9fb0 con 0x55c5694cc650 2024-05-08T18:05:08.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.850+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94306363605024] v27'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9988040040 con 0x7f9984076040 2024-05-08T18:05:08.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.851+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.851+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94306363611904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694da3a0 con 0x55c5694cc650 2024-05-08T18:05:08.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.851+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694dad60 con 0x55c5694cc650 2024-05-08T18:05:08.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.851+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94306363609184] v27'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9988040040 con 0x55c5694c7c60 2024-05-08T18:05:08.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.852+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.852+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94306363615408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694dacd0 con 0x55c5694cc650 2024-05-08T18:05:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.852+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694dbc20 con 0x55c5694cc650 2024-05-08T18:05:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.852+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94306363611904] v27'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.853+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94306363615408] v27'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.853+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.853+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94306363617616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694db770 con 0x55c5694cc650 2024-05-08T18:05:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.853+0000 7f9992c0f800 20 add_watcher() i=0 2024-05-08T18:05:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.853+0000 7f9992c0f800 20 add_watcher() i=1 2024-05-08T18:05:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.853+0000 7f9992c0f800 20 add_watcher() i=2 2024-05-08T18:05:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.853+0000 7f9992c0f800 20 add_watcher() i=3 2024-05-08T18:05:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.853+0000 7f9992c0f800 20 add_watcher() i=4 2024-05-08T18:05:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.853+0000 7f9992c0f800 20 add_watcher() i=5 2024-05-08T18:05:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.853+0000 7f9992c0f800 20 add_watcher() i=6 2024-05-08T18:05:08.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.854+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94306363617616] v27'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.854+0000 7f9992c0f800 20 add_watcher() i=7 2024-05-08T18:05:08.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.854+0000 7f9992c0f800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:08.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.854+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c5694dec50 con 0x55c5694c7c60 2024-05-08T18:05:08.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.854+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9988040040 con 0x55c5694c7c60 2024-05-08T18:05:08.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.854+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94306363619184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694dc040 con 0x55c5694c7c60 2024-05-08T18:05:08.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.855+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94306363619184] v27'121 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9988040040 con 0x55c5694c7c60 2024-05-08T18:05:08.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.855+0000 7f9844fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:08.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.855+0000 7f9992c0f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:08.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.855+0000 7f9992c0f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:08.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.855+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e7390 con 0x55c5694c7c60 2024-05-08T18:05:08.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.856+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'21 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9988040040 con 0x55c5694c7c60 2024-05-08T18:05:08.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.857+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e7a60 con 0x55c5694cc650 2024-05-08T18:05:08.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.858+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.858+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e7a60 con 0x55c5694d1000 2024-05-08T18:05:08.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.858+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.859+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e8070 con 0x55c5694c7c60 2024-05-08T18:05:08.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.859+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'122 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9988040040 con 0x55c5694c7c60 2024-05-08T18:05:08.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.859+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e8520 con 0x7f9984076040 2024-05-08T18:05:08.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.860+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'35 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9988040040 con 0x7f9984076040 2024-05-08T18:05:08.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.860+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e51f0 con 0x55c5694d1000 2024-05-08T18:05:08.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.861+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'23 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.861+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e51f0 con 0x55c5694c7c60 2024-05-08T18:05:08.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.863+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9988040040 con 0x55c5694c7c60 2024-05-08T18:05:08.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.863+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e92c0 con 0x55c5694cc650 2024-05-08T18:05:08.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.864+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'23 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.864+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e92c0 con 0x7f9984076040 2024-05-08T18:05:08.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.865+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9988040040 con 0x7f9984076040 2024-05-08T18:05:08.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.865+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e92c0 con 0x55c5694d1000 2024-05-08T18:05:08.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.865+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'21 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.866+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e9700 con 0x7f9984076040 2024-05-08T18:05:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.866+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'36 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9988040040 con 0x7f9984076040 2024-05-08T18:05:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.867+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e9700 con 0x55c5694cc650 2024-05-08T18:05:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.868+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'17 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.868+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e9700 con 0x7f9984076040 2024-05-08T18:05:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.869+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9988040040 con 0x7f9984076040 2024-05-08T18:05:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.869+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e9700 con 0x55c5694cc650 2024-05-08T18:05:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.870+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'29 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.870+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e9790 con 0x55c5694c7c60 2024-05-08T18:05:08.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.871+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'123 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9988040040 con 0x55c5694c7c60 2024-05-08T18:05:08.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.871+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e9790 con 0x55c5694d1000 2024-05-08T18:05:08.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.872+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'30 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.872+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e9820 con 0x7f9984076040 2024-05-08T18:05:08.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.873+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'37 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9988040040 con 0x7f9984076040 2024-05-08T18:05:08.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.873+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e9820 con 0x55c5694d1000 2024-05-08T18:05:08.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.874+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'15 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.874+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e99f0 con 0x55c5694c7c60 2024-05-08T18:05:08.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.875+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'23 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9988040040 con 0x55c5694c7c60 2024-05-08T18:05:08.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.875+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e99f0 con 0x55c5694c7c60 2024-05-08T18:05:08.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.876+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'124 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9988040040 con 0x55c5694c7c60 2024-05-08T18:05:08.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.876+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e99f0 con 0x55c5694d1000 2024-05-08T18:05:08.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.877+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'31 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.877+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e99f0 con 0x7f9984076040 2024-05-08T18:05:08.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.878+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'25 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9988040040 con 0x7f9984076040 2024-05-08T18:05:08.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.878+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e99f0 con 0x55c5694d1000 2024-05-08T18:05:08.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.879+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.879+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e99f0 con 0x55c5694d1000 2024-05-08T18:05:08.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.880+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.880+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e99f0 con 0x55c5694d1000 2024-05-08T18:05:08.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.881+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'11 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.881+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e99f0 con 0x55c5694d1000 2024-05-08T18:05:08.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.882+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'24 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.882+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e99f0 con 0x55c5694cc650 2024-05-08T18:05:08.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.883+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.883+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e99f0 con 0x55c5694d1000 2024-05-08T18:05:08.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.884+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'15 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.884+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e9960 con 0x55c5694cc650 2024-05-08T18:05:08.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.885+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'24 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.885+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e9960 con 0x55c5694d1000 2024-05-08T18:05:08.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.886+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'32 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.886+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e9960 con 0x55c5694cc650 2024-05-08T18:05:08.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.887+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'30 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.887+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694e9f10 con 0x7f9984076040 2024-05-08T18:05:08.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.888+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9988040040 con 0x7f9984076040 2024-05-08T18:05:08.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.888+0000 7f97bf7fe640 20 reqs_thread_entry: start 2024-05-08T18:05:08.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.888+0000 7f9992c0f800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:08.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.888+0000 7f9992c0f800 20 Filter name: none 2024-05-08T18:05:08.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.888+0000 7f97bd7fa640 20 reqs_thread_entry: start 2024-05-08T18:05:08.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.888+0000 7f9992c0f800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-05-08T18:05:08.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.888+0000 7f9992c0f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:08.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.888+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c569514950 con 0x55c5694cc650 2024-05-08T18:05:08.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.889+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(94 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.889+0000 7f9992c0f800 20 rados_obj.operate() r=0 bl.length=380 2024-05-08T18:05:08.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.889+0000 7f9992c0f800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-05-08T18:05:08.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.889+0000 7f9992c0f800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-05-08T18:05:08.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.889+0000 7f9992c0f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:08.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.889+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:95 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c5695173a0 con 0x55c5694cc650 2024-05-08T18:05:08.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.889+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(95 fud.buckets [call out=15b] v0'0 uv2 ondisk = 0) ==== 155+0+15 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.889+0000 7f9992c0f800 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-05-08T18:05:08.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.889+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:96 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c569515380 con 0x55c5694cc650 2024-05-08T18:05:08.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.890+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(96 p5YnriCv1nAtykxBrupQ [delete] v27'4 uv4 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.890+0000 7f9992c0f800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-05-08T18:05:08.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.890+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [notify cookie 94306363858576 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c569517a30 con 0x55c5694d1000 2024-05-08T18:05:08.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.891+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== watch-notify(notify (1) cookie 94306363598176 notify 115964116995 ret 0) ==== 212+0+0 (crc 0 0 0) 0x55c5694e9080 con 0x55c5694d1000 2024-05-08T18:05:08.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.891+0000 7f9871ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116995 cookie 94306363598176 notifier 4522 bl.length()=170 2024-05-08T18:05:08.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.891+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 28 ==== osd_op_reply(97 notify.2 [notify cookie 94306363858576 out=8b] v0'0 uv126 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.891+0000 7f9871ffb640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f9854003d60 con 0x55c5694d1000 2024-05-08T18:05:08.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.891+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 29 ==== osd_op_reply(98 notify.2 [notify-ack] v0'0 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.891+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 30 ==== watch-notify(notify_complete (2) cookie 94306363858576 notify 115964116995 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55c5694e5590 con 0x55c5694d1000 2024-05-08T18:05:08.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.891+0000 7f9992c0f800 10 removing email index: 2024-05-08T18:05:08.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.891+0000 7f9992c0f800 10 removing user buckets index 2024-05-08T18:05:08.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.891+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:99 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c569517990 con 0x55c5694cc650 2024-05-08T18:05:08.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.892+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(99 fud.buckets [delete] v27'4 uv4 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.892+0000 7f9992c0f800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2024-05-08T18:05:08.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.892+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:100 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94306363865488 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c569516ea0 con 0x55c5694cc650 2024-05-08T18:05:08.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.892+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== watch-notify(notify (1) cookie 94306363615408 notify 115964117001 ret 0) ==== 202+0+0 (crc 0 0 0) 0x55c5694e92c0 con 0x55c5694cc650 2024-05-08T18:05:08.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.892+0000 7f9871ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117001 cookie 94306363615408 notifier 4522 bl.length()=160 2024-05-08T18:05:08.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.892+0000 7f9871ffb640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f9854003d60 con 0x55c5694cc650 2024-05-08T18:05:08.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.892+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== osd_op_reply(100 notify.6 [notify cookie 94306363865488 out=8b] v0'0 uv126 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.893+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 31 ==== watch-notify(notify_complete (2) cookie 94306363865488 notify 115964117001 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55c5694a89e0 con 0x55c5694cc650 2024-05-08T18:05:08.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.893+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 32 ==== osd_op_reply(101 notify.6 [notify-ack] v0'0 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.893+0000 7f9992c0f800 10 removing user index: fud 2024-05-08T18:05:08.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.893+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c569517990 con 0x55c5694cc650 2024-05-08T18:05:08.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.894+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 33 ==== osd_op_reply(102 fud [call,call,delete] v27'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.894+0000 7f9992c0f800 10 removing default.rgw.meta+users.uid+fud from cache 2024-05-08T18:05:08.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.894+0000 7f9992c0f800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-05-08T18:05:08.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.894+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:103 7.3 7:c609908c:::notify.5:head [notify cookie 94306363865488 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55c569515e90 con 0x55c5694cc650 2024-05-08T18:05:08.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.894+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 34 ==== watch-notify(notify (1) cookie 94306363611904 notify 115964117002 ret 0) ==== 194+0+0 (crc 0 0 0) 0x55c5694a89e0 con 0x55c5694cc650 2024-05-08T18:05:08.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.894+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 35 ==== osd_op_reply(103 notify.5 [notify cookie 94306363865488 out=8b] v0'0 uv126 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.894+0000 7f98717fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117002 cookie 94306363611904 notifier 4522 bl.length()=152 2024-05-08T18:05:08.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.894+0000 7f98717fa640 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:104 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f97c40032c0 con 0x55c5694cc650 2024-05-08T18:05:08.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.895+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 36 ==== osd_op_reply(104 notify.5 [notify-ack] v0'0 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.895+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 37 ==== watch-notify(notify_complete (2) cookie 94306363865488 notify 115964117002 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55c56948cd70 con 0x55c5694cc650 2024-05-08T18:05:08.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.896+0000 7f9844fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:08.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.896+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:105 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94306363619184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c569517990 con 0x55c5694c7c60 2024-05-08T18:05:08.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.897+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(105 data_loggenerations_metadata [watch unwatch cookie 94306363619184] v27'125 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9988040040 con 0x55c5694c7c60 2024-05-08T18:05:08.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.897+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:106 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94306363594800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c569517990 con 0x7f9984076040 2024-05-08T18:05:08.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.898+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(106 notify.0 [watch unwatch cookie 94306363594800] v27'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9988040040 con 0x7f9984076040 2024-05-08T18:05:08.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.898+0000 7f9992c0f800 20 remove_watcher() i=0 2024-05-08T18:05:08.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.898+0000 7f9992c0f800 2 removed watcher, disabling cache 2024-05-08T18:05:08.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.898+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:107 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94306363592800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d5830 con 0x55c5694c7c60 2024-05-08T18:05:08.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.899+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(107 notify.1 [watch unwatch cookie 94306363592800] v27'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9988040040 con 0x55c5694c7c60 2024-05-08T18:05:08.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.899+0000 7f9992c0f800 20 remove_watcher() i=1 2024-05-08T18:05:08.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.899+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94306363598176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d5060 con 0x55c5694d1000 2024-05-08T18:05:08.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.900+0000 7f998da0c640 1 -- 172.21.15.50:0/25617652 <== osd.3 v2:172.21.15.50:6800/3130492744 31 ==== osd_op_reply(108 notify.2 [watch unwatch cookie 94306363598176] v27'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9984081070 con 0x55c5694d1000 2024-05-08T18:05:08.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.900+0000 7f9992c0f800 20 remove_watcher() i=2 2024-05-08T18:05:08.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.900+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94306363605024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d6560 con 0x7f9984076040 2024-05-08T18:05:08.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.901+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(109 notify.3 [watch unwatch cookie 94306363605024] v27'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9988040040 con 0x7f9984076040 2024-05-08T18:05:08.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.901+0000 7f9992c0f800 20 remove_watcher() i=3 2024-05-08T18:05:08.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.901+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:110 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94306363609184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d6560 con 0x55c5694c7c60 2024-05-08T18:05:08.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.902+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(110 notify.4 [watch unwatch cookie 94306363609184] v27'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9988040040 con 0x55c5694c7c60 2024-05-08T18:05:08.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.902+0000 7f9992c0f800 20 remove_watcher() i=4 2024-05-08T18:05:08.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.902+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:111 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94306363611904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d9060 con 0x55c5694cc650 2024-05-08T18:05:08.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.903+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 38 ==== osd_op_reply(111 notify.5 [watch unwatch cookie 94306363611904] v27'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.903+0000 7f9992c0f800 20 remove_watcher() i=5 2024-05-08T18:05:08.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.903+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94306363615408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d6560 con 0x55c5694cc650 2024-05-08T18:05:08.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.904+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 39 ==== osd_op_reply(112 notify.6 [watch unwatch cookie 94306363615408] v27'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.904+0000 7f9992c0f800 20 remove_watcher() i=6 2024-05-08T18:05:08.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.904+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:113 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94306363617616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55c5694d9b00 con 0x55c5694cc650 2024-05-08T18:05:08.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f998e20d640 1 -- 172.21.15.50:0/25617652 <== osd.0 v2:172.21.15.46:6800/337143975 40 ==== osd_op_reply(113 notify.7 [watch unwatch cookie 94306363617616] v27'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9980018490 con 0x55c5694cc650 2024-05-08T18:05:08.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 20 remove_watcher() i=7 2024-05-08T18:05:08.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c5694cc650 msgr2=0x55c5694cea40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c5694cc650 0x55c5694cea40 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c5694c7c60 msgr2=0x55c5694ca120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c5694c7c60 0x55c5694ca120 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9984076040 msgr2=0x7f9984083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9984076040 0x7f9984083410 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c5694d1000 msgr2=0x55c5694d34c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c5694d1000 0x55c5694d34c0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f985803bba0 msgr2=0x7f985803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f985803bba0 0x7f985803e060 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9984052fd0 tx=0x7f998404f000 comp rx=0 tx=0).stop 2024-05-08T18:05:08.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f998ea0e640 1 -- 172.21.15.50:0/25617652 reap_dead start 2024-05-08T18:05:08.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694b5160 msgr2=0x55c5694bc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694b5160 0x55c5694bc580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9980049930 tx=0x7f9980020f20 comp rx=0 tx=0).stop 2024-05-08T18:05:08.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 shutdown_connections 2024-05-08T18:05:08.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 --2- 172.21.15.50:0/25617652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694b5160 0x55c5694bc580 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 >> 172.21.15.50:0/25617652 conn(0x55c5694b1430 msgr2=0x55c5694aea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 shutdown_connections 2024-05-08T18:05:08.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.905+0000 7f9992c0f800 1 -- 172.21.15.50:0/25617652 wait complete. 2024-05-08T18:05:08.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 -- 172.21.15.50:0/1065208596 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f986803bb50 msgr2=0x7f986803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 --2- 172.21.15.50:0/1065208596 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f986803bb50 0x7f986803e010 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55c56948e020 tx=0x7f9984005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:08.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 -- 172.21.15.50:0/1065208596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694a8080 msgr2=0x55c5694a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 --2- 172.21.15.50:0/1065208596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694a8080 0x55c5694a84a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9980018740 tx=0x7f998001af10 comp rx=0 tx=0).stop 2024-05-08T18:05:08.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 -- 172.21.15.50:0/1065208596 shutdown_connections 2024-05-08T18:05:08.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 --2- 172.21.15.50:0/1065208596 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f986803bb50 0x7f986803e010 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 --2- 172.21.15.50:0/1065208596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c5694a8080 0x55c5694a84a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 -- 172.21.15.50:0/1065208596 >> 172.21.15.50:0/1065208596 conn(0x55c56949c120 msgr2=0x55c56949c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 -- 172.21.15.50:0/1065208596 shutdown_connections 2024-05-08T18:05:08.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 -- 172.21.15.50:0/1065208596 wait complete. 2024-05-08T18:05:08.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 -- 172.21.15.50:0/986759349 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c5694933f0 msgr2=0x55c5694957e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55c5694933f0 0x55c5694957e0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 -- 172.21.15.50:0/986759349 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c56938eaf0 msgr2=0x55c569490b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55c56938eaf0 0x55c569490b30 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 -- 172.21.15.50:0/986759349 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c5694972e0 msgr2=0x55c5694996d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55c5694972e0 0x55c5694996d0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 -- 172.21.15.50:0/986759349 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f987403bb50 msgr2=0x7f987403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f987403bb50 0x7f987403e010 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9984054000 tx=0x7f998404e000 comp rx=0 tx=0).stop 2024-05-08T18:05:08.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 -- 172.21.15.50:0/986759349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938cfb0 msgr2=0x55c5694898f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 --2- 172.21.15.50:0/986759349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c56938cfb0 0x55c5694898f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55c569482ae0 tx=0x7f9980000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f998ea0e640 1 -- 172.21.15.50:0/986759349 reap_dead start 2024-05-08T18:05:08.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 -- 172.21.15.50:0/986759349 shutdown_connections 2024-05-08T18:05:08.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 -- 172.21.15.50:0/986759349 >> 172.21.15.50:0/986759349 conn(0x55c569388ad0 msgr2=0x55c56947be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.906+0000 7f9992c0f800 1 -- 172.21.15.50:0/986759349 shutdown_connections 2024-05-08T18:05:08.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.907+0000 7f9992c0f800 1 -- 172.21.15.50:0/986759349 wait complete. 2024-05-08T18:05:08.915 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-05-08T18:05:08.915 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-05-08T18:05:08.915 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --tenant testx --uid tenanteduser --display-name tenanted-user --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2024-05-08T18:05:08.971 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:08.971 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:08.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.973+0000 7f3f35319800 1 Processor -- start 2024-05-08T18:05:08.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.973+0000 7f3f35319800 1 -- start start 2024-05-08T18:05:08.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.973+0000 7f3f35319800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56369bd15df0 0x56369bdaa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.973+0000 7f3f35319800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56369bd162a0 con 0x56369bd17130 2024-05-08T18:05:08.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.973+0000 7f3f35319800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x56369bc41e20 con 0x56369bd15df0 2024-05-08T18:05:08.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.973+0000 7f3f32610640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56369bd15df0 0x56369bdaa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.973+0000 7f3f32e11640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x56369bd17130 0x56369bd156e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43228/0 (socket says 172.21.15.50:43228) 2024-05-08T18:05:08.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.973+0000 7f3f32e11640 1 -- 172.21.15.50:0/2727583023 learned_addr learned my addr 172.21.15.50:0/2727583023 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:08.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.974+0000 7f3f32610640 1 -- 172.21.15.50:0/2727583023 >> v1:172.21.15.46:6789/0 conn(0x56369bd17130 legacy=0x56369bd156e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.974+0000 7f3f32610640 1 -- 172.21.15.50:0/2727583023 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56369bd164a0 con 0x56369bd15df0 2024-05-08T18:05:08.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.974+0000 7f3f31e0f640 1 -- 172.21.15.50:0/2727583023 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1907877814 0 0) 0x56369bd162a0 con 0x56369bd17130 2024-05-08T18:05:08.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.976+0000 7f3f32610640 1 --2- 172.21.15.50:0/2727583023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 0x56369bdaa5a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800a470 tx=0x7f3f2801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f07466271208a84 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.976+0000 7f3f31e0f640 1 -- 172.21.15.50:0/2727583023 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f28057070 con 0x56369bd15df0 2024-05-08T18:05:08.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.976+0000 7f3f31e0f640 1 -- 172.21.15.50:0/2727583023 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f28052570 con 0x56369bd15df0 2024-05-08T18:05:08.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.976+0000 7f3f31e0f640 1 -- 172.21.15.50:0/2727583023 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f28052870 con 0x56369bd15df0 2024-05-08T18:05:08.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.976+0000 7f3f35319800 1 -- 172.21.15.50:0/2727583023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 msgr2=0x56369bdaa5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.976+0000 7f3f35319800 1 --2- 172.21.15.50:0/2727583023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 0x56369bdaa5a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3f2800a470 tx=0x7f3f2801ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:08.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.976+0000 7f3f35319800 1 -- 172.21.15.50:0/2727583023 shutdown_connections 2024-05-08T18:05:08.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.976+0000 7f3f35319800 1 --2- 172.21.15.50:0/2727583023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 0x56369bdaa5a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.976+0000 7f3f35319800 1 -- 172.21.15.50:0/2727583023 >> 172.21.15.50:0/2727583023 conn(0x56369bd9cce0 msgr2=0x56369bd9f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.976+0000 7f3f35319800 1 -- 172.21.15.50:0/2727583023 shutdown_connections 2024-05-08T18:05:08.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.976+0000 7f3f35319800 1 -- 172.21.15.50:0/2727583023 wait complete. 2024-05-08T18:05:08.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.992+0000 7f3f35319800 1 Processor -- start 2024-05-08T18:05:08.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.992+0000 7f3f35319800 1 -- start start 2024-05-08T18:05:08.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.992+0000 7f3f35319800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 0x56369be94c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.992+0000 7f3f35319800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56369bcecb20 con 0x56369bd15df0 2024-05-08T18:05:08.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.992+0000 7f3f32e11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 0x56369be94c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.992+0000 7f3f32e11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 0x56369be94c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38104/0 (socket says 172.21.15.50:38104) 2024-05-08T18:05:08.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.992+0000 7f3f32e11640 1 -- 172.21.15.50:0/3427296258 learned_addr learned my addr 172.21.15.50:0/3427296258 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.992+0000 7f3f32e11640 1 -- 172.21.15.50:0/3427296258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56369be95250 con 0x56369bd15df0 2024-05-08T18:05:08.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.993+0000 7f3f32e11640 1 --2- 172.21.15.50:0/3427296258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 0x56369be94c80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3f24001db0 tx=0x7f3f2400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0cc917e69c3705 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.993+0000 7f3e20e09640 1 -- 172.21.15.50:0/3427296258 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f24012020 con 0x56369bd15df0 2024-05-08T18:05:08.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.993+0000 7f3e20e09640 1 -- 172.21.15.50:0/3427296258 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f2400e5c0 con 0x56369bd15df0 2024-05-08T18:05:08.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.993+0000 7f3e20e09640 1 -- 172.21.15.50:0/3427296258 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f2400e8e0 con 0x56369bd15df0 2024-05-08T18:05:08.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.994+0000 7f3f35319800 1 -- 172.21.15.50:0/3427296258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 msgr2=0x56369be94c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:08.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.994+0000 7f3f35319800 1 --2- 172.21.15.50:0/3427296258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 0x56369be94c80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3f24001db0 tx=0x7f3f2400b6c0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.994+0000 7f3f35319800 1 -- 172.21.15.50:0/3427296258 shutdown_connections 2024-05-08T18:05:08.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.994+0000 7f3f35319800 1 --2- 172.21.15.50:0/3427296258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 0x56369be94c80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:08.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.994+0000 7f3f35319800 1 -- 172.21.15.50:0/3427296258 >> 172.21.15.50:0/3427296258 conn(0x56369bd9cce0 msgr2=0x56369be905a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:08.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.994+0000 7f3f35319800 1 -- 172.21.15.50:0/3427296258 shutdown_connections 2024-05-08T18:05:08.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.994+0000 7f3f35319800 1 -- 172.21.15.50:0/3427296258 wait complete. 2024-05-08T18:05:08.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.994+0000 7f3f35319800 1 Processor -- start 2024-05-08T18:05:08.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.994+0000 7f3f35319800 1 -- start start 2024-05-08T18:05:08.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.994+0000 7f3f35319800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 0x56369be9c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.994+0000 7f3f35319800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56369bdaade0 con 0x56369bd15df0 2024-05-08T18:05:08.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.994+0000 7f3f32e11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 0x56369be9c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.994+0000 7f3f32e11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 0x56369be9c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38118/0 (socket says 172.21.15.50:38118) 2024-05-08T18:05:08.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.994+0000 7f3f32e11640 1 -- 172.21.15.50:0/2039769947 learned_addr learned my addr 172.21.15.50:0/2039769947 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:08.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.995+0000 7f3f32e11640 1 -- 172.21.15.50:0/2039769947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56369be9cd40 con 0x56369bd15df0 2024-05-08T18:05:08.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.995+0000 7f3f32e11640 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 0x56369be9c800 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3f24000c00 tx=0x7f3f24000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.995+0000 7f3e1f606640 1 -- 172.21.15.50:0/2039769947 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f24012030 con 0x56369bd15df0 2024-05-08T18:05:08.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.995+0000 7f3e1f606640 1 -- 172.21.15.50:0/2039769947 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f2401a750 con 0x56369bd15df0 2024-05-08T18:05:08.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.995+0000 7f3e1f606640 1 -- 172.21.15.50:0/2039769947 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f2401aa70 con 0x56369bd15df0 2024-05-08T18:05:08.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.995+0000 7f3f35319800 1 -- 172.21.15.50:0/2039769947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56369be9cfd0 con 0x56369bd15df0 2024-05-08T18:05:08.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.995+0000 7f3f35319800 1 -- 172.21.15.50:0/2039769947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56369be9fc80 con 0x56369bd15df0 2024-05-08T18:05:08.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.996+0000 7f3e1f606640 1 -- 172.21.15.50:0/2039769947 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3f24015020 con 0x56369bd15df0 2024-05-08T18:05:08.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.996+0000 7f3e1f606640 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e1803bba0 0x7f3e1803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.996+0000 7f3e1f606640 1 -- 172.21.15.50:0/2039769947 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f3f24020030 con 0x56369bd15df0 2024-05-08T18:05:08.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.996+0000 7f3f32610640 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e1803bba0 0x7f3e1803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.996+0000 7f3f35319800 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56369bd17130 0x56369bea3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:08.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.996+0000 7f3f35319800 1 -- 172.21.15.50:0/2039769947 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bea3f80 con 0x56369bd17130 2024-05-08T18:05:08.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.997+0000 7f3f33612640 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56369bd17130 0x56369bea3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:08.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.997+0000 7f3f32610640 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e1803bba0 0x7f3e1803e060 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3f28054000 tx=0x7f3f2804e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:08.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.997+0000 7f3f33612640 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56369bd17130 0x56369bea3a40 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.997+0000 7f3f33612640 1 -- 172.21.15.50:0/2039769947 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bd17130 2024-05-08T18:05:09.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.997+0000 7f3f35319800 1 -- 172.21.15.50:0/2039769947 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bea0ff0 con 0x56369bd17130 2024-05-08T18:05:09.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.998+0000 7f3f33612640 1 -- 172.21.15.50:0/2039769947 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bd17130 2024-05-08T18:05:09.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.998+0000 7f3f35319800 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56369bea62a0 0x56369bea8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.998+0000 7f3f35319800 1 -- 172.21.15.50:0/2039769947 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bea8d80 con 0x56369bea62a0 2024-05-08T18:05:09.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.998+0000 7f3f32e11640 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56369bea62a0 0x56369bea8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.998+0000 7f3f32e11640 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56369bea62a0 0x56369bea8690 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.998+0000 7f3f32e11640 1 -- 172.21.15.50:0/2039769947 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f3f240430c0 con 0x56369bea62a0 2024-05-08T18:05:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.998+0000 7f3f35319800 1 -- 172.21.15.50:0/2039769947 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bea5140 con 0x56369bd17130 2024-05-08T18:05:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.999+0000 7f3f33612640 1 -- 172.21.15.50:0/2039769947 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bd17130 2024-05-08T18:05:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.999+0000 7f3f35319800 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56369beaa1e0 0x56369beac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.999+0000 7f3f35319800 1 -- 172.21.15.50:0/2039769947 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369beaccd0 con 0x56369beaa1e0 2024-05-08T18:05:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.999+0000 7f3f32610640 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56369beaa1e0 0x56369beac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:08.999+0000 7f3f32610640 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56369beaa1e0 0x56369beac5d0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.000+0000 7f3f32610640 1 -- 172.21.15.50:0/2039769947 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3f28052940 con 0x56369beaa1e0 2024-05-08T18:05:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.000+0000 7f3f35319800 1 Processor -- start 2024-05-08T18:05:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.000+0000 7f3f35319800 1 -- start start 2024-05-08T18:05:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.000+0000 7f3f35319800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bebafc0 0x56369bebb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.000+0000 7f3f35319800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56369bc93330 con 0x56369bebafc0 2024-05-08T18:05:09.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.000+0000 7f3f32e11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bebafc0 0x56369bebb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.000+0000 7f3f32e11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bebafc0 0x56369bebb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38120/0 (socket says 172.21.15.50:38120) 2024-05-08T18:05:09.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.000+0000 7f3f32e11640 1 -- 172.21.15.50:0/1444399698 learned_addr learned my addr 172.21.15.50:0/1444399698 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.001+0000 7f3f32e11640 1 -- 172.21.15.50:0/1444399698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56369bebb920 con 0x56369bebafc0 2024-05-08T18:05:09.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.001+0000 7f3f32e11640 1 --2- 172.21.15.50:0/1444399698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bebafc0 0x56369bebb3e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400b700 tx=0x7f3f24040f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.001+0000 7f3e1ce01640 1 -- 172.21.15.50:0/1444399698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f24049e20 con 0x56369bebafc0 2024-05-08T18:05:09.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.001+0000 7f3e1ce01640 1 -- 172.21.15.50:0/1444399698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f24049fc0 con 0x56369bebafc0 2024-05-08T18:05:09.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.001+0000 7f3f35319800 1 -- 172.21.15.50:0/1444399698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56369bebc390 con 0x56369bebafc0 2024-05-08T18:05:09.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.001+0000 7f3e1ce01640 1 -- 172.21.15.50:0/1444399698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f2404a2a0 con 0x56369bebafc0 2024-05-08T18:05:09.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.001+0000 7f3f35319800 1 -- 172.21.15.50:0/1444399698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56369bebc890 con 0x56369bebafc0 2024-05-08T18:05:09.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.002+0000 7f3e1ce01640 1 -- 172.21.15.50:0/1444399698 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3f240523d0 con 0x56369bebafc0 2024-05-08T18:05:09.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.002+0000 7f3e1ce01640 1 --2- 172.21.15.50:0/1444399698 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e0c03bb50 0x7f3e0c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.002+0000 7f3e1ce01640 1 -- 172.21.15.50:0/1444399698 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f3f24049070 con 0x56369bebafc0 2024-05-08T18:05:09.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.002+0000 7f3f32610640 1 --2- 172.21.15.50:0/1444399698 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e0c03bb50 0x7f3e0c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.002+0000 7f3f35319800 1 Processor -- start 2024-05-08T18:05:09.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.002+0000 7f3f35319800 1 -- start start 2024-05-08T18:05:09.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.002+0000 7f3f32610640 1 --2- 172.21.15.50:0/1444399698 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e0c03bb50 0x7f3e0c03e010 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x56369bea4850 tx=0x7f3f28005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.002+0000 7f3f35319800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bec8120 0x56369bec8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.003+0000 7f3f35319800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56369be955d0 con 0x56369bec8120 2024-05-08T18:05:09.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.003+0000 7f3f32e11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bec8120 0x56369bec8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.003+0000 7f3f32e11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bec8120 0x56369bec8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38134/0 (socket says 172.21.15.50:38134) 2024-05-08T18:05:09.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.003+0000 7f3f32e11640 1 -- 172.21.15.50:0/3451487649 learned_addr learned my addr 172.21.15.50:0/3451487649 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.003+0000 7f3f32e11640 1 -- 172.21.15.50:0/3451487649 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56369bec8ad0 con 0x56369bec8120 2024-05-08T18:05:09.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.003+0000 7f3f32e11640 1 --2- 172.21.15.50:0/3451487649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bec8120 0x56369bec8500 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3f2402d5a0 tx=0x7f3f24016dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4d2d45187b09e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.003+0000 7f3e167fc640 1 -- 172.21.15.50:0/3451487649 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f24060070 con 0x56369bec8120 2024-05-08T18:05:09.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.003+0000 7f3e167fc640 1 -- 172.21.15.50:0/3451487649 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f2403c070 con 0x56369bec8120 2024-05-08T18:05:09.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.003+0000 7f3e167fc640 1 -- 172.21.15.50:0/3451487649 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f240167f0 con 0x56369bec8120 2024-05-08T18:05:09.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.004+0000 7f3f35319800 1 -- 172.21.15.50:0/3451487649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bec8120 msgr2=0x56369bec8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.004+0000 7f3f35319800 1 --2- 172.21.15.50:0/3451487649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bec8120 0x56369bec8500 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3f2402d5a0 tx=0x7f3f24016dd0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.004+0000 7f3f35319800 1 -- 172.21.15.50:0/3451487649 shutdown_connections 2024-05-08T18:05:09.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.004+0000 7f3f35319800 1 --2- 172.21.15.50:0/3451487649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bec8120 0x56369bec8500 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.004+0000 7f3f35319800 1 -- 172.21.15.50:0/3451487649 >> 172.21.15.50:0/3451487649 conn(0x56369bec43f0 msgr2=0x56369bec2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.004+0000 7f3f35319800 1 -- 172.21.15.50:0/3451487649 shutdown_connections 2024-05-08T18:05:09.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.004+0000 7f3f35319800 1 -- 172.21.15.50:0/3451487649 wait complete. 2024-05-08T18:05:09.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.004+0000 7f3f35319800 1 Processor -- start 2024-05-08T18:05:09.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.004+0000 7f3f35319800 1 -- start start 2024-05-08T18:05:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.004+0000 7f3f35319800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bec8120 0x56369becf550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.004+0000 7f3f35319800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56369bec91e0 con 0x56369bec8120 2024-05-08T18:05:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.004+0000 7f3f32e11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bec8120 0x56369becf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.004+0000 7f3f32e11640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bec8120 0x56369becf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38140/0 (socket says 172.21.15.50:38140) 2024-05-08T18:05:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.004+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 learned_addr learned my addr 172.21.15.50:0/3309682085 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.005+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56369becfa90 con 0x56369bec8120 2024-05-08T18:05:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.005+0000 7f3f32e11640 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bec8120 0x56369becf550 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3f2402d5a0 tx=0x7f3f24005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.005+0000 7f3e14ff9640 1 -- 172.21.15.50:0/3309682085 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f2406a070 con 0x56369bec8120 2024-05-08T18:05:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.005+0000 7f3e14ff9640 1 -- 172.21.15.50:0/3309682085 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f24037480 con 0x56369bec8120 2024-05-08T18:05:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.005+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56369becfa90 con 0x56369bec8120 2024-05-08T18:05:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.005+0000 7f3e14ff9640 1 -- 172.21.15.50:0/3309682085 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f24060040 con 0x56369bec8120 2024-05-08T18:05:09.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.005+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56369becfa90 con 0x56369bec8120 2024-05-08T18:05:09.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.006+0000 7f3e14ff9640 1 -- 172.21.15.50:0/3309682085 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3f2406d070 con 0x56369bec8120 2024-05-08T18:05:09.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.006+0000 7f3e14ff9640 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dfc03bb00 0x7f3dfc03dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.006+0000 7f3e14ff9640 1 -- 172.21.15.50:0/3309682085 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f3f24058070 con 0x56369bec8120 2024-05-08T18:05:09.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.006+0000 7f3f32610640 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dfc03bb00 0x7f3dfc03dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.007+0000 7f3f32610640 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dfc03bb00 0x7f3dfc03dfc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3f28052fd0 tx=0x7f3f2804f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.007+0000 7f3f35319800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.007+0000 7f3f35319800 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56369bedab00 0x56369bedcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.007+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bedd710 con 0x56369bedab00 2024-05-08T18:05:09.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.007+0000 7f3f33612640 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56369bedab00 0x56369bedcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.007+0000 7f3f33612640 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56369bedab00 0x56369bedcfc0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.007+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bedab00 2024-05-08T18:05:09.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.007+0000 7f3f35319800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.007+0000 7f3f35319800 20 realm 2024-05-08T18:05:09.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.007+0000 7f3f35319800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.007+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369beddb20 con 0x56369bedab00 2024-05-08T18:05:09.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.008+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bedab00 2024-05-08T18:05:09.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.008+0000 7f3f35319800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.008+0000 7f3f35319800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:09.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.008+0000 7f3f35319800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.008+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bede3f0 con 0x56369bedab00 2024-05-08T18:05:09.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.008+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bedab00 2024-05-08T18:05:09.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.008+0000 7f3f35319800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.008+0000 7f3f35319800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.008+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bede790 con 0x56369bedab00 2024-05-08T18:05:09.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.008+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bedab00 2024-05-08T18:05:09.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.008+0000 7f3f35319800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:09.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.008+0000 7f3f35319800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.008+0000 7f3f35319800 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56369bedf4f0 0x56369bee18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.008+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bee1ef0 con 0x56369bedf4f0 2024-05-08T18:05:09.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.008+0000 7f3f32e11640 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56369bedf4f0 0x56369bee18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.009+0000 7f3f32e11640 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56369bedf4f0 0x56369bee18e0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.009+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.009+0000 7f3f35319800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:09.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.009+0000 7f3f35319800 20 searching for the correct realm 2024-05-08T18:05:09.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.009+0000 7f3f35319800 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56369bee3e80 0x56369bee6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.009+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56369bee6a10 con 0x56369bee3e80 2024-05-08T18:05:09.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.009+0000 7f3f32610640 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56369bee3e80 0x56369bee6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.009+0000 7f3f32610640 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56369bee3e80 0x56369bee6340 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.010+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.010+0000 7f3f32610640 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3f28076040 0x7f3f28083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.010+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56369bee6a10 con 0x7f3f28076040 2024-05-08T18:05:09.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.010+0000 7f3f33612640 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3f28076040 0x7f3f28083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.010+0000 7f3f33612640 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3f28076040 0x7f3f28083410 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.010+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x7f3f28076040 2024-05-08T18:05:09.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.010+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56369bede790 con 0x56369bedf4f0 2024-05-08T18:05:09.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.011+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.011+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56369bede790 con 0x56369bedab00 2024-05-08T18:05:09.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.011+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bedab00 2024-05-08T18:05:09.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.011+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56369bede790 con 0x56369bedab00 2024-05-08T18:05:09.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.011+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bedab00 2024-05-08T18:05:09.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.011+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56369bede790 con 0x56369bedf4f0 2024-05-08T18:05:09.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.011+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.011+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56369bede790 con 0x56369bedf4f0 2024-05-08T18:05:09.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.012+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.012+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56369bede790 con 0x7f3f28076040 2024-05-08T18:05:09.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.012+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x7f3f28076040 2024-05-08T18:05:09.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.012+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56369bede790 con 0x56369bee3e80 2024-05-08T18:05:09.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.012+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.012+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56369bede790 con 0x56369bedab00 2024-05-08T18:05:09.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.012+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bedab00 2024-05-08T18:05:09.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.012+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56369bede790 con 0x56369bedf4f0 2024-05-08T18:05:09.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.013+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.013+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56369bede790 con 0x56369bedab00 2024-05-08T18:05:09.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.013+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bedab00 2024-05-08T18:05:09.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.013+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56369bede790 con 0x7f3f28076040 2024-05-08T18:05:09.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.013+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x7f3f28076040 2024-05-08T18:05:09.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.013+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56369bede790 con 0x56369bee3e80 2024-05-08T18:05:09.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.014+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.014+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x56369bee3e80 2024-05-08T18:05:09.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.014+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.014+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x56369bedf4f0 2024-05-08T18:05:09.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.014+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.014+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x56369bee3e80 2024-05-08T18:05:09.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.014+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.014+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x56369bee3e80 2024-05-08T18:05:09.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.015+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.015+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x56369bedab00 2024-05-08T18:05:09.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.015+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bedab00 2024-05-08T18:05:09.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.015+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x56369bedab00 2024-05-08T18:05:09.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.015+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bedab00 2024-05-08T18:05:09.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.015+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x56369bedf4f0 2024-05-08T18:05:09.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.015+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.015+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x56369bee3e80 2024-05-08T18:05:09.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.016+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.016+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x7f3f28076040 2024-05-08T18:05:09.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.016+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x7f3f28076040 2024-05-08T18:05:09.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.016+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x56369bedab00 2024-05-08T18:05:09.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.016+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bedab00 2024-05-08T18:05:09.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.016+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x56369bedf4f0 2024-05-08T18:05:09.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.016+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.016+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x56369bee3e80 2024-05-08T18:05:09.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.017+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.017+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x56369bedf4f0 2024-05-08T18:05:09.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.017+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.017+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x7f3f28076040 2024-05-08T18:05:09.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.017+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x7f3f28076040 2024-05-08T18:05:09.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.017+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x56369bee3e80 2024-05-08T18:05:09.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.017+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.017+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x56369bedf4f0 2024-05-08T18:05:09.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.018+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.018+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x56369bedab00 2024-05-08T18:05:09.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.018+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bedab00 2024-05-08T18:05:09.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.018+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f28090040 con 0x7f3f28076040 2024-05-08T18:05:09.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.018+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x7f3f28076040 2024-05-08T18:05:09.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.018+0000 7f3f35319800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:09.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.018+0000 7f3f35319800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:09.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.018+0000 7f3f35319800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:09.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.018+0000 7f3f35319800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:09.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.018+0000 7f3f35319800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:09.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.018+0000 7f3f35319800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:09.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.018+0000 7f3f35319800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.018+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bee27b0 con 0x56369bedab00 2024-05-08T18:05:09.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bedab00 2024-05-08T18:05:09.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bee27b0 con 0x56369bedab00 2024-05-08T18:05:09.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x56369bedab00 2024-05-08T18:05:09.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:09.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bee27b0 con 0x56369bee3e80 2024-05-08T18:05:09.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:09.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 20 zone default found 2024-05-08T18:05:09.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 4 Realm: () 2024-05-08T18:05:09.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:09.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:09.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:09.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 20 zonegroup default 2024-05-08T18:05:09.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bee3140 con 0x56369bedf4f0 2024-05-08T18:05:09.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.019+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bee2cf0 con 0x56369bee3e80 2024-05-08T18:05:09.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.020+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.020+0000 7f3f35319800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.020+0000 7f3f35319800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.020+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bee8dc0 con 0x56369bee3e80 2024-05-08T18:05:09.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.020+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.020+0000 7f3f35319800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.020+0000 7f3f35319800 20 started sync module instance, tier type = 2024-05-08T18:05:09.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.020+0000 7f3f35319800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:09.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.020+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369bee81b0 con 0x7f3f28076040 2024-05-08T18:05:09.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.021+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f2c00a4a0 con 0x7f3f28076040 2024-05-08T18:05:09.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.021+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94792544323280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369bee81b0 con 0x7f3f28076040 2024-05-08T18:05:09.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.021+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369bee9840 con 0x56369bedab00 2024-05-08T18:05:09.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.022+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94792544323280] v27'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f2c040040 con 0x7f3f28076040 2024-05-08T18:05:09.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.022+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f2c040040 con 0x56369bedab00 2024-05-08T18:05:09.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.022+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94792544327024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369bee81b0 con 0x56369bedab00 2024-05-08T18:05:09.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.022+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beea290 con 0x56369bee3e80 2024-05-08T18:05:09.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.024+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.024+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94792544329120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beeab30 con 0x56369bee3e80 2024-05-08T18:05:09.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.024+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beeb5b0 con 0x7f3f28076040 2024-05-08T18:05:09.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.024+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94792544327024] v27'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f2c040040 con 0x56369bedab00 2024-05-08T18:05:09.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.025+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94792544329120] v27'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.025+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f2c040040 con 0x7f3f28076040 2024-05-08T18:05:09.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.025+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94792544331392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369bee7d00 con 0x7f3f28076040 2024-05-08T18:05:09.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.025+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beebe80 con 0x56369bedab00 2024-05-08T18:05:09.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.026+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f2c040040 con 0x56369bedab00 2024-05-08T18:05:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.026+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94792544336256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beec2a0 con 0x56369bedab00 2024-05-08T18:05:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.026+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beecd90 con 0x56369bedf4f0 2024-05-08T18:05:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.026+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94792544331392] v27'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f2c040040 con 0x7f3f28076040 2024-05-08T18:05:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.027+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.027+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94792544340064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beed180 con 0x56369bedf4f0 2024-05-08T18:05:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.027+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beedb40 con 0x56369bedf4f0 2024-05-08T18:05:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.027+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94792544336256] v27'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f2c040040 con 0x56369bedab00 2024-05-08T18:05:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.028+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94792544340064] v27'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.028+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.028+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94792544337312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beedb40 con 0x56369bedf4f0 2024-05-08T18:05:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.028+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beee3d0 con 0x56369bedf4f0 2024-05-08T18:05:09.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.029+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94792544337312] v27'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.029+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.029+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94792544343696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beedf60 con 0x56369bedf4f0 2024-05-08T18:05:09.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.029+0000 7f3f35319800 20 add_watcher() i=0 2024-05-08T18:05:09.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.029+0000 7f3f35319800 20 add_watcher() i=1 2024-05-08T18:05:09.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.029+0000 7f3f35319800 20 add_watcher() i=2 2024-05-08T18:05:09.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.029+0000 7f3f35319800 20 add_watcher() i=3 2024-05-08T18:05:09.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.029+0000 7f3f35319800 20 add_watcher() i=4 2024-05-08T18:05:09.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.029+0000 7f3f35319800 20 add_watcher() i=5 2024-05-08T18:05:09.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.029+0000 7f3f35319800 20 add_watcher() i=6 2024-05-08T18:05:09.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.030+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94792544343696] v27'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.030+0000 7f3f35319800 20 add_watcher() i=7 2024-05-08T18:05:09.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.030+0000 7f3f35319800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:09.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.030+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bef1820 con 0x56369bedab00 2024-05-08T18:05:09.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.030+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f3f2c040040 con 0x56369bedab00 2024-05-08T18:05:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.030+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94792544345216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beeece0 con 0x56369bedab00 2024-05-08T18:05:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.031+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94792544345216] v27'126 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3f2c040040 con 0x56369bedab00 2024-05-08T18:05:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.031+0000 7f3de97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.031+0000 7f3f35319800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.031+0000 7f3f35319800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.032+0000 7f3f35319800 5 note: GC not initialized 2024-05-08T18:05:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.032+0000 7f3d63fff640 20 reqs_thread_entry: start 2024-05-08T18:05:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.032+0000 7f3f35319800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.032+0000 7f3f35319800 20 Filter name: none 2024-05-08T18:05:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.032+0000 7f3d61ffb640 20 reqs_thread_entry: start 2024-05-08T18:05:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.032+0000 7f3f35319800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-05-08T18:05:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.032+0000 7f3f35319800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.032+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bf24f20 con 0x56369bedf4f0 2024-05-08T18:05:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.033+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(62 testx$tenanteduser [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 246+0+0 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.033+0000 7f3f35319800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.033+0000 7f3f35319800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x0 2024-05-08T18:05:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.033+0000 7f3f35319800 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-05-08T18:05:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.033+0000 7f3f35319800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-05-08T18:05:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.033+0000 7f3f35319800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.033+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bf23610 con 0x56369bedf4f0 2024-05-08T18:05:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.033+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.033+0000 7f3f35319800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.033+0000 7f3f35319800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-05-08T18:05:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.033+0000 7f3f35319800 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-05-08T18:05:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.033+0000 7f3f35319800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-05-08T18:05:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.033+0000 7f3f35319800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-05-08T18:05:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.033+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:64 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [delete,create,call version.set in=58b,writefull 0~419 in=419b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369bf27c70 con 0x56369bedf4f0 2024-05-08T18:05:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.034+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(64 testx$tenanteduser [delete,create,call,writefull 0~419] v27'5 uv5 ondisk = 0) ==== 288+0+0 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.034+0000 7f3f35319800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2024-05-08T18:05:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.034+0000 7f3f35319800 10 moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-05-08T18:05:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.034+0000 7f3f35319800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2024-05-08T18:05:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.034+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:65 7.e 7:7759931f:::notify.3:head [notify cookie 94792544581104 in=622b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bf28740 con 0x7f3f28076040 2024-05-08T18:05:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.035+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== watch-notify(notify (1) cookie 94792544331392 notify 115964117004 ret 0) ==== 652+0+0 (crc 0 0 0) 0x7f3f2c018940 con 0x7f3f28076040 2024-05-08T18:05:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.035+0000 7f3e167fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117004 cookie 94792544331392 notifier 4529 bl.length()=610 2024-05-08T18:05:09.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.035+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(65 notify.3 [notify cookie 94792544581104 out=8b] v0'0 uv129 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3f2c040040 con 0x7f3f28076040 2024-05-08T18:05:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.035+0000 7f3e167fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2024-05-08T18:05:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.035+0000 7f3e167fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-05-08T18:05:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.035+0000 7f3e167fc640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f3df8003ea0 con 0x7f3f28076040 2024-05-08T18:05:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.035+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(66 notify.3 [notify-ack] v0'0 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f2c040040 con 0x7f3f28076040 2024-05-08T18:05:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.035+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== watch-notify(notify_complete (2) cookie 94792544581104 notify 115964117004 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f3f2c018b10 con 0x7f3f28076040 2024-05-08T18:05:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.035+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369bf275c0 con 0x56369bedf4f0 2024-05-08T18:05:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.036+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~22] v27'6 uv6 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.036+0000 7f3f35319800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-05-08T18:05:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.036+0000 7f3f35319800 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-05-08T18:05:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.036+0000 7f3f35319800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-05-08T18:05:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.036+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94792544581008 in=204b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56369bf28740 con 0x56369bee3e80 2024-05-08T18:05:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.037+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== watch-notify(notify (1) cookie 94792544329120 notify 115964116996 ret 0) ==== 234+0+0 (crc 0 0 0) 0x7f3f2808a040 con 0x56369bee3e80 2024-05-08T18:05:09.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.037+0000 7f3e167fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116996 cookie 94792544329120 notifier 4529 bl.length()=192 2024-05-08T18:05:09.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.037+0000 7f3e167fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-05-08T18:05:09.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.037+0000 7f3e167fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-05-08T18:05:09.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.037+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(68 notify.2 [notify cookie 94792544581008 out=8b] v0'0 uv129 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.037+0000 7f3e167fc640 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f3df8003ea0 con 0x56369bee3e80 2024-05-08T18:05:09.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.037+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== watch-notify(notify_complete (2) cookie 94792544581008 notify 115964116996 ret 0) ==== 42+0+48 (crc 0 0 0) 0x56369bd164a0 con 0x56369bee3e80 2024-05-08T18:05:09.049 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.037+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.049 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"testx$tenanteduser","display_name":"tenanted-user","email":"","suspended":0,"max_buckets":1,"subusers":[],"keys":[{"user":"testx$tenanteduser","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-05-08T18:05:09.035266Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:09.035252Z","tags":[],"group_ids":[]} 2024-05-08T18:05:09.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.038+0000 7f3de97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:09.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.038+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94792544345216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369bf23d40 con 0x56369bedab00 2024-05-08T18:05:09.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.039+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94792544345216] v27'127 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3f2c040040 con 0x56369bedab00 2024-05-08T18:05:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.039+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94792544323280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beedc50 con 0x7f3f28076040 2024-05-08T18:05:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.040+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94792544323280] v27'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f2c040040 con 0x7f3f28076040 2024-05-08T18:05:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.040+0000 7f3f35319800 20 remove_watcher() i=0 2024-05-08T18:05:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.040+0000 7f3f35319800 2 removed watcher, disabling cache 2024-05-08T18:05:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.040+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94792544327024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369bee86b0 con 0x56369bedab00 2024-05-08T18:05:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.041+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94792544327024] v27'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f2c040040 con 0x56369bedab00 2024-05-08T18:05:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.041+0000 7f3f35319800 20 remove_watcher() i=1 2024-05-08T18:05:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.041+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94792544329120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369bee9570 con 0x56369bee3e80 2024-05-08T18:05:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.042+0000 7f3f32610640 1 -- 172.21.15.50:0/3309682085 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94792544329120] v27'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f28081070 con 0x56369bee3e80 2024-05-08T18:05:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.042+0000 7f3f35319800 20 remove_watcher() i=2 2024-05-08T18:05:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.042+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94792544331392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369bee9570 con 0x7f3f28076040 2024-05-08T18:05:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.043+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94792544331392] v27'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f2c040040 con 0x7f3f28076040 2024-05-08T18:05:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.043+0000 7f3f35319800 20 remove_watcher() i=3 2024-05-08T18:05:09.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.043+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94792544336256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beea680 con 0x56369bedab00 2024-05-08T18:05:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.044+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94792544336256] v27'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f2c040040 con 0x56369bedab00 2024-05-08T18:05:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.044+0000 7f3f35319800 20 remove_watcher() i=4 2024-05-08T18:05:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.044+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94792544340064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beea680 con 0x56369bedf4f0 2024-05-08T18:05:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.045+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94792544340064] v27'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.045+0000 7f3f35319800 20 remove_watcher() i=5 2024-05-08T18:05:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.045+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94792544337312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beec860 con 0x56369bedf4f0 2024-05-08T18:05:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.046+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94792544337312] v27'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.046+0000 7f3f35319800 20 remove_watcher() i=6 2024-05-08T18:05:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.046+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94792544343696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56369beea680 con 0x56369bedf4f0 2024-05-08T18:05:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f32e11640 1 -- 172.21.15.50:0/3309682085 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94792544343696] v27'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f24076070 con 0x56369bedf4f0 2024-05-08T18:05:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f35319800 20 remove_watcher() i=7 2024-05-08T18:05:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56369bedf4f0 msgr2=0x56369bee18e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f35319800 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56369bedf4f0 0x56369bee18e0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56369bedab00 msgr2=0x56369bedcfc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f35319800 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56369bedab00 0x56369bedcfc0 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3f28076040 msgr2=0x7f3f28083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f35319800 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3f28076040 0x7f3f28083410 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56369bee3e80 msgr2=0x56369bee6340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f35319800 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56369bee3e80 0x56369bee6340 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dfc03bb00 msgr2=0x7f3dfc03dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f35319800 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3dfc03bb00 0x7f3dfc03dfc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3f28052fd0 tx=0x7f3f2804f000 comp rx=0 tx=0).stop 2024-05-08T18:05:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bec8120 msgr2=0x56369becf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f33612640 1 -- 172.21.15.50:0/3309682085 reap_dead start 2024-05-08T18:05:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f35319800 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bec8120 0x56369becf550 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3f2402d5a0 tx=0x7f3f24005ea0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 shutdown_connections 2024-05-08T18:05:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f35319800 1 --2- 172.21.15.50:0/3309682085 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bec8120 0x56369becf550 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.047+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 >> 172.21.15.50:0/3309682085 conn(0x56369bec43f0 msgr2=0x56369bec1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 shutdown_connections 2024-05-08T18:05:09.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 -- 172.21.15.50:0/3309682085 wait complete. 2024-05-08T18:05:09.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 -- 172.21.15.50:0/1444399698 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e0c03bb50 msgr2=0x7f3e0c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 --2- 172.21.15.50:0/1444399698 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e0c03bb50 0x7f3e0c03e010 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x56369bea4850 tx=0x7f3f28005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:09.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 -- 172.21.15.50:0/1444399698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bebafc0 msgr2=0x56369bebb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 --2- 172.21.15.50:0/1444399698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bebafc0 0x56369bebb3e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3f2400b700 tx=0x7f3f24040f50 comp rx=0 tx=0).stop 2024-05-08T18:05:09.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 -- 172.21.15.50:0/1444399698 shutdown_connections 2024-05-08T18:05:09.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 --2- 172.21.15.50:0/1444399698 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e0c03bb50 0x7f3e0c03e010 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 --2- 172.21.15.50:0/1444399698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bebafc0 0x56369bebb3e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 -- 172.21.15.50:0/1444399698 >> 172.21.15.50:0/1444399698 conn(0x56369beaf060 msgr2=0x56369beaf460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 -- 172.21.15.50:0/1444399698 shutdown_connections 2024-05-08T18:05:09.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 -- 172.21.15.50:0/1444399698 wait complete. 2024-05-08T18:05:09.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 -- 172.21.15.50:0/2039769947 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56369bea62a0 msgr2=0x56369bea8690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56369bea62a0 0x56369bea8690 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 -- 172.21.15.50:0/2039769947 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56369bd17130 msgr2=0x56369bea3a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56369bd17130 0x56369bea3a40 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 -- 172.21.15.50:0/2039769947 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56369beaa1e0 msgr2=0x56369beac5d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56369beaa1e0 0x56369beac5d0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 -- 172.21.15.50:0/2039769947 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e1803bba0 msgr2=0x7f3e1803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e1803bba0 0x7f3e1803e060 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3f28054000 tx=0x7f3f2804e000 comp rx=0 tx=0).stop 2024-05-08T18:05:09.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 -- 172.21.15.50:0/2039769947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 msgr2=0x56369be9c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 --2- 172.21.15.50:0/2039769947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56369bd15df0 0x56369be9c800 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3f24000c00 tx=0x7f3f24000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f33612640 1 -- 172.21.15.50:0/2039769947 reap_dead start 2024-05-08T18:05:09.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 -- 172.21.15.50:0/2039769947 shutdown_connections 2024-05-08T18:05:09.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.048+0000 7f3f35319800 1 -- 172.21.15.50:0/2039769947 >> 172.21.15.50:0/2039769947 conn(0x56369bd9cce0 msgr2=0x56369be8eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.049+0000 7f3f35319800 1 -- 172.21.15.50:0/2039769947 shutdown_connections 2024-05-08T18:05:09.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.049+0000 7f3f35319800 1 -- 172.21.15.50:0/2039769947 wait complete. 2024-05-08T18:05:09.060 DEBUG:tasks.util.rgw: json result: {'user_id': 'testx$tenanteduser', 'display_name': 'tenanted-user', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'testx$tenanteduser', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-05-08T18:05:09.035266Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:09.035252Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:09.060 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-05-08T18:05:09.061 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-05-08T18:05:09.061 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket /myfoo --tenant testx --uid tenanteduser 2024-05-08T18:05:09.110 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:09.110 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.112+0000 7f6f3d698800 1 Processor -- start 2024-05-08T18:05:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.112+0000 7f6f3d698800 1 -- start start 2024-05-08T18:05:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.112+0000 7f6f3d698800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x555cf7a22b50 0x555cf7a22f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.112+0000 7f6f3d698800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x555cf7a235f0 con 0x555cf7a21860 2024-05-08T18:05:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.112+0000 7f6f3d698800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x555cf78c2e20 con 0x555cf7a22b50 2024-05-08T18:05:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.113+0000 7f6f3b197640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x555cf7a22b50 0x555cf7a22f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.113+0000 7f6f3a996640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x555cf7a21860 0x555cf7a21ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43236/0 (socket says 172.21.15.50:43236) 2024-05-08T18:05:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.113+0000 7f6f3a996640 1 -- 172.21.15.50:0/3331506884 learned_addr learned my addr 172.21.15.50:0/3331506884 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:09.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.113+0000 7f6f3b197640 1 -- 172.21.15.50:0/3331506884 >> v1:172.21.15.46:6789/0 conn(0x555cf7a21860 legacy=0x555cf7a21ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.113+0000 7f6f3b197640 1 -- 172.21.15.50:0/3331506884 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf7a223f0 con 0x555cf7a22b50 2024-05-08T18:05:09.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.113+0000 7f6f3a195640 1 -- 172.21.15.50:0/3331506884 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3429967819 0 0) 0x555cf7a235f0 con 0x555cf7a21860 2024-05-08T18:05:09.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.115+0000 7f6f3b197640 1 --2- 172.21.15.50:0/3331506884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a22b50 0x555cf7a22f70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00a470 tx=0x7f6f2c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac887280320b572a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.115+0000 7f6f3a195640 1 -- 172.21.15.50:0/3331506884 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f2c057070 con 0x555cf7a22b50 2024-05-08T18:05:09.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.115+0000 7f6f3a195640 1 -- 172.21.15.50:0/3331506884 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2c04adc0 con 0x555cf7a22b50 2024-05-08T18:05:09.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.115+0000 7f6f3a195640 1 -- 172.21.15.50:0/3331506884 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f2c052590 con 0x555cf7a22b50 2024-05-08T18:05:09.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.115+0000 7f6f3d698800 1 -- 172.21.15.50:0/3331506884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a22b50 msgr2=0x555cf7a22f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.115+0000 7f6f3d698800 1 --2- 172.21.15.50:0/3331506884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a22b50 0x555cf7a22f70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00a470 tx=0x7f6f2c01ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:09.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.116+0000 7f6f3d698800 1 -- 172.21.15.50:0/3331506884 shutdown_connections 2024-05-08T18:05:09.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.116+0000 7f6f3d698800 1 --2- 172.21.15.50:0/3331506884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a22b50 0x555cf7a22f70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.116+0000 7f6f3d698800 1 -- 172.21.15.50:0/3331506884 >> 172.21.15.50:0/3331506884 conn(0x555cf7a1c2f0 msgr2=0x555cf7a1e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.116+0000 7f6f3d698800 1 -- 172.21.15.50:0/3331506884 shutdown_connections 2024-05-08T18:05:09.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.116+0000 7f6f3d698800 1 -- 172.21.15.50:0/3331506884 wait complete. 2024-05-08T18:05:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.131+0000 7f6f3d698800 1 Processor -- start 2024-05-08T18:05:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.131+0000 7f6f3d698800 1 -- start start 2024-05-08T18:05:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.131+0000 7f6f3d698800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a21860 0x555cf7a21290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.131+0000 7f6f3d698800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555cf796da30 con 0x555cf7a21860 2024-05-08T18:05:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.132+0000 7f6f3b197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a21860 0x555cf7a21290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.132+0000 7f6f3b197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a21860 0x555cf7a21290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38170/0 (socket says 172.21.15.50:38170) 2024-05-08T18:05:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.132+0000 7f6f3b197640 1 -- 172.21.15.50:0/885895689 learned_addr learned my addr 172.21.15.50:0/885895689 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.132+0000 7f6f3b197640 1 -- 172.21.15.50:0/885895689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf7b14590 con 0x555cf7a21860 2024-05-08T18:05:09.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.132+0000 7f6f3b197640 1 --2- 172.21.15.50:0/885895689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a21860 0x555cf7a21290 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c000c00 tx=0x7f6f2c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89ca9790804b288 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.132+0000 7f6e2960a640 1 -- 172.21.15.50:0/885895689 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f2c057040 con 0x555cf7a21860 2024-05-08T18:05:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.133+0000 7f6e2960a640 1 -- 172.21.15.50:0/885895689 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2c063070 con 0x555cf7a21860 2024-05-08T18:05:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.133+0000 7f6e2960a640 1 -- 172.21.15.50:0/885895689 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f2c05d470 con 0x555cf7a21860 2024-05-08T18:05:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.133+0000 7f6f3d698800 1 -- 172.21.15.50:0/885895689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a21860 msgr2=0x555cf7a21290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.133+0000 7f6f3d698800 1 --2- 172.21.15.50:0/885895689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a21860 0x555cf7a21290 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c000c00 tx=0x7f6f2c04c000 comp rx=0 tx=0).stop 2024-05-08T18:05:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.133+0000 7f6f3d698800 1 -- 172.21.15.50:0/885895689 shutdown_connections 2024-05-08T18:05:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.133+0000 7f6f3d698800 1 --2- 172.21.15.50:0/885895689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a21860 0x555cf7a21290 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.133+0000 7f6f3d698800 1 -- 172.21.15.50:0/885895689 >> 172.21.15.50:0/885895689 conn(0x555cf7a1c2f0 msgr2=0x555cf7b0e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.133+0000 7f6f3d698800 1 -- 172.21.15.50:0/885895689 shutdown_connections 2024-05-08T18:05:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.133+0000 7f6f3d698800 1 -- 172.21.15.50:0/885895689 wait complete. 2024-05-08T18:05:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.134+0000 7f6f3d698800 1 Processor -- start 2024-05-08T18:05:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.134+0000 7f6f3d698800 1 -- start start 2024-05-08T18:05:09.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.134+0000 7f6f3d698800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a21860 0x555cf7b1d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.134+0000 7f6f3d698800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555cf7a2ad10 con 0x555cf7a21860 2024-05-08T18:05:09.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.134+0000 7f6f3b197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a21860 0x555cf7b1d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.134+0000 7f6f3b197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a21860 0x555cf7b1d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38178/0 (socket says 172.21.15.50:38178) 2024-05-08T18:05:09.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.134+0000 7f6f3b197640 1 -- 172.21.15.50:0/2017692848 learned_addr learned my addr 172.21.15.50:0/2017692848 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.135+0000 7f6f3b197640 1 -- 172.21.15.50:0/2017692848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf7b1db70 con 0x555cf7a21860 2024-05-08T18:05:09.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.135+0000 7f6f3b197640 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a21860 0x555cf7b1d630 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c000c00 tx=0x7f6f2c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.135+0000 7f6e27e07640 1 -- 172.21.15.50:0/2017692848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f2c06c070 con 0x555cf7a21860 2024-05-08T18:05:09.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.135+0000 7f6e27e07640 1 -- 172.21.15.50:0/2017692848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2c05d8f0 con 0x555cf7a21860 2024-05-08T18:05:09.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.135+0000 7f6e27e07640 1 -- 172.21.15.50:0/2017692848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f2c057070 con 0x555cf7a21860 2024-05-08T18:05:09.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.135+0000 7f6f3d698800 1 -- 172.21.15.50:0/2017692848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555cf7b1de00 con 0x555cf7a21860 2024-05-08T18:05:09.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.135+0000 7f6f3d698800 1 -- 172.21.15.50:0/2017692848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf7b20ab0 con 0x555cf7a21860 2024-05-08T18:05:09.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.136+0000 7f6e27e07640 1 -- 172.21.15.50:0/2017692848 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6f2c05dc10 con 0x555cf7a21860 2024-05-08T18:05:09.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.136+0000 7f6e27e07640 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6e2003bba0 0x7f6e2003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.136+0000 7f6e27e07640 1 -- 172.21.15.50:0/2017692848 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f6f2c094ae0 con 0x555cf7a21860 2024-05-08T18:05:09.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.136+0000 7f6f3a996640 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6e2003bba0 0x7f6e2003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.137+0000 7f6f3d698800 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555cf7a22b50 0x555cf7b24870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.137+0000 7f6f3d698800 1 -- 172.21.15.50:0/2017692848 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b24db0 con 0x555cf7a22b50 2024-05-08T18:05:09.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.137+0000 7f6f3b998640 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555cf7a22b50 0x555cf7b24870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.137+0000 7f6f3a996640 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6e2003bba0 0x7f6e2003e060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6f30000c00 tx=0x7f6f300013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.137+0000 7f6f3b998640 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555cf7a22b50 0x555cf7b24870 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.137+0000 7f6f3b998640 1 -- 172.21.15.50:0/2017692848 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7a22b50 2024-05-08T18:05:09.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.137+0000 7f6f3d698800 1 -- 172.21.15.50:0/2017692848 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b21e20 con 0x555cf7a22b50 2024-05-08T18:05:09.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.137+0000 7f6f3b998640 1 -- 172.21.15.50:0/2017692848 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7a22b50 2024-05-08T18:05:09.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.138+0000 7f6f3d698800 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555cf7b27080 0x555cf7b29470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.138+0000 7f6f3d698800 1 -- 172.21.15.50:0/2017692848 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b29b40 con 0x555cf7b27080 2024-05-08T18:05:09.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.138+0000 7f6f3b197640 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555cf7b27080 0x555cf7b29470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.138+0000 7f6f3b197640 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555cf7b27080 0x555cf7b29470 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.138+0000 7f6f3b197640 1 -- 172.21.15.50:0/2017692848 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b27080 2024-05-08T18:05:09.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.138+0000 7f6f3d698800 1 -- 172.21.15.50:0/2017692848 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b25ef0 con 0x555cf7a22b50 2024-05-08T18:05:09.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.139+0000 7f6f3b998640 1 -- 172.21.15.50:0/2017692848 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7a22b50 2024-05-08T18:05:09.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.139+0000 7f6f3d698800 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555cf7b2af80 0x555cf7b2d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.139+0000 7f6f3d698800 1 -- 172.21.15.50:0/2017692848 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b2dab0 con 0x555cf7b2af80 2024-05-08T18:05:09.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.139+0000 7f6f3a996640 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555cf7b2af80 0x555cf7b2d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.139+0000 7f6f3a996640 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555cf7b2af80 0x555cf7b2d370 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.139+0000 7f6f3a996640 1 -- 172.21.15.50:0/2017692848 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f6f3000f670 con 0x555cf7b2af80 2024-05-08T18:05:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.140+0000 7f6f3d698800 1 Processor -- start 2024-05-08T18:05:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.140+0000 7f6f3d698800 1 -- start start 2024-05-08T18:05:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.140+0000 7f6f3d698800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b3bdf0 0x555cf7b3c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.140+0000 7f6f3d698800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555cf7914330 con 0x555cf7b3bdf0 2024-05-08T18:05:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.140+0000 7f6f3b197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b3bdf0 0x555cf7b3c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.140+0000 7f6f3b197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b3bdf0 0x555cf7b3c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38194/0 (socket says 172.21.15.50:38194) 2024-05-08T18:05:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.140+0000 7f6f3b197640 1 -- 172.21.15.50:0/4224538954 learned_addr learned my addr 172.21.15.50:0/4224538954 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.140+0000 7f6f3b197640 1 -- 172.21.15.50:0/4224538954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf7b3c750 con 0x555cf7b3bdf0 2024-05-08T18:05:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.140+0000 7f6f3b197640 1 --2- 172.21.15.50:0/4224538954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b3bdf0 0x555cf7b3c210 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009af0 tx=0x7f6f2c0096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.141+0000 7f6e25602640 1 -- 172.21.15.50:0/4224538954 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f2c068cc0 con 0x555cf7b3bdf0 2024-05-08T18:05:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.141+0000 7f6f3d698800 1 -- 172.21.15.50:0/4224538954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555cf7b3d1c0 con 0x555cf7b3bdf0 2024-05-08T18:05:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.141+0000 7f6e25602640 1 -- 172.21.15.50:0/4224538954 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2c067a50 con 0x555cf7b3bdf0 2024-05-08T18:05:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.141+0000 7f6e25602640 1 -- 172.21.15.50:0/4224538954 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f2c068e60 con 0x555cf7b3bdf0 2024-05-08T18:05:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.141+0000 7f6f3d698800 1 -- 172.21.15.50:0/4224538954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf7b3d6c0 con 0x555cf7b3bdf0 2024-05-08T18:05:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.141+0000 7f6e25602640 1 -- 172.21.15.50:0/4224538954 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6f2c084070 con 0x555cf7b3bdf0 2024-05-08T18:05:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.142+0000 7f6e25602640 1 --2- 172.21.15.50:0/4224538954 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6e1403bb50 0x7f6e1403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.142+0000 7f6e25602640 1 -- 172.21.15.50:0/4224538954 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f6f2c08a020 con 0x555cf7b3bdf0 2024-05-08T18:05:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.142+0000 7f6f3a996640 1 --2- 172.21.15.50:0/4224538954 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6e1403bb50 0x7f6e1403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.142+0000 7f6f3d698800 1 Processor -- start 2024-05-08T18:05:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.142+0000 7f6f3d698800 1 -- start start 2024-05-08T18:05:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.142+0000 7f6f3a996640 1 --2- 172.21.15.50:0/4224538954 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6e1403bb50 0x7f6e1403e010 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x555cf7b25220 tx=0x7f6f30006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.142+0000 7f6f3d698800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b48dc0 0x555cf7b491e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.142+0000 7f6f3d698800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555cf7b18c60 con 0x555cf7b48dc0 2024-05-08T18:05:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.142+0000 7f6f3b197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b48dc0 0x555cf7b491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.142+0000 7f6f3b197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b48dc0 0x555cf7b491e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38204/0 (socket says 172.21.15.50:38204) 2024-05-08T18:05:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.142+0000 7f6f3b197640 1 -- 172.21.15.50:0/420526020 learned_addr learned my addr 172.21.15.50:0/420526020 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.143+0000 7f6f3b197640 1 -- 172.21.15.50:0/420526020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf7b497b0 con 0x555cf7b48dc0 2024-05-08T18:05:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.143+0000 7f6f3b197640 1 --2- 172.21.15.50:0/420526020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b48dc0 0x555cf7b491e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009900 tx=0x7f6f2c0919b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62d88a8f61ac3e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.143+0000 7f6e1effd640 1 -- 172.21.15.50:0/420526020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f2c067390 con 0x555cf7b48dc0 2024-05-08T18:05:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.143+0000 7f6e1effd640 1 -- 172.21.15.50:0/420526020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2c08b2c0 con 0x555cf7b48dc0 2024-05-08T18:05:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.143+0000 7f6e1effd640 1 -- 172.21.15.50:0/420526020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f2c067530 con 0x555cf7b48dc0 2024-05-08T18:05:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.143+0000 7f6f3d698800 1 -- 172.21.15.50:0/420526020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b48dc0 msgr2=0x555cf7b491e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.143+0000 7f6f3d698800 1 --2- 172.21.15.50:0/420526020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b48dc0 0x555cf7b491e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009900 tx=0x7f6f2c0919b0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.143+0000 7f6f3d698800 1 -- 172.21.15.50:0/420526020 shutdown_connections 2024-05-08T18:05:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.143+0000 7f6f3d698800 1 --2- 172.21.15.50:0/420526020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b48dc0 0x555cf7b491e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.143+0000 7f6f3d698800 1 -- 172.21.15.50:0/420526020 >> 172.21.15.50:0/420526020 conn(0x555cf7b450d0 msgr2=0x555cf7b43130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.143+0000 7f6f3d698800 1 -- 172.21.15.50:0/420526020 shutdown_connections 2024-05-08T18:05:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.143+0000 7f6f3d698800 1 -- 172.21.15.50:0/420526020 wait complete. 2024-05-08T18:05:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.144+0000 7f6f3d698800 1 Processor -- start 2024-05-08T18:05:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.144+0000 7f6f3d698800 1 -- start start 2024-05-08T18:05:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.144+0000 7f6f3d698800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b48dc0 0x555cf7b50160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.144+0000 7f6f3d698800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555cf7b49e40 con 0x555cf7b48dc0 2024-05-08T18:05:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.144+0000 7f6f3b197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b48dc0 0x555cf7b50160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.144+0000 7f6f3b197640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b48dc0 0x555cf7b50160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38214/0 (socket says 172.21.15.50:38214) 2024-05-08T18:05:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.144+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 learned_addr learned my addr 172.21.15.50:0/692100361 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.144+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cf7b506a0 con 0x555cf7b48dc0 2024-05-08T18:05:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.145+0000 7f6f3b197640 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b48dc0 0x555cf7b50160 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c08fa00 tx=0x7f6f2c0a3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.145+0000 7f6e1d7fa640 1 -- 172.21.15.50:0/692100361 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f2c091190 con 0x555cf7b48dc0 2024-05-08T18:05:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.145+0000 7f6e1d7fa640 1 -- 172.21.15.50:0/692100361 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f2c091330 con 0x555cf7b48dc0 2024-05-08T18:05:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.145+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555cf7b506a0 con 0x555cf7b48dc0 2024-05-08T18:05:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.145+0000 7f6e1d7fa640 1 -- 172.21.15.50:0/692100361 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f2c08b620 con 0x555cf7b48dc0 2024-05-08T18:05:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.145+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cf7b506a0 con 0x555cf7b48dc0 2024-05-08T18:05:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.146+0000 7f6e1d7fa640 1 -- 172.21.15.50:0/692100361 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6f2c08b7c0 con 0x555cf7b48dc0 2024-05-08T18:05:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.146+0000 7f6e1d7fa640 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6e0403bba0 0x7f6e0403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.146+0000 7f6e1d7fa640 1 -- 172.21.15.50:0/692100361 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f6f2c0ad980 con 0x555cf7b48dc0 2024-05-08T18:05:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.146+0000 7f6f3a996640 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6e0403bba0 0x7f6e0403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.146+0000 7f6f3a996640 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6e0403bba0 0x7f6e0403e060 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000ffd0 tx=0x7f6f30023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.147+0000 7f6f3d698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.147+0000 7f6f3d698800 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555cf7b5b850 0x555cf7b5dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.147+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5e460 con 0x555cf7b5b850 2024-05-08T18:05:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.147+0000 7f6f3b998640 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555cf7b5b850 0x555cf7b5dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.147+0000 7f6f3b998640 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555cf7b5b850 0x555cf7b5dd10 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.147+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7b5b850 2024-05-08T18:05:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.147+0000 7f6f3d698800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.147+0000 7f6f3d698800 20 realm 2024-05-08T18:05:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.147+0000 7f6f3d698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.147+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5e870 con 0x555cf7b5b850 2024-05-08T18:05:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.147+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7b5b850 2024-05-08T18:05:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.147+0000 7f6f3d698800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.147+0000 7f6f3d698800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.147+0000 7f6f3d698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.147+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f140 con 0x555cf7b5b850 2024-05-08T18:05:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.148+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7b5b850 2024-05-08T18:05:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.148+0000 7f6f3d698800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.148+0000 7f6f3d698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.148+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b5b850 2024-05-08T18:05:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.148+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7b5b850 2024-05-08T18:05:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.148+0000 7f6f3d698800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.148+0000 7f6f3d698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.148+0000 7f6f3d698800 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555cf7b60240 0x555cf7b62630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.148+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b62bf0 con 0x555cf7b60240 2024-05-08T18:05:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.148+0000 7f6f3b197640 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555cf7b60240 0x555cf7b62630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.149+0000 7f6f3b197640 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555cf7b60240 0x555cf7b62630 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.149+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.149+0000 7f6f3d698800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.149+0000 7f6f3d698800 20 searching for the correct realm 2024-05-08T18:05:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.149+0000 7f6f3d698800 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555cf7b64ba0 0x555cf7b67060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.149+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b67730 con 0x555cf7b64ba0 2024-05-08T18:05:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.149+0000 7f6f3a996640 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555cf7b64ba0 0x555cf7b67060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.149+0000 7f6f3a996640 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555cf7b64ba0 0x555cf7b67060 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.150+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f30037040 con 0x555cf7b64ba0 2024-05-08T18:05:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.150+0000 7f6f3a996640 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6f3002e910 0x7f6f3003a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.150+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b67730 con 0x7f6f3002e910 2024-05-08T18:05:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.150+0000 7f6f3b998640 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6f3002e910 0x7f6f3003a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.150+0000 7f6f3b998640 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6f3002e910 0x7f6f3003a3c0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.150+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f3400a4a0 con 0x7f6f3002e910 2024-05-08T18:05:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.150+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b60240 2024-05-08T18:05:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.151+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.151+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b5b850 2024-05-08T18:05:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.151+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7b5b850 2024-05-08T18:05:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.151+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b5b850 2024-05-08T18:05:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.151+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7b5b850 2024-05-08T18:05:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.151+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b60240 2024-05-08T18:05:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.152+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.152+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b60240 2024-05-08T18:05:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.152+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.152+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x7f6f3002e910 2024-05-08T18:05:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.152+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f3400a4a0 con 0x7f6f3002e910 2024-05-08T18:05:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.152+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b64ba0 2024-05-08T18:05:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.152+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f30037040 con 0x555cf7b64ba0 2024-05-08T18:05:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.152+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b5b850 2024-05-08T18:05:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.153+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7b5b850 2024-05-08T18:05:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.153+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b60240 2024-05-08T18:05:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.153+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.153+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b5b850 2024-05-08T18:05:09.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.153+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7b5b850 2024-05-08T18:05:09.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.153+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x7f6f3002e910 2024-05-08T18:05:09.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.153+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f6f3400a4a0 con 0x7f6f3002e910 2024-05-08T18:05:09.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.153+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b64ba0 2024-05-08T18:05:09.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.154+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f30037040 con 0x555cf7b64ba0 2024-05-08T18:05:09.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.154+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b64ba0 2024-05-08T18:05:09.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.154+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f30037040 con 0x555cf7b64ba0 2024-05-08T18:05:09.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.154+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b60240 2024-05-08T18:05:09.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.154+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.154+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b64ba0 2024-05-08T18:05:09.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.154+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f30037040 con 0x555cf7b64ba0 2024-05-08T18:05:09.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.155+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b64ba0 2024-05-08T18:05:09.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.155+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f6f30037040 con 0x555cf7b64ba0 2024-05-08T18:05:09.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.155+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b5b850 2024-05-08T18:05:09.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.155+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7b5b850 2024-05-08T18:05:09.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.155+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b5b850 2024-05-08T18:05:09.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.155+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7b5b850 2024-05-08T18:05:09.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.155+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b60240 2024-05-08T18:05:09.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.156+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.156+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b64ba0 2024-05-08T18:05:09.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.156+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f30037040 con 0x555cf7b64ba0 2024-05-08T18:05:09.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.156+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x7f6f3002e910 2024-05-08T18:05:09.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.156+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f3400a4a0 con 0x7f6f3002e910 2024-05-08T18:05:09.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.156+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b5b850 2024-05-08T18:05:09.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.157+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7b5b850 2024-05-08T18:05:09.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.157+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b60240 2024-05-08T18:05:09.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.157+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.157+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b64ba0 2024-05-08T18:05:09.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.157+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f30037040 con 0x555cf7b64ba0 2024-05-08T18:05:09.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.157+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b60240 2024-05-08T18:05:09.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.157+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.157+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x7f6f3002e910 2024-05-08T18:05:09.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.158+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f3400a4a0 con 0x7f6f3002e910 2024-05-08T18:05:09.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.158+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b64ba0 2024-05-08T18:05:09.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.158+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f30037040 con 0x555cf7b64ba0 2024-05-08T18:05:09.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.158+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b60240 2024-05-08T18:05:09.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.158+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.158+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x555cf7b5b850 2024-05-08T18:05:09.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.158+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7b5b850 2024-05-08T18:05:09.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.158+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b5f4e0 con 0x7f6f3002e910 2024-05-08T18:05:09.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6f3400a4a0 con 0x7f6f3002e910 2024-05-08T18:05:09.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3d698800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:09.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3d698800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:09.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3d698800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:09.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3d698800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:09.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3d698800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:09.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3d698800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:09.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3d698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b634b0 con 0x555cf7b5b850 2024-05-08T18:05:09.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7b5b850 2024-05-08T18:05:09.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3d698800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3d698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b634b0 con 0x555cf7b5b850 2024-05-08T18:05:09.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6f3400a4a0 con 0x555cf7b5b850 2024-05-08T18:05:09.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3d698800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:09.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3d698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b634b0 con 0x555cf7b64ba0 2024-05-08T18:05:09.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f6f30037040 con 0x555cf7b64ba0 2024-05-08T18:05:09.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.159+0000 7f6f3d698800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:09.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 20 zone default found 2024-05-08T18:05:09.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 4 Realm: () 2024-05-08T18:05:09.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:09.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:09.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:09.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 20 zonegroup default 2024-05-08T18:05:09.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b63e20 con 0x555cf7b60240 2024-05-08T18:05:09.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b69620 con 0x555cf7b64ba0 2024-05-08T18:05:09.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6f30037040 con 0x555cf7b64ba0 2024-05-08T18:05:09.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b69af0 con 0x555cf7b64ba0 2024-05-08T18:05:09.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6f30037040 con 0x555cf7b64ba0 2024-05-08T18:05:09.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 20 started sync module instance, tier type = 2024-05-08T18:05:09.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:09.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.160+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b68a30 con 0x7f6f3002e910 2024-05-08T18:05:09.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.161+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f3400a4a0 con 0x7f6f3002e910 2024-05-08T18:05:09.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.161+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93857781288784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b68a30 con 0x7f6f3002e910 2024-05-08T18:05:09.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.162+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b6a5b0 con 0x555cf7b5b850 2024-05-08T18:05:09.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.163+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93857781288784] v27'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x7f6f3002e910 2024-05-08T18:05:09.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.163+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x555cf7b5b850 2024-05-08T18:05:09.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.163+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93857781293248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b68a30 con 0x555cf7b5b850 2024-05-08T18:05:09.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.163+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b6b020 con 0x555cf7b64ba0 2024-05-08T18:05:09.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.164+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93857781293248] v27'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x555cf7b5b850 2024-05-08T18:05:09.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.164+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f30037040 con 0x555cf7b64ba0 2024-05-08T18:05:09.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.164+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93857781286272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b689a0 con 0x555cf7b64ba0 2024-05-08T18:05:09.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.164+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b6bf10 con 0x7f6f3002e910 2024-05-08T18:05:09.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.165+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x7f6f3002e910 2024-05-08T18:05:09.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.165+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93857781300352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b6c7b0 con 0x7f6f3002e910 2024-05-08T18:05:09.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.165+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b6d280 con 0x555cf7b5b850 2024-05-08T18:05:09.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.165+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93857781286272] v27'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f30037040 con 0x555cf7b64ba0 2024-05-08T18:05:09.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.166+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x555cf7b5b850 2024-05-08T18:05:09.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.166+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93857781304688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b6d240 con 0x555cf7b5b850 2024-05-08T18:05:09.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.166+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93857781300352] v27'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x7f6f3002e910 2024-05-08T18:05:09.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.166+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b6dc90 con 0x555cf7b60240 2024-05-08T18:05:09.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.167+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93857781304688] v27'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x555cf7b5b850 2024-05-08T18:05:09.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.167+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.167+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93857781302144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b6b440 con 0x555cf7b60240 2024-05-08T18:05:09.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.167+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b6e4e0 con 0x555cf7b60240 2024-05-08T18:05:09.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.168+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93857781302144] v27'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.168+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.168+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93857781307360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b6ed30 con 0x555cf7b60240 2024-05-08T18:05:09.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.168+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b6f6f0 con 0x555cf7b60240 2024-05-08T18:05:09.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.169+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93857781307360] v27'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.169+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.169+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93857781312096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b6f6f0 con 0x555cf7b60240 2024-05-08T18:05:09.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.169+0000 7f6f3d698800 20 add_watcher() i=0 2024-05-08T18:05:09.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.169+0000 7f6f3d698800 20 add_watcher() i=1 2024-05-08T18:05:09.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.169+0000 7f6f3d698800 20 add_watcher() i=2 2024-05-08T18:05:09.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.169+0000 7f6f3d698800 20 add_watcher() i=3 2024-05-08T18:05:09.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.169+0000 7f6f3d698800 20 add_watcher() i=4 2024-05-08T18:05:09.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.169+0000 7f6f3d698800 20 add_watcher() i=5 2024-05-08T18:05:09.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.169+0000 7f6f3d698800 20 add_watcher() i=6 2024-05-08T18:05:09.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.170+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93857781312096] v27'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.170+0000 7f6f3d698800 20 add_watcher() i=7 2024-05-08T18:05:09.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.170+0000 7f6f3d698800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:09.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.170+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b72a50 con 0x555cf7b5b850 2024-05-08T18:05:09.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.171+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f6f34040040 con 0x555cf7b5b850 2024-05-08T18:05:09.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.171+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93857781314480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b6fc80 con 0x555cf7b5b850 2024-05-08T18:05:09.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.172+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93857781314480] v27'128 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x555cf7b5b850 2024-05-08T18:05:09.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.172+0000 7f6df1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:09.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.172+0000 7f6f3d698800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:09.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.172+0000 7f6f3d698800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:09.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.172+0000 7f6f3d698800 5 note: GC not initialized 2024-05-08T18:05:09.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.172+0000 7f6df0fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:09.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.172+0000 7f6f3d698800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:09.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.172+0000 7f6f3d698800 20 Filter name: none 2024-05-08T18:05:09.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.172+0000 7f6d6a7fc640 20 reqs_thread_entry: start 2024-05-08T18:05:09.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.172+0000 7f6f3d698800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-05-08T18:05:09.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.172+0000 7f6f3d698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.172+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7ba6250 con 0x555cf7b60240 2024-05-08T18:05:09.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.173+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(62 testx$tenanteduser [call out=48b,read 0~419 out=419b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 246+0+542 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.173+0000 7f6f3d698800 20 rados_obj.operate() r=0 bl.length=419 2024-05-08T18:05:09.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.173+0000 7f6f3d698800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2024-05-08T18:05:09.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.173+0000 7f6f3d698800 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-05-08T18:05:09.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.173+0000 7f6f3d698800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:09.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.173+0000 7f6f3d698800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-05-08T18:05:09.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.173+0000 7f6f3d698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.173+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7ba5ca0 con 0x7f6f3002e910 2024-05-08T18:05:09.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.173+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv25 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f6f34040040 con 0x7f6f3002e910 2024-05-08T18:05:09.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.173+0000 7f6f3d698800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:09.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.173+0000 7f6f3d698800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-05-08T18:05:09.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.173+0000 7f6f3d698800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:09.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.173+0000 7f6f3d698800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 : miss 2024-05-08T18:05:09.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.173+0000 7f6f3d698800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.173+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:64 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7ba9a20 con 0x555cf7b60240 2024-05-08T18:05:09.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.174+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(64 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.174+0000 7f6f3d698800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:09.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.174+0000 7f6f3d698800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:09.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.174+0000 7f6f3d698800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:09.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.174+0000 7f6f3d698800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:09.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.174+0000 7f6f3d698800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:09.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.174+0000 7f6f3d698800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 2024-05-08T18:05:09.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.174+0000 7f6f3d698800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : hit (requested=0x13, cached=0x13) 2024-05-08T18:05:09.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.174+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7badda0 con 0x7f6f3002e910 2024-05-08T18:05:09.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.175+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(65 foo.buckets [call] v27'26 uv26 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x7f6f3002e910 2024-05-08T18:05:09.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.175+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 8.1f 8:fa027c55:root::.bucket.meta.testx%3amyfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [create,call version.inc in=38b,writefull 0~412 in=412b,setxattr user.rgw.acl (203) in=215b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7bb1690 con 0x7f6f3002e910 2024-05-08T18:05:09.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.176+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(66 .bucket.meta.testx:myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [create,call,writefull 0~412,setxattr (203)] v27'1 uv1 ondisk = 0) ==== 338+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x7f6f3002e910 2024-05-08T18:05:09.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.176+0000 7f6f3d698800 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x7 2024-05-08T18:05:09.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.176+0000 7f6f3d698800 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:09.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.176+0000 7f6f3d698800 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-08T18:05:09.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.176+0000 7f6f3d698800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1, ofs0, ns] 2024-05-08T18:05:09.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.176+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:67 7.3 7:c609908c:::notify.5:head [notify cookie 93857781576896 in=859b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7bb1100 con 0x555cf7b60240 2024-05-08T18:05:09.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.176+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== watch-notify(notify (1) cookie 93857781302144 notify 115964117003 ret 0) ==== 889+0+0 (crc 0 0 0) 0x555cf7ba8ff0 con 0x555cf7b60240 2024-05-08T18:05:09.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.176+0000 7f6e1effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117003 cookie 93857781302144 notifier 4536 bl.length()=847 2024-05-08T18:05:09.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.176+0000 7f6e1effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x7 2024-05-08T18:05:09.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.176+0000 7f6e1effd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.testx:myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:09.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.176+0000 7f6e1effd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-05-08T18:05:09.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.176+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(67 notify.5 [notify cookie 93857781576896 out=8b] v0'0 uv132 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.177+0000 7f6e1effd640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f6e00003ea0 con 0x555cf7b60240 2024-05-08T18:05:09.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.177+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(68 notify.5 [notify-ack] v0'0 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.177+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== watch-notify(notify_complete (2) cookie 93857781576896 notify 115964117003 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f6f2c080040 con 0x555cf7b60240 2024-05-08T18:05:09.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.177+0000 7f6f3d698800 20 handle_bi_update(): bucket=testx:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): orig_sources= new_sources= 2024-05-08T18:05:09.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.177+0000 7f6f3d698800 20 handle_bi_update(): bucket=testx:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): potential sources added=[] removed=[] 2024-05-08T18:05:09.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.177+0000 7f6f3d698800 20 handle_bi_update(): bucket=testx:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): orig_dests= new_dests= 2024-05-08T18:05:09.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.177+0000 7f6f3d698800 20 handle_bi_update(): bucket=testx:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): potential dests added=[] removed=[] 2024-05-08T18:05:09.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.177+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7bb11e0 con 0x555cf7b5b850 2024-05-08T18:05:09.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.178+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(69 testx$tenanteduser.buckets [call] v27'1 uv1 ondisk = 0) ==== 170+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x555cf7b5b850 2024-05-08T18:05:09.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.178+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [delete,create,call version.inc in=38b,writefull 0~193 in=193b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7bb11e0 con 0x555cf7b5b850 2024-05-08T18:05:09.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.179+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(70 testx/myfoo [delete,create,call,writefull 0~193] v27'1 uv1 ondisk = 0) ==== 281+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x555cf7b5b850 2024-05-08T18:05:09.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.179+0000 7f6f3d698800 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2024-05-08T18:05:09.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.179+0000 7f6f3d698800 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2024-05-08T18:05:09.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.179+0000 7f6f3d698800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2024-05-08T18:05:09.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.179+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify cookie 93857781583152 in=360b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7bb1150 con 0x555cf7b5b850 2024-05-08T18:05:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.179+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== watch-notify(notify (1) cookie 93857781304688 notify 115964117009 ret 0) ==== 390+0+0 (crc 0 0 0) 0x555cf7bb1580 con 0x555cf7b5b850 2024-05-08T18:05:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.179+0000 7f6e1e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117009 cookie 93857781304688 notifier 4536 bl.length()=348 2024-05-08T18:05:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.179+0000 7f6e1e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2024-05-08T18:05:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.179+0000 7f6e1e7fc640 10 rgw watcher librados: moving default.rgw.meta+root+testx/myfoo to cache LRU end 2024-05-08T18:05:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.179+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(71 notify.4 [notify cookie 93857781583152 out=8b] v0'0 uv132 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6f34040040 con 0x555cf7b5b850 2024-05-08T18:05:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.179+0000 7f6e1e7fc640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f6d70003400 con 0x555cf7b5b850 2024-05-08T18:05:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.180+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(72 notify.4 [notify-ack] v0'0 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x555cf7b5b850 2024-05-08T18:05:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.180+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== watch-notify(notify_complete (2) cookie 93857781583152 notify 115964117009 ret 0) ==== 42+0+48 (crc 0 0 0) 0x555cf7bb04d0 con 0x555cf7b5b850 2024-05-08T18:05:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.180+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:73 8.9 8:91b71841:root::myfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7bb11e0 con 0x7f6f3002e910 2024-05-08T18:05:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.181+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(73 myfoo [call,delete] v27'27 uv27 ondisk = 0) ==== 191+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x7f6f3002e910 2024-05-08T18:05:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.181+0000 7f6f3d698800 10 removing default.rgw.meta+root+myfoo from cache 2024-05-08T18:05:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.181+0000 7f6f3d698800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-05-08T18:05:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.181+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [notify cookie 93857781583152 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7bb1150 con 0x555cf7b60240 2024-05-08T18:05:09.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.181+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== watch-notify(notify (1) cookie 93857781307360 notify 115964117004 ret 0) ==== 191+0+0 (crc 0 0 0) 0x7f6f2c080040 con 0x555cf7b60240 2024-05-08T18:05:09.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.181+0000 7f6e1effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117004 cookie 93857781307360 notifier 4536 bl.length()=149 2024-05-08T18:05:09.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.181+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(74 notify.6 [notify cookie 93857781583152 out=8b] v0'0 uv132 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.181+0000 7f6e1effd640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f6e00003ea0 con 0x555cf7b60240 2024-05-08T18:05:09.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.182+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(75 notify.6 [notify-ack] v0'0 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.182+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== watch-notify(notify_complete (2) cookie 93857781583152 notify 115964117004 ret 0) ==== 42+0+48 (crc 0 0 0) 0x555cf7b20ab0 con 0x555cf7b60240 2024-05-08T18:05:09.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.182+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7bb11e0 con 0x555cf7b60240 2024-05-08T18:05:09.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.183+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(76 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call,delete] v27'4 uv4 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.183+0000 7f6f3d698800 10 removing default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 from cache 2024-05-08T18:05:09.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.183+0000 7f6f3d698800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1, ofs0, ns] 2024-05-08T18:05:09.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.183+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [notify cookie 93857781551856 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x555cf7bb11e0 con 0x555cf7b60240 2024-05-08T18:05:09.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.183+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== watch-notify(notify (1) cookie 93857781307360 notify 115964117005 ret 0) ==== 248+0+0 (crc 0 0 0) 0x555cf7b20ab0 con 0x555cf7b60240 2024-05-08T18:05:09.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.183+0000 7f6e1e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117005 cookie 93857781307360 notifier 4536 bl.length()=206 2024-05-08T18:05:09.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.183+0000 7f6e1e7fc640 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f6d70003400 con 0x555cf7b60240 2024-05-08T18:05:09.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.183+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== osd_op_reply(77 notify.6 [notify cookie 93857781551856 out=8b] v0'0 uv132 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.183+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 31 ==== osd_op_reply(78 notify.6 [notify-ack] v0'0 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.183+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 32 ==== watch-notify(notify_complete (2) cookie 93857781551856 notify 115964117005 ret 0) ==== 42+0+48 (crc 0 0 0) 0x555cf7b1de00 con 0x555cf7b60240 2024-05-08T18:05:09.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.184+0000 7f6df1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:09.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.184+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:79 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93857781314480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b8e9f0 con 0x555cf7b5b850 2024-05-08T18:05:09.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.185+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(79 data_loggenerations_metadata [watch unwatch cookie 93857781314480] v27'129 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x555cf7b5b850 2024-05-08T18:05:09.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.185+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:80 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93857781288784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b8e9f0 con 0x7f6f3002e910 2024-05-08T18:05:09.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.186+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(80 notify.0 [watch unwatch cookie 93857781288784] v27'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x7f6f3002e910 2024-05-08T18:05:09.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.186+0000 7f6f3d698800 20 remove_watcher() i=0 2024-05-08T18:05:09.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.186+0000 7f6f3d698800 2 removed watcher, disabling cache 2024-05-08T18:05:09.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.186+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93857781293248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b8e9f0 con 0x555cf7b5b850 2024-05-08T18:05:09.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.187+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(81 notify.1 [watch unwatch cookie 93857781293248] v27'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x555cf7b5b850 2024-05-08T18:05:09.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.187+0000 7f6f3d698800 20 remove_watcher() i=1 2024-05-08T18:05:09.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.187+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93857781286272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b8e9f0 con 0x555cf7b64ba0 2024-05-08T18:05:09.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.188+0000 7f6f3a996640 1 -- 172.21.15.50:0/692100361 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(82 notify.2 [watch unwatch cookie 93857781286272] v27'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f30037040 con 0x555cf7b64ba0 2024-05-08T18:05:09.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.188+0000 7f6f3d698800 20 remove_watcher() i=2 2024-05-08T18:05:09.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.188+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93857781300352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b6a0c0 con 0x7f6f3002e910 2024-05-08T18:05:09.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.189+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 notify.3 [watch unwatch cookie 93857781300352] v27'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x7f6f3002e910 2024-05-08T18:05:09.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.189+0000 7f6f3d698800 20 remove_watcher() i=3 2024-05-08T18:05:09.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.189+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:84 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93857781304688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b68580 con 0x555cf7b5b850 2024-05-08T18:05:09.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.190+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(84 notify.4 [watch unwatch cookie 93857781304688] v27'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f34040040 con 0x555cf7b5b850 2024-05-08T18:05:09.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.190+0000 7f6f3d698800 20 remove_watcher() i=4 2024-05-08T18:05:09.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.190+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:85 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93857781302144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b6a0c0 con 0x555cf7b60240 2024-05-08T18:05:09.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.191+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 33 ==== osd_op_reply(85 notify.5 [watch unwatch cookie 93857781302144] v27'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.191+0000 7f6f3d698800 20 remove_watcher() i=5 2024-05-08T18:05:09.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.191+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:86 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93857781307360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b68580 con 0x555cf7b60240 2024-05-08T18:05:09.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.192+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 34 ==== osd_op_reply(86 notify.6 [watch unwatch cookie 93857781307360] v27'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.192+0000 7f6f3d698800 20 remove_watcher() i=6 2024-05-08T18:05:09.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.192+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:87 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93857781312096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x555cf7b6a0c0 con 0x555cf7b60240 2024-05-08T18:05:09.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3b197640 1 -- 172.21.15.50:0/692100361 <== osd.0 v2:172.21.15.46:6800/337143975 35 ==== osd_op_reply(87 notify.7 [watch unwatch cookie 93857781312096] v27'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6f2c05a040 con 0x555cf7b60240 2024-05-08T18:05:09.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 20 remove_watcher() i=7 2024-05-08T18:05:09.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555cf7b60240 msgr2=0x555cf7b62630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555cf7b60240 0x555cf7b62630 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555cf7b5b850 msgr2=0x555cf7b5dd10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555cf7b5b850 0x555cf7b5dd10 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6f3002e910 msgr2=0x7f6f3003a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6f3002e910 0x7f6f3003a3c0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555cf7b64ba0 msgr2=0x555cf7b67060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555cf7b64ba0 0x555cf7b67060 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6e0403bba0 msgr2=0x7f6e0403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6e0403bba0 0x7f6e0403e060 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000ffd0 tx=0x7f6f30023040 comp rx=0 tx=0).stop 2024-05-08T18:05:09.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b48dc0 msgr2=0x555cf7b50160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3b998640 1 -- 172.21.15.50:0/692100361 reap_dead start 2024-05-08T18:05:09.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b48dc0 0x555cf7b50160 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c08fa00 tx=0x7f6f2c0a3000 comp rx=0 tx=0).stop 2024-05-08T18:05:09.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 shutdown_connections 2024-05-08T18:05:09.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 1 --2- 172.21.15.50:0/692100361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b48dc0 0x555cf7b50160 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 >> 172.21.15.50:0/692100361 conn(0x555cf7b450d0 msgr2=0x555cf7b42730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.193+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 shutdown_connections 2024-05-08T18:05:09.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 -- 172.21.15.50:0/692100361 wait complete. 2024-05-08T18:05:09.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 -- 172.21.15.50:0/4224538954 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6e1403bb50 msgr2=0x7f6e1403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 --2- 172.21.15.50:0/4224538954 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6e1403bb50 0x7f6e1403e010 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x555cf7b25220 tx=0x7f6f30006000 comp rx=0 tx=0).stop 2024-05-08T18:05:09.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 -- 172.21.15.50:0/4224538954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b3bdf0 msgr2=0x555cf7b3c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 --2- 172.21.15.50:0/4224538954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b3bdf0 0x555cf7b3c210 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009af0 tx=0x7f6f2c0096e0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 -- 172.21.15.50:0/4224538954 shutdown_connections 2024-05-08T18:05:09.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 --2- 172.21.15.50:0/4224538954 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6e1403bb50 0x7f6e1403e010 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 --2- 172.21.15.50:0/4224538954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7b3bdf0 0x555cf7b3c210 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 -- 172.21.15.50:0/4224538954 >> 172.21.15.50:0/4224538954 conn(0x555cf7b2fe90 msgr2=0x555cf7b30290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 -- 172.21.15.50:0/4224538954 shutdown_connections 2024-05-08T18:05:09.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 -- 172.21.15.50:0/4224538954 wait complete. 2024-05-08T18:05:09.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 -- 172.21.15.50:0/2017692848 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555cf7b27080 msgr2=0x555cf7b29470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555cf7b27080 0x555cf7b29470 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 -- 172.21.15.50:0/2017692848 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555cf7a22b50 msgr2=0x555cf7b24870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555cf7a22b50 0x555cf7b24870 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 -- 172.21.15.50:0/2017692848 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555cf7b2af80 msgr2=0x555cf7b2d370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555cf7b2af80 0x555cf7b2d370 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 -- 172.21.15.50:0/2017692848 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6e2003bba0 msgr2=0x7f6e2003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6e2003bba0 0x7f6e2003e060 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6f30000c00 tx=0x7f6f300013b0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 -- 172.21.15.50:0/2017692848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a21860 msgr2=0x555cf7b1d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 --2- 172.21.15.50:0/2017692848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555cf7a21860 0x555cf7b1d630 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c000c00 tx=0x7f6f2c002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3b998640 1 -- 172.21.15.50:0/2017692848 reap_dead start 2024-05-08T18:05:09.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 -- 172.21.15.50:0/2017692848 shutdown_connections 2024-05-08T18:05:09.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.194+0000 7f6f3d698800 1 -- 172.21.15.50:0/2017692848 >> 172.21.15.50:0/2017692848 conn(0x555cf7a1c2f0 msgr2=0x555cf7b0ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.195+0000 7f6f3d698800 1 -- 172.21.15.50:0/2017692848 shutdown_connections 2024-05-08T18:05:09.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.195+0000 7f6f3d698800 1 -- 172.21.15.50:0/2017692848 wait complete. 2024-05-08T18:05:09.212 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:testx/myfoo'] 2024-05-08T18:05:09.212 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:testx/myfoo'] 2024-05-08T18:05:09.212 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:testx/myfoo 2024-05-08T18:05:09.256 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:09.256 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:09.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.258+0000 7f7f112bb800 1 Processor -- start 2024-05-08T18:05:09.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.258+0000 7f7f112bb800 1 -- start start 2024-05-08T18:05:09.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.258+0000 7f7f112bb800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55946a6ebc90 0x55946a6ee080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.258+0000 7f7f112bb800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55946a6ee5c0 con 0x55946a6e8180 2024-05-08T18:05:09.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.258+0000 7f7f112bb800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55946a587e20 con 0x55946a6ebc90 2024-05-08T18:05:09.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.259+0000 7f7f0edba640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55946a6e8180 0x55946a6ea570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43246/0 (socket says 172.21.15.50:43246) 2024-05-08T18:05:09.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.259+0000 7f7f0edba640 1 -- 172.21.15.50:0/2201896 learned_addr learned my addr 172.21.15.50:0/2201896 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:09.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.259+0000 7f7f0e5b9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55946a6ebc90 0x55946a6ee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.259+0000 7f7f0ddb8640 1 -- 172.21.15.50:0/2201896 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3023501287 0 0) 0x55946a6ee5c0 con 0x55946a6e8180 2024-05-08T18:05:09.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.259+0000 7f7f0ddb8640 1 -- 172.21.15.50:0/2201896 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ef8001e10 con 0x55946a6e8180 2024-05-08T18:05:09.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.259+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2201896 >> v1:172.21.15.46:6789/0 conn(0x55946a6e8180 legacy=0x55946a6ea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.259+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2201896 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946a6ef7a0 con 0x55946a6ebc90 2024-05-08T18:05:09.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.261+0000 7f7f0e5b9640 1 --2- 172.21.15.50:0/2201896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6ebc90 0x55946a6ee080 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400a470 tx=0x7f7f0401cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0feb115fe40351a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.261+0000 7f7f0ddb8640 1 -- 172.21.15.50:0/2201896 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f04057070 con 0x55946a6ebc90 2024-05-08T18:05:09.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.261+0000 7f7f0ddb8640 1 -- 172.21.15.50:0/2201896 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f04052570 con 0x55946a6ebc90 2024-05-08T18:05:09.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.261+0000 7f7f0ddb8640 1 -- 172.21.15.50:0/2201896 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f04052870 con 0x55946a6ebc90 2024-05-08T18:05:09.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.262+0000 7f7f112bb800 1 -- 172.21.15.50:0/2201896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6ebc90 msgr2=0x55946a6ee080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.262+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2201896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6ebc90 0x55946a6ee080 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400a470 tx=0x7f7f0401cdb0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.262+0000 7f7f112bb800 1 -- 172.21.15.50:0/2201896 shutdown_connections 2024-05-08T18:05:09.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.262+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2201896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6ebc90 0x55946a6ee080 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.262+0000 7f7f112bb800 1 -- 172.21.15.50:0/2201896 >> 172.21.15.50:0/2201896 conn(0x55946a6e1ad0 msgr2=0x55946a6e3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.262+0000 7f7f112bb800 1 -- 172.21.15.50:0/2201896 shutdown_connections 2024-05-08T18:05:09.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.262+0000 7f7f112bb800 1 -- 172.21.15.50:0/2201896 wait complete. 2024-05-08T18:05:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.277+0000 7f7f112bb800 1 Processor -- start 2024-05-08T18:05:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.277+0000 7f7f112bb800 1 -- start start 2024-05-08T18:05:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.277+0000 7f7f112bb800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6e8180 0x55946a7d8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.277+0000 7f7f112bb800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55946a632a10 con 0x55946a6e8180 2024-05-08T18:05:09.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.277+0000 7f7f0edba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6e8180 0x55946a7d8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.277+0000 7f7f0edba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6e8180 0x55946a7d8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38238/0 (socket says 172.21.15.50:38238) 2024-05-08T18:05:09.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.277+0000 7f7f0edba640 1 -- 172.21.15.50:0/3298689630 learned_addr learned my addr 172.21.15.50:0/3298689630 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.278+0000 7f7f0edba640 1 -- 172.21.15.50:0/3298689630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946a7d8fe0 con 0x55946a6e8180 2024-05-08T18:05:09.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.278+0000 7f7f0edba640 1 --2- 172.21.15.50:0/3298689630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6e8180 0x55946a7d8a10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7f00001de0 tx=0x7f7f0000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20dce03206930976 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.278+0000 7f7dfce09640 1 -- 172.21.15.50:0/3298689630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f00012020 con 0x55946a6e8180 2024-05-08T18:05:09.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.278+0000 7f7dfce09640 1 -- 172.21.15.50:0/3298689630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0000e470 con 0x55946a6e8180 2024-05-08T18:05:09.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.278+0000 7f7dfce09640 1 -- 172.21.15.50:0/3298689630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f0000e7b0 con 0x55946a6e8180 2024-05-08T18:05:09.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.279+0000 7f7f112bb800 1 -- 172.21.15.50:0/3298689630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6e8180 msgr2=0x55946a7d8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.279+0000 7f7f112bb800 1 --2- 172.21.15.50:0/3298689630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6e8180 0x55946a7d8a10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f7f00001de0 tx=0x7f7f0000b6c0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.279+0000 7f7f112bb800 1 -- 172.21.15.50:0/3298689630 shutdown_connections 2024-05-08T18:05:09.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.279+0000 7f7f112bb800 1 --2- 172.21.15.50:0/3298689630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6e8180 0x55946a7d8a10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.279+0000 7f7f112bb800 1 -- 172.21.15.50:0/3298689630 >> 172.21.15.50:0/3298689630 conn(0x55946a6e1ad0 msgr2=0x55946a7d4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.279+0000 7f7f112bb800 1 -- 172.21.15.50:0/3298689630 shutdown_connections 2024-05-08T18:05:09.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.279+0000 7f7f112bb800 1 -- 172.21.15.50:0/3298689630 wait complete. 2024-05-08T18:05:09.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.279+0000 7f7f112bb800 1 Processor -- start 2024-05-08T18:05:09.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.279+0000 7f7f112bb800 1 -- start start 2024-05-08T18:05:09.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.279+0000 7f7f112bb800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6e8180 0x55946a7e0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.279+0000 7f7f112bb800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55946a6eea90 con 0x55946a6e8180 2024-05-08T18:05:09.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.280+0000 7f7f0edba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6e8180 0x55946a7e0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.280+0000 7f7f0edba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6e8180 0x55946a7e0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38244/0 (socket says 172.21.15.50:38244) 2024-05-08T18:05:09.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.280+0000 7f7f0edba640 1 -- 172.21.15.50:0/2995376839 learned_addr learned my addr 172.21.15.50:0/2995376839 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.280+0000 7f7f0edba640 1 -- 172.21.15.50:0/2995376839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946a7e0aa0 con 0x55946a6e8180 2024-05-08T18:05:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.280+0000 7f7f0edba640 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6e8180 0x55946a7e0560 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000b700 tx=0x7f7f000040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.280+0000 7f7dfb606640 1 -- 172.21.15.50:0/2995376839 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f00012030 con 0x55946a6e8180 2024-05-08T18:05:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.281+0000 7f7dfb606640 1 -- 172.21.15.50:0/2995376839 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0000be70 con 0x55946a6e8180 2024-05-08T18:05:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.281+0000 7f7f112bb800 1 -- 172.21.15.50:0/2995376839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55946a7e0d30 con 0x55946a6e8180 2024-05-08T18:05:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.281+0000 7f7dfb606640 1 -- 172.21.15.50:0/2995376839 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f0001a8d0 con 0x55946a6e8180 2024-05-08T18:05:09.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.281+0000 7f7f112bb800 1 -- 172.21.15.50:0/2995376839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55946a7e39e0 con 0x55946a6e8180 2024-05-08T18:05:09.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.281+0000 7f7dfb606640 1 -- 172.21.15.50:0/2995376839 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7f00015020 con 0x55946a6e8180 2024-05-08T18:05:09.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.282+0000 7f7dfb606640 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7df403bba0 0x7f7df403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.282+0000 7f7dfb606640 1 -- 172.21.15.50:0/2995376839 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7f00020030 con 0x55946a6e8180 2024-05-08T18:05:09.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.282+0000 7f7f0e5b9640 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7df403bba0 0x7f7df403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.282+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55946a6ebc90 0x55946a7e77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.282+0000 7f7f112bb800 1 -- 172.21.15.50:0/2995376839 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a7e7d20 con 0x55946a6ebc90 2024-05-08T18:05:09.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.282+0000 7f7f0f5bb640 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55946a6ebc90 0x55946a7e77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.282+0000 7f7f0e5b9640 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7df403bba0 0x7f7df403e060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7f04054000 tx=0x7f7f0404e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.282+0000 7f7f0f5bb640 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55946a6ebc90 0x55946a7e77a0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.283+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2995376839 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a6ebc90 2024-05-08T18:05:09.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.283+0000 7f7f112bb800 1 -- 172.21.15.50:0/2995376839 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a7e4d70 con 0x55946a6ebc90 2024-05-08T18:05:09.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.283+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2995376839 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a6ebc90 2024-05-08T18:05:09.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.283+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55946a7ea030 0x55946a7ec420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.283+0000 7f7f112bb800 1 -- 172.21.15.50:0/2995376839 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a7ecb10 con 0x55946a7ea030 2024-05-08T18:05:09.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.283+0000 7f7f0edba640 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55946a7ea030 0x55946a7ec420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.283+0000 7f7f0edba640 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55946a7ea030 0x55946a7ec420 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.284+0000 7f7f0edba640 1 -- 172.21.15.50:0/2995376839 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7f00018430 con 0x55946a7ea030 2024-05-08T18:05:09.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.284+0000 7f7f112bb800 1 -- 172.21.15.50:0/2995376839 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a7e8ed0 con 0x55946a6ebc90 2024-05-08T18:05:09.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.284+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2995376839 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a6ebc90 2024-05-08T18:05:09.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.284+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55946a7edf70 0x55946a7f0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.284+0000 7f7f112bb800 1 -- 172.21.15.50:0/2995376839 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a7f0a60 con 0x55946a7edf70 2024-05-08T18:05:09.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.284+0000 7f7f0e5b9640 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55946a7edf70 0x55946a7f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.285+0000 7f7f0e5b9640 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55946a7edf70 0x55946a7f0360 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.285+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2995376839 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7f04052940 con 0x55946a7edf70 2024-05-08T18:05:09.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.285+0000 7f7f112bb800 1 Processor -- start 2024-05-08T18:05:09.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.285+0000 7f7f112bb800 1 -- start start 2024-05-08T18:05:09.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.285+0000 7f7f112bb800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a7fedd0 0x55946a7ff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.285+0000 7f7f112bb800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55946a5d9330 con 0x55946a7fedd0 2024-05-08T18:05:09.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.286+0000 7f7f0edba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a7fedd0 0x55946a7ff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.286+0000 7f7f0edba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a7fedd0 0x55946a7ff1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38254/0 (socket says 172.21.15.50:38254) 2024-05-08T18:05:09.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.286+0000 7f7f0edba640 1 -- 172.21.15.50:0/2660513014 learned_addr learned my addr 172.21.15.50:0/2660513014 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.286+0000 7f7f0edba640 1 -- 172.21.15.50:0/2660513014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946a7ff730 con 0x55946a7fedd0 2024-05-08T18:05:09.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.286+0000 7f7f0edba640 1 --2- 172.21.15.50:0/2660513014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a7fedd0 0x55946a7ff1f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7f00022860 tx=0x7f7f0000c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.286+0000 7f7df8e01640 1 -- 172.21.15.50:0/2660513014 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f00022b00 con 0x55946a7fedd0 2024-05-08T18:05:09.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.286+0000 7f7df8e01640 1 -- 172.21.15.50:0/2660513014 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f00022ca0 con 0x55946a7fedd0 2024-05-08T18:05:09.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.286+0000 7f7f112bb800 1 -- 172.21.15.50:0/2660513014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55946a8001a0 con 0x55946a7fedd0 2024-05-08T18:05:09.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.286+0000 7f7df8e01640 1 -- 172.21.15.50:0/2660513014 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f00016570 con 0x55946a7fedd0 2024-05-08T18:05:09.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.286+0000 7f7f112bb800 1 -- 172.21.15.50:0/2660513014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55946a8001a0 con 0x55946a7fedd0 2024-05-08T18:05:09.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.287+0000 7f7df8e01640 1 -- 172.21.15.50:0/2660513014 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7f00016710 con 0x55946a7fedd0 2024-05-08T18:05:09.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.287+0000 7f7df8e01640 1 --2- 172.21.15.50:0/2660513014 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7de803bba0 0x7f7de803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.287+0000 7f7df8e01640 1 -- 172.21.15.50:0/2660513014 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7f00038c80 con 0x55946a7fedd0 2024-05-08T18:05:09.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.287+0000 7f7f0e5b9640 1 --2- 172.21.15.50:0/2660513014 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7de803bba0 0x7f7de803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.288+0000 7f7f112bb800 1 Processor -- start 2024-05-08T18:05:09.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.288+0000 7f7f112bb800 1 -- start start 2024-05-08T18:05:09.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.288+0000 7f7f0e5b9640 1 --2- 172.21.15.50:0/2660513014 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7de803bba0 0x7f7de803e060 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55946a7e85e0 tx=0x7f7f04005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.288+0000 7f7f112bb800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a80bce0 0x55946a80c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.288+0000 7f7f112bb800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55946a7d9360 con 0x55946a80bce0 2024-05-08T18:05:09.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.288+0000 7f7f0edba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a80bce0 0x55946a80c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.288+0000 7f7f0edba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a80bce0 0x55946a80c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38270/0 (socket says 172.21.15.50:38270) 2024-05-08T18:05:09.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.288+0000 7f7f0edba640 1 -- 172.21.15.50:0/4038843335 learned_addr learned my addr 172.21.15.50:0/4038843335 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.288+0000 7f7f0edba640 1 -- 172.21.15.50:0/4038843335 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946a80c6d0 con 0x55946a80bce0 2024-05-08T18:05:09.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.288+0000 7f7f0edba640 1 --2- 172.21.15.50:0/4038843335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a80bce0 0x55946a80c100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7f00019fd0 tx=0x7f7f00043f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97067d1dfa1e83b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.288+0000 7f7df27fc640 1 -- 172.21.15.50:0/4038843335 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f00024930 con 0x55946a80bce0 2024-05-08T18:05:09.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.288+0000 7f7df27fc640 1 -- 172.21.15.50:0/4038843335 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f00024cd0 con 0x55946a80bce0 2024-05-08T18:05:09.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.288+0000 7f7df27fc640 1 -- 172.21.15.50:0/4038843335 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f00043680 con 0x55946a80bce0 2024-05-08T18:05:09.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.289+0000 7f7f112bb800 1 -- 172.21.15.50:0/4038843335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a80bce0 msgr2=0x55946a80c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.289+0000 7f7f112bb800 1 --2- 172.21.15.50:0/4038843335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a80bce0 0x55946a80c100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7f00019fd0 tx=0x7f7f00043f40 comp rx=0 tx=0).stop 2024-05-08T18:05:09.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.289+0000 7f7f112bb800 1 -- 172.21.15.50:0/4038843335 shutdown_connections 2024-05-08T18:05:09.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.289+0000 7f7f112bb800 1 --2- 172.21.15.50:0/4038843335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a80bce0 0x55946a80c100 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.289+0000 7f7f112bb800 1 -- 172.21.15.50:0/4038843335 >> 172.21.15.50:0/4038843335 conn(0x55946a808040 msgr2=0x55946a8060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.289+0000 7f7f112bb800 1 -- 172.21.15.50:0/4038843335 shutdown_connections 2024-05-08T18:05:09.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.289+0000 7f7f112bb800 1 -- 172.21.15.50:0/4038843335 wait complete. 2024-05-08T18:05:09.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.289+0000 7f7f112bb800 1 Processor -- start 2024-05-08T18:05:09.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.289+0000 7f7f112bb800 1 -- start start 2024-05-08T18:05:09.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.290+0000 7f7f112bb800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a80bce0 0x55946a6e7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.290+0000 7f7f112bb800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55946a80cd80 con 0x55946a80bce0 2024-05-08T18:05:09.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.290+0000 7f7f0edba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a80bce0 0x55946a6e7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.290+0000 7f7f0edba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a80bce0 0x55946a6e7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38284/0 (socket says 172.21.15.50:38284) 2024-05-08T18:05:09.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.290+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 learned_addr learned my addr 172.21.15.50:0/2370516345 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.290+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55946a6e61b0 con 0x55946a80bce0 2024-05-08T18:05:09.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.290+0000 7f7f0edba640 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a80bce0 0x55946a6e7ac0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7f0004bf30 tx=0x7f7f00018c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.290+0000 7f7df0ff9640 1 -- 172.21.15.50:0/2370516345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f0006d070 con 0x55946a80bce0 2024-05-08T18:05:09.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.290+0000 7f7df0ff9640 1 -- 172.21.15.50:0/2370516345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f00043a10 con 0x55946a80bce0 2024-05-08T18:05:09.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.290+0000 7f7df0ff9640 1 -- 172.21.15.50:0/2370516345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f000526b0 con 0x55946a80bce0 2024-05-08T18:05:09.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.290+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55946a6e61b0 con 0x55946a80bce0 2024-05-08T18:05:09.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.290+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55946a6e61b0 con 0x55946a80bce0 2024-05-08T18:05:09.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.291+0000 7f7df0ff9640 1 -- 172.21.15.50:0/2370516345 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7f00070070 con 0x55946a80bce0 2024-05-08T18:05:09.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.291+0000 7f7df0ff9640 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7dd803bba0 0x7f7dd803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.291+0000 7f7df0ff9640 1 -- 172.21.15.50:0/2370516345 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7f00059030 con 0x55946a80bce0 2024-05-08T18:05:09.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.292+0000 7f7f0e5b9640 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7dd803bba0 0x7f7dd803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.292+0000 7f7f112bb800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.292+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55946a820a80 0x55946a822f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.292+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a823690 con 0x55946a820a80 2024-05-08T18:05:09.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.292+0000 7f7f0f5bb640 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55946a820a80 0x55946a822f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.292+0000 7f7f0e5b9640 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7dd803bba0 0x7f7dd803e060 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7f04052fd0 tx=0x7f7f0404f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.292+0000 7f7f0f5bb640 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55946a820a80 0x55946a822f40 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.292+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a820a80 2024-05-08T18:05:09.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.292+0000 7f7f112bb800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.292+0000 7f7f112bb800 20 realm 2024-05-08T18:05:09.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.292+0000 7f7f112bb800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.292+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a823aa0 con 0x55946a820a80 2024-05-08T18:05:09.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.293+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a820a80 2024-05-08T18:05:09.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.293+0000 7f7f112bb800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.293+0000 7f7f112bb800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:09.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.293+0000 7f7f112bb800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.293+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a824370 con 0x55946a820a80 2024-05-08T18:05:09.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.293+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a820a80 2024-05-08T18:05:09.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.293+0000 7f7f112bb800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.293+0000 7f7f112bb800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.293+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a824710 con 0x55946a820a80 2024-05-08T18:05:09.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.293+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a820a80 2024-05-08T18:05:09.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.293+0000 7f7f112bb800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:09.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.293+0000 7f7f112bb800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.293+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55946a825200 0x55946a8275f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.293+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a827d50 con 0x55946a825200 2024-05-08T18:05:09.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.293+0000 7f7f0edba640 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55946a825200 0x55946a8275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.294+0000 7f7f0edba640 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55946a825200 0x55946a8275f0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.294+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.294+0000 7f7f112bb800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:09.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.294+0000 7f7f112bb800 20 searching for the correct realm 2024-05-08T18:05:09.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.294+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55946a829e00 0x55946a82c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.294+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55946a82c990 con 0x55946a829e00 2024-05-08T18:05:09.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.294+0000 7f7f0e5b9640 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55946a829e00 0x55946a82c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.294+0000 7f7f0e5b9640 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55946a829e00 0x55946a82c2c0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.295+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f04081070 con 0x55946a829e00 2024-05-08T18:05:09.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.295+0000 7f7f0e5b9640 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7f04076040 0x7f7f04083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.295+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55946a82c990 con 0x7f7f04076040 2024-05-08T18:05:09.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.295+0000 7f7f0f5bb640 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7f04076040 0x7f7f04083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.295+0000 7f7f0f5bb640 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7f04076040 0x7f7f04083410 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.295+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0800a4a0 con 0x7f7f04076040 2024-05-08T18:05:09.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.295+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55946a824710 con 0x55946a825200 2024-05-08T18:05:09.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.296+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.296+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55946a824710 con 0x55946a820a80 2024-05-08T18:05:09.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.296+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a820a80 2024-05-08T18:05:09.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.296+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55946a824710 con 0x55946a820a80 2024-05-08T18:05:09.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.296+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a820a80 2024-05-08T18:05:09.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.296+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55946a824710 con 0x55946a825200 2024-05-08T18:05:09.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.296+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.296+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55946a824710 con 0x55946a825200 2024-05-08T18:05:09.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.297+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.297+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55946a824710 con 0x7f7f04076040 2024-05-08T18:05:09.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.297+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0800a4a0 con 0x7f7f04076040 2024-05-08T18:05:09.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.297+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55946a824710 con 0x55946a829e00 2024-05-08T18:05:09.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.297+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f04081070 con 0x55946a829e00 2024-05-08T18:05:09.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.297+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55946a824710 con 0x55946a820a80 2024-05-08T18:05:09.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.297+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a820a80 2024-05-08T18:05:09.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.297+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55946a824710 con 0x55946a825200 2024-05-08T18:05:09.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.298+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.298+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55946a824710 con 0x55946a820a80 2024-05-08T18:05:09.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.298+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a820a80 2024-05-08T18:05:09.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.298+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55946a824710 con 0x7f7f04076040 2024-05-08T18:05:09.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.298+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7f0800a4a0 con 0x7f7f04076040 2024-05-08T18:05:09.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.298+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55946a824710 con 0x55946a829e00 2024-05-08T18:05:09.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.298+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f04081070 con 0x55946a829e00 2024-05-08T18:05:09.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.298+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x55946a829e00 2024-05-08T18:05:09.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.299+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f04081070 con 0x55946a829e00 2024-05-08T18:05:09.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.299+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x55946a825200 2024-05-08T18:05:09.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.299+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.299+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x55946a829e00 2024-05-08T18:05:09.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.299+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f04081070 con 0x55946a829e00 2024-05-08T18:05:09.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.299+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x55946a829e00 2024-05-08T18:05:09.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.299+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7f04081070 con 0x55946a829e00 2024-05-08T18:05:09.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.299+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x55946a820a80 2024-05-08T18:05:09.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.300+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a820a80 2024-05-08T18:05:09.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.300+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x55946a820a80 2024-05-08T18:05:09.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.300+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a820a80 2024-05-08T18:05:09.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.300+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x55946a825200 2024-05-08T18:05:09.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.300+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.300+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x55946a829e00 2024-05-08T18:05:09.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.300+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f04081070 con 0x55946a829e00 2024-05-08T18:05:09.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.300+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x7f7f04076040 2024-05-08T18:05:09.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.301+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0800a4a0 con 0x7f7f04076040 2024-05-08T18:05:09.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.301+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x55946a820a80 2024-05-08T18:05:09.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.301+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a820a80 2024-05-08T18:05:09.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.301+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x55946a825200 2024-05-08T18:05:09.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.301+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.301+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x55946a829e00 2024-05-08T18:05:09.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.301+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f04081070 con 0x55946a829e00 2024-05-08T18:05:09.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.301+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x55946a825200 2024-05-08T18:05:09.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.302+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.302+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x7f7f04076040 2024-05-08T18:05:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.302+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0800a4a0 con 0x7f7f04076040 2024-05-08T18:05:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.302+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x55946a829e00 2024-05-08T18:05:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.302+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f04081070 con 0x55946a829e00 2024-05-08T18:05:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.302+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x55946a825200 2024-05-08T18:05:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.302+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.303+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x55946a820a80 2024-05-08T18:05:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.303+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a820a80 2024-05-08T18:05:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.303+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7f04090040 con 0x7f7f04076040 2024-05-08T18:05:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.303+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7f0800a4a0 con 0x7f7f04076040 2024-05-08T18:05:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.303+0000 7f7f112bb800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.303+0000 7f7f112bb800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.303+0000 7f7f112bb800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.303+0000 7f7f112bb800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.303+0000 7f7f112bb800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.303+0000 7f7f112bb800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.303+0000 7f7f112bb800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.303+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a828700 con 0x55946a820a80 2024-05-08T18:05:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.303+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a820a80 2024-05-08T18:05:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.303+0000 7f7f112bb800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.303+0000 7f7f112bb800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.303+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a828700 con 0x55946a820a80 2024-05-08T18:05:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7f0800a4a0 con 0x55946a820a80 2024-05-08T18:05:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a8286d0 con 0x55946a829e00 2024-05-08T18:05:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7f04081070 con 0x55946a829e00 2024-05-08T18:05:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 20 zone default found 2024-05-08T18:05:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 4 Realm: () 2024-05-08T18:05:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 20 zonegroup default 2024-05-08T18:05:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a828fa0 con 0x55946a825200 2024-05-08T18:05:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a82e7f0 con 0x55946a829e00 2024-05-08T18:05:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7f04081070 con 0x55946a829e00 2024-05-08T18:05:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.304+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a82eb90 con 0x55946a829e00 2024-05-08T18:05:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.305+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7f04081070 con 0x55946a829e00 2024-05-08T18:05:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.305+0000 7f7f112bb800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.305+0000 7f7f112bb800 20 started sync module instance, tier type = 2024-05-08T18:05:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.305+0000 7f7f112bb800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.305+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a82dc90 con 0x7f7f04076040 2024-05-08T18:05:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.306+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f0800a4a0 con 0x7f7f04076040 2024-05-08T18:05:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.306+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94095930483136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a82dc90 con 0x7f7f04076040 2024-05-08T18:05:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.306+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a82f590 con 0x55946a820a80 2024-05-08T18:05:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.307+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94095930483136] v27'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f08040040 con 0x7f7f04076040 2024-05-08T18:05:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.307+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f08040040 con 0x55946a820a80 2024-05-08T18:05:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.307+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94095930486944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a82dc90 con 0x55946a820a80 2024-05-08T18:05:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.307+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a830010 con 0x55946a829e00 2024-05-08T18:05:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.308+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f04081070 con 0x55946a829e00 2024-05-08T18:05:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.308+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94095930489632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a8308e0 con 0x55946a829e00 2024-05-08T18:05:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.308+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a831370 con 0x7f7f04076040 2024-05-08T18:05:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.309+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94095930486944] v27'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f08040040 con 0x55946a820a80 2024-05-08T18:05:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.309+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94095930489632] v27'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f04081070 con 0x55946a829e00 2024-05-08T18:05:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.310+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f08040040 con 0x7f7f04076040 2024-05-08T18:05:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.310+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94095930491952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a830e70 con 0x7f7f04076040 2024-05-08T18:05:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.310+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a831c40 con 0x55946a820a80 2024-05-08T18:05:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.311+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94095930491952] v27'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f08040040 con 0x7f7f04076040 2024-05-08T18:05:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.311+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f08040040 con 0x55946a820a80 2024-05-08T18:05:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.311+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94095930480608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a831bf0 con 0x55946a820a80 2024-05-08T18:05:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.311+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a8325f0 con 0x55946a825200 2024-05-08T18:05:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.312+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94095930480608] v27'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f08040040 con 0x55946a820a80 2024-05-08T18:05:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.312+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.312+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94095930496832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a831b60 con 0x55946a825200 2024-05-08T18:05:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.312+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a8333a0 con 0x55946a825200 2024-05-08T18:05:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.313+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94095930496832] v27'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.313+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.313+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94095930502080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a833c70 con 0x55946a825200 2024-05-08T18:05:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.313+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a834630 con 0x55946a825200 2024-05-08T18:05:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.314+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94095930502080] v27'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.314+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.314+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94095930507648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a834a50 con 0x55946a825200 2024-05-08T18:05:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.314+0000 7f7f112bb800 20 add_watcher() i=0 2024-05-08T18:05:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.314+0000 7f7f112bb800 20 add_watcher() i=1 2024-05-08T18:05:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.314+0000 7f7f112bb800 20 add_watcher() i=2 2024-05-08T18:05:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.314+0000 7f7f112bb800 20 add_watcher() i=3 2024-05-08T18:05:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.314+0000 7f7f112bb800 20 add_watcher() i=4 2024-05-08T18:05:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.314+0000 7f7f112bb800 20 add_watcher() i=5 2024-05-08T18:05:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.314+0000 7f7f112bb800 20 add_watcher() i=6 2024-05-08T18:05:09.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.315+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94095930507648] v27'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.315+0000 7f7f112bb800 20 add_watcher() i=7 2024-05-08T18:05:09.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.315+0000 7f7f112bb800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:09.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.315+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a837ce0 con 0x55946a820a80 2024-05-08T18:05:09.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.315+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7f08040040 con 0x55946a820a80 2024-05-08T18:05:09.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.315+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94095930509072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a834fe0 con 0x55946a820a80 2024-05-08T18:05:09.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.316+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94095930509072] v27'130 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7f08040040 con 0x55946a820a80 2024-05-08T18:05:09.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.316+0000 7f7dc57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:09.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.316+0000 7f7f112bb800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:09.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.316+0000 7f7f112bb800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:09.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.316+0000 7f7f112bb800 5 note: GC not initialized 2024-05-08T18:05:09.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.316+0000 7f7d3ffff640 20 reqs_thread_entry: start 2024-05-08T18:05:09.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.317+0000 7f7f112bb800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:09.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.317+0000 7f7f112bb800 20 Filter name: none 2024-05-08T18:05:09.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.317+0000 7f7d3dffb640 20 reqs_thread_entry: start 2024-05-08T18:05:09.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.317+0000 7f7f112bb800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.317+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55946a86ad10 con 0x55946a820a80 2024-05-08T18:05:09.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.317+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 testx/myfoo [call out=48b,stat out=16b,read 0~193 out=193b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 281+0+332 (crc 0 0 0) 0x7f7f08040040 con 0x55946a820a80 2024-05-08T18:05:09.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.317+0000 7f7f112bb800 20 rados_obj.operate() r=0 bl.length=193 2024-05-08T18:05:09.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.318+0000 7f7dc57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:09.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.318+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94095930509072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a81e1e0 con 0x55946a820a80 2024-05-08T18:05:09.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.319+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94095930509072] v27'131 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7f08040040 con 0x55946a820a80 2024-05-08T18:05:09.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.319+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94095930483136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a868d70 con 0x7f7f04076040 2024-05-08T18:05:09.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.320+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94095930483136] v27'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f08040040 con 0x7f7f04076040 2024-05-08T18:05:09.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.320+0000 7f7f112bb800 20 remove_watcher() i=0 2024-05-08T18:05:09.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.320+0000 7f7f112bb800 2 removed watcher, disabling cache 2024-05-08T18:05:09.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.320+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94095930486944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a868d70 con 0x55946a820a80 2024-05-08T18:05:09.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.321+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94095930486944] v27'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f08040040 con 0x55946a820a80 2024-05-08T18:05:09.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.321+0000 7f7f112bb800 20 remove_watcher() i=1 2024-05-08T18:05:09.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.321+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94095930489632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a868d70 con 0x55946a829e00 2024-05-08T18:05:09.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.322+0000 7f7f0e5b9640 1 -- 172.21.15.50:0/2370516345 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94095930489632] v27'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f04081070 con 0x55946a829e00 2024-05-08T18:05:09.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.322+0000 7f7f112bb800 20 remove_watcher() i=2 2024-05-08T18:05:09.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.322+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94095930491952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a868d70 con 0x7f7f04076040 2024-05-08T18:05:09.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.323+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94095930491952] v27'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f08040040 con 0x7f7f04076040 2024-05-08T18:05:09.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.323+0000 7f7f112bb800 20 remove_watcher() i=3 2024-05-08T18:05:09.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.323+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94095930480608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a868d70 con 0x55946a820a80 2024-05-08T18:05:09.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.324+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94095930480608] v27'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f08040040 con 0x55946a820a80 2024-05-08T18:05:09.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.324+0000 7f7f112bb800 20 remove_watcher() i=4 2024-05-08T18:05:09.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.324+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94095930496832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a868d70 con 0x55946a825200 2024-05-08T18:05:09.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.325+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94095930496832] v27'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.325+0000 7f7f112bb800 20 remove_watcher() i=5 2024-05-08T18:05:09.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.325+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94095930502080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a868d70 con 0x55946a825200 2024-05-08T18:05:09.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.326+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94095930502080] v27'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.326+0000 7f7f112bb800 20 remove_watcher() i=6 2024-05-08T18:05:09.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.326+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94095930507648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55946a868d70 con 0x55946a825200 2024-05-08T18:05:09.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f0edba640 1 -- 172.21.15.50:0/2370516345 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94095930507648] v27'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7f0006f030 con 0x55946a825200 2024-05-08T18:05:09.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f112bb800 20 remove_watcher() i=7 2024-05-08T18:05:09.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55946a825200 msgr2=0x55946a8275f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55946a825200 0x55946a8275f0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55946a820a80 msgr2=0x55946a822f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55946a820a80 0x55946a822f40 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7f04076040 msgr2=0x7f7f04083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7f04076040 0x7f7f04083410 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55946a829e00 msgr2=0x55946a82c2c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55946a829e00 0x55946a82c2c0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7dd803bba0 msgr2=0x7f7dd803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7dd803bba0 0x7f7dd803e060 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7f04052fd0 tx=0x7f7f0404f000 comp rx=0 tx=0).stop 2024-05-08T18:05:09.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2370516345 reap_dead start 2024-05-08T18:05:09.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a80bce0 msgr2=0x55946a6e7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a80bce0 0x55946a6e7ac0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7f0004bf30 tx=0x7f7f00018c20 comp rx=0 tx=0).stop 2024-05-08T18:05:09.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 shutdown_connections 2024-05-08T18:05:09.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2370516345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a80bce0 0x55946a6e7ac0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.327+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 >> 172.21.15.50:0/2370516345 conn(0x55946a808040 msgr2=0x55946a8056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 shutdown_connections 2024-05-08T18:05:09.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 -- 172.21.15.50:0/2370516345 wait complete. 2024-05-08T18:05:09.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 -- 172.21.15.50:0/2660513014 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7de803bba0 msgr2=0x7f7de803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2660513014 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7de803bba0 0x7f7de803e060 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55946a7e85e0 tx=0x7f7f04005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:09.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 -- 172.21.15.50:0/2660513014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a7fedd0 msgr2=0x55946a7ff1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2660513014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a7fedd0 0x55946a7ff1f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7f00022860 tx=0x7f7f0000c6c0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 -- 172.21.15.50:0/2660513014 shutdown_connections 2024-05-08T18:05:09.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2660513014 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7de803bba0 0x7f7de803e060 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2660513014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a7fedd0 0x55946a7ff1f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 -- 172.21.15.50:0/2660513014 >> 172.21.15.50:0/2660513014 conn(0x55946a7f2e70 msgr2=0x55946a7f3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 -- 172.21.15.50:0/2660513014 shutdown_connections 2024-05-08T18:05:09.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 -- 172.21.15.50:0/2660513014 wait complete. 2024-05-08T18:05:09.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 -- 172.21.15.50:0/2995376839 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55946a7ea030 msgr2=0x55946a7ec420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55946a7ea030 0x55946a7ec420 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 -- 172.21.15.50:0/2995376839 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55946a6ebc90 msgr2=0x55946a7e77a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55946a6ebc90 0x55946a7e77a0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 -- 172.21.15.50:0/2995376839 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55946a7edf70 msgr2=0x55946a7f0360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55946a7edf70 0x55946a7f0360 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 -- 172.21.15.50:0/2995376839 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7df403bba0 msgr2=0x7f7df403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7df403bba0 0x7f7df403e060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7f04054000 tx=0x7f7f0404e000 comp rx=0 tx=0).stop 2024-05-08T18:05:09.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 -- 172.21.15.50:0/2995376839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6e8180 msgr2=0x55946a7e0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f112bb800 1 --2- 172.21.15.50:0/2995376839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55946a6e8180 0x55946a7e0560 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000b700 tx=0x7f7f000040e0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.328+0000 7f7f0f5bb640 1 -- 172.21.15.50:0/2995376839 reap_dead start 2024-05-08T18:05:09.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.329+0000 7f7f112bb800 1 -- 172.21.15.50:0/2995376839 shutdown_connections 2024-05-08T18:05:09.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.329+0000 7f7f112bb800 1 -- 172.21.15.50:0/2995376839 >> 172.21.15.50:0/2995376839 conn(0x55946a6e1ad0 msgr2=0x55946a7d2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.329+0000 7f7f112bb800 1 -- 172.21.15.50:0/2995376839 shutdown_connections 2024-05-08T18:05:09.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.329+0000 7f7f112bb800 1 -- 172.21.15.50:0/2995376839 wait complete. 2024-05-08T18:05:09.346 INFO:teuthology.orchestra.run.smithi050.stdout:{"key":"bucket:testx/myfoo","ver":{"tag":"IccuB4GRYEnpPlFDqOihsPHV","ver":2},"mtime":"2024-05-08T18:05:09.180112Z","data":{"bucket":{"name":"myfoo","marker":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1","bucket_id":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1","tenant":"testx","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""}},"owner":"testx$tenanteduser","creation_time":"2024-05-08T18:05:06.706377Z","linked":true,"has_bucket_info":false}} 2024-05-08T18:05:09.346 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:testx/myfoo', 'ver': {'tag': 'IccuB4GRYEnpPlFDqOihsPHV', 'ver': 2}, 'mtime': '2024-05-08T18:05:09.180112Z', 'data': {'bucket': {'name': 'myfoo', 'marker': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1', 'bucket_id': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1', 'tenant': 'testx', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'testx$tenanteduser', 'creation_time': '2024-05-08T18:05:06.706377Z', 'linked': True, 'has_bucket_info': False}} 2024-05-08T18:05:09.346 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2024-05-08T18:05:09.346 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2024-05-08T18:05:09.347 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket testx/myfoo --uid foo 2024-05-08T18:05:09.434 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:09.435 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:09.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.436+0000 7fc044cdf800 1 Processor -- start 2024-05-08T18:05:09.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.436+0000 7fc044cdf800 1 -- start start 2024-05-08T18:05:09.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.436+0000 7fc044cdf800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56424585f960 0x56424585fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.436+0000 7fc044cdf800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5642458603b0 con 0x564245862950 2024-05-08T18:05:09.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.436+0000 7fc044cdf800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x564245701e20 con 0x56424585f960 2024-05-08T18:05:09.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.437+0000 7fc0427de640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56424585f960 0x56424585fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.437+0000 7fc041fdd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x564245862950 0x564245862e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43252/0 (socket says 172.21.15.50:43252) 2024-05-08T18:05:09.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.437+0000 7fc041fdd640 1 -- 172.21.15.50:0/118630939 learned_addr learned my addr 172.21.15.50:0/118630939 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:09.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.437+0000 7fc0427de640 1 -- 172.21.15.50:0/118630939 >> v1:172.21.15.46:6789/0 conn(0x564245862950 legacy=0x564245862e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.437+0000 7fc0417dc640 1 -- 172.21.15.50:0/118630939 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1209465604 0 0) 0x5642458603b0 con 0x564245862950 2024-05-08T18:05:09.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.437+0000 7fc0427de640 1 -- 172.21.15.50:0/118630939 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x564245863510 con 0x56424585f960 2024-05-08T18:05:09.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.439+0000 7fc0427de640 1 --2- 172.21.15.50:0/118630939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 0x56424585fd80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc03400a470 tx=0x7fc03401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1af3430a45d3eae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.439+0000 7fc0417dc640 1 -- 172.21.15.50:0/118630939 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc034057070 con 0x56424585f960 2024-05-08T18:05:09.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.439+0000 7fc0417dc640 1 -- 172.21.15.50:0/118630939 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03404adc0 con 0x56424585f960 2024-05-08T18:05:09.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.439+0000 7fc0417dc640 1 -- 172.21.15.50:0/118630939 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc034052590 con 0x56424585f960 2024-05-08T18:05:09.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.439+0000 7fc044cdf800 1 -- 172.21.15.50:0/118630939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 msgr2=0x56424585fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.440+0000 7fc044cdf800 1 --2- 172.21.15.50:0/118630939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 0x56424585fd80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc03400a470 tx=0x7fc03401ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:09.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.440+0000 7fc044cdf800 1 -- 172.21.15.50:0/118630939 shutdown_connections 2024-05-08T18:05:09.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.440+0000 7fc044cdf800 1 --2- 172.21.15.50:0/118630939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 0x56424585fd80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.440+0000 7fc044cdf800 1 -- 172.21.15.50:0/118630939 >> 172.21.15.50:0/118630939 conn(0x56424585b2f0 msgr2=0x56424585d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.440+0000 7fc044cdf800 1 -- 172.21.15.50:0/118630939 shutdown_connections 2024-05-08T18:05:09.441 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.440+0000 7fc044cdf800 1 -- 172.21.15.50:0/118630939 wait complete. 2024-05-08T18:05:09.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.455+0000 7fc044cdf800 1 Processor -- start 2024-05-08T18:05:09.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.455+0000 7fc044cdf800 1 -- start start 2024-05-08T18:05:09.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.455+0000 7fc044cdf800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 0x564245952a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.455+0000 7fc044cdf800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5642457aca30 con 0x56424585f960 2024-05-08T18:05:09.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.455+0000 7fc0427de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 0x564245952a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.455+0000 7fc0427de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 0x564245952a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38302/0 (socket says 172.21.15.50:38302) 2024-05-08T18:05:09.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.455+0000 7fc0427de640 1 -- 172.21.15.50:0/278719779 learned_addr learned my addr 172.21.15.50:0/278719779 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.456+0000 7fc0427de640 1 -- 172.21.15.50:0/278719779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564245953040 con 0x56424585f960 2024-05-08T18:05:09.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.456+0000 7fc0427de640 1 --2- 172.21.15.50:0/278719779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 0x564245952a70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc034055040 tx=0x7fc03404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5037835b9877a8a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.456+0000 7fbf30608640 1 -- 172.21.15.50:0/278719779 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc034057040 con 0x56424585f960 2024-05-08T18:05:09.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.456+0000 7fbf30608640 1 -- 172.21.15.50:0/278719779 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc034062070 con 0x56424585f960 2024-05-08T18:05:09.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.456+0000 7fbf30608640 1 -- 172.21.15.50:0/278719779 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc03405d470 con 0x56424585f960 2024-05-08T18:05:09.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.457+0000 7fc044cdf800 1 -- 172.21.15.50:0/278719779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 msgr2=0x564245952a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.457+0000 7fc044cdf800 1 --2- 172.21.15.50:0/278719779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 0x564245952a70 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc034055040 tx=0x7fc03404c000 comp rx=0 tx=0).stop 2024-05-08T18:05:09.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.457+0000 7fc044cdf800 1 -- 172.21.15.50:0/278719779 shutdown_connections 2024-05-08T18:05:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.457+0000 7fc044cdf800 1 --2- 172.21.15.50:0/278719779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 0x564245952a70 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.457+0000 7fc044cdf800 1 -- 172.21.15.50:0/278719779 >> 172.21.15.50:0/278719779 conn(0x56424585b2f0 msgr2=0x56424594e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.457+0000 7fc044cdf800 1 -- 172.21.15.50:0/278719779 shutdown_connections 2024-05-08T18:05:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.457+0000 7fc044cdf800 1 -- 172.21.15.50:0/278719779 wait complete. 2024-05-08T18:05:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.457+0000 7fc044cdf800 1 Processor -- start 2024-05-08T18:05:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.457+0000 7fc044cdf800 1 -- start start 2024-05-08T18:05:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.457+0000 7fc044cdf800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 0x5642458621f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.457+0000 7fc044cdf800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56424586ae20 con 0x56424585f960 2024-05-08T18:05:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.458+0000 7fc0427de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 0x5642458621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.458+0000 7fc0427de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 0x5642458621f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38304/0 (socket says 172.21.15.50:38304) 2024-05-08T18:05:09.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.458+0000 7fc0427de640 1 -- 172.21.15.50:0/3560137342 learned_addr learned my addr 172.21.15.50:0/3560137342 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.458+0000 7fc0427de640 1 -- 172.21.15.50:0/3560137342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564245862730 con 0x56424585f960 2024-05-08T18:05:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.458+0000 7fc0427de640 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 0x5642458621f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc03401ce60 tx=0x7fc034002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.458+0000 7fbf2ee05640 1 -- 172.21.15.50:0/3560137342 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc03406b070 con 0x56424585f960 2024-05-08T18:05:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.458+0000 7fbf2ee05640 1 -- 172.21.15.50:0/3560137342 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03405d970 con 0x56424585f960 2024-05-08T18:05:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.458+0000 7fc044cdf800 1 -- 172.21.15.50:0/3560137342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564245860980 con 0x56424585f960 2024-05-08T18:05:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.458+0000 7fbf2ee05640 1 -- 172.21.15.50:0/3560137342 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc034057070 con 0x56424585f960 2024-05-08T18:05:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.458+0000 7fc044cdf800 1 -- 172.21.15.50:0/3560137342 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564245860ea0 con 0x56424585f960 2024-05-08T18:05:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.459+0000 7fbf2ee05640 1 -- 172.21.15.50:0/3560137342 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fc03405dc90 con 0x56424585f960 2024-05-08T18:05:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.459+0000 7fbf2ee05640 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf2803bba0 0x7fbf2803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.459+0000 7fbf2ee05640 1 -- 172.21.15.50:0/3560137342 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fc034094510 con 0x56424585f960 2024-05-08T18:05:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.460+0000 7fc041fdd640 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf2803bba0 0x7fbf2803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.460+0000 7fc044cdf800 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564245862950 0x564245963950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.460+0000 7fc044cdf800 1 -- 172.21.15.50:0/3560137342 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564245963e90 con 0x564245862950 2024-05-08T18:05:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.460+0000 7fc042fdf640 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564245862950 0x564245963950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.460+0000 7fc041fdd640 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf2803bba0 0x7fbf2803e060 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc038000c00 tx=0x7fc0380013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.460+0000 7fc042fdf640 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564245862950 0x564245963950 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.460+0000 7fc042fdf640 1 -- 172.21.15.50:0/3560137342 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc03c00a4a0 con 0x564245862950 2024-05-08T18:05:09.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.460+0000 7fc044cdf800 1 -- 172.21.15.50:0/3560137342 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564245861300 con 0x564245862950 2024-05-08T18:05:09.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.461+0000 7fc042fdf640 1 -- 172.21.15.50:0/3560137342 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc03c00a4a0 con 0x564245862950 2024-05-08T18:05:09.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.461+0000 7fc044cdf800 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564245966100 0x5642459684f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.461+0000 7fc044cdf800 1 -- 172.21.15.50:0/3560137342 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564245968bc0 con 0x564245966100 2024-05-08T18:05:09.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.461+0000 7fc0427de640 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564245966100 0x5642459684f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.461+0000 7fc0427de640 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564245966100 0x5642459684f0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.461+0000 7fc0427de640 1 -- 172.21.15.50:0/3560137342 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fc03405a040 con 0x564245966100 2024-05-08T18:05:09.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.461+0000 7fc044cdf800 1 -- 172.21.15.50:0/3560137342 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564245964fa0 con 0x564245862950 2024-05-08T18:05:09.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.462+0000 7fc042fdf640 1 -- 172.21.15.50:0/3560137342 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc03c00a4a0 con 0x564245862950 2024-05-08T18:05:09.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.462+0000 7fc044cdf800 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56424596a000 0x56424596c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.462+0000 7fc044cdf800 1 -- 172.21.15.50:0/3560137342 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56424596cb30 con 0x56424596a000 2024-05-08T18:05:09.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.462+0000 7fc041fdd640 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56424596a000 0x56424596c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.462+0000 7fc041fdd640 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56424596a000 0x56424596c3f0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.462+0000 7fc041fdd640 1 -- 172.21.15.50:0/3560137342 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fc03800f670 con 0x56424596a000 2024-05-08T18:05:09.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.463+0000 7fc044cdf800 1 Processor -- start 2024-05-08T18:05:09.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.463+0000 7fc044cdf800 1 -- start start 2024-05-08T18:05:09.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.463+0000 7fc044cdf800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424597aea0 0x56424597b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.463+0000 7fc044cdf800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564245753330 con 0x56424597aea0 2024-05-08T18:05:09.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.463+0000 7fc0427de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424597aea0 0x56424597b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.463+0000 7fc0427de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424597aea0 0x56424597b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38310/0 (socket says 172.21.15.50:38310) 2024-05-08T18:05:09.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.463+0000 7fc0427de640 1 -- 172.21.15.50:0/3874975289 learned_addr learned my addr 172.21.15.50:0/3874975289 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.464+0000 7fc0427de640 1 -- 172.21.15.50:0/3874975289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56424597b800 con 0x56424597aea0 2024-05-08T18:05:09.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.464+0000 7fc0427de640 1 --2- 172.21.15.50:0/3874975289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424597aea0 0x56424597b2c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc03405bc90 tx=0x7fc0340094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.464+0000 7fbf27fff640 1 -- 172.21.15.50:0/3874975289 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc03408f070 con 0x56424597aea0 2024-05-08T18:05:09.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.464+0000 7fbf27fff640 1 -- 172.21.15.50:0/3874975289 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc034079dd0 con 0x56424597aea0 2024-05-08T18:05:09.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.464+0000 7fc044cdf800 1 -- 172.21.15.50:0/3874975289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56424597c270 con 0x56424597aea0 2024-05-08T18:05:09.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.464+0000 7fbf27fff640 1 -- 172.21.15.50:0/3874975289 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc034092b70 con 0x56424597aea0 2024-05-08T18:05:09.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.464+0000 7fc044cdf800 1 -- 172.21.15.50:0/3874975289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56424597c770 con 0x56424597aea0 2024-05-08T18:05:09.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.465+0000 7fbf27fff640 1 -- 172.21.15.50:0/3874975289 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fc034092070 con 0x56424597aea0 2024-05-08T18:05:09.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.465+0000 7fbf27fff640 1 --2- 172.21.15.50:0/3874975289 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf1c03bba0 0x7fbf1c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.465+0000 7fbf27fff640 1 -- 172.21.15.50:0/3874975289 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fc034087070 con 0x56424597aea0 2024-05-08T18:05:09.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.465+0000 7fc041fdd640 1 --2- 172.21.15.50:0/3874975289 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf1c03bba0 0x7fbf1c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.465+0000 7fc044cdf800 1 Processor -- start 2024-05-08T18:05:09.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.465+0000 7fc044cdf800 1 -- start start 2024-05-08T18:05:09.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.465+0000 7fc041fdd640 1 --2- 172.21.15.50:0/3874975289 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf1c03bba0 0x7fbf1c03e060 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x5642459642d0 tx=0x7fc038006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.466+0000 7fc044cdf800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564245987fa0 0x5642459883c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.466+0000 7fc044cdf800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564245953300 con 0x564245987fa0 2024-05-08T18:05:09.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.466+0000 7fc0427de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564245987fa0 0x5642459883c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.466+0000 7fc0427de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564245987fa0 0x5642459883c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38312/0 (socket says 172.21.15.50:38312) 2024-05-08T18:05:09.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.466+0000 7fc0427de640 1 -- 172.21.15.50:0/3073002830 learned_addr learned my addr 172.21.15.50:0/3073002830 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.466+0000 7fc0427de640 1 -- 172.21.15.50:0/3073002830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564245988990 con 0x564245987fa0 2024-05-08T18:05:09.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.466+0000 7fc0427de640 1 --2- 172.21.15.50:0/3073002830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564245987fa0 0x5642459883c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc03405ba60 tx=0x7fc03406eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217f174b37a8c256 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.466+0000 7fbf25ffb640 1 -- 172.21.15.50:0/3073002830 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc03405cbd0 con 0x564245987fa0 2024-05-08T18:05:09.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.466+0000 7fbf25ffb640 1 -- 172.21.15.50:0/3073002830 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0340709a0 con 0x564245987fa0 2024-05-08T18:05:09.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.466+0000 7fbf25ffb640 1 -- 172.21.15.50:0/3073002830 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc034092d10 con 0x564245987fa0 2024-05-08T18:05:09.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.467+0000 7fc044cdf800 1 -- 172.21.15.50:0/3073002830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564245987fa0 msgr2=0x5642459883c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.467+0000 7fc044cdf800 1 --2- 172.21.15.50:0/3073002830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564245987fa0 0x5642459883c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fc03405ba60 tx=0x7fc03406eb50 comp rx=0 tx=0).stop 2024-05-08T18:05:09.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.467+0000 7fc044cdf800 1 -- 172.21.15.50:0/3073002830 shutdown_connections 2024-05-08T18:05:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.467+0000 7fc044cdf800 1 --2- 172.21.15.50:0/3073002830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564245987fa0 0x5642459883c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.467+0000 7fc044cdf800 1 -- 172.21.15.50:0/3073002830 >> 172.21.15.50:0/3073002830 conn(0x5642459842b0 msgr2=0x564245982310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.467+0000 7fc044cdf800 1 -- 172.21.15.50:0/3073002830 shutdown_connections 2024-05-08T18:05:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.467+0000 7fc044cdf800 1 -- 172.21.15.50:0/3073002830 wait complete. 2024-05-08T18:05:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.467+0000 7fc044cdf800 1 Processor -- start 2024-05-08T18:05:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.467+0000 7fc044cdf800 1 -- start start 2024-05-08T18:05:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.467+0000 7fc044cdf800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564245987fa0 0x56424598f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.467+0000 7fc044cdf800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564245989020 con 0x564245987fa0 2024-05-08T18:05:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.468+0000 7fc0427de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564245987fa0 0x56424598f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.468+0000 7fc0427de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564245987fa0 0x56424598f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38326/0 (socket says 172.21.15.50:38326) 2024-05-08T18:05:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.468+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 learned_addr learned my addr 172.21.15.50:0/4106731120 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.468+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56424598f8c0 con 0x564245987fa0 2024-05-08T18:05:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.468+0000 7fc0427de640 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564245987fa0 0x56424598f380 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc034044700 tx=0x7fc034094f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.468+0000 7fbf13fff640 1 -- 172.21.15.50:0/4106731120 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc0340b9070 con 0x564245987fa0 2024-05-08T18:05:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.468+0000 7fbf13fff640 1 -- 172.21.15.50:0/4106731120 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03406e360 con 0x564245987fa0 2024-05-08T18:05:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.468+0000 7fbf13fff640 1 -- 172.21.15.50:0/4106731120 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc03407c0d0 con 0x564245987fa0 2024-05-08T18:05:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.468+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56424598f8c0 con 0x564245987fa0 2024-05-08T18:05:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.468+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56424598f8c0 con 0x564245987fa0 2024-05-08T18:05:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.469+0000 7fbf13fff640 1 -- 172.21.15.50:0/4106731120 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fc0340bc070 con 0x564245987fa0 2024-05-08T18:05:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.469+0000 7fbf13fff640 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf0c03bba0 0x7fbf0c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.469+0000 7fbf13fff640 1 -- 172.21.15.50:0/4106731120 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fc0340bfca0 con 0x564245987fa0 2024-05-08T18:05:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.469+0000 7fc041fdd640 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf0c03bba0 0x7fbf0c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.470+0000 7fc041fdd640 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf0c03bba0 0x7fbf0c03e060 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc03800ffd0 tx=0x7fc038023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.470+0000 7fc044cdf800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.470+0000 7fc044cdf800 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56424599a930 0x56424599cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.470+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56424599d540 con 0x56424599a930 2024-05-08T18:05:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.470+0000 7fc042fdf640 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56424599a930 0x56424599cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.470+0000 7fc042fdf640 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56424599a930 0x56424599cdf0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.470+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc03c00a4a0 con 0x56424599a930 2024-05-08T18:05:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc044cdf800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc044cdf800 20 realm 2024-05-08T18:05:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc044cdf800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56424599d950 con 0x56424599a930 2024-05-08T18:05:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc03c00a4a0 con 0x56424599a930 2024-05-08T18:05:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc044cdf800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc044cdf800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc044cdf800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56424599e220 con 0x56424599a930 2024-05-08T18:05:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc03c00a4a0 con 0x56424599a930 2024-05-08T18:05:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc044cdf800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc044cdf800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56424599e5c0 con 0x56424599a930 2024-05-08T18:05:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc03c00a4a0 con 0x56424599a930 2024-05-08T18:05:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc044cdf800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc044cdf800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc044cdf800 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56424599f320 0x5642459a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.471+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5642459a1cd0 con 0x56424599f320 2024-05-08T18:05:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.472+0000 7fc0427de640 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56424599f320 0x5642459a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.472+0000 7fc0427de640 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56424599f320 0x5642459a1710 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.472+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.472+0000 7fc044cdf800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.472+0000 7fc044cdf800 20 searching for the correct realm 2024-05-08T18:05:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.472+0000 7fc044cdf800 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5642459a3c60 0x5642459a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.472+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5642459a67f0 con 0x5642459a3c60 2024-05-08T18:05:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.472+0000 7fc041fdd640 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5642459a3c60 0x5642459a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.473+0000 7fc041fdd640 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5642459a3c60 0x5642459a6120 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.473+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc038037040 con 0x5642459a3c60 2024-05-08T18:05:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.473+0000 7fc041fdd640 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fc03802e910 0x7fc03803a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.473+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5642459a67f0 con 0x7fc03802e910 2024-05-08T18:05:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.473+0000 7fc042fdf640 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fc03802e910 0x7fc03803a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.473+0000 7fc042fdf640 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fc03802e910 0x7fc03803a3c0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.474+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc03c00a4a0 con 0x7fc03802e910 2024-05-08T18:05:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.474+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56424599e5c0 con 0x56424599f320 2024-05-08T18:05:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.474+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.474+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56424599e5c0 con 0x56424599a930 2024-05-08T18:05:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.474+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc03c00a4a0 con 0x56424599a930 2024-05-08T18:05:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.474+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56424599e5c0 con 0x56424599a930 2024-05-08T18:05:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.474+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc03c00a4a0 con 0x56424599a930 2024-05-08T18:05:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.474+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56424599e5c0 con 0x56424599f320 2024-05-08T18:05:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.475+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.475+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56424599e5c0 con 0x56424599f320 2024-05-08T18:05:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.475+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.475+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x7fc03802e910 2024-05-08T18:05:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.475+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc03c00a4a0 con 0x7fc03802e910 2024-05-08T18:05:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.475+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x5642459a3c60 2024-05-08T18:05:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.475+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc038037040 con 0x5642459a3c60 2024-05-08T18:05:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.475+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x56424599a930 2024-05-08T18:05:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.476+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc03c00a4a0 con 0x56424599a930 2024-05-08T18:05:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.476+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x56424599f320 2024-05-08T18:05:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.476+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.476+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x56424599a930 2024-05-08T18:05:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.476+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc03c00a4a0 con 0x56424599a930 2024-05-08T18:05:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.476+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x7fc03802e910 2024-05-08T18:05:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.476+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fc03c00a4a0 con 0x7fc03802e910 2024-05-08T18:05:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.476+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x5642459a3c60 2024-05-08T18:05:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.477+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc038037040 con 0x5642459a3c60 2024-05-08T18:05:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.477+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x5642459a3c60 2024-05-08T18:05:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.477+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc038037040 con 0x5642459a3c60 2024-05-08T18:05:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.477+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x56424599f320 2024-05-08T18:05:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.477+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.477+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x5642459a3c60 2024-05-08T18:05:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.477+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc038037040 con 0x5642459a3c60 2024-05-08T18:05:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.478+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x5642459a3c60 2024-05-08T18:05:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.478+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fc038037040 con 0x5642459a3c60 2024-05-08T18:05:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.478+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x56424599a930 2024-05-08T18:05:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.478+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc03c00a4a0 con 0x56424599a930 2024-05-08T18:05:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.478+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x56424599a930 2024-05-08T18:05:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.478+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fc03c00a4a0 con 0x56424599a930 2024-05-08T18:05:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.478+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x56424599f320 2024-05-08T18:05:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.479+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.479+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x5642459a3c60 2024-05-08T18:05:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.479+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc038037040 con 0x5642459a3c60 2024-05-08T18:05:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.479+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x7fc03802e910 2024-05-08T18:05:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.479+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc03c00a4a0 con 0x7fc03802e910 2024-05-08T18:05:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.479+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x56424599a930 2024-05-08T18:05:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.479+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc03c00a4a0 con 0x56424599a930 2024-05-08T18:05:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.479+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x56424599f320 2024-05-08T18:05:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.480+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.480+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x5642459a3c60 2024-05-08T18:05:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.480+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc038037040 con 0x5642459a3c60 2024-05-08T18:05:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.480+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x56424599f320 2024-05-08T18:05:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.481+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.481+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x7fc03802e910 2024-05-08T18:05:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.481+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc03c00a4a0 con 0x7fc03802e910 2024-05-08T18:05:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.481+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x5642459a3c60 2024-05-08T18:05:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.481+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc038037040 con 0x5642459a3c60 2024-05-08T18:05:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.481+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x56424599f320 2024-05-08T18:05:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x56424599a930 2024-05-08T18:05:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fc03c00a4a0 con 0x56424599a930 2024-05-08T18:05:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc0340cb080 con 0x7fc03802e910 2024-05-08T18:05:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc03c00a4a0 con 0x7fc03802e910 2024-05-08T18:05:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc044cdf800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc044cdf800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc044cdf800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc044cdf800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc044cdf800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc044cdf800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc044cdf800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5642459a2590 con 0x56424599a930 2024-05-08T18:05:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc03c00a4a0 con 0x56424599a930 2024-05-08T18:05:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc044cdf800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc044cdf800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.482+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5642459a2590 con 0x56424599a930 2024-05-08T18:05:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.483+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc03c00a4a0 con 0x56424599a930 2024-05-08T18:05:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.483+0000 7fc044cdf800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.483+0000 7fc044cdf800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.483+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5642459a2590 con 0x5642459a3c60 2024-05-08T18:05:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.483+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fc038037040 con 0x5642459a3c60 2024-05-08T18:05:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.483+0000 7fc044cdf800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.483+0000 7fc044cdf800 20 zone default found 2024-05-08T18:05:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.483+0000 7fc044cdf800 4 Realm: () 2024-05-08T18:05:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.483+0000 7fc044cdf800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.483+0000 7fc044cdf800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.483+0000 7fc044cdf800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.483+0000 7fc044cdf800 20 zonegroup default 2024-05-08T18:05:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.483+0000 7fc044cdf800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.483+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5642459a2f60 con 0x56424599f320 2024-05-08T18:05:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.483+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.483+0000 7fc044cdf800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.484+0000 7fc044cdf800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.484+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5642459a2b00 con 0x5642459a3c60 2024-05-08T18:05:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.484+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc038037040 con 0x5642459a3c60 2024-05-08T18:05:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.484+0000 7fc044cdf800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.484+0000 7fc044cdf800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.484+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5642459a8ba0 con 0x5642459a3c60 2024-05-08T18:05:09.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.484+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc038037040 con 0x5642459a3c60 2024-05-08T18:05:09.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.484+0000 7fc044cdf800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.484+0000 7fc044cdf800 20 started sync module instance, tier type = 2024-05-08T18:05:09.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.484+0000 7fc044cdf800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:09.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.484+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459a7f90 con 0x7fc03802e910 2024-05-08T18:05:09.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.485+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc03c00a4a0 con 0x7fc03802e910 2024-05-08T18:05:09.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.485+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94842635584688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459a7f90 con 0x7fc03802e910 2024-05-08T18:05:09.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.485+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459a96c0 con 0x56424599a930 2024-05-08T18:05:09.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.486+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94842635584688] v27'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x7fc03802e910 2024-05-08T18:05:09.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.487+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x56424599a930 2024-05-08T18:05:09.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.487+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94842635588048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459a7f90 con 0x56424599a930 2024-05-08T18:05:09.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.487+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459aa190 con 0x5642459a3c60 2024-05-08T18:05:09.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.488+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc038037040 con 0x5642459a3c60 2024-05-08T18:05:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.488+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94842635590784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459aaa30 con 0x5642459a3c60 2024-05-08T18:05:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.488+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459ab510 con 0x7fc03802e910 2024-05-08T18:05:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.488+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94842635588048] v27'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x56424599a930 2024-05-08T18:05:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.489+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94842635590784] v27'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc038037040 con 0x5642459a3c60 2024-05-08T18:05:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.489+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x7fc03802e910 2024-05-08T18:05:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.489+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94842635593088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459a7ae0 con 0x7fc03802e910 2024-05-08T18:05:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.489+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459abdb0 con 0x56424599a930 2024-05-08T18:05:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.490+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x56424599a930 2024-05-08T18:05:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.490+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94842635598000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459abcd0 con 0x56424599a930 2024-05-08T18:05:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.490+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459accb0 con 0x56424599f320 2024-05-08T18:05:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.490+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94842635593088] v27'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x7fc03802e910 2024-05-08T18:05:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.491+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.491+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94842635595808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459ad0a0 con 0x56424599f320 2024-05-08T18:05:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.491+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459ada60 con 0x56424599f320 2024-05-08T18:05:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.491+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94842635598000] v27'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x56424599a930 2024-05-08T18:05:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.492+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94842635595808] v27'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.492+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.492+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94842635601856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459ad5b0 con 0x56424599f320 2024-05-08T18:05:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.492+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459adf70 con 0x56424599f320 2024-05-08T18:05:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.493+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94842635601856] v27'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.493+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.493+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94842635600288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459adf70 con 0x56424599f320 2024-05-08T18:05:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.493+0000 7fc044cdf800 20 add_watcher() i=0 2024-05-08T18:05:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.493+0000 7fc044cdf800 20 add_watcher() i=1 2024-05-08T18:05:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.493+0000 7fc044cdf800 20 add_watcher() i=2 2024-05-08T18:05:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.493+0000 7fc044cdf800 20 add_watcher() i=3 2024-05-08T18:05:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.493+0000 7fc044cdf800 20 add_watcher() i=4 2024-05-08T18:05:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.493+0000 7fc044cdf800 20 add_watcher() i=5 2024-05-08T18:05:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.493+0000 7fc044cdf800 20 add_watcher() i=6 2024-05-08T18:05:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.494+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94842635600288] v27'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.494+0000 7fc044cdf800 20 add_watcher() i=7 2024-05-08T18:05:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.494+0000 7fc044cdf800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.494+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5642459b12f0 con 0x56424599a930 2024-05-08T18:05:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.494+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fc03c040040 con 0x56424599a930 2024-05-08T18:05:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.494+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94842635619536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459ae0e0 con 0x56424599a930 2024-05-08T18:05:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.495+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94842635619536] v27'132 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x56424599a930 2024-05-08T18:05:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.495+0000 7fbef8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.495+0000 7fc044cdf800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.495+0000 7fc044cdf800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.495+0000 7fc044cdf800 5 note: GC not initialized 2024-05-08T18:05:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.496+0000 7fbe737fe640 20 reqs_thread_entry: start 2024-05-08T18:05:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.496+0000 7fc044cdf800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.496+0000 7fc044cdf800 20 Filter name: none 2024-05-08T18:05:09.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.496+0000 7fbe717fa640 20 reqs_thread_entry: start 2024-05-08T18:05:09.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.496+0000 7fc044cdf800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:09.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.496+0000 7fc044cdf800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.496+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5642459e48a0 con 0x56424599a930 2024-05-08T18:05:09.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.496+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fc03c040040 con 0x56424599a930 2024-05-08T18:05:09.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.496+0000 7fc044cdf800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:09.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.496+0000 7fc044cdf800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:09.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.496+0000 7fc044cdf800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:09.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.496+0000 7fc044cdf800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:09.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.496+0000 7fc044cdf800 10 cache get: name=default.rgw.meta+root+testx/myfoo : miss 2024-05-08T18:05:09.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.496+0000 7fc044cdf800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.496+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5642459e6980 con 0x56424599a930 2024-05-08T18:05:09.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.497+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 testx/myfoo [call out=48b,read 0~193 out=193b] v0'0 uv1 ondisk = 0) ==== 197+0+241 (crc 0 0 0) 0x7fc03c040040 con 0x56424599a930 2024-05-08T18:05:09.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.497+0000 7fc044cdf800 20 rados_obj.operate() r=0 bl.length=193 2024-05-08T18:05:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.497+0000 7fc044cdf800 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x11 2024-05-08T18:05:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.497+0000 7fc044cdf800 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2024-05-08T18:05:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.497+0000 7fc044cdf800 10 cache get: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 : miss 2024-05-08T18:05:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.497+0000 7fc044cdf800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.497+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 8.1f 8:fa027c55:root::.bucket.meta.testx%3amyfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5642459e8210 con 0x7fc03802e910 2024-05-08T18:05:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.497+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(64 .bucket.meta.testx:myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~412 out=412b,getxattrs out=298b] v0'0 uv1 ondisk = 0) ==== 338+0+774 (crc 0 0 0) 0x7fc03c040040 con 0x7fc03802e910 2024-05-08T18:05:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.497+0000 7fc044cdf800 20 rados_obj.operate() r=0 bl.length=412 2024-05-08T18:05:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.497+0000 7fc044cdf800 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.497+0000 7fc044cdf800 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.497+0000 7fc044cdf800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.497+0000 7fc044cdf800 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-05-08T18:05:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.497+0000 7fc044cdf800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.testx:myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 2024-05-08T18:05:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.497+0000 7fc044cdf800 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-05-08T18:05:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.497+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459ec860 con 0x56424599a930 2024-05-08T18:05:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.499+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 testx$tenanteduser.buckets [call] v27'2 uv2 ondisk = 0) ==== 170+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x56424599a930 2024-05-08T18:05:09.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.499+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:66 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [create,call version.inc in=38b,writefull 0~379 in=379b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459f0170 con 0x56424599f320 2024-05-08T18:05:09.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.500+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(66 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [create,call,writefull 0~379,setxattr (123)] v27'5 uv5 ondisk = 0) ==== 332+0+0 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.500+0000 7fc044cdf800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x7 2024-05-08T18:05:09.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.500+0000 7fc044cdf800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:09.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.500+0000 7fc044cdf800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:09.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.500+0000 7fc044cdf800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1, ofs0, ns] 2024-05-08T18:05:09.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.500+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:67 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94842635869344 in=740b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5642459ef940 con 0x56424599f320 2024-05-08T18:05:09.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.500+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== watch-notify(notify (1) cookie 94842635601856 notify 115964117006 ret 0) ==== 770+0+0 (crc 0 0 0) 0x5642459ec290 con 0x56424599f320 2024-05-08T18:05:09.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.500+0000 7fbf25ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117006 cookie 94842635601856 notifier 4550 bl.length()=728 2024-05-08T18:05:09.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.500+0000 7fbf25ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x7 2024-05-08T18:05:09.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.500+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(67 notify.6 [notify cookie 94842635869344 out=8b] v0'0 uv138 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.500+0000 7fbf25ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:09.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.500+0000 7fbf25ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:09.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.500+0000 7fbf25ffb640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fbf08003ea0 con 0x56424599f320 2024-05-08T18:05:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.501+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(68 notify.6 [notify-ack] v0'0 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.501+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== watch-notify(notify_complete (2) cookie 94842635869344 notify 115964117006 ret 0) ==== 42+0+48 (crc 0 0 0) 0x56424597b800 con 0x56424599f320 2024-05-08T18:05:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.501+0000 7fc044cdf800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): orig_sources= new_sources= 2024-05-08T18:05:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.501+0000 7fc044cdf800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): potential sources added=[] removed=[] 2024-05-08T18:05:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.501+0000 7fc044cdf800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): orig_dests= new_dests= 2024-05-08T18:05:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.501+0000 7fc044cdf800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): potential dests added=[] removed=[] 2024-05-08T18:05:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.501+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:69 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459f0340 con 0x7fc03802e910 2024-05-08T18:05:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.502+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(69 foo.buckets [call] v27'28 uv28 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x7fc03802e910 2024-05-08T18:05:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.502+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459eb450 con 0x7fc03802e910 2024-05-08T18:05:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.503+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v27'29 uv29 ondisk = 0) ==== 275+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x7fc03802e910 2024-05-08T18:05:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.503+0000 7fc044cdf800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-05-08T18:05:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.503+0000 7fc044cdf800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.503+0000 7fc044cdf800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-05-08T18:05:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.503+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94842635860032 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5642459eec90 con 0x56424599f320 2024-05-08T18:05:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.503+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== watch-notify(notify (1) cookie 94842635601856 notify 115964117007 ret 0) ==== 356+0+0 (crc 0 0 0) 0x564245860ea0 con 0x56424599f320 2024-05-08T18:05:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.503+0000 7fbf25ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117007 cookie 94842635601856 notifier 4550 bl.length()=314 2024-05-08T18:05:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.503+0000 7fbf25ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-05-08T18:05:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.503+0000 7fbf25ffb640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.503+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(71 notify.6 [notify cookie 94842635860032 out=8b] v0'0 uv138 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.503+0000 7fbf25ffb640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fbf08003ea0 con 0x56424599f320 2024-05-08T18:05:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.504+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== watch-notify(notify_complete (2) cookie 94842635860032 notify 115964117007 ret 0) ==== 42+0+48 (crc 0 0 0) 0x564245860980 con 0x56424599f320 2024-05-08T18:05:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.504+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.504+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:73 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459ebcf0 con 0x56424599a930 2024-05-08T18:05:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.504+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(73 testx/myfoo [call,delete] v27'2 uv2 ondisk = 0) ==== 197+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x56424599a930 2024-05-08T18:05:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.504+0000 7fc044cdf800 10 removing default.rgw.meta+root+testx/myfoo from cache 2024-05-08T18:05:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.504+0000 7fc044cdf800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2024-05-08T18:05:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.504+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [notify cookie 94842635860032 in=167b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5642459edc80 con 0x56424599a930 2024-05-08T18:05:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.505+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== watch-notify(notify (1) cookie 94842635598000 notify 115964117010 ret 0) ==== 197+0+0 (crc 0 0 0) 0x5642459f0540 con 0x56424599a930 2024-05-08T18:05:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.505+0000 7fbf257fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117010 cookie 94842635598000 notifier 4550 bl.length()=155 2024-05-08T18:05:09.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.505+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(74 notify.4 [notify cookie 94842635860032 out=8b] v0'0 uv138 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fc03c040040 con 0x56424599a930 2024-05-08T18:05:09.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.505+0000 7fbf257fa640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fbe780032c0 con 0x56424599a930 2024-05-08T18:05:09.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.505+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== watch-notify(notify_complete (2) cookie 94842635860032 notify 115964117010 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5642459f0340 con 0x56424599a930 2024-05-08T18:05:09.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.505+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(75 notify.4 [notify-ack] v0'0 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x56424599a930 2024-05-08T18:05:09.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.505+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:76 8.1f 8:fa027c55:root::.bucket.meta.testx%3amyfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459ebcc0 con 0x7fc03802e910 2024-05-08T18:05:09.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.506+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(76 .bucket.meta.testx:myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call,delete] v27'2 uv2 ondisk = 0) ==== 254+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x7fc03802e910 2024-05-08T18:05:09.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.506+0000 7fc044cdf800 10 removing default.rgw.meta+root+.bucket.meta.testx:myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 from cache 2024-05-08T18:05:09.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.506+0000 7fc044cdf800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1, ofs0, ns] 2024-05-08T18:05:09.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.506+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.3 7:c609908c:::notify.5:head [notify cookie 94842635844288 in=224b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5642459ebcc0 con 0x56424599f320 2024-05-08T18:05:09.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.507+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== watch-notify(notify (1) cookie 94842635595808 notify 115964117008 ret 0) ==== 254+0+0 (crc 0 0 0) 0x564245860980 con 0x56424599f320 2024-05-08T18:05:09.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.507+0000 7fbf257fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117008 cookie 94842635595808 notifier 4550 bl.length()=212 2024-05-08T18:05:09.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.507+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(77 notify.5 [notify cookie 94842635844288 out=8b] v0'0 uv138 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.507+0000 7fbf257fa640 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fbe78003330 con 0x56424599f320 2024-05-08T18:05:09.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.507+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(78 notify.5 [notify-ack] v0'0 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.507+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== watch-notify(notify_complete (2) cookie 94842635844288 notify 115964117008 ret 0) ==== 42+0+48 (crc 0 0 0) 0x564245862730 con 0x56424599f320 2024-05-08T18:05:09.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.508+0000 7fbef8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:09.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.508+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:79 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94842635619536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459eb810 con 0x56424599a930 2024-05-08T18:05:09.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.509+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(79 data_loggenerations_metadata [watch unwatch cookie 94842635619536] v27'133 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x56424599a930 2024-05-08T18:05:09.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.509+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:80 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94842635584688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459b0c60 con 0x7fc03802e910 2024-05-08T18:05:09.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.510+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(80 notify.0 [watch unwatch cookie 94842635584688] v27'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x7fc03802e910 2024-05-08T18:05:09.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.510+0000 7fc044cdf800 20 remove_watcher() i=0 2024-05-08T18:05:09.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.510+0000 7fc044cdf800 2 removed watcher, disabling cache 2024-05-08T18:05:09.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.510+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94842635588048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459a8490 con 0x56424599a930 2024-05-08T18:05:09.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.511+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(81 notify.1 [watch unwatch cookie 94842635588048] v27'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x56424599a930 2024-05-08T18:05:09.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.511+0000 7fc044cdf800 20 remove_watcher() i=1 2024-05-08T18:05:09.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.511+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94842635590784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459a91d0 con 0x5642459a3c60 2024-05-08T18:05:09.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.512+0000 7fc041fdd640 1 -- 172.21.15.50:0/4106731120 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(82 notify.2 [watch unwatch cookie 94842635590784] v27'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc038037040 con 0x5642459a3c60 2024-05-08T18:05:09.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.512+0000 7fc044cdf800 20 remove_watcher() i=2 2024-05-08T18:05:09.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.512+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94842635593088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459a8490 con 0x7fc03802e910 2024-05-08T18:05:09.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.513+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 notify.3 [watch unwatch cookie 94842635593088] v27'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x7fc03802e910 2024-05-08T18:05:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.513+0000 7fc044cdf800 20 remove_watcher() i=3 2024-05-08T18:05:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.513+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:84 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94842635598000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459a8490 con 0x56424599a930 2024-05-08T18:05:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.514+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(84 notify.4 [watch unwatch cookie 94842635598000] v27'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc03c040040 con 0x56424599a930 2024-05-08T18:05:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.514+0000 7fc044cdf800 20 remove_watcher() i=4 2024-05-08T18:05:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.514+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:85 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94842635595808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459ab8b0 con 0x56424599f320 2024-05-08T18:05:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.515+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 31 ==== osd_op_reply(85 notify.5 [watch unwatch cookie 94842635595808] v27'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.515+0000 7fc044cdf800 20 remove_watcher() i=5 2024-05-08T18:05:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.515+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:86 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94842635601856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459ab8b0 con 0x56424599f320 2024-05-08T18:05:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.516+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 32 ==== osd_op_reply(86 notify.6 [watch unwatch cookie 94842635601856] v27'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.516+0000 7fc044cdf800 20 remove_watcher() i=6 2024-05-08T18:05:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.516+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:87 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94842635600288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5642459ab8b0 con 0x56424599f320 2024-05-08T18:05:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.517+0000 7fc0427de640 1 -- 172.21.15.50:0/4106731120 <== osd.0 v2:172.21.15.46:6800/337143975 33 ==== osd_op_reply(87 notify.7 [watch unwatch cookie 94842635600288] v27'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc0340bb030 con 0x56424599f320 2024-05-08T18:05:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.517+0000 7fc044cdf800 20 remove_watcher() i=7 2024-05-08T18:05:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.517+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56424599f320 msgr2=0x5642459a1710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.517+0000 7fc044cdf800 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56424599f320 0x5642459a1710 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.517+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56424599a930 msgr2=0x56424599cdf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.517+0000 7fc044cdf800 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56424599a930 0x56424599cdf0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.517+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fc03802e910 msgr2=0x7fc03803a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.517+0000 7fc044cdf800 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fc03802e910 0x7fc03803a3c0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.517+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5642459a3c60 msgr2=0x5642459a6120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.517+0000 7fc044cdf800 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5642459a3c60 0x5642459a6120 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf0c03bba0 msgr2=0x7fbf0c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf0c03bba0 0x7fbf0c03e060 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc03800ffd0 tx=0x7fc038023040 comp rx=0 tx=0).stop 2024-05-08T18:05:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564245987fa0 msgr2=0x56424598f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 --2- 172.21.15.50:0/4106731120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564245987fa0 0x56424598f380 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc034044700 tx=0x7fc034094f60 comp rx=0 tx=0).stop 2024-05-08T18:05:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 reap_dead start 2024-05-08T18:05:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc042fdf640 1 -- 172.21.15.50:0/4106731120 reap_dead start 2024-05-08T18:05:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 shutdown_connections 2024-05-08T18:05:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 >> 172.21.15.50:0/4106731120 conn(0x5642459842b0 msgr2=0x564245981910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 shutdown_connections 2024-05-08T18:05:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 -- 172.21.15.50:0/4106731120 wait complete. 2024-05-08T18:05:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 -- 172.21.15.50:0/3874975289 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf1c03bba0 msgr2=0x7fbf1c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 --2- 172.21.15.50:0/3874975289 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf1c03bba0 0x7fbf1c03e060 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x5642459642d0 tx=0x7fc038006000 comp rx=0 tx=0).stop 2024-05-08T18:05:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 -- 172.21.15.50:0/3874975289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424597aea0 msgr2=0x56424597b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 --2- 172.21.15.50:0/3874975289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424597aea0 0x56424597b2c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fc03405bc90 tx=0x7fc0340094d0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 -- 172.21.15.50:0/3874975289 shutdown_connections 2024-05-08T18:05:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 --2- 172.21.15.50:0/3874975289 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf1c03bba0 0x7fbf1c03e060 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 --2- 172.21.15.50:0/3874975289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424597aea0 0x56424597b2c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 -- 172.21.15.50:0/3874975289 >> 172.21.15.50:0/3874975289 conn(0x56424596ef40 msgr2=0x56424596f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 -- 172.21.15.50:0/3874975289 shutdown_connections 2024-05-08T18:05:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 -- 172.21.15.50:0/3874975289 wait complete. 2024-05-08T18:05:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 -- 172.21.15.50:0/3560137342 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564245966100 msgr2=0x5642459684f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564245966100 0x5642459684f0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 -- 172.21.15.50:0/3560137342 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564245862950 msgr2=0x564245963950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564245862950 0x564245963950 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 -- 172.21.15.50:0/3560137342 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56424596a000 msgr2=0x56424596c3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.518+0000 7fc044cdf800 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56424596a000 0x56424596c3f0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.519+0000 7fc044cdf800 1 -- 172.21.15.50:0/3560137342 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf2803bba0 msgr2=0x7fbf2803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.519+0000 7fc044cdf800 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbf2803bba0 0x7fbf2803e060 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc038000c00 tx=0x7fc0380013b0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.519+0000 7fc044cdf800 1 -- 172.21.15.50:0/3560137342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 msgr2=0x5642458621f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.519+0000 7fc044cdf800 1 --2- 172.21.15.50:0/3560137342 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56424585f960 0x5642458621f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc03401ce60 tx=0x7fc034002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.519+0000 7fc042fdf640 1 -- 172.21.15.50:0/3560137342 reap_dead start 2024-05-08T18:05:09.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.519+0000 7fc044cdf800 1 -- 172.21.15.50:0/3560137342 shutdown_connections 2024-05-08T18:05:09.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.519+0000 7fc044cdf800 1 -- 172.21.15.50:0/3560137342 >> 172.21.15.50:0/3560137342 conn(0x56424585b2f0 msgr2=0x56424594cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.519+0000 7fc044cdf800 1 -- 172.21.15.50:0/3560137342 shutdown_connections 2024-05-08T18:05:09.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.519+0000 7fc044cdf800 1 -- 172.21.15.50:0/3560137342 wait complete. 2024-05-08T18:05:09.531 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-05-08T18:05:09.531 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-05-08T18:05:09.532 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --tenant testx --uid tenanteduser 2024-05-08T18:05:09.579 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:09.579 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:09.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.581+0000 7f2f16c19800 1 Processor -- start 2024-05-08T18:05:09.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.581+0000 7f2f16c19800 1 -- start start 2024-05-08T18:05:09.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.581+0000 7f2f16c19800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5644b5f3faf0 0x5644b5f485f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.581+0000 7f2f16c19800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5644b5eb3030 con 0x5644b5f3dfb0 2024-05-08T18:05:09.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.581+0000 7f2f16c19800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5644b5ddfe20 con 0x5644b5f3faf0 2024-05-08T18:05:09.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.582+0000 7f2f13f10640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5644b5f3faf0 0x5644b5f485f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.582+0000 7f2f14711640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5644b5f3dfb0 0x5644b5f3e3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43254/0 (socket says 172.21.15.50:43254) 2024-05-08T18:05:09.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.582+0000 7f2f14711640 1 -- 172.21.15.50:0/2977182677 learned_addr learned my addr 172.21.15.50:0/2977182677 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:09.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.582+0000 7f2f13f10640 1 -- 172.21.15.50:0/2977182677 >> v1:172.21.15.46:6789/0 conn(0x5644b5f3dfb0 legacy=0x5644b5f3e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.582+0000 7f2f1370f640 1 -- 172.21.15.50:0/2977182677 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 20793633 0 0) 0x5644b5eb3030 con 0x5644b5f3dfb0 2024-05-08T18:05:09.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.582+0000 7f2f13f10640 1 -- 172.21.15.50:0/2977182677 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b5f49b40 con 0x5644b5f3faf0 2024-05-08T18:05:09.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.584+0000 7f2f13f10640 1 --2- 172.21.15.50:0/2977182677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3faf0 0x5644b5f485f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00a470 tx=0x7f2efc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c45c0cb1dd3a50b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.584+0000 7f2f1370f640 1 -- 172.21.15.50:0/2977182677 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2efc057070 con 0x5644b5f3faf0 2024-05-08T18:05:09.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.584+0000 7f2f1370f640 1 -- 172.21.15.50:0/2977182677 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2efc04adc0 con 0x5644b5f3faf0 2024-05-08T18:05:09.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.584+0000 7f2f1370f640 1 -- 172.21.15.50:0/2977182677 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2efc052590 con 0x5644b5f3faf0 2024-05-08T18:05:09.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.584+0000 7f2f16c19800 1 -- 172.21.15.50:0/2977182677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3faf0 msgr2=0x5644b5f485f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.584+0000 7f2f16c19800 1 --2- 172.21.15.50:0/2977182677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3faf0 0x5644b5f485f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00a470 tx=0x7f2efc01ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:09.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.585+0000 7f2f16c19800 1 -- 172.21.15.50:0/2977182677 shutdown_connections 2024-05-08T18:05:09.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.585+0000 7f2f16c19800 1 --2- 172.21.15.50:0/2977182677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3faf0 0x5644b5f485f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.585+0000 7f2f16c19800 1 -- 172.21.15.50:0/2977182677 >> 172.21.15.50:0/2977182677 conn(0x5644b5f39af0 msgr2=0x5644b5f3bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.585+0000 7f2f16c19800 1 -- 172.21.15.50:0/2977182677 shutdown_connections 2024-05-08T18:05:09.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.585+0000 7f2f16c19800 1 -- 172.21.15.50:0/2977182677 wait complete. 2024-05-08T18:05:09.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.600+0000 7f2f16c19800 1 Processor -- start 2024-05-08T18:05:09.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.600+0000 7f2f16c19800 1 -- start start 2024-05-08T18:05:09.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.600+0000 7f2f16c19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3dfb0 0x5644b6032d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.600+0000 7f2f16c19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5644b5e8aa30 con 0x5644b5f3dfb0 2024-05-08T18:05:09.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.600+0000 7f2f14711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3dfb0 0x5644b6032d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.600+0000 7f2f14711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3dfb0 0x5644b6032d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38336/0 (socket says 172.21.15.50:38336) 2024-05-08T18:05:09.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.600+0000 7f2f14711640 1 -- 172.21.15.50:0/1285954627 learned_addr learned my addr 172.21.15.50:0/1285954627 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.601+0000 7f2f14711640 1 -- 172.21.15.50:0/1285954627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b60332d0 con 0x5644b5f3dfb0 2024-05-08T18:05:09.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.601+0000 7f2f14711640 1 --2- 172.21.15.50:0/1285954627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3dfb0 0x5644b6032d00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2f04006f70 tx=0x7f2f04001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5739e52b5702e125 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.601+0000 7f2e0260c640 1 -- 172.21.15.50:0/1285954627 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f04003450 con 0x5644b5f3dfb0 2024-05-08T18:05:09.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.601+0000 7f2e0260c640 1 -- 172.21.15.50:0/1285954627 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f04003610 con 0x5644b5f3dfb0 2024-05-08T18:05:09.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.601+0000 7f2e0260c640 1 -- 172.21.15.50:0/1285954627 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f040039b0 con 0x5644b5f3dfb0 2024-05-08T18:05:09.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.602+0000 7f2f16c19800 1 -- 172.21.15.50:0/1285954627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3dfb0 msgr2=0x5644b6032d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.602+0000 7f2f16c19800 1 --2- 172.21.15.50:0/1285954627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3dfb0 0x5644b6032d00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2f04006f70 tx=0x7f2f04001b00 comp rx=0 tx=0).stop 2024-05-08T18:05:09.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.602+0000 7f2f16c19800 1 -- 172.21.15.50:0/1285954627 shutdown_connections 2024-05-08T18:05:09.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.602+0000 7f2f16c19800 1 --2- 172.21.15.50:0/1285954627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3dfb0 0x5644b6032d00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.602+0000 7f2f16c19800 1 -- 172.21.15.50:0/1285954627 >> 172.21.15.50:0/1285954627 conn(0x5644b5f39af0 msgr2=0x5644b602e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.602+0000 7f2f16c19800 1 -- 172.21.15.50:0/1285954627 shutdown_connections 2024-05-08T18:05:09.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.602+0000 7f2f16c19800 1 -- 172.21.15.50:0/1285954627 wait complete. 2024-05-08T18:05:09.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.602+0000 7f2f16c19800 1 Processor -- start 2024-05-08T18:05:09.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.602+0000 7f2f16c19800 1 -- start start 2024-05-08T18:05:09.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.602+0000 7f2f16c19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3dfb0 0x5644b603a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.602+0000 7f2f16c19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5644b5f48e00 con 0x5644b5f3dfb0 2024-05-08T18:05:09.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.603+0000 7f2f14711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3dfb0 0x5644b603a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.603+0000 7f2f14711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3dfb0 0x5644b603a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38346/0 (socket says 172.21.15.50:38346) 2024-05-08T18:05:09.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.603+0000 7f2f14711640 1 -- 172.21.15.50:0/4237611277 learned_addr learned my addr 172.21.15.50:0/4237611277 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.603+0000 7f2f14711640 1 -- 172.21.15.50:0/4237611277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b603adc0 con 0x5644b5f3dfb0 2024-05-08T18:05:09.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.603+0000 7f2f14711640 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3dfb0 0x5644b603a880 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2f04000f80 tx=0x7f2f04001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.603+0000 7f2e00e09640 1 -- 172.21.15.50:0/4237611277 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f04014990 con 0x5644b5f3dfb0 2024-05-08T18:05:09.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.603+0000 7f2e00e09640 1 -- 172.21.15.50:0/4237611277 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f04023070 con 0x5644b5f3dfb0 2024-05-08T18:05:09.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.603+0000 7f2f16c19800 1 -- 172.21.15.50:0/4237611277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5644b603b050 con 0x5644b5f3dfb0 2024-05-08T18:05:09.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.603+0000 7f2e00e09640 1 -- 172.21.15.50:0/4237611277 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f04014ca0 con 0x5644b5f3dfb0 2024-05-08T18:05:09.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.603+0000 7f2f16c19800 1 -- 172.21.15.50:0/4237611277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b603dcb0 con 0x5644b5f3dfb0 2024-05-08T18:05:09.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.604+0000 7f2e00e09640 1 -- 172.21.15.50:0/4237611277 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2f04010070 con 0x5644b5f3dfb0 2024-05-08T18:05:09.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.604+0000 7f2e00e09640 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2dfc03bba0 0x7f2dfc03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.604+0000 7f2e00e09640 1 -- 172.21.15.50:0/4237611277 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f2f04018070 con 0x5644b5f3dfb0 2024-05-08T18:05:09.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.605+0000 7f2f13f10640 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2dfc03bba0 0x7f2dfc03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.605+0000 7f2f16c19800 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5644b5f3faf0 0x5644b6041a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.605+0000 7f2f16c19800 1 -- 172.21.15.50:0/4237611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b6042000 con 0x5644b5f3faf0 2024-05-08T18:05:09.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.605+0000 7f2f14f12640 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5644b5f3faf0 0x5644b6041a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.605+0000 7f2f13f10640 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2dfc03bba0 0x7f2dfc03e060 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2efc054000 tx=0x7f2efc04e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.605+0000 7f2f14f12640 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5644b5f3faf0 0x5644b6041a70 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.605+0000 7f2f14f12640 1 -- 172.21.15.50:0/4237611277 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2f0c0083e0 con 0x5644b5f3faf0 2024-05-08T18:05:09.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.605+0000 7f2f16c19800 1 -- 172.21.15.50:0/4237611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b603f020 con 0x5644b5f3faf0 2024-05-08T18:05:09.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.606+0000 7f2f14f12640 1 -- 172.21.15.50:0/4237611277 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2f0c0083e0 con 0x5644b5f3faf0 2024-05-08T18:05:09.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.606+0000 7f2f16c19800 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5644b6044330 0x5644b6046720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.606+0000 7f2f16c19800 1 -- 172.21.15.50:0/4237611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b6046e10 con 0x5644b6044330 2024-05-08T18:05:09.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.606+0000 7f2f14711640 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5644b6044330 0x5644b6046720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.606+0000 7f2f14711640 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5644b6044330 0x5644b6046720 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.606+0000 7f2f14711640 1 -- 172.21.15.50:0/4237611277 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b6044330 2024-05-08T18:05:09.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.606+0000 7f2f16c19800 1 -- 172.21.15.50:0/4237611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b60431d0 con 0x5644b5f3faf0 2024-05-08T18:05:09.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.607+0000 7f2f14f12640 1 -- 172.21.15.50:0/4237611277 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2f0c0083e0 con 0x5644b5f3faf0 2024-05-08T18:05:09.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.607+0000 7f2f16c19800 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5644b6048270 0x5644b604a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.607+0000 7f2f16c19800 1 -- 172.21.15.50:0/4237611277 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b604ada0 con 0x5644b6048270 2024-05-08T18:05:09.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.607+0000 7f2f13f10640 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5644b6048270 0x5644b604a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.607+0000 7f2f13f10640 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5644b6048270 0x5644b604a660 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.608+0000 7f2f13f10640 1 -- 172.21.15.50:0/4237611277 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f2efc05a040 con 0x5644b6048270 2024-05-08T18:05:09.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.608+0000 7f2f16c19800 1 Processor -- start 2024-05-08T18:05:09.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.608+0000 7f2f16c19800 1 -- start start 2024-05-08T18:05:09.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.608+0000 7f2f16c19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60590d0 0x5644b60594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.608+0000 7f2f16c19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5644b5e31330 con 0x5644b60590d0 2024-05-08T18:05:09.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.608+0000 7f2f14711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60590d0 0x5644b60594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.608+0000 7f2f14711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60590d0 0x5644b60594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38350/0 (socket says 172.21.15.50:38350) 2024-05-08T18:05:09.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.608+0000 7f2f14711640 1 -- 172.21.15.50:0/536975330 learned_addr learned my addr 172.21.15.50:0/536975330 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.608+0000 7f2f14711640 1 -- 172.21.15.50:0/536975330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b6059a30 con 0x5644b60590d0 2024-05-08T18:05:09.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.609+0000 7f2f14711640 1 --2- 172.21.15.50:0/536975330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60590d0 0x5644b60594f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2f04002250 tx=0x7f2f04012e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.609+0000 7f2df9ffb640 1 -- 172.21.15.50:0/536975330 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f0401d530 con 0x5644b60590d0 2024-05-08T18:05:09.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.609+0000 7f2df9ffb640 1 -- 172.21.15.50:0/536975330 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0401d8d0 con 0x5644b60590d0 2024-05-08T18:05:09.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.609+0000 7f2f16c19800 1 -- 172.21.15.50:0/536975330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5644b605a4a0 con 0x5644b60590d0 2024-05-08T18:05:09.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.609+0000 7f2df9ffb640 1 -- 172.21.15.50:0/536975330 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f0401d350 con 0x5644b60590d0 2024-05-08T18:05:09.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.609+0000 7f2f16c19800 1 -- 172.21.15.50:0/536975330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b605a9a0 con 0x5644b60590d0 2024-05-08T18:05:09.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.610+0000 7f2df9ffb640 1 -- 172.21.15.50:0/536975330 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2f04028670 con 0x5644b60590d0 2024-05-08T18:05:09.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.610+0000 7f2df9ffb640 1 --2- 172.21.15.50:0/536975330 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2dec03bb50 0x7f2dec03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.610+0000 7f2df9ffb640 1 -- 172.21.15.50:0/536975330 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f2f0403d8c0 con 0x5644b60590d0 2024-05-08T18:05:09.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.610+0000 7f2f13f10640 1 --2- 172.21.15.50:0/536975330 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2dec03bb50 0x7f2dec03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.610+0000 7f2f16c19800 1 Processor -- start 2024-05-08T18:05:09.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.610+0000 7f2f16c19800 1 -- start start 2024-05-08T18:05:09.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.610+0000 7f2f13f10640 1 --2- 172.21.15.50:0/536975330 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2dec03bb50 0x7f2dec03e010 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x5644b60424d0 tx=0x7f2efc005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.610+0000 7f2f16c19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60663d0 0x5644b60667f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.610+0000 7f2f16c19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5644b6033650 con 0x5644b60663d0 2024-05-08T18:05:09.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.611+0000 7f2f14711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60663d0 0x5644b60667f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.611+0000 7f2f14711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60663d0 0x5644b60667f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38362/0 (socket says 172.21.15.50:38362) 2024-05-08T18:05:09.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.611+0000 7f2f14711640 1 -- 172.21.15.50:0/934126170 learned_addr learned my addr 172.21.15.50:0/934126170 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.611+0000 7f2f14711640 1 -- 172.21.15.50:0/934126170 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b6066dc0 con 0x5644b60663d0 2024-05-08T18:05:09.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.611+0000 7f2f14711640 1 --2- 172.21.15.50:0/934126170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60663d0 0x5644b60667f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2f04034fb0 tx=0x7f2f04049180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0603c7dbac97d7a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.611+0000 7f2df37fe640 1 -- 172.21.15.50:0/934126170 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f040329b0 con 0x5644b60663d0 2024-05-08T18:05:09.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.611+0000 7f2df37fe640 1 -- 172.21.15.50:0/934126170 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f040050d0 con 0x5644b60663d0 2024-05-08T18:05:09.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.611+0000 7f2df37fe640 1 -- 172.21.15.50:0/934126170 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f0404a210 con 0x5644b60663d0 2024-05-08T18:05:09.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.612+0000 7f2f16c19800 1 -- 172.21.15.50:0/934126170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60663d0 msgr2=0x5644b60667f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.612+0000 7f2f16c19800 1 --2- 172.21.15.50:0/934126170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60663d0 0x5644b60667f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2f04034fb0 tx=0x7f2f04049180 comp rx=0 tx=0).stop 2024-05-08T18:05:09.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.612+0000 7f2f16c19800 1 -- 172.21.15.50:0/934126170 shutdown_connections 2024-05-08T18:05:09.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.612+0000 7f2f16c19800 1 --2- 172.21.15.50:0/934126170 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60663d0 0x5644b60667f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.612+0000 7f2f16c19800 1 -- 172.21.15.50:0/934126170 >> 172.21.15.50:0/934126170 conn(0x5644b60626f0 msgr2=0x5644b6060750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.612+0000 7f2f16c19800 1 -- 172.21.15.50:0/934126170 shutdown_connections 2024-05-08T18:05:09.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.612+0000 7f2f16c19800 1 -- 172.21.15.50:0/934126170 wait complete. 2024-05-08T18:05:09.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.612+0000 7f2f16c19800 1 Processor -- start 2024-05-08T18:05:09.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.612+0000 7f2f16c19800 1 -- start start 2024-05-08T18:05:09.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.612+0000 7f2f16c19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60663d0 0x5644b606d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.612+0000 7f2f16c19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5644b60674e0 con 0x5644b60663d0 2024-05-08T18:05:09.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.612+0000 7f2f14711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60663d0 0x5644b606d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.612+0000 7f2f14711640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60663d0 0x5644b606d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38378/0 (socket says 172.21.15.50:38378) 2024-05-08T18:05:09.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.612+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 learned_addr learned my addr 172.21.15.50:0/549626291 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.613+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644b606dd60 con 0x5644b60663d0 2024-05-08T18:05:09.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.613+0000 7f2f14711640 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60663d0 0x5644b606d820 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2f04034fb0 tx=0x7f2f04034ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.613+0000 7f2df1ffb640 1 -- 172.21.15.50:0/549626291 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f040329b0 con 0x5644b60663d0 2024-05-08T18:05:09.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.613+0000 7f2df1ffb640 1 -- 172.21.15.50:0/549626291 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0403dd50 con 0x5644b60663d0 2024-05-08T18:05:09.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.613+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5644b606dd60 con 0x5644b60663d0 2024-05-08T18:05:09.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.613+0000 7f2df1ffb640 1 -- 172.21.15.50:0/549626291 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f040278e0 con 0x5644b60663d0 2024-05-08T18:05:09.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.613+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644b606dd60 con 0x5644b60663d0 2024-05-08T18:05:09.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.614+0000 7f2df1ffb640 1 -- 172.21.15.50:0/549626291 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2f0406f020 con 0x5644b60663d0 2024-05-08T18:05:09.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.614+0000 7f2df1ffb640 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2de403bba0 0x7f2de403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.614+0000 7f2df1ffb640 1 -- 172.21.15.50:0/549626291 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f2f0406d020 con 0x5644b60663d0 2024-05-08T18:05:09.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.614+0000 7f2f13f10640 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2de403bba0 0x7f2de403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.615+0000 7f2f13f10640 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2de403bba0 0x7f2de403e060 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2efc000c00 tx=0x7f2efc060040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.615+0000 7f2f16c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.615+0000 7f2f16c19800 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5644b6078ef0 0x5644b607b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.615+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b607bb00 con 0x5644b6078ef0 2024-05-08T18:05:09.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.615+0000 7f2f14f12640 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5644b6078ef0 0x5644b607b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.615+0000 7f2f14f12640 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5644b6078ef0 0x5644b607b3b0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.615+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2f0c0083e0 con 0x5644b6078ef0 2024-05-08T18:05:09.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f16c19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f16c19800 20 realm 2024-05-08T18:05:09.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f16c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b607bf10 con 0x5644b6078ef0 2024-05-08T18:05:09.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2f0c0083e0 con 0x5644b6078ef0 2024-05-08T18:05:09.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f16c19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f16c19800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:09.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f16c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6078ef0 2024-05-08T18:05:09.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2f0c0083e0 con 0x5644b6078ef0 2024-05-08T18:05:09.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f16c19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f16c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b607cb80 con 0x5644b6078ef0 2024-05-08T18:05:09.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2f0c0083e0 con 0x5644b6078ef0 2024-05-08T18:05:09.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f16c19800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:09.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f16c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f16c19800 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5644b607d540 0x5644b607f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.616+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b607ff80 con 0x5644b607d540 2024-05-08T18:05:09.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.617+0000 7f2f14711640 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5644b607d540 0x5644b607f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.617+0000 7f2f14711640 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5644b607d540 0x5644b607f930 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.617+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.617+0000 7f2f16c19800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.617+0000 7f2f16c19800 20 searching for the correct realm 2024-05-08T18:05:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.617+0000 7f2f16c19800 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5644b6081f10 0x5644b60843d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.617+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b6084aa0 con 0x5644b6081f10 2024-05-08T18:05:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.617+0000 7f2f13f10640 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5644b6081f10 0x5644b60843d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.618+0000 7f2f13f10640 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5644b6081f10 0x5644b60843d0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.618+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.618+0000 7f2f13f10640 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2efc052b30 0x7f2efc080930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.618+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b6084aa0 con 0x7f2efc052b30 2024-05-08T18:05:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.618+0000 7f2f14f12640 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2efc052b30 0x7f2efc080930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.618+0000 7f2f14f12640 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2efc052b30 0x7f2efc080930 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.619+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0c0083e0 con 0x7f2efc052b30 2024-05-08T18:05:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.619+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b607d540 2024-05-08T18:05:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.619+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.619+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6078ef0 2024-05-08T18:05:09.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.619+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0c0083e0 con 0x5644b6078ef0 2024-05-08T18:05:09.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.619+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6078ef0 2024-05-08T18:05:09.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.619+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.619+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b607d540 2024-05-08T18:05:09.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.620+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.620+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b607d540 2024-05-08T18:05:09.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.620+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.620+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x7f2efc052b30 2024-05-08T18:05:09.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.620+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.620+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6081f10 2024-05-08T18:05:09.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.620+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.620+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6078ef0 2024-05-08T18:05:09.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.621+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.621+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b607d540 2024-05-08T18:05:09.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.621+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.621+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6078ef0 2024-05-08T18:05:09.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.621+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.621+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x7f2efc052b30 2024-05-08T18:05:09.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.621+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.621+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6081f10 2024-05-08T18:05:09.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.622+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.622+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6081f10 2024-05-08T18:05:09.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.622+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.622+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b607d540 2024-05-08T18:05:09.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.622+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.622+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6081f10 2024-05-08T18:05:09.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.622+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.622+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6081f10 2024-05-08T18:05:09.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.622+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.622+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6078ef0 2024-05-08T18:05:09.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.623+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.623+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6078ef0 2024-05-08T18:05:09.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.623+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.623+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b607d540 2024-05-08T18:05:09.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.623+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.623+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6081f10 2024-05-08T18:05:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.624+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.624+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x7f2efc052b30 2024-05-08T18:05:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.624+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.624+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6078ef0 2024-05-08T18:05:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.624+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.624+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b607d540 2024-05-08T18:05:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.624+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.624+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6081f10 2024-05-08T18:05:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.625+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.625+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b607d540 2024-05-08T18:05:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.625+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.625+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x7f2efc052b30 2024-05-08T18:05:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.625+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.625+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6081f10 2024-05-08T18:05:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.625+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.625+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b607d540 2024-05-08T18:05:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x5644b6078ef0 2024-05-08T18:05:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5644b607c7e0 con 0x7f2efc052b30 2024-05-08T18:05:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f16c19800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f16c19800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f16c19800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f16c19800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f16c19800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f16c19800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:09.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f16c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b6080320 con 0x5644b6078ef0 2024-05-08T18:05:09.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f16c19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f16c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.626+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b6080320 con 0x5644b6078ef0 2024-05-08T18:05:09.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f16c19800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:09.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f16c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b6080320 con 0x5644b6081f10 2024-05-08T18:05:09.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f16c19800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:09.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f16c19800 20 zone default found 2024-05-08T18:05:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f16c19800 4 Realm: () 2024-05-08T18:05:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f16c19800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f16c19800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f16c19800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f16c19800 20 zonegroup default 2024-05-08T18:05:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f16c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b6080cb0 con 0x5644b607d540 2024-05-08T18:05:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f16c19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f16c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b6080860 con 0x5644b6081f10 2024-05-08T18:05:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.627+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.628+0000 7f2f16c19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.628+0000 7f2f16c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.628+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b6086e90 con 0x5644b6081f10 2024-05-08T18:05:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.628+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.628+0000 7f2f16c19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.628+0000 7f2f16c19800 20 started sync module instance, tier type = 2024-05-08T18:05:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.628+0000 7f2f16c19800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.628+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6086240 con 0x7f2efc052b30 2024-05-08T18:05:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.629+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.629+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94853111768928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6086240 con 0x7f2efc052b30 2024-05-08T18:05:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.629+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6087910 con 0x5644b6078ef0 2024-05-08T18:05:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.630+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94853111768928] v27'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.630+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.630+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94853111766896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6087910 con 0x5644b6078ef0 2024-05-08T18:05:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.630+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b60882d0 con 0x5644b6081f10 2024-05-08T18:05:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.631+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.631+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94853111774752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6088b70 con 0x5644b6081f10 2024-05-08T18:05:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.631+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6089630 con 0x7f2efc052b30 2024-05-08T18:05:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.631+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94853111766896] v27'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.632+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94853111774752] v27'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.632+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.632+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94853111776960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6087460 con 0x7f2efc052b30 2024-05-08T18:05:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.632+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6089f10 con 0x5644b6078ef0 2024-05-08T18:05:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.633+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.633+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94853111781792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b608a300 con 0x5644b6078ef0 2024-05-08T18:05:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.633+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b608ae20 con 0x5644b607d540 2024-05-08T18:05:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.633+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94853111776960] v27'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.634+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94853111781792] v27'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.634+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.634+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94853111782848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6089100 con 0x5644b607d540 2024-05-08T18:05:09.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.634+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b608b6c0 con 0x5644b607d540 2024-05-08T18:05:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.635+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94853111782848] v27'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.635+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.635+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94853111788048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b608bae0 con 0x5644b607d540 2024-05-08T18:05:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.635+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b608c4a0 con 0x5644b607d540 2024-05-08T18:05:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.636+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94853111788048] v27'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.636+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.636+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94853111789584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b608c4a0 con 0x5644b607d540 2024-05-08T18:05:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.636+0000 7f2f16c19800 20 add_watcher() i=0 2024-05-08T18:05:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.636+0000 7f2f16c19800 20 add_watcher() i=1 2024-05-08T18:05:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.636+0000 7f2f16c19800 20 add_watcher() i=2 2024-05-08T18:05:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.636+0000 7f2f16c19800 20 add_watcher() i=3 2024-05-08T18:05:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.636+0000 7f2f16c19800 20 add_watcher() i=4 2024-05-08T18:05:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.636+0000 7f2f16c19800 20 add_watcher() i=5 2024-05-08T18:05:09.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.636+0000 7f2f16c19800 20 add_watcher() i=6 2024-05-08T18:05:09.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.637+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94853111789584] v27'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.637+0000 7f2f16c19800 20 add_watcher() i=7 2024-05-08T18:05:09.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.637+0000 7f2f16c19800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:09.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.637+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b608f8a0 con 0x5644b6078ef0 2024-05-08T18:05:09.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.638+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.638+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94853111792064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b608cd60 con 0x5644b6078ef0 2024-05-08T18:05:09.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.639+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94853111792064] v27'134 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.639+0000 7f2dca7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:09.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.639+0000 7f2f16c19800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:09.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.639+0000 7f2f16c19800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:09.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.639+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b608f520 con 0x5644b6078ef0 2024-05-08T18:05:09.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.640+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'22 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.640+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b60988f0 con 0x5644b607d540 2024-05-08T18:05:09.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.641+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.641+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b60988f0 con 0x5644b6081f10 2024-05-08T18:05:09.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.642+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.642+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b60988f0 con 0x5644b6078ef0 2024-05-08T18:05:09.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.643+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'135 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.643+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b60991e0 con 0x7f2efc052b30 2024-05-08T18:05:09.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.644+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'38 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.644+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6099750 con 0x5644b6081f10 2024-05-08T18:05:09.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.645+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'25 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.645+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6099b50 con 0x5644b6078ef0 2024-05-08T18:05:09.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.646+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.646+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6099b50 con 0x5644b607d540 2024-05-08T18:05:09.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.648+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'25 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.648+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6099be0 con 0x7f2efc052b30 2024-05-08T18:05:09.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.649+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.649+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6099be0 con 0x5644b6081f10 2024-05-08T18:05:09.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.650+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'22 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.650+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a1b0 con 0x7f2efc052b30 2024-05-08T18:05:09.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.651+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'39 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.651+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b607d540 2024-05-08T18:05:09.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.652+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'18 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.652+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x7f2efc052b30 2024-05-08T18:05:09.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.653+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.653+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b607d540 2024-05-08T18:05:09.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.654+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'31 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.654+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b6078ef0 2024-05-08T18:05:09.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.655+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'136 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.655+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b6081f10 2024-05-08T18:05:09.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.656+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'33 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.656+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x7f2efc052b30 2024-05-08T18:05:09.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.657+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'40 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.657+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b6081f10 2024-05-08T18:05:09.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.658+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'16 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.658+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b6078ef0 2024-05-08T18:05:09.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.659+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'24 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.659+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b6078ef0 2024-05-08T18:05:09.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.660+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'137 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.660+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b6081f10 2024-05-08T18:05:09.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.661+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'34 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.661+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x7f2efc052b30 2024-05-08T18:05:09.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.662+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'26 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.662+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b6081f10 2024-05-08T18:05:09.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.663+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.663+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b6081f10 2024-05-08T18:05:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.664+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'22 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.664+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b6081f10 2024-05-08T18:05:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.665+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.665+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b6081f10 2024-05-08T18:05:09.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.666+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'26 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.666+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b607d540 2024-05-08T18:05:09.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.668+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.668+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b6081f10 2024-05-08T18:05:09.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.669+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'16 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.669+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b607d540 2024-05-08T18:05:09.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.670+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'26 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.670+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b6081f10 2024-05-08T18:05:09.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.671+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'35 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.671+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x5644b607d540 2024-05-08T18:05:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.672+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'32 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.672+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b609a170 con 0x7f2efc052b30 2024-05-08T18:05:09.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.673+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'22 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.674+0000 7f2dc97e2640 20 reqs_thread_entry: start 2024-05-08T18:05:09.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.674+0000 7f2f16c19800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:09.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.674+0000 7f2f16c19800 20 Filter name: none 2024-05-08T18:05:09.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.674+0000 7f2d3affd640 20 reqs_thread_entry: start 2024-05-08T18:05:09.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.674+0000 7f2f16c19800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-05-08T18:05:09.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.674+0000 7f2f16c19800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.674+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:94 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b60c5440 con 0x5644b607d540 2024-05-08T18:05:09.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.674+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(94 testx$tenanteduser [call out=48b,read 0~419 out=419b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 246+0+542 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.675+0000 7f2f16c19800 20 rados_obj.operate() r=0 bl.length=419 2024-05-08T18:05:09.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.675+0000 7f2f16c19800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2024-05-08T18:05:09.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.675+0000 7f2f16c19800 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-05-08T18:05:09.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.675+0000 7f2f16c19800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:09.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.675+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:95 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b60c4e90 con 0x5644b6078ef0 2024-05-08T18:05:09.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.675+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(95 testx$tenanteduser.buckets [call out=15b] v0'0 uv2 ondisk = 0) ==== 170+0+15 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.675+0000 7f2f16c19800 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-05-08T18:05:09.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.675+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:96 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b60c4e90 con 0x5644b607d540 2024-05-08T18:05:09.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.676+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(96 p5YnriCv1nAtykxBrupQ [delete] v27'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.676+0000 7f2f16c19800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-05-08T18:05:09.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.676+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [notify cookie 94853112031760 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b60c82f0 con 0x5644b6081f10 2024-05-08T18:05:09.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.677+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== watch-notify(notify (1) cookie 94853111774752 notify 115964116997 ret 0) ==== 212+0+0 (crc 0 0 0) 0x5644b60997a0 con 0x5644b6081f10 2024-05-08T18:05:09.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.677+0000 7f2df37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116997 cookie 94853111774752 notifier 4557 bl.length()=170 2024-05-08T18:05:09.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.677+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 28 ==== osd_op_reply(97 notify.2 [notify cookie 94853112031760 out=8b] v0'0 uv141 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.677+0000 7f2df37fe640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f2dd8003d60 con 0x5644b6081f10 2024-05-08T18:05:09.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.677+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 29 ==== osd_op_reply(98 notify.2 [notify-ack] v0'0 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.677+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 30 ==== watch-notify(notify_complete (2) cookie 94853112031760 notify 115964116997 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5644b60993b0 con 0x5644b6081f10 2024-05-08T18:05:09.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.677+0000 7f2f16c19800 10 removing email index: 2024-05-08T18:05:09.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.677+0000 7f2f16c19800 10 removing user buckets index 2024-05-08T18:05:09.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.677+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:99 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b60c5eb0 con 0x5644b6078ef0 2024-05-08T18:05:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.678+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(99 testx$tenanteduser.buckets [delete] v27'3 uv3 ondisk = 0) ==== 170+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.678+0000 7f2f16c19800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser.buckets, ofs0, ns] 2024-05-08T18:05:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.678+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [notify cookie 94853112031760 in=187b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b60c82f0 con 0x5644b6078ef0 2024-05-08T18:05:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.678+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== watch-notify(notify (1) cookie 94853111781792 notify 115964117011 ret 0) ==== 217+0+0 (crc 0 0 0) 0x5644b6099580 con 0x5644b6078ef0 2024-05-08T18:05:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.678+0000 7f2df37fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117011 cookie 94853111781792 notifier 4557 bl.length()=175 2024-05-08T18:05:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.678+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(100 notify.4 [notify cookie 94853112031760 out=8b] v0'0 uv141 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.678+0000 7f2df37fe640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f2dd8003d60 con 0x5644b6078ef0 2024-05-08T18:05:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.678+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(101 notify.4 [notify-ack] v0'0 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.679+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 32 ==== watch-notify(notify_complete (2) cookie 94853112031760 notify 115964117011 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5644b60978e0 con 0x5644b6078ef0 2024-05-08T18:05:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.679+0000 7f2f16c19800 10 removing user index: testx$tenanteduser 2024-05-08T18:05:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.679+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b60ca050 con 0x5644b607d540 2024-05-08T18:05:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.680+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(102 testx$tenanteduser [call,call,delete] v27'8 uv8 ondisk = 0) ==== 246+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.680+0000 7f2f16c19800 10 removing default.rgw.meta+users.uid+testx$tenanteduser from cache 2024-05-08T18:05:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.680+0000 7f2f16c19800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2024-05-08T18:05:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.680+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:103 7.e 7:7759931f:::notify.3:head [notify cookie 94853112044448 in=179b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5644b60c9fc0 con 0x7f2efc052b30 2024-05-08T18:05:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.680+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== watch-notify(notify (1) cookie 94853111776960 notify 115964117005 ret 0) ==== 209+0+0 (crc 0 0 0) 0x5644b60978e0 con 0x7f2efc052b30 2024-05-08T18:05:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.680+0000 7f2df2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117005 cookie 94853111776960 notifier 4557 bl.length()=167 2024-05-08T18:05:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.680+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(103 notify.3 [notify cookie 94853112044448 out=8b] v0'0 uv141 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.680+0000 7f2df2ffd640 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:104 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f2d480032c0 con 0x7f2efc052b30 2024-05-08T18:05:09.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.680+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 20 ==== watch-notify(notify_complete (2) cookie 94853112044448 notify 115964117005 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5644b608f2e0 con 0x7f2efc052b30 2024-05-08T18:05:09.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.680+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 21 ==== osd_op_reply(104 notify.3 [notify-ack] v0'0 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.681+0000 7f2dca7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:09.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.681+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:105 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94853111792064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6096e00 con 0x5644b6078ef0 2024-05-08T18:05:09.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.682+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 33 ==== osd_op_reply(105 data_loggenerations_metadata [watch unwatch cookie 94853111792064] v27'138 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.682+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:106 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94853111768928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b608c190 con 0x7f2efc052b30 2024-05-08T18:05:09.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.683+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 22 ==== osd_op_reply(106 notify.0 [watch unwatch cookie 94853111768928] v27'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.683+0000 7f2f16c19800 20 remove_watcher() i=0 2024-05-08T18:05:09.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.683+0000 7f2f16c19800 2 removed watcher, disabling cache 2024-05-08T18:05:09.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.683+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:107 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94853111766896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6086760 con 0x5644b6078ef0 2024-05-08T18:05:09.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.684+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 34 ==== osd_op_reply(107 notify.1 [watch unwatch cookie 94853111766896] v27'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.684+0000 7f2f16c19800 20 remove_watcher() i=1 2024-05-08T18:05:09.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.685+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94853111774752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6085f70 con 0x5644b6081f10 2024-05-08T18:05:09.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.685+0000 7f2f13f10640 1 -- 172.21.15.50:0/549626291 <== osd.3 v2:172.21.15.50:6800/3130492744 31 ==== osd_op_reply(108 notify.2 [watch unwatch cookie 94853111774752] v27'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2efc070070 con 0x5644b6081f10 2024-05-08T18:05:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.685+0000 7f2f16c19800 20 remove_watcher() i=2 2024-05-08T18:05:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.685+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94853111776960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6085f70 con 0x7f2efc052b30 2024-05-08T18:05:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.686+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.2 v2:172.21.15.50:6811/3039646617 23 ==== osd_op_reply(109 notify.3 [watch unwatch cookie 94853111776960] v27'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x7f2efc052b30 2024-05-08T18:05:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.686+0000 7f2f16c19800 20 remove_watcher() i=3 2024-05-08T18:05:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.686+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:110 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94853111781792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6085f70 con 0x5644b6078ef0 2024-05-08T18:05:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.687+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 <== osd.1 v2:172.21.15.50:6808/2141923372 35 ==== osd_op_reply(110 notify.4 [watch unwatch cookie 94853111781792] v27'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0c026040 con 0x5644b6078ef0 2024-05-08T18:05:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.687+0000 7f2f16c19800 20 remove_watcher() i=4 2024-05-08T18:05:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.687+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:111 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94853111782848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b60899a0 con 0x5644b607d540 2024-05-08T18:05:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.688+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(111 notify.5 [watch unwatch cookie 94853111782848] v27'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.688+0000 7f2f16c19800 20 remove_watcher() i=5 2024-05-08T18:05:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.688+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94853111788048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b6085f70 con 0x5644b607d540 2024-05-08T18:05:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.690+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(112 notify.6 [watch unwatch cookie 94853111788048] v27'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.690+0000 7f2f16c19800 20 remove_watcher() i=6 2024-05-08T18:05:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.690+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:113 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94853111789584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5644b608b210 con 0x5644b607d540 2024-05-08T18:05:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f14711640 1 -- 172.21.15.50:0/549626291 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== osd_op_reply(113 notify.7 [watch unwatch cookie 94853111789584] v27'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f0401b070 con 0x5644b607d540 2024-05-08T18:05:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 20 remove_watcher() i=7 2024-05-08T18:05:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5644b607d540 msgr2=0x5644b607f930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5644b607d540 0x5644b607f930 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5644b6078ef0 msgr2=0x5644b607b3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5644b6078ef0 0x5644b607b3b0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2efc052b30 msgr2=0x7f2efc080930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2efc052b30 0x7f2efc080930 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5644b6081f10 msgr2=0x5644b60843d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5644b6081f10 0x5644b60843d0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2de403bba0 msgr2=0x7f2de403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2de403bba0 0x7f2de403e060 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2efc000c00 tx=0x7f2efc060040 comp rx=0 tx=0).stop 2024-05-08T18:05:09.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60663d0 msgr2=0x5644b606d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60663d0 0x5644b606d820 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f2f04034fb0 tx=0x7f2f04034ee0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f14f12640 1 -- 172.21.15.50:0/549626291 reap_dead start 2024-05-08T18:05:09.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 shutdown_connections 2024-05-08T18:05:09.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 --2- 172.21.15.50:0/549626291 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60663d0 0x5644b606d820 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 >> 172.21.15.50:0/549626291 conn(0x5644b60626f0 msgr2=0x5644b605fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 shutdown_connections 2024-05-08T18:05:09.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 -- 172.21.15.50:0/549626291 wait complete. 2024-05-08T18:05:09.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 -- 172.21.15.50:0/536975330 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2dec03bb50 msgr2=0x7f2dec03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 --2- 172.21.15.50:0/536975330 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2dec03bb50 0x7f2dec03e010 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x5644b60424d0 tx=0x7f2efc005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:09.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 -- 172.21.15.50:0/536975330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60590d0 msgr2=0x5644b60594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.691+0000 7f2f16c19800 1 --2- 172.21.15.50:0/536975330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60590d0 0x5644b60594f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f2f04002250 tx=0x7f2f04012e20 comp rx=0 tx=0).stop 2024-05-08T18:05:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 -- 172.21.15.50:0/536975330 shutdown_connections 2024-05-08T18:05:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 --2- 172.21.15.50:0/536975330 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2dec03bb50 0x7f2dec03e010 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 --2- 172.21.15.50:0/536975330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b60590d0 0x5644b60594f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 -- 172.21.15.50:0/536975330 >> 172.21.15.50:0/536975330 conn(0x5644b604d170 msgr2=0x5644b604d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 -- 172.21.15.50:0/536975330 shutdown_connections 2024-05-08T18:05:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 -- 172.21.15.50:0/536975330 wait complete. 2024-05-08T18:05:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 -- 172.21.15.50:0/4237611277 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5644b6044330 msgr2=0x5644b6046720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5644b6044330 0x5644b6046720 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 -- 172.21.15.50:0/4237611277 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5644b5f3faf0 msgr2=0x5644b6041a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5644b5f3faf0 0x5644b6041a70 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 -- 172.21.15.50:0/4237611277 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5644b6048270 msgr2=0x5644b604a660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5644b6048270 0x5644b604a660 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 -- 172.21.15.50:0/4237611277 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2dfc03bba0 msgr2=0x7f2dfc03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2dfc03bba0 0x7f2dfc03e060 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2efc054000 tx=0x7f2efc04e000 comp rx=0 tx=0).stop 2024-05-08T18:05:09.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 -- 172.21.15.50:0/4237611277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3dfb0 msgr2=0x5644b603a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 --2- 172.21.15.50:0/4237611277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5644b5f3dfb0 0x5644b603a880 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f2f04000f80 tx=0x7f2f04001250 comp rx=0 tx=0).stop 2024-05-08T18:05:09.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f14f12640 1 -- 172.21.15.50:0/4237611277 reap_dead start 2024-05-08T18:05:09.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 -- 172.21.15.50:0/4237611277 shutdown_connections 2024-05-08T18:05:09.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 -- 172.21.15.50:0/4237611277 >> 172.21.15.50:0/4237611277 conn(0x5644b5f39af0 msgr2=0x5644b602ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 -- 172.21.15.50:0/4237611277 shutdown_connections 2024-05-08T18:05:09.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.692+0000 7f2f16c19800 1 -- 172.21.15.50:0/4237611277 wait complete. 2024-05-08T18:05:09.709 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2024-05-08T18:05:09.712 INFO:tasks.radosgw_admin:RL: get_obj myfoo foo bytes_out=0 bytes_in=4 failed=False 2024-05-08T18:05:09.763 INFO:tasks.radosgw_admin:RL: list_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:09.763 INFO:tasks.util.rgw:rgwadmin: client.0 : ['object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2024-05-08T18:05:09.763 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2024-05-08T18:05:09.763 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph object rm --bucket myfoo --object four 2024-05-08T18:05:09.812 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:09.812 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:09.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.814+0000 7fa135d2b800 1 Processor -- start 2024-05-08T18:05:09.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.814+0000 7fa135d2b800 1 -- start start 2024-05-08T18:05:09.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.814+0000 7fa135d2b800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x559513d25960 0x559513d25d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.814+0000 7fa135d2b800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559513d263b0 con 0x559513d28950 2024-05-08T18:05:09.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.814+0000 7fa135d2b800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x559513bc7e20 con 0x559513d25960 2024-05-08T18:05:09.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.814+0000 7fa131ae8640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x559513d25960 0x559513d25d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.814+0000 7fa1312e7640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x559513d28950 0x559513d28e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43270/0 (socket says 172.21.15.50:43270) 2024-05-08T18:05:09.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.814+0000 7fa1312e7640 1 -- 172.21.15.50:0/1868218006 learned_addr learned my addr 172.21.15.50:0/1868218006 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:09.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.815+0000 7fa131ae8640 1 -- 172.21.15.50:0/1868218006 >> v1:172.21.15.46:6789/0 conn(0x559513d28950 legacy=0x559513d28e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.815+0000 7fa130ae6640 1 -- 172.21.15.50:0/1868218006 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1937879926 0 0) 0x559513d263b0 con 0x559513d28950 2024-05-08T18:05:09.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.815+0000 7fa131ae8640 1 -- 172.21.15.50:0/1868218006 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559513d29510 con 0x559513d25960 2024-05-08T18:05:09.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.817+0000 7fa131ae8640 1 --2- 172.21.15.50:0/1868218006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 0x559513d25d80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa12400a470 tx=0x7fa12401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37170a86b74960e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.817+0000 7fa130ae6640 1 -- 172.21.15.50:0/1868218006 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa124057070 con 0x559513d25960 2024-05-08T18:05:09.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.817+0000 7fa130ae6640 1 -- 172.21.15.50:0/1868218006 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa12404adc0 con 0x559513d25960 2024-05-08T18:05:09.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.817+0000 7fa130ae6640 1 -- 172.21.15.50:0/1868218006 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa124052590 con 0x559513d25960 2024-05-08T18:05:09.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.817+0000 7fa135d2b800 1 -- 172.21.15.50:0/1868218006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 msgr2=0x559513d25d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.817 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.817+0000 7fa135d2b800 1 --2- 172.21.15.50:0/1868218006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 0x559513d25d80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa12400a470 tx=0x7fa12401ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:09.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.818+0000 7fa135d2b800 1 -- 172.21.15.50:0/1868218006 shutdown_connections 2024-05-08T18:05:09.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.818+0000 7fa135d2b800 1 --2- 172.21.15.50:0/1868218006 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 0x559513d25d80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.818+0000 7fa135d2b800 1 -- 172.21.15.50:0/1868218006 >> 172.21.15.50:0/1868218006 conn(0x559513d212f0 msgr2=0x559513d23710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.818+0000 7fa135d2b800 1 -- 172.21.15.50:0/1868218006 shutdown_connections 2024-05-08T18:05:09.818 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.818+0000 7fa135d2b800 1 -- 172.21.15.50:0/1868218006 wait complete. 2024-05-08T18:05:09.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.834+0000 7fa135d2b800 1 Processor -- start 2024-05-08T18:05:09.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.834+0000 7fa135d2b800 1 -- start start 2024-05-08T18:05:09.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.834+0000 7fa135d2b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 0x559513e18a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.834+0000 7fa135d2b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559513c72a30 con 0x559513d25960 2024-05-08T18:05:09.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.835+0000 7fa131ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 0x559513e18a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.835+0000 7fa131ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 0x559513e18a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38396/0 (socket says 172.21.15.50:38396) 2024-05-08T18:05:09.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.835+0000 7fa131ae8640 1 -- 172.21.15.50:0/187639728 learned_addr learned my addr 172.21.15.50:0/187639728 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.835+0000 7fa131ae8640 1 -- 172.21.15.50:0/187639728 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559513e19040 con 0x559513d25960 2024-05-08T18:05:09.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.835+0000 7fa131ae8640 1 --2- 172.21.15.50:0/187639728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 0x559513e18a70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa124055040 tx=0x7fa12404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf60e001e4abc64 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.835+0000 7fa01fe07640 1 -- 172.21.15.50:0/187639728 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa124057040 con 0x559513d25960 2024-05-08T18:05:09.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.835+0000 7fa01fe07640 1 -- 172.21.15.50:0/187639728 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa124062070 con 0x559513d25960 2024-05-08T18:05:09.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.835+0000 7fa01fe07640 1 -- 172.21.15.50:0/187639728 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa12405d470 con 0x559513d25960 2024-05-08T18:05:09.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.836+0000 7fa135d2b800 1 -- 172.21.15.50:0/187639728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 msgr2=0x559513e18a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.836+0000 7fa135d2b800 1 --2- 172.21.15.50:0/187639728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 0x559513e18a70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa124055040 tx=0x7fa12404c000 comp rx=0 tx=0).stop 2024-05-08T18:05:09.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.836+0000 7fa135d2b800 1 -- 172.21.15.50:0/187639728 shutdown_connections 2024-05-08T18:05:09.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.836+0000 7fa135d2b800 1 --2- 172.21.15.50:0/187639728 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 0x559513e18a70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.836+0000 7fa135d2b800 1 -- 172.21.15.50:0/187639728 >> 172.21.15.50:0/187639728 conn(0x559513d212f0 msgr2=0x559513e14390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.836+0000 7fa135d2b800 1 -- 172.21.15.50:0/187639728 shutdown_connections 2024-05-08T18:05:09.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.836+0000 7fa135d2b800 1 -- 172.21.15.50:0/187639728 wait complete. 2024-05-08T18:05:09.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.836+0000 7fa135d2b800 1 Processor -- start 2024-05-08T18:05:09.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.836+0000 7fa135d2b800 1 -- start start 2024-05-08T18:05:09.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.837+0000 7fa135d2b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 0x559513d281f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.837+0000 7fa135d2b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559513d30e20 con 0x559513d25960 2024-05-08T18:05:09.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.837+0000 7fa131ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 0x559513d281f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.837+0000 7fa131ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 0x559513d281f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38408/0 (socket says 172.21.15.50:38408) 2024-05-08T18:05:09.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.837+0000 7fa131ae8640 1 -- 172.21.15.50:0/1761730860 learned_addr learned my addr 172.21.15.50:0/1761730860 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.837+0000 7fa131ae8640 1 -- 172.21.15.50:0/1761730860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559513d28730 con 0x559513d25960 2024-05-08T18:05:09.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.837+0000 7fa131ae8640 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 0x559513d281f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa12401ce60 tx=0x7fa124002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.838+0000 7fa01e604640 1 -- 172.21.15.50:0/1761730860 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa12406b070 con 0x559513d25960 2024-05-08T18:05:09.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.838+0000 7fa01e604640 1 -- 172.21.15.50:0/1761730860 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa12405d970 con 0x559513d25960 2024-05-08T18:05:09.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.838+0000 7fa135d2b800 1 -- 172.21.15.50:0/1761730860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559513d26980 con 0x559513d25960 2024-05-08T18:05:09.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.838+0000 7fa01e604640 1 -- 172.21.15.50:0/1761730860 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa124057070 con 0x559513d25960 2024-05-08T18:05:09.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.838+0000 7fa135d2b800 1 -- 172.21.15.50:0/1761730860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559513d26ea0 con 0x559513d25960 2024-05-08T18:05:09.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.839+0000 7fa01e604640 1 -- 172.21.15.50:0/1761730860 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa12405dc90 con 0x559513d25960 2024-05-08T18:05:09.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.839+0000 7fa01e604640 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa01803bba0 0x7fa01803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.839+0000 7fa01e604640 1 -- 172.21.15.50:0/1761730860 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fa124094580 con 0x559513d25960 2024-05-08T18:05:09.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.839+0000 7fa1312e7640 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa01803bba0 0x7fa01803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.839+0000 7fa135d2b800 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559513d28950 0x559513e29950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.839+0000 7fa135d2b800 1 -- 172.21.15.50:0/1761730860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513e29e90 con 0x559513d28950 2024-05-08T18:05:09.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.840+0000 7fa1322e9640 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559513d28950 0x559513e29950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.840+0000 7fa1312e7640 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa01803bba0 0x7fa01803e060 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa128000c00 tx=0x7fa1280013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.840+0000 7fa1322e9640 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559513d28950 0x559513e29950 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.840+0000 7fa1322e9640 1 -- 172.21.15.50:0/1761730860 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513d28950 2024-05-08T18:05:09.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.840+0000 7fa135d2b800 1 -- 172.21.15.50:0/1761730860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513d273f0 con 0x559513d28950 2024-05-08T18:05:09.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.840+0000 7fa1322e9640 1 -- 172.21.15.50:0/1761730860 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513d28950 2024-05-08T18:05:09.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.840+0000 7fa135d2b800 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559513e2c1d0 0x559513e2e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.840+0000 7fa135d2b800 1 -- 172.21.15.50:0/1761730860 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513e2ecb0 con 0x559513e2c1d0 2024-05-08T18:05:09.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.841+0000 7fa131ae8640 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559513e2c1d0 0x559513e2e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.841+0000 7fa131ae8640 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559513e2c1d0 0x559513e2e5c0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.841+0000 7fa131ae8640 1 -- 172.21.15.50:0/1761730860 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fa12405a040 con 0x559513e2c1d0 2024-05-08T18:05:09.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.842+0000 7fa135d2b800 1 -- 172.21.15.50:0/1761730860 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513e2b070 con 0x559513d28950 2024-05-08T18:05:09.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.842+0000 7fa1322e9640 1 -- 172.21.15.50:0/1761730860 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513d28950 2024-05-08T18:05:09.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.842+0000 7fa135d2b800 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559513e30110 0x559513e32500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.842+0000 7fa135d2b800 1 -- 172.21.15.50:0/1761730860 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513e32c00 con 0x559513e30110 2024-05-08T18:05:09.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.842+0000 7fa1312e7640 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559513e30110 0x559513e32500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.842+0000 7fa1312e7640 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559513e30110 0x559513e32500 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.843+0000 7fa1312e7640 1 -- 172.21.15.50:0/1761730860 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fa12800f670 con 0x559513e30110 2024-05-08T18:05:09.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.843+0000 7fa135d2b800 1 Processor -- start 2024-05-08T18:05:09.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.843+0000 7fa135d2b800 1 -- start start 2024-05-08T18:05:09.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.843+0000 7fa135d2b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e45370 0x559513e45790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.843+0000 7fa135d2b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559513c19330 con 0x559513e45370 2024-05-08T18:05:09.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.844+0000 7fa131ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e45370 0x559513e45790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.844+0000 7fa131ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e45370 0x559513e45790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38422/0 (socket says 172.21.15.50:38422) 2024-05-08T18:05:09.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.844+0000 7fa131ae8640 1 -- 172.21.15.50:0/1625973994 learned_addr learned my addr 172.21.15.50:0/1625973994 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.844+0000 7fa131ae8640 1 -- 172.21.15.50:0/1625973994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559513e45cd0 con 0x559513e45370 2024-05-08T18:05:09.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.844+0000 7fa131ae8640 1 --2- 172.21.15.50:0/1625973994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e45370 0x559513e45790 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa124092910 tx=0x7fa124009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.844+0000 7fa0177fe640 1 -- 172.21.15.50:0/1625973994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa124090070 con 0x559513e45370 2024-05-08T18:05:09.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.844+0000 7fa0177fe640 1 -- 172.21.15.50:0/1625973994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa12406ed60 con 0x559513e45370 2024-05-08T18:05:09.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.844+0000 7fa135d2b800 1 -- 172.21.15.50:0/1625973994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559513e46740 con 0x559513e45370 2024-05-08T18:05:09.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.844+0000 7fa0177fe640 1 -- 172.21.15.50:0/1625973994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa12406e490 con 0x559513e45370 2024-05-08T18:05:09.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.844+0000 7fa135d2b800 1 -- 172.21.15.50:0/1625973994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559513e46b80 con 0x559513e45370 2024-05-08T18:05:09.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.845+0000 7fa0177fe640 1 -- 172.21.15.50:0/1625973994 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa12406e630 con 0x559513e45370 2024-05-08T18:05:09.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.845+0000 7fa0177fe640 1 --2- 172.21.15.50:0/1625973994 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa00c03bbf0 0x7fa00c03e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.845+0000 7fa0177fe640 1 -- 172.21.15.50:0/1625973994 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fa124088070 con 0x559513e45370 2024-05-08T18:05:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.845+0000 7fa1312e7640 1 --2- 172.21.15.50:0/1625973994 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa00c03bbf0 0x7fa00c03e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.846+0000 7fa135d2b800 1 Processor -- start 2024-05-08T18:05:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.846+0000 7fa135d2b800 1 -- start start 2024-05-08T18:05:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.846+0000 7fa1312e7640 1 --2- 172.21.15.50:0/1625973994 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa00c03bbf0 0x7fa00c03e0b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x559513e2a780 tx=0x7fa128006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.846+0000 7fa135d2b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e52490 0x559513e528b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.846+0000 7fa135d2b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559513e19300 con 0x559513e52490 2024-05-08T18:05:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.846+0000 7fa131ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e52490 0x559513e528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.846+0000 7fa131ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e52490 0x559513e528b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38434/0 (socket says 172.21.15.50:38434) 2024-05-08T18:05:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.846+0000 7fa131ae8640 1 -- 172.21.15.50:0/2899745159 learned_addr learned my addr 172.21.15.50:0/2899745159 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.847+0000 7fa131ae8640 1 -- 172.21.15.50:0/2899745159 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559513e52e80 con 0x559513e52490 2024-05-08T18:05:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.847+0000 7fa131ae8640 1 --2- 172.21.15.50:0/2899745159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e52490 0x559513e528b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa12408bb80 tx=0x7fa12408b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f565fa941ebc54b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.847+0000 7fa0157fa640 1 -- 172.21.15.50:0/2899745159 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1240a6070 con 0x559513e52490 2024-05-08T18:05:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.847+0000 7fa0157fa640 1 -- 172.21.15.50:0/2899745159 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa12408be20 con 0x559513e52490 2024-05-08T18:05:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.847+0000 7fa0157fa640 1 -- 172.21.15.50:0/2899745159 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa124082030 con 0x559513e52490 2024-05-08T18:05:09.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.847+0000 7fa135d2b800 1 -- 172.21.15.50:0/2899745159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e52490 msgr2=0x559513e528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.847+0000 7fa135d2b800 1 --2- 172.21.15.50:0/2899745159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e52490 0x559513e528b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa12408bb80 tx=0x7fa12408b6b0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.848+0000 7fa135d2b800 1 -- 172.21.15.50:0/2899745159 shutdown_connections 2024-05-08T18:05:09.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.848+0000 7fa135d2b800 1 --2- 172.21.15.50:0/2899745159 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e52490 0x559513e528b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.848+0000 7fa135d2b800 1 -- 172.21.15.50:0/2899745159 >> 172.21.15.50:0/2899745159 conn(0x559513e4e760 msgr2=0x559513e4c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.848+0000 7fa135d2b800 1 -- 172.21.15.50:0/2899745159 shutdown_connections 2024-05-08T18:05:09.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.848+0000 7fa135d2b800 1 -- 172.21.15.50:0/2899745159 wait complete. 2024-05-08T18:05:09.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.848+0000 7fa135d2b800 1 Processor -- start 2024-05-08T18:05:09.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.848+0000 7fa135d2b800 1 -- start start 2024-05-08T18:05:09.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.848+0000 7fa135d2b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e52490 0x559513e597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.848+0000 7fa135d2b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559513e534e0 con 0x559513e52490 2024-05-08T18:05:09.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.848+0000 7fa131ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e52490 0x559513e597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.848+0000 7fa131ae8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e52490 0x559513e597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38440/0 (socket says 172.21.15.50:38440) 2024-05-08T18:05:09.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.848+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 learned_addr learned my addr 172.21.15.50:0/76161776 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:09.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.849+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559513e59ce0 con 0x559513e52490 2024-05-08T18:05:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.849+0000 7fa131ae8640 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e52490 0x559513e597a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa12408bdb0 tx=0x7fa124073f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.849+0000 7fa0037fe640 1 -- 172.21.15.50:0/76161776 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1240922c0 con 0x559513e52490 2024-05-08T18:05:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.849+0000 7fa0037fe640 1 -- 172.21.15.50:0/76161776 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa124092460 con 0x559513e52490 2024-05-08T18:05:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.849+0000 7fa0037fe640 1 -- 172.21.15.50:0/76161776 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa124081830 con 0x559513e52490 2024-05-08T18:05:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.849+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559513e59ce0 con 0x559513e52490 2024-05-08T18:05:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.849+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559513e59ce0 con 0x559513e52490 2024-05-08T18:05:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.850+0000 7fa0037fe640 1 -- 172.21.15.50:0/76161776 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa1240bc030 con 0x559513e52490 2024-05-08T18:05:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.850+0000 7fa0037fe640 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9ffc03bba0 0x7f9ffc03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.850+0000 7fa0037fe640 1 -- 172.21.15.50:0/76161776 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fa1240ba070 con 0x559513e52490 2024-05-08T18:05:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.850+0000 7fa1312e7640 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9ffc03bba0 0x7f9ffc03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.851+0000 7fa1312e7640 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9ffc03bba0 0x7f9ffc03e060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x559513e33000 tx=0x7fa128023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.851+0000 7fa135d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.851+0000 7fa135d2b800 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559513e64cc0 0x559513e67180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.851+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513e678d0 con 0x559513e64cc0 2024-05-08T18:05:09.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.852+0000 7fa1322e9640 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559513e64cc0 0x559513e67180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.852+0000 7fa1322e9640 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559513e64cc0 0x559513e67180 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.852+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513e64cc0 2024-05-08T18:05:09.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.852+0000 7fa135d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.852+0000 7fa135d2b800 20 realm 2024-05-08T18:05:09.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.852+0000 7fa135d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.852+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513e67ce0 con 0x559513e64cc0 2024-05-08T18:05:09.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.852+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513e64cc0 2024-05-08T18:05:09.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.852+0000 7fa135d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.852+0000 7fa135d2b800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:09.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.852+0000 7fa135d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.852+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513e685b0 con 0x559513e64cc0 2024-05-08T18:05:09.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.853+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513e64cc0 2024-05-08T18:05:09.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.853+0000 7fa135d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.853+0000 7fa135d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.853+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e64cc0 2024-05-08T18:05:09.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.853+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513e64cc0 2024-05-08T18:05:09.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.853+0000 7fa135d2b800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:09.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.853+0000 7fa135d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.853+0000 7fa135d2b800 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559513e696b0 0x559513e6baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.853+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513e6c0b0 con 0x559513e696b0 2024-05-08T18:05:09.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.853+0000 7fa131ae8640 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559513e696b0 0x559513e6baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.854+0000 7fa131ae8640 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559513e696b0 0x559513e6baa0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.854+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.854+0000 7fa135d2b800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:09.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.854+0000 7fa135d2b800 20 searching for the correct realm 2024-05-08T18:05:09.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.854+0000 7fa135d2b800 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559513e6e060 0x559513e70520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.854+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e70bf0 con 0x559513e6e060 2024-05-08T18:05:09.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.854+0000 7fa1312e7640 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559513e6e060 0x559513e70520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.854+0000 7fa1312e7640 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559513e6e060 0x559513e70520 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.854+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.854+0000 7fa1312e7640 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa12802ea80 0x7fa12803a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.855+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e70bf0 con 0x7fa12802ea80 2024-05-08T18:05:09.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.855+0000 7fa1322e9640 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa12802ea80 0x7fa12803a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.855+0000 7fa1322e9640 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa12802ea80 0x7fa12803a3c0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.855+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa12c00a4a0 con 0x7fa12802ea80 2024-05-08T18:05:09.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.855+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e696b0 2024-05-08T18:05:09.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.856+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.860 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.856+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e64cc0 2024-05-08T18:05:09.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.856+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513e64cc0 2024-05-08T18:05:09.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.856+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e64cc0 2024-05-08T18:05:09.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.856+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513e64cc0 2024-05-08T18:05:09.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.856+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e696b0 2024-05-08T18:05:09.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.857+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.857+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e696b0 2024-05-08T18:05:09.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.857+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.857+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x7fa12802ea80 2024-05-08T18:05:09.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.857+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa12c00a4a0 con 0x7fa12802ea80 2024-05-08T18:05:09.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.857+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e6e060 2024-05-08T18:05:09.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.857+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.857+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e64cc0 2024-05-08T18:05:09.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.858+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513e64cc0 2024-05-08T18:05:09.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.858+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e696b0 2024-05-08T18:05:09.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.858+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.858+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e64cc0 2024-05-08T18:05:09.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.858+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513e64cc0 2024-05-08T18:05:09.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.858+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x7fa12802ea80 2024-05-08T18:05:09.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.858+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fa12c00a4a0 con 0x7fa12802ea80 2024-05-08T18:05:09.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.858+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e6e060 2024-05-08T18:05:09.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.859+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.859+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e6e060 2024-05-08T18:05:09.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.859+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.859+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e696b0 2024-05-08T18:05:09.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.859+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.859+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e6e060 2024-05-08T18:05:09.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.859+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.859+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e6e060 2024-05-08T18:05:09.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.859+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.860+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e64cc0 2024-05-08T18:05:09.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.860+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513e64cc0 2024-05-08T18:05:09.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.860+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e64cc0 2024-05-08T18:05:09.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.860+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513e64cc0 2024-05-08T18:05:09.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.860+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e696b0 2024-05-08T18:05:09.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.860+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.860+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e6e060 2024-05-08T18:05:09.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.861+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.861+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x7fa12802ea80 2024-05-08T18:05:09.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.861+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa12c00a4a0 con 0x7fa12802ea80 2024-05-08T18:05:09.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.861+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e64cc0 2024-05-08T18:05:09.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.861+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513e64cc0 2024-05-08T18:05:09.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.861+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e696b0 2024-05-08T18:05:09.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.861+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.861+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e6e060 2024-05-08T18:05:09.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.862+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.862+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e696b0 2024-05-08T18:05:09.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.862+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.862+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x7fa12802ea80 2024-05-08T18:05:09.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.862+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa12c00a4a0 con 0x7fa12802ea80 2024-05-08T18:05:09.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.862+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e6e060 2024-05-08T18:05:09.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.862+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.862+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e696b0 2024-05-08T18:05:09.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x559513e64cc0 2024-05-08T18:05:09.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513e64cc0 2024-05-08T18:05:09.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559513e68950 con 0x7fa12802ea80 2024-05-08T18:05:09.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa12c00a4a0 con 0x7fa12802ea80 2024-05-08T18:05:09.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa135d2b800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:09.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa135d2b800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:09.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa135d2b800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:09.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa135d2b800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:09.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa135d2b800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:09.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa135d2b800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:09.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa135d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513e6c970 con 0x559513e64cc0 2024-05-08T18:05:09.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513e64cc0 2024-05-08T18:05:09.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa135d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa135d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.863+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513e6c970 con 0x559513e64cc0 2024-05-08T18:05:09.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa12c00a4a0 con 0x559513e64cc0 2024-05-08T18:05:09.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:09.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513e6c970 con 0x559513e6e060 2024-05-08T18:05:09.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:09.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 20 zone default found 2024-05-08T18:05:09.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 4 Realm: () 2024-05-08T18:05:09.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:09.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:09.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:09.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 20 zonegroup default 2024-05-08T18:05:09.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513e6d2a0 con 0x559513e696b0 2024-05-08T18:05:09.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513e72ab0 con 0x559513e6e060 2024-05-08T18:05:09.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.864+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513e72fa0 con 0x559513e6e060 2024-05-08T18:05:09.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.865+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.865+0000 7fa135d2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:09.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.865+0000 7fa135d2b800 20 started sync module instance, tier type = 2024-05-08T18:05:09.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.865+0000 7fa135d2b800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:09.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.865+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e71ef0 con 0x7fa12802ea80 2024-05-08T18:05:09.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.866+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa12c00a4a0 con 0x7fa12802ea80 2024-05-08T18:05:09.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.866+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94098772403216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e71ef0 con 0x7fa12802ea80 2024-05-08T18:05:09.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.866+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e73a60 con 0x559513e64cc0 2024-05-08T18:05:09.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.867+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94098772403216] v27'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x7fa12802ea80 2024-05-08T18:05:09.878 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.867+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.867+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94098772407664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e71ef0 con 0x559513e64cc0 2024-05-08T18:05:09.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.867+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e744f0 con 0x559513e6e060 2024-05-08T18:05:09.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.868+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94098772407664] v27'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.868+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.868+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94098772400704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e71e60 con 0x559513e6e060 2024-05-08T18:05:09.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.868+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e753b0 con 0x7fa12802ea80 2024-05-08T18:05:09.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.869+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94098772400704] v27'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.870+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x7fa12802ea80 2024-05-08T18:05:09.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.870+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94098772414208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e75320 con 0x7fa12802ea80 2024-05-08T18:05:09.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.870+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e76140 con 0x559513e64cc0 2024-05-08T18:05:09.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.871+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94098772414208] v27'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x7fa12802ea80 2024-05-08T18:05:09.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.871+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.871+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94098772416272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e76140 con 0x559513e64cc0 2024-05-08T18:05:09.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.871+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e76140 con 0x559513e696b0 2024-05-08T18:05:09.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.872+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94098772416272] v27'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.872+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.872+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94098772417328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e76e30 con 0x559513e696b0 2024-05-08T18:05:09.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.872+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e778f0 con 0x559513e696b0 2024-05-08T18:05:09.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.873+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94098772417328] v27'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.873+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.873+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94098772423616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e769c0 con 0x559513e696b0 2024-05-08T18:05:09.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.873+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e786d0 con 0x559513e696b0 2024-05-08T18:05:09.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.874+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94098772423616] v27'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.874+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.874+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94098772427168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e78af0 con 0x559513e696b0 2024-05-08T18:05:09.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.874+0000 7fa135d2b800 20 add_watcher() i=0 2024-05-08T18:05:09.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.874+0000 7fa135d2b800 20 add_watcher() i=1 2024-05-08T18:05:09.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.874+0000 7fa135d2b800 20 add_watcher() i=2 2024-05-08T18:05:09.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.874+0000 7fa135d2b800 20 add_watcher() i=3 2024-05-08T18:05:09.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.874+0000 7fa135d2b800 20 add_watcher() i=4 2024-05-08T18:05:09.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.874+0000 7fa135d2b800 20 add_watcher() i=5 2024-05-08T18:05:09.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.874+0000 7fa135d2b800 20 add_watcher() i=6 2024-05-08T18:05:09.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.875+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94098772427168] v27'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.875+0000 7fa135d2b800 20 add_watcher() i=7 2024-05-08T18:05:09.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.875+0000 7fa135d2b800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:09.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.875+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513e7bef0 con 0x559513e64cc0 2024-05-08T18:05:09.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.875+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.875+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94098772425424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e793b0 con 0x559513e64cc0 2024-05-08T18:05:09.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.876+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94098772425424] v27'139 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.876+0000 7f9fe37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:09.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.876+0000 7fa135d2b800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:09.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.876+0000 7fa135d2b800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:09.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.876+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e7bbe0 con 0x559513e64cc0 2024-05-08T18:05:09.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.877+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'23 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.877+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e84f40 con 0x559513e696b0 2024-05-08T18:05:09.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.878+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.878+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e84f40 con 0x559513e6e060 2024-05-08T18:05:09.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.879+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.879+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e859a0 con 0x559513e64cc0 2024-05-08T18:05:09.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.880+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'140 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.880+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e845b0 con 0x7fa12802ea80 2024-05-08T18:05:09.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.881+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'41 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x7fa12802ea80 2024-05-08T18:05:09.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.881+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e845b0 con 0x559513e6e060 2024-05-08T18:05:09.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.882+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'27 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.882+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e866d0 con 0x559513e64cc0 2024-05-08T18:05:09.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.883+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.883+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e86b80 con 0x559513e696b0 2024-05-08T18:05:09.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.884+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'27 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.884+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e86b80 con 0x7fa12802ea80 2024-05-08T18:05:09.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.885+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x7fa12802ea80 2024-05-08T18:05:09.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.885+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e86b80 con 0x559513e6e060 2024-05-08T18:05:09.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.886+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'23 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.886+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e870f0 con 0x7fa12802ea80 2024-05-08T18:05:09.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.887+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'42 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x7fa12802ea80 2024-05-08T18:05:09.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.887+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e870f0 con 0x559513e696b0 2024-05-08T18:05:09.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.889+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'19 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.889+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e870f0 con 0x7fa12802ea80 2024-05-08T18:05:09.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.890+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x7fa12802ea80 2024-05-08T18:05:09.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.890+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e870f0 con 0x559513e696b0 2024-05-08T18:05:09.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.891+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'33 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.891+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e870f0 con 0x559513e64cc0 2024-05-08T18:05:09.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.892+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'141 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.892+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e870f0 con 0x559513e6e060 2024-05-08T18:05:09.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.893+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'36 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.893+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e83630 con 0x7fa12802ea80 2024-05-08T18:05:09.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.894+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'43 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x7fa12802ea80 2024-05-08T18:05:09.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.894+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e83630 con 0x559513e6e060 2024-05-08T18:05:09.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.895+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'17 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.895+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e83630 con 0x559513e64cc0 2024-05-08T18:05:09.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.896+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'25 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.896+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e82f20 con 0x559513e64cc0 2024-05-08T18:05:09.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.896+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'142 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.896+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e83630 con 0x559513e6e060 2024-05-08T18:05:09.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.897+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'37 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.897+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e82f20 con 0x7fa12802ea80 2024-05-08T18:05:09.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.898+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'27 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x7fa12802ea80 2024-05-08T18:05:09.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.898+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e82f20 con 0x559513e6e060 2024-05-08T18:05:09.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.899+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.899+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e82f20 con 0x559513e6e060 2024-05-08T18:05:09.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.900+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'23 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.900+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e82f20 con 0x559513e6e060 2024-05-08T18:05:09.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.901+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.901+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e82f20 con 0x559513e6e060 2024-05-08T18:05:09.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.902+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'28 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.902+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e82f20 con 0x559513e696b0 2024-05-08T18:05:09.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.904+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.904+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e876e0 con 0x559513e6e060 2024-05-08T18:05:09.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.904+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'17 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.904+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e876e0 con 0x559513e696b0 2024-05-08T18:05:09.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.906+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'28 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.906+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e876e0 con 0x559513e6e060 2024-05-08T18:05:09.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.907+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'38 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.907+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e876e0 con 0x559513e696b0 2024-05-08T18:05:09.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.908+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'34 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.908+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e876e0 con 0x7fa12802ea80 2024-05-08T18:05:09.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.909+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'23 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x7fa12802ea80 2024-05-08T18:05:09.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.909+0000 7f9fe27fc640 20 reqs_thread_entry: start 2024-05-08T18:05:09.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.909+0000 7fa135d2b800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:09.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.909+0000 7fa135d2b800 20 Filter name: none 2024-05-08T18:05:09.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.909+0000 7f9f53fff640 20 reqs_thread_entry: start 2024-05-08T18:05:09.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.909+0000 7fa135d2b800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-05-08T18:05:09.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.909+0000 7fa135d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.909+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:94 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513eb2460 con 0x7fa12802ea80 2024-05-08T18:05:09.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(94 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv29 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fa12c040040 con 0x7fa12802ea80 2024-05-08T18:05:09.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa135d2b800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:09.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa135d2b800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-05-08T18:05:09.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa135d2b800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:09.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa135d2b800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 : miss 2024-05-08T18:05:09.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa135d2b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:09.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:95 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513eb3ef0 con 0x559513e696b0 2024-05-08T18:05:09.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(95 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv5 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa135d2b800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:09.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa135d2b800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:09.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa135d2b800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:09.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa135d2b800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:09.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa135d2b800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:09.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa135d2b800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 2024-05-08T18:05:09.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa135d2b800 20 get_obj_state: octx=0x7fa124070ae0 obj=myfoo:four state=0x559513eb51d8 s->prefetch_data=0 2024-05-08T18:05:09.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:96 3.31s0 3:8fc50a0f:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1_four:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559513eb6000 con 0x559513e696b0 2024-05-08T18:05:09.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.910+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(96 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1_four [getxattrs out=849b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 234+0+865 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.911+0000 7fa135d2b800 10 manifest: total_size = 4 2024-05-08T18:05:09.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.911+0000 7fa135d2b800 20 get_obj_state: setting s->obj_tag to 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.4075766027446092483 2024-05-08T18:05:09.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.911+0000 7fa135d2b800 20 get_obj_state: octx=0x7fa124070ae0 obj=myfoo:four state=0x559513eb51d8 s->prefetch_data=0 2024-05-08T18:05:09.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.911+0000 7fa135d2b800 20 prepare_atomic_modification: state is not atomic. state=0x559513eb51d8 2024-05-08T18:05:09.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.911+0000 7fa135d2b800 20 bucket index object: :.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.6 2024-05-08T18:05:09.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.911+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:97 4.3 4:c17a4869:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=182b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513eba910 con 0x559513e64cc0 2024-05-08T18:05:09.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.912+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(97 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.6 [stat,call,call] v27'4 uv4 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.912+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:98 3.31s0 3:8fc50a0f:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1_four:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513eba910 con 0x559513e696b0 2024-05-08T18:05:09.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.913+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(98 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1_four [call] v27'2 uv2 ondisk = 0) ==== 192+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.913+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:99 4.3 4:c17a4869:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=259b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513ebb3a0 con 0x559513e64cc0 2024-05-08T18:05:09.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.914+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(99 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.6 [stat,call,call] v27'5 uv5 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.914+0000 7f9fe37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:09.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.914+0000 7fa0157fa640 20 handle_completion(): completion ok for obj=four 2024-05-08T18:05:09.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.915+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94098772425424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513ebb3a0 con 0x559513e64cc0 2024-05-08T18:05:09.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.915+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(100 data_loggenerations_metadata [watch unwatch cookie 94098772425424] v27'143 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.915+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:101 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94098772403216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513ebb3a0 con 0x7fa12802ea80 2024-05-08T18:05:09.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.916+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(101 notify.0 [watch unwatch cookie 94098772403216] v27'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x7fa12802ea80 2024-05-08T18:05:09.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.916+0000 7fa135d2b800 20 remove_watcher() i=0 2024-05-08T18:05:09.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.916+0000 7fa135d2b800 2 removed watcher, disabling cache 2024-05-08T18:05:09.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.916+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:102 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94098772407664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513ebb3a0 con 0x559513e64cc0 2024-05-08T18:05:09.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.918+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(102 notify.1 [watch unwatch cookie 94098772407664] v27'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.918+0000 7fa135d2b800 20 remove_watcher() i=1 2024-05-08T18:05:09.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.918+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:103 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94098772400704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513ebb3a0 con 0x559513e6e060 2024-05-08T18:05:09.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.919+0000 7fa1312e7640 1 -- 172.21.15.50:0/76161776 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(103 notify.2 [watch unwatch cookie 94098772400704] v27'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa128037040 con 0x559513e6e060 2024-05-08T18:05:09.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.919+0000 7fa135d2b800 20 remove_watcher() i=2 2024-05-08T18:05:09.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.919+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:104 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94098772414208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513ebb3a0 con 0x7fa12802ea80 2024-05-08T18:05:09.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.920+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.2 v2:172.21.15.50:6811/3039646617 20 ==== osd_op_reply(104 notify.3 [watch unwatch cookie 94098772414208] v27'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x7fa12802ea80 2024-05-08T18:05:09.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.920+0000 7fa135d2b800 20 remove_watcher() i=3 2024-05-08T18:05:09.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.920+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:105 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94098772416272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513ebb3a0 con 0x559513e64cc0 2024-05-08T18:05:09.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.921+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(105 notify.4 [watch unwatch cookie 94098772416272] v27'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa12c040040 con 0x559513e64cc0 2024-05-08T18:05:09.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.921+0000 7fa135d2b800 20 remove_watcher() i=4 2024-05-08T18:05:09.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.921+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:106 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94098772417328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513ebb3a0 con 0x559513e696b0 2024-05-08T18:05:09.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.922+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(106 notify.5 [watch unwatch cookie 94098772417328] v27'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.922+0000 7fa135d2b800 20 remove_watcher() i=5 2024-05-08T18:05:09.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.922+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:107 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94098772423616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e75bc0 con 0x559513e696b0 2024-05-08T18:05:09.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.923+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(107 notify.6 [watch unwatch cookie 94098772423616] v27'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.923+0000 7fa135d2b800 20 remove_watcher() i=6 2024-05-08T18:05:09.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.923+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:108 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94098772427168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559513e73570 con 0x559513e696b0 2024-05-08T18:05:09.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa131ae8640 1 -- 172.21.15.50:0/76161776 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== osd_op_reply(108 notify.7 [watch unwatch cookie 94098772427168] v27'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1240a8030 con 0x559513e696b0 2024-05-08T18:05:09.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 20 remove_watcher() i=7 2024-05-08T18:05:09.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559513e696b0 msgr2=0x559513e6baa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559513e696b0 0x559513e6baa0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559513e64cc0 msgr2=0x559513e67180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559513e64cc0 0x559513e67180 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa12802ea80 msgr2=0x7fa12803a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa12802ea80 0x7fa12803a3c0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559513e6e060 msgr2=0x559513e70520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559513e6e060 0x559513e70520 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9ffc03bba0 msgr2=0x7f9ffc03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9ffc03bba0 0x7f9ffc03e060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x559513e33000 tx=0x7fa128023040 comp rx=0 tx=0).stop 2024-05-08T18:05:09.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa1322e9640 1 -- 172.21.15.50:0/76161776 reap_dead start 2024-05-08T18:05:09.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e52490 msgr2=0x559513e597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e52490 0x559513e597a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa12408bdb0 tx=0x7fa124073f60 comp rx=0 tx=0).stop 2024-05-08T18:05:09.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 shutdown_connections 2024-05-08T18:05:09.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 --2- 172.21.15.50:0/76161776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e52490 0x559513e597a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 >> 172.21.15.50:0/76161776 conn(0x559513e4e760 msgr2=0x559513e4bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 shutdown_connections 2024-05-08T18:05:09.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.924+0000 7fa135d2b800 1 -- 172.21.15.50:0/76161776 wait complete. 2024-05-08T18:05:09.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 -- 172.21.15.50:0/1625973994 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa00c03bbf0 msgr2=0x7fa00c03e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 --2- 172.21.15.50:0/1625973994 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa00c03bbf0 0x7fa00c03e0b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x559513e2a780 tx=0x7fa128006000 comp rx=0 tx=0).stop 2024-05-08T18:05:09.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 -- 172.21.15.50:0/1625973994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e45370 msgr2=0x559513e45790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 --2- 172.21.15.50:0/1625973994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e45370 0x559513e45790 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa124092910 tx=0x7fa124009640 comp rx=0 tx=0).stop 2024-05-08T18:05:09.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 -- 172.21.15.50:0/1625973994 shutdown_connections 2024-05-08T18:05:09.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 --2- 172.21.15.50:0/1625973994 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa00c03bbf0 0x7fa00c03e0b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 --2- 172.21.15.50:0/1625973994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513e45370 0x559513e45790 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 -- 172.21.15.50:0/1625973994 >> 172.21.15.50:0/1625973994 conn(0x559513e20210 msgr2=0x559513e20610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 -- 172.21.15.50:0/1625973994 shutdown_connections 2024-05-08T18:05:09.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 -- 172.21.15.50:0/1625973994 wait complete. 2024-05-08T18:05:09.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 -- 172.21.15.50:0/1761730860 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559513e2c1d0 msgr2=0x559513e2e5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x559513e2c1d0 0x559513e2e5c0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 -- 172.21.15.50:0/1761730860 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559513d28950 msgr2=0x559513e29950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x559513d28950 0x559513e29950 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 -- 172.21.15.50:0/1761730860 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559513e30110 msgr2=0x559513e32500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559513e30110 0x559513e32500 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 -- 172.21.15.50:0/1761730860 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa01803bba0 msgr2=0x7fa01803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa01803bba0 0x7fa01803e060 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa128000c00 tx=0x7fa1280013b0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 -- 172.21.15.50:0/1761730860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 msgr2=0x559513d281f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 --2- 172.21.15.50:0/1761730860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559513d25960 0x559513d281f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa12401ce60 tx=0x7fa124002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa1322e9640 1 -- 172.21.15.50:0/1761730860 reap_dead start 2024-05-08T18:05:09.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 -- 172.21.15.50:0/1761730860 shutdown_connections 2024-05-08T18:05:09.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 -- 172.21.15.50:0/1761730860 >> 172.21.15.50:0/1761730860 conn(0x559513d212f0 msgr2=0x559513e12b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 -- 172.21.15.50:0/1761730860 shutdown_connections 2024-05-08T18:05:09.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.925+0000 7fa135d2b800 1 -- 172.21.15.50:0/1761730860 wait complete. 2024-05-08T18:05:09.934 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-05-08T18:05:09.934 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-05-08T18:05:09.934 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-05-08T18:05:09.979 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:09.979 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:09.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.981+0000 7f9b8ad21800 1 Processor -- start 2024-05-08T18:05:09.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.981+0000 7f9b8ad21800 1 -- start start 2024-05-08T18:05:09.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.981+0000 7f9b8ad21800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56273ad2af70 0x56273ad2b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.981+0000 7f9b8ad21800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56273ac9f8c0 con 0x56273ad2b8d0 2024-05-08T18:05:09.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.981+0000 7f9b8ad21800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x56273abcce20 con 0x56273ad2af70 2024-05-08T18:05:09.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.981+0000 7f9b86a52640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56273ad2af70 0x56273ad2b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.981+0000 7f9b86251640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x56273ad2b8d0 0x56273ad2bd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43274/0 (socket says 172.21.15.50:43274) 2024-05-08T18:05:09.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.981+0000 7f9b86251640 1 -- 172.21.15.50:0/1083056376 learned_addr learned my addr 172.21.15.50:0/1083056376 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:09.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.982+0000 7f9b86a52640 1 -- 172.21.15.50:0/1083056376 >> v1:172.21.15.46:6789/0 conn(0x56273ad2b8d0 legacy=0x56273ad2bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.982+0000 7f9b86a52640 1 -- 172.21.15.50:0/1083056376 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56273ad2c490 con 0x56273ad2af70 2024-05-08T18:05:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.984+0000 7f9b86a52640 1 --2- 172.21.15.50:0/1083056376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 0x56273ad2b390 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800a470 tx=0x7f9b7801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4fb54c38d35bf7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.984+0000 7f9b85a50640 1 -- 172.21.15.50:0/1083056376 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b78057070 con 0x56273ad2af70 2024-05-08T18:05:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.984+0000 7f9b85a50640 1 -- 172.21.15.50:0/1083056376 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b78052570 con 0x56273ad2af70 2024-05-08T18:05:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.984+0000 7f9b85a50640 1 -- 172.21.15.50:0/1083056376 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b78052870 con 0x56273ad2af70 2024-05-08T18:05:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.984+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1083056376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 msgr2=0x56273ad2b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.984+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/1083056376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 0x56273ad2b390 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800a470 tx=0x7f9b7801ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.984+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1083056376 shutdown_connections 2024-05-08T18:05:09.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.984+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/1083056376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 0x56273ad2b390 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.984+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1083056376 >> 172.21.15.50:0/1083056376 conn(0x56273ad26b50 msgr2=0x56273ad28f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.984+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1083056376 shutdown_connections 2024-05-08T18:05:09.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.984+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1083056376 wait complete. 2024-05-08T18:05:09.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.999+0000 7f9b8ad21800 1 Processor -- start 2024-05-08T18:05:09.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:09.999+0000 7f9b8ad21800 1 -- start start 2024-05-08T18:05:09.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.000+0000 7f9b8ad21800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 0x56273ae1ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:09.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.000+0000 7f9b8ad21800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56273ac77a10 con 0x56273ad2af70 2024-05-08T18:05:09.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.000+0000 7f9b86a52640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 0x56273ae1ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:09.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.000+0000 7f9b86a52640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 0x56273ae1ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38452/0 (socket says 172.21.15.50:38452) 2024-05-08T18:05:09.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.000+0000 7f9b86a52640 1 -- 172.21.15.50:0/2549620869 learned_addr learned my addr 172.21.15.50:0/2549620869 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.000+0000 7f9b86a52640 1 -- 172.21.15.50:0/2549620869 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56273ae20530 con 0x56273ad2af70 2024-05-08T18:05:10.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.000+0000 7f9b86a52640 1 --2- 172.21.15.50:0/2549620869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 0x56273ae1ff60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9b78055040 tx=0x7f9b7804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cab924c95934c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.001+0000 7f9a74e09640 1 -- 172.21.15.50:0/2549620869 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b78057070 con 0x56273ad2af70 2024-05-08T18:05:10.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.001+0000 7f9a74e09640 1 -- 172.21.15.50:0/2549620869 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b78063070 con 0x56273ad2af70 2024-05-08T18:05:10.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.001+0000 7f9a74e09640 1 -- 172.21.15.50:0/2549620869 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b78052720 con 0x56273ad2af70 2024-05-08T18:05:10.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.001+0000 7f9b8ad21800 1 -- 172.21.15.50:0/2549620869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 msgr2=0x56273ae1ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.001+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/2549620869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 0x56273ae1ff60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9b78055040 tx=0x7f9b7804c000 comp rx=0 tx=0).stop 2024-05-08T18:05:10.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.001+0000 7f9b8ad21800 1 -- 172.21.15.50:0/2549620869 shutdown_connections 2024-05-08T18:05:10.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.001+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/2549620869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 0x56273ae1ff60 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.001+0000 7f9b8ad21800 1 -- 172.21.15.50:0/2549620869 >> 172.21.15.50:0/2549620869 conn(0x56273ad26b50 msgr2=0x56273ae1b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.001+0000 7f9b8ad21800 1 -- 172.21.15.50:0/2549620869 shutdown_connections 2024-05-08T18:05:10.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.001+0000 7f9b8ad21800 1 -- 172.21.15.50:0/2549620869 wait complete. 2024-05-08T18:05:10.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.002+0000 7f9b8ad21800 1 Processor -- start 2024-05-08T18:05:10.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.002+0000 7f9b8ad21800 1 -- start start 2024-05-08T18:05:10.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.002+0000 7f9b8ad21800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 0x56273ae27a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.002+0000 7f9b8ad21800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56273ad2c9c0 con 0x56273ad2af70 2024-05-08T18:05:10.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.002+0000 7f9b86a52640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 0x56273ae27a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.002+0000 7f9b86a52640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 0x56273ae27a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38466/0 (socket says 172.21.15.50:38466) 2024-05-08T18:05:10.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.002+0000 7f9b86a52640 1 -- 172.21.15.50:0/1325966140 learned_addr learned my addr 172.21.15.50:0/1325966140 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.002+0000 7f9b86a52640 1 -- 172.21.15.50:0/1325966140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56273ae27f50 con 0x56273ad2af70 2024-05-08T18:05:10.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.003+0000 7f9b86a52640 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 0x56273ae27a10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9b78060040 tx=0x7f9b78002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.003+0000 7f9a73606640 1 -- 172.21.15.50:0/1325966140 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b78052e30 con 0x56273ad2af70 2024-05-08T18:05:10.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.003+0000 7f9a73606640 1 -- 172.21.15.50:0/1325966140 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7806c070 con 0x56273ad2af70 2024-05-08T18:05:10.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.003+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1325966140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56273ae281e0 con 0x56273ad2af70 2024-05-08T18:05:10.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.003+0000 7f9a73606640 1 -- 172.21.15.50:0/1325966140 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b7805e5f0 con 0x56273ad2af70 2024-05-08T18:05:10.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.003+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1325966140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56273ae2ae90 con 0x56273ad2af70 2024-05-08T18:05:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.004+0000 7f9a73606640 1 -- 172.21.15.50:0/1325966140 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9b7805e790 con 0x56273ad2af70 2024-05-08T18:05:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.004+0000 7f9a73606640 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9a6c03bb50 0x7f9a6c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.004+0000 7f9a73606640 1 -- 172.21.15.50:0/1325966140 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f9b78094d60 con 0x56273ad2af70 2024-05-08T18:05:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.004+0000 7f9b86251640 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9a6c03bb50 0x7f9a6c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.004+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56273ad2b8d0 0x56273ae2ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.004+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1325966140 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae2f190 con 0x56273ad2b8d0 2024-05-08T18:05:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.004+0000 7f9b87253640 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56273ad2b8d0 0x56273ae2ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.004+0000 7f9b86251640 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9a6c03bb50 0x7f9a6c03e010 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c001d70 tx=0x7f9b7c0012d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.005+0000 7f9b87253640 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56273ad2b8d0 0x56273ae2ec50 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.005+0000 7f9b87253640 1 -- 172.21.15.50:0/1325966140 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ad2b8d0 2024-05-08T18:05:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.005+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1325966140 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae2fa70 con 0x56273ad2b8d0 2024-05-08T18:05:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.005+0000 7f9b87253640 1 -- 172.21.15.50:0/1325966140 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ad2b8d0 2024-05-08T18:05:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.005+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56273ae31380 0x56273ae33820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.005+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1325966140 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae33fb0 con 0x56273ae31380 2024-05-08T18:05:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.005+0000 7f9b86a52640 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56273ae31380 0x56273ae33820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.006+0000 7f9b86a52640 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56273ae31380 0x56273ae33820 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.006+0000 7f9b86a52640 1 -- 172.21.15.50:0/1325966140 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f9b78009b60 con 0x56273ae31380 2024-05-08T18:05:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.006+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1325966140 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae302c0 con 0x56273ad2b8d0 2024-05-08T18:05:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.006+0000 7f9b87253640 1 -- 172.21.15.50:0/1325966140 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ad2b8d0 2024-05-08T18:05:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.006+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56273ae35410 0x56273ae37800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.006+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1325966140 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae37f40 con 0x56273ae35410 2024-05-08T18:05:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.006+0000 7f9b86251640 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56273ae35410 0x56273ae37800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.007+0000 7f9b86251640 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56273ae35410 0x56273ae37800 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.007+0000 7f9b86251640 1 -- 172.21.15.50:0/1325966140 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae35410 2024-05-08T18:05:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.007+0000 7f9b8ad21800 1 Processor -- start 2024-05-08T18:05:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.007+0000 7f9b8ad21800 1 -- start start 2024-05-08T18:05:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.007+0000 7f9b8ad21800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae46270 0x56273ae46690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.007+0000 7f9b8ad21800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56273ac1e330 con 0x56273ae46270 2024-05-08T18:05:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.008+0000 7f9b86a52640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae46270 0x56273ae46690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.008+0000 7f9b86a52640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae46270 0x56273ae46690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38472/0 (socket says 172.21.15.50:38472) 2024-05-08T18:05:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.008+0000 7f9b86a52640 1 -- 172.21.15.50:0/3985579381 learned_addr learned my addr 172.21.15.50:0/3985579381 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.008+0000 7f9b86a52640 1 -- 172.21.15.50:0/3985579381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56273ae46bd0 con 0x56273ae46270 2024-05-08T18:05:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.008+0000 7f9b86a52640 1 --2- 172.21.15.50:0/3985579381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae46270 0x56273ae46690 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9b78093210 tx=0x7f9b7805ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.008+0000 7f9a70e01640 1 -- 172.21.15.50:0/3985579381 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b7808a070 con 0x56273ae46270 2024-05-08T18:05:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.008+0000 7f9a70e01640 1 -- 172.21.15.50:0/3985579381 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b78058040 con 0x56273ae46270 2024-05-08T18:05:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.008+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3985579381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56273ae47640 con 0x56273ae46270 2024-05-08T18:05:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.008+0000 7f9a70e01640 1 -- 172.21.15.50:0/3985579381 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b78009830 con 0x56273ae46270 2024-05-08T18:05:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.008+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3985579381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56273ae47b40 con 0x56273ae46270 2024-05-08T18:05:10.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.009+0000 7f9a70e01640 1 -- 172.21.15.50:0/3985579381 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9b78099020 con 0x56273ae46270 2024-05-08T18:05:10.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.009+0000 7f9a70e01640 1 --2- 172.21.15.50:0/3985579381 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9a6003bba0 0x7f9a6003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.009+0000 7f9a70e01640 1 -- 172.21.15.50:0/3985579381 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f9b7807b040 con 0x56273ae46270 2024-05-08T18:05:10.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.009+0000 7f9b86251640 1 --2- 172.21.15.50:0/3985579381 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9a6003bba0 0x7f9a6003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.010+0000 7f9b8ad21800 1 Processor -- start 2024-05-08T18:05:10.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.010+0000 7f9b8ad21800 1 -- start start 2024-05-08T18:05:10.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.010+0000 7f9b86251640 1 --2- 172.21.15.50:0/3985579381 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9a6003bba0 0x7f9a6003e060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00bf40 tx=0x7f9b7c00f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.010+0000 7f9b8ad21800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae53440 0x56273ae53860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.010+0000 7f9b8ad21800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56273ae207f0 con 0x56273ae53440 2024-05-08T18:05:10.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.010+0000 7f9b86a52640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae53440 0x56273ae53860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.010+0000 7f9b86a52640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae53440 0x56273ae53860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38482/0 (socket says 172.21.15.50:38482) 2024-05-08T18:05:10.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.010+0000 7f9b86a52640 1 -- 172.21.15.50:0/2249639713 learned_addr learned my addr 172.21.15.50:0/2249639713 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.010+0000 7f9b86a52640 1 -- 172.21.15.50:0/2249639713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56273ae53e30 con 0x56273ae53440 2024-05-08T18:05:10.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.011+0000 7f9b86a52640 1 --2- 172.21.15.50:0/2249639713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae53440 0x56273ae53860 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9b7805efd0 tx=0x7f9b7805b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e818c28642b68cf1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.011+0000 7f9a6a7fc640 1 -- 172.21.15.50:0/2249639713 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b7805b4a0 con 0x56273ae53440 2024-05-08T18:05:10.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.011+0000 7f9a6a7fc640 1 -- 172.21.15.50:0/2249639713 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b780aa070 con 0x56273ae53440 2024-05-08T18:05:10.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.011+0000 7f9a6a7fc640 1 -- 172.21.15.50:0/2249639713 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b7809be40 con 0x56273ae53440 2024-05-08T18:05:10.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.011+0000 7f9b8ad21800 1 -- 172.21.15.50:0/2249639713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae53440 msgr2=0x56273ae53860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.011+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/2249639713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae53440 0x56273ae53860 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f9b7805efd0 tx=0x7f9b7805b7c0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.011+0000 7f9b8ad21800 1 -- 172.21.15.50:0/2249639713 shutdown_connections 2024-05-08T18:05:10.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.011+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/2249639713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae53440 0x56273ae53860 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.011+0000 7f9b8ad21800 1 -- 172.21.15.50:0/2249639713 >> 172.21.15.50:0/2249639713 conn(0x56273ae4f710 msgr2=0x56273ae4d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.011+0000 7f9b8ad21800 1 -- 172.21.15.50:0/2249639713 shutdown_connections 2024-05-08T18:05:10.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.011+0000 7f9b8ad21800 1 -- 172.21.15.50:0/2249639713 wait complete. 2024-05-08T18:05:10.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.012+0000 7f9b8ad21800 1 Processor -- start 2024-05-08T18:05:10.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.012+0000 7f9b8ad21800 1 -- start start 2024-05-08T18:05:10.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.012+0000 7f9b8ad21800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae53440 0x56273ae5a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.012+0000 7f9b8ad21800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56273ae54490 con 0x56273ae53440 2024-05-08T18:05:10.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.012+0000 7f9b86a52640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae53440 0x56273ae5a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.012+0000 7f9b86a52640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae53440 0x56273ae5a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38492/0 (socket says 172.21.15.50:38492) 2024-05-08T18:05:10.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.012+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 learned_addr learned my addr 172.21.15.50:0/3549568071 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.012+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56273ae5ac00 con 0x56273ae53440 2024-05-08T18:05:10.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.012+0000 7f9b86a52640 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae53440 0x56273ae5a6c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9b78060ff0 tx=0x7f9b78060f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.013+0000 7f9a68ff9640 1 -- 172.21.15.50:0/3549568071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b780bd070 con 0x56273ae53440 2024-05-08T18:05:10.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.013+0000 7f9a68ff9640 1 -- 172.21.15.50:0/3549568071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7805b4a0 con 0x56273ae53440 2024-05-08T18:05:10.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.013+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56273ae5ac00 con 0x56273ae53440 2024-05-08T18:05:10.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.013+0000 7f9a68ff9640 1 -- 172.21.15.50:0/3549568071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b78085540 con 0x56273ae53440 2024-05-08T18:05:10.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.013+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56273ae5ac00 con 0x56273ae53440 2024-05-08T18:05:10.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.013+0000 7f9a68ff9640 1 -- 172.21.15.50:0/3549568071 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9b780c0070 con 0x56273ae53440 2024-05-08T18:05:10.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.014+0000 7f9a68ff9640 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9a5003bba0 0x7f9a5003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.014+0000 7f9a68ff9640 1 -- 172.21.15.50:0/3549568071 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f9b780c3ca0 con 0x56273ae53440 2024-05-08T18:05:10.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.014+0000 7f9b86251640 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9a5003bba0 0x7f9a5003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.014+0000 7f9b8ad21800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.014+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56273ae65cb0 0x56273ae68170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.014+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae688c0 con 0x56273ae65cb0 2024-05-08T18:05:10.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.014+0000 7f9b87253640 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56273ae65cb0 0x56273ae68170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.014+0000 7f9b86251640 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9a5003bba0 0x7f9a5003e060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00c010 tx=0x7f9b7c010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.014+0000 7f9b87253640 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56273ae65cb0 0x56273ae68170 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ae65cb0 2024-05-08T18:05:10.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b8ad21800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b8ad21800 20 realm 2024-05-08T18:05:10.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b8ad21800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae68cd0 con 0x56273ae65cb0 2024-05-08T18:05:10.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ae65cb0 2024-05-08T18:05:10.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b8ad21800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b8ad21800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:10.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b8ad21800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae695a0 con 0x56273ae65cb0 2024-05-08T18:05:10.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ae65cb0 2024-05-08T18:05:10.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b8ad21800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b8ad21800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae65cb0 2024-05-08T18:05:10.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ae65cb0 2024-05-08T18:05:10.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b8ad21800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:10.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b8ad21800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56273ae6a430 0x56273ae6c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.015+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae6cf60 con 0x56273ae6a430 2024-05-08T18:05:10.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.016+0000 7f9b86a52640 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56273ae6a430 0x56273ae6c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.016+0000 7f9b86a52640 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56273ae6a430 0x56273ae6c820 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.016+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.016+0000 7f9b8ad21800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:10.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.016+0000 7f9b8ad21800 20 searching for the correct realm 2024-05-08T18:05:10.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.016+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56273ae6f000 0x56273ae714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.016+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae71b90 con 0x56273ae6f000 2024-05-08T18:05:10.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.016+0000 7f9b86251640 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56273ae6f000 0x56273ae714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.017+0000 7f9b86251640 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56273ae6f000 0x56273ae714c0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.017+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.017+0000 7f9b86251640 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9b7c037040 0x7f9b7c036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.017+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae71b90 con 0x7f9b7c037040 2024-05-08T18:05:10.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.017+0000 7f9b87253640 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9b7c037040 0x7f9b7c036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.017+0000 7f9b87253640 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9b7c037040 0x7f9b7c036410 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.018+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b8000a4a0 con 0x7f9b7c037040 2024-05-08T18:05:10.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.018+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6a430 2024-05-08T18:05:10.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.018+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.018+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae65cb0 2024-05-08T18:05:10.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.018+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ae65cb0 2024-05-08T18:05:10.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.018+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae65cb0 2024-05-08T18:05:10.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.018+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ae65cb0 2024-05-08T18:05:10.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.018+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6a430 2024-05-08T18:05:10.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.019+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.019+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6a430 2024-05-08T18:05:10.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.019+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.019+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x7f9b7c037040 2024-05-08T18:05:10.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.019+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b8000a4a0 con 0x7f9b7c037040 2024-05-08T18:05:10.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.019+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6f000 2024-05-08T18:05:10.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.019+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.019+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae65cb0 2024-05-08T18:05:10.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.020+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ae65cb0 2024-05-08T18:05:10.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.020+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6a430 2024-05-08T18:05:10.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.020+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.020+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae65cb0 2024-05-08T18:05:10.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.020+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ae65cb0 2024-05-08T18:05:10.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.020+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x7f9b7c037040 2024-05-08T18:05:10.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.020+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9b8000a4a0 con 0x7f9b7c037040 2024-05-08T18:05:10.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.020+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6f000 2024-05-08T18:05:10.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.021+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.021+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6f000 2024-05-08T18:05:10.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.021+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.021+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6a430 2024-05-08T18:05:10.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.021+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.021+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6f000 2024-05-08T18:05:10.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.021+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.021+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6f000 2024-05-08T18:05:10.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.022+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.022+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae65cb0 2024-05-08T18:05:10.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.022+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ae65cb0 2024-05-08T18:05:10.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.022+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae65cb0 2024-05-08T18:05:10.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.022+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ae65cb0 2024-05-08T18:05:10.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.022+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6a430 2024-05-08T18:05:10.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.022+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.022+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6f000 2024-05-08T18:05:10.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.023+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.023+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x7f9b7c037040 2024-05-08T18:05:10.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.023+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b8000a4a0 con 0x7f9b7c037040 2024-05-08T18:05:10.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.023+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae65cb0 2024-05-08T18:05:10.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.023+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ae65cb0 2024-05-08T18:05:10.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.023+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6a430 2024-05-08T18:05:10.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.024+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.024+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6f000 2024-05-08T18:05:10.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.024+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.024+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6a430 2024-05-08T18:05:10.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.024+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.024+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x7f9b7c037040 2024-05-08T18:05:10.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.024+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b8000a4a0 con 0x7f9b7c037040 2024-05-08T18:05:10.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.024+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6f000 2024-05-08T18:05:10.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.025+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.025+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae6a430 2024-05-08T18:05:10.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.025+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.025+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x56273ae65cb0 2024-05-08T18:05:10.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.025+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ae65cb0 2024-05-08T18:05:10.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.025+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56273ae69940 con 0x7f9b7c037040 2024-05-08T18:05:10.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.025+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b8000a4a0 con 0x7f9b7c037040 2024-05-08T18:05:10.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.025+0000 7f9b8ad21800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:10.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.025+0000 7f9b8ad21800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:10.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.025+0000 7f9b8ad21800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:10.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.025+0000 7f9b8ad21800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:10.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.025+0000 7f9b8ad21800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:10.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.025+0000 7f9b8ad21800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:10.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.025+0000 7f9b8ad21800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.025+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae6d900 con 0x56273ae65cb0 2024-05-08T18:05:10.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ae65cb0 2024-05-08T18:05:10.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b8ad21800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b8ad21800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae6d900 con 0x56273ae65cb0 2024-05-08T18:05:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9b8000a4a0 con 0x56273ae65cb0 2024-05-08T18:05:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b8ad21800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b8ad21800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae6d900 con 0x56273ae6f000 2024-05-08T18:05:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b8ad21800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b8ad21800 20 zone default found 2024-05-08T18:05:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b8ad21800 4 Realm: () 2024-05-08T18:05:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b8ad21800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b8ad21800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b8ad21800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b8ad21800 20 zonegroup default 2024-05-08T18:05:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b8ad21800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.026+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae6e1a0 con 0x56273ae6a430 2024-05-08T18:05:10.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.027+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.027+0000 7f9b8ad21800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.027+0000 7f9b8ad21800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.027+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae739f0 con 0x56273ae6f000 2024-05-08T18:05:10.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.027+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.027+0000 7f9b8ad21800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.027+0000 7f9b8ad21800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.027+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae73d90 con 0x56273ae6f000 2024-05-08T18:05:10.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.028+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.028+0000 7f9b8ad21800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.028+0000 7f9b8ad21800 20 started sync module instance, tier type = 2024-05-08T18:05:10.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.028+0000 7f9b8ad21800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:10.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.028+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae72e90 con 0x7f9b7c037040 2024-05-08T18:05:10.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.029+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b8000a4a0 con 0x7f9b7c037040 2024-05-08T18:05:10.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.029+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94726491943824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae72e90 con 0x7f9b7c037040 2024-05-08T18:05:10.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.029+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae74800 con 0x56273ae65cb0 2024-05-08T18:05:10.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.030+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94726491943824] v27'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b80040040 con 0x7f9b7c037040 2024-05-08T18:05:10.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.030+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b80040040 con 0x56273ae65cb0 2024-05-08T18:05:10.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.030+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94726491947760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae72e90 con 0x56273ae65cb0 2024-05-08T18:05:10.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.030+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae752b0 con 0x56273ae6f000 2024-05-08T18:05:10.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.031+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94726491947760] v27'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b80040040 con 0x56273ae65cb0 2024-05-08T18:05:10.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.031+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.031+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94726491941344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae72e00 con 0x56273ae6f000 2024-05-08T18:05:10.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.031+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae76170 con 0x7f9b7c037040 2024-05-08T18:05:10.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.032+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94726491941344] v27'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.032+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b80040040 con 0x7f9b7c037040 2024-05-08T18:05:10.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.032+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94726491954368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae760e0 con 0x7f9b7c037040 2024-05-08T18:05:10.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.032+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae76f30 con 0x56273ae65cb0 2024-05-08T18:05:10.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.033+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b80040040 con 0x56273ae65cb0 2024-05-08T18:05:10.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.033+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94726491954368] v27'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b80040040 con 0x7f9b7c037040 2024-05-08T18:05:10.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.033+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94726491957776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae76f30 con 0x56273ae65cb0 2024-05-08T18:05:10.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.034+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae765a0 con 0x56273ae6a430 2024-05-08T18:05:10.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.034+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.034+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94726491957776] v27'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b80040040 con 0x56273ae65cb0 2024-05-08T18:05:10.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.034+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94726491961664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae77d60 con 0x56273ae6a430 2024-05-08T18:05:10.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.035+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae78720 con 0x56273ae6a430 2024-05-08T18:05:10.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.035+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.035+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94726491964016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae78b80 con 0x56273ae6a430 2024-05-08T18:05:10.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.035+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae79660 con 0x56273ae6a430 2024-05-08T18:05:10.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.036+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94726491961664] v27'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.036+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94726491964016] v27'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.036+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.036+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94726491960096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae79170 con 0x56273ae6a430 2024-05-08T18:05:10.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.037+0000 7f9b8ad21800 20 add_watcher() i=0 2024-05-08T18:05:10.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.037+0000 7f9b8ad21800 20 add_watcher() i=1 2024-05-08T18:05:10.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.037+0000 7f9b8ad21800 20 add_watcher() i=2 2024-05-08T18:05:10.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.037+0000 7f9b8ad21800 20 add_watcher() i=3 2024-05-08T18:05:10.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.037+0000 7f9b8ad21800 20 add_watcher() i=4 2024-05-08T18:05:10.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.037+0000 7f9b8ad21800 20 add_watcher() i=5 2024-05-08T18:05:10.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.037+0000 7f9b8ad21800 20 add_watcher() i=6 2024-05-08T18:05:10.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.037+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94726491960096] v27'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.037+0000 7f9b8ad21800 20 add_watcher() i=7 2024-05-08T18:05:10.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.037+0000 7f9b8ad21800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:10.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.037+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273ae7c570 con 0x56273ae65cb0 2024-05-08T18:05:10.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.038+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9b80040040 con 0x56273ae65cb0 2024-05-08T18:05:10.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.038+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94726491965072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae79ac0 con 0x56273ae65cb0 2024-05-08T18:05:10.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.039+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94726491965072] v27'144 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9b80040040 con 0x56273ae65cb0 2024-05-08T18:05:10.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.039+0000 7f9a3d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:10.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.039+0000 7f9b8ad21800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:10.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.039+0000 7f9b8ad21800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:10.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.039+0000 7f9b8ad21800 5 note: GC not initialized 2024-05-08T18:05:10.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.039+0000 7f99b7fff640 20 reqs_thread_entry: start 2024-05-08T18:05:10.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.039+0000 7f9b8ad21800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:10.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.039+0000 7f9b8ad21800 20 Filter name: none 2024-05-08T18:05:10.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.040+0000 7f99b5ffb640 20 reqs_thread_entry: start 2024-05-08T18:05:10.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.040+0000 7f9b8ad21800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.040+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273aeafb00 con 0x7f9b7c037040 2024-05-08T18:05:10.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.040+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv29 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f9b80040040 con 0x7f9b7c037040 2024-05-08T18:05:10.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.040+0000 7f9b8ad21800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:10.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.040+0000 7f9b8ad21800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.040+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273aeb0390 con 0x56273ae6a430 2024-05-08T18:05:10.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.040+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv5 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.040+0000 7f9b8ad21800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:10.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.040+0000 7f9b8ad21800 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-08T18:05:10.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.040+0000 7f9b8ad21800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.040+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273aeb0540 con 0x7f9b7c037040 2024-05-08T18:05:10.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.041+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv29 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f9b80040040 con 0x7f9b7c037040 2024-05-08T18:05:10.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.041+0000 7f9b8ad21800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:10.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.041+0000 7f9b8ad21800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.041+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:65 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273aeaeea0 con 0x56273ae6a430 2024-05-08T18:05:10.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.041+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(65 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv5 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.041+0000 7f9b8ad21800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:10.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.041+0000 7f9b8ad21800 20 couldn't put binfo cache entry, might have raced with data changes 2024-05-08T18:05:10.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.041+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:66 4.37 4:eecadb7d:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273aeb2e60 con 0x56273ae6a430 2024-05-08T18:05:10.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.041+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 4.22 4:46cfe895:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273aeb5750 con 0x56273ae65cb0 2024-05-08T18:05:10.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.041+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 4.0 4:018ce329:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273aeb82f0 con 0x56273ae6f000 2024-05-08T18:05:10.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.041+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 4.12 4:4afe9c7f:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273aebaca0 con 0x56273ae65cb0 2024-05-08T18:05:10.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.041+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:70 4.d 4:b2c4ac36:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273aebd5d0 con 0x56273ae6f000 2024-05-08T18:05:10.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.041+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 4.33 4:cc412025:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273aec02e0 con 0x56273ae6a430 2024-05-08T18:05:10.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.041+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 4.3 4:c17a4869:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273aec2e70 con 0x56273ae65cb0 2024-05-08T18:05:10.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.041+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:73 4.6 4:61ff1ea9:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273aec5940 con 0x56273ae6f000 2024-05-08T18:05:10.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.041+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 4.1c 4:3ae7d9ef:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273aec87a0 con 0x56273ae6a430 2024-05-08T18:05:10.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.042+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 4.f 4:f2293886:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273aecb300 con 0x56273ae65cb0 2024-05-08T18:05:10.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.042+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 4.21 4:87ec338a:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56273aece090 con 0x56273ae6a430 2024-05-08T18:05:10.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.042+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(66 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.042+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(67 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9b80040040 con 0x56273ae65cb0 2024-05-08T18:05:10.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.042+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(68 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.042+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(70 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.042+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(73 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.042+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(69 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9b80040040 con 0x56273ae65cb0 2024-05-08T18:05:10.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.042+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(74 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.8 [call out=124b] v0'0 uv7 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.042+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(71 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.5 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.042+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(72 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.6 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f9b80040040 con 0x56273ae65cb0 2024-05-08T18:05:10.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.042+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(76 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.042+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(75 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9b80040040 con 0x56273ae65cb0 2024-05-08T18:05:10.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.043+0000 7f9a3d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:10.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.043+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94726491965072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273aec87a0 con 0x56273ae65cb0 2024-05-08T18:05:10.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.044+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94726491965072] v27'145 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9b80040040 con 0x56273ae65cb0 2024-05-08T18:05:10.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.044+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94726491943824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae98060 con 0x7f9b7c037040 2024-05-08T18:05:10.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.045+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94726491943824] v27'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b80040040 con 0x7f9b7c037040 2024-05-08T18:05:10.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.045+0000 7f9b8ad21800 20 remove_watcher() i=0 2024-05-08T18:05:10.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.045+0000 7f9b8ad21800 2 removed watcher, disabling cache 2024-05-08T18:05:10.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.045+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94726491947760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae98060 con 0x56273ae65cb0 2024-05-08T18:05:10.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.046+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94726491947760] v27'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b80040040 con 0x56273ae65cb0 2024-05-08T18:05:10.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.046+0000 7f9b8ad21800 20 remove_watcher() i=1 2024-05-08T18:05:10.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.046+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94726491941344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae98060 con 0x56273ae6f000 2024-05-08T18:05:10.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.047+0000 7f9b86251640 1 -- 172.21.15.50:0/3549568071 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94726491941344] v27'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b7c009050 con 0x56273ae6f000 2024-05-08T18:05:10.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.047+0000 7f9b8ad21800 20 remove_watcher() i=2 2024-05-08T18:05:10.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.047+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94726491954368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae98060 con 0x7f9b7c037040 2024-05-08T18:05:10.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.048+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94726491954368] v27'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b80040040 con 0x7f9b7c037040 2024-05-08T18:05:10.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.048+0000 7f9b8ad21800 20 remove_watcher() i=3 2024-05-08T18:05:10.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.048+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94726491957776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae98060 con 0x56273ae65cb0 2024-05-08T18:05:10.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.049+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94726491957776] v27'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b80040040 con 0x56273ae65cb0 2024-05-08T18:05:10.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.049+0000 7f9b8ad21800 20 remove_watcher() i=4 2024-05-08T18:05:10.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.049+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94726491961664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae98060 con 0x56273ae6a430 2024-05-08T18:05:10.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.050+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94726491961664] v27'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.050+0000 7f9b8ad21800 20 remove_watcher() i=5 2024-05-08T18:05:10.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.050+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94726491964016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae98060 con 0x56273ae6a430 2024-05-08T18:05:10.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.051+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94726491964016] v27'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.051+0000 7f9b8ad21800 20 remove_watcher() i=6 2024-05-08T18:05:10.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.051+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94726491960096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56273ae98060 con 0x56273ae6a430 2024-05-08T18:05:10.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.052+0000 7f9b86a52640 1 -- 172.21.15.50:0/3549568071 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94726491960096] v27'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b780bf030 con 0x56273ae6a430 2024-05-08T18:05:10.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.052+0000 7f9b8ad21800 20 remove_watcher() i=7 2024-05-08T18:05:10.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.052+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56273ae6a430 msgr2=0x56273ae6c820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56273ae6a430 0x56273ae6c820 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56273ae65cb0 msgr2=0x56273ae68170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.062 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56273ae65cb0 0x56273ae68170 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9b7c037040 msgr2=0x7f9b7c036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9b7c037040 0x7f9b7c036410 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56273ae6f000 msgr2=0x56273ae714c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56273ae6f000 0x56273ae714c0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9a5003bba0 msgr2=0x7f9a5003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9a5003bba0 0x7f9a5003e060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00c010 tx=0x7f9b7c010000 comp rx=0 tx=0).stop 2024-05-08T18:05:10.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b87253640 1 -- 172.21.15.50:0/3549568071 reap_dead start 2024-05-08T18:05:10.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae53440 msgr2=0x56273ae5a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae53440 0x56273ae5a6c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9b78060ff0 tx=0x7f9b78060f40 comp rx=0 tx=0).stop 2024-05-08T18:05:10.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 shutdown_connections 2024-05-08T18:05:10.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/3549568071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae53440 0x56273ae5a6c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 >> 172.21.15.50:0/3549568071 conn(0x56273ae4f710 msgr2=0x56273ae4cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 shutdown_connections 2024-05-08T18:05:10.063 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3549568071 wait complete. 2024-05-08T18:05:10.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3985579381 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9a6003bba0 msgr2=0x7f9a6003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/3985579381 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9a6003bba0 0x7f9a6003e060 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00bf40 tx=0x7f9b7c00f000 comp rx=0 tx=0).stop 2024-05-08T18:05:10.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3985579381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae46270 msgr2=0x56273ae46690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/3985579381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae46270 0x56273ae46690 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9b78093210 tx=0x7f9b7805ed20 comp rx=0 tx=0).stop 2024-05-08T18:05:10.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3985579381 shutdown_connections 2024-05-08T18:05:10.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/3985579381 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9a6003bba0 0x7f9a6003e060 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/3985579381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ae46270 0x56273ae46690 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3985579381 >> 172.21.15.50:0/3985579381 conn(0x56273ae3a310 msgr2=0x56273ae3a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3985579381 shutdown_connections 2024-05-08T18:05:10.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.053+0000 7f9b8ad21800 1 -- 172.21.15.50:0/3985579381 wait complete. 2024-05-08T18:05:10.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.054+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1325966140 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56273ae31380 msgr2=0x56273ae33820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.054+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56273ae31380 0x56273ae33820 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.054+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1325966140 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56273ad2b8d0 msgr2=0x56273ae2ec50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.054+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56273ad2b8d0 0x56273ae2ec50 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.054+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1325966140 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56273ae35410 msgr2=0x56273ae37800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.054+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56273ae35410 0x56273ae37800 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.054+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1325966140 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9a6c03bb50 msgr2=0x7f9a6c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.054+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9a6c03bb50 0x7f9a6c03e010 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c001d70 tx=0x7f9b7c0012d0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.054+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1325966140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 msgr2=0x56273ae27a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.054+0000 7f9b8ad21800 1 --2- 172.21.15.50:0/1325966140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56273ad2af70 0x56273ae27a10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9b78060040 tx=0x7f9b78002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.054+0000 7f9b87253640 1 -- 172.21.15.50:0/1325966140 reap_dead start 2024-05-08T18:05:10.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.054+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1325966140 shutdown_connections 2024-05-08T18:05:10.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.054+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1325966140 >> 172.21.15.50:0/1325966140 conn(0x56273ad26b50 msgr2=0x56273ae1a080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.054+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1325966140 shutdown_connections 2024-05-08T18:05:10.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.054+0000 7f9b8ad21800 1 -- 172.21.15.50:0/1325966140 wait complete. 2024-05-08T18:05:10.066 INFO:teuthology.orchestra.run.smithi050.stdout:{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"36133b23-abaa-446a-bf67-ceb1d1559ceb","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1","marker":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1","index_type":"Normal","index_generation":0,"num_shards":11,"object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-05-08T18:05:09.500805Z","creation_time":"2024-05-08T18:05:06.706377Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{"rgw.main":{"size":0,"size_actual":0,"size_utilized":0,"size_kb":0,"size_kb_actual":0,"size_kb_utilized":0,"num_objects":0}},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}} 2024-05-08T18:05:10.066 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': '36133b23-abaa-446a-bf67-ceb1d1559ceb', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1', 'marker': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-05-08T18:05:09.500805Z', 'creation_time': '2024-05-08T18:05:06.706377Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-05-08T18:05:10.067 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'list'] 2024-05-08T18:05:10.067 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'list'] 2024-05-08T18:05:10.067 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log list 2024-05-08T18:05:10.110 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:10.111 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:10.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.112+0000 7f7df4485800 1 Processor -- start 2024-05-08T18:05:10.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.112+0000 7f7df4485800 1 -- start start 2024-05-08T18:05:10.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.112+0000 7f7df4485800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55637b2d9fa0 0x55637b2da3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.112+0000 7f7df4485800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55637b24f120 con 0x55637b2da900 2024-05-08T18:05:10.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.112+0000 7f7df4485800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55637b17be20 con 0x55637b2d9fa0 2024-05-08T18:05:10.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.113+0000 7f7df1f7d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55637b2d9fa0 0x55637b2da3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.113+0000 7f7df177c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55637b2da900 0x55637b2dadb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43286/0 (socket says 172.21.15.50:43286) 2024-05-08T18:05:10.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.113+0000 7f7df177c640 1 -- 172.21.15.50:0/1201489363 learned_addr learned my addr 172.21.15.50:0/1201489363 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:10.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.113+0000 7f7df1f7d640 1 -- 172.21.15.50:0/1201489363 >> v1:172.21.15.46:6789/0 conn(0x55637b2da900 legacy=0x55637b2dadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.113+0000 7f7df1f7d640 1 -- 172.21.15.50:0/1201489363 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637b2db4c0 con 0x55637b2d9fa0 2024-05-08T18:05:10.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.115+0000 7f7df1f7d640 1 --2- 172.21.15.50:0/1201489363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 0x55637b2da3c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7de400a470 tx=0x7f7de401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1274c5d31725c60 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.115+0000 7f7df0f7b640 1 -- 172.21.15.50:0/1201489363 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de4057070 con 0x55637b2d9fa0 2024-05-08T18:05:10.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.115+0000 7f7df0f7b640 1 -- 172.21.15.50:0/1201489363 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de4052570 con 0x55637b2d9fa0 2024-05-08T18:05:10.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.115+0000 7f7df0f7b640 1 -- 172.21.15.50:0/1201489363 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de4052870 con 0x55637b2d9fa0 2024-05-08T18:05:10.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.115+0000 7f7df4485800 1 -- 172.21.15.50:0/1201489363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 msgr2=0x55637b2da3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.115+0000 7f7df4485800 1 --2- 172.21.15.50:0/1201489363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 0x55637b2da3c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7de400a470 tx=0x7f7de401ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:10.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.115+0000 7f7df4485800 1 -- 172.21.15.50:0/1201489363 shutdown_connections 2024-05-08T18:05:10.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.115+0000 7f7df4485800 1 --2- 172.21.15.50:0/1201489363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 0x55637b2da3c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.115+0000 7f7df4485800 1 -- 172.21.15.50:0/1201489363 >> 172.21.15.50:0/1201489363 conn(0x55637b2d5ac0 msgr2=0x55637b2d7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.115+0000 7f7df4485800 1 -- 172.21.15.50:0/1201489363 shutdown_connections 2024-05-08T18:05:10.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.115+0000 7f7df4485800 1 -- 172.21.15.50:0/1201489363 wait complete. 2024-05-08T18:05:10.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.131+0000 7f7df4485800 1 Processor -- start 2024-05-08T18:05:10.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.131+0000 7f7df4485800 1 -- start start 2024-05-08T18:05:10.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.131+0000 7f7df4485800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 0x55637b3cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.131+0000 7f7df4485800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55637b226a00 con 0x55637b2d9fa0 2024-05-08T18:05:10.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.131+0000 7f7df1f7d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 0x55637b3cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.131+0000 7f7df1f7d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 0x55637b3cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38498/0 (socket says 172.21.15.50:38498) 2024-05-08T18:05:10.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.131+0000 7f7df1f7d640 1 -- 172.21.15.50:0/4244940092 learned_addr learned my addr 172.21.15.50:0/4244940092 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.132+0000 7f7df1f7d640 1 -- 172.21.15.50:0/4244940092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637b3cf540 con 0x55637b2d9fa0 2024-05-08T18:05:10.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.132+0000 7f7df1f7d640 1 --2- 172.21.15.50:0/4244940092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 0x55637b3cef70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7de4055040 tx=0x7f7de404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a74d7fc284ca2e86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.132+0000 7f7cdfe07640 1 -- 172.21.15.50:0/4244940092 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de4057070 con 0x55637b2d9fa0 2024-05-08T18:05:10.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.132+0000 7f7cdfe07640 1 -- 172.21.15.50:0/4244940092 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de4063070 con 0x55637b2d9fa0 2024-05-08T18:05:10.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.132+0000 7f7cdfe07640 1 -- 172.21.15.50:0/4244940092 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de4052720 con 0x55637b2d9fa0 2024-05-08T18:05:10.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.133+0000 7f7df4485800 1 -- 172.21.15.50:0/4244940092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 msgr2=0x55637b3cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.133+0000 7f7df4485800 1 --2- 172.21.15.50:0/4244940092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 0x55637b3cef70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7de4055040 tx=0x7f7de404c000 comp rx=0 tx=0).stop 2024-05-08T18:05:10.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.133+0000 7f7df4485800 1 -- 172.21.15.50:0/4244940092 shutdown_connections 2024-05-08T18:05:10.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.133+0000 7f7df4485800 1 --2- 172.21.15.50:0/4244940092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 0x55637b3cef70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.133+0000 7f7df4485800 1 -- 172.21.15.50:0/4244940092 >> 172.21.15.50:0/4244940092 conn(0x55637b2d5ac0 msgr2=0x55637b3ca890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.133+0000 7f7df4485800 1 -- 172.21.15.50:0/4244940092 shutdown_connections 2024-05-08T18:05:10.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.133+0000 7f7df4485800 1 -- 172.21.15.50:0/4244940092 wait complete. 2024-05-08T18:05:10.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.133+0000 7f7df4485800 1 Processor -- start 2024-05-08T18:05:10.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.133+0000 7f7df4485800 1 -- start start 2024-05-08T18:05:10.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.133+0000 7f7df4485800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 0x55637b3d69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.133+0000 7f7df4485800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55637b2db9f0 con 0x55637b2d9fa0 2024-05-08T18:05:10.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.134+0000 7f7df1f7d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 0x55637b3d69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.134+0000 7f7df1f7d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 0x55637b3d69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38500/0 (socket says 172.21.15.50:38500) 2024-05-08T18:05:10.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.134+0000 7f7df1f7d640 1 -- 172.21.15.50:0/1390405526 learned_addr learned my addr 172.21.15.50:0/1390405526 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.134+0000 7f7df1f7d640 1 -- 172.21.15.50:0/1390405526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637b3d6f30 con 0x55637b2d9fa0 2024-05-08T18:05:10.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.134+0000 7f7df1f7d640 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 0x55637b3d69f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7de4060040 tx=0x7f7de4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.134+0000 7f7cde604640 1 -- 172.21.15.50:0/1390405526 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de4052e30 con 0x55637b2d9fa0 2024-05-08T18:05:10.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.134+0000 7f7cde604640 1 -- 172.21.15.50:0/1390405526 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de406c070 con 0x55637b2d9fa0 2024-05-08T18:05:10.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.134+0000 7f7df4485800 1 -- 172.21.15.50:0/1390405526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55637b3d71c0 con 0x55637b2d9fa0 2024-05-08T18:05:10.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.134+0000 7f7cde604640 1 -- 172.21.15.50:0/1390405526 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de405e5f0 con 0x55637b2d9fa0 2024-05-08T18:05:10.135 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.135+0000 7f7df4485800 1 -- 172.21.15.50:0/1390405526 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637b3d9e70 con 0x55637b2d9fa0 2024-05-08T18:05:10.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.135+0000 7f7cde604640 1 -- 172.21.15.50:0/1390405526 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7de405e790 con 0x55637b2d9fa0 2024-05-08T18:05:10.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.136+0000 7f7cde604640 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7cd803bb50 0x7f7cd803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.136+0000 7f7cde604640 1 -- 172.21.15.50:0/1390405526 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7de4094e20 con 0x55637b2d9fa0 2024-05-08T18:05:10.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.136+0000 7f7df177c640 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7cd803bb50 0x7f7cd803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.136+0000 7f7df4485800 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55637b2da900 0x55637b3ddc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.136+0000 7f7df4485800 1 -- 172.21.15.50:0/1390405526 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b3de170 con 0x55637b2da900 2024-05-08T18:05:10.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.136+0000 7f7df277e640 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55637b2da900 0x55637b3ddc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.136+0000 7f7df177c640 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7cd803bb50 0x7f7cd803e010 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7de8001d70 tx=0x7f7de80012d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.136+0000 7f7df277e640 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55637b2da900 0x55637b3ddc30 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.137+0000 7f7df277e640 1 -- 172.21.15.50:0/1390405526 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b2da900 2024-05-08T18:05:10.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.137+0000 7f7df4485800 1 -- 172.21.15.50:0/1390405526 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b3dea50 con 0x55637b2da900 2024-05-08T18:05:10.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.137+0000 7f7df277e640 1 -- 172.21.15.50:0/1390405526 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b2da900 2024-05-08T18:05:10.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.137+0000 7f7df4485800 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55637b3e0360 0x55637b3e2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.137+0000 7f7df4485800 1 -- 172.21.15.50:0/1390405526 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b3e2f90 con 0x55637b3e0360 2024-05-08T18:05:10.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.137+0000 7f7df1f7d640 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55637b3e0360 0x55637b3e2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.137+0000 7f7df1f7d640 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55637b3e0360 0x55637b3e2800 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.138+0000 7f7df1f7d640 1 -- 172.21.15.50:0/1390405526 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7de4009b60 con 0x55637b3e0360 2024-05-08T18:05:10.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.138+0000 7f7df4485800 1 -- 172.21.15.50:0/1390405526 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b3df2a0 con 0x55637b2da900 2024-05-08T18:05:10.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.138+0000 7f7df277e640 1 -- 172.21.15.50:0/1390405526 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b2da900 2024-05-08T18:05:10.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.138+0000 7f7df4485800 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55637b3e43f0 0x55637b3e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.138+0000 7f7df4485800 1 -- 172.21.15.50:0/1390405526 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b3e6f20 con 0x55637b3e43f0 2024-05-08T18:05:10.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.138+0000 7f7df177c640 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55637b3e43f0 0x55637b3e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.139+0000 7f7df177c640 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55637b3e43f0 0x55637b3e67e0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.139+0000 7f7df177c640 1 -- 172.21.15.50:0/1390405526 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7de8009050 con 0x55637b3e43f0 2024-05-08T18:05:10.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.139+0000 7f7df4485800 1 Processor -- start 2024-05-08T18:05:10.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.139+0000 7f7df4485800 1 -- start start 2024-05-08T18:05:10.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.139+0000 7f7df4485800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b3f5250 0x55637b3f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.139+0000 7f7df4485800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55637b1cd330 con 0x55637b3f5250 2024-05-08T18:05:10.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.140+0000 7f7df1f7d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b3f5250 0x55637b3f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.140+0000 7f7df1f7d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b3f5250 0x55637b3f5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38510/0 (socket says 172.21.15.50:38510) 2024-05-08T18:05:10.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.140+0000 7f7df1f7d640 1 -- 172.21.15.50:0/1930440541 learned_addr learned my addr 172.21.15.50:0/1930440541 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.140+0000 7f7df1f7d640 1 -- 172.21.15.50:0/1930440541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637b3f5bb0 con 0x55637b3f5250 2024-05-08T18:05:10.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.140+0000 7f7df1f7d640 1 --2- 172.21.15.50:0/1930440541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b3f5250 0x55637b3f5670 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7de4093210 tx=0x7f7de405c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.140+0000 7f7cd77fe640 1 -- 172.21.15.50:0/1930440541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de4089070 con 0x55637b3f5250 2024-05-08T18:05:10.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.140+0000 7f7cd77fe640 1 -- 172.21.15.50:0/1930440541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de4058040 con 0x55637b3f5250 2024-05-08T18:05:10.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.140+0000 7f7df4485800 1 -- 172.21.15.50:0/1930440541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55637b3f6620 con 0x55637b3f5250 2024-05-08T18:05:10.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.140+0000 7f7cd77fe640 1 -- 172.21.15.50:0/1930440541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de40095d0 con 0x55637b3f5250 2024-05-08T18:05:10.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.140+0000 7f7df4485800 1 -- 172.21.15.50:0/1930440541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637b3f6b20 con 0x55637b3f5250 2024-05-08T18:05:10.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.141+0000 7f7cd77fe640 1 -- 172.21.15.50:0/1930440541 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7de408c070 con 0x55637b3f5250 2024-05-08T18:05:10.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.141+0000 7f7cd77fe640 1 --2- 172.21.15.50:0/1930440541 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ccc03bb50 0x7f7ccc03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.141+0000 7f7cd77fe640 1 -- 172.21.15.50:0/1930440541 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7de407b050 con 0x55637b3f5250 2024-05-08T18:05:10.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.141+0000 7f7df177c640 1 --2- 172.21.15.50:0/1930440541 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ccc03bb50 0x7f7ccc03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.142+0000 7f7df4485800 1 Processor -- start 2024-05-08T18:05:10.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.142+0000 7f7df4485800 1 -- start start 2024-05-08T18:05:10.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.142+0000 7f7df177c640 1 --2- 172.21.15.50:0/1930440541 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ccc03bb50 0x7f7ccc03e010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7de800bf40 tx=0x7f7de800f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.142+0000 7f7df4485800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b4023d0 0x55637b4027f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.142+0000 7f7df4485800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55637b3cf800 con 0x55637b4023d0 2024-05-08T18:05:10.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.142+0000 7f7df1f7d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b4023d0 0x55637b4027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.142+0000 7f7df1f7d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b4023d0 0x55637b4027f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38514/0 (socket says 172.21.15.50:38514) 2024-05-08T18:05:10.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.142+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3112667182 learned_addr learned my addr 172.21.15.50:0/3112667182 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.142+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3112667182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637b402dc0 con 0x55637b4023d0 2024-05-08T18:05:10.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.142+0000 7f7df1f7d640 1 --2- 172.21.15.50:0/3112667182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b4023d0 0x55637b4027f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7de408c660 tx=0x7f7de405b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eba4ab7b61728f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.143+0000 7f7cd57fa640 1 -- 172.21.15.50:0/3112667182 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de40ac070 con 0x55637b4023d0 2024-05-08T18:05:10.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.143+0000 7f7cd57fa640 1 -- 172.21.15.50:0/3112667182 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de40098a0 con 0x55637b4023d0 2024-05-08T18:05:10.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.143+0000 7f7cd57fa640 1 -- 172.21.15.50:0/3112667182 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de4071c00 con 0x55637b4023d0 2024-05-08T18:05:10.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.143+0000 7f7df4485800 1 -- 172.21.15.50:0/3112667182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b4023d0 msgr2=0x55637b4027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.143+0000 7f7df4485800 1 --2- 172.21.15.50:0/3112667182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b4023d0 0x55637b4027f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7de408c660 tx=0x7f7de405b580 comp rx=0 tx=0).stop 2024-05-08T18:05:10.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.143+0000 7f7df4485800 1 -- 172.21.15.50:0/3112667182 shutdown_connections 2024-05-08T18:05:10.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.143+0000 7f7df4485800 1 --2- 172.21.15.50:0/3112667182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b4023d0 0x55637b4027f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.143+0000 7f7df4485800 1 -- 172.21.15.50:0/3112667182 >> 172.21.15.50:0/3112667182 conn(0x55637b3fe6e0 msgr2=0x55637b3fc740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.143+0000 7f7df4485800 1 -- 172.21.15.50:0/3112667182 shutdown_connections 2024-05-08T18:05:10.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.143+0000 7f7df4485800 1 -- 172.21.15.50:0/3112667182 wait complete. 2024-05-08T18:05:10.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.143+0000 7f7df4485800 1 Processor -- start 2024-05-08T18:05:10.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.144+0000 7f7df4485800 1 -- start start 2024-05-08T18:05:10.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.144+0000 7f7df4485800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b4023d0 0x55637b409810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.144+0000 7f7df4485800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55637b403450 con 0x55637b4023d0 2024-05-08T18:05:10.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.144+0000 7f7df1f7d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b4023d0 0x55637b409810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.144+0000 7f7df1f7d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b4023d0 0x55637b409810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38526/0 (socket says 172.21.15.50:38526) 2024-05-08T18:05:10.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.144+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 learned_addr learned my addr 172.21.15.50:0/3595317482 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.144+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637b409d50 con 0x55637b4023d0 2024-05-08T18:05:10.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.144+0000 7f7df1f7d640 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b4023d0 0x55637b409810 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7de4068010 tx=0x7f7de406ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.144+0000 7f7cc37fe640 1 -- 172.21.15.50:0/3595317482 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de40b6020 con 0x55637b4023d0 2024-05-08T18:05:10.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.144+0000 7f7cc37fe640 1 -- 172.21.15.50:0/3595317482 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7de4053b20 con 0x55637b4023d0 2024-05-08T18:05:10.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.144+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55637b409d50 con 0x55637b4023d0 2024-05-08T18:05:10.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.144+0000 7f7cc37fe640 1 -- 172.21.15.50:0/3595317482 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7de40ac040 con 0x55637b4023d0 2024-05-08T18:05:10.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.145+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637b409d50 con 0x55637b4023d0 2024-05-08T18:05:10.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.145+0000 7f7cc37fe640 1 -- 172.21.15.50:0/3595317482 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7de40b9070 con 0x55637b4023d0 2024-05-08T18:05:10.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.146+0000 7f7cc37fe640 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7cbc03bba0 0x7f7cbc03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.146+0000 7f7cc37fe640 1 -- 172.21.15.50:0/3595317482 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f7de40a3030 con 0x55637b4023d0 2024-05-08T18:05:10.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.146+0000 7f7df177c640 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7cbc03bba0 0x7f7cbc03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.146+0000 7f7df4485800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.146+0000 7f7df4485800 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55637b414ea0 0x55637b417360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.146+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b417ab0 con 0x55637b414ea0 2024-05-08T18:05:10.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.146+0000 7f7df177c640 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7cbc03bba0 0x7f7cbc03e060 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7de800c010 tx=0x7f7de8010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.146+0000 7f7df277e640 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55637b414ea0 0x55637b417360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.147+0000 7f7df277e640 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55637b414ea0 0x55637b417360 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.147+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b414ea0 2024-05-08T18:05:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.147+0000 7f7df4485800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.147+0000 7f7df4485800 20 realm 2024-05-08T18:05:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.147+0000 7f7df4485800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.147+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b417ec0 con 0x55637b414ea0 2024-05-08T18:05:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.147+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b414ea0 2024-05-08T18:05:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.147+0000 7f7df4485800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.147+0000 7f7df4485800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.147+0000 7f7df4485800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.147+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b418790 con 0x55637b414ea0 2024-05-08T18:05:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.147+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b414ea0 2024-05-08T18:05:10.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.147+0000 7f7df4485800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.147+0000 7f7df4485800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.147+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b414ea0 2024-05-08T18:05:10.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.148+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b414ea0 2024-05-08T18:05:10.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.148+0000 7f7df4485800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:10.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.148+0000 7f7df4485800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.148+0000 7f7df4485800 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55637b419620 0x55637b41ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.148+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b41c150 con 0x55637b419620 2024-05-08T18:05:10.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.148+0000 7f7df1f7d640 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55637b419620 0x55637b41ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.148+0000 7f7df1f7d640 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55637b419620 0x55637b41ba10 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.148+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.148+0000 7f7df4485800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:10.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.148+0000 7f7df4485800 20 searching for the correct realm 2024-05-08T18:05:10.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.148+0000 7f7df4485800 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55637b41e1f0 0x55637b4206b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.148+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b420d80 con 0x55637b41e1f0 2024-05-08T18:05:10.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.149+0000 7f7df177c640 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55637b41e1f0 0x55637b4206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.149+0000 7f7df177c640 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55637b41e1f0 0x55637b4206b0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.149+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.149+0000 7f7df177c640 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7de8037040 0x7f7de8036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.149+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b420d80 con 0x7f7de8037040 2024-05-08T18:05:10.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.149+0000 7f7df277e640 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7de8037040 0x7f7de8036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.150+0000 7f7df277e640 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7de8037040 0x7f7de8036410 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.150+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7dec00a4a0 con 0x7f7de8037040 2024-05-08T18:05:10.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.150+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b419620 2024-05-08T18:05:10.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.150+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.150+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b414ea0 2024-05-08T18:05:10.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.150+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b414ea0 2024-05-08T18:05:10.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.150+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b414ea0 2024-05-08T18:05:10.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.151+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b414ea0 2024-05-08T18:05:10.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.151+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b419620 2024-05-08T18:05:10.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.151+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.151+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b419620 2024-05-08T18:05:10.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.151+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.151+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x7f7de8037040 2024-05-08T18:05:10.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.152+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7dec00a4a0 con 0x7f7de8037040 2024-05-08T18:05:10.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.152+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b41e1f0 2024-05-08T18:05:10.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.152+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.152+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b414ea0 2024-05-08T18:05:10.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.152+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b414ea0 2024-05-08T18:05:10.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.152+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b419620 2024-05-08T18:05:10.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.152+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.152+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b414ea0 2024-05-08T18:05:10.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.153+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b414ea0 2024-05-08T18:05:10.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.153+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x7f7de8037040 2024-05-08T18:05:10.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.153+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7dec00a4a0 con 0x7f7de8037040 2024-05-08T18:05:10.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.153+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b41e1f0 2024-05-08T18:05:10.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.153+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.153+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b41e1f0 2024-05-08T18:05:10.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.153+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.154+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b419620 2024-05-08T18:05:10.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.154+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.154+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b41e1f0 2024-05-08T18:05:10.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.154+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.154+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b41e1f0 2024-05-08T18:05:10.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.154+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.154+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b414ea0 2024-05-08T18:05:10.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.155+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b414ea0 2024-05-08T18:05:10.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.155+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b414ea0 2024-05-08T18:05:10.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.155+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b414ea0 2024-05-08T18:05:10.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.155+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b419620 2024-05-08T18:05:10.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.155+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.155+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b41e1f0 2024-05-08T18:05:10.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.155+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.155+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x7f7de8037040 2024-05-08T18:05:10.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.156+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7dec00a4a0 con 0x7f7de8037040 2024-05-08T18:05:10.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.156+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b414ea0 2024-05-08T18:05:10.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.156+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b414ea0 2024-05-08T18:05:10.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.156+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b419620 2024-05-08T18:05:10.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.156+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.156+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b41e1f0 2024-05-08T18:05:10.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.156+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.156+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b419620 2024-05-08T18:05:10.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.157+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.157+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x7f7de8037040 2024-05-08T18:05:10.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.157+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7dec00a4a0 con 0x7f7de8037040 2024-05-08T18:05:10.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.157+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b41e1f0 2024-05-08T18:05:10.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.157+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.157+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b419620 2024-05-08T18:05:10.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b414ea0 2024-05-08T18:05:10.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b414ea0 2024-05-08T18:05:10.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x7f7de8037040 2024-05-08T18:05:10.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7dec00a4a0 con 0x7f7de8037040 2024-05-08T18:05:10.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df4485800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:10.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df4485800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:10.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df4485800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:10.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df4485800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:10.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df4485800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:10.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df4485800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:10.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df4485800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b41caf0 con 0x55637b414ea0 2024-05-08T18:05:10.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b414ea0 2024-05-08T18:05:10.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df4485800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df4485800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.158+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b41caf0 con 0x55637b414ea0 2024-05-08T18:05:10.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7dec00a4a0 con 0x55637b414ea0 2024-05-08T18:05:10.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df4485800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:10.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df4485800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b41caf0 con 0x55637b41e1f0 2024-05-08T18:05:10.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df4485800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:10.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df4485800 20 zone default found 2024-05-08T18:05:10.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df4485800 4 Realm: () 2024-05-08T18:05:10.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df4485800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:10.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df4485800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:10.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df4485800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:10.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df4485800 20 zonegroup default 2024-05-08T18:05:10.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df4485800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b41d390 con 0x55637b419620 2024-05-08T18:05:10.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df4485800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df4485800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b422be0 con 0x55637b41e1f0 2024-05-08T18:05:10.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.159+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.160+0000 7f7df4485800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.160+0000 7f7df4485800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.160+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b422f80 con 0x55637b41e1f0 2024-05-08T18:05:10.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.160+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.160+0000 7f7df4485800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.160+0000 7f7df4485800 20 started sync module instance, tier type = 2024-05-08T18:05:10.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.160+0000 7f7df4485800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:10.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.160+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b422080 con 0x7f7de8037040 2024-05-08T18:05:10.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.161+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7dec00a4a0 con 0x7f7de8037040 2024-05-08T18:05:10.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.161+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93885758055856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b422080 con 0x7f7de8037040 2024-05-08T18:05:10.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.161+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b4239f0 con 0x55637b414ea0 2024-05-08T18:05:10.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.162+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93885758055856] v27'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7dec040040 con 0x7f7de8037040 2024-05-08T18:05:10.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.162+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7dec040040 con 0x55637b414ea0 2024-05-08T18:05:10.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.162+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93885758059744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b422080 con 0x55637b414ea0 2024-05-08T18:05:10.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.162+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b424490 con 0x55637b41e1f0 2024-05-08T18:05:10.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.163+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93885758059744] v27'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7dec040040 con 0x55637b414ea0 2024-05-08T18:05:10.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.163+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.163+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93885758053328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b421ff0 con 0x55637b41e1f0 2024-05-08T18:05:10.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.163+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b425350 con 0x7f7de8037040 2024-05-08T18:05:10.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.164+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93885758053328] v27'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.164+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7dec040040 con 0x7f7de8037040 2024-05-08T18:05:10.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.164+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93885758066336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b425770 con 0x7f7de8037040 2024-05-08T18:05:10.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.165+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b426130 con 0x55637b414ea0 2024-05-08T18:05:10.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.166+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7dec040040 con 0x55637b414ea0 2024-05-08T18:05:10.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.166+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93885758066336] v27'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7dec040040 con 0x7f7de8037040 2024-05-08T18:05:10.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.166+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93885758069888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b426550 con 0x55637b414ea0 2024-05-08T18:05:10.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.166+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b4252e0 con 0x55637b419620 2024-05-08T18:05:10.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.167+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93885758069888] v27'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7dec040040 con 0x55637b414ea0 2024-05-08T18:05:10.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.167+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.167+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93885758071424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b426f70 con 0x55637b419620 2024-05-08T18:05:10.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.167+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b427820 con 0x55637b419620 2024-05-08T18:05:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.168+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93885758071424] v27'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.168+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.168+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93885758073536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b427a00 con 0x55637b419620 2024-05-08T18:05:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.168+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b428250 con 0x55637b419620 2024-05-08T18:05:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.169+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v27'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.169+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93885758078032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b428680 con 0x55637b419620 2024-05-08T18:05:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.169+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 93885758073536] v27'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.169+0000 7f7df4485800 20 add_watcher() i=0 2024-05-08T18:05:10.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.169+0000 7f7df4485800 20 add_watcher() i=1 2024-05-08T18:05:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.169+0000 7f7df4485800 20 add_watcher() i=2 2024-05-08T18:05:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.169+0000 7f7df4485800 20 add_watcher() i=3 2024-05-08T18:05:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.169+0000 7f7df4485800 20 add_watcher() i=4 2024-05-08T18:05:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.169+0000 7f7df4485800 20 add_watcher() i=5 2024-05-08T18:05:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.169+0000 7f7df4485800 20 add_watcher() i=6 2024-05-08T18:05:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.170+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93885758078032] v27'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.170+0000 7f7df4485800 20 add_watcher() i=7 2024-05-08T18:05:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.170+0000 7f7df4485800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.170+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55637b428680 con 0x55637b414ea0 2024-05-08T18:05:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.170+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7dec040040 con 0x55637b414ea0 2024-05-08T18:05:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.170+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93885758079248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b428680 con 0x55637b414ea0 2024-05-08T18:05:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.171+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93885758079248] v27'146 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7dec040040 con 0x55637b414ea0 2024-05-08T18:05:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.171+0000 7f7ca37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.171+0000 7f7df4485800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:10.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.171+0000 7f7df4485800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.171+0000 7f7df4485800 5 note: GC not initialized 2024-05-08T18:05:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.172+0000 7f7ca27fc640 20 reqs_thread_entry: start 2024-05-08T18:05:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.172+0000 7f7df4485800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.172+0000 7f7df4485800 20 Filter name: none 2024-05-08T18:05:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.172+0000 7f7c13fff640 20 reqs_thread_entry: start 2024-05-08T18:05:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.172+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 6.0 6:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b45e1f0 con 0x55637b419620 2024-05-08T18:05:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.172+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(62 [pgnls start_epoch 27 out=254b] v27'34 uv18 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.172+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.10 6:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b4252e0 con 0x55637b419620 2024-05-08T18:05:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.173+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(63 [pgnls start_epoch 27 out=131b] v27'5 uv4 ondisk = 1) ==== 144+0+131 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.173+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.8 6:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b4252e0 con 0x55637b41e1f0 2024-05-08T18:05:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.173+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 [pgnls start_epoch 27 out=295b] v27'23 uv15 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.173+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:65 6.18 6:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b4252e0 con 0x55637b419620 2024-05-08T18:05:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.173+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(65 [pgnls start_epoch 27 out=172b] v27'18 uv9 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.173+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 6.4 6:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b4252e0 con 0x55637b414ea0 2024-05-08T18:05:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.174+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(66 [pgnls start_epoch 27 out=335b] v27'146 uv23 ondisk = 1) ==== 144+0+335 (crc 0 0 0) 0x7f7dec040040 con 0x55637b414ea0 2024-05-08T18:05:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.174+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 6.14 6:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x7f7de8037040 2024-05-08T18:05:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.174+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(67 [pgnls start_epoch 27 out=213b] v27'10 uv10 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7f7dec040040 con 0x7f7de8037040 2024-05-08T18:05:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.174+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 6.c 6:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b41e1f0 2024-05-08T18:05:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.174+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(68 [pgnls start_epoch 27 out=254b] v27'10 uv10 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.174+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 6.1c 6:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b41e1f0 2024-05-08T18:05:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.175+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(69 [pgnls start_epoch 27 out=295b] v27'23 uv15 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.175+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:70 6.2 6:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b41e1f0 2024-05-08T18:05:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.175+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(70 [pgnls start_epoch 27 out=213b] v27'10 uv10 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.175+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:71 6.12 6:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x7f7de8037040 2024-05-08T18:05:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.175+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(71 [pgnls start_epoch 27 out=172b] v27'17 uv9 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7f7dec040040 con 0x7f7de8037040 2024-05-08T18:05:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.175+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.a 6:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x7f7de8037040 2024-05-08T18:05:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.175+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 [pgnls start_epoch 27 out=336b] v27'27 uv19 ondisk = 1) ==== 144+0+336 (crc 0 0 0) 0x7f7dec049160 con 0x7f7de8037040 2024-05-08T18:05:10.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.175+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:73 6.1a 6:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b41e1f0 2024-05-08T18:05:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.176+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(73 [pgnls start_epoch 27 out=131b] v27'17 uv9 ondisk = 1) ==== 144+0+131 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.176+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:74 6.6 6:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b41e1f0 2024-05-08T18:05:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.176+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(74 [pgnls start_epoch 27 out=254b] v27'12 uv12 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.176+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:75 6.16 6:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b41e1f0 2024-05-08T18:05:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.176+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(75 [pgnls start_epoch 27 out=90b] v27'17 uv9 ondisk = 1) ==== 144+0+90 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.176+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.e 6:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b414ea0 2024-05-08T18:05:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.176+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(76 [pgnls start_epoch 27 out=213b] v27'19 uv11 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7f7dec049160 con 0x55637b414ea0 2024-05-08T18:05:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.176+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.1e 6:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b41e1f0 2024-05-08T18:05:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.177+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(77 [pgnls start_epoch 27 out=254b] v27'10 uv10 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.177+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:78 6.1 6:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55637b418b30 con 0x55637b414ea0 2024-05-08T18:05:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.177+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(78 [pgnls start_epoch 27 out=254b] v27'14 uv14 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7f7dec049160 con 0x55637b414ea0 2024-05-08T18:05:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.177+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.11 6:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7dec04e040 con 0x55637b41e1f0 2024-05-08T18:05:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.177+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(79 [pgnls start_epoch 27 out=213b] v27'19 uv11 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.177+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:80 6.9 6:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7dec04e040 con 0x55637b41e1f0 2024-05-08T18:05:10.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.178+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(80 [pgnls start_epoch 27 out=172b] v27'28 uv12 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.178+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.19 6:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7dec04e040 con 0x55637b414ea0 2024-05-08T18:05:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.178+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(81 [pgnls start_epoch 27 out=295b] v27'23 uv15 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7f7dec049160 con 0x55637b414ea0 2024-05-08T18:05:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.178+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:82 6.5 6:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7dec04e040 con 0x7f7de8037040 2024-05-08T18:05:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.178+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(82 [pgnls start_epoch 27 out=276b] v27'15 uv15 ondisk = 1) ==== 144+0+276 (crc 0 0 0) 0x7f7dec049160 con 0x7f7de8037040 2024-05-08T18:05:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.178+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.15 6:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7dec04e040 con 0x7f7de8037040 2024-05-08T18:05:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.178+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(83 [pgnls start_epoch 27 out=172b] v27'17 uv9 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7f7dec049160 con 0x7f7de8037040 2024-05-08T18:05:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.178+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:84 6.d 6:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7dec04e040 con 0x55637b414ea0 2024-05-08T18:05:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.179+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(84 [pgnls start_epoch 27 out=295b] v27'25 uv17 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7f7dec049160 con 0x55637b414ea0 2024-05-08T18:05:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.179+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:85 6.1d 6:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7dec04e040 con 0x55637b419620 2024-05-08T18:05:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.179+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(85 [pgnls start_epoch 27 out=172b] v27'28 uv12 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.179+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:86 6.3 6:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7dec04e040 con 0x7f7de8037040 2024-05-08T18:05:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.179+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(86 [pgnls start_epoch 27 out=213b] v27'43 uv19 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7f7dec049160 con 0x7f7de8037040 2024-05-08T18:05:10.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.179+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:87 6.13 6:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7dec04e040 con 0x55637b419620 2024-05-08T18:05:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.180+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(87 [pgnls start_epoch 27 out=172b] v27'19 uv11 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.180+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:88 6.b 6:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7dec04e040 con 0x55637b41e1f0 2024-05-08T18:05:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.180+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(88 [pgnls start_epoch 27 out=172b] v27'17 uv9 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.180+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1b 6:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7dec04e040 con 0x55637b41e1f0 2024-05-08T18:05:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.181+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(89 [pgnls start_epoch 27 out=90b] v27'13 uv5 ondisk = 1) ==== 144+0+90 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.181+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:90 6.7 6:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7dec04e040 con 0x55637b41e1f0 2024-05-08T18:05:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.181+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(90 [pgnls start_epoch 27 out=90b] v27'38 uv13 ondisk = 1) ==== 144+0+90 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.181+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:91 6.17 6:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7dec04e040 con 0x55637b414ea0 2024-05-08T18:05:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.181+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(91 [pgnls start_epoch 27 out=131b] v27'6 uv6 ondisk = 1) ==== 144+0+131 (crc 0 0 0) 0x7f7dec049160 con 0x55637b414ea0 2024-05-08T18:05:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.181+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:92 6.f 6:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7dec04e040 con 0x7f7de8037040 2024-05-08T18:05:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.181+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(92 [pgnls start_epoch 27 out=295b] v27'23 uv15 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7f7dec049160 con 0x7f7de8037040 2024-05-08T18:05:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.181+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:93 6.1f 6:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f7dec04e040 con 0x55637b419620 2024-05-08T18:05:10.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.182+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(93 [pgnls start_epoch 27 out=213b] v27'19 uv11 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.192 INFO:teuthology.orchestra.run.smithi050.stdout:["obj_delete_at_hint.0000000078","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","obj_delete_at_hint.0000000028","obj_delete_at_hint.0000000040","obj_delete_at_hint.0000000015","obj_delete_at_hint.0000000069","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","obj_delete_at_hint.0000000024","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","data_loggenerations_metadata","obj_delete_at_hint.0000000076","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","obj_delete_at_hint.0000000088","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","obj_delete_at_hint.0000000110","obj_delete_at_hint.0000000096","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","obj_delete_at_hint.0000000006","obj_delete_at_hint.0000000029","obj_delete_at_hint.0000000089","obj_delete_at_hint.0000000022","obj_delete_at_hint.0000000016","obj_delete_at_hint.0000000041","obj_delete_at_hint.0000000018","obj_delete_at_hint.0000000092","obj_delete_at_hint.0000000014","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","obj_delete_at_hint.0000000064","obj_delete_at_hint.0000000071","obj_delete_at_hint.0000000074","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","obj_delete_at_hint.0000000053","obj_delete_at_hint.0000000091","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","obj_delete_at_hint.0000000045","obj_delete_at_hint.0000000002","obj_delete_at_hint.0000000116","obj_delete_at_hint.0000000034","obj_delete_at_hint.0000000101","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","obj_delete_at_hint.0000000066","obj_delete_at_hint.0000000068","obj_delete_at_hint.0000000085","obj_delete_at_hint.0000000073","obj_delete_at_hint.0000000038","obj_delete_at_hint.0000000118","obj_delete_at_hint.0000000036","obj_delete_at_hint.0000000103","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","obj_delete_at_hint.0000000027","obj_delete_at_hint.0000000019","obj_delete_at_hint.0000000039","obj_delete_at_hint.0000000100","obj_delete_at_hint.0000000093","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","obj_delete_at_hint.0000000122","obj_delete_at_hint.0000000057","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","data_log.0","obj_delete_at_hint.0000000012","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","obj_delete_at_hint.0000000111","obj_delete_at_hint.0000000048","obj_delete_at_hint.0000000020","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","obj_delete_at_hint.0000000072","obj_delete_at_hint.0000000062","obj_delete_at_hint.0000000115","obj_delete_at_hint.0000000033","obj_delete_at_hint.0000000025","obj_delete_at_hint.0000000123","obj_delete_at_hint.0000000108","obj_delete_at_hint.0000000094","obj_delete_at_hint.0000000060","obj_delete_at_hint.0000000109","obj_delete_at_hint.0000000013","obj_delete_at_hint.0000000051","obj_delete_at_hint.0000000106","obj_delete_at_hint.0000000017","obj_delete_at_hint.0000000023","obj_delete_at_hint.0000000107","obj_delete_at_hint.0000000046","obj_delete_at_hint.0000000067","obj_delete_at_hint.0000000086","obj_delete_at_hint.0000000050","obj_delete_at_hint.0000000102","obj_delete_at_hint.0000000124","obj_delete_at_hint.0000000001","obj_delete_at_hint.0000000126","obj_delete_at_hint.0000000055","obj_delete_at_hint.0000000061","obj_delete_at_hint.0000000097","obj_delete_at_hint.0000000042","obj_delete_at_hint.0000000075","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030"] 2024-05-08T18:05:10.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.184+0000 7f7ca37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:10.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.184+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93885758079248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b45e1b0 con 0x55637b414ea0 2024-05-08T18:05:10.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.185+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 93885758079248] v27'147 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7dec049160 con 0x55637b414ea0 2024-05-08T18:05:10.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.185+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93885758055856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b45c6a0 con 0x7f7de8037040 2024-05-08T18:05:10.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.186+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 93885758055856] v27'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7dec049160 con 0x7f7de8037040 2024-05-08T18:05:10.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.186+0000 7f7df4485800 20 remove_watcher() i=0 2024-05-08T18:05:10.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.186+0000 7f7df4485800 2 removed watcher, disabling cache 2024-05-08T18:05:10.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.186+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93885758059744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b45c6a0 con 0x55637b414ea0 2024-05-08T18:05:10.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.187+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 93885758059744] v27'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7dec049160 con 0x55637b414ea0 2024-05-08T18:05:10.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.187+0000 7f7df4485800 20 remove_watcher() i=1 2024-05-08T18:05:10.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.187+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93885758053328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b45c6a0 con 0x55637b41e1f0 2024-05-08T18:05:10.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.188+0000 7f7df177c640 1 -- 172.21.15.50:0/3595317482 <== osd.3 v2:172.21.15.50:6800/3130492744 28 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 93885758053328] v27'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7de8009050 con 0x55637b41e1f0 2024-05-08T18:05:10.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.188+0000 7f7df4485800 20 remove_watcher() i=2 2024-05-08T18:05:10.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.188+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93885758066336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b421bd0 con 0x7f7de8037040 2024-05-08T18:05:10.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.189+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 93885758066336] v27'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7dec049160 con 0x7f7de8037040 2024-05-08T18:05:10.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.189+0000 7f7df4485800 20 remove_watcher() i=3 2024-05-08T18:05:10.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.189+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93885758069888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b424ea0 con 0x55637b414ea0 2024-05-08T18:05:10.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.190+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 93885758069888] v27'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7dec049160 con 0x55637b414ea0 2024-05-08T18:05:10.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.190+0000 7f7df4485800 20 remove_watcher() i=4 2024-05-08T18:05:10.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.190+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93885758071424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b421bd0 con 0x55637b419620 2024-05-08T18:05:10.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.191+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 93885758071424] v27'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.191+0000 7f7df4485800 20 remove_watcher() i=5 2024-05-08T18:05:10.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.191+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93885758073536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b425c80 con 0x55637b419620 2024-05-08T18:05:10.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.192+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 93885758073536] v27'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.192+0000 7f7df4485800 20 remove_watcher() i=6 2024-05-08T18:05:10.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.192+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93885758078032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55637b424ea0 con 0x55637b419620 2024-05-08T18:05:10.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df1f7d640 1 -- 172.21.15.50:0/3595317482 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 93885758078032] v27'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7de40b8030 con 0x55637b419620 2024-05-08T18:05:10.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 20 remove_watcher() i=7 2024-05-08T18:05:10.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55637b419620 msgr2=0x55637b41ba10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55637b419620 0x55637b41ba10 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55637b414ea0 msgr2=0x55637b417360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55637b414ea0 0x55637b417360 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7de8037040 msgr2=0x7f7de8036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7de8037040 0x7f7de8036410 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55637b41e1f0 msgr2=0x55637b4206b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55637b41e1f0 0x55637b4206b0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7cbc03bba0 msgr2=0x7f7cbc03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7cbc03bba0 0x7f7cbc03e060 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7de800c010 tx=0x7f7de8010000 comp rx=0 tx=0).stop 2024-05-08T18:05:10.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b4023d0 msgr2=0x55637b409810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df277e640 1 -- 172.21.15.50:0/3595317482 reap_dead start 2024-05-08T18:05:10.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b4023d0 0x55637b409810 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7de4068010 tx=0x7f7de406ffa0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 shutdown_connections 2024-05-08T18:05:10.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 --2- 172.21.15.50:0/3595317482 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b4023d0 0x55637b409810 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 >> 172.21.15.50:0/3595317482 conn(0x55637b3fe6e0 msgr2=0x55637b3fbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 shutdown_connections 2024-05-08T18:05:10.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 -- 172.21.15.50:0/3595317482 wait complete. 2024-05-08T18:05:10.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 -- 172.21.15.50:0/1930440541 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ccc03bb50 msgr2=0x7f7ccc03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 --2- 172.21.15.50:0/1930440541 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ccc03bb50 0x7f7ccc03e010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7de800bf40 tx=0x7f7de800f000 comp rx=0 tx=0).stop 2024-05-08T18:05:10.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 -- 172.21.15.50:0/1930440541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b3f5250 msgr2=0x55637b3f5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 --2- 172.21.15.50:0/1930440541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b3f5250 0x55637b3f5670 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7de4093210 tx=0x7f7de405c670 comp rx=0 tx=0).stop 2024-05-08T18:05:10.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 -- 172.21.15.50:0/1930440541 shutdown_connections 2024-05-08T18:05:10.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 --2- 172.21.15.50:0/1930440541 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ccc03bb50 0x7f7ccc03e010 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 --2- 172.21.15.50:0/1930440541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b3f5250 0x55637b3f5670 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.193+0000 7f7df4485800 1 -- 172.21.15.50:0/1930440541 >> 172.21.15.50:0/1930440541 conn(0x55637b3e92f0 msgr2=0x55637b3e96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df4485800 1 -- 172.21.15.50:0/1930440541 shutdown_connections 2024-05-08T18:05:10.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df4485800 1 -- 172.21.15.50:0/1930440541 wait complete. 2024-05-08T18:05:10.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df4485800 1 -- 172.21.15.50:0/1390405526 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55637b3e0360 msgr2=0x55637b3e2800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df4485800 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55637b3e0360 0x55637b3e2800 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df4485800 1 -- 172.21.15.50:0/1390405526 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55637b2da900 msgr2=0x55637b3ddc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df4485800 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55637b2da900 0x55637b3ddc30 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df4485800 1 -- 172.21.15.50:0/1390405526 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55637b3e43f0 msgr2=0x55637b3e67e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df4485800 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55637b3e43f0 0x55637b3e67e0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df4485800 1 -- 172.21.15.50:0/1390405526 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7cd803bb50 msgr2=0x7f7cd803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df4485800 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7cd803bb50 0x7f7cd803e010 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7de8001d70 tx=0x7f7de80012d0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df4485800 1 -- 172.21.15.50:0/1390405526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 msgr2=0x55637b3d69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df4485800 1 --2- 172.21.15.50:0/1390405526 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55637b2d9fa0 0x55637b3d69f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7de4060040 tx=0x7f7de4002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df277e640 1 -- 172.21.15.50:0/1390405526 reap_dead start 2024-05-08T18:05:10.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df4485800 1 -- 172.21.15.50:0/1390405526 shutdown_connections 2024-05-08T18:05:10.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df4485800 1 -- 172.21.15.50:0/1390405526 >> 172.21.15.50:0/1390405526 conn(0x55637b2d5ac0 msgr2=0x55637b3c9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df4485800 1 -- 172.21.15.50:0/1390405526 shutdown_connections 2024-05-08T18:05:10.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.194+0000 7f7df4485800 1 -- 172.21.15.50:0/1390405526 wait complete. 2024-05-08T18:05:10.203 DEBUG:tasks.util.rgw: json result: ['obj_delete_at_hint.0000000078', 'obj_delete_at_hint.0000000070', 'obj_delete_at_hint.0000000104', 'obj_delete_at_hint.0000000026', 'obj_delete_at_hint.0000000028', 'obj_delete_at_hint.0000000040', 'obj_delete_at_hint.0000000015', 'obj_delete_at_hint.0000000069', 'obj_delete_at_hint.0000000095', 'obj_delete_at_hint.0000000003', 'obj_delete_at_hint.0000000047', 'obj_delete_at_hint.0000000035', 'obj_delete_at_hint.0000000037', 'obj_delete_at_hint.0000000024', 'obj_delete_at_hint.0000000000', 'obj_delete_at_hint.0000000031', 'data_loggenerations_metadata', 'obj_delete_at_hint.0000000076', 'obj_delete_at_hint.0000000113', 'obj_delete_at_hint.0000000005', 'obj_delete_at_hint.0000000011', 'obj_delete_at_hint.0000000058', 'obj_delete_at_hint.0000000052', 'obj_delete_at_hint.0000000088', 'obj_delete_at_hint.0000000080', 'obj_delete_at_hint.0000000090', 'obj_delete_at_hint.0000000110', 'obj_delete_at_hint.0000000096', 'obj_delete_at_hint.0000000087', 'obj_delete_at_hint.0000000008', 'obj_delete_at_hint.0000000006', 'obj_delete_at_hint.0000000029', 'obj_delete_at_hint.0000000089', 'obj_delete_at_hint.0000000022', 'obj_delete_at_hint.0000000016', 'obj_delete_at_hint.0000000041', 'obj_delete_at_hint.0000000018', 'obj_delete_at_hint.0000000092', 'obj_delete_at_hint.0000000014', 'obj_delete_at_hint.0000000112', 'obj_delete_at_hint.0000000007', 'obj_delete_at_hint.0000000021', 'obj_delete_at_hint.0000000064', 'obj_delete_at_hint.0000000071', 'obj_delete_at_hint.0000000074', 'obj_delete_at_hint.0000000081', 'obj_delete_at_hint.0000000009', 'obj_delete_at_hint.0000000121', 'obj_delete_at_hint.0000000125', 'obj_delete_at_hint.0000000082', 'obj_delete_at_hint.0000000105', 'obj_delete_at_hint.0000000059', 'obj_delete_at_hint.0000000077', 'obj_delete_at_hint.0000000032', 'obj_delete_at_hint.0000000053', 'obj_delete_at_hint.0000000091', 'obj_delete_at_hint.0000000065', 'obj_delete_at_hint.0000000083', 'obj_delete_at_hint.0000000010', 'obj_delete_at_hint.0000000045', 'obj_delete_at_hint.0000000002', 'obj_delete_at_hint.0000000116', 'obj_delete_at_hint.0000000034', 'obj_delete_at_hint.0000000101', 'obj_delete_at_hint.0000000079', 'obj_delete_at_hint.0000000049', 'obj_delete_at_hint.0000000117', 'obj_delete_at_hint.0000000044', 'obj_delete_at_hint.0000000066', 'obj_delete_at_hint.0000000068', 'obj_delete_at_hint.0000000085', 'obj_delete_at_hint.0000000073', 'obj_delete_at_hint.0000000038', 'obj_delete_at_hint.0000000118', 'obj_delete_at_hint.0000000036', 'obj_delete_at_hint.0000000103', 'obj_delete_at_hint.0000000119', 'obj_delete_at_hint.0000000098', 'obj_delete_at_hint.0000000027', 'obj_delete_at_hint.0000000019', 'obj_delete_at_hint.0000000039', 'obj_delete_at_hint.0000000100', 'obj_delete_at_hint.0000000093', 'obj_delete_at_hint.0000000004', 'obj_delete_at_hint.0000000063', 'obj_delete_at_hint.0000000122', 'obj_delete_at_hint.0000000057', 'obj_delete_at_hint.0000000054', 'obj_delete_at_hint.0000000114', 'data_log.0', 'obj_delete_at_hint.0000000012', 'obj_delete_at_hint.0000000084', 'obj_delete_at_hint.0000000043', 'obj_delete_at_hint.0000000111', 'obj_delete_at_hint.0000000048', 'obj_delete_at_hint.0000000020', 'obj_delete_at_hint.0000000099', 'obj_delete_at_hint.0000000056', 'obj_delete_at_hint.0000000072', 'obj_delete_at_hint.0000000062', 'obj_delete_at_hint.0000000115', 'obj_delete_at_hint.0000000033', 'obj_delete_at_hint.0000000025', 'obj_delete_at_hint.0000000123', 'obj_delete_at_hint.0000000108', 'obj_delete_at_hint.0000000094', 'obj_delete_at_hint.0000000060', 'obj_delete_at_hint.0000000109', 'obj_delete_at_hint.0000000013', 'obj_delete_at_hint.0000000051', 'obj_delete_at_hint.0000000106', 'obj_delete_at_hint.0000000017', 'obj_delete_at_hint.0000000023', 'obj_delete_at_hint.0000000107', 'obj_delete_at_hint.0000000046', 'obj_delete_at_hint.0000000067', 'obj_delete_at_hint.0000000086', 'obj_delete_at_hint.0000000050', 'obj_delete_at_hint.0000000102', 'obj_delete_at_hint.0000000124', 'obj_delete_at_hint.0000000001', 'obj_delete_at_hint.0000000126', 'obj_delete_at_hint.0000000055', 'obj_delete_at_hint.0000000061', 'obj_delete_at_hint.0000000097', 'obj_delete_at_hint.0000000042', 'obj_delete_at_hint.0000000075', 'obj_delete_at_hint.0000000120', 'obj_delete_at_hint.0000000030'] 2024-05-08T18:05:10.203 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2024-05-08T18:05:10.203 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2024-05-08T18:05:10.204 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2024-05-08T18:05:10.289 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:10.289 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:10.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.291+0000 7f0d1bd2d800 1 Processor -- start 2024-05-08T18:05:10.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.291+0000 7f0d1bd2d800 1 -- start start 2024-05-08T18:05:10.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.291+0000 7f0d1bd2d800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5586c522fbf0 0x5586c5231fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.291+0000 7f0d1bd2d800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5586c5232520 con 0x5586c522c0e0 2024-05-08T18:05:10.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.291+0000 7f0d1bd2d800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5586c50cbe20 con 0x5586c522fbf0 2024-05-08T18:05:10.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.291+0000 7f0d172bd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5586c522fbf0 0x5586c5231fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.291+0000 7f0d17abe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5586c522c0e0 0x5586c522e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43294/0 (socket says 172.21.15.50:43294) 2024-05-08T18:05:10.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.291+0000 7f0d17abe640 1 -- 172.21.15.50:0/81102229 learned_addr learned my addr 172.21.15.50:0/81102229 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:10.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.292+0000 7f0d16abc640 1 -- 172.21.15.50:0/81102229 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2165893567 0 0) 0x5586c5232520 con 0x5586c522c0e0 2024-05-08T18:05:10.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.292+0000 7f0d16abc640 1 -- 172.21.15.50:0/81102229 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d04001e10 con 0x5586c522c0e0 2024-05-08T18:05:10.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.292+0000 7f0d172bd640 1 -- 172.21.15.50:0/81102229 >> v1:172.21.15.46:6789/0 conn(0x5586c522c0e0 legacy=0x5586c522e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.292+0000 7f0d172bd640 1 -- 172.21.15.50:0/81102229 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586c5233700 con 0x5586c522fbf0 2024-05-08T18:05:10.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.293+0000 7f0d172bd640 1 --2- 172.21.15.50:0/81102229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522fbf0 0x5586c5231fe0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000a470 tx=0x7f0d0001cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83881138a660a79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.294+0000 7f0d16abc640 1 -- 172.21.15.50:0/81102229 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d00057070 con 0x5586c522fbf0 2024-05-08T18:05:10.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.294+0000 7f0d16abc640 1 -- 172.21.15.50:0/81102229 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d00052570 con 0x5586c522fbf0 2024-05-08T18:05:10.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.294+0000 7f0d16abc640 1 -- 172.21.15.50:0/81102229 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d00052850 con 0x5586c522fbf0 2024-05-08T18:05:10.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.294+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/81102229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522fbf0 msgr2=0x5586c5231fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.294+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/81102229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522fbf0 0x5586c5231fe0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000a470 tx=0x7f0d0001cdb0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.294+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/81102229 shutdown_connections 2024-05-08T18:05:10.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.294+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/81102229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522fbf0 0x5586c5231fe0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.294+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/81102229 >> 172.21.15.50:0/81102229 conn(0x5586c5225b50 msgr2=0x5586c5227f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.294+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/81102229 shutdown_connections 2024-05-08T18:05:10.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.294+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/81102229 wait complete. 2024-05-08T18:05:10.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.309+0000 7f0d1bd2d800 1 Processor -- start 2024-05-08T18:05:10.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.309+0000 7f0d1bd2d800 1 -- start start 2024-05-08T18:05:10.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.309+0000 7f0d1bd2d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522c0e0 0x5586c531c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.310+0000 7f0d1bd2d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5586c5176a10 con 0x5586c522c0e0 2024-05-08T18:05:10.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.310+0000 7f0d17abe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522c0e0 0x5586c531c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.310+0000 7f0d17abe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522c0e0 0x5586c531c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38554/0 (socket says 172.21.15.50:38554) 2024-05-08T18:05:10.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.310+0000 7f0d17abe640 1 -- 172.21.15.50:0/635238241 learned_addr learned my addr 172.21.15.50:0/635238241 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.310+0000 7f0d17abe640 1 -- 172.21.15.50:0/635238241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586c531cec0 con 0x5586c522c0e0 2024-05-08T18:05:10.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.310+0000 7f0d17abe640 1 --2- 172.21.15.50:0/635238241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522c0e0 0x5586c531c8f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0d08001de0 tx=0x7f0d0800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ec7d1d866508c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.310+0000 7f0c05e0b640 1 -- 172.21.15.50:0/635238241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d08012020 con 0x5586c522c0e0 2024-05-08T18:05:10.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.310+0000 7f0c05e0b640 1 -- 172.21.15.50:0/635238241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d0800e470 con 0x5586c522c0e0 2024-05-08T18:05:10.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.311+0000 7f0c05e0b640 1 -- 172.21.15.50:0/635238241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d0800e7b0 con 0x5586c522c0e0 2024-05-08T18:05:10.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.311+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/635238241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522c0e0 msgr2=0x5586c531c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.311+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/635238241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522c0e0 0x5586c531c8f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0d08001de0 tx=0x7f0d0800b6c0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.311+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/635238241 shutdown_connections 2024-05-08T18:05:10.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.311+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/635238241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522c0e0 0x5586c531c8f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.311+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/635238241 >> 172.21.15.50:0/635238241 conn(0x5586c5225b50 msgr2=0x5586c5318210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.311+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/635238241 shutdown_connections 2024-05-08T18:05:10.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.311+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/635238241 wait complete. 2024-05-08T18:05:10.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.311+0000 7f0d1bd2d800 1 Processor -- start 2024-05-08T18:05:10.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.312+0000 7f0d1bd2d800 1 -- start start 2024-05-08T18:05:10.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.312+0000 7f0d1bd2d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522c0e0 0x5586c5324460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.312+0000 7f0d1bd2d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5586c52329f0 con 0x5586c522c0e0 2024-05-08T18:05:10.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.312+0000 7f0d17abe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522c0e0 0x5586c5324460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.312+0000 7f0d17abe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522c0e0 0x5586c5324460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38560/0 (socket says 172.21.15.50:38560) 2024-05-08T18:05:10.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.312+0000 7f0d17abe640 1 -- 172.21.15.50:0/2159951613 learned_addr learned my addr 172.21.15.50:0/2159951613 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.312+0000 7f0d17abe640 1 -- 172.21.15.50:0/2159951613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586c53249a0 con 0x5586c522c0e0 2024-05-08T18:05:10.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.312+0000 7f0d17abe640 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522c0e0 0x5586c5324460 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800b700 tx=0x7f0d080040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.312+0000 7f0bfffff640 1 -- 172.21.15.50:0/2159951613 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d08012030 con 0x5586c522c0e0 2024-05-08T18:05:10.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.312+0000 7f0bfffff640 1 -- 172.21.15.50:0/2159951613 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d0801a590 con 0x5586c522c0e0 2024-05-08T18:05:10.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.312+0000 7f0bfffff640 1 -- 172.21.15.50:0/2159951613 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d0801a870 con 0x5586c522c0e0 2024-05-08T18:05:10.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.312+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2159951613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5586c5324c30 con 0x5586c522c0e0 2024-05-08T18:05:10.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.313+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2159951613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586c53278e0 con 0x5586c522c0e0 2024-05-08T18:05:10.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.313+0000 7f0bfffff640 1 -- 172.21.15.50:0/2159951613 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f0d08015020 con 0x5586c522c0e0 2024-05-08T18:05:10.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.314+0000 7f0bfffff640 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0c0003bba0 0x7f0c0003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.314+0000 7f0bfffff640 1 -- 172.21.15.50:0/2159951613 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f0d08020030 con 0x5586c522c0e0 2024-05-08T18:05:10.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.314+0000 7f0d172bd640 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0c0003bba0 0x7f0c0003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.314+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5586c522fbf0 0x5586c532b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.314+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2159951613 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c532bbe0 con 0x5586c522fbf0 2024-05-08T18:05:10.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.314+0000 7f0d182bf640 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5586c522fbf0 0x5586c532b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.314+0000 7f0d172bd640 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0c0003bba0 0x7f0c0003e060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0d00054000 tx=0x7f0d0004e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.314+0000 7f0d182bf640 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5586c522fbf0 0x5586c532b6a0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.315+0000 7f0d182bf640 1 -- 172.21.15.50:0/2159951613 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c522fbf0 2024-05-08T18:05:10.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.315+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2159951613 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c5328c70 con 0x5586c522fbf0 2024-05-08T18:05:10.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.315+0000 7f0d182bf640 1 -- 172.21.15.50:0/2159951613 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c522fbf0 2024-05-08T18:05:10.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.315+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5586c532df30 0x5586c5330320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.315+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2159951613 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c5330a10 con 0x5586c532df30 2024-05-08T18:05:10.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.315+0000 7f0d17abe640 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5586c532df30 0x5586c5330320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.315+0000 7f0d17abe640 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5586c532df30 0x5586c5330320 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.316+0000 7f0d17abe640 1 -- 172.21.15.50:0/2159951613 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f0d08018430 con 0x5586c532df30 2024-05-08T18:05:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.316+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2159951613 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c532cdd0 con 0x5586c522fbf0 2024-05-08T18:05:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.316+0000 7f0d182bf640 1 -- 172.21.15.50:0/2159951613 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c522fbf0 2024-05-08T18:05:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.316+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5586c5331e70 0x5586c5334260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.316+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2159951613 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c5334960 con 0x5586c5331e70 2024-05-08T18:05:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.316+0000 7f0d172bd640 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5586c5331e70 0x5586c5334260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.316+0000 7f0d172bd640 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5586c5331e70 0x5586c5334260 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.317+0000 7f0d172bd640 1 -- 172.21.15.50:0/2159951613 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f0d000528e0 con 0x5586c5331e70 2024-05-08T18:05:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.317+0000 7f0d1bd2d800 1 Processor -- start 2024-05-08T18:05:10.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.317+0000 7f0d1bd2d800 1 -- start start 2024-05-08T18:05:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.317+0000 7f0d1bd2d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c5342c90 0x5586c53430b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.317+0000 7f0d1bd2d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5586c511d330 con 0x5586c5342c90 2024-05-08T18:05:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.317+0000 7f0d17abe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c5342c90 0x5586c53430b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.317+0000 7f0d17abe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c5342c90 0x5586c53430b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38566/0 (socket says 172.21.15.50:38566) 2024-05-08T18:05:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.317+0000 7f0d17abe640 1 -- 172.21.15.50:0/332619826 learned_addr learned my addr 172.21.15.50:0/332619826 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.318+0000 7f0d17abe640 1 -- 172.21.15.50:0/332619826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586c53435f0 con 0x5586c5342c90 2024-05-08T18:05:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.318+0000 7f0d17abe640 1 --2- 172.21.15.50:0/332619826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c5342c90 0x5586c53430b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0d0802cb00 tx=0x7f0d08043360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.318+0000 7f0bfd7fa640 1 -- 172.21.15.50:0/332619826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d0802cb70 con 0x5586c5342c90 2024-05-08T18:05:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.318+0000 7f0bfd7fa640 1 -- 172.21.15.50:0/332619826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d0802cd10 con 0x5586c5342c90 2024-05-08T18:05:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.318+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/332619826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5586c5344060 con 0x5586c5342c90 2024-05-08T18:05:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.318+0000 7f0bfd7fa640 1 -- 172.21.15.50:0/332619826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d08018950 con 0x5586c5342c90 2024-05-08T18:05:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.318+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/332619826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586c5344060 con 0x5586c5342c90 2024-05-08T18:05:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.319+0000 7f0bfd7fa640 1 -- 172.21.15.50:0/332619826 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f0d08018b10 con 0x5586c5342c90 2024-05-08T18:05:10.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.319+0000 7f0bfd7fa640 1 --2- 172.21.15.50:0/332619826 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0bf003bba0 0x7f0bf003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.319+0000 7f0bfd7fa640 1 -- 172.21.15.50:0/332619826 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f0d08038d20 con 0x5586c5342c90 2024-05-08T18:05:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.319+0000 7f0d172bd640 1 --2- 172.21.15.50:0/332619826 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0bf003bba0 0x7f0bf003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.319+0000 7f0d1bd2d800 1 Processor -- start 2024-05-08T18:05:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.319+0000 7f0d1bd2d800 1 -- start start 2024-05-08T18:05:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.320+0000 7f0d1bd2d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522b7e0 0x5586c5229ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.320+0000 7f0d1bd2d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5586c531d240 con 0x5586c522b7e0 2024-05-08T18:05:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.320+0000 7f0d172bd640 1 --2- 172.21.15.50:0/332619826 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0bf003bba0 0x7f0bf003e060 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x5586c532c4e0 tx=0x7f0d00005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.320+0000 7f0d17abe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522b7e0 0x5586c5229ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.320+0000 7f0d17abe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522b7e0 0x5586c5229ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38570/0 (socket says 172.21.15.50:38570) 2024-05-08T18:05:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.320+0000 7f0d17abe640 1 -- 172.21.15.50:0/2753020891 learned_addr learned my addr 172.21.15.50:0/2753020891 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.320+0000 7f0d17abe640 1 -- 172.21.15.50:0/2753020891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586c522bc90 con 0x5586c522b7e0 2024-05-08T18:05:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.320+0000 7f0d17abe640 1 --2- 172.21.15.50:0/2753020891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522b7e0 0x5586c5229ed0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0d080423f0 tx=0x7f0d0800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302605f1e00942a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.320+0000 7f0bf6ffd640 1 -- 172.21.15.50:0/2753020891 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d0803d040 con 0x5586c522b7e0 2024-05-08T18:05:10.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.320+0000 7f0bf6ffd640 1 -- 172.21.15.50:0/2753020891 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d0804a6c0 con 0x5586c522b7e0 2024-05-08T18:05:10.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.320+0000 7f0bf6ffd640 1 -- 172.21.15.50:0/2753020891 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d08042d50 con 0x5586c522b7e0 2024-05-08T18:05:10.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.321+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2753020891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522b7e0 msgr2=0x5586c5229ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.321+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/2753020891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522b7e0 0x5586c5229ed0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0d080423f0 tx=0x7f0d0800df80 comp rx=0 tx=0).stop 2024-05-08T18:05:10.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.321+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2753020891 shutdown_connections 2024-05-08T18:05:10.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.321+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/2753020891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522b7e0 0x5586c5229ed0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.321+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2753020891 >> 172.21.15.50:0/2753020891 conn(0x5586c534bf20 msgr2=0x5586c5349f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.321+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2753020891 shutdown_connections 2024-05-08T18:05:10.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.321+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2753020891 wait complete. 2024-05-08T18:05:10.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.321+0000 7f0d1bd2d800 1 Processor -- start 2024-05-08T18:05:10.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.321+0000 7f0d1bd2d800 1 -- start start 2024-05-08T18:05:10.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.321+0000 7f0d1bd2d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522b7e0 0x5586c5359110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.321+0000 7f0d1bd2d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5586c522a410 con 0x5586c522b7e0 2024-05-08T18:05:10.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.321+0000 7f0d17abe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522b7e0 0x5586c5359110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.322+0000 7f0d17abe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522b7e0 0x5586c5359110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38586/0 (socket says 172.21.15.50:38586) 2024-05-08T18:05:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.322+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 learned_addr learned my addr 172.21.15.50:0/42160731 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.322+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586c5359650 con 0x5586c522b7e0 2024-05-08T18:05:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.322+0000 7f0d17abe640 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522b7e0 0x5586c5359110 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0d080519d0 tx=0x7f0d080388a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.322+0000 7f0bf57fa640 1 -- 172.21.15.50:0/42160731 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d08016570 con 0x5586c522b7e0 2024-05-08T18:05:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.322+0000 7f0bf57fa640 1 -- 172.21.15.50:0/42160731 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d08037430 con 0x5586c522b7e0 2024-05-08T18:05:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.322+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5586c5359650 con 0x5586c522b7e0 2024-05-08T18:05:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.322+0000 7f0bf57fa640 1 -- 172.21.15.50:0/42160731 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d08016710 con 0x5586c522b7e0 2024-05-08T18:05:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.322+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586c5359650 con 0x5586c522b7e0 2024-05-08T18:05:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.323+0000 7f0bf57fa640 1 -- 172.21.15.50:0/42160731 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f0d08069070 con 0x5586c522b7e0 2024-05-08T18:05:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.323+0000 7f0bf57fa640 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0be003bba0 0x7f0be003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.323+0000 7f0bf57fa640 1 -- 172.21.15.50:0/42160731 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f0d0805a070 con 0x5586c522b7e0 2024-05-08T18:05:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.323+0000 7f0d172bd640 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0be003bba0 0x7f0be003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.324+0000 7f0d172bd640 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0be003bba0 0x7f0be003e060 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0d00052fd0 tx=0x7f0d0004f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.324+0000 7f0d1bd2d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.324+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5586c5364710 0x5586c5366bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.324+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c5367320 con 0x5586c5364710 2024-05-08T18:05:10.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.324+0000 7f0d182bf640 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5586c5364710 0x5586c5366bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d182bf640 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5586c5364710 0x5586c5366bd0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c5364710 2024-05-08T18:05:10.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d1bd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d1bd2d800 20 realm 2024-05-08T18:05:10.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d1bd2d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c5367730 con 0x5586c5364710 2024-05-08T18:05:10.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c5364710 2024-05-08T18:05:10.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d1bd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d1bd2d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:10.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d1bd2d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c5368000 con 0x5586c5364710 2024-05-08T18:05:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c5364710 2024-05-08T18:05:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d1bd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d1bd2d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c53683a0 con 0x5586c5364710 2024-05-08T18:05:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c5364710 2024-05-08T18:05:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d1bd2d800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d1bd2d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5586c5369100 0x5586c536b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.325+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c536bb00 con 0x5586c5369100 2024-05-08T18:05:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.326+0000 7f0d17abe640 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5586c5369100 0x5586c536b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.326+0000 7f0d17abe640 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5586c5369100 0x5586c536b4f0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.326+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.326+0000 7f0d1bd2d800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:10.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.326+0000 7f0d1bd2d800 20 searching for the correct realm 2024-05-08T18:05:10.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.326+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5586c536dab0 0x5586c536ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.326+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5586c5370640 con 0x5586c536dab0 2024-05-08T18:05:10.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.326+0000 7f0d172bd640 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5586c536dab0 0x5586c536ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.327+0000 7f0d172bd640 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5586c536dab0 0x5586c536ff70 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.327+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d00081070 con 0x5586c536dab0 2024-05-08T18:05:10.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.327+0000 7f0d172bd640 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f0d00076040 0x7f0d0005e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.327+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5586c5370640 con 0x7f0d00076040 2024-05-08T18:05:10.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.327+0000 7f0d182bf640 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f0d00076040 0x7f0d0005e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.327+0000 7f0d182bf640 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f0d00076040 0x7f0d0005e480 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.328+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d1000a4a0 con 0x7f0d00076040 2024-05-08T18:05:10.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.328+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5586c53683a0 con 0x5586c5369100 2024-05-08T18:05:10.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.328+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.328+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5586c53683a0 con 0x5586c5364710 2024-05-08T18:05:10.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.328+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c5364710 2024-05-08T18:05:10.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.328+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5586c53683a0 con 0x5586c5364710 2024-05-08T18:05:10.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.328+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c5364710 2024-05-08T18:05:10.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.328+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5586c53683a0 con 0x5586c5369100 2024-05-08T18:05:10.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.329+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.329+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5586c53683a0 con 0x5586c5369100 2024-05-08T18:05:10.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.329+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.329+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5586c53683a0 con 0x7f0d00076040 2024-05-08T18:05:10.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.329+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d1000a4a0 con 0x7f0d00076040 2024-05-08T18:05:10.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.329+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5586c53683a0 con 0x5586c536dab0 2024-05-08T18:05:10.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.329+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d00081070 con 0x5586c536dab0 2024-05-08T18:05:10.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.329+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5586c53683a0 con 0x5586c5364710 2024-05-08T18:05:10.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.330+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c5364710 2024-05-08T18:05:10.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.330+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5586c53683a0 con 0x5586c5369100 2024-05-08T18:05:10.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.330+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.330+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5586c53683a0 con 0x5586c5364710 2024-05-08T18:05:10.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.330+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c5364710 2024-05-08T18:05:10.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.330+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5586c53683a0 con 0x7f0d00076040 2024-05-08T18:05:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.330+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f0d1000a4a0 con 0x7f0d00076040 2024-05-08T18:05:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.330+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5586c53683a0 con 0x5586c536dab0 2024-05-08T18:05:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.331+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d00081070 con 0x5586c536dab0 2024-05-08T18:05:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.331+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x5586c536dab0 2024-05-08T18:05:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.331+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d00081070 con 0x5586c536dab0 2024-05-08T18:05:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.331+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x5586c5369100 2024-05-08T18:05:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.331+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.331+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x5586c536dab0 2024-05-08T18:05:10.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.331+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d00081070 con 0x5586c536dab0 2024-05-08T18:05:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.331+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x5586c536dab0 2024-05-08T18:05:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.332+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f0d00081070 con 0x5586c536dab0 2024-05-08T18:05:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.332+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x5586c5364710 2024-05-08T18:05:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.332+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c5364710 2024-05-08T18:05:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.332+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x5586c5364710 2024-05-08T18:05:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.332+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c5364710 2024-05-08T18:05:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.332+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x5586c5369100 2024-05-08T18:05:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.333+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.333+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x5586c536dab0 2024-05-08T18:05:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.333+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d00081070 con 0x5586c536dab0 2024-05-08T18:05:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.333+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x7f0d00076040 2024-05-08T18:05:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.333+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d1000a4a0 con 0x7f0d00076040 2024-05-08T18:05:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.333+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x5586c5364710 2024-05-08T18:05:10.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.333+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c5364710 2024-05-08T18:05:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.333+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x5586c5369100 2024-05-08T18:05:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.334+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.334+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x5586c536dab0 2024-05-08T18:05:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.334+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d00081070 con 0x5586c536dab0 2024-05-08T18:05:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.334+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x5586c5369100 2024-05-08T18:05:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.334+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.334+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x7f0d00076040 2024-05-08T18:05:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.335+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d1000a4a0 con 0x7f0d00076040 2024-05-08T18:05:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.335+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x5586c536dab0 2024-05-08T18:05:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.335+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d00081070 con 0x5586c536dab0 2024-05-08T18:05:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.335+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x5586c5369100 2024-05-08T18:05:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.335+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.335+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x5586c5364710 2024-05-08T18:05:10.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c5364710 2024-05-08T18:05:10.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f0d00090040 con 0x7f0d00076040 2024-05-08T18:05:10.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d1000a4a0 con 0x7f0d00076040 2024-05-08T18:05:10.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d1bd2d800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:10.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d1bd2d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:10.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d1bd2d800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:10.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d1bd2d800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:10.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d1bd2d800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:10.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d1bd2d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:10.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d1bd2d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c536c3c0 con 0x5586c5364710 2024-05-08T18:05:10.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c5364710 2024-05-08T18:05:10.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d1bd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d1bd2d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c536c3c0 con 0x5586c5364710 2024-05-08T18:05:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0d1000a4a0 con 0x5586c5364710 2024-05-08T18:05:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.336+0000 7f0d1bd2d800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d1bd2d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c536c3c0 con 0x5586c536dab0 2024-05-08T18:05:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f0d00081070 con 0x5586c536dab0 2024-05-08T18:05:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d1bd2d800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d1bd2d800 20 zone default found 2024-05-08T18:05:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d1bd2d800 4 Realm: () 2024-05-08T18:05:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d1bd2d800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d1bd2d800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d1bd2d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d1bd2d800 20 zonegroup default 2024-05-08T18:05:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d1bd2d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c536cd20 con 0x5586c5369100 2024-05-08T18:05:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d1bd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d1bd2d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c5372530 con 0x5586c536dab0 2024-05-08T18:05:10.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0d00081070 con 0x5586c536dab0 2024-05-08T18:05:10.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.337+0000 7f0d1bd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.338+0000 7f0d1bd2d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.338+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c5372a00 con 0x5586c536dab0 2024-05-08T18:05:10.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.338+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0d00081070 con 0x5586c536dab0 2024-05-08T18:05:10.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.338+0000 7f0d1bd2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.338+0000 7f0d1bd2d800 20 started sync module instance, tier type = 2024-05-08T18:05:10.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.338+0000 7f0d1bd2d800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:10.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.338+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c5371940 con 0x7f0d00076040 2024-05-08T18:05:10.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.339+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d1000a4a0 con 0x7f0d00076040 2024-05-08T18:05:10.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.339+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94037322702432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c5371940 con 0x7f0d00076040 2024-05-08T18:05:10.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.339+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c5373480 con 0x5586c5364710 2024-05-08T18:05:10.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.340+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94037322702432] v27'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d10040040 con 0x7f0d00076040 2024-05-08T18:05:10.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.340+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d10040040 con 0x5586c5364710 2024-05-08T18:05:10.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.340+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94037322707376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c5371940 con 0x5586c5364710 2024-05-08T18:05:10.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.340+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c5373f10 con 0x5586c536dab0 2024-05-08T18:05:10.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.341+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94037322707376] v27'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d10040040 con 0x5586c5364710 2024-05-08T18:05:10.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.341+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d00081070 con 0x5586c536dab0 2024-05-08T18:05:10.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.342+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94037322699920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c53718b0 con 0x5586c536dab0 2024-05-08T18:05:10.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.342+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c5374dd0 con 0x7f0d00076040 2024-05-08T18:05:10.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.343+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94037322699920] v27'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d00081070 con 0x5586c536dab0 2024-05-08T18:05:10.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.343+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d10040040 con 0x7f0d00076040 2024-05-08T18:05:10.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.343+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94037322713376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c53751f0 con 0x7f0d00076040 2024-05-08T18:05:10.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.343+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c5375bb0 con 0x5586c5364710 2024-05-08T18:05:10.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.344+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94037322713376] v27'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d10040040 con 0x7f0d00076040 2024-05-08T18:05:10.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.344+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d10040040 con 0x5586c5364710 2024-05-08T18:05:10.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.344+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94037322714912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c5375bb0 con 0x5586c5364710 2024-05-08T18:05:10.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.344+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c5376570 con 0x5586c5369100 2024-05-08T18:05:10.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.345+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94037322714912] v27'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d10040040 con 0x5586c5364710 2024-05-08T18:05:10.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.345+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.345+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94037322716928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c5375b20 con 0x5586c5369100 2024-05-08T18:05:10.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.345+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c53772d0 con 0x5586c5369100 2024-05-08T18:05:10.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.346+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94037322716928] v27'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.346+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.346+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94037322722064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c5377b20 con 0x5586c5369100 2024-05-08T18:05:10.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.346+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c53784e0 con 0x5586c5369100 2024-05-08T18:05:10.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.347+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94037322722064] v27'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.347+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.347+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94037322725488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c53784e0 con 0x5586c5369100 2024-05-08T18:05:10.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.347+0000 7f0d1bd2d800 20 add_watcher() i=0 2024-05-08T18:05:10.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.347+0000 7f0d1bd2d800 20 add_watcher() i=1 2024-05-08T18:05:10.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.347+0000 7f0d1bd2d800 20 add_watcher() i=2 2024-05-08T18:05:10.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.347+0000 7f0d1bd2d800 20 add_watcher() i=3 2024-05-08T18:05:10.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.347+0000 7f0d1bd2d800 20 add_watcher() i=4 2024-05-08T18:05:10.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.347+0000 7f0d1bd2d800 20 add_watcher() i=5 2024-05-08T18:05:10.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.347+0000 7f0d1bd2d800 20 add_watcher() i=6 2024-05-08T18:05:10.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.348+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94037322725488] v27'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.348+0000 7f0d1bd2d800 20 add_watcher() i=7 2024-05-08T18:05:10.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.348+0000 7f0d1bd2d800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:10.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.348+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c537b8e0 con 0x5586c5364710 2024-05-08T18:05:10.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.348+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f0d10040040 con 0x5586c5364710 2024-05-08T18:05:10.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.348+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94037322727936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c5378dd0 con 0x5586c5364710 2024-05-08T18:05:10.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.349+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94037322727936] v27'148 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0d10040040 con 0x5586c5364710 2024-05-08T18:05:10.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.349+0000 7f0bcdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:10.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.350+0000 7f0d1bd2d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:10.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.350+0000 7f0d1bd2d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:10.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.350+0000 7f0d1bd2d800 5 note: GC not initialized 2024-05-08T18:05:10.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.350+0000 7f0bccfe1640 20 reqs_thread_entry: start 2024-05-08T18:05:10.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.350+0000 7f0d1bd2d800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:10.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.350+0000 7f0d1bd2d800 20 Filter name: none 2024-05-08T18:05:10.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.350+0000 7f0b467fc640 20 reqs_thread_entry: start 2024-05-08T18:05:10.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.350+0000 7f0d1bd2d800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:10.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.350+0000 7f0d1bd2d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.350+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c53aede0 con 0x5586c5364710 2024-05-08T18:05:10.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f0d10040040 con 0x5586c5364710 2024-05-08T18:05:10.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d1bd2d800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:10.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d1bd2d800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:10.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d1bd2d800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:10.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d1bd2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:10.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c53b1870 con 0x7f0d00076040 2024-05-08T18:05:10.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv28 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f0d10040040 con 0x7f0d00076040 2024-05-08T18:05:10.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d1bd2d800 20 disabling bucket name=myfoo 2024-05-08T18:05:10.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d1bd2d800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-05-08T18:05:10.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d1bd2d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c53af850 con 0x7f0d00076040 2024-05-08T18:05:10.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(64 myfoo [read 0~165 out=165b] v0'0 uv29 ondisk = 0) ==== 149+0+165 (crc 0 0 0) 0x7f0d10040040 con 0x7f0d00076040 2024-05-08T18:05:10.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d1bd2d800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:10.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d1bd2d800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2024-05-08T18:05:10.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d1bd2d800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:10.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d1bd2d800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 : miss 2024-05-08T18:05:10.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d1bd2d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.351+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:65 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c53b2100 con 0x5586c5369100 2024-05-08T18:05:10.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.352+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(65 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv5 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.352+0000 7f0d1bd2d800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:10.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.352+0000 7f0d1bd2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:10.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.352+0000 7f0d1bd2d800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:10.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.352+0000 7f0d1bd2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:10.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.352+0000 7f0d1bd2d800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:10.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.352+0000 7f0d1bd2d800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 2024-05-08T18:05:10.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.352+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:66 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~379 in=379b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c53b9ae0 con 0x5586c5369100 2024-05-08T18:05:10.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.353+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(66 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [delete,create,call,call,writefull 0~379,setxattr (123)] v27'6 uv6 ondisk = 0) ==== 416+0+0 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.353+0000 7f0d1bd2d800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:10.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.353+0000 7f0d1bd2d800 10 moving default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:10.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.353+0000 7f0d1bd2d800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:10.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.353+0000 7f0d1bd2d800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1, ofs0, ns] 2024-05-08T18:05:10.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.353+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:67 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94037322963264 in=764b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c53b5190 con 0x5586c5369100 2024-05-08T18:05:10.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.354+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== watch-notify(notify (1) cookie 94037322722064 notify 115964117009 ret 0) ==== 794+0+0 (crc 0 0 0) 0x5586c53b1600 con 0x5586c5369100 2024-05-08T18:05:10.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.354+0000 7f0bf6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117009 cookie 94037322722064 notifier 4585 bl.length()=752 2024-05-08T18:05:10.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.354+0000 7f0bf6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:10.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.354+0000 7f0bf6ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:10.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.354+0000 7f0bf6ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:10.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.354+0000 7f0bf6ffd640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f0bdc003ea0 con 0x5586c5369100 2024-05-08T18:05:10.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.354+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(67 notify.6 [notify cookie 94037322963264 out=8b] v0'0 uv153 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.354+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(68 notify.6 [notify-ack] v0'0 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.354+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== watch-notify(notify_complete (2) cookie 94037322963264 notify 115964117009 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f0d08032040 con 0x5586c5369100 2024-05-08T18:05:10.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.354+0000 7f0d1bd2d800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): orig_sources= new_sources= 2024-05-08T18:05:10.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.354+0000 7f0d1bd2d800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): potential sources added=[] removed=[] 2024-05-08T18:05:10.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.354+0000 7f0d1bd2d800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): orig_dests= new_dests= 2024-05-08T18:05:10.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.354+0000 7f0d1bd2d800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): potential dests added=[] removed=[] 2024-05-08T18:05:10.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.354+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c53b4ca0 con 0x5586c5364710 2024-05-08T18:05:10.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.355+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(69 foo [delete,create,call,call,writefull 0~391] v27'12 uv12 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f0d10040040 con 0x5586c5364710 2024-05-08T18:05:10.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.355+0000 7f0d1bd2d800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:10.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.355+0000 7f0d1bd2d800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:10.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.355+0000 7f0d1bd2d800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-05-08T18:05:10.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.355+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify cookie 94037322967296 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5586c53b4ca0 con 0x5586c5364710 2024-05-08T18:05:10.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.355+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== watch-notify(notify (1) cookie 94037322714912 notify 115964117012 ret 0) ==== 609+0+0 (crc 0 0 0) 0x7f0d10038450 con 0x5586c5364710 2024-05-08T18:05:10.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.355+0000 7f0bf67fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117012 cookie 94037322714912 notifier 4585 bl.length()=567 2024-05-08T18:05:10.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.355+0000 7f0bf67fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:10.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.355+0000 7f0bf67fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:10.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.355+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(70 notify.4 [notify cookie 94037322967296 out=8b] v0'0 uv153 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f0d10040040 con 0x5586c5364710 2024-05-08T18:05:10.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.355+0000 7f0bf67fc640 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f0b4c003430 con 0x5586c5364710 2024-05-08T18:05:10.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.356+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== watch-notify(notify_complete (2) cookie 94037322967296 notify 115964117012 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f0d10038620 con 0x5586c5364710 2024-05-08T18:05:10.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.356+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(71 notify.4 [notify-ack] v0'0 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d10040040 con 0x5586c5364710 2024-05-08T18:05:10.367 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":1,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.357+0000 7f0bcdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.357+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94037322727936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c53b31c0 con 0x5586c5364710 2024-05-08T18:05:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.358+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(72 data_loggenerations_metadata [watch unwatch cookie 94037322727936] v27'149 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0d10040040 con 0x5586c5364710 2024-05-08T18:05:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.358+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:73 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94037322702432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c53781d0 con 0x7f0d00076040 2024-05-08T18:05:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.359+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(73 notify.0 [watch unwatch cookie 94037322702432] v27'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d10040040 con 0x7f0d00076040 2024-05-08T18:05:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.359+0000 7f0d1bd2d800 20 remove_watcher() i=0 2024-05-08T18:05:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.359+0000 7f0d1bd2d800 2 removed watcher, disabling cache 2024-05-08T18:05:10.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.359+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:74 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94037322707376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c5371e40 con 0x5586c5364710 2024-05-08T18:05:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.360+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(74 notify.1 [watch unwatch cookie 94037322707376] v27'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d10040040 con 0x5586c5364710 2024-05-08T18:05:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.360+0000 7f0d1bd2d800 20 remove_watcher() i=1 2024-05-08T18:05:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.360+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:75 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94037322699920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c53731b0 con 0x5586c536dab0 2024-05-08T18:05:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.361+0000 7f0d172bd640 1 -- 172.21.15.50:0/42160731 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(75 notify.2 [watch unwatch cookie 94037322699920] v27'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d00081070 con 0x5586c536dab0 2024-05-08T18:05:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.361+0000 7f0d1bd2d800 20 remove_watcher() i=2 2024-05-08T18:05:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.361+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:76 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94037322713376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c53731b0 con 0x7f0d00076040 2024-05-08T18:05:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.362+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(76 notify.3 [watch unwatch cookie 94037322713376] v27'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d10040040 con 0x7f0d00076040 2024-05-08T18:05:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.362+0000 7f0d1bd2d800 20 remove_watcher() i=3 2024-05-08T18:05:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.362+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94037322714912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c5374920 con 0x5586c5364710 2024-05-08T18:05:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.363+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(77 notify.4 [watch unwatch cookie 94037322714912] v27'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d10040040 con 0x5586c5364710 2024-05-08T18:05:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.363+0000 7f0d1bd2d800 20 remove_watcher() i=4 2024-05-08T18:05:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.363+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94037322716928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c53731b0 con 0x5586c5369100 2024-05-08T18:05:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.364+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(78 notify.5 [watch unwatch cookie 94037322716928] v27'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.364+0000 7f0d1bd2d800 20 remove_watcher() i=5 2024-05-08T18:05:10.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.364+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:79 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94037322722064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c5374f20 con 0x5586c5369100 2024-05-08T18:05:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.365+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(79 notify.6 [watch unwatch cookie 94037322722064] v27'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.365+0000 7f0d1bd2d800 20 remove_watcher() i=6 2024-05-08T18:05:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.365+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:80 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94037322725488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5586c5374920 con 0x5586c5369100 2024-05-08T18:05:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d17abe640 1 -- 172.21.15.50:0/42160731 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(80 notify.7 [watch unwatch cookie 94037322725488] v27'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d08072020 con 0x5586c5369100 2024-05-08T18:05:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 20 remove_watcher() i=7 2024-05-08T18:05:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5586c5369100 msgr2=0x5586c536b4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5586c5369100 0x5586c536b4f0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5586c5364710 msgr2=0x5586c5366bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5586c5364710 0x5586c5366bd0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f0d00076040 msgr2=0x7f0d0005e480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f0d00076040 0x7f0d0005e480 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5586c536dab0 msgr2=0x5586c536ff70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5586c536dab0 0x5586c536ff70 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0be003bba0 msgr2=0x7f0be003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0be003bba0 0x7f0be003e060 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0d00052fd0 tx=0x7f0d0004f000 comp rx=0 tx=0).stop 2024-05-08T18:05:10.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522b7e0 msgr2=0x5586c5359110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522b7e0 0x5586c5359110 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0d080519d0 tx=0x7f0d080388a0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d182bf640 1 -- 172.21.15.50:0/42160731 reap_dead start 2024-05-08T18:05:10.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 shutdown_connections 2024-05-08T18:05:10.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/42160731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522b7e0 0x5586c5359110 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 >> 172.21.15.50:0/42160731 conn(0x5586c534bf20 msgr2=0x5586c5349580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 shutdown_connections 2024-05-08T18:05:10.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.366+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/42160731 wait complete. 2024-05-08T18:05:10.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/332619826 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0bf003bba0 msgr2=0x7f0bf003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/332619826 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0bf003bba0 0x7f0bf003e060 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x5586c532c4e0 tx=0x7f0d00005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:10.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/332619826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c5342c90 msgr2=0x5586c53430b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/332619826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c5342c90 0x5586c53430b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0d0802cb00 tx=0x7f0d08043360 comp rx=0 tx=0).stop 2024-05-08T18:05:10.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/332619826 shutdown_connections 2024-05-08T18:05:10.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/332619826 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0bf003bba0 0x7f0bf003e060 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/332619826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c5342c90 0x5586c53430b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/332619826 >> 172.21.15.50:0/332619826 conn(0x5586c5336d30 msgr2=0x5586c5337130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/332619826 shutdown_connections 2024-05-08T18:05:10.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/332619826 wait complete. 2024-05-08T18:05:10.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2159951613 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5586c532df30 msgr2=0x5586c5330320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5586c532df30 0x5586c5330320 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2159951613 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5586c522fbf0 msgr2=0x5586c532b6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5586c522fbf0 0x5586c532b6a0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2159951613 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5586c5331e70 msgr2=0x5586c5334260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5586c5331e70 0x5586c5334260 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2159951613 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0c0003bba0 msgr2=0x7f0c0003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0c0003bba0 0x7f0c0003e060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0d00054000 tx=0x7f0d0004e000 comp rx=0 tx=0).stop 2024-05-08T18:05:10.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2159951613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522c0e0 msgr2=0x5586c5324460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 --2- 172.21.15.50:0/2159951613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5586c522c0e0 0x5586c5324460 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800b700 tx=0x7f0d080040e0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d182bf640 1 -- 172.21.15.50:0/2159951613 reap_dead start 2024-05-08T18:05:10.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2159951613 shutdown_connections 2024-05-08T18:05:10.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2159951613 >> 172.21.15.50:0/2159951613 conn(0x5586c5225b50 msgr2=0x5586c5316a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2159951613 shutdown_connections 2024-05-08T18:05:10.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.367+0000 7f0d1bd2d800 1 -- 172.21.15.50:0/2159951613 wait complete. 2024-05-08T18:05:10.378 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:10.380 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2024-05-08T18:05:10.380 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2024-05-08T18:05:10.380 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2024-05-08T18:05:10.425 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:10.425 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:10.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.427+0000 7f3f4a325800 1 Processor -- start 2024-05-08T18:05:10.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.427+0000 7f3f4a325800 1 -- start start 2024-05-08T18:05:10.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.427+0000 7f3f4a325800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56463d428af0 0x56463d4315f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.427+0000 7f3f4a325800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56463d39bff0 con 0x56463d426fb0 2024-05-08T18:05:10.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.427+0000 7f3f4a325800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x56463d2c8e20 con 0x56463d428af0 2024-05-08T18:05:10.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.428+0000 7f3f458af640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56463d428af0 0x56463d4315f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.428+0000 7f3f460b0640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x56463d426fb0 0x56463d4273d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:43300/0 (socket says 172.21.15.50:43300) 2024-05-08T18:05:10.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.428+0000 7f3f458af640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56463d428af0 0x56463d4315f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38594/0 (socket says 172.21.15.50:38594) 2024-05-08T18:05:10.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.428+0000 7f3f460b0640 1 -- 172.21.15.50:0/579390757 learned_addr learned my addr 172.21.15.50:0/579390757 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:10.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.428+0000 7f3f458af640 1 -- 172.21.15.50:0/579390757 >> v1:172.21.15.46:6789/0 conn(0x56463d426fb0 legacy=0x56463d4273d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.428+0000 7f3f458af640 1 -- 172.21.15.50:0/579390757 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463d39c1c0 con 0x56463d428af0 2024-05-08T18:05:10.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.430+0000 7f3f458af640 1 --2- 172.21.15.50:0/579390757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d428af0 0x56463d4315f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00a470 tx=0x7f3f3c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23322f31daa8a035 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.430+0000 7f3f450ae640 1 -- 172.21.15.50:0/579390757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f3c057070 con 0x56463d428af0 2024-05-08T18:05:10.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.430+0000 7f3f450ae640 1 -- 172.21.15.50:0/579390757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3c052570 con 0x56463d428af0 2024-05-08T18:05:10.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.430+0000 7f3f450ae640 1 -- 172.21.15.50:0/579390757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f3c052870 con 0x56463d428af0 2024-05-08T18:05:10.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.430+0000 7f3f4a325800 1 -- 172.21.15.50:0/579390757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d428af0 msgr2=0x56463d4315f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.430+0000 7f3f4a325800 1 --2- 172.21.15.50:0/579390757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d428af0 0x56463d4315f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00a470 tx=0x7f3f3c01ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:10.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.430+0000 7f3f4a325800 1 -- 172.21.15.50:0/579390757 shutdown_connections 2024-05-08T18:05:10.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.430+0000 7f3f4a325800 1 --2- 172.21.15.50:0/579390757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d428af0 0x56463d4315f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.430+0000 7f3f4a325800 1 -- 172.21.15.50:0/579390757 >> 172.21.15.50:0/579390757 conn(0x56463d422ad0 msgr2=0x56463d424f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.430+0000 7f3f4a325800 1 -- 172.21.15.50:0/579390757 shutdown_connections 2024-05-08T18:05:10.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.430+0000 7f3f4a325800 1 -- 172.21.15.50:0/579390757 wait complete. 2024-05-08T18:05:10.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.445+0000 7f3f4a325800 1 Processor -- start 2024-05-08T18:05:10.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.446+0000 7f3f4a325800 1 -- start start 2024-05-08T18:05:10.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.446+0000 7f3f4a325800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d426fb0 0x56463d51bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.446+0000 7f3f4a325800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56463d373a10 con 0x56463d426fb0 2024-05-08T18:05:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.446+0000 7f3f460b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d426fb0 0x56463d51bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.446+0000 7f3f460b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d426fb0 0x56463d51bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38600/0 (socket says 172.21.15.50:38600) 2024-05-08T18:05:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.446+0000 7f3f460b0640 1 -- 172.21.15.50:0/1807714360 learned_addr learned my addr 172.21.15.50:0/1807714360 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.446+0000 7f3f460b0640 1 -- 172.21.15.50:0/1807714360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463d51c2b0 con 0x56463d426fb0 2024-05-08T18:05:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.447+0000 7f3f460b0640 1 --2- 172.21.15.50:0/1807714360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d426fb0 0x56463d51bce0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3f38001dc0 tx=0x7f3f3800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f61d0acc041d01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.447+0000 7f3e34608640 1 -- 172.21.15.50:0/1807714360 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f38012020 con 0x56463d426fb0 2024-05-08T18:05:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.447+0000 7f3e34608640 1 -- 172.21.15.50:0/1807714360 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f3800e470 con 0x56463d426fb0 2024-05-08T18:05:10.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.447+0000 7f3e34608640 1 -- 172.21.15.50:0/1807714360 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f3800e7b0 con 0x56463d426fb0 2024-05-08T18:05:10.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.447+0000 7f3f4a325800 1 -- 172.21.15.50:0/1807714360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d426fb0 msgr2=0x56463d51bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.447+0000 7f3f4a325800 1 --2- 172.21.15.50:0/1807714360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d426fb0 0x56463d51bce0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3f38001dc0 tx=0x7f3f3800b6c0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.447+0000 7f3f4a325800 1 -- 172.21.15.50:0/1807714360 shutdown_connections 2024-05-08T18:05:10.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.447+0000 7f3f4a325800 1 --2- 172.21.15.50:0/1807714360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d426fb0 0x56463d51bce0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.447+0000 7f3f4a325800 1 -- 172.21.15.50:0/1807714360 >> 172.21.15.50:0/1807714360 conn(0x56463d422ad0 msgr2=0x56463d517600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.447+0000 7f3f4a325800 1 -- 172.21.15.50:0/1807714360 shutdown_connections 2024-05-08T18:05:10.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.448+0000 7f3f4a325800 1 -- 172.21.15.50:0/1807714360 wait complete. 2024-05-08T18:05:10.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.448+0000 7f3f4a325800 1 Processor -- start 2024-05-08T18:05:10.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.448+0000 7f3f4a325800 1 -- start start 2024-05-08T18:05:10.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.448+0000 7f3f4a325800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d426fb0 0x56463d5238e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.448+0000 7f3f4a325800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56463d431e00 con 0x56463d426fb0 2024-05-08T18:05:10.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.448+0000 7f3f460b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d426fb0 0x56463d5238e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.448+0000 7f3f460b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d426fb0 0x56463d5238e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38606/0 (socket says 172.21.15.50:38606) 2024-05-08T18:05:10.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.448+0000 7f3f460b0640 1 -- 172.21.15.50:0/1339901790 learned_addr learned my addr 172.21.15.50:0/1339901790 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.448+0000 7f3f460b0640 1 -- 172.21.15.50:0/1339901790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463d523e20 con 0x56463d426fb0 2024-05-08T18:05:10.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.449+0000 7f3f460b0640 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d426fb0 0x56463d5238e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800b700 tx=0x7f3f38000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.449+0000 7f3e32e05640 1 -- 172.21.15.50:0/1339901790 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f38012030 con 0x56463d426fb0 2024-05-08T18:05:10.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.449+0000 7f3e32e05640 1 -- 172.21.15.50:0/1339901790 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f38001420 con 0x56463d426fb0 2024-05-08T18:05:10.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.449+0000 7f3e32e05640 1 -- 172.21.15.50:0/1339901790 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f3801a7a0 con 0x56463d426fb0 2024-05-08T18:05:10.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.449+0000 7f3f4a325800 1 -- 172.21.15.50:0/1339901790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56463d5240b0 con 0x56463d426fb0 2024-05-08T18:05:10.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.449+0000 7f3f4a325800 1 -- 172.21.15.50:0/1339901790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56463d526d10 con 0x56463d426fb0 2024-05-08T18:05:10.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.450+0000 7f3e32e05640 1 -- 172.21.15.50:0/1339901790 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3f38015020 con 0x56463d426fb0 2024-05-08T18:05:10.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.450+0000 7f3e32e05640 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e2c03bba0 0x7f3e2c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.450+0000 7f3e32e05640 1 -- 172.21.15.50:0/1339901790 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f3f38020030 con 0x56463d426fb0 2024-05-08T18:05:10.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.450+0000 7f3f458af640 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e2c03bba0 0x7f3e2c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.450+0000 7f3f4a325800 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56463d428af0 0x56463d52aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.450+0000 7f3f4a325800 1 -- 172.21.15.50:0/1339901790 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d52b010 con 0x56463d428af0 2024-05-08T18:05:10.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.450+0000 7f3f468b1640 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56463d428af0 0x56463d52aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.451+0000 7f3f458af640 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e2c03bba0 0x7f3e2c03e060 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c054000 tx=0x7f3f3c04e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.451+0000 7f3f468b1640 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56463d428af0 0x56463d52aad0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.451+0000 7f3f468b1640 1 -- 172.21.15.50:0/1339901790 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d428af0 2024-05-08T18:05:10.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.451+0000 7f3f4a325800 1 -- 172.21.15.50:0/1339901790 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d528080 con 0x56463d428af0 2024-05-08T18:05:10.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.451+0000 7f3f468b1640 1 -- 172.21.15.50:0/1339901790 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d428af0 2024-05-08T18:05:10.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.451+0000 7f3f4a325800 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56463d52d330 0x56463d52f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.451+0000 7f3f4a325800 1 -- 172.21.15.50:0/1339901790 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d52fe10 con 0x56463d52d330 2024-05-08T18:05:10.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.452+0000 7f3f460b0640 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56463d52d330 0x56463d52f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.452+0000 7f3f460b0640 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56463d52d330 0x56463d52f720 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.452+0000 7f3f460b0640 1 -- 172.21.15.50:0/1339901790 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f3f380430c0 con 0x56463d52d330 2024-05-08T18:05:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.452+0000 7f3f4a325800 1 -- 172.21.15.50:0/1339901790 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d52c1d0 con 0x56463d428af0 2024-05-08T18:05:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.452+0000 7f3f468b1640 1 -- 172.21.15.50:0/1339901790 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d428af0 2024-05-08T18:05:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.453+0000 7f3f4a325800 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56463d531270 0x56463d533660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.453+0000 7f3f4a325800 1 -- 172.21.15.50:0/1339901790 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d533d60 con 0x56463d531270 2024-05-08T18:05:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.453+0000 7f3f458af640 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56463d531270 0x56463d533660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.453+0000 7f3f458af640 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56463d531270 0x56463d533660 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.453+0000 7f3f458af640 1 -- 172.21.15.50:0/1339901790 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3f3c052a60 con 0x56463d531270 2024-05-08T18:05:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.454+0000 7f3f4a325800 1 Processor -- start 2024-05-08T18:05:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.454+0000 7f3f4a325800 1 -- start start 2024-05-08T18:05:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.454+0000 7f3f4a325800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d542050 0x56463d542470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.454+0000 7f3f4a325800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56463d31a330 con 0x56463d542050 2024-05-08T18:05:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.454+0000 7f3f460b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d542050 0x56463d542470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.454+0000 7f3f460b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d542050 0x56463d542470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38618/0 (socket says 172.21.15.50:38618) 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.454+0000 7f3f460b0640 1 -- 172.21.15.50:0/3024885172 learned_addr learned my addr 172.21.15.50:0/3024885172 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.454+0000 7f3f460b0640 1 -- 172.21.15.50:0/3024885172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463d5429b0 con 0x56463d542050 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.454+0000 7f3f460b0640 1 --2- 172.21.15.50:0/3024885172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d542050 0x56463d542470 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800c4d0 tx=0x7f3f38018b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.454+0000 7f3e2bfff640 1 -- 172.21.15.50:0/3024885172 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f38049db0 con 0x56463d542050 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.455+0000 7f3e2bfff640 1 -- 172.21.15.50:0/3024885172 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f38049f50 con 0x56463d542050 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.455+0000 7f3e2bfff640 1 -- 172.21.15.50:0/3024885172 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f3804a270 con 0x56463d542050 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.455+0000 7f3f4a325800 1 -- 172.21.15.50:0/3024885172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56463d543420 con 0x56463d542050 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.455+0000 7f3f4a325800 1 -- 172.21.15.50:0/3024885172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56463d543920 con 0x56463d542050 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.455+0000 7f3e2bfff640 1 -- 172.21.15.50:0/3024885172 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3f3803d250 con 0x56463d542050 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.455+0000 7f3e2bfff640 1 --2- 172.21.15.50:0/3024885172 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e2003bba0 0x7f3e2003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.455+0000 7f3e2bfff640 1 -- 172.21.15.50:0/3024885172 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f3f3804fe80 con 0x56463d542050 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.456+0000 7f3f458af640 1 --2- 172.21.15.50:0/3024885172 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e2003bba0 0x7f3e2003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.456+0000 7f3f4a325800 1 Processor -- start 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.456+0000 7f3f4a325800 1 -- start start 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.456+0000 7f3f458af640 1 --2- 172.21.15.50:0/3024885172 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e2003bba0 0x7f3e2003e060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x56463d52b8e0 tx=0x7f3f3c005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.456+0000 7f3f4a325800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d54f150 0x56463d54f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.456+0000 7f3f4a325800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56463d51c630 con 0x56463d54f150 2024-05-08T18:05:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.456+0000 7f3f460b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d54f150 0x56463d54f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.456+0000 7f3f460b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d54f150 0x56463d54f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38626/0 (socket says 172.21.15.50:38626) 2024-05-08T18:05:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.456+0000 7f3f460b0640 1 -- 172.21.15.50:0/888121913 learned_addr learned my addr 172.21.15.50:0/888121913 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.456+0000 7f3f460b0640 1 -- 172.21.15.50:0/888121913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463d54fb40 con 0x56463d54f150 2024-05-08T18:05:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.457+0000 7f3f460b0640 1 --2- 172.21.15.50:0/888121913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d54f150 0x56463d54f570 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3f3804f530 tx=0x7f3f3804f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24d696e07567778 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.457+0000 7f3e29ffb640 1 -- 172.21.15.50:0/888121913 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f38062070 con 0x56463d54f150 2024-05-08T18:05:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.457+0000 7f3e29ffb640 1 -- 172.21.15.50:0/888121913 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f38037440 con 0x56463d54f150 2024-05-08T18:05:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.457+0000 7f3e29ffb640 1 -- 172.21.15.50:0/888121913 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f38059040 con 0x56463d54f150 2024-05-08T18:05:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.457+0000 7f3f4a325800 1 -- 172.21.15.50:0/888121913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d54f150 msgr2=0x56463d54f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.457+0000 7f3f4a325800 1 --2- 172.21.15.50:0/888121913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d54f150 0x56463d54f570 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3f3804f530 tx=0x7f3f3804f370 comp rx=0 tx=0).stop 2024-05-08T18:05:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.457+0000 7f3f4a325800 1 -- 172.21.15.50:0/888121913 shutdown_connections 2024-05-08T18:05:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.457+0000 7f3f4a325800 1 --2- 172.21.15.50:0/888121913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d54f150 0x56463d54f570 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.457+0000 7f3f4a325800 1 -- 172.21.15.50:0/888121913 >> 172.21.15.50:0/888121913 conn(0x56463d54b460 msgr2=0x56463d5494c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.457+0000 7f3f4a325800 1 -- 172.21.15.50:0/888121913 shutdown_connections 2024-05-08T18:05:10.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.457+0000 7f3f4a325800 1 -- 172.21.15.50:0/888121913 wait complete. 2024-05-08T18:05:10.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.458+0000 7f3f4a325800 1 Processor -- start 2024-05-08T18:05:10.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.458+0000 7f3f4a325800 1 -- start start 2024-05-08T18:05:10.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.458+0000 7f3f4a325800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d54f150 0x56463d5564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.458+0000 7f3f4a325800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56463d5501d0 con 0x56463d54f150 2024-05-08T18:05:10.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.458+0000 7f3f460b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d54f150 0x56463d5564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.458+0000 7f3f460b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d54f150 0x56463d5564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:38634/0 (socket says 172.21.15.50:38634) 2024-05-08T18:05:10.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.458+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 learned_addr learned my addr 172.21.15.50:0/740560256 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:10.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.458+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56463d556a20 con 0x56463d54f150 2024-05-08T18:05:10.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.458+0000 7f3f460b0640 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d54f150 0x56463d5564e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009ff0 tx=0x7f3f38009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.459+0000 7f3e17fff640 1 -- 172.21.15.50:0/740560256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f3806a070 con 0x56463d54f150 2024-05-08T18:05:10.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.459+0000 7f3e17fff640 1 -- 172.21.15.50:0/740560256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f380527d0 con 0x56463d54f150 2024-05-08T18:05:10.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.459+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56463d556a20 con 0x56463d54f150 2024-05-08T18:05:10.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.459+0000 7f3e17fff640 1 -- 172.21.15.50:0/740560256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f38062040 con 0x56463d54f150 2024-05-08T18:05:10.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.459+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56463d556a20 con 0x56463d54f150 2024-05-08T18:05:10.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.459+0000 7f3e17fff640 1 -- 172.21.15.50:0/740560256 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3f3806d070 con 0x56463d54f150 2024-05-08T18:05:10.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.460+0000 7f3e17fff640 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e1003bba0 0x7f3e1003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.460+0000 7f3e17fff640 1 -- 172.21.15.50:0/740560256 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f3f38070cb0 con 0x56463d54f150 2024-05-08T18:05:10.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.460+0000 7f3f458af640 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e1003bba0 0x7f3e1003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.460+0000 7f3f458af640 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e1003bba0 0x7f3e1003e060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c055040 tx=0x7f3f3c04f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.460+0000 7f3f4a325800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.460+0000 7f3f4a325800 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56463d561b90 0x56463d564050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.460+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d5647a0 con 0x56463d561b90 2024-05-08T18:05:10.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.461+0000 7f3f468b1640 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56463d561b90 0x56463d564050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.461+0000 7f3f468b1640 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56463d561b90 0x56463d564050 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.477 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.461+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d561b90 2024-05-08T18:05:10.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.461+0000 7f3f4a325800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.461+0000 7f3f4a325800 20 realm 2024-05-08T18:05:10.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.461+0000 7f3f4a325800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.461+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d564bb0 con 0x56463d561b90 2024-05-08T18:05:10.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.461+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d561b90 2024-05-08T18:05:10.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.461+0000 7f3f4a325800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.461+0000 7f3f4a325800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:10.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.461+0000 7f3f4a325800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.461+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d565480 con 0x56463d561b90 2024-05-08T18:05:10.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.462+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d561b90 2024-05-08T18:05:10.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.462+0000 7f3f4a325800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.462+0000 7f3f4a325800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.478 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.462+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d565820 con 0x56463d561b90 2024-05-08T18:05:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.462+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d561b90 2024-05-08T18:05:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.462+0000 7f3f4a325800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.462+0000 7f3f4a325800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.462+0000 7f3f4a325800 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56463d566580 0x56463d568970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.462+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d568f80 con 0x56463d566580 2024-05-08T18:05:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.462+0000 7f3f460b0640 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56463d566580 0x56463d568970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.462+0000 7f3f460b0640 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56463d566580 0x56463d568970 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.463+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.463+0000 7f3f4a325800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.463+0000 7f3f4a325800 20 searching for the correct realm 2024-05-08T18:05:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.463+0000 7f3f4a325800 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56463d56af30 0x56463d56d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.463+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56463d56dac0 con 0x56463d56af30 2024-05-08T18:05:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.463+0000 7f3f458af640 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56463d56af30 0x56463d56d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.463+0000 7f3f458af640 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56463d56af30 0x56463d56d3f0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.463+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f3c081070 con 0x56463d56af30 2024-05-08T18:05:10.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.463+0000 7f3f458af640 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3f3c076040 0x7f3f3c083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:10.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.464+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56463d56dac0 con 0x7f3f3c076040 2024-05-08T18:05:10.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.464+0000 7f3f468b1640 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3f3c076040 0x7f3f3c083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:10.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.464+0000 7f3f468b1640 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3f3c076040 0x7f3f3c083410 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:10.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.464+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f4000a4a0 con 0x7f3f3c076040 2024-05-08T18:05:10.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.464+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56463d565820 con 0x56463d566580 2024-05-08T18:05:10.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.464+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.464+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56463d565820 con 0x56463d561b90 2024-05-08T18:05:10.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.465+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d561b90 2024-05-08T18:05:10.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.465+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56463d565820 con 0x56463d561b90 2024-05-08T18:05:10.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.465+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d561b90 2024-05-08T18:05:10.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.465+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56463d565820 con 0x56463d566580 2024-05-08T18:05:10.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.465+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.465+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56463d565820 con 0x56463d566580 2024-05-08T18:05:10.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.465+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.465+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56463d565820 con 0x7f3f3c076040 2024-05-08T18:05:10.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.466+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f4000a4a0 con 0x7f3f3c076040 2024-05-08T18:05:10.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.466+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56463d565820 con 0x56463d56af30 2024-05-08T18:05:10.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.466+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f3c081070 con 0x56463d56af30 2024-05-08T18:05:10.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.466+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56463d565820 con 0x56463d561b90 2024-05-08T18:05:10.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.466+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d561b90 2024-05-08T18:05:10.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.466+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56463d565820 con 0x56463d566580 2024-05-08T18:05:10.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.466+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.466+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56463d565820 con 0x56463d561b90 2024-05-08T18:05:10.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.467+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d561b90 2024-05-08T18:05:10.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.467+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56463d565820 con 0x7f3f3c076040 2024-05-08T18:05:10.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.467+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f3f4000a4a0 con 0x7f3f3c076040 2024-05-08T18:05:10.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.467+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56463d565820 con 0x56463d56af30 2024-05-08T18:05:10.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.467+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f3c081070 con 0x56463d56af30 2024-05-08T18:05:10.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.467+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x56463d56af30 2024-05-08T18:05:10.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.467+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f3c081070 con 0x56463d56af30 2024-05-08T18:05:10.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.468+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x56463d566580 2024-05-08T18:05:10.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.468+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.468+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x56463d56af30 2024-05-08T18:05:10.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.468+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f3c081070 con 0x56463d56af30 2024-05-08T18:05:10.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.468+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x56463d56af30 2024-05-08T18:05:10.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.468+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f3f3c081070 con 0x56463d56af30 2024-05-08T18:05:10.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.468+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x56463d561b90 2024-05-08T18:05:10.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.469+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d561b90 2024-05-08T18:05:10.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.469+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x56463d561b90 2024-05-08T18:05:10.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.469+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d561b90 2024-05-08T18:05:10.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.469+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x56463d566580 2024-05-08T18:05:10.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.469+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.469+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x56463d56af30 2024-05-08T18:05:10.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.469+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f3c081070 con 0x56463d56af30 2024-05-08T18:05:10.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.469+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x7f3f3c076040 2024-05-08T18:05:10.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.470+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f4000a4a0 con 0x7f3f3c076040 2024-05-08T18:05:10.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.470+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x56463d561b90 2024-05-08T18:05:10.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.470+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d561b90 2024-05-08T18:05:10.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.470+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x56463d566580 2024-05-08T18:05:10.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.470+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.470+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x56463d56af30 2024-05-08T18:05:10.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.470+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f3c081070 con 0x56463d56af30 2024-05-08T18:05:10.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.470+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x56463d566580 2024-05-08T18:05:10.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.471+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.471+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x7f3f3c076040 2024-05-08T18:05:10.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.471+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f4000a4a0 con 0x7f3f3c076040 2024-05-08T18:05:10.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.471+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x56463d56af30 2024-05-08T18:05:10.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.471+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f3c081070 con 0x56463d56af30 2024-05-08T18:05:10.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.471+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x56463d566580 2024-05-08T18:05:10.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.471+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.471+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x56463d561b90 2024-05-08T18:05:10.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d561b90 2024-05-08T18:05:10.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f3f3c090040 con 0x7f3f3c076040 2024-05-08T18:05:10.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3f4000a4a0 con 0x7f3f3c076040 2024-05-08T18:05:10.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f4a325800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:10.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f4a325800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:10.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f4a325800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:10.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f4a325800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:10.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f4a325800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:10.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f4a325800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:10.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f4a325800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d569840 con 0x56463d561b90 2024-05-08T18:05:10.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d561b90 2024-05-08T18:05:10.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f4a325800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f4a325800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d569840 con 0x56463d561b90 2024-05-08T18:05:10.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3f4000a4a0 con 0x56463d561b90 2024-05-08T18:05:10.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f4a325800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:10.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f4a325800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.472+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d569840 con 0x56463d56af30 2024-05-08T18:05:10.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f3f3c081070 con 0x56463d56af30 2024-05-08T18:05:10.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f4a325800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:10.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f4a325800 20 zone default found 2024-05-08T18:05:10.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f4a325800 4 Realm: () 2024-05-08T18:05:10.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f4a325800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:10.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f4a325800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:10.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f4a325800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:10.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f4a325800 20 zonegroup default 2024-05-08T18:05:10.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f4a325800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d56a1e0 con 0x56463d566580 2024-05-08T18:05:10.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f4a325800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f4a325800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d56f980 con 0x56463d56af30 2024-05-08T18:05:10.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3f3c081070 con 0x56463d56af30 2024-05-08T18:05:10.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f4a325800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f4a325800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.473+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d56fde0 con 0x56463d56af30 2024-05-08T18:05:10.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.474+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3f3c081070 con 0x56463d56af30 2024-05-08T18:05:10.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.474+0000 7f3f4a325800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:10.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.474+0000 7f3f4a325800 20 started sync module instance, tier type = 2024-05-08T18:05:10.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.474+0000 7f3f4a325800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:10.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.474+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d56edc0 con 0x7f3f3c076040 2024-05-08T18:05:10.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.475+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f4000a4a0 con 0x7f3f3c076040 2024-05-08T18:05:10.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.475+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94859676807872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d56edc0 con 0x7f3f3c076040 2024-05-08T18:05:10.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.475+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d570970 con 0x56463d561b90 2024-05-08T18:05:10.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.476+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f40040040 con 0x56463d561b90 2024-05-08T18:05:10.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.476+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94859676807872] v27'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f40040040 con 0x7f3f3c076040 2024-05-08T18:05:10.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.476+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94859676812352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d570970 con 0x56463d561b90 2024-05-08T18:05:10.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.476+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d56e950 con 0x56463d56af30 2024-05-08T18:05:10.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.477+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94859676812352] v27'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f40040040 con 0x56463d561b90 2024-05-08T18:05:10.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.477+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f3c081070 con 0x56463d56af30 2024-05-08T18:05:10.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.477+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94859676814688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d571910 con 0x56463d56af30 2024-05-08T18:05:10.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.477+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d5722d0 con 0x7f3f3c076040 2024-05-08T18:05:10.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.478+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f40040040 con 0x7f3f3c076040 2024-05-08T18:05:10.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.478+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94859676814688] v27'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f3c081070 con 0x56463d56af30 2024-05-08T18:05:10.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.478+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94859676818976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d5726f0 con 0x7f3f3c076040 2024-05-08T18:05:10.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.478+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d5730b0 con 0x56463d561b90 2024-05-08T18:05:10.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.479+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94859676818976] v27'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f40040040 con 0x7f3f3c076040 2024-05-08T18:05:10.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.479+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f40040040 con 0x56463d561b90 2024-05-08T18:05:10.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.479+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94859676820512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d5730b0 con 0x56463d561b90 2024-05-08T18:05:10.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.479+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d573a70 con 0x56463d566580 2024-05-08T18:05:10.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.480+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94859676820512] v27'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f40040040 con 0x56463d561b90 2024-05-08T18:05:10.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.480+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.480+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94859676822528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d573020 con 0x56463d566580 2024-05-08T18:05:10.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.480+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d5747f0 con 0x56463d566580 2024-05-08T18:05:10.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.481+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94859676822528] v27'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.481+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.481+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94859676827632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d575040 con 0x56463d566580 2024-05-08T18:05:10.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.481+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d575a00 con 0x56463d566580 2024-05-08T18:05:10.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.482+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94859676827632] v27'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.483+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.483+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94859676833104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d575e20 con 0x56463d566580 2024-05-08T18:05:10.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.483+0000 7f3f4a325800 20 add_watcher() i=0 2024-05-08T18:05:10.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.483+0000 7f3f4a325800 20 add_watcher() i=1 2024-05-08T18:05:10.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.483+0000 7f3f4a325800 20 add_watcher() i=2 2024-05-08T18:05:10.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.483+0000 7f3f4a325800 20 add_watcher() i=3 2024-05-08T18:05:10.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.483+0000 7f3f4a325800 20 add_watcher() i=4 2024-05-08T18:05:10.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.483+0000 7f3f4a325800 20 add_watcher() i=5 2024-05-08T18:05:10.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.483+0000 7f3f4a325800 20 add_watcher() i=6 2024-05-08T18:05:10.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.483+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94859676833104] v27'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.483+0000 7f3f4a325800 20 add_watcher() i=7 2024-05-08T18:05:10.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.483+0000 7f3f4a325800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:10.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.483+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d579100 con 0x56463d561b90 2024-05-08T18:05:10.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.484+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f3f40040040 con 0x56463d561b90 2024-05-08T18:05:10.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.484+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94859676834528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d5763b0 con 0x56463d561b90 2024-05-08T18:05:10.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.485+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94859676834528] v27'150 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3f40040040 con 0x56463d561b90 2024-05-08T18:05:10.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.485+0000 7f3dfcfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:10.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.485+0000 7f3f4a325800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:10.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.485+0000 7f3f4a325800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:10.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.485+0000 7f3f4a325800 5 note: GC not initialized 2024-05-08T18:05:10.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.485+0000 7f3d777fe640 20 reqs_thread_entry: start 2024-05-08T18:05:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.485+0000 7f3f4a325800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.486+0000 7f3f4a325800 20 Filter name: none 2024-05-08T18:05:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.486+0000 7f3d757fa640 20 reqs_thread_entry: start 2024-05-08T18:05:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.486+0000 7f3f4a325800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.486+0000 7f3f4a325800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.486+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d5ac700 con 0x56463d561b90 2024-05-08T18:05:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.486+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv12 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f3f40040040 con 0x56463d561b90 2024-05-08T18:05:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.486+0000 7f3f4a325800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.486+0000 7f3f4a325800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.486+0000 7f3f4a325800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.486+0000 7f3f4a325800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.486+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d5af190 con 0x7f3f3c076040 2024-05-08T18:05:10.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.486+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv28 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f3f40040040 con 0x7f3f3c076040 2024-05-08T18:05:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 20 enabling bucket name=myfoo 2024-05-08T18:05:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-05-08T18:05:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d5ad170 con 0x7f3f3c076040 2024-05-08T18:05:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(64 myfoo [read 0~165 out=165b] v0'0 uv29 ondisk = 0) ==== 149+0+165 (crc 0 0 0) 0x7f3f40040040 con 0x7f3f3c076040 2024-05-08T18:05:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2024-05-08T18:05:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 : miss 2024-05-08T18:05:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 20 rados->read ofs=0 len=0 2024-05-08T18:05:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:65 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d5afa30 con 0x56463d566580 2024-05-08T18:05:10.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(65 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv6 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 2024-05-08T18:05:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.487+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:66 8.1e 8:7e0edeea:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~379 in=379b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d5b7240 con 0x56463d566580 2024-05-08T18:05:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(66 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 [delete,create,call,call,writefull 0~379,setxattr (123)] v27'7 uv7 ondisk = 0) ==== 416+0+0 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3f4a325800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3f4a325800 10 moving default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3f4a325800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3f4a325800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1, ofs0, ns] 2024-05-08T18:05:10.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:67 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94859677070000 in=764b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d5b28f0 con 0x56463d566580 2024-05-08T18:05:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== watch-notify(notify (1) cookie 94859676827632 notify 115964117010 ret 0) ==== 794+0+0 (crc 0 0 0) 0x56463d5aef70 con 0x56463d566580 2024-05-08T18:05:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3e29ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117010 cookie 94859676827632 notifier 4592 bl.length()=752 2024-05-08T18:05:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3e29ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 info.flags=0x17 2024-05-08T18:05:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(67 notify.6 [notify cookie 94859677070000 out=8b] v0'0 uv156 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3e29ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1 to cache LRU end 2024-05-08T18:05:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3e29ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3e29ffb640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f3e0c003ea0 con 0x56463d566580 2024-05-08T18:05:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(68 notify.6 [notify-ack] v0'0 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== watch-notify(notify_complete (2) cookie 94859677070000 notify 115964117010 ret 0) ==== 42+0+48 (crc 0 0 0) 0x56463d5429b0 con 0x56463d566580 2024-05-08T18:05:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3f4a325800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): orig_sources= new_sources= 2024-05-08T18:05:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3f4a325800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): potential sources added=[] removed=[] 2024-05-08T18:05:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3f4a325800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): orig_dests= new_dests= 2024-05-08T18:05:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3f4a325800 20 handle_bi_update(): bucket=:myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1]): potential dests added=[] removed=[] 2024-05-08T18:05:10.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.489+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d5af280 con 0x56463d561b90 2024-05-08T18:05:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.491+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(69 foo [delete,create,call,call,writefull 0~391] v27'13 uv13 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f3f40040040 con 0x56463d561b90 2024-05-08T18:05:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.491+0000 7f3f4a325800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.491+0000 7f3f4a325800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.491+0000 7f3f4a325800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-05-08T18:05:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.491+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify cookie 94859677069952 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56463d5b1810 con 0x56463d561b90 2024-05-08T18:05:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.491+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== watch-notify(notify (1) cookie 94859676820512 notify 115964117013 ret 0) ==== 609+0+0 (crc 0 0 0) 0x7f3f40018ca0 con 0x56463d561b90 2024-05-08T18:05:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.491+0000 7f3e29ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117013 cookie 94859676820512 notifier 4592 bl.length()=567 2024-05-08T18:05:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.491+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(70 notify.4 [notify cookie 94859677069952 out=8b] v0'0 uv156 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3f40040040 con 0x56463d561b90 2024-05-08T18:05:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.491+0000 7f3e29ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.491+0000 7f3e29ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.491+0000 7f3e29ffb640 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f3e0c003ea0 con 0x56463d561b90 2024-05-08T18:05:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.491+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(71 notify.4 [notify-ack] v0'0 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f40040040 con 0x56463d561b90 2024-05-08T18:05:10.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.491+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== watch-notify(notify_complete (2) cookie 94859677069952 notify 115964117013 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f3f40048040 con 0x56463d561b90 2024-05-08T18:05:10.504 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:10.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.492+0000 7f3dfcfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:10.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.492+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94859676834528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d578790 con 0x56463d561b90 2024-05-08T18:05:10.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.493+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(72 data_loggenerations_metadata [watch unwatch cookie 94859676834528] v27'151 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3f40040040 con 0x56463d561b90 2024-05-08T18:05:10.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.493+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:73 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94859676807872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d575ae0 con 0x7f3f3c076040 2024-05-08T18:05:10.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.494+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(73 notify.0 [watch unwatch cookie 94859676807872] v27'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f40040040 con 0x7f3f3c076040 2024-05-08T18:05:10.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.494+0000 7f3f4a325800 20 remove_watcher() i=0 2024-05-08T18:05:10.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.494+0000 7f3f4a325800 2 removed watcher, disabling cache 2024-05-08T18:05:10.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.494+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:74 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94859676812352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d575ae0 con 0x56463d561b90 2024-05-08T18:05:10.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.495+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(74 notify.1 [watch unwatch cookie 94859676812352] v27'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f40040040 con 0x56463d561b90 2024-05-08T18:05:10.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.495+0000 7f3f4a325800 20 remove_watcher() i=1 2024-05-08T18:05:10.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.495+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:75 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94859676814688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d575ae0 con 0x56463d56af30 2024-05-08T18:05:10.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.496+0000 7f3f458af640 1 -- 172.21.15.50:0/740560256 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(75 notify.2 [watch unwatch cookie 94859676814688] v27'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f3c081070 con 0x56463d56af30 2024-05-08T18:05:10.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.496+0000 7f3f4a325800 20 remove_watcher() i=2 2024-05-08T18:05:10.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.496+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:76 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94859676818976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d570d60 con 0x7f3f3c076040 2024-05-08T18:05:10.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.497+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(76 notify.3 [watch unwatch cookie 94859676818976] v27'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f40040040 con 0x7f3f3c076040 2024-05-08T18:05:10.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.497+0000 7f3f4a325800 20 remove_watcher() i=3 2024-05-08T18:05:10.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.497+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94859676820512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d575ae0 con 0x56463d561b90 2024-05-08T18:05:10.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.498+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(77 notify.4 [watch unwatch cookie 94859676820512] v27'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f40040040 con 0x56463d561b90 2024-05-08T18:05:10.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.498+0000 7f3f4a325800 20 remove_watcher() i=4 2024-05-08T18:05:10.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.498+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94859676822528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d571e20 con 0x56463d566580 2024-05-08T18:05:10.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.499+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(78 notify.5 [watch unwatch cookie 94859676822528] v27'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.499+0000 7f3f4a325800 20 remove_watcher() i=5 2024-05-08T18:05:10.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.499+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:79 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94859676827632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d572420 con 0x56463d566580 2024-05-08T18:05:10.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.500+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(79 notify.6 [watch unwatch cookie 94859676827632] v27'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.500+0000 7f3f4a325800 20 remove_watcher() i=6 2024-05-08T18:05:10.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.500+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:80 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94859676833104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56463d575ae0 con 0x56463d566580 2024-05-08T18:05:10.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f460b0640 1 -- 172.21.15.50:0/740560256 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(80 notify.7 [watch unwatch cookie 94859676833104] v27'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3f38076070 con 0x56463d566580 2024-05-08T18:05:10.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 20 remove_watcher() i=7 2024-05-08T18:05:10.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56463d566580 msgr2=0x56463d568970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56463d566580 0x56463d568970 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56463d561b90 msgr2=0x56463d564050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56463d561b90 0x56463d564050 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3f3c076040 msgr2=0x7f3f3c083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3f3c076040 0x7f3f3c083410 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56463d56af30 msgr2=0x56463d56d3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56463d56af30 0x56463d56d3f0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e1003bba0 msgr2=0x7f3e1003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e1003bba0 0x7f3e1003e060 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c055040 tx=0x7f3f3c04f000 comp rx=0 tx=0).stop 2024-05-08T18:05:10.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d54f150 msgr2=0x56463d5564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 1 --2- 172.21.15.50:0/740560256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d54f150 0x56463d5564e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3f38009ff0 tx=0x7f3f38009f80 comp rx=0 tx=0).stop 2024-05-08T18:05:10.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 reap_dead start 2024-05-08T18:05:10.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f468b1640 1 -- 172.21.15.50:0/740560256 reap_dead start 2024-05-08T18:05:10.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 shutdown_connections 2024-05-08T18:05:10.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 >> 172.21.15.50:0/740560256 conn(0x56463d54b460 msgr2=0x56463d548ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 shutdown_connections 2024-05-08T18:05:10.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.501+0000 7f3f4a325800 1 -- 172.21.15.50:0/740560256 wait complete. 2024-05-08T18:05:10.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 -- 172.21.15.50:0/3024885172 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e2003bba0 msgr2=0x7f3e2003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 --2- 172.21.15.50:0/3024885172 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e2003bba0 0x7f3e2003e060 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x56463d52b8e0 tx=0x7f3f3c005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:10.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 -- 172.21.15.50:0/3024885172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d542050 msgr2=0x56463d542470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 --2- 172.21.15.50:0/3024885172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d542050 0x56463d542470 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800c4d0 tx=0x7f3f38018b80 comp rx=0 tx=0).stop 2024-05-08T18:05:10.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 -- 172.21.15.50:0/3024885172 shutdown_connections 2024-05-08T18:05:10.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 --2- 172.21.15.50:0/3024885172 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e2003bba0 0x7f3e2003e060 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 --2- 172.21.15.50:0/3024885172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d542050 0x56463d542470 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 -- 172.21.15.50:0/3024885172 >> 172.21.15.50:0/3024885172 conn(0x56463d5360f0 msgr2=0x56463d5364f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 -- 172.21.15.50:0/3024885172 shutdown_connections 2024-05-08T18:05:10.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 -- 172.21.15.50:0/3024885172 wait complete. 2024-05-08T18:05:10.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 -- 172.21.15.50:0/1339901790 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56463d52d330 msgr2=0x56463d52f720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56463d52d330 0x56463d52f720 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 -- 172.21.15.50:0/1339901790 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56463d428af0 msgr2=0x56463d52aad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56463d428af0 0x56463d52aad0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 -- 172.21.15.50:0/1339901790 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56463d531270 msgr2=0x56463d533660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56463d531270 0x56463d533660 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 -- 172.21.15.50:0/1339901790 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e2c03bba0 msgr2=0x7f3e2c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3e2c03bba0 0x7f3e2c03e060 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c054000 tx=0x7f3f3c04e000 comp rx=0 tx=0).stop 2024-05-08T18:05:10.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 -- 172.21.15.50:0/1339901790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d426fb0 msgr2=0x56463d5238e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:10.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 --2- 172.21.15.50:0/1339901790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56463d426fb0 0x56463d5238e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3f3800b700 tx=0x7f3f38000bc0 comp rx=0 tx=0).stop 2024-05-08T18:05:10.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f468b1640 1 -- 172.21.15.50:0/1339901790 reap_dead start 2024-05-08T18:05:10.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 -- 172.21.15.50:0/1339901790 shutdown_connections 2024-05-08T18:05:10.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 -- 172.21.15.50:0/1339901790 >> 172.21.15.50:0/1339901790 conn(0x56463d422ad0 msgr2=0x56463d515e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:10.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 -- 172.21.15.50:0/1339901790 shutdown_connections 2024-05-08T18:05:10.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:10.502+0000 7f3f4a325800 1 -- 172.21.15.50:0/1339901790 wait complete. 2024-05-08T18:05:10.516 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:10.523 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-05-08T18:05:10.759 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=30000000 bytes_in=0 failed=False 2024-05-08T18:05:10.767 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:25.768 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2024-05-08T18:05:25.768 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2024-05-08T18:05:25.769 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2024-05-08T18:05:25.823 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:25.823 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:25.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.826+0000 7f5c03610800 1 Processor -- start 2024-05-08T18:05:25.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.826+0000 7f5c03610800 1 -- start start 2024-05-08T18:05:25.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.826+0000 7f5c03610800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x564af45eac90 0x564af45ed080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:25.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.826+0000 7f5c03610800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x564af45ed5c0 con 0x564af45e7180 2024-05-08T18:05:25.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.826+0000 7f5c03610800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x564af4486e20 con 0x564af45eac90 2024-05-08T18:05:25.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.827+0000 7f5bfe4f5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x564af45eac90 0x564af45ed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:25.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.827+0000 7f5bfecf6640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x564af45e7180 0x564af45e9570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:60336/0 (socket says 172.21.15.50:60336) 2024-05-08T18:05:25.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.827+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2044534701 learned_addr learned my addr 172.21.15.50:0/2044534701 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:25.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.827+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2044534701 >> v1:172.21.15.46:6789/0 conn(0x564af45e7180 legacy=0x564af45e9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.827+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2044534701 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af45ee7a0 con 0x564af45eac90 2024-05-08T18:05:25.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.829+0000 7f5bfe4f5640 1 --2- 172.21.15.50:0/2044534701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45eac90 0x564af45ed080 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400a470 tx=0x7f5bf401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a35f754f13c6e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:25.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.830+0000 7f5bfdcf4640 1 -- 172.21.15.50:0/2044534701 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf4057070 con 0x564af45eac90 2024-05-08T18:05:25.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.830+0000 7f5bfdcf4640 1 -- 172.21.15.50:0/2044534701 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf4052570 con 0x564af45eac90 2024-05-08T18:05:25.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.830+0000 7f5bfdcf4640 1 -- 172.21.15.50:0/2044534701 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf4052870 con 0x564af45eac90 2024-05-08T18:05:25.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.830+0000 7f5c03610800 1 -- 172.21.15.50:0/2044534701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45eac90 msgr2=0x564af45ed080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.830+0000 7f5c03610800 1 --2- 172.21.15.50:0/2044534701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45eac90 0x564af45ed080 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400a470 tx=0x7f5bf401ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:25.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.830+0000 7f5c03610800 1 -- 172.21.15.50:0/2044534701 shutdown_connections 2024-05-08T18:05:25.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.830+0000 7f5c03610800 1 --2- 172.21.15.50:0/2044534701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45eac90 0x564af45ed080 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:25.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.830+0000 7f5c03610800 1 -- 172.21.15.50:0/2044534701 >> 172.21.15.50:0/2044534701 conn(0x564af45e0ad0 msgr2=0x564af45e2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:25.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.830+0000 7f5c03610800 1 -- 172.21.15.50:0/2044534701 shutdown_connections 2024-05-08T18:05:25.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.830+0000 7f5c03610800 1 -- 172.21.15.50:0/2044534701 wait complete. 2024-05-08T18:05:25.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.850+0000 7f5c03610800 1 Processor -- start 2024-05-08T18:05:25.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.850+0000 7f5c03610800 1 -- start start 2024-05-08T18:05:25.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.850+0000 7f5c03610800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e7180 0x564af46d7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:25.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.850+0000 7f5c03610800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564af4531a10 con 0x564af45e7180 2024-05-08T18:05:25.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.850+0000 7f5bfecf6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e7180 0x564af46d7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:25.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.850+0000 7f5bfecf6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e7180 0x564af46d7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43600/0 (socket says 172.21.15.50:43600) 2024-05-08T18:05:25.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.850+0000 7f5bfecf6640 1 -- 172.21.15.50:0/4245189224 learned_addr learned my addr 172.21.15.50:0/4245189224 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:25.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.851+0000 7f5bfecf6640 1 -- 172.21.15.50:0/4245189224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af46d8130 con 0x564af45e7180 2024-05-08T18:05:25.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.851+0000 7f5bfecf6640 1 --2- 172.21.15.50:0/4245189224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e7180 0x564af46d7b60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0001e10 tx=0x7f5bf000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478c553efe313133 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:25.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.851+0000 7f5aece09640 1 -- 172.21.15.50:0/4245189224 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf0012020 con 0x564af45e7180 2024-05-08T18:05:25.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.851+0000 7f5aece09640 1 -- 172.21.15.50:0/4245189224 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf000e5a0 con 0x564af45e7180 2024-05-08T18:05:25.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.851+0000 7f5aece09640 1 -- 172.21.15.50:0/4245189224 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf000e8a0 con 0x564af45e7180 2024-05-08T18:05:25.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.852+0000 7f5c03610800 1 -- 172.21.15.50:0/4245189224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e7180 msgr2=0x564af46d7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.852+0000 7f5c03610800 1 --2- 172.21.15.50:0/4245189224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e7180 0x564af46d7b60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0001e10 tx=0x7f5bf000b780 comp rx=0 tx=0).stop 2024-05-08T18:05:25.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.852+0000 7f5c03610800 1 -- 172.21.15.50:0/4245189224 shutdown_connections 2024-05-08T18:05:25.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.852+0000 7f5c03610800 1 --2- 172.21.15.50:0/4245189224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e7180 0x564af46d7b60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:25.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.852+0000 7f5c03610800 1 -- 172.21.15.50:0/4245189224 >> 172.21.15.50:0/4245189224 conn(0x564af45e0ad0 msgr2=0x564af46d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:25.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.852+0000 7f5c03610800 1 -- 172.21.15.50:0/4245189224 shutdown_connections 2024-05-08T18:05:25.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.852+0000 7f5c03610800 1 -- 172.21.15.50:0/4245189224 wait complete. 2024-05-08T18:05:25.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.852+0000 7f5c03610800 1 Processor -- start 2024-05-08T18:05:25.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.852+0000 7f5c03610800 1 -- start start 2024-05-08T18:05:25.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.853+0000 7f5c03610800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e7180 0x564af46df700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:25.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.853+0000 7f5c03610800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564af45eda90 con 0x564af45e7180 2024-05-08T18:05:25.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.853+0000 7f5bfecf6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e7180 0x564af46df700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:25.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.853+0000 7f5bfecf6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e7180 0x564af46df700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43612/0 (socket says 172.21.15.50:43612) 2024-05-08T18:05:25.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.853+0000 7f5bfecf6640 1 -- 172.21.15.50:0/1211371285 learned_addr learned my addr 172.21.15.50:0/1211371285 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:25.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.853+0000 7f5bfecf6640 1 -- 172.21.15.50:0/1211371285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af46dfc40 con 0x564af45e7180 2024-05-08T18:05:25.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.853+0000 7f5bfecf6640 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e7180 0x564af46df700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x564af46d88f0 tx=0x7f5bf0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:25.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.853+0000 7f5aeb606640 1 -- 172.21.15.50:0/1211371285 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf0012030 con 0x564af45e7180 2024-05-08T18:05:25.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.853+0000 7f5aeb606640 1 -- 172.21.15.50:0/1211371285 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf0001470 con 0x564af45e7180 2024-05-08T18:05:25.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.854+0000 7f5c03610800 1 -- 172.21.15.50:0/1211371285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564af46dfed0 con 0x564af45e7180 2024-05-08T18:05:25.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.854+0000 7f5aeb606640 1 -- 172.21.15.50:0/1211371285 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf001a7a0 con 0x564af45e7180 2024-05-08T18:05:25.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.854+0000 7f5c03610800 1 -- 172.21.15.50:0/1211371285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564af46e2b80 con 0x564af45e7180 2024-05-08T18:05:25.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.855+0000 7f5aeb606640 1 -- 172.21.15.50:0/1211371285 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f5bf0015020 con 0x564af45e7180 2024-05-08T18:05:25.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.855+0000 7f5aeb606640 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f5ae403bba0 0x7f5ae403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:25.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.855+0000 7f5aeb606640 1 -- 172.21.15.50:0/1211371285 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f5bf0020030 con 0x564af45e7180 2024-05-08T18:05:25.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.855+0000 7f5bfe4f5640 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f5ae403bba0 0x7f5ae403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:25.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.855+0000 7f5c03610800 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564af45eac90 0x564af46e6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:25.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.855+0000 7f5c03610800 1 -- 172.21.15.50:0/1211371285 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af46e6e80 con 0x564af45eac90 2024-05-08T18:05:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.855+0000 7f5bff4f7640 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564af45eac90 0x564af46e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.855+0000 7f5bfe4f5640 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f5ae403bba0 0x7f5ae403e060 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4054000 tx=0x7f5bf404e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.856+0000 7f5bff4f7640 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564af45eac90 0x564af46e6940 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.856+0000 7f5bff4f7640 1 -- 172.21.15.50:0/1211371285 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af45eac90 2024-05-08T18:05:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.856+0000 7f5c03610800 1 -- 172.21.15.50:0/1211371285 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af46e3ef0 con 0x564af45eac90 2024-05-08T18:05:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.856+0000 7f5bff4f7640 1 -- 172.21.15.50:0/1211371285 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af45eac90 2024-05-08T18:05:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.856+0000 7f5c03610800 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564af46e91a0 0x564af46eb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.856+0000 7f5c03610800 1 -- 172.21.15.50:0/1211371285 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af46ebc80 con 0x564af46e91a0 2024-05-08T18:05:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.857+0000 7f5bfecf6640 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564af46e91a0 0x564af46eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.857+0000 7f5bfecf6640 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564af46e91a0 0x564af46eb590 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.857+0000 7f5bfecf6640 1 -- 172.21.15.50:0/1211371285 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f5bf00430c0 con 0x564af46e91a0 2024-05-08T18:05:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.857+0000 7f5c03610800 1 -- 172.21.15.50:0/1211371285 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af46e8040 con 0x564af45eac90 2024-05-08T18:05:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.858+0000 7f5bff4f7640 1 -- 172.21.15.50:0/1211371285 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af45eac90 2024-05-08T18:05:25.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.858+0000 7f5c03610800 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564af46ed0e0 0x564af46ef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:25.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.858+0000 7f5c03610800 1 -- 172.21.15.50:0/1211371285 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af46efbd0 con 0x564af46ed0e0 2024-05-08T18:05:25.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.858+0000 7f5bfe4f5640 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564af46ed0e0 0x564af46ef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:25.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.858+0000 7f5bfe4f5640 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564af46ed0e0 0x564af46ef4d0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:25.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.859+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/1211371285 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f5bf4052a40 con 0x564af46ed0e0 2024-05-08T18:05:25.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.859+0000 7f5c03610800 1 Processor -- start 2024-05-08T18:05:25.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.859+0000 7f5c03610800 1 -- start start 2024-05-08T18:05:25.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.859+0000 7f5c03610800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e6d20 0x564af45e5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:25.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.859+0000 7f5c03610800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564af44d8330 con 0x564af45e6d20 2024-05-08T18:05:25.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.859+0000 7f5bfecf6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e6d20 0x564af45e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:25.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.859+0000 7f5bfecf6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e6d20 0x564af45e5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43624/0 (socket says 172.21.15.50:43624) 2024-05-08T18:05:25.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.860+0000 7f5bfecf6640 1 -- 172.21.15.50:0/3364257535 learned_addr learned my addr 172.21.15.50:0/3364257535 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:25.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.860+0000 7f5bfecf6640 1 -- 172.21.15.50:0/3364257535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af45e56d0 con 0x564af45e6d20 2024-05-08T18:05:25.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.860+0000 7f5bfecf6640 1 --2- 172.21.15.50:0/3364257535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e6d20 0x564af45e5190 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0050f60 tx=0x7f5bf004a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:25.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.860+0000 7f5ae8e01640 1 -- 172.21.15.50:0/3364257535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf000cd70 con 0x564af45e6d20 2024-05-08T18:05:25.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.860+0000 7f5ae8e01640 1 -- 172.21.15.50:0/3364257535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf0048070 con 0x564af45e6d20 2024-05-08T18:05:25.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.860+0000 7f5ae8e01640 1 -- 172.21.15.50:0/3364257535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf0016500 con 0x564af45e6d20 2024-05-08T18:05:25.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.860+0000 7f5c03610800 1 -- 172.21.15.50:0/3364257535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564af45e5960 con 0x564af45e6d20 2024-05-08T18:05:25.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.860+0000 7f5c03610800 1 -- 172.21.15.50:0/3364257535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564af45e5e60 con 0x564af45e6d20 2024-05-08T18:05:25.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.861+0000 7f5ae8e01640 1 -- 172.21.15.50:0/3364257535 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f5bf00168b0 con 0x564af45e6d20 2024-05-08T18:05:25.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.861+0000 7f5ae8e01640 1 --2- 172.21.15.50:0/3364257535 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f5ad803bba0 0x7f5ad803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:25.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.861+0000 7f5ae8e01640 1 -- 172.21.15.50:0/3364257535 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f5bf00030d0 con 0x564af45e6d20 2024-05-08T18:05:25.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.861+0000 7f5bfe4f5640 1 --2- 172.21.15.50:0/3364257535 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f5ad803bba0 0x7f5ad803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:25.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.862+0000 7f5bfe4f5640 1 --2- 172.21.15.50:0/3364257535 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f5ad803bba0 0x7f5ad803e060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x564af46e7750 tx=0x7f5bf4005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:25.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.862+0000 7f5c03610800 1 Processor -- start 2024-05-08T18:05:25.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.862+0000 7f5c03610800 1 -- start start 2024-05-08T18:05:25.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.862+0000 7f5c03610800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af470d2a0 0x564af470d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:25.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.862+0000 7f5c03610800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564af46d84b0 con 0x564af470d2a0 2024-05-08T18:05:25.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.863+0000 7f5bfecf6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af470d2a0 0x564af470d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:25.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.863+0000 7f5bfecf6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af470d2a0 0x564af470d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43630/0 (socket says 172.21.15.50:43630) 2024-05-08T18:05:25.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.863+0000 7f5bfecf6640 1 -- 172.21.15.50:0/3416771304 learned_addr learned my addr 172.21.15.50:0/3416771304 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:25.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.863+0000 7f5bfecf6640 1 -- 172.21.15.50:0/3416771304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af470dc90 con 0x564af470d2a0 2024-05-08T18:05:25.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.863+0000 7f5bfecf6640 1 --2- 172.21.15.50:0/3416771304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af470d2a0 0x564af470d6c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5bf001afd0 tx=0x7f5bf00245e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b71a56ef6ae8b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:25.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.863+0000 7f5ae27fc640 1 -- 172.21.15.50:0/3416771304 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf0044440 con 0x564af470d2a0 2024-05-08T18:05:25.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.863+0000 7f5ae27fc640 1 -- 172.21.15.50:0/3416771304 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf004abd0 con 0x564af470d2a0 2024-05-08T18:05:25.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.863+0000 7f5ae27fc640 1 -- 172.21.15.50:0/3416771304 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf00445e0 con 0x564af470d2a0 2024-05-08T18:05:25.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.864+0000 7f5c03610800 1 -- 172.21.15.50:0/3416771304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af470d2a0 msgr2=0x564af470d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.864+0000 7f5c03610800 1 --2- 172.21.15.50:0/3416771304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af470d2a0 0x564af470d6c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5bf001afd0 tx=0x7f5bf00245e0 comp rx=0 tx=0).stop 2024-05-08T18:05:25.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.864+0000 7f5c03610800 1 -- 172.21.15.50:0/3416771304 shutdown_connections 2024-05-08T18:05:25.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.864+0000 7f5c03610800 1 --2- 172.21.15.50:0/3416771304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af470d2a0 0x564af470d6c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:25.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.864+0000 7f5c03610800 1 -- 172.21.15.50:0/3416771304 >> 172.21.15.50:0/3416771304 conn(0x564af4709570 msgr2=0x564af47075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:25.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.864+0000 7f5c03610800 1 -- 172.21.15.50:0/3416771304 shutdown_connections 2024-05-08T18:05:25.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.864+0000 7f5c03610800 1 -- 172.21.15.50:0/3416771304 wait complete. 2024-05-08T18:05:25.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.864+0000 7f5c03610800 1 Processor -- start 2024-05-08T18:05:25.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.864+0000 7f5c03610800 1 -- start start 2024-05-08T18:05:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.864+0000 7f5c03610800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af470d2a0 0x564af4714700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.864+0000 7f5c03610800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564af470e360 con 0x564af470d2a0 2024-05-08T18:05:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.864+0000 7f5bfecf6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af470d2a0 0x564af4714700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.864+0000 7f5bfecf6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af470d2a0 0x564af4714700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43636/0 (socket says 172.21.15.50:43636) 2024-05-08T18:05:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.864+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 learned_addr learned my addr 172.21.15.50:0/2058148234 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.865+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564af4714c40 con 0x564af470d2a0 2024-05-08T18:05:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.865+0000 7f5bfecf6640 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af470d2a0 0x564af4714700 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0024620 tx=0x7f5bf0007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.865+0000 7f5ae0ff9640 1 -- 172.21.15.50:0/2058148234 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf00523e0 con 0x564af470d2a0 2024-05-08T18:05:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.865+0000 7f5ae0ff9640 1 -- 172.21.15.50:0/2058148234 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bf004ad80 con 0x564af470d2a0 2024-05-08T18:05:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.865+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564af4714c40 con 0x564af470d2a0 2024-05-08T18:05:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.865+0000 7f5ae0ff9640 1 -- 172.21.15.50:0/2058148234 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bf0040360 con 0x564af470d2a0 2024-05-08T18:05:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.865+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564af47151a0 con 0x564af470d2a0 2024-05-08T18:05:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.866+0000 7f5ae0ff9640 1 -- 172.21.15.50:0/2058148234 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f5bf003e360 con 0x564af470d2a0 2024-05-08T18:05:25.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.867+0000 7f5ae0ff9640 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f5ac803bd40 0x7f5ac803e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:25.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.867+0000 7f5ae0ff9640 1 -- 172.21.15.50:0/2058148234 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f5bf0071c90 con 0x564af470d2a0 2024-05-08T18:05:25.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.867+0000 7f5bfe4f5640 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f5ac803bd40 0x7f5ac803e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:25.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.867+0000 7f5c03610800 20 rados->read ofs=0 len=0 2024-05-08T18:05:25.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.867+0000 7f5c03610800 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564af471fd60 0x564af4722220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:25.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.867+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af4722970 con 0x564af471fd60 2024-05-08T18:05:25.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.867+0000 7f5bff4f7640 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564af471fd60 0x564af4722220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:25.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.868+0000 7f5bfe4f5640 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f5ac803bd40 0x7f5ac803e200 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4052fd0 tx=0x7f5bf404f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:25.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.868+0000 7f5bff4f7640 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564af471fd60 0x564af4722220 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:25.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.868+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af471fd60 2024-05-08T18:05:25.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.868+0000 7f5c03610800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:25.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.868+0000 7f5c03610800 20 realm 2024-05-08T18:05:25.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.868+0000 7f5c03610800 20 rados->read ofs=0 len=0 2024-05-08T18:05:25.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.868+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af4722d80 con 0x564af471fd60 2024-05-08T18:05:25.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.868+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af471fd60 2024-05-08T18:05:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.868+0000 7f5c03610800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.868+0000 7f5c03610800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.868+0000 7f5c03610800 20 rados->read ofs=0 len=0 2024-05-08T18:05:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.868+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af4723650 con 0x564af471fd60 2024-05-08T18:05:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.869+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af471fd60 2024-05-08T18:05:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.869+0000 7f5c03610800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.869+0000 7f5c03610800 20 rados->read ofs=0 len=0 2024-05-08T18:05:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.869+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af47239f0 con 0x564af471fd60 2024-05-08T18:05:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.869+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af471fd60 2024-05-08T18:05:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.869+0000 7f5c03610800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.869+0000 7f5c03610800 20 rados->read ofs=0 len=0 2024-05-08T18:05:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.869+0000 7f5c03610800 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564af47244e0 0x564af47268d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.869+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af4727030 con 0x564af47244e0 2024-05-08T18:05:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.869+0000 7f5bfecf6640 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564af47244e0 0x564af47268d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:25.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.869+0000 7f5bfecf6640 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564af47244e0 0x564af47268d0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.870+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.870+0000 7f5c03610800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.870+0000 7f5c03610800 20 searching for the correct realm 2024-05-08T18:05:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.870+0000 7f5c03610800 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564af47290e0 0x564af472b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.870+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564af472bc70 con 0x564af47290e0 2024-05-08T18:05:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.870+0000 7f5bfe4f5640 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564af47290e0 0x564af472b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.870+0000 7f5bfe4f5640 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564af47290e0 0x564af472b5a0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.871+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.871+0000 7f5bfe4f5640 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f5bf4076040 0x7f5bf4083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.871+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564af472bc70 con 0x7f5bf4076040 2024-05-08T18:05:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.871+0000 7f5bff4f7640 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f5bf4076040 0x7f5bf4083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.871+0000 7f5bff4f7640 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f5bf4076040 0x7f5bf4083410 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.871+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf800a4a0 con 0x7f5bf4076040 2024-05-08T18:05:25.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.871+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564af47239f0 con 0x564af47244e0 2024-05-08T18:05:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.872+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.872+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564af47239f0 con 0x564af471fd60 2024-05-08T18:05:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.872+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af471fd60 2024-05-08T18:05:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.872+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564af47239f0 con 0x564af471fd60 2024-05-08T18:05:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.873+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af471fd60 2024-05-08T18:05:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.873+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564af47239f0 con 0x564af47244e0 2024-05-08T18:05:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.873+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.873+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564af47239f0 con 0x564af47244e0 2024-05-08T18:05:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.873+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.873+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564af47239f0 con 0x7f5bf4076040 2024-05-08T18:05:25.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.874+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf800a4a0 con 0x7f5bf4076040 2024-05-08T18:05:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.874+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564af47239f0 con 0x564af47290e0 2024-05-08T18:05:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.874+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.874+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564af47239f0 con 0x564af471fd60 2024-05-08T18:05:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.874+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af471fd60 2024-05-08T18:05:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.874+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564af47239f0 con 0x564af47244e0 2024-05-08T18:05:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.875+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.875+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564af47239f0 con 0x564af471fd60 2024-05-08T18:05:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.875+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af471fd60 2024-05-08T18:05:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.875+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564af47239f0 con 0x7f5bf4076040 2024-05-08T18:05:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.875+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f5bf800a4a0 con 0x7f5bf4076040 2024-05-08T18:05:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.875+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564af47239f0 con 0x564af47290e0 2024-05-08T18:05:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.875+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.875+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x564af47290e0 2024-05-08T18:05:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.876+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.876+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x564af47244e0 2024-05-08T18:05:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.876+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.876+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x564af47290e0 2024-05-08T18:05:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.876+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.876+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x564af47290e0 2024-05-08T18:05:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.877+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.877+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x564af471fd60 2024-05-08T18:05:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.877+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af471fd60 2024-05-08T18:05:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.877+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x564af471fd60 2024-05-08T18:05:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.877+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af471fd60 2024-05-08T18:05:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.877+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x564af47244e0 2024-05-08T18:05:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.877+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.877+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x564af47290e0 2024-05-08T18:05:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.878+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.878+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x7f5bf4076040 2024-05-08T18:05:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.878+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf800a4a0 con 0x7f5bf4076040 2024-05-08T18:05:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.878+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x564af471fd60 2024-05-08T18:05:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.878+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af471fd60 2024-05-08T18:05:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.878+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x564af47244e0 2024-05-08T18:05:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.879+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.879+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x564af47290e0 2024-05-08T18:05:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.879+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.879+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x564af47244e0 2024-05-08T18:05:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.880+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.880+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x7f5bf4076040 2024-05-08T18:05:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.880+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf800a4a0 con 0x7f5bf4076040 2024-05-08T18:05:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.880+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x564af47290e0 2024-05-08T18:05:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.880+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.880+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x564af47244e0 2024-05-08T18:05:25.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x564af471fd60 2024-05-08T18:05:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af471fd60 2024-05-08T18:05:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f5bf4090040 con 0x7f5bf4076040 2024-05-08T18:05:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5bf800a4a0 con 0x7f5bf4076040 2024-05-08T18:05:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5c03610800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5c03610800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5c03610800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5c03610800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5c03610800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5c03610800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5c03610800 20 rados->read ofs=0 len=0 2024-05-08T18:05:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af47279e0 con 0x564af471fd60 2024-05-08T18:05:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af471fd60 2024-05-08T18:05:25.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5c03610800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:25.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5c03610800 20 rados->read ofs=0 len=0 2024-05-08T18:05:25.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.881+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af47279e0 con 0x564af471fd60 2024-05-08T18:05:25.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.882+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5bf800a4a0 con 0x564af471fd60 2024-05-08T18:05:25.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.882+0000 7f5c03610800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:25.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.882+0000 7f5c03610800 20 rados->read ofs=0 len=0 2024-05-08T18:05:25.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.882+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af47279b0 con 0x564af47290e0 2024-05-08T18:05:25.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.882+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.882+0000 7f5c03610800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:25.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.882+0000 7f5c03610800 20 zone default found 2024-05-08T18:05:25.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.882+0000 7f5c03610800 4 Realm: () 2024-05-08T18:05:25.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.882+0000 7f5c03610800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:25.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.882+0000 7f5c03610800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:25.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.882+0000 7f5c03610800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:25.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.882+0000 7f5c03610800 20 zonegroup default 2024-05-08T18:05:25.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.882+0000 7f5c03610800 20 rados->read ofs=0 len=0 2024-05-08T18:05:25.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.882+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af4728280 con 0x564af47244e0 2024-05-08T18:05:25.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.883+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.883+0000 7f5c03610800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:25.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.883+0000 7f5c03610800 20 rados->read ofs=0 len=0 2024-05-08T18:05:25.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.883+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af472dad0 con 0x564af47290e0 2024-05-08T18:05:25.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.883+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.883+0000 7f5c03610800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:25.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.883+0000 7f5c03610800 20 rados->read ofs=0 len=0 2024-05-08T18:05:25.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.883+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af472de70 con 0x564af47290e0 2024-05-08T18:05:25.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.883+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.883+0000 7f5c03610800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:25.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.883+0000 7f5c03610800 20 started sync module instance, tier type = 2024-05-08T18:05:25.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.883+0000 7f5c03610800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:25.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.883+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af472cf70 con 0x7f5bf4076040 2024-05-08T18:05:25.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.884+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf800a4a0 con 0x7f5bf4076040 2024-05-08T18:05:25.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.884+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94879928734912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af472cf70 con 0x7f5bf4076040 2024-05-08T18:05:25.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.884+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af472e870 con 0x564af471fd60 2024-05-08T18:05:25.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.885+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94879928734912] v27'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x7f5bf4076040 2024-05-08T18:05:25.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.886+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.886+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94879928738688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af472cf70 con 0x564af471fd60 2024-05-08T18:05:25.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.886+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af472f2c0 con 0x564af47290e0 2024-05-08T18:05:25.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.887+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94879928738688] v27'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.887+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.887+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94879928732352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af472cee0 con 0x564af47290e0 2024-05-08T18:05:25.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.887+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4730180 con 0x7f5bf4076040 2024-05-08T18:05:25.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.888+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94879928732352] v27'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.888+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x7f5bf4076040 2024-05-08T18:05:25.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.888+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94879928745168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af47305a0 con 0x7f5bf4076040 2024-05-08T18:05:25.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.888+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4730f60 con 0x564af471fd60 2024-05-08T18:05:25.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.889+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94879928745168] v27'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x7f5bf4076040 2024-05-08T18:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.889+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.889+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94879928746704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4730f60 con 0x564af471fd60 2024-05-08T18:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.889+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4731920 con 0x564af47244e0 2024-05-08T18:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.891+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94879928746704] v27'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.891+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.891+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94879928748720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4730ed0 con 0x564af47244e0 2024-05-08T18:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.891+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4732660 con 0x564af47244e0 2024-05-08T18:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.892+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94879928748720] v27'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.892+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.892+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94879928753824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4732f00 con 0x564af47244e0 2024-05-08T18:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.892+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af47338c0 con 0x564af47244e0 2024-05-08T18:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.893+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94879928753824] v27'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.893+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.893+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94879928759312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4733ce0 con 0x564af47244e0 2024-05-08T18:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.893+0000 7f5c03610800 20 add_watcher() i=0 2024-05-08T18:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.893+0000 7f5c03610800 20 add_watcher() i=1 2024-05-08T18:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.893+0000 7f5c03610800 20 add_watcher() i=2 2024-05-08T18:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.893+0000 7f5c03610800 20 add_watcher() i=3 2024-05-08T18:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.893+0000 7f5c03610800 20 add_watcher() i=4 2024-05-08T18:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.893+0000 7f5c03610800 20 add_watcher() i=5 2024-05-08T18:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.893+0000 7f5c03610800 20 add_watcher() i=6 2024-05-08T18:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.894+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94879928759312] v27'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.894+0000 7f5c03610800 20 add_watcher() i=7 2024-05-08T18:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.894+0000 7f5c03610800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.894+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af4736f70 con 0x564af471fd60 2024-05-08T18:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.895+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.895+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94879928760736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4734270 con 0x564af471fd60 2024-05-08T18:05:25.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.896+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94879928760736] v27'152 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.896+0000 7f5ab57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:25.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.896+0000 7f5c03610800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:25.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.896+0000 7f5c03610800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:25.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.896+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af473f6a0 con 0x564af471fd60 2024-05-08T18:05:25.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.897+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'24 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.897+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af473fc10 con 0x564af47244e0 2024-05-08T18:05:25.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.898+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.898+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af473fc10 con 0x564af47290e0 2024-05-08T18:05:25.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.899+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.899+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4740180 con 0x564af471fd60 2024-05-08T18:05:25.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.900+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'153 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.900+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4740560 con 0x7f5bf4076040 2024-05-08T18:05:25.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.901+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'44 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x7f5bf4076040 2024-05-08T18:05:25.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.901+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af47409c0 con 0x564af47290e0 2024-05-08T18:05:25.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.902+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'29 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.902+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4740f30 con 0x564af471fd60 2024-05-08T18:05:25.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.904+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'20 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.904+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af47413e0 con 0x564af47244e0 2024-05-08T18:05:25.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.905+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'29 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.905+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741470 con 0x7f5bf4076040 2024-05-08T18:05:25.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.906+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x7f5bf4076040 2024-05-08T18:05:25.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.906+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741500 con 0x564af47290e0 2024-05-08T18:05:25.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.907+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'24 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.907+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741590 con 0x7f5bf4076040 2024-05-08T18:05:25.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.908+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'45 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x7f5bf4076040 2024-05-08T18:05:25.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.908+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741590 con 0x564af47244e0 2024-05-08T18:05:25.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.909+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'21 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.909+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741590 con 0x7f5bf4076040 2024-05-08T18:05:25.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.910+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x7f5bf4076040 2024-05-08T18:05:25.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.910+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741590 con 0x564af47244e0 2024-05-08T18:05:25.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.911+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'35 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.912+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af47415c0 con 0x564af471fd60 2024-05-08T18:05:25.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.912+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'154 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.913+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af47415c0 con 0x564af47290e0 2024-05-08T18:05:25.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.914+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'39 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.914+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741590 con 0x7f5bf4076040 2024-05-08T18:05:25.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.914+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'46 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x7f5bf4076040 2024-05-08T18:05:25.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.915+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741590 con 0x564af47290e0 2024-05-08T18:05:25.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.915+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'18 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.915+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741500 con 0x564af471fd60 2024-05-08T18:05:25.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.917+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'26 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.917+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741500 con 0x564af471fd60 2024-05-08T18:05:25.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.918+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'155 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.918+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741500 con 0x564af47290e0 2024-05-08T18:05:25.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.919+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'40 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.919+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741500 con 0x7f5bf4076040 2024-05-08T18:05:25.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.920+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'28 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x7f5bf4076040 2024-05-08T18:05:25.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.920+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741bb0 con 0x564af47290e0 2024-05-08T18:05:25.918 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.921+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'20 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.921+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741bb0 con 0x564af47290e0 2024-05-08T18:05:25.919 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.922+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'24 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.922+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741bb0 con 0x564af47290e0 2024-05-08T18:05:25.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.923+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'15 uv14 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.923+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741bb0 con 0x564af47290e0 2024-05-08T18:05:25.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.924+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'30 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.924+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741bb0 con 0x564af47244e0 2024-05-08T18:05:25.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.925+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'20 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.925+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741bb0 con 0x564af47290e0 2024-05-08T18:05:25.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.926+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'18 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.926+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741bb0 con 0x564af47244e0 2024-05-08T18:05:25.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.927+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'30 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.927+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741bb0 con 0x564af47290e0 2024-05-08T18:05:25.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.928+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'41 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.928+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741bb0 con 0x564af47244e0 2024-05-08T18:05:25.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.930+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'36 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.930+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af4741bb0 con 0x7f5bf4076040 2024-05-08T18:05:25.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.931+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'24 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f5bf8040040 con 0x7f5bf4076040 2024-05-08T18:05:25.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.931+0000 7f5a2ffff640 20 reqs_thread_entry: start 2024-05-08T18:05:25.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.931+0000 7f5c03610800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:25.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.931+0000 7f5c03610800 20 Filter name: none 2024-05-08T18:05:25.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.931+0000 7f5a2dffb640 20 reqs_thread_entry: start 2024-05-08T18:05:25.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.931+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:94 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af471fd60 2024-05-08T18:05:25.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.932+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(94 gc.0 [call out=15b] v0'0 uv3 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.932+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:95 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af471fd60 2024-05-08T18:05:25.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.932+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(95 gc.0 [call out=24b] v0'0 uv3 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.932+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:96 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af471fd60 2024-05-08T18:05:25.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.932+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(96 gc.0 [call out=15b] v0'0 uv3 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.932+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:97 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47244e0 2024-05-08T18:05:25.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.932+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(97 gc.1 [call out=15b] v0'0 uv5 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.933+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:98 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af47244e0 2024-05-08T18:05:25.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.933+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(98 gc.1 [call out=24b] v0'0 uv5 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.933+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:99 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47244e0 2024-05-08T18:05:25.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.933+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(99 gc.1 [call out=15b] v0'0 uv5 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.933+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:100 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.933+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(100 gc.2 [call out=15b] v0'0 uv3 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.933+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:101 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af47290e0 2024-05-08T18:05:25.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.934+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 28 ==== osd_op_reply(101 gc.2 [call out=24b] v0'0 uv3 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.934+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:102 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.934+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 29 ==== osd_op_reply(102 gc.2 [call out=15b] v0'0 uv3 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.934+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:103 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af471fd60 2024-05-08T18:05:25.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.934+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(103 gc.3 [call out=15b] v0'0 uv9 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.934+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:104 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af471fd60 2024-05-08T18:05:25.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.934+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(104 gc.3 [call out=24b] v0'0 uv9 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.934+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:105 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af471fd60 2024-05-08T18:05:25.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.935+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 32 ==== osd_op_reply(105 gc.3 [call out=15b] v0'0 uv9 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf8040040 con 0x564af471fd60 2024-05-08T18:05:25.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.935+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:106 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x7f5bf4076040 2024-05-08T18:05:25.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.935+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(106 gc.4 [call out=15b] v0'0 uv7 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf8040040 con 0x7f5bf4076040 2024-05-08T18:05:25.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.935+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:107 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x7f5bf4076040 2024-05-08T18:05:25.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.935+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(107 gc.4 [call out=24b] v0'0 uv7 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.935+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:108 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x7f5bf4076040 2024-05-08T18:05:25.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.936+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 20 ==== osd_op_reply(108 gc.4 [call out=15b] v0'0 uv7 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.936+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:109 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.936+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 30 ==== osd_op_reply(109 gc.5 [call out=15b] v0'0 uv4 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.936+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:110 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af47290e0 2024-05-08T18:05:25.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.936+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 31 ==== osd_op_reply(110 gc.5 [call out=24b] v0'0 uv4 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.936+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:111 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.936+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 32 ==== osd_op_reply(111 gc.5 [call out=15b] v0'0 uv4 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.936+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:112 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af471fd60 2024-05-08T18:05:25.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.937+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 33 ==== osd_op_reply(112 gc.6 [call out=15b] v0'0 uv5 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x564af471fd60 2024-05-08T18:05:25.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.937+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:113 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af471fd60 2024-05-08T18:05:25.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.937+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 34 ==== osd_op_reply(113 gc.6 [call out=24b] v0'0 uv5 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5bf804b020 con 0x564af471fd60 2024-05-08T18:05:25.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.937+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:114 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af471fd60 2024-05-08T18:05:25.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.937+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 35 ==== osd_op_reply(114 gc.6 [call out=15b] v0'0 uv5 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x564af471fd60 2024-05-08T18:05:25.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.937+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:115 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47244e0 2024-05-08T18:05:25.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.938+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(115 gc.7 [call out=15b] v0'0 uv4 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.938+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:116 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af47244e0 2024-05-08T18:05:25.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.938+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(116 gc.7 [call out=24b] v0'0 uv4 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.938+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:117 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47244e0 2024-05-08T18:05:25.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.938+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== osd_op_reply(117 gc.7 [call out=15b] v0'0 uv4 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.939+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:118 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x7f5bf4076040 2024-05-08T18:05:25.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.939+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 21 ==== osd_op_reply(118 gc.8 [call out=15b] v0'0 uv3 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.939+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:119 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x7f5bf4076040 2024-05-08T18:05:25.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.939+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 22 ==== osd_op_reply(119 gc.8 [call out=24b] v0'0 uv3 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.939+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:120 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x7f5bf4076040 2024-05-08T18:05:25.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.939+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 23 ==== osd_op_reply(120 gc.8 [call out=15b] v0'0 uv3 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.939+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:121 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.940+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 33 ==== osd_op_reply(121 gc.9 [call out=15b] v0'0 uv3 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.940+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:122 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af47290e0 2024-05-08T18:05:25.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.940+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 34 ==== osd_op_reply(122 gc.9 [call out=24b] v0'0 uv3 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.940+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:123 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.940+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 35 ==== osd_op_reply(123 gc.9 [call out=15b] v0'0 uv3 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.940+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:124 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x7f5bf4076040 2024-05-08T18:05:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.941+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 24 ==== osd_op_reply(124 gc.10 [call out=15b] v0'0 uv11 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.941+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:125 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x7f5bf4076040 2024-05-08T18:05:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.941+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 25 ==== osd_op_reply(125 gc.10 [call out=24b] v0'0 uv11 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.941+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:126 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x7f5bf4076040 2024-05-08T18:05:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.941+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 26 ==== osd_op_reply(126 gc.10 [call out=15b] v0'0 uv11 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.941+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:127 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47244e0 2024-05-08T18:05:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.942+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 31 ==== osd_op_reply(127 gc.11 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.942+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:128 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af47244e0 2024-05-08T18:05:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.942+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 32 ==== osd_op_reply(128 gc.11 [call out=24b] v0'0 uv7 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.942+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:129 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47244e0 2024-05-08T18:05:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.942+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 33 ==== osd_op_reply(129 gc.11 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.942+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:130 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x7f5bf4076040 2024-05-08T18:05:25.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.942+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 27 ==== osd_op_reply(130 gc.12 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.942+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:131 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x7f5bf4076040 2024-05-08T18:05:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.943+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 28 ==== osd_op_reply(131 gc.12 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.943+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:132 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x7f5bf4076040 2024-05-08T18:05:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.943+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 29 ==== osd_op_reply(132 gc.12 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.943+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:133 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47244e0 2024-05-08T18:05:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.943+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 34 ==== osd_op_reply(133 gc.13 [call out=15b] v0'0 uv8 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.944+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:134 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af47244e0 2024-05-08T18:05:25.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.944+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 35 ==== osd_op_reply(134 gc.13 [call out=24b] v0'0 uv8 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.944+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:135 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47244e0 2024-05-08T18:05:25.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.944+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 36 ==== osd_op_reply(135 gc.13 [call out=15b] v0'0 uv8 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.944+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:136 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af471fd60 2024-05-08T18:05:25.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.945+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 36 ==== osd_op_reply(136 gc.14 [call out=15b] v0'0 uv13 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x564af471fd60 2024-05-08T18:05:25.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.945+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:137 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af471fd60 2024-05-08T18:05:25.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.945+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 37 ==== osd_op_reply(137 gc.14 [call out=24b] v0'0 uv13 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf804b020 con 0x564af471fd60 2024-05-08T18:05:25.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.945+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:138 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af471fd60 2024-05-08T18:05:25.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.945+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 38 ==== osd_op_reply(138 gc.14 [call out=15b] v0'0 uv13 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x564af471fd60 2024-05-08T18:05:25.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.945+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:139 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.945+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 36 ==== osd_op_reply(139 gc.15 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.945+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:140 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af47290e0 2024-05-08T18:05:25.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.946+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 37 ==== osd_op_reply(140 gc.15 [call out=24b] v0'0 uv7 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.946+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:141 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.946+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 38 ==== osd_op_reply(141 gc.15 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.946+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:142 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x7f5bf4076040 2024-05-08T18:05:25.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.946+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 30 ==== osd_op_reply(142 gc.16 [call out=15b] v0'0 uv13 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.946+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:143 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x7f5bf4076040 2024-05-08T18:05:25.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.946+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 31 ==== osd_op_reply(143 gc.16 [call out=24b] v0'0 uv13 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.946+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:144 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x7f5bf4076040 2024-05-08T18:05:25.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.947+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 32 ==== osd_op_reply(144 gc.16 [call out=15b] v0'0 uv13 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.947+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:145 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.947+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 39 ==== osd_op_reply(145 gc.17 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.947+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:146 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af47290e0 2024-05-08T18:05:25.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.947+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 40 ==== osd_op_reply(146 gc.17 [call out=24b] v0'0 uv7 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.947+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:147 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.948+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 41 ==== osd_op_reply(147 gc.17 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.948+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:148 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af471fd60 2024-05-08T18:05:25.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.948+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 39 ==== osd_op_reply(148 gc.18 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x564af471fd60 2024-05-08T18:05:25.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.948+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:149 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af471fd60 2024-05-08T18:05:25.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.948+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 40 ==== osd_op_reply(149 gc.18 [call out=24b] v0'0 uv5 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf804b020 con 0x564af471fd60 2024-05-08T18:05:25.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.948+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:150 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af471fd60 2024-05-08T18:05:25.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.948+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 41 ==== osd_op_reply(150 gc.18 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x564af471fd60 2024-05-08T18:05:25.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.948+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:151 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af471fd60 2024-05-08T18:05:25.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.949+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 42 ==== osd_op_reply(151 gc.19 [call out=15b] v0'0 uv15 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x564af471fd60 2024-05-08T18:05:25.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.949+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:152 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af471fd60 2024-05-08T18:05:25.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.949+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 43 ==== osd_op_reply(152 gc.19 [call out=24b] v0'0 uv15 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf804b020 con 0x564af471fd60 2024-05-08T18:05:25.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.949+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:153 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af471fd60 2024-05-08T18:05:25.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.949+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 44 ==== osd_op_reply(153 gc.19 [call out=15b] v0'0 uv15 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x564af471fd60 2024-05-08T18:05:25.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.949+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:154 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.949+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 42 ==== osd_op_reply(154 gc.20 [call out=15b] v0'0 uv9 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.949+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:155 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af47290e0 2024-05-08T18:05:25.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.950+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 43 ==== osd_op_reply(155 gc.20 [call out=24b] v0'0 uv9 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.950+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:156 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.950+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 44 ==== osd_op_reply(156 gc.20 [call out=15b] v0'0 uv9 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.950+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:157 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x7f5bf4076040 2024-05-08T18:05:25.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.950+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 33 ==== osd_op_reply(157 gc.21 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.950+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:158 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x7f5bf4076040 2024-05-08T18:05:25.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.950+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 34 ==== osd_op_reply(158 gc.21 [call out=24b] v0'0 uv7 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.950+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:159 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x7f5bf4076040 2024-05-08T18:05:25.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.951+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 35 ==== osd_op_reply(159 gc.21 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.951+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:160 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.951+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 45 ==== osd_op_reply(160 gc.22 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.951+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:161 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af47290e0 2024-05-08T18:05:25.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.951+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 46 ==== osd_op_reply(161 gc.22 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.951+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:162 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.951+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 47 ==== osd_op_reply(162 gc.22 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.951+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:163 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.952+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 48 ==== osd_op_reply(163 gc.23 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.952+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:164 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af47290e0 2024-05-08T18:05:25.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.952+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 49 ==== osd_op_reply(164 gc.23 [call out=24b] v0'0 uv5 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.952+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:165 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.953+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 50 ==== osd_op_reply(165 gc.23 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.953+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:166 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.953+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 51 ==== osd_op_reply(166 gc.24 [call out=15b] v0'0 uv14 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.953+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:167 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476c1e0 con 0x564af47290e0 2024-05-08T18:05:25.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.953+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 52 ==== osd_op_reply(167 gc.24 [call out=24b] v0'0 uv14 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.953+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:168 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476d1f0 con 0x564af47290e0 2024-05-08T18:05:25.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.953+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 53 ==== osd_op_reply(168 gc.24 [call out=1702b] v0'0 uv14 ondisk = 0) ==== 149+0+1702 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.953+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:169 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476dc30 con 0x564af47290e0 2024-05-08T18:05:25.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.953+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 54 ==== osd_op_reply(169 gc.25 [call out=15b] v0'0 uv10 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.954+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:170 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476cc20 con 0x564af47290e0 2024-05-08T18:05:25.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.954+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 55 ==== osd_op_reply(170 gc.25 [call out=24b] v0'0 uv10 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.954+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:171 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476dc30 con 0x564af47290e0 2024-05-08T18:05:25.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.954+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 56 ==== osd_op_reply(171 gc.25 [call out=15b] v0'0 uv10 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.954+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:172 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476dc30 con 0x564af47244e0 2024-05-08T18:05:25.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.954+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 37 ==== osd_op_reply(172 gc.26 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.954+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:173 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476cc20 con 0x564af47244e0 2024-05-08T18:05:25.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.955+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 38 ==== osd_op_reply(173 gc.26 [call out=24b] v0'0 uv5 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.955+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:174 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476dc30 con 0x564af47244e0 2024-05-08T18:05:25.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.955+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 39 ==== osd_op_reply(174 gc.26 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.955+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:175 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476dc30 con 0x564af47290e0 2024-05-08T18:05:25.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.956+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 57 ==== osd_op_reply(175 gc.27 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.956+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:176 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476cc20 con 0x564af47290e0 2024-05-08T18:05:25.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.956+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 58 ==== osd_op_reply(176 gc.27 [call out=24b] v0'0 uv7 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.956+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:177 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476dc30 con 0x564af47290e0 2024-05-08T18:05:25.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.956+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 59 ==== osd_op_reply(177 gc.27 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.956+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:178 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476dc30 con 0x564af47244e0 2024-05-08T18:05:25.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.957+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 40 ==== osd_op_reply(178 gc.28 [call out=15b] v0'0 uv8 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.957+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:179 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476cc20 con 0x564af47244e0 2024-05-08T18:05:25.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.957+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 41 ==== osd_op_reply(179 gc.28 [call out=24b] v0'0 uv8 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.957+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:180 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476dc30 con 0x564af47244e0 2024-05-08T18:05:25.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.957+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 42 ==== osd_op_reply(180 gc.28 [call out=15b] v0'0 uv8 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.957+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:181 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476dc30 con 0x564af47290e0 2024-05-08T18:05:25.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.958+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 60 ==== osd_op_reply(181 gc.29 [call out=15b] v0'0 uv11 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.958+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:182 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476cc20 con 0x564af47290e0 2024-05-08T18:05:25.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.958+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 61 ==== osd_op_reply(182 gc.29 [call out=24b] v0'0 uv11 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.958+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:183 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476dc30 con 0x564af47290e0 2024-05-08T18:05:25.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.958+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 62 ==== osd_op_reply(183 gc.29 [call out=15b] v0'0 uv11 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.958+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:184 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476dc30 con 0x564af47244e0 2024-05-08T18:05:25.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.958+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 43 ==== osd_op_reply(184 gc.30 [call out=15b] v0'0 uv4 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.958+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:185 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476cc20 con 0x564af47244e0 2024-05-08T18:05:25.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.959+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 44 ==== osd_op_reply(185 gc.30 [call out=24b] v0'0 uv4 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.959+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:186 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476dc30 con 0x564af47244e0 2024-05-08T18:05:25.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.959+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 45 ==== osd_op_reply(186 gc.30 [call out=15b] v0'0 uv4 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.959+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:187 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476dc30 con 0x7f5bf4076040 2024-05-08T18:05:25.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.960+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 36 ==== osd_op_reply(187 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.960+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:188 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476cc20 con 0x7f5bf4076040 2024-05-08T18:05:25.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.960+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 37 ==== osd_op_reply(188 gc.31 [call out=24b] v0'0 uv5 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.960+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:189 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564af476dc30 con 0x7f5bf4076040 2024-05-08T18:05:25.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.960+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 38 ==== osd_op_reply(189 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.961+0000 7f5ab57e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:25.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.961+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:190 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94879928760736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af473d5b0 con 0x564af471fd60 2024-05-08T18:05:25.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.962+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 45 ==== osd_op_reply(190 data_loggenerations_metadata [watch unwatch cookie 94879928760736] v27'156 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5bf804b020 con 0x564af471fd60 2024-05-08T18:05:25.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.962+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:191 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94879928734912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af47339a0 con 0x7f5bf4076040 2024-05-08T18:05:25.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.963+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 39 ==== osd_op_reply(191 notify.0 [watch unwatch cookie 94879928734912] v27'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.963+0000 7f5c03610800 20 remove_watcher() i=0 2024-05-08T18:05:25.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.963+0000 7f5c03610800 2 removed watcher, disabling cache 2024-05-08T18:05:25.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.963+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:192 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94879928738688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af47339a0 con 0x564af471fd60 2024-05-08T18:05:25.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.964+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 46 ==== osd_op_reply(192 notify.1 [watch unwatch cookie 94879928738688] v27'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf804b020 con 0x564af471fd60 2024-05-08T18:05:25.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.964+0000 7f5c03610800 20 remove_watcher() i=1 2024-05-08T18:05:25.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.964+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:193 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94879928732352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af472e380 con 0x564af47290e0 2024-05-08T18:05:25.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.965+0000 7f5bfe4f5640 1 -- 172.21.15.50:0/2058148234 <== osd.3 v2:172.21.15.50:6800/3130492744 63 ==== osd_op_reply(193 notify.2 [watch unwatch cookie 94879928732352] v27'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf4081070 con 0x564af47290e0 2024-05-08T18:05:25.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.965+0000 7f5c03610800 20 remove_watcher() i=2 2024-05-08T18:05:25.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.965+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:194 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94879928745168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af47339a0 con 0x7f5bf4076040 2024-05-08T18:05:25.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.966+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.2 v2:172.21.15.50:6811/3039646617 40 ==== osd_op_reply(194 notify.3 [watch unwatch cookie 94879928745168] v27'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf804b020 con 0x7f5bf4076040 2024-05-08T18:05:25.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.966+0000 7f5c03610800 20 remove_watcher() i=3 2024-05-08T18:05:25.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.966+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:195 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94879928746704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af472cac0 con 0x564af471fd60 2024-05-08T18:05:25.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.967+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 <== osd.1 v2:172.21.15.50:6808/2141923372 47 ==== osd_op_reply(195 notify.4 [watch unwatch cookie 94879928746704] v27'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf804b020 con 0x564af471fd60 2024-05-08T18:05:25.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.967+0000 7f5c03610800 20 remove_watcher() i=4 2024-05-08T18:05:25.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.967+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:196 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94879928748720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af472fcd0 con 0x564af47244e0 2024-05-08T18:05:25.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.968+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 46 ==== osd_op_reply(196 notify.5 [watch unwatch cookie 94879928748720] v27'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.968+0000 7f5c03610800 20 remove_watcher() i=5 2024-05-08T18:05:25.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.968+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:197 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94879928753824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af47339a0 con 0x564af47244e0 2024-05-08T18:05:25.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.969+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 47 ==== osd_op_reply(197 notify.6 [watch unwatch cookie 94879928753824] v27'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.969+0000 7f5c03610800 20 remove_watcher() i=6 2024-05-08T18:05:25.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.969+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:198 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94879928759312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564af47302d0 con 0x564af47244e0 2024-05-08T18:05:25.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.970+0000 7f5bfecf6640 1 -- 172.21.15.50:0/2058148234 <== osd.0 v2:172.21.15.46:6800/337143975 48 ==== osd_op_reply(198 notify.7 [watch unwatch cookie 94879928759312] v27'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5bf0077070 con 0x564af47244e0 2024-05-08T18:05:25.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.970+0000 7f5c03610800 20 remove_watcher() i=7 2024-05-08T18:05:25.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564af47244e0 msgr2=0x564af47268d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564af47244e0 0x564af47268d0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:25.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564af471fd60 msgr2=0x564af4722220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564af471fd60 0x564af4722220 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:25.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f5bf4076040 msgr2=0x7f5bf4083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f5bf4076040 0x7f5bf4083410 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:25.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564af47290e0 msgr2=0x564af472b5a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564af47290e0 0x564af472b5a0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:25.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f5ac803bd40 msgr2=0x7f5ac803e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f5ac803bd40 0x7f5ac803e200 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4052fd0 tx=0x7f5bf404f000 comp rx=0 tx=0).stop 2024-05-08T18:05:25.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af470d2a0 msgr2=0x564af4714700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5bff4f7640 1 -- 172.21.15.50:0/2058148234 reap_dead start 2024-05-08T18:05:25.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af470d2a0 0x564af4714700 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0024620 tx=0x7f5bf0007f60 comp rx=0 tx=0).stop 2024-05-08T18:05:25.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 shutdown_connections 2024-05-08T18:05:25.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 --2- 172.21.15.50:0/2058148234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af470d2a0 0x564af4714700 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:25.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 >> 172.21.15.50:0/2058148234 conn(0x564af4709570 msgr2=0x564af4706bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:25.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 shutdown_connections 2024-05-08T18:05:25.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 -- 172.21.15.50:0/2058148234 wait complete. 2024-05-08T18:05:25.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 -- 172.21.15.50:0/3364257535 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f5ad803bba0 msgr2=0x7f5ad803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 --2- 172.21.15.50:0/3364257535 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f5ad803bba0 0x7f5ad803e060 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x564af46e7750 tx=0x7f5bf4005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:25.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 -- 172.21.15.50:0/3364257535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e6d20 msgr2=0x564af45e5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 --2- 172.21.15.50:0/3364257535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e6d20 0x564af45e5190 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f5bf0050f60 tx=0x7f5bf004a1a0 comp rx=0 tx=0).stop 2024-05-08T18:05:25.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 -- 172.21.15.50:0/3364257535 shutdown_connections 2024-05-08T18:05:25.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 --2- 172.21.15.50:0/3364257535 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f5ad803bba0 0x7f5ad803e060 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:25.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 --2- 172.21.15.50:0/3364257535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e6d20 0x564af45e5190 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:25.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 -- 172.21.15.50:0/3364257535 >> 172.21.15.50:0/3364257535 conn(0x564af46f1f20 msgr2=0x564af46f2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:25.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 -- 172.21.15.50:0/3364257535 shutdown_connections 2024-05-08T18:05:25.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.971+0000 7f5c03610800 1 -- 172.21.15.50:0/3364257535 wait complete. 2024-05-08T18:05:25.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.972+0000 7f5c03610800 1 -- 172.21.15.50:0/1211371285 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564af46e91a0 msgr2=0x564af46eb590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.972+0000 7f5c03610800 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564af46e91a0 0x564af46eb590 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:25.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.972+0000 7f5c03610800 1 -- 172.21.15.50:0/1211371285 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564af45eac90 msgr2=0x564af46e6940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.972+0000 7f5c03610800 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564af45eac90 0x564af46e6940 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:25.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.972+0000 7f5c03610800 1 -- 172.21.15.50:0/1211371285 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564af46ed0e0 msgr2=0x564af46ef4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.972+0000 7f5c03610800 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564af46ed0e0 0x564af46ef4d0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:25.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.972+0000 7f5c03610800 1 -- 172.21.15.50:0/1211371285 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f5ae403bba0 msgr2=0x7f5ae403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.972+0000 7f5c03610800 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f5ae403bba0 0x7f5ae403e060 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4054000 tx=0x7f5bf404e000 comp rx=0 tx=0).stop 2024-05-08T18:05:25.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.972+0000 7f5c03610800 1 -- 172.21.15.50:0/1211371285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e7180 msgr2=0x564af46df700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:25.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.972+0000 7f5c03610800 1 --2- 172.21.15.50:0/1211371285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564af45e7180 0x564af46df700 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x564af46d88f0 tx=0x7f5bf0000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:25.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.972+0000 7f5bff4f7640 1 -- 172.21.15.50:0/1211371285 reap_dead start 2024-05-08T18:05:25.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.972+0000 7f5c03610800 1 -- 172.21.15.50:0/1211371285 shutdown_connections 2024-05-08T18:05:25.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.972+0000 7f5c03610800 1 -- 172.21.15.50:0/1211371285 >> 172.21.15.50:0/1211371285 conn(0x564af45e0ad0 msgr2=0x564af46d1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:25.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.972+0000 7f5c03610800 1 -- 172.21.15.50:0/1211371285 shutdown_connections 2024-05-08T18:05:25.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:25.972+0000 7f5c03610800 1 -- 172.21.15.50:0/1211371285 wait complete. 2024-05-08T18:05:25.979 INFO:teuthology.orchestra.run.smithi050.stdout:[{"tag":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.14930053639123495669\u0000","time":"2024-05-08T18:05:25.768649+0000","objs":[{"pool":"default.rgw.buckets.data","oid":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_1","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_2","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_3","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_4","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_5","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_6","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_7","key":"","instance":""}]}] 2024-05-08T18:05:25.979 DEBUG:tasks.util.rgw: json result: [{'tag': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.14930053639123495669\x00', 'time': '2024-05-08T18:05:25.768649+0000', 'objs': [{'pool': 'default.rgw.buckets.data', 'oid': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_1', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_2', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_3', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_4', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_5', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_6', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_7', 'key': '', 'instance': ''}]}] 2024-05-08T18:05:25.979 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process'] 2024-05-08T18:05:25.980 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process'] 2024-05-08T18:05:25.980 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process 2024-05-08T18:05:26.023 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:26.023 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:26.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.026+0000 7fad9640e800 1 Processor -- start 2024-05-08T18:05:26.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.026+0000 7fad9640e800 1 -- start start 2024-05-08T18:05:26.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.026+0000 7fad9640e800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x557126a9f6e0 0x557126a9fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.026+0000 7fad9640e800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x557126aa0180 con 0x557126aa08a0 2024-05-08T18:05:26.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.026+0000 7fad9640e800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55712693fe20 con 0x557126a9f6e0 2024-05-08T18:05:26.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.027+0000 7fad93f06640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x557126a9f6e0 0x557126a9fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.027+0000 7fad93705640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x557126aa08a0 0x557126aa0d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:60338/0 (socket says 172.21.15.50:60338) 2024-05-08T18:05:26.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.027+0000 7fad93705640 1 -- 172.21.15.50:0/2385524214 learned_addr learned my addr 172.21.15.50:0/2385524214 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:26.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.027+0000 7fad93f06640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x557126a9f6e0 0x557126a9fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43638/0 (socket says 172.21.15.50:43638) 2024-05-08T18:05:26.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.027+0000 7fad93f06640 1 -- 172.21.15.50:0/2385524214 >> v1:172.21.15.46:6789/0 conn(0x557126aa08a0 legacy=0x557126aa0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.027+0000 7fad93f06640 1 -- 172.21.15.50:0/2385524214 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x557126aa1460 con 0x557126a9f6e0 2024-05-08T18:05:26.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.029+0000 7fad93f06640 1 --2- 172.21.15.50:0/2385524214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 0x557126a9fb00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fad8400a470 tx=0x7fad8401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ae90226239b794 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.029+0000 7fad92f04640 1 -- 172.21.15.50:0/2385524214 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad84057070 con 0x557126a9f6e0 2024-05-08T18:05:26.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.029+0000 7fad92f04640 1 -- 172.21.15.50:0/2385524214 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8404adc0 con 0x557126a9f6e0 2024-05-08T18:05:26.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.029+0000 7fad92f04640 1 -- 172.21.15.50:0/2385524214 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad84052590 con 0x557126a9f6e0 2024-05-08T18:05:26.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.030+0000 7fad9640e800 1 -- 172.21.15.50:0/2385524214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 msgr2=0x557126a9fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.030+0000 7fad9640e800 1 --2- 172.21.15.50:0/2385524214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 0x557126a9fb00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fad8400a470 tx=0x7fad8401ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:26.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.030+0000 7fad9640e800 1 -- 172.21.15.50:0/2385524214 shutdown_connections 2024-05-08T18:05:26.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.030+0000 7fad9640e800 1 --2- 172.21.15.50:0/2385524214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 0x557126a9fb00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.030+0000 7fad9640e800 1 -- 172.21.15.50:0/2385524214 >> 172.21.15.50:0/2385524214 conn(0x557126a9ac60 msgr2=0x557126a9d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.030+0000 7fad9640e800 1 -- 172.21.15.50:0/2385524214 shutdown_connections 2024-05-08T18:05:26.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.030+0000 7fad9640e800 1 -- 172.21.15.50:0/2385524214 wait complete. 2024-05-08T18:05:26.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.045+0000 7fad9640e800 1 Processor -- start 2024-05-08T18:05:26.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.045+0000 7fad9640e800 1 -- start start 2024-05-08T18:05:26.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.045+0000 7fad9640e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 0x557126b90bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.045+0000 7fad9640e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5571269eaa00 con 0x557126a9f6e0 2024-05-08T18:05:26.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.045+0000 7fad93f06640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 0x557126b90bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.045+0000 7fad93f06640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 0x557126b90bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43646/0 (socket says 172.21.15.50:43646) 2024-05-08T18:05:26.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.045+0000 7fad93f06640 1 -- 172.21.15.50:0/603674433 learned_addr learned my addr 172.21.15.50:0/603674433 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.046+0000 7fad93f06640 1 -- 172.21.15.50:0/603674433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557126b911c0 con 0x557126a9f6e0 2024-05-08T18:05:26.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.046+0000 7fad93f06640 1 --2- 172.21.15.50:0/603674433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 0x557126b90bf0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fad84055040 tx=0x7fad8404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49033af1c219efcc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.046+0000 7fac81e0b640 1 -- 172.21.15.50:0/603674433 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad84057040 con 0x557126a9f6e0 2024-05-08T18:05:26.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.046+0000 7fac81e0b640 1 -- 172.21.15.50:0/603674433 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad84062070 con 0x557126a9f6e0 2024-05-08T18:05:26.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.046+0000 7fac81e0b640 1 -- 172.21.15.50:0/603674433 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad8405d470 con 0x557126a9f6e0 2024-05-08T18:05:26.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.047+0000 7fad9640e800 1 -- 172.21.15.50:0/603674433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 msgr2=0x557126b90bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.047+0000 7fad9640e800 1 --2- 172.21.15.50:0/603674433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 0x557126b90bf0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fad84055040 tx=0x7fad8404c000 comp rx=0 tx=0).stop 2024-05-08T18:05:26.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.047+0000 7fad9640e800 1 -- 172.21.15.50:0/603674433 shutdown_connections 2024-05-08T18:05:26.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.047+0000 7fad9640e800 1 --2- 172.21.15.50:0/603674433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 0x557126b90bf0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.047+0000 7fad9640e800 1 -- 172.21.15.50:0/603674433 >> 172.21.15.50:0/603674433 conn(0x557126a9ac60 msgr2=0x557126b8c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.047+0000 7fad9640e800 1 -- 172.21.15.50:0/603674433 shutdown_connections 2024-05-08T18:05:26.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.047+0000 7fad9640e800 1 -- 172.21.15.50:0/603674433 wait complete. 2024-05-08T18:05:26.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.047+0000 7fad9640e800 1 Processor -- start 2024-05-08T18:05:26.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.047+0000 7fad9640e800 1 -- start start 2024-05-08T18:05:26.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.047+0000 7fad9640e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 0x557126b98640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.047+0000 7fad9640e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557126aa8d70 con 0x557126a9f6e0 2024-05-08T18:05:26.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.047+0000 7fad93f06640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 0x557126b98640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.047+0000 7fad93f06640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 0x557126b98640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43660/0 (socket says 172.21.15.50:43660) 2024-05-08T18:05:26.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.047+0000 7fad93f06640 1 -- 172.21.15.50:0/4158942153 learned_addr learned my addr 172.21.15.50:0/4158942153 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.048+0000 7fad93f06640 1 -- 172.21.15.50:0/4158942153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557126b98b80 con 0x557126a9f6e0 2024-05-08T18:05:26.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.048+0000 7fad93f06640 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 0x557126b98640 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fad8405f040 tx=0x7fad84002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.048+0000 7fac7bfff640 1 -- 172.21.15.50:0/4158942153 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad8406b070 con 0x557126a9f6e0 2024-05-08T18:05:26.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.048+0000 7fac7bfff640 1 -- 172.21.15.50:0/4158942153 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad8405d940 con 0x557126a9f6e0 2024-05-08T18:05:26.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.048+0000 7fac7bfff640 1 -- 172.21.15.50:0/4158942153 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad84057070 con 0x557126a9f6e0 2024-05-08T18:05:26.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.048+0000 7fad9640e800 1 -- 172.21.15.50:0/4158942153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557126b98e10 con 0x557126a9f6e0 2024-05-08T18:05:26.048 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.048+0000 7fad9640e800 1 -- 172.21.15.50:0/4158942153 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557126b9bac0 con 0x557126a9f6e0 2024-05-08T18:05:26.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.049+0000 7fac7bfff640 1 -- 172.21.15.50:0/4158942153 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fad8405dc20 con 0x557126a9f6e0 2024-05-08T18:05:26.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.049+0000 7fac7bfff640 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac7c03bb50 0x7fac7c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.049+0000 7fac7bfff640 1 -- 172.21.15.50:0/4158942153 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fad840946a0 con 0x557126a9f6e0 2024-05-08T18:05:26.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.049+0000 7fad93705640 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac7c03bb50 0x7fac7c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.050+0000 7fad9640e800 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557126aa08a0 0x557126b9f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.050+0000 7fad9640e800 1 -- 172.21.15.50:0/4158942153 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126b9fe00 con 0x557126aa08a0 2024-05-08T18:05:26.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.050+0000 7fad94707640 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557126aa08a0 0x557126b9f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.050+0000 7fad93705640 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac7c03bb50 0x7fac7c03e010 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fad88001d70 tx=0x7fad880012d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.050+0000 7fad94707640 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557126aa08a0 0x557126b9f880 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.050+0000 7fad94707640 1 -- 172.21.15.50:0/4158942153 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126aa08a0 2024-05-08T18:05:26.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.050+0000 7fad9640e800 1 -- 172.21.15.50:0/4158942153 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126ba06d0 con 0x557126aa08a0 2024-05-08T18:05:26.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.051+0000 7fad94707640 1 -- 172.21.15.50:0/4158942153 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126aa08a0 2024-05-08T18:05:26.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.051+0000 7fad9640e800 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557126ba1fe0 0x557126ba4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.051+0000 7fad9640e800 1 -- 172.21.15.50:0/4158942153 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126ba4c00 con 0x557126ba1fe0 2024-05-08T18:05:26.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.051+0000 7fad93f06640 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557126ba1fe0 0x557126ba4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.051+0000 7fad93f06640 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557126ba1fe0 0x557126ba4470 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.051+0000 7fad93f06640 1 -- 172.21.15.50:0/4158942153 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fad8405a040 con 0x557126ba1fe0 2024-05-08T18:05:26.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.052+0000 7fad9640e800 1 -- 172.21.15.50:0/4158942153 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126ba0f20 con 0x557126aa08a0 2024-05-08T18:05:26.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.052+0000 7fad94707640 1 -- 172.21.15.50:0/4158942153 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126aa08a0 2024-05-08T18:05:26.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.052+0000 7fad9640e800 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557126ba6060 0x557126ba8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.052+0000 7fad9640e800 1 -- 172.21.15.50:0/4158942153 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126ba8b40 con 0x557126ba6060 2024-05-08T18:05:26.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.052+0000 7fad93705640 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557126ba6060 0x557126ba8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.052+0000 7fad93705640 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557126ba6060 0x557126ba8450 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.052+0000 7fad93705640 1 -- 172.21.15.50:0/4158942153 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fad88009050 con 0x557126ba6060 2024-05-08T18:05:26.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.053+0000 7fad9640e800 1 Processor -- start 2024-05-08T18:05:26.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.053+0000 7fad9640e800 1 -- start start 2024-05-08T18:05:26.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.053+0000 7fad9640e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126bb6eb0 0x557126bb72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.053+0000 7fad9640e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557126991330 con 0x557126bb6eb0 2024-05-08T18:05:26.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.053+0000 7fad93f06640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126bb6eb0 0x557126bb72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.053+0000 7fad93f06640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126bb6eb0 0x557126bb72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43676/0 (socket says 172.21.15.50:43676) 2024-05-08T18:05:26.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.053+0000 7fad93f06640 1 -- 172.21.15.50:0/2617423776 learned_addr learned my addr 172.21.15.50:0/2617423776 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.054+0000 7fad93f06640 1 -- 172.21.15.50:0/2617423776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557126bb7810 con 0x557126bb6eb0 2024-05-08T18:05:26.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.054+0000 7fad93f06640 1 --2- 172.21.15.50:0/2617423776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126bb6eb0 0x557126bb72d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fad84092a30 tx=0x7fad8405b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.054+0000 7fac797fa640 1 -- 172.21.15.50:0/2617423776 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad84086070 con 0x557126bb6eb0 2024-05-08T18:05:26.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.054+0000 7fac797fa640 1 -- 172.21.15.50:0/2617423776 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad84091380 con 0x557126bb6eb0 2024-05-08T18:05:26.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.054+0000 7fad9640e800 1 -- 172.21.15.50:0/2617423776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557126bb8280 con 0x557126bb6eb0 2024-05-08T18:05:26.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.054+0000 7fac797fa640 1 -- 172.21.15.50:0/2617423776 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad84052620 con 0x557126bb6eb0 2024-05-08T18:05:26.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.054+0000 7fad9640e800 1 -- 172.21.15.50:0/2617423776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557126bb8780 con 0x557126bb6eb0 2024-05-08T18:05:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.055+0000 7fac797fa640 1 -- 172.21.15.50:0/2617423776 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fad84089070 con 0x557126bb6eb0 2024-05-08T18:05:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.055+0000 7fac797fa640 1 --2- 172.21.15.50:0/2617423776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac6c03bba0 0x7fac6c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.055+0000 7fac797fa640 1 -- 172.21.15.50:0/2617423776 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fad84096080 con 0x557126bb6eb0 2024-05-08T18:05:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.055+0000 7fad93705640 1 --2- 172.21.15.50:0/2617423776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac6c03bba0 0x7fac6c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.055+0000 7fad9640e800 1 Processor -- start 2024-05-08T18:05:26.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.055+0000 7fad9640e800 1 -- start start 2024-05-08T18:05:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.055+0000 7fad93705640 1 --2- 172.21.15.50:0/2617423776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac6c03bba0 0x7fac6c03e060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fad880016b0 tx=0x7fad8800f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.055+0000 7fad9640e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a14c00 0x557126a15020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.055+0000 7fad9640e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557126b91480 con 0x557126a14c00 2024-05-08T18:05:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.056+0000 7fad93f06640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a14c00 0x557126a15020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.056+0000 7fad93f06640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a14c00 0x557126a15020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43684/0 (socket says 172.21.15.50:43684) 2024-05-08T18:05:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.056+0000 7fad93f06640 1 -- 172.21.15.50:0/587899286 learned_addr learned my addr 172.21.15.50:0/587899286 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.056+0000 7fad93f06640 1 -- 172.21.15.50:0/587899286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557126a13410 con 0x557126a14c00 2024-05-08T18:05:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.056+0000 7fad93f06640 1 --2- 172.21.15.50:0/587899286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a14c00 0x557126a15020 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fad8405d490 tx=0x7fad8408b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac22cc6d90c8cb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.056+0000 7fac72ffd640 1 -- 172.21.15.50:0/587899286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad84081db0 con 0x557126a14c00 2024-05-08T18:05:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.056+0000 7fac72ffd640 1 -- 172.21.15.50:0/587899286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad84081400 con 0x557126a14c00 2024-05-08T18:05:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.056+0000 7fac72ffd640 1 -- 172.21.15.50:0/587899286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad84070460 con 0x557126a14c00 2024-05-08T18:05:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.057+0000 7fad9640e800 1 -- 172.21.15.50:0/587899286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a14c00 msgr2=0x557126a15020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.057+0000 7fad9640e800 1 --2- 172.21.15.50:0/587899286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a14c00 0x557126a15020 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fad8405d490 tx=0x7fad8408b810 comp rx=0 tx=0).stop 2024-05-08T18:05:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.057+0000 7fad9640e800 1 -- 172.21.15.50:0/587899286 shutdown_connections 2024-05-08T18:05:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.057+0000 7fad9640e800 1 --2- 172.21.15.50:0/587899286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a14c00 0x557126a15020 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.057+0000 7fad9640e800 1 -- 172.21.15.50:0/587899286 >> 172.21.15.50:0/587899286 conn(0x557126bc02e0 msgr2=0x557126bbe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.057+0000 7fad9640e800 1 -- 172.21.15.50:0/587899286 shutdown_connections 2024-05-08T18:05:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.057+0000 7fad9640e800 1 -- 172.21.15.50:0/587899286 wait complete. 2024-05-08T18:05:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.057+0000 7fad9640e800 1 Processor -- start 2024-05-08T18:05:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.057+0000 7fad9640e800 1 -- start start 2024-05-08T18:05:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.057+0000 7fad9640e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a14c00 0x557126bcd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.057+0000 7fad9640e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557126a13820 con 0x557126a14c00 2024-05-08T18:05:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.058+0000 7fad93f06640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a14c00 0x557126bcd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.058+0000 7fad93f06640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a14c00 0x557126bcd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43696/0 (socket says 172.21.15.50:43696) 2024-05-08T18:05:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.058+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 learned_addr learned my addr 172.21.15.50:0/1847279976 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.058+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557126bcdb20 con 0x557126a14c00 2024-05-08T18:05:26.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.058+0000 7fad93f06640 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a14c00 0x557126bcd5e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fad84070f90 tx=0x7fad84091730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.058+0000 7fac717fa640 1 -- 172.21.15.50:0/1847279976 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad840b1070 con 0x557126a14c00 2024-05-08T18:05:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.058+0000 7fac717fa640 1 -- 172.21.15.50:0/1847279976 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad840949c0 con 0x557126a14c00 2024-05-08T18:05:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.058+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557126bcdb20 con 0x557126a14c00 2024-05-08T18:05:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.058+0000 7fac717fa640 1 -- 172.21.15.50:0/1847279976 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad84094ca0 con 0x557126a14c00 2024-05-08T18:05:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.059+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557126bcdb20 con 0x557126a14c00 2024-05-08T18:05:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.059+0000 7fac717fa640 1 -- 172.21.15.50:0/1847279976 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fad8408b8c0 con 0x557126a14c00 2024-05-08T18:05:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.059+0000 7fac717fa640 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac5c03bb50 0x7fac5c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.060+0000 7fac717fa640 1 -- 172.21.15.50:0/1847279976 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fad840b6c60 con 0x557126a14c00 2024-05-08T18:05:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.060+0000 7fad93705640 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac5c03bb50 0x7fac5c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.060+0000 7fad93705640 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac5c03bb50 0x7fac5c03e010 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fad8800efd0 tx=0x7fad88010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.060+0000 7fad9640e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.060+0000 7fad9640e800 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557126bd8d40 0x557126bdb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.060+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126bdb950 con 0x557126bd8d40 2024-05-08T18:05:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.060+0000 7fad94707640 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557126bd8d40 0x557126bdb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.061+0000 7fad94707640 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557126bd8d40 0x557126bdb200 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.061+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126bd8d40 2024-05-08T18:05:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.061+0000 7fad9640e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.061+0000 7fad9640e800 20 realm 2024-05-08T18:05:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.061+0000 7fad9640e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.061+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126bdbd60 con 0x557126bd8d40 2024-05-08T18:05:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.061+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126bd8d40 2024-05-08T18:05:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.061+0000 7fad9640e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.061+0000 7fad9640e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.061+0000 7fad9640e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.061+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc630 con 0x557126bd8d40 2024-05-08T18:05:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.061+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126bd8d40 2024-05-08T18:05:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.061+0000 7fad9640e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.061+0000 7fad9640e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.061+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bd8d40 2024-05-08T18:05:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.062+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126bd8d40 2024-05-08T18:05:26.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.062+0000 7fad9640e800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.062+0000 7fad9640e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.062+0000 7fad9640e800 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557126bdd730 0x557126bdfb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.062+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126be0130 con 0x557126bdd730 2024-05-08T18:05:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.062+0000 7fad93f06640 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557126bdd730 0x557126bdfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.062+0000 7fad93f06640 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557126bdd730 0x557126bdfb20 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.063+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.063+0000 7fad9640e800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.063+0000 7fad9640e800 20 searching for the correct realm 2024-05-08T18:05:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.063+0000 7fad9640e800 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557126be20e0 0x557126be45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.063+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126be4c70 con 0x557126be20e0 2024-05-08T18:05:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.063+0000 7fad93705640 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557126be20e0 0x557126be45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.063+0000 7fad93705640 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557126be20e0 0x557126be45a0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.063+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.063+0000 7fad93705640 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fad88037040 0x7fad88036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.063+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126be4c70 con 0x7fad88037040 2024-05-08T18:05:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.063+0000 7fad94707640 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fad88037040 0x7fad88036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.064+0000 7fad94707640 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fad88037040 0x7fad88036410 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.064+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad8c00a4a0 con 0x7fad88037040 2024-05-08T18:05:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.064+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bdd730 2024-05-08T18:05:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.064+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.064+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bd8d40 2024-05-08T18:05:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.065+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126bd8d40 2024-05-08T18:05:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.065+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bd8d40 2024-05-08T18:05:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.065+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126bd8d40 2024-05-08T18:05:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.065+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bdd730 2024-05-08T18:05:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.065+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.065+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bdd730 2024-05-08T18:05:26.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.065+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.065+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x7fad88037040 2024-05-08T18:05:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.066+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad8c00a4a0 con 0x7fad88037040 2024-05-08T18:05:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.066+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126be20e0 2024-05-08T18:05:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.066+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.066+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bd8d40 2024-05-08T18:05:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.066+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126bd8d40 2024-05-08T18:05:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.066+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bdd730 2024-05-08T18:05:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.066+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.066+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bd8d40 2024-05-08T18:05:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.067+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126bd8d40 2024-05-08T18:05:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.067+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x7fad88037040 2024-05-08T18:05:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.067+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fad8c00a4a0 con 0x7fad88037040 2024-05-08T18:05:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.067+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126be20e0 2024-05-08T18:05:26.072 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.067+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.067+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126be20e0 2024-05-08T18:05:26.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.067+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.073 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.067+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bdd730 2024-05-08T18:05:26.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.068+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.068+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126be20e0 2024-05-08T18:05:26.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.068+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.068+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126be20e0 2024-05-08T18:05:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.068+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.068+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bd8d40 2024-05-08T18:05:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.068+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126bd8d40 2024-05-08T18:05:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.069+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bd8d40 2024-05-08T18:05:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.069+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126bd8d40 2024-05-08T18:05:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.069+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bdd730 2024-05-08T18:05:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.069+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.069+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126be20e0 2024-05-08T18:05:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.069+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.069+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x7fad88037040 2024-05-08T18:05:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.069+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad8c00a4a0 con 0x7fad88037040 2024-05-08T18:05:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.070+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bd8d40 2024-05-08T18:05:26.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.070+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126bd8d40 2024-05-08T18:05:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.070+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bdd730 2024-05-08T18:05:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.070+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.070+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126be20e0 2024-05-08T18:05:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.070+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.070+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bdd730 2024-05-08T18:05:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.071+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.071+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x7fad88037040 2024-05-08T18:05:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.071+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad8c00a4a0 con 0x7fad88037040 2024-05-08T18:05:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.071+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126be20e0 2024-05-08T18:05:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.071+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.071+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bdd730 2024-05-08T18:05:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x557126bd8d40 2024-05-08T18:05:26.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126bd8d40 2024-05-08T18:05:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x557126bdc9d0 con 0x7fad88037040 2024-05-08T18:05:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fad8c00a4a0 con 0x7fad88037040 2024-05-08T18:05:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad9640e800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad9640e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad9640e800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad9640e800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad9640e800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad9640e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad9640e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126be09f0 con 0x557126bd8d40 2024-05-08T18:05:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126bd8d40 2024-05-08T18:05:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad9640e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad9640e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.072+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126be09f0 con 0x557126bd8d40 2024-05-08T18:05:26.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fad8c00a4a0 con 0x557126bd8d40 2024-05-08T18:05:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad9640e800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad9640e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126be09f0 con 0x557126be20e0 2024-05-08T18:05:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad9640e800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad9640e800 20 zone default found 2024-05-08T18:05:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad9640e800 4 Realm: () 2024-05-08T18:05:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad9640e800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad9640e800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad9640e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad9640e800 20 zonegroup default 2024-05-08T18:05:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad9640e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126be1320 con 0x557126bdd730 2024-05-08T18:05:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad9640e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad9640e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.073+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126be6b30 con 0x557126be20e0 2024-05-08T18:05:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.074+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.074+0000 7fad9640e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.074+0000 7fad9640e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.074+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126be6fe0 con 0x557126be20e0 2024-05-08T18:05:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.074+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.074+0000 7fad9640e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.074+0000 7fad9640e800 20 started sync module instance, tier type = 2024-05-08T18:05:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.074+0000 7fad9640e800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.074+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126be5f70 con 0x7fad88037040 2024-05-08T18:05:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.075+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad8c00a4a0 con 0x7fad88037040 2024-05-08T18:05:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.075+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93944469677168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126be5f70 con 0x7fad88037040 2024-05-08T18:05:26.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.075+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126be7ae0 con 0x557126bd8d40 2024-05-08T18:05:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.076+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93944469677168] v27'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.076+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.076+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93944469681584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126be5f70 con 0x557126bd8d40 2024-05-08T18:05:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.076+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126be8560 con 0x557126be20e0 2024-05-08T18:05:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.078+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.078+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93944469684336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126be8e30 con 0x557126be20e0 2024-05-08T18:05:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.078+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126be98f0 con 0x7fad88037040 2024-05-08T18:05:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.078+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93944469681584] v27'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.079+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93944469684336] v27'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.079+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.079+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93944469686656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126be5ac0 con 0x7fad88037040 2024-05-08T18:05:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.079+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bea1c0 con 0x557126bd8d40 2024-05-08T18:05:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.080+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.080+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93944469691456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bea5b0 con 0x557126bd8d40 2024-05-08T18:05:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.080+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126beb0a0 con 0x557126bdd730 2024-05-08T18:05:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.080+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93944469686656] v27'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.081+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93944469691456] v27'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.081+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.081+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93944469692512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126be9420 con 0x557126bdd730 2024-05-08T18:05:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.081+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126beb940 con 0x557126bdd730 2024-05-08T18:05:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.082+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.082+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93944469699808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bec5b0 con 0x557126bdd730 2024-05-08T18:05:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.082+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126becfb0 con 0x557126bdd730 2024-05-08T18:05:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.082+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 93944469692512] v27'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.083+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93944469699808] v27'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.083+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.083+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93944469695344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bec5b0 con 0x557126bdd730 2024-05-08T18:05:26.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.083+0000 7fad9640e800 20 add_watcher() i=0 2024-05-08T18:05:26.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.083+0000 7fad9640e800 20 add_watcher() i=1 2024-05-08T18:05:26.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.083+0000 7fad9640e800 20 add_watcher() i=2 2024-05-08T18:05:26.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.083+0000 7fad9640e800 20 add_watcher() i=3 2024-05-08T18:05:26.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.083+0000 7fad9640e800 20 add_watcher() i=4 2024-05-08T18:05:26.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.083+0000 7fad9640e800 20 add_watcher() i=5 2024-05-08T18:05:26.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.083+0000 7fad9640e800 20 add_watcher() i=6 2024-05-08T18:05:26.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.084+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93944469695344] v27'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.084+0000 7fad9640e800 20 add_watcher() i=7 2024-05-08T18:05:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.084+0000 7fad9640e800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.084+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126befe90 con 0x557126bd8d40 2024-05-08T18:05:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.085+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.085+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93944469701328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bed310 con 0x557126bd8d40 2024-05-08T18:05:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.086+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93944469701328] v27'157 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.086+0000 7fac49fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.086+0000 7fad9640e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.086+0000 7fad9640e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.086+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf8820 con 0x557126bd8d40 2024-05-08T18:05:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.087+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'25 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.087+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf8e70 con 0x557126bdd730 2024-05-08T18:05:26.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.088+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'20 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.088+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf8e70 con 0x557126be20e0 2024-05-08T18:05:26.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.089+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.089+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf9930 con 0x557126bd8d40 2024-05-08T18:05:26.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.090+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'158 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.090+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf66b0 con 0x7fad88037040 2024-05-08T18:05:26.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.091+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'47 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.091+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf66b0 con 0x557126be20e0 2024-05-08T18:05:26.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.092+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'31 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.092+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bfa660 con 0x557126bd8d40 2024-05-08T18:05:26.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.094+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.094+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bfab10 con 0x557126bdd730 2024-05-08T18:05:26.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.095+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'31 uv4 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.095+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bfab10 con 0x7fad88037040 2024-05-08T18:05:26.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.096+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.096+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bfab10 con 0x557126be20e0 2024-05-08T18:05:26.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.097+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'25 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.097+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf56a0 con 0x7fad88037040 2024-05-08T18:05:26.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.098+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'48 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.098+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126bdd730 2024-05-08T18:05:26.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.099+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'22 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.099+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x7fad88037040 2024-05-08T18:05:26.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.100+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.100+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126bdd730 2024-05-08T18:05:26.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.101+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'37 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.101+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126bd8d40 2024-05-08T18:05:26.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.102+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'159 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.102+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126be20e0 2024-05-08T18:05:26.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.103+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'42 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.103+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x7fad88037040 2024-05-08T18:05:26.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.104+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'49 uv13 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.104+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126be20e0 2024-05-08T18:05:26.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.105+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'19 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.105+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126bd8d40 2024-05-08T18:05:26.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.106+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'27 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.106+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126bd8d40 2024-05-08T18:05:26.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.107+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'160 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.107+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126be20e0 2024-05-08T18:05:26.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.108+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'43 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.108+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x7fad88037040 2024-05-08T18:05:26.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.109+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'29 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.109+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126be20e0 2024-05-08T18:05:26.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.110+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'21 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.108 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.110+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126be20e0 2024-05-08T18:05:26.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.111+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'25 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.111+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126be20e0 2024-05-08T18:05:26.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.112+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'16 uv14 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.112+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126be20e0 2024-05-08T18:05:26.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.113+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'32 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.113+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126bdd730 2024-05-08T18:05:26.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.114+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.114+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126be20e0 2024-05-08T18:05:26.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.115+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'19 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.115+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126bdd730 2024-05-08T18:05:26.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.116+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'32 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.116+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126be20e0 2024-05-08T18:05:26.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.117+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'44 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.117+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x557126bdd730 2024-05-08T18:05:26.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.118+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'38 uv4 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.118+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bf5c90 con 0x7fad88037040 2024-05-08T18:05:26.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.119+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'25 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.119+0000 7fac48fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:26.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.120+0000 7fad9640e800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:26.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.120+0000 7fad9640e800 20 Filter name: none 2024-05-08T18:05:26.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.120+0000 7fabc27fc640 20 reqs_thread_entry: start 2024-05-08T18:05:26.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.120+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=1 2024-05-08T18:05:26.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.120+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:94 6.d 6:b019ea83:gc::gc.18:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126bd8d40 2024-05-08T18:05:26.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.121+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(94 gc.18 [call] v27'28 uv28 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.121+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:95 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126bd8d40 2024-05-08T18:05:26.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.121+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(95 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.121+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.120 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.121+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:96 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c26680 con 0x557126bd8d40 2024-05-08T18:05:26.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.121+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(96 gc.18 [call out=24b] v0'0 uv28 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.121+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:97 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126bd8d40 2024-05-08T18:05:26.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.122+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(97 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.122+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.122+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:98 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126bd8d40 2024-05-08T18:05:26.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.122+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(98 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.122+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.122+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:99 6.d 6:b019ea83:gc::gc.18:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126bd8d40 2024-05-08T18:05:26.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.123+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 32 ==== osd_op_reply(99 gc.18 [call] v27'29 uv29 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.123+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=1 2024-05-08T18:05:26.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.123+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:100 6.4 6:24c8e1ae:gc::gc.19:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126bd8d40 2024-05-08T18:05:26.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.124+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 33 ==== osd_op_reply(100 gc.19 [call] v27'161 uv161 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.124+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:101 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126bd8d40 2024-05-08T18:05:26.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.124+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 34 ==== osd_op_reply(101 gc.19 [call out=15b] v0'0 uv161 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.124+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.124+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:102 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c26680 con 0x557126bd8d40 2024-05-08T18:05:26.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.124+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 35 ==== osd_op_reply(102 gc.19 [call out=24b] v0'0 uv161 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.124+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:103 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126bd8d40 2024-05-08T18:05:26.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.125+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 36 ==== osd_op_reply(103 gc.19 [call out=15b] v0'0 uv161 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.125+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.125+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:104 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126bd8d40 2024-05-08T18:05:26.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.125+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 37 ==== osd_op_reply(104 gc.19 [call out=15b] v0'0 uv161 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.125+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.125+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:105 6.4 6:24c8e1ae:gc::gc.19:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126bd8d40 2024-05-08T18:05:26.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.126+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 38 ==== osd_op_reply(105 gc.19 [call] v27'162 uv162 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.126+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=1 2024-05-08T18:05:26.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.126+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:106 6.7 6:e622f76a:gc::gc.20:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.127+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(106 gc.20 [call] v27'45 uv45 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.127+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:107 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.127+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 28 ==== osd_op_reply(107 gc.20 [call out=15b] v0'0 uv45 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.127+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.127+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:108 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c26680 con 0x557126be20e0 2024-05-08T18:05:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.127+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 29 ==== osd_op_reply(108 gc.20 [call out=24b] v0'0 uv45 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.127+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:109 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.128+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 30 ==== osd_op_reply(109 gc.20 [call out=15b] v0'0 uv45 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.128+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.128+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:110 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.128+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 31 ==== osd_op_reply(110 gc.20 [call out=15b] v0'0 uv45 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.128+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.128+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:111 6.7 6:e622f76a:gc::gc.20:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.129+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 32 ==== osd_op_reply(111 gc.20 [call] v27'46 uv46 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.129+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=1 2024-05-08T18:05:26.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.129+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:112 6.a 6:5127e642:gc::gc.21:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x7fad88037040 2024-05-08T18:05:26.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.130+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(112 gc.21 [call] v27'30 uv30 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.130+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:113 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x7fad88037040 2024-05-08T18:05:26.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.130+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(113 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.130+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.131+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:114 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c26680 con 0x7fad88037040 2024-05-08T18:05:26.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.131+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 20 ==== osd_op_reply(114 gc.21 [call out=24b] v0'0 uv30 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.131+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:115 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x7fad88037040 2024-05-08T18:05:26.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.131+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 21 ==== osd_op_reply(115 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.131+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.131+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:116 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x7fad88037040 2024-05-08T18:05:26.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.131+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 22 ==== osd_op_reply(116 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.131+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.131+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:117 6.a 6:5127e642:gc::gc.21:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x7fad88037040 2024-05-08T18:05:26.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.132+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 23 ==== osd_op_reply(117 gc.21 [call] v27'31 uv31 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.132+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=1 2024-05-08T18:05:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.132+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:118 6.11 6:8eb99c53:gc::gc.22:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.133+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 33 ==== osd_op_reply(118 gc.22 [call] v27'22 uv22 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.133+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:119 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.133+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 34 ==== osd_op_reply(119 gc.22 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.133+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.133+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:120 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c26680 con 0x557126be20e0 2024-05-08T18:05:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.134+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 35 ==== osd_op_reply(120 gc.22 [call out=24b] v0'0 uv22 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.134+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:121 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.134+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 36 ==== osd_op_reply(121 gc.22 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.134+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.134+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:122 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.134+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 37 ==== osd_op_reply(122 gc.22 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.134+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.134+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:123 6.11 6:8eb99c53:gc::gc.22:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.135+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 38 ==== osd_op_reply(123 gc.22 [call] v27'23 uv23 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.135+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=1 2024-05-08T18:05:26.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.135+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:124 6.8 6:17d7e8ed:gc::gc.23:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.136 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.136+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 39 ==== osd_op_reply(124 gc.23 [call] v27'26 uv26 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.136+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:125 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.136+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 40 ==== osd_op_reply(125 gc.23 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.136+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.136+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:126 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c26680 con 0x557126be20e0 2024-05-08T18:05:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.137+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 41 ==== osd_op_reply(126 gc.23 [call out=24b] v0'0 uv26 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.137+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:127 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.137+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 42 ==== osd_op_reply(127 gc.23 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.137+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.137+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:128 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.137+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 43 ==== osd_op_reply(128 gc.23 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.137+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.137+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:129 6.8 6:17d7e8ed:gc::gc.23:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.138+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 44 ==== osd_op_reply(129 gc.23 [call] v27'27 uv27 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.138+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=1 2024-05-08T18:05:26.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.138+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:130 6.1b 6:db9d130d:gc::gc.24:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.139+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 45 ==== osd_op_reply(130 gc.24 [call] v27'17 uv17 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.139+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:131 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.139+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 46 ==== osd_op_reply(131 gc.24 [call out=15b] v0'0 uv17 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.139+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.139+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:132 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c26680 con 0x557126be20e0 2024-05-08T18:05:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.139+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 47 ==== osd_op_reply(132 gc.24 [call out=24b] v0'0 uv17 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.139+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:133 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 48 ==== osd_op_reply(133 gc.24 [call out=15b] v0'0 uv17 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:134 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c27690 con 0x557126be20e0 2024-05-08T18:05:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 49 ==== osd_op_reply(134 gc.24 [call out=1702b] v0'0 uv17 ondisk = 0) ==== 149+0+1702 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-08T18:05:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 20 garbage collection: RGWGC::process iterating over entry tag='1c041ffd-e40a-486a-88e5-3547b85128ea.4221.14930053639123495669', time=2024-05-08T18:05:25.768649+0000, chain.objs.size()=7 2024-05-08T18:05:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_1 2024-05-08T18:05:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:135 3.24s0 3:250a72cf:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_1:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c27990 con 0x557126bdd730 2024-05-08T18:05:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_2 2024-05-08T18:05:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:136 3.3cs0 3:3cbf96c1:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_2:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a5b0 con 0x7fad88037040 2024-05-08T18:05:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_3 2024-05-08T18:05:26.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:137 3.cs0 3:3014903e:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_3:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2cf30 con 0x557126be20e0 2024-05-08T18:05:26.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_4 2024-05-08T18:05:26.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:138 3.2bs0 3:d5ab4bc8:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_4:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2f940 con 0x557126bd8d40 2024-05-08T18:05:26.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_5 2024-05-08T18:05:26.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:139 3.17s0 3:eb26ea78:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_5:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2cf30 con 0x557126bd8d40 2024-05-08T18:05:26.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_6 2024-05-08T18:05:26.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:140 3.24s0 3:2540a711:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_6:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c34740 con 0x557126bdd730 2024-05-08T18:05:26.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_7 2024-05-08T18:05:26.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.140+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:141 3.23s0 3:c6e4573a:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_7:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c36fb0 con 0x557126bdd730 2024-05-08T18:05:26.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.143+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 50 ==== osd_op_reply(137 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_3 [call] v27'3 uv3 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.144+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(135 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_1 [call] v27'3 uv3 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.144+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(141 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_7 [call] v27'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.144+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 39 ==== osd_op_reply(138 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_4 [call] v27'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.144+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 24 ==== osd_op_reply(136 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_2 [call] v27'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.144+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(140 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_6 [call] v27'4 uv4 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.145+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 40 ==== osd_op_reply(139 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.1__shadow_.i7BhOLMk-wMkRqs2c6fCbmmyd9k2qgM_5 [call] v27'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.146+0000 7fad9640e800 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-08T18:05:26.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.146+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:142 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_remove_entries in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2aa60 con 0x557126be20e0 2024-05-08T18:05:26.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.147+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 51 ==== osd_op_reply(142 gc.24 [call] v27'18 uv18 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.147+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:143 6.1b 6:db9d130d:gc::gc.24:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2aa60 con 0x557126be20e0 2024-05-08T18:05:26.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.148+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 52 ==== osd_op_reply(143 gc.24 [call] v27'19 uv19 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.148+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=1 2024-05-08T18:05:26.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.148+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:144 6.9 6:921ecdb1:gc::gc.25:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2aa60 con 0x557126be20e0 2024-05-08T18:05:26.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.149+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 53 ==== osd_op_reply(144 gc.25 [call] v27'33 uv33 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.149+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:145 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2aa60 con 0x557126be20e0 2024-05-08T18:05:26.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.150+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 54 ==== osd_op_reply(145 gc.25 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.150+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.150+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:146 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c28eb0 con 0x557126be20e0 2024-05-08T18:05:26.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.150+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 55 ==== osd_op_reply(146 gc.25 [call out=24b] v0'0 uv33 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.150+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:147 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2aa60 con 0x557126be20e0 2024-05-08T18:05:26.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.150+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 56 ==== osd_op_reply(147 gc.25 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.150+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.150+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:148 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2aa60 con 0x557126be20e0 2024-05-08T18:05:26.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.150+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 57 ==== osd_op_reply(148 gc.25 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.150+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.150+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:149 6.9 6:921ecdb1:gc::gc.25:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2aa60 con 0x557126be20e0 2024-05-08T18:05:26.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.151+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 58 ==== osd_op_reply(149 gc.25 [call] v27'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.151+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=1 2024-05-08T18:05:26.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.151+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:150 6.1f 6:fde4dd55:gc::gc.26:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2aa60 con 0x557126bdd730 2024-05-08T18:05:26.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.152+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(150 gc.26 [call] v27'22 uv22 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.152+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:151 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2aa60 con 0x557126bdd730 2024-05-08T18:05:26.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.153+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(151 gc.26 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.153+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.153+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:152 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c28e60 con 0x557126bdd730 2024-05-08T18:05:26.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.153+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== osd_op_reply(152 gc.26 [call out=24b] v0'0 uv22 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.153+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:153 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2aa60 con 0x557126bdd730 2024-05-08T18:05:26.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.153+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 31 ==== osd_op_reply(153 gc.26 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.153+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.153+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:154 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2aa60 con 0x557126bdd730 2024-05-08T18:05:26.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.154+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 32 ==== osd_op_reply(154 gc.26 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.154+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.154+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:155 6.1f 6:fde4dd55:gc::gc.26:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2aa60 con 0x557126bdd730 2024-05-08T18:05:26.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.155+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 33 ==== osd_op_reply(155 gc.26 [call] v27'23 uv23 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.155+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=1 2024-05-08T18:05:26.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.155+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:156 6.1a 6:5909e671:gc::gc.27:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2aa60 con 0x557126be20e0 2024-05-08T18:05:26.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.156+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 59 ==== osd_op_reply(156 gc.27 [call] v27'20 uv20 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.156+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:157 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2aa10 con 0x557126be20e0 2024-05-08T18:05:26.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.156+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 60 ==== osd_op_reply(157 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.156+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.156+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:158 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2aa10 con 0x557126be20e0 2024-05-08T18:05:26.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.156+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 61 ==== osd_op_reply(158 gc.27 [call out=24b] v0'0 uv20 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.156+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:159 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126be20e0 2024-05-08T18:05:26.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.157+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 62 ==== osd_op_reply(159 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.157+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.157+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:160 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126be20e0 2024-05-08T18:05:26.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.157+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 63 ==== osd_op_reply(160 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.157+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.157+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:161 6.1a 6:5909e671:gc::gc.27:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126be20e0 2024-05-08T18:05:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.158+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 64 ==== osd_op_reply(161 gc.27 [call] v27'21 uv21 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.158+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=1 2024-05-08T18:05:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.158+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:162 6.1d 6:b88e3271:gc::gc.28:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126bdd730 2024-05-08T18:05:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.159+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 34 ==== osd_op_reply(162 gc.28 [call] v27'33 uv33 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.159+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:163 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126bdd730 2024-05-08T18:05:26.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.159+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 35 ==== osd_op_reply(163 gc.28 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.159+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.159+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:164 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126bdd730 2024-05-08T18:05:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.160+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 36 ==== osd_op_reply(164 gc.28 [call out=24b] v0'0 uv33 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.160+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:165 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126bdd730 2024-05-08T18:05:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.160+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 37 ==== osd_op_reply(165 gc.28 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.160+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.160+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:166 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126bdd730 2024-05-08T18:05:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.160+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 38 ==== osd_op_reply(166 gc.28 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.160+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.160+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:167 6.1d 6:b88e3271:gc::gc.28:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126bdd730 2024-05-08T18:05:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.161+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 39 ==== osd_op_reply(167 gc.28 [call] v27'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.161+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=1 2024-05-08T18:05:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.161+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:168 6.7 6:e47c9322:gc::gc.29:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126be20e0 2024-05-08T18:05:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.162+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 65 ==== osd_op_reply(168 gc.29 [call] v27'47 uv47 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.162+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:169 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126be20e0 2024-05-08T18:05:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.162+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 66 ==== osd_op_reply(169 gc.29 [call out=15b] v0'0 uv47 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.163+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.163+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:170 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126be20e0 2024-05-08T18:05:26.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.163+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 67 ==== osd_op_reply(170 gc.29 [call out=24b] v0'0 uv47 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.163+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:171 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126be20e0 2024-05-08T18:05:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.163+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 68 ==== osd_op_reply(171 gc.29 [call out=15b] v0'0 uv47 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.163+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.163+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:172 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126be20e0 2024-05-08T18:05:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.163+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 69 ==== osd_op_reply(172 gc.29 [call out=15b] v0'0 uv47 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.163+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.163+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:173 6.7 6:e47c9322:gc::gc.29:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126be20e0 2024-05-08T18:05:26.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.164+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 70 ==== osd_op_reply(173 gc.29 [call] v27'48 uv48 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.164+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=1 2024-05-08T18:05:26.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.164+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:174 6.0 6:013d6c33:gc::gc.30:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126bdd730 2024-05-08T18:05:26.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.165+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 40 ==== osd_op_reply(174 gc.30 [call] v27'39 uv39 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.165+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:175 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126bdd730 2024-05-08T18:05:26.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.166+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 41 ==== osd_op_reply(175 gc.30 [call out=15b] v0'0 uv39 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.166+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.166+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:176 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c28e60 con 0x557126bdd730 2024-05-08T18:05:26.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.166+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 42 ==== osd_op_reply(176 gc.30 [call out=24b] v0'0 uv39 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.166+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:177 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126bdd730 2024-05-08T18:05:26.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.167+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 43 ==== osd_op_reply(177 gc.30 [call out=15b] v0'0 uv39 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.167+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.167+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:178 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126bdd730 2024-05-08T18:05:26.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.167+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 44 ==== osd_op_reply(178 gc.30 [call out=15b] v0'0 uv39 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.167+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.167+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:179 6.0 6:013d6c33:gc::gc.30:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x557126bdd730 2024-05-08T18:05:26.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.168+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 45 ==== osd_op_reply(179 gc.30 [call] v27'40 uv40 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.168+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=1 2024-05-08T18:05:26.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.168+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:180 6.f 6:f24cb4a8:gc::gc.31:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c2a9f0 con 0x7fad88037040 2024-05-08T18:05:26.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.169+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 25 ==== osd_op_reply(180 gc.31 [call] v27'26 uv26 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.169+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:181 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.169+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 26 ==== osd_op_reply(181 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.169+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.169+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:182 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.170+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 27 ==== osd_op_reply(182 gc.31 [call out=24b] v0'0 uv26 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.170+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:183 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.170+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 28 ==== osd_op_reply(183 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.170+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.170+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:184 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.170+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 29 ==== osd_op_reply(184 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.170+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.170+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:185 6.f 6:f24cb4a8:gc::gc.31:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.171+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 30 ==== osd_op_reply(185 gc.31 [call] v27'27 uv27 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.171+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=1 2024-05-08T18:05:26.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.171+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:186 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.172+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 41 ==== osd_op_reply(186 gc.0 [call] v27'26 uv26 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.172+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:187 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.172+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 42 ==== osd_op_reply(187 gc.0 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.172+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.172+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:188 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2beb0 con 0x557126bd8d40 2024-05-08T18:05:26.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.173+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 43 ==== osd_op_reply(188 gc.0 [call out=24b] v0'0 uv26 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.173+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:189 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2beb0 con 0x557126bd8d40 2024-05-08T18:05:26.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.173+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 44 ==== osd_op_reply(189 gc.0 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.173+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.173+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:190 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2beb0 con 0x557126bd8d40 2024-05-08T18:05:26.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.173+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 45 ==== osd_op_reply(190 gc.0 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.173+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.173+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:191 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.174+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 46 ==== osd_op_reply(191 gc.0 [call] v27'27 uv27 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.174+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=1 2024-05-08T18:05:26.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.174+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:192 6.18 6:1e85421f:gc::gc.1:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.176+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 46 ==== osd_op_reply(192 gc.1 [call] v27'21 uv21 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.176+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:193 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.176+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 47 ==== osd_op_reply(193 gc.1 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.176+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.176+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:194 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.176+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 48 ==== osd_op_reply(194 gc.1 [call out=24b] v0'0 uv21 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.176+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:195 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.177+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 49 ==== osd_op_reply(195 gc.1 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.177+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.177+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:196 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.177+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 50 ==== osd_op_reply(196 gc.1 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.177+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.177+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:197 6.18 6:1e85421f:gc::gc.1:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.178+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 51 ==== osd_op_reply(197 gc.1 [call] v27'22 uv22 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.178+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=1 2024-05-08T18:05:26.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.178+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:198 6.b 6:d242335b:gc::gc.2:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.179+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 71 ==== osd_op_reply(198 gc.2 [call] v27'20 uv20 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.179+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:199 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.180+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 72 ==== osd_op_reply(199 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.180+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.180+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:200 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.180+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 73 ==== osd_op_reply(200 gc.2 [call out=24b] v0'0 uv20 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.180+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:201 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.180+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 74 ==== osd_op_reply(201 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.180+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.180+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:202 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.180+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 75 ==== osd_op_reply(202 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.180+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.180+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:203 6.b 6:d242335b:gc::gc.2:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.181+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 76 ==== osd_op_reply(203 gc.2 [call] v27'21 uv21 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.181+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=1 2024-05-08T18:05:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.181+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:204 6.4 6:2166d8e1:gc::gc.3:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.182+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 47 ==== osd_op_reply(204 gc.3 [call] v27'163 uv163 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.182+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:205 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.182+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 48 ==== osd_op_reply(205 gc.3 [call out=15b] v0'0 uv163 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.182+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.182+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:206 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2beb0 con 0x557126bd8d40 2024-05-08T18:05:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.183+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 49 ==== osd_op_reply(206 gc.3 [call out=24b] v0'0 uv163 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.183+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:207 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.183+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 50 ==== osd_op_reply(207 gc.3 [call out=15b] v0'0 uv163 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.183+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.183+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:208 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.183+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 51 ==== osd_op_reply(208 gc.3 [call out=15b] v0'0 uv163 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.183+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.183+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:209 6.4 6:2166d8e1:gc::gc.3:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.184+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 52 ==== osd_op_reply(209 gc.3 [call] v27'164 uv164 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.184+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=1 2024-05-08T18:05:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.184+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:210 6.3 6:c60da066:gc::gc.4:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.185+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 31 ==== osd_op_reply(210 gc.4 [call] v27'50 uv50 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.185+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:211 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.185+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 32 ==== osd_op_reply(211 gc.4 [call out=15b] v0'0 uv50 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.185+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.185+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:212 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.186+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 33 ==== osd_op_reply(212 gc.4 [call out=24b] v0'0 uv50 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.186+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:213 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.186+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 34 ==== osd_op_reply(213 gc.4 [call out=15b] v0'0 uv50 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.186+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.186+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:214 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.186+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 35 ==== osd_op_reply(214 gc.4 [call out=15b] v0'0 uv50 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.186+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.186+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:215 6.3 6:c60da066:gc::gc.4:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.187+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 36 ==== osd_op_reply(215 gc.4 [call] v27'51 uv51 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.187+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=1 2024-05-08T18:05:26.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.187+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:216 6.9 6:954be8d2:gc::gc.5:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.188+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 77 ==== osd_op_reply(216 gc.5 [call] v27'35 uv35 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.188+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:217 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.188+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 78 ==== osd_op_reply(217 gc.5 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.188+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.188+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:218 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.189+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 79 ==== osd_op_reply(218 gc.5 [call out=24b] v0'0 uv35 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.189+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:219 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.189+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 80 ==== osd_op_reply(219 gc.5 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.189+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.189+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:220 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.189+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 81 ==== osd_op_reply(220 gc.5 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.189+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.189+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:221 6.9 6:954be8d2:gc::gc.5:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.190+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 82 ==== osd_op_reply(221 gc.5 [call] v27'36 uv36 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.190+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=1 2024-05-08T18:05:26.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.190+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:222 6.e 6:748722ff:gc::gc.6:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.191+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 53 ==== osd_op_reply(222 gc.6 [call] v27'22 uv22 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.191+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:223 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.191+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 54 ==== osd_op_reply(223 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.192+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.192+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:224 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.192+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 55 ==== osd_op_reply(224 gc.6 [call out=24b] v0'0 uv22 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.192+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:225 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.192+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 56 ==== osd_op_reply(225 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.192+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.192+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:226 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.192+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 57 ==== osd_op_reply(226 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.192+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.192+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:227 6.e 6:748722ff:gc::gc.6:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.193+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 58 ==== osd_op_reply(227 gc.6 [call] v27'23 uv23 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.193+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=1 2024-05-08T18:05:26.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.193+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:228 6.1d 6:beaa4be4:gc::gc.7:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.194+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 52 ==== osd_op_reply(228 gc.7 [call] v27'35 uv35 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.194+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:229 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.194+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 53 ==== osd_op_reply(229 gc.7 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.194+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.195+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:230 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.195+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 54 ==== osd_op_reply(230 gc.7 [call out=24b] v0'0 uv35 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.195+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:231 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.195+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 55 ==== osd_op_reply(231 gc.7 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.195+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.195+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:232 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.195+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 56 ==== osd_op_reply(232 gc.7 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.195+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.195+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:233 6.1d 6:beaa4be4:gc::gc.7:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.196+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 57 ==== osd_op_reply(233 gc.7 [call] v27'36 uv36 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.196+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=1 2024-05-08T18:05:26.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.196+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:234 6.12 6:4f78e9a2:gc::gc.8:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.197+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 37 ==== osd_op_reply(234 gc.8 [call] v27'20 uv20 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.197+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:235 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.197+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 38 ==== osd_op_reply(235 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.197+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.197+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:236 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.198+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 39 ==== osd_op_reply(236 gc.8 [call out=24b] v0'0 uv20 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.198+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:237 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.198+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 40 ==== osd_op_reply(237 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.198+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.198+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:238 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.198+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 41 ==== osd_op_reply(238 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.198+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.198+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:239 6.12 6:4f78e9a2:gc::gc.8:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.199+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 42 ==== osd_op_reply(239 gc.8 [call] v27'21 uv21 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.199+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=1 2024-05-08T18:05:26.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.199+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:240 6.1c 6:3e426232:gc::gc.9:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.200+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 83 ==== osd_op_reply(240 gc.9 [call] v27'26 uv26 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.200+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:241 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.200+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 84 ==== osd_op_reply(241 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.200+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.200+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:242 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c2beb0 con 0x557126be20e0 2024-05-08T18:05:26.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.201+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 85 ==== osd_op_reply(242 gc.9 [call out=24b] v0'0 uv26 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.201+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:243 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.201+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 86 ==== osd_op_reply(243 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.201+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.201+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:244 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.201+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 87 ==== osd_op_reply(244 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.201+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.201+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:245 6.1c 6:3e426232:gc::gc.9:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.202+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 88 ==== osd_op_reply(245 gc.9 [call] v27'27 uv27 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.202+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=1 2024-05-08T18:05:26.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.202+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:246 6.3 6:c31f41d7:gc::gc.10:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.203+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 43 ==== osd_op_reply(246 gc.10 [call] v27'52 uv52 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.203+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:247 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.203+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 44 ==== osd_op_reply(247 gc.10 [call out=15b] v0'0 uv52 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.203+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.203+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:248 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.204+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 45 ==== osd_op_reply(248 gc.10 [call out=24b] v0'0 uv52 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.204+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:249 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.204+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 46 ==== osd_op_reply(249 gc.10 [call out=15b] v0'0 uv52 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.204+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.204+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:250 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.204+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 47 ==== osd_op_reply(250 gc.10 [call out=15b] v0'0 uv52 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.204+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.204+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:251 6.3 6:c31f41d7:gc::gc.10:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.205+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 48 ==== osd_op_reply(251 gc.10 [call] v27'53 uv53 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.205+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=1 2024-05-08T18:05:26.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.205+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:252 6.13 6:c8333aab:gc::gc.11:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.206+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 58 ==== osd_op_reply(252 gc.11 [call] v27'23 uv23 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.206+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:253 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.207+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 59 ==== osd_op_reply(253 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.207+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.207+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:254 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.207+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 60 ==== osd_op_reply(254 gc.11 [call out=24b] v0'0 uv23 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.207+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:255 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.207+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 61 ==== osd_op_reply(255 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.207+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.207+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:256 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.208+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 62 ==== osd_op_reply(256 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.208+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.208+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:257 6.13 6:c8333aab:gc::gc.11:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.209+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 63 ==== osd_op_reply(257 gc.11 [call] v27'24 uv24 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.209+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=1 2024-05-08T18:05:26.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.209+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:258 6.15 6:ab569f81:gc::gc.12:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.210+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 49 ==== osd_op_reply(258 gc.12 [call] v27'20 uv20 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.210+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:259 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.210+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 50 ==== osd_op_reply(259 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.210+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.210+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:260 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.210+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 51 ==== osd_op_reply(260 gc.12 [call out=24b] v0'0 uv20 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.210+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:261 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.211+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 52 ==== osd_op_reply(261 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.211+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.211+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:262 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.211+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 53 ==== osd_op_reply(262 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.211+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.211+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:263 6.15 6:ab569f81:gc::gc.12:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.212+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 54 ==== osd_op_reply(263 gc.12 [call] v27'21 uv21 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.212+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=1 2024-05-08T18:05:26.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.212+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:264 6.0 6:0667bb13:gc::gc.13:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.214+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 64 ==== osd_op_reply(264 gc.13 [call] v27'41 uv41 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.214+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:265 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.214+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 65 ==== osd_op_reply(265 gc.13 [call out=15b] v0'0 uv41 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.214+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.214+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:266 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.214+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 66 ==== osd_op_reply(266 gc.13 [call out=24b] v0'0 uv41 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.214+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:267 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.215+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 67 ==== osd_op_reply(267 gc.13 [call out=15b] v0'0 uv41 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.215+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.215+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:268 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.215+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 68 ==== osd_op_reply(268 gc.13 [call out=15b] v0'0 uv41 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.215+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.215+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:269 6.0 6:0667bb13:gc::gc.13:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bdd730 2024-05-08T18:05:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.216+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 69 ==== osd_op_reply(269 gc.13 [call] v27'42 uv42 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.216+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=1 2024-05-08T18:05:26.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.216+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:270 6.4 6:227f63b5:gc::gc.14:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.217+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 59 ==== osd_op_reply(270 gc.14 [call] v27'165 uv165 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.217+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:271 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.218+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 60 ==== osd_op_reply(271 gc.14 [call out=15b] v0'0 uv165 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.218+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.218+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:272 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.218+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 61 ==== osd_op_reply(272 gc.14 [call out=24b] v0'0 uv165 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.218+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:273 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.218+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 62 ==== osd_op_reply(273 gc.14 [call out=15b] v0'0 uv165 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.218+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.218+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:274 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.218+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 63 ==== osd_op_reply(274 gc.14 [call out=15b] v0'0 uv165 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.218+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.218+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:275 6.4 6:227f63b5:gc::gc.14:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.219+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 64 ==== osd_op_reply(275 gc.14 [call] v27'166 uv166 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.219+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=1 2024-05-08T18:05:26.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.219+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:276 6.7 6:e07a6548:gc::gc.15:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.220+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 89 ==== osd_op_reply(276 gc.15 [call] v27'49 uv49 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.220+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:277 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.221+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 90 ==== osd_op_reply(277 gc.15 [call out=15b] v0'0 uv49 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.219 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.221+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.221+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:278 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.221+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 91 ==== osd_op_reply(278 gc.15 [call out=24b] v0'0 uv49 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.221+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:279 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.221+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 92 ==== osd_op_reply(279 gc.15 [call out=15b] v0'0 uv49 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.221+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.221+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:280 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.221+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 93 ==== osd_op_reply(280 gc.15 [call out=15b] v0'0 uv49 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.221+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.221+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:281 6.7 6:e07a6548:gc::gc.15:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.223+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 94 ==== osd_op_reply(281 gc.15 [call] v27'50 uv50 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.223+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=1 2024-05-08T18:05:26.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.223+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:282 6.3 6:c623b173:gc::gc.16:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.224+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 55 ==== osd_op_reply(282 gc.16 [call] v27'54 uv54 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.224+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:283 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.224+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 56 ==== osd_op_reply(283 gc.16 [call out=15b] v0'0 uv54 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.224+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.224+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:284 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.224+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 57 ==== osd_op_reply(284 gc.16 [call out=24b] v0'0 uv54 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.224+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:285 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.224+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 58 ==== osd_op_reply(285 gc.16 [call out=15b] v0'0 uv54 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.224+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.224+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:286 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.225+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 59 ==== osd_op_reply(286 gc.16 [call out=15b] v0'0 uv54 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.225+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.223 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.225+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:287 6.3 6:c623b173:gc::gc.16:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x7fad88037040 2024-05-08T18:05:26.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.225+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 60 ==== osd_op_reply(287 gc.16 [call] v27'55 uv55 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.225+0000 7fad9640e800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=1 2024-05-08T18:05:26.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.225+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:288 6.16 6:6849fd06:gc::gc.17:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.226+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 95 ==== osd_op_reply(288 gc.17 [call] v27'20 uv20 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.226+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:289 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.227+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 96 ==== osd_op_reply(289 gc.17 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.227+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.227+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:290 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.227+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 97 ==== osd_op_reply(290 gc.17 [call out=24b] v0'0 uv20 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.227+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:291 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.227+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 98 ==== osd_op_reply(291 gc.17 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.227+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:05:26.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.227+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:292 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.228+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 99 ==== osd_op_reply(292 gc.17 [call out=15b] v0'0 uv20 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.228+0000 7fad9640e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:05:26.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.228+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:293 6.16 6:6849fd06:gc::gc.17:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126be20e0 2024-05-08T18:05:26.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.229+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 100 ==== osd_op_reply(293 gc.17 [call] v27'21 uv21 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.229+0000 7fac49fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:26.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.230+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:294 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93944469701328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126c25360 con 0x557126bd8d40 2024-05-08T18:05:26.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.230+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 65 ==== osd_op_reply(294 data_loggenerations_metadata [watch unwatch cookie 93944469701328] v27'167 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.230+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:295 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93944469677168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126bec2a0 con 0x7fad88037040 2024-05-08T18:05:26.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.231+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 61 ==== osd_op_reply(295 notify.0 [watch unwatch cookie 93944469677168] v27'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.231+0000 7fad9640e800 20 remove_watcher() i=0 2024-05-08T18:05:26.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.231+0000 7fad9640e800 2 removed watcher, disabling cache 2024-05-08T18:05:26.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.231+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:296 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93944469681584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126be6470 con 0x557126bd8d40 2024-05-08T18:05:26.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.233+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 66 ==== osd_op_reply(296 notify.1 [watch unwatch cookie 93944469681584] v27'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.233+0000 7fad9640e800 20 remove_watcher() i=1 2024-05-08T18:05:26.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.233+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:297 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93944469684336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126be6470 con 0x557126be20e0 2024-05-08T18:05:26.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.234+0000 7fad93705640 1 -- 172.21.15.50:0/1847279976 <== osd.3 v2:172.21.15.50:6800/3130492744 101 ==== osd_op_reply(297 notify.2 [watch unwatch cookie 93944469684336] v27'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad88009050 con 0x557126be20e0 2024-05-08T18:05:26.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.234+0000 7fad9640e800 20 remove_watcher() i=2 2024-05-08T18:05:26.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.234+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:298 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93944469686656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126be6470 con 0x7fad88037040 2024-05-08T18:05:26.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.234+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.2 v2:172.21.15.50:6811/3039646617 62 ==== osd_op_reply(298 notify.3 [watch unwatch cookie 93944469686656] v27'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x7fad88037040 2024-05-08T18:05:26.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.234+0000 7fad9640e800 20 remove_watcher() i=3 2024-05-08T18:05:26.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.234+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:299 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93944469691456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126be8980 con 0x557126bd8d40 2024-05-08T18:05:26.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.236+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 <== osd.1 v2:172.21.15.50:6808/2141923372 67 ==== osd_op_reply(299 notify.4 [watch unwatch cookie 93944469691456] v27'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad8c040040 con 0x557126bd8d40 2024-05-08T18:05:26.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.236+0000 7fad9640e800 20 remove_watcher() i=4 2024-05-08T18:05:26.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.236+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:300 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93944469692512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126be6470 con 0x557126bdd730 2024-05-08T18:05:26.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.237+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 70 ==== osd_op_reply(300 notify.5 [watch unwatch cookie 93944469692512] v27'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.237+0000 7fad9640e800 20 remove_watcher() i=5 2024-05-08T18:05:26.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.237+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:301 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93944469699808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126be8980 con 0x557126bdd730 2024-05-08T18:05:26.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.238+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 71 ==== osd_op_reply(301 notify.6 [watch unwatch cookie 93944469699808] v27'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.238+0000 7fad9640e800 20 remove_watcher() i=6 2024-05-08T18:05:26.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.238+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:302 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93944469695344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x557126be6470 con 0x557126bdd730 2024-05-08T18:05:26.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad93f06640 1 -- 172.21.15.50:0/1847279976 <== osd.0 v2:172.21.15.46:6800/337143975 72 ==== osd_op_reply(302 notify.7 [watch unwatch cookie 93944469695344] v27'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fad840a0070 con 0x557126bdd730 2024-05-08T18:05:26.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 20 remove_watcher() i=7 2024-05-08T18:05:26.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557126bdd730 msgr2=0x557126bdfb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557126bdd730 0x557126bdfb20 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557126bd8d40 msgr2=0x557126bdb200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557126bd8d40 0x557126bdb200 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fad88037040 msgr2=0x7fad88036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fad88037040 0x7fad88036410 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557126be20e0 msgr2=0x557126be45a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557126be20e0 0x557126be45a0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac5c03bb50 msgr2=0x7fac5c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac5c03bb50 0x7fac5c03e010 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fad8800efd0 tx=0x7fad88010000 comp rx=0 tx=0).stop 2024-05-08T18:05:26.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad94707640 1 -- 172.21.15.50:0/1847279976 reap_dead start 2024-05-08T18:05:26.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a14c00 msgr2=0x557126bcd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a14c00 0x557126bcd5e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fad84070f90 tx=0x7fad84091730 comp rx=0 tx=0).stop 2024-05-08T18:05:26.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 shutdown_connections 2024-05-08T18:05:26.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 --2- 172.21.15.50:0/1847279976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a14c00 0x557126bcd5e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 >> 172.21.15.50:0/1847279976 conn(0x557126bc02e0 msgr2=0x557126bbd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 shutdown_connections 2024-05-08T18:05:26.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.239+0000 7fad9640e800 1 -- 172.21.15.50:0/1847279976 wait complete. 2024-05-08T18:05:26.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 -- 172.21.15.50:0/2617423776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac6c03bba0 msgr2=0x7fac6c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 --2- 172.21.15.50:0/2617423776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac6c03bba0 0x7fac6c03e060 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fad880016b0 tx=0x7fad8800f000 comp rx=0 tx=0).stop 2024-05-08T18:05:26.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 -- 172.21.15.50:0/2617423776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126bb6eb0 msgr2=0x557126bb72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 --2- 172.21.15.50:0/2617423776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126bb6eb0 0x557126bb72d0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fad84092a30 tx=0x7fad8405b5a0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 -- 172.21.15.50:0/2617423776 shutdown_connections 2024-05-08T18:05:26.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 --2- 172.21.15.50:0/2617423776 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac6c03bba0 0x7fac6c03e060 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 --2- 172.21.15.50:0/2617423776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126bb6eb0 0x557126bb72d0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 -- 172.21.15.50:0/2617423776 >> 172.21.15.50:0/2617423776 conn(0x557126baaf50 msgr2=0x557126bab350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 -- 172.21.15.50:0/2617423776 shutdown_connections 2024-05-08T18:05:26.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 -- 172.21.15.50:0/2617423776 wait complete. 2024-05-08T18:05:26.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 -- 172.21.15.50:0/4158942153 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557126ba1fe0 msgr2=0x557126ba4470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557126ba1fe0 0x557126ba4470 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 -- 172.21.15.50:0/4158942153 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557126aa08a0 msgr2=0x557126b9f880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557126aa08a0 0x557126b9f880 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 -- 172.21.15.50:0/4158942153 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557126ba6060 msgr2=0x557126ba8450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557126ba6060 0x557126ba8450 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 -- 172.21.15.50:0/4158942153 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac7c03bb50 msgr2=0x7fac7c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac7c03bb50 0x7fac7c03e010 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fad88001d70 tx=0x7fad880012d0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 -- 172.21.15.50:0/4158942153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 msgr2=0x557126b98640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 --2- 172.21.15.50:0/4158942153 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557126a9f6e0 0x557126b98640 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fad8405f040 tx=0x7fad84002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad94707640 1 -- 172.21.15.50:0/4158942153 reap_dead start 2024-05-08T18:05:26.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 -- 172.21.15.50:0/4158942153 shutdown_connections 2024-05-08T18:05:26.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 -- 172.21.15.50:0/4158942153 >> 172.21.15.50:0/4158942153 conn(0x557126a9ac60 msgr2=0x557126b8ad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 -- 172.21.15.50:0/4158942153 shutdown_connections 2024-05-08T18:05:26.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.240+0000 7fad9640e800 1 -- 172.21.15.50:0/4158942153 wait complete. 2024-05-08T18:05:26.247 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2024-05-08T18:05:26.247 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2024-05-08T18:05:26.248 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2024-05-08T18:05:26.332 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:26.332 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:26.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.335+0000 7fae0d11f800 1 Processor -- start 2024-05-08T18:05:26.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.335+0000 7fae0d11f800 1 -- start start 2024-05-08T18:05:26.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.335+0000 7fae0d11f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x563c1f5269e0 0x563c1f528dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.335+0000 7fae0d11f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x563c1f5293a0 con 0x563c1f522550 2024-05-08T18:05:26.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.335+0000 7fae0d11f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x563c1f3c3e20 con 0x563c1f5269e0 2024-05-08T18:05:26.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.336+0000 7fae085c2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x563c1f5269e0 0x563c1f528dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.336+0000 7fae08dc3640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x563c1f522550 0x563c1f522970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:60348/0 (socket says 172.21.15.50:60348) 2024-05-08T18:05:26.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.336+0000 7fae08dc3640 1 -- 172.21.15.50:0/2326202988 learned_addr learned my addr 172.21.15.50:0/2326202988 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:26.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.336+0000 7fae085c2640 1 -- 172.21.15.50:0/2326202988 >> v1:172.21.15.46:6789/0 conn(0x563c1f522550 legacy=0x563c1f522970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.334 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.336+0000 7fae085c2640 1 -- 172.21.15.50:0/2326202988 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c1f52a5d0 con 0x563c1f5269e0 2024-05-08T18:05:26.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.338+0000 7fae085c2640 1 --2- 172.21.15.50:0/2326202988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f5269e0 0x563c1f528dd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00a470 tx=0x7fadfc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc61b2c912f02e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.338+0000 7fae07dc1640 1 -- 172.21.15.50:0/2326202988 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadfc057070 con 0x563c1f5269e0 2024-05-08T18:05:26.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.338+0000 7fae07dc1640 1 -- 172.21.15.50:0/2326202988 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadfc052570 con 0x563c1f5269e0 2024-05-08T18:05:26.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.338+0000 7fae07dc1640 1 -- 172.21.15.50:0/2326202988 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadfc052870 con 0x563c1f5269e0 2024-05-08T18:05:26.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.338+0000 7fae0d11f800 1 -- 172.21.15.50:0/2326202988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f5269e0 msgr2=0x563c1f528dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.338+0000 7fae0d11f800 1 --2- 172.21.15.50:0/2326202988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f5269e0 0x563c1f528dd0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fadfc00a470 tx=0x7fadfc01ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:26.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.338+0000 7fae0d11f800 1 -- 172.21.15.50:0/2326202988 shutdown_connections 2024-05-08T18:05:26.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.338+0000 7fae0d11f800 1 --2- 172.21.15.50:0/2326202988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f5269e0 0x563c1f528dd0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.339+0000 7fae0d11f800 1 -- 172.21.15.50:0/2326202988 >> 172.21.15.50:0/2326202988 conn(0x563c1f51db50 msgr2=0x563c1f51ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.339+0000 7fae0d11f800 1 -- 172.21.15.50:0/2326202988 shutdown_connections 2024-05-08T18:05:26.337 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.339+0000 7fae0d11f800 1 -- 172.21.15.50:0/2326202988 wait complete. 2024-05-08T18:05:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.354+0000 7fae0d11f800 1 Processor -- start 2024-05-08T18:05:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.354+0000 7fae0d11f800 1 -- start start 2024-05-08T18:05:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.354+0000 7fae0d11f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f522550 0x563c1f6128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.354+0000 7fae0d11f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563c1f46ea10 con 0x563c1f522550 2024-05-08T18:05:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.354+0000 7fae08dc3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f522550 0x563c1f6128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.354+0000 7fae08dc3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f522550 0x563c1f6128f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43720/0 (socket says 172.21.15.50:43720) 2024-05-08T18:05:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.354+0000 7fae08dc3640 1 -- 172.21.15.50:0/4090331818 learned_addr learned my addr 172.21.15.50:0/4090331818 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.354+0000 7fae08dc3640 1 -- 172.21.15.50:0/4090331818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c1f612ec0 con 0x563c1f522550 2024-05-08T18:05:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.355+0000 7fae08dc3640 1 --2- 172.21.15.50:0/4090331818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f522550 0x563c1f6128f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fadf8001e40 tx=0x7fadf8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6749db0da672ec98 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.355+0000 7facf6e0d640 1 -- 172.21.15.50:0/4090331818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf8012020 con 0x563c1f522550 2024-05-08T18:05:26.353 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.355+0000 7facf6e0d640 1 -- 172.21.15.50:0/4090331818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf800e5a0 con 0x563c1f522550 2024-05-08T18:05:26.354 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.355+0000 7facf6e0d640 1 -- 172.21.15.50:0/4090331818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf800e8a0 con 0x563c1f522550 2024-05-08T18:05:26.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.355+0000 7fae0d11f800 1 -- 172.21.15.50:0/4090331818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f522550 msgr2=0x563c1f6128f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.355+0000 7fae0d11f800 1 --2- 172.21.15.50:0/4090331818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f522550 0x563c1f6128f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fadf8001e40 tx=0x7fadf8001e70 comp rx=0 tx=0).stop 2024-05-08T18:05:26.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.356+0000 7fae0d11f800 1 -- 172.21.15.50:0/4090331818 shutdown_connections 2024-05-08T18:05:26.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.356+0000 7fae0d11f800 1 --2- 172.21.15.50:0/4090331818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f522550 0x563c1f6128f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.356+0000 7fae0d11f800 1 -- 172.21.15.50:0/4090331818 >> 172.21.15.50:0/4090331818 conn(0x563c1f51db50 msgr2=0x563c1f60e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.356+0000 7fae0d11f800 1 -- 172.21.15.50:0/4090331818 shutdown_connections 2024-05-08T18:05:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.356+0000 7fae0d11f800 1 -- 172.21.15.50:0/4090331818 wait complete. 2024-05-08T18:05:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.356+0000 7fae0d11f800 1 Processor -- start 2024-05-08T18:05:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.356+0000 7fae0d11f800 1 -- start start 2024-05-08T18:05:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.356+0000 7fae0d11f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f522550 0x563c1f61a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.356+0000 7fae0d11f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563c1f529870 con 0x563c1f522550 2024-05-08T18:05:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.356+0000 7fae08dc3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f522550 0x563c1f61a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.356+0000 7fae08dc3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f522550 0x563c1f61a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43734/0 (socket says 172.21.15.50:43734) 2024-05-08T18:05:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.356+0000 7fae08dc3640 1 -- 172.21.15.50:0/433930700 learned_addr learned my addr 172.21.15.50:0/433930700 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.357+0000 7fae08dc3640 1 -- 172.21.15.50:0/433930700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c1f61a9d0 con 0x563c1f522550 2024-05-08T18:05:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.357+0000 7fae08dc3640 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f522550 0x563c1f61a490 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fadf8000c00 tx=0x7fadf8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.357+0000 7facf560a640 1 -- 172.21.15.50:0/433930700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf8012030 con 0x563c1f522550 2024-05-08T18:05:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.357+0000 7fae0d11f800 1 -- 172.21.15.50:0/433930700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563c1f61ac60 con 0x563c1f522550 2024-05-08T18:05:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.357+0000 7facf560a640 1 -- 172.21.15.50:0/433930700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf800ee50 con 0x563c1f522550 2024-05-08T18:05:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.357+0000 7facf560a640 1 -- 172.21.15.50:0/433930700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf801a940 con 0x563c1f522550 2024-05-08T18:05:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.357+0000 7fae0d11f800 1 -- 172.21.15.50:0/433930700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c1f61d910 con 0x563c1f522550 2024-05-08T18:05:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.358+0000 7facf560a640 1 -- 172.21.15.50:0/433930700 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fadf8015020 con 0x563c1f522550 2024-05-08T18:05:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.358+0000 7facf560a640 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7facf003bb50 0x7facf003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.358+0000 7facf560a640 1 -- 172.21.15.50:0/433930700 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fadf8020030 con 0x563c1f522550 2024-05-08T18:05:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.358+0000 7fae085c2640 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7facf003bb50 0x7facf003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.358+0000 7fae0d11f800 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x563c1f5269e0 0x563c1f6216d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.358+0000 7fae0d11f800 1 -- 172.21.15.50:0/433930700 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f621c60 con 0x563c1f5269e0 2024-05-08T18:05:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.359+0000 7fae095c4640 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x563c1f5269e0 0x563c1f6216d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.359+0000 7fae085c2640 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7facf003bb50 0x7facf003e010 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fadfc054000 tx=0x7fadfc04e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.359+0000 7fae095c4640 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x563c1f5269e0 0x563c1f6216d0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.359+0000 7fae095c4640 1 -- 172.21.15.50:0/433930700 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f5269e0 2024-05-08T18:05:26.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.359+0000 7fae0d11f800 1 -- 172.21.15.50:0/433930700 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f61eca0 con 0x563c1f5269e0 2024-05-08T18:05:26.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.359+0000 7fae095c4640 1 -- 172.21.15.50:0/433930700 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f5269e0 2024-05-08T18:05:26.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.359+0000 7fae0d11f800 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x563c1f623fb0 0x563c1f6263a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.359+0000 7fae0d11f800 1 -- 172.21.15.50:0/433930700 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f626a90 con 0x563c1f623fb0 2024-05-08T18:05:26.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.360+0000 7fae08dc3640 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x563c1f623fb0 0x563c1f6263a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.360+0000 7fae08dc3640 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x563c1f623fb0 0x563c1f6263a0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.360+0000 7fae08dc3640 1 -- 172.21.15.50:0/433930700 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fadf803fd70 con 0x563c1f623fb0 2024-05-08T18:05:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.360+0000 7fae0d11f800 1 -- 172.21.15.50:0/433930700 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f622e50 con 0x563c1f5269e0 2024-05-08T18:05:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.361+0000 7fae095c4640 1 -- 172.21.15.50:0/433930700 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f5269e0 2024-05-08T18:05:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.361+0000 7fae0d11f800 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x563c1f627ef0 0x563c1f62a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.361+0000 7fae0d11f800 1 -- 172.21.15.50:0/433930700 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f62a9e0 con 0x563c1f627ef0 2024-05-08T18:05:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.361+0000 7fae085c2640 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x563c1f627ef0 0x563c1f62a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.361+0000 7fae085c2640 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x563c1f627ef0 0x563c1f62a2e0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.361+0000 7fae085c2640 1 -- 172.21.15.50:0/433930700 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fadfc052940 con 0x563c1f627ef0 2024-05-08T18:05:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.362+0000 7fae0d11f800 1 Processor -- start 2024-05-08T18:05:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.362+0000 7fae0d11f800 1 -- start start 2024-05-08T18:05:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.362+0000 7fae0d11f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f638d50 0x563c1f639170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.362+0000 7fae0d11f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563c1f415330 con 0x563c1f638d50 2024-05-08T18:05:26.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.362+0000 7fae08dc3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f638d50 0x563c1f639170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.362+0000 7fae08dc3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f638d50 0x563c1f639170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43742/0 (socket says 172.21.15.50:43742) 2024-05-08T18:05:26.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.362+0000 7fae08dc3640 1 -- 172.21.15.50:0/1736428341 learned_addr learned my addr 172.21.15.50:0/1736428341 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.362+0000 7fae08dc3640 1 -- 172.21.15.50:0/1736428341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c1f6396b0 con 0x563c1f638d50 2024-05-08T18:05:26.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.363+0000 7fae08dc3640 1 --2- 172.21.15.50:0/1736428341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f638d50 0x563c1f639170 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fadf80510b0 tx=0x7fadf8043500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.363+0000 7facee7fc640 1 -- 172.21.15.50:0/1736428341 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf8018a80 con 0x563c1f638d50 2024-05-08T18:05:26.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.363+0000 7facee7fc640 1 -- 172.21.15.50:0/1736428341 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf8047070 con 0x563c1f638d50 2024-05-08T18:05:26.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.363+0000 7fae0d11f800 1 -- 172.21.15.50:0/1736428341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563c1f63a120 con 0x563c1f638d50 2024-05-08T18:05:26.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.363+0000 7facee7fc640 1 -- 172.21.15.50:0/1736428341 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf803d9c0 con 0x563c1f638d50 2024-05-08T18:05:26.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.363+0000 7fae0d11f800 1 -- 172.21.15.50:0/1736428341 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c1f63a120 con 0x563c1f638d50 2024-05-08T18:05:26.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.364+0000 7facee7fc640 1 -- 172.21.15.50:0/1736428341 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fadf803db60 con 0x563c1f638d50 2024-05-08T18:05:26.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.364+0000 7facee7fc640 1 --2- 172.21.15.50:0/1736428341 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7face003bba0 0x7face003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.364+0000 7facee7fc640 1 -- 172.21.15.50:0/1736428341 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fadf8038cd0 con 0x563c1f638d50 2024-05-08T18:05:26.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.364+0000 7fae085c2640 1 --2- 172.21.15.50:0/1736428341 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7face003bba0 0x7face003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.364+0000 7fae0d11f800 1 Processor -- start 2024-05-08T18:05:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.364+0000 7fae0d11f800 1 -- start start 2024-05-08T18:05:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.364+0000 7fae085c2640 1 --2- 172.21.15.50:0/1736428341 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7face003bba0 0x7face003e060 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x563c1f622560 tx=0x7fadfc005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.364+0000 7fae0d11f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f645c60 0x563c1f646080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.364+0000 7fae0d11f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563c1f613240 con 0x563c1f645c60 2024-05-08T18:05:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.365+0000 7fae08dc3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f645c60 0x563c1f646080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.365+0000 7fae08dc3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f645c60 0x563c1f646080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43756/0 (socket says 172.21.15.50:43756) 2024-05-08T18:05:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.365+0000 7fae08dc3640 1 -- 172.21.15.50:0/2774259862 learned_addr learned my addr 172.21.15.50:0/2774259862 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.365+0000 7fae08dc3640 1 -- 172.21.15.50:0/2774259862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c1f646650 con 0x563c1f645c60 2024-05-08T18:05:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.365+0000 7fae08dc3640 1 --2- 172.21.15.50:0/2774259862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f645c60 0x563c1f646080 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fadf804bfd0 tx=0x7fadf8043d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161272fdd22d8fb7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.365+0000 7facdffff640 1 -- 172.21.15.50:0/2774259862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf80225b0 con 0x563c1f645c60 2024-05-08T18:05:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.365+0000 7facdffff640 1 -- 172.21.15.50:0/2774259862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf8022770 con 0x563c1f645c60 2024-05-08T18:05:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.365+0000 7facdffff640 1 -- 172.21.15.50:0/2774259862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf803fb70 con 0x563c1f645c60 2024-05-08T18:05:26.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.366+0000 7fae0d11f800 1 -- 172.21.15.50:0/2774259862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f645c60 msgr2=0x563c1f646080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.366+0000 7fae0d11f800 1 --2- 172.21.15.50:0/2774259862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f645c60 0x563c1f646080 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fadf804bfd0 tx=0x7fadf8043d70 comp rx=0 tx=0).stop 2024-05-08T18:05:26.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.366+0000 7fae0d11f800 1 -- 172.21.15.50:0/2774259862 shutdown_connections 2024-05-08T18:05:26.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.366+0000 7fae0d11f800 1 --2- 172.21.15.50:0/2774259862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f645c60 0x563c1f646080 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.366+0000 7fae0d11f800 1 -- 172.21.15.50:0/2774259862 >> 172.21.15.50:0/2774259862 conn(0x563c1f641f80 msgr2=0x563c1f63ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.366+0000 7fae0d11f800 1 -- 172.21.15.50:0/2774259862 shutdown_connections 2024-05-08T18:05:26.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.366+0000 7fae0d11f800 1 -- 172.21.15.50:0/2774259862 wait complete. 2024-05-08T18:05:26.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.366+0000 7fae0d11f800 1 Processor -- start 2024-05-08T18:05:26.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.366+0000 7fae0d11f800 1 -- start start 2024-05-08T18:05:26.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.366+0000 7fae0d11f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f645c60 0x563c1f64d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.366+0000 7fae0d11f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563c1f646d70 con 0x563c1f645c60 2024-05-08T18:05:26.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.367+0000 7fae08dc3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f645c60 0x563c1f64d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.367+0000 7fae08dc3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f645c60 0x563c1f64d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43764/0 (socket says 172.21.15.50:43764) 2024-05-08T18:05:26.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.367+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 learned_addr learned my addr 172.21.15.50:0/1461934309 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.367+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c1f64d5f0 con 0x563c1f645c60 2024-05-08T18:05:26.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.367+0000 7fae08dc3640 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f645c60 0x563c1f64d0b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fadf8022b40 tx=0x7fadf800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.367+0000 7facde7fc640 1 -- 172.21.15.50:0/1461934309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf8056040 con 0x563c1f645c60 2024-05-08T18:05:26.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.367+0000 7facde7fc640 1 -- 172.21.15.50:0/1461934309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadf803e000 con 0x563c1f645c60 2024-05-08T18:05:26.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.367+0000 7facde7fc640 1 -- 172.21.15.50:0/1461934309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadf80223d0 con 0x563c1f645c60 2024-05-08T18:05:26.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.367+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563c1f64d5f0 con 0x563c1f645c60 2024-05-08T18:05:26.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.367+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c1f64d5f0 con 0x563c1f645c60 2024-05-08T18:05:26.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.368+0000 7facde7fc640 1 -- 172.21.15.50:0/1461934309 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fadf803e1b0 con 0x563c1f645c60 2024-05-08T18:05:26.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.368+0000 7facde7fc640 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7facd403bba0 0x7facd403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.368+0000 7facde7fc640 1 -- 172.21.15.50:0/1461934309 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fadf806ecb0 con 0x563c1f645c60 2024-05-08T18:05:26.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.368+0000 7fae085c2640 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7facd403bba0 0x7facd403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.368+0000 7fae0d11f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.369+0000 7fae0d11f800 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x563c1f658590 0x563c1f65aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.369+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65b150 con 0x563c1f658590 2024-05-08T18:05:26.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.369+0000 7fae095c4640 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x563c1f658590 0x563c1f65aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.369+0000 7fae085c2640 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7facd403bba0 0x7facd403e060 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fadfc052fd0 tx=0x7fadfc04f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.369+0000 7fae095c4640 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x563c1f658590 0x563c1f65aa50 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.369+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f658590 2024-05-08T18:05:26.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.369+0000 7fae0d11f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.369+0000 7fae0d11f800 20 realm 2024-05-08T18:05:26.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.369+0000 7fae0d11f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.369+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65b560 con 0x563c1f658590 2024-05-08T18:05:26.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.369+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f658590 2024-05-08T18:05:26.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.370+0000 7fae0d11f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.370+0000 7fae0d11f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:26.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.370+0000 7fae0d11f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.370+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65be30 con 0x563c1f658590 2024-05-08T18:05:26.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.370+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f658590 2024-05-08T18:05:26.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.370+0000 7fae0d11f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.370+0000 7fae0d11f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.370+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65c1d0 con 0x563c1f658590 2024-05-08T18:05:26.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.370+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f658590 2024-05-08T18:05:26.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.370+0000 7fae0d11f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:26.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.370+0000 7fae0d11f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.370+0000 7fae0d11f800 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x563c1f65ccc0 0x563c1f65f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.370+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65f810 con 0x563c1f65ccc0 2024-05-08T18:05:26.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.370+0000 7fae08dc3640 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x563c1f65ccc0 0x563c1f65f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.371+0000 7fae08dc3640 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x563c1f65ccc0 0x563c1f65f0b0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.371+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.371+0000 7fae0d11f800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:26.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.371+0000 7fae0d11f800 20 searching for the correct realm 2024-05-08T18:05:26.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.371+0000 7fae0d11f800 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x563c1f6618b0 0x563c1f663d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.371+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563c1f664440 con 0x563c1f6618b0 2024-05-08T18:05:26.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.371+0000 7fae085c2640 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x563c1f6618b0 0x563c1f663d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.371+0000 7fae085c2640 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x563c1f6618b0 0x563c1f663d70 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.372+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.372+0000 7fae085c2640 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fadfc076040 0x7fadfc083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.372+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563c1f664440 con 0x7fadfc076040 2024-05-08T18:05:26.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.372+0000 7fae095c4640 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fadfc076040 0x7fadfc083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.372+0000 7fae095c4640 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fadfc076040 0x7fadfc083410 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.372+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae0000a4a0 con 0x7fadfc076040 2024-05-08T18:05:26.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.372+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65c1d0 con 0x563c1f65ccc0 2024-05-08T18:05:26.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.373+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.373+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65c1d0 con 0x563c1f658590 2024-05-08T18:05:26.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.373+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f658590 2024-05-08T18:05:26.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.373+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65c1d0 con 0x563c1f658590 2024-05-08T18:05:26.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.373+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f658590 2024-05-08T18:05:26.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.373+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65c1d0 con 0x563c1f65ccc0 2024-05-08T18:05:26.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.374+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.374+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65c1d0 con 0x563c1f65ccc0 2024-05-08T18:05:26.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.374+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.374+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65c1d0 con 0x7fadfc076040 2024-05-08T18:05:26.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.374+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae0000a4a0 con 0x7fadfc076040 2024-05-08T18:05:26.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.374+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65c1d0 con 0x563c1f6618b0 2024-05-08T18:05:26.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.374+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.374+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65c1d0 con 0x563c1f658590 2024-05-08T18:05:26.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.375+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f658590 2024-05-08T18:05:26.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.375+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65c1d0 con 0x563c1f65ccc0 2024-05-08T18:05:26.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.375+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.375+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65c1d0 con 0x563c1f658590 2024-05-08T18:05:26.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.375+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f658590 2024-05-08T18:05:26.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.375+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65c1d0 con 0x7fadfc076040 2024-05-08T18:05:26.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.375+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fae0000a4a0 con 0x7fadfc076040 2024-05-08T18:05:26.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.375+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563c1f65c1d0 con 0x563c1f6618b0 2024-05-08T18:05:26.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.376+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.376+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x563c1f6618b0 2024-05-08T18:05:26.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.376+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.376+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x563c1f65ccc0 2024-05-08T18:05:26.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.376+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.376+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x563c1f6618b0 2024-05-08T18:05:26.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.377+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.377+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x563c1f6618b0 2024-05-08T18:05:26.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.377+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.377+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x563c1f658590 2024-05-08T18:05:26.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.377+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f658590 2024-05-08T18:05:26.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.377+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x563c1f658590 2024-05-08T18:05:26.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.377+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f658590 2024-05-08T18:05:26.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.377+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x563c1f65ccc0 2024-05-08T18:05:26.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.378+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.378+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x563c1f6618b0 2024-05-08T18:05:26.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.378+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.378+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x7fadfc076040 2024-05-08T18:05:26.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.378+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae0000a4a0 con 0x7fadfc076040 2024-05-08T18:05:26.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.378+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x563c1f658590 2024-05-08T18:05:26.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.378+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f658590 2024-05-08T18:05:26.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.378+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x563c1f65ccc0 2024-05-08T18:05:26.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.379+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.379+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x563c1f6618b0 2024-05-08T18:05:26.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.379+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.379+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x563c1f65ccc0 2024-05-08T18:05:26.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.380+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.380+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x7fadfc076040 2024-05-08T18:05:26.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.380+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae0000a4a0 con 0x7fadfc076040 2024-05-08T18:05:26.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.380+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x563c1f6618b0 2024-05-08T18:05:26.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.380+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.380+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x563c1f65ccc0 2024-05-08T18:05:26.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.380+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.380+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x563c1f658590 2024-05-08T18:05:26.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f658590 2024-05-08T18:05:26.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fadfc090040 con 0x7fadfc076040 2024-05-08T18:05:26.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fae0000a4a0 con 0x7fadfc076040 2024-05-08T18:05:26.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae0d11f800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:26.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae0d11f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:26.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae0d11f800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:26.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae0d11f800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:26.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae0d11f800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:26.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae0d11f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:26.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae0d11f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6601b0 con 0x563c1f658590 2024-05-08T18:05:26.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f658590 2024-05-08T18:05:26.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae0d11f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae0d11f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6601b0 con 0x563c1f658590 2024-05-08T18:05:26.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fae0000a4a0 con 0x563c1f658590 2024-05-08T18:05:26.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae0d11f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:26.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae0d11f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.381+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f660160 con 0x563c1f6618b0 2024-05-08T18:05:26.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae0d11f800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:26.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae0d11f800 20 zone default found 2024-05-08T18:05:26.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae0d11f800 4 Realm: () 2024-05-08T18:05:26.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae0d11f800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:26.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae0d11f800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:26.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae0d11f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:26.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae0d11f800 20 zonegroup default 2024-05-08T18:05:26.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae0d11f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f660a50 con 0x563c1f65ccc0 2024-05-08T18:05:26.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae0d11f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae0d11f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6662a0 con 0x563c1f6618b0 2024-05-08T18:05:26.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae0d11f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae0d11f800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.382+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f666640 con 0x563c1f6618b0 2024-05-08T18:05:26.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.383+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.383+0000 7fae0d11f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.383+0000 7fae0d11f800 20 started sync module instance, tier type = 2024-05-08T18:05:26.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.383+0000 7fae0d11f800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:26.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.383+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f665740 con 0x7fadfc076040 2024-05-08T18:05:26.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.384+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae0000a4a0 con 0x7fadfc076040 2024-05-08T18:05:26.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.384+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94816224828560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f665740 con 0x7fadfc076040 2024-05-08T18:05:26.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.384+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6670a0 con 0x563c1f658590 2024-05-08T18:05:26.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.385+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94816224828560] v27'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae00040040 con 0x7fadfc076040 2024-05-08T18:05:26.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.385+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.385+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94816224832384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f665740 con 0x563c1f658590 2024-05-08T18:05:26.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.385+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f667b30 con 0x563c1f6618b0 2024-05-08T18:05:26.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.386+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94816224832384] v27'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.386+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.386+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94816224826000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6656b0 con 0x563c1f6618b0 2024-05-08T18:05:26.396 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.386+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6689f0 con 0x7fadfc076040 2024-05-08T18:05:26.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.387+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94816224826000] v27'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.387+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae00040040 con 0x7fadfc076040 2024-05-08T18:05:26.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.387+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94816224838976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f668e10 con 0x7fadfc076040 2024-05-08T18:05:26.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.387+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6697d0 con 0x563c1f658590 2024-05-08T18:05:26.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.388+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.388+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94816224842528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f669bf0 con 0x563c1f658590 2024-05-08T18:05:26.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.388+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f66a670 con 0x563c1f65ccc0 2024-05-08T18:05:26.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.388+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94816224838976] v27'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae00040040 con 0x7fadfc076040 2024-05-08T18:05:26.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.389+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94816224842528] v27'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.389+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.389+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94816224840032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f669bf0 con 0x563c1f65ccc0 2024-05-08T18:05:26.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.390+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f66af40 con 0x563c1f65ccc0 2024-05-08T18:05:26.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.390+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94816224840032] v27'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.391+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.391+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94816224843584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f66b790 con 0x563c1f65ccc0 2024-05-08T18:05:26.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.391+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f66c040 con 0x563c1f65ccc0 2024-05-08T18:05:26.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.392+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v27'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.392+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94816224843584] v27'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.392+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94816224852784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f66c2a0 con 0x563c1f65ccc0 2024-05-08T18:05:26.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.392+0000 7fae0d11f800 20 add_watcher() i=0 2024-05-08T18:05:26.401 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.392+0000 7fae0d11f800 20 add_watcher() i=1 2024-05-08T18:05:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.392+0000 7fae0d11f800 20 add_watcher() i=2 2024-05-08T18:05:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.392+0000 7fae0d11f800 20 add_watcher() i=3 2024-05-08T18:05:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.392+0000 7fae0d11f800 20 add_watcher() i=4 2024-05-08T18:05:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.392+0000 7fae0d11f800 20 add_watcher() i=5 2024-05-08T18:05:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.392+0000 7fae0d11f800 20 add_watcher() i=6 2024-05-08T18:05:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.393+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94816224852784] v27'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.393+0000 7fae0d11f800 20 add_watcher() i=7 2024-05-08T18:05:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.393+0000 7fae0d11f800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.393+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f66f7e0 con 0x563c1f658590 2024-05-08T18:05:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.393+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.393+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94816224850656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f66ccd0 con 0x563c1f658590 2024-05-08T18:05:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.394+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94816224850656] v27'168 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.394+0000 7facbefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:26.402 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.394+0000 7fae0d11f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.394+0000 7fae0d11f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.394+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f677dc0 con 0x563c1f658590 2024-05-08T18:05:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.396+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'28 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.396+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f678410 con 0x563c1f65ccc0 2024-05-08T18:05:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.397+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'23 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.397+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f678830 con 0x563c1f6618b0 2024-05-08T18:05:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.398+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'22 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.398+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f678a30 con 0x563c1f658590 2024-05-08T18:05:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.399+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'169 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.399+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f675fb0 con 0x7fadfc076040 2024-05-08T18:05:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.400+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'56 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fae00040040 con 0x7fadfc076040 2024-05-08T18:05:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.400+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f675fb0 con 0x563c1f6618b0 2024-05-08T18:05:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.400+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'37 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.400+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679760 con 0x563c1f658590 2024-05-08T18:05:26.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.401+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'24 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.402+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679cd0 con 0x563c1f65ccc0 2024-05-08T18:05:26.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.402+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'37 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.403+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679cd0 con 0x7fadfc076040 2024-05-08T18:05:26.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.403+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'22 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fae00040040 con 0x7fadfc076040 2024-05-08T18:05:26.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.403+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679fb0 con 0x563c1f6618b0 2024-05-08T18:05:26.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.404+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'28 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.404+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f67a010 con 0x7fadfc076040 2024-05-08T18:05:26.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.405+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'57 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fae00040040 con 0x7fadfc076040 2024-05-08T18:05:26.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.405+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f67a010 con 0x563c1f65ccc0 2024-05-08T18:05:26.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.406+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'25 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.406+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f67a0a0 con 0x7fadfc076040 2024-05-08T18:05:26.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.407+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'22 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fae00040040 con 0x7fadfc076040 2024-05-08T18:05:26.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.407+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f67a0a0 con 0x563c1f65ccc0 2024-05-08T18:05:26.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.408+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'43 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.409+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f67a0a0 con 0x563c1f658590 2024-05-08T18:05:26.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.409+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'170 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.410+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f67a0a0 con 0x563c1f6618b0 2024-05-08T18:05:26.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.411+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'51 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.411+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f67a160 con 0x7fadfc076040 2024-05-08T18:05:26.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.411+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'58 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fae00040040 con 0x7fadfc076040 2024-05-08T18:05:26.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.411+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f67a160 con 0x563c1f6618b0 2024-05-08T18:05:26.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.412+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'22 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.412+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679be0 con 0x563c1f658590 2024-05-08T18:05:26.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.413+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'30 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.413+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679be0 con 0x563c1f658590 2024-05-08T18:05:26.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.414+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'171 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.414+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679be0 con 0x563c1f6618b0 2024-05-08T18:05:26.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.416+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'52 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.416+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679be0 con 0x7fadfc076040 2024-05-08T18:05:26.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.417+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'32 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fae00040040 con 0x7fadfc076040 2024-05-08T18:05:26.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.417+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679be0 con 0x563c1f6618b0 2024-05-08T18:05:26.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.417+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'24 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.417+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679be0 con 0x563c1f6618b0 2024-05-08T18:05:26.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.418+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'28 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.418+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679be0 con 0x563c1f6618b0 2024-05-08T18:05:26.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.419+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'20 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.419+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679be0 con 0x563c1f6618b0 2024-05-08T18:05:26.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.420+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'38 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.420+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679be0 con 0x563c1f65ccc0 2024-05-08T18:05:26.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.421+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'24 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.421+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679be0 con 0x563c1f6618b0 2024-05-08T18:05:26.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.422+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'22 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.422+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679be0 con 0x563c1f65ccc0 2024-05-08T18:05:26.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.423+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'38 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.424+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679be0 con 0x563c1f6618b0 2024-05-08T18:05:26.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.424+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'53 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.422 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.424+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679be0 con 0x563c1f65ccc0 2024-05-08T18:05:26.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.425+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'44 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.425+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f679be0 con 0x7fadfc076040 2024-05-08T18:05:26.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.426+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'28 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fae00040040 con 0x7fadfc076040 2024-05-08T18:05:26.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.427+0000 7facbdfe3640 20 reqs_thread_entry: start 2024-05-08T18:05:26.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.427+0000 7fae0d11f800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:26.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.427+0000 7fae0d11f800 20 Filter name: none 2024-05-08T18:05:26.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.427+0000 7fac2f7fe640 20 reqs_thread_entry: start 2024-05-08T18:05:26.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.427+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:94 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f658590 2024-05-08T18:05:26.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.427+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(94 gc.0 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.427+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:95 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f658590 2024-05-08T18:05:26.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.428+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(95 gc.0 [call out=24b] v0'0 uv27 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.428+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:96 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f658590 2024-05-08T18:05:26.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.428+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(96 gc.0 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.428+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:97 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f65ccc0 2024-05-08T18:05:26.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.428+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(97 gc.1 [call out=15b] v0'0 uv22 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.428+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:98 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f65ccc0 2024-05-08T18:05:26.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.429+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(98 gc.1 [call out=24b] v0'0 uv22 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.429+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:99 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f65ccc0 2024-05-08T18:05:26.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.429+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(99 gc.1 [call out=15b] v0'0 uv22 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.429+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:100 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.429+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(100 gc.2 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.429+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:101 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f6618b0 2024-05-08T18:05:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.430+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 28 ==== osd_op_reply(101 gc.2 [call out=24b] v0'0 uv21 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.430+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:102 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.430+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 29 ==== osd_op_reply(102 gc.2 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.430+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:103 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f658590 2024-05-08T18:05:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.430+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(103 gc.3 [call out=15b] v0'0 uv164 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.430+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:104 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f658590 2024-05-08T18:05:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.430+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(104 gc.3 [call out=24b] v0'0 uv164 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.430+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:105 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f658590 2024-05-08T18:05:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.431+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 32 ==== osd_op_reply(105 gc.3 [call out=15b] v0'0 uv164 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fae00040040 con 0x563c1f658590 2024-05-08T18:05:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.431+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:106 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x7fadfc076040 2024-05-08T18:05:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.431+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(106 gc.4 [call out=15b] v0'0 uv51 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fae00040040 con 0x7fadfc076040 2024-05-08T18:05:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.431+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:107 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x7fadfc076040 2024-05-08T18:05:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.431+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(107 gc.4 [call out=24b] v0'0 uv51 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fae00040040 con 0x7fadfc076040 2024-05-08T18:05:26.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.431+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:108 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x7fadfc076040 2024-05-08T18:05:26.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.431+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 20 ==== osd_op_reply(108 gc.4 [call out=15b] v0'0 uv51 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.432+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:109 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.432+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 30 ==== osd_op_reply(109 gc.5 [call out=15b] v0'0 uv36 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.432+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:110 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f6618b0 2024-05-08T18:05:26.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.432+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 31 ==== osd_op_reply(110 gc.5 [call out=24b] v0'0 uv36 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.432+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:111 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.432+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 32 ==== osd_op_reply(111 gc.5 [call out=15b] v0'0 uv36 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.432+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:112 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f658590 2024-05-08T18:05:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.433+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 33 ==== osd_op_reply(112 gc.6 [call out=15b] v0'0 uv23 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x563c1f658590 2024-05-08T18:05:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.433+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:113 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f658590 2024-05-08T18:05:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.433+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 34 ==== osd_op_reply(113 gc.6 [call out=24b] v0'0 uv23 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fae0004b020 con 0x563c1f658590 2024-05-08T18:05:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.433+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:114 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f658590 2024-05-08T18:05:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.433+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 35 ==== osd_op_reply(114 gc.6 [call out=15b] v0'0 uv23 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x563c1f658590 2024-05-08T18:05:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.433+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:115 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f65ccc0 2024-05-08T18:05:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.433+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(115 gc.7 [call out=15b] v0'0 uv36 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.433+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:116 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f65ccc0 2024-05-08T18:05:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.434+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(116 gc.7 [call out=24b] v0'0 uv36 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.434+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:117 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f65ccc0 2024-05-08T18:05:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.434+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== osd_op_reply(117 gc.7 [call out=15b] v0'0 uv36 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.434+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:118 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x7fadfc076040 2024-05-08T18:05:26.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.434+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 21 ==== osd_op_reply(118 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.434+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:119 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x7fadfc076040 2024-05-08T18:05:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.435+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 22 ==== osd_op_reply(119 gc.8 [call out=24b] v0'0 uv21 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.435+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:120 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x7fadfc076040 2024-05-08T18:05:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.435+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 23 ==== osd_op_reply(120 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.435+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:121 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.435+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 33 ==== osd_op_reply(121 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.435+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:122 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f6618b0 2024-05-08T18:05:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.435+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 34 ==== osd_op_reply(122 gc.9 [call out=24b] v0'0 uv27 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.435+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:123 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.436+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 35 ==== osd_op_reply(123 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.436+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:124 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x7fadfc076040 2024-05-08T18:05:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.436+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 24 ==== osd_op_reply(124 gc.10 [call out=15b] v0'0 uv53 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.436+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:125 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x7fadfc076040 2024-05-08T18:05:26.438 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.436+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 25 ==== osd_op_reply(125 gc.10 [call out=24b] v0'0 uv53 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.436+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:126 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x7fadfc076040 2024-05-08T18:05:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.436+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 26 ==== osd_op_reply(126 gc.10 [call out=15b] v0'0 uv53 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.436+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:127 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f65ccc0 2024-05-08T18:05:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.437+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 31 ==== osd_op_reply(127 gc.11 [call out=15b] v0'0 uv24 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.437+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:128 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f65ccc0 2024-05-08T18:05:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.437+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 32 ==== osd_op_reply(128 gc.11 [call out=24b] v0'0 uv24 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.437+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:129 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f65ccc0 2024-05-08T18:05:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.437+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 33 ==== osd_op_reply(129 gc.11 [call out=15b] v0'0 uv24 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.437+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:130 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x7fadfc076040 2024-05-08T18:05:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.438+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 27 ==== osd_op_reply(130 gc.12 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.438+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:131 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x7fadfc076040 2024-05-08T18:05:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.438+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 28 ==== osd_op_reply(131 gc.12 [call out=24b] v0'0 uv21 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.438+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:132 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x7fadfc076040 2024-05-08T18:05:26.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.438+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 29 ==== osd_op_reply(132 gc.12 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.438+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:133 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f65ccc0 2024-05-08T18:05:26.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.439+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 34 ==== osd_op_reply(133 gc.13 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.439+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:134 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f65ccc0 2024-05-08T18:05:26.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.439+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 35 ==== osd_op_reply(134 gc.13 [call out=24b] v0'0 uv42 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.439+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:135 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f65ccc0 2024-05-08T18:05:26.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.440+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 36 ==== osd_op_reply(135 gc.13 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.440+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:136 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f658590 2024-05-08T18:05:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.440+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 36 ==== osd_op_reply(136 gc.14 [call out=15b] v0'0 uv166 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x563c1f658590 2024-05-08T18:05:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.440+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:137 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f658590 2024-05-08T18:05:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.440+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 37 ==== osd_op_reply(137 gc.14 [call out=24b] v0'0 uv166 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fae0004b020 con 0x563c1f658590 2024-05-08T18:05:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.440+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:138 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f658590 2024-05-08T18:05:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.441+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 38 ==== osd_op_reply(138 gc.14 [call out=15b] v0'0 uv166 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x563c1f658590 2024-05-08T18:05:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.441+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:139 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.441+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 36 ==== osd_op_reply(139 gc.15 [call out=15b] v0'0 uv50 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.441+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:140 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f6618b0 2024-05-08T18:05:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.441+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 37 ==== osd_op_reply(140 gc.15 [call out=24b] v0'0 uv50 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.441+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:141 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.441+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 38 ==== osd_op_reply(141 gc.15 [call out=15b] v0'0 uv50 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.442+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:142 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x7fadfc076040 2024-05-08T18:05:26.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.442+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 30 ==== osd_op_reply(142 gc.16 [call out=15b] v0'0 uv55 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.442+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:143 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x7fadfc076040 2024-05-08T18:05:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.442+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 31 ==== osd_op_reply(143 gc.16 [call out=24b] v0'0 uv55 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.442+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:144 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x7fadfc076040 2024-05-08T18:05:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.442+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 32 ==== osd_op_reply(144 gc.16 [call out=15b] v0'0 uv55 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.442+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:145 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.443+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 39 ==== osd_op_reply(145 gc.17 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.443+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:146 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f6618b0 2024-05-08T18:05:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.443+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 40 ==== osd_op_reply(146 gc.17 [call out=24b] v0'0 uv21 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.443+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:147 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.443+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 41 ==== osd_op_reply(147 gc.17 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.443+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:148 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f658590 2024-05-08T18:05:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.443+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 39 ==== osd_op_reply(148 gc.18 [call out=15b] v0'0 uv29 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x563c1f658590 2024-05-08T18:05:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.443+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:149 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f658590 2024-05-08T18:05:26.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.444+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 40 ==== osd_op_reply(149 gc.18 [call out=24b] v0'0 uv29 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fae0004b020 con 0x563c1f658590 2024-05-08T18:05:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.444+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:150 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f658590 2024-05-08T18:05:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.444+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 41 ==== osd_op_reply(150 gc.18 [call out=15b] v0'0 uv29 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x563c1f658590 2024-05-08T18:05:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.444+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:151 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f658590 2024-05-08T18:05:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.444+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 42 ==== osd_op_reply(151 gc.19 [call out=15b] v0'0 uv162 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x563c1f658590 2024-05-08T18:05:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.444+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:152 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f658590 2024-05-08T18:05:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.444+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 43 ==== osd_op_reply(152 gc.19 [call out=24b] v0'0 uv162 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fae0004b020 con 0x563c1f658590 2024-05-08T18:05:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.444+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:153 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f658590 2024-05-08T18:05:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.445+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 44 ==== osd_op_reply(153 gc.19 [call out=15b] v0'0 uv162 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x563c1f658590 2024-05-08T18:05:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.445+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:154 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.445+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 42 ==== osd_op_reply(154 gc.20 [call out=15b] v0'0 uv46 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.445+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:155 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f6618b0 2024-05-08T18:05:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.445+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 43 ==== osd_op_reply(155 gc.20 [call out=24b] v0'0 uv46 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.445+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:156 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.446+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 44 ==== osd_op_reply(156 gc.20 [call out=15b] v0'0 uv46 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.446+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:157 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x7fadfc076040 2024-05-08T18:05:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.446+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 33 ==== osd_op_reply(157 gc.21 [call out=15b] v0'0 uv31 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.446+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:158 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x7fadfc076040 2024-05-08T18:05:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.446+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 34 ==== osd_op_reply(158 gc.21 [call out=24b] v0'0 uv31 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.446+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:159 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x7fadfc076040 2024-05-08T18:05:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.446+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 35 ==== osd_op_reply(159 gc.21 [call out=15b] v0'0 uv31 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.446+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:160 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.447+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 45 ==== osd_op_reply(160 gc.22 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.447+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:161 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f6618b0 2024-05-08T18:05:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.447+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 46 ==== osd_op_reply(161 gc.22 [call out=24b] v0'0 uv23 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.447+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:162 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.447+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 47 ==== osd_op_reply(162 gc.22 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.447+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:163 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.447+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 48 ==== osd_op_reply(163 gc.23 [call out=15b] v0'0 uv27 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.448+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:164 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f6618b0 2024-05-08T18:05:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.448+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 49 ==== osd_op_reply(164 gc.23 [call out=24b] v0'0 uv27 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.448+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:165 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.448+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 50 ==== osd_op_reply(165 gc.23 [call out=15b] v0'0 uv27 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.448+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:166 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.448+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 51 ==== osd_op_reply(166 gc.24 [call out=15b] v0'0 uv19 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.448+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:167 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f6618b0 2024-05-08T18:05:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.449+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 52 ==== osd_op_reply(167 gc.24 [call out=24b] v0'0 uv19 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.449+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:168 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.449+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 53 ==== osd_op_reply(168 gc.24 [call out=15b] v0'0 uv19 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.449+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:169 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.449+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 54 ==== osd_op_reply(169 gc.25 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.449+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:170 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f6618b0 2024-05-08T18:05:26.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.449+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 55 ==== osd_op_reply(170 gc.25 [call out=24b] v0'0 uv34 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.449+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:171 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.450+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 56 ==== osd_op_reply(171 gc.25 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.450+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:172 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f65ccc0 2024-05-08T18:05:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.450+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 37 ==== osd_op_reply(172 gc.26 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.450+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:173 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f65ccc0 2024-05-08T18:05:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.450+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 38 ==== osd_op_reply(173 gc.26 [call out=24b] v0'0 uv23 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.450+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:174 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f65ccc0 2024-05-08T18:05:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.451+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 39 ==== osd_op_reply(174 gc.26 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.451+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:175 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.451+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 57 ==== osd_op_reply(175 gc.27 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.451+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:176 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f6618b0 2024-05-08T18:05:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.451+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 58 ==== osd_op_reply(176 gc.27 [call out=24b] v0'0 uv21 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.451+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:177 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.451+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 59 ==== osd_op_reply(177 gc.27 [call out=15b] v0'0 uv21 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.451+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:178 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f65ccc0 2024-05-08T18:05:26.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.452+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 40 ==== osd_op_reply(178 gc.28 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.452+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:179 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f65ccc0 2024-05-08T18:05:26.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.452+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 41 ==== osd_op_reply(179 gc.28 [call out=24b] v0'0 uv34 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.452+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:180 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f65ccc0 2024-05-08T18:05:26.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.452+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 42 ==== osd_op_reply(180 gc.28 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.452+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:181 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.453+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 60 ==== osd_op_reply(181 gc.29 [call out=15b] v0'0 uv48 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.453+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:182 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f6618b0 2024-05-08T18:05:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.453+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 61 ==== osd_op_reply(182 gc.29 [call out=24b] v0'0 uv48 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.453+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:183 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f6618b0 2024-05-08T18:05:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.453+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 62 ==== osd_op_reply(183 gc.29 [call out=15b] v0'0 uv48 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.453+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:184 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f65ccc0 2024-05-08T18:05:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.453+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 43 ==== osd_op_reply(184 gc.30 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.453+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:185 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x563c1f65ccc0 2024-05-08T18:05:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.454+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 44 ==== osd_op_reply(185 gc.30 [call out=24b] v0'0 uv40 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.454+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:186 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x563c1f65ccc0 2024-05-08T18:05:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.454+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 45 ==== osd_op_reply(186 gc.30 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.454+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:187 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x7fadfc076040 2024-05-08T18:05:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.454+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 36 ==== osd_op_reply(187 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.454+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:188 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a4d80 con 0x7fadfc076040 2024-05-08T18:05:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.454+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 37 ==== osd_op_reply(188 gc.31 [call out=24b] v0'0 uv27 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.454+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:189 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563c1f6a5d90 con 0x7fadfc076040 2024-05-08T18:05:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.455+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 38 ==== osd_op_reply(189 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.456+0000 7facbefe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.456+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:190 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94816224850656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f66ebd0 con 0x563c1f658590 2024-05-08T18:05:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.457+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 45 ==== osd_op_reply(190 data_loggenerations_metadata [watch unwatch cookie 94816224850656] v27'172 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fae0004b020 con 0x563c1f658590 2024-05-08T18:05:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.457+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:191 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94816224828560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f66b2e0 con 0x7fadfc076040 2024-05-08T18:05:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.458+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 39 ==== osd_op_reply(191 notify.0 [watch unwatch cookie 94816224828560] v27'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.458+0000 7fae0d11f800 20 remove_watcher() i=0 2024-05-08T18:05:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.458+0000 7fae0d11f800 2 removed watcher, disabling cache 2024-05-08T18:05:26.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.458+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:192 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94816224832384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f66b2e0 con 0x563c1f658590 2024-05-08T18:05:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.459+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 46 ==== osd_op_reply(192 notify.1 [watch unwatch cookie 94816224832384] v27'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae0004b020 con 0x563c1f658590 2024-05-08T18:05:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.459+0000 7fae0d11f800 20 remove_watcher() i=1 2024-05-08T18:05:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.459+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:193 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94816224826000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f666b80 con 0x563c1f6618b0 2024-05-08T18:05:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.460+0000 7fae085c2640 1 -- 172.21.15.50:0/1461934309 <== osd.3 v2:172.21.15.50:6800/3130492744 63 ==== osd_op_reply(193 notify.2 [watch unwatch cookie 94816224826000] v27'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadfc081070 con 0x563c1f6618b0 2024-05-08T18:05:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.460+0000 7fae0d11f800 20 remove_watcher() i=2 2024-05-08T18:05:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.460+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:194 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94816224838976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f66b2e0 con 0x7fadfc076040 2024-05-08T18:05:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.461+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.2 v2:172.21.15.50:6811/3039646617 40 ==== osd_op_reply(194 notify.3 [watch unwatch cookie 94816224838976] v27'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae0004b020 con 0x7fadfc076040 2024-05-08T18:05:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.461+0000 7fae0d11f800 20 remove_watcher() i=3 2024-05-08T18:05:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.461+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:195 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94816224842528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f665290 con 0x563c1f658590 2024-05-08T18:05:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.462+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 <== osd.1 v2:172.21.15.50:6808/2141923372 47 ==== osd_op_reply(195 notify.4 [watch unwatch cookie 94816224842528] v27'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fae0004b020 con 0x563c1f658590 2024-05-08T18:05:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.462+0000 7fae0d11f800 20 remove_watcher() i=4 2024-05-08T18:05:26.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.462+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:196 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94816224840032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f666b80 con 0x563c1f65ccc0 2024-05-08T18:05:26.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.463+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 46 ==== osd_op_reply(196 notify.5 [watch unwatch cookie 94816224840032] v27'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.463+0000 7fae0d11f800 20 remove_watcher() i=5 2024-05-08T18:05:26.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.463+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:197 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94816224843584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f66b2e0 con 0x563c1f65ccc0 2024-05-08T18:05:26.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.464+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 47 ==== osd_op_reply(197 notify.6 [watch unwatch cookie 94816224843584] v27'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.464+0000 7fae0d11f800 20 remove_watcher() i=6 2024-05-08T18:05:26.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.464+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:198 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94816224852784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563c1f665290 con 0x563c1f65ccc0 2024-05-08T18:05:26.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.465+0000 7fae08dc3640 1 -- 172.21.15.50:0/1461934309 <== osd.0 v2:172.21.15.46:6800/337143975 48 ==== osd_op_reply(198 notify.7 [watch unwatch cookie 94816224852784] v27'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadf805b070 con 0x563c1f65ccc0 2024-05-08T18:05:26.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.465+0000 7fae0d11f800 20 remove_watcher() i=7 2024-05-08T18:05:26.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.465+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x563c1f65ccc0 msgr2=0x563c1f65f0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.465+0000 7fae0d11f800 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x563c1f65ccc0 0x563c1f65f0b0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.465+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x563c1f658590 msgr2=0x563c1f65aa50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.465+0000 7fae0d11f800 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x563c1f658590 0x563c1f65aa50 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.465+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fadfc076040 msgr2=0x7fadfc083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.465+0000 7fae0d11f800 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fadfc076040 0x7fadfc083410 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.465+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x563c1f6618b0 msgr2=0x563c1f663d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.465+0000 7fae0d11f800 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x563c1f6618b0 0x563c1f663d70 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7facd403bba0 msgr2=0x7facd403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7facd403bba0 0x7facd403e060 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fadfc052fd0 tx=0x7fadfc04f000 comp rx=0 tx=0).stop 2024-05-08T18:05:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f645c60 msgr2=0x563c1f64d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 --2- 172.21.15.50:0/1461934309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f645c60 0x563c1f64d0b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fadf8022b40 tx=0x7fadf800a340 comp rx=0 tx=0).stop 2024-05-08T18:05:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 reap_dead start 2024-05-08T18:05:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae095c4640 1 -- 172.21.15.50:0/1461934309 reap_dead start 2024-05-08T18:05:26.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 shutdown_connections 2024-05-08T18:05:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 >> 172.21.15.50:0/1461934309 conn(0x563c1f641f80 msgr2=0x563c1f63f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 shutdown_connections 2024-05-08T18:05:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 -- 172.21.15.50:0/1461934309 wait complete. 2024-05-08T18:05:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 -- 172.21.15.50:0/1736428341 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7face003bba0 msgr2=0x7face003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 --2- 172.21.15.50:0/1736428341 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7face003bba0 0x7face003e060 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x563c1f622560 tx=0x7fadfc005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 -- 172.21.15.50:0/1736428341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f638d50 msgr2=0x563c1f639170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 --2- 172.21.15.50:0/1736428341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f638d50 0x563c1f639170 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fadf80510b0 tx=0x7fadf8043500 comp rx=0 tx=0).stop 2024-05-08T18:05:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 -- 172.21.15.50:0/1736428341 shutdown_connections 2024-05-08T18:05:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 --2- 172.21.15.50:0/1736428341 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7face003bba0 0x7face003e060 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 --2- 172.21.15.50:0/1736428341 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f638d50 0x563c1f639170 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 -- 172.21.15.50:0/1736428341 >> 172.21.15.50:0/1736428341 conn(0x563c1f62cdf0 msgr2=0x563c1f62d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 -- 172.21.15.50:0/1736428341 shutdown_connections 2024-05-08T18:05:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.466+0000 7fae0d11f800 1 -- 172.21.15.50:0/1736428341 wait complete. 2024-05-08T18:05:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.467+0000 7fae0d11f800 1 -- 172.21.15.50:0/433930700 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x563c1f623fb0 msgr2=0x563c1f6263a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.467+0000 7fae0d11f800 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x563c1f623fb0 0x563c1f6263a0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.467+0000 7fae0d11f800 1 -- 172.21.15.50:0/433930700 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x563c1f5269e0 msgr2=0x563c1f6216d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.467+0000 7fae0d11f800 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x563c1f5269e0 0x563c1f6216d0 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.467+0000 7fae0d11f800 1 -- 172.21.15.50:0/433930700 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x563c1f627ef0 msgr2=0x563c1f62a2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.467+0000 7fae0d11f800 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x563c1f627ef0 0x563c1f62a2e0 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.467+0000 7fae0d11f800 1 -- 172.21.15.50:0/433930700 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7facf003bb50 msgr2=0x7facf003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.467+0000 7fae0d11f800 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7facf003bb50 0x7facf003e010 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fadfc054000 tx=0x7fadfc04e000 comp rx=0 tx=0).stop 2024-05-08T18:05:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.467+0000 7fae0d11f800 1 -- 172.21.15.50:0/433930700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f522550 msgr2=0x563c1f61a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.467+0000 7fae0d11f800 1 --2- 172.21.15.50:0/433930700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563c1f522550 0x563c1f61a490 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fadf8000c00 tx=0x7fadf8000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.467+0000 7fae095c4640 1 -- 172.21.15.50:0/433930700 reap_dead start 2024-05-08T18:05:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.467+0000 7fae0d11f800 1 -- 172.21.15.50:0/433930700 shutdown_connections 2024-05-08T18:05:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.467+0000 7fae0d11f800 1 -- 172.21.15.50:0/433930700 >> 172.21.15.50:0/433930700 conn(0x563c1f51db50 msgr2=0x563c1f60ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.467+0000 7fae0d11f800 1 -- 172.21.15.50:0/433930700 shutdown_connections 2024-05-08T18:05:26.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.467+0000 7fae0d11f800 1 -- 172.21.15.50:0/433930700 wait complete. 2024-05-08T18:05:26.474 INFO:teuthology.orchestra.run.smithi050.stdout:[] 2024-05-08T18:05:26.474 DEBUG:tasks.util.rgw: json result: [] 2024-05-08T18:05:26.474 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-05-08T18:05:26.474 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-05-08T18:05:26.475 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-05-08T18:05:26.520 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:26.520 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:26.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.523+0000 7f6b2a719800 1 Processor -- start 2024-05-08T18:05:26.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.523+0000 7f6b2a719800 1 -- start start 2024-05-08T18:05:26.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.523+0000 7f6b2a719800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55621fa39ae0 0x55621fa3bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.523+0000 7f6b2a719800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55621fa3c4a0 con 0x55621fa356f0 2024-05-08T18:05:26.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.523+0000 7f6b2a719800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55621f8d5e20 con 0x55621fa39ae0 2024-05-08T18:05:26.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.523+0000 7f6b27a10640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55621fa39ae0 0x55621fa3bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.523+0000 7f6b28211640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55621fa356f0 0x55621fa35b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:60364/0 (socket says 172.21.15.50:60364) 2024-05-08T18:05:26.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.523+0000 7f6b27a10640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55621fa39ae0 0x55621fa3bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43766/0 (socket says 172.21.15.50:43766) 2024-05-08T18:05:26.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.523+0000 7f6b28211640 1 -- 172.21.15.50:0/2493350522 learned_addr learned my addr 172.21.15.50:0/2493350522 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:26.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.523+0000 7f6b27a10640 1 -- 172.21.15.50:0/2493350522 >> v1:172.21.15.46:6789/0 conn(0x55621fa356f0 legacy=0x55621fa35b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.523+0000 7f6b27a10640 1 -- 172.21.15.50:0/2493350522 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55621fa3d6d0 con 0x55621fa39ae0 2024-05-08T18:05:26.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.525+0000 7f6b27a10640 1 --2- 172.21.15.50:0/2493350522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa39ae0 0x55621fa3bed0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00a470 tx=0x7f6b1c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d775985674f5ba00 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.526+0000 7f6b2720f640 1 -- 172.21.15.50:0/2493350522 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b1c057070 con 0x55621fa39ae0 2024-05-08T18:05:26.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.526+0000 7f6b2720f640 1 -- 172.21.15.50:0/2493350522 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1c052570 con 0x55621fa39ae0 2024-05-08T18:05:26.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.526+0000 7f6b2720f640 1 -- 172.21.15.50:0/2493350522 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b1c052870 con 0x55621fa39ae0 2024-05-08T18:05:26.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.526+0000 7f6b2a719800 1 -- 172.21.15.50:0/2493350522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa39ae0 msgr2=0x55621fa3bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.526+0000 7f6b2a719800 1 --2- 172.21.15.50:0/2493350522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa39ae0 0x55621fa3bed0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00a470 tx=0x7f6b1c01ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:26.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.526+0000 7f6b2a719800 1 -- 172.21.15.50:0/2493350522 shutdown_connections 2024-05-08T18:05:26.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.526+0000 7f6b2a719800 1 --2- 172.21.15.50:0/2493350522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa39ae0 0x55621fa3bed0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.526+0000 7f6b2a719800 1 -- 172.21.15.50:0/2493350522 >> 172.21.15.50:0/2493350522 conn(0x55621fa30c70 msgr2=0x55621fa330b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.526+0000 7f6b2a719800 1 -- 172.21.15.50:0/2493350522 shutdown_connections 2024-05-08T18:05:26.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.526+0000 7f6b2a719800 1 -- 172.21.15.50:0/2493350522 wait complete. 2024-05-08T18:05:26.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.541+0000 7f6b2a719800 1 Processor -- start 2024-05-08T18:05:26.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.541+0000 7f6b2a719800 1 -- start start 2024-05-08T18:05:26.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.542+0000 7f6b2a719800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa356f0 0x55621fb26b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.542+0000 7f6b2a719800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55621f980a10 con 0x55621fa356f0 2024-05-08T18:05:26.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.542+0000 7f6b28211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa356f0 0x55621fb26b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.542+0000 7f6b28211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa356f0 0x55621fb26b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43770/0 (socket says 172.21.15.50:43770) 2024-05-08T18:05:26.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.542+0000 7f6b28211640 1 -- 172.21.15.50:0/2168429441 learned_addr learned my addr 172.21.15.50:0/2168429441 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.542+0000 7f6b28211640 1 -- 172.21.15.50:0/2168429441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55621fb270e0 con 0x55621fa356f0 2024-05-08T18:05:26.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.542+0000 7f6b28211640 1 --2- 172.21.15.50:0/2168429441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa356f0 0x55621fb26b10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6b18001e40 tx=0x7f6b18001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f00d8c6252d00ce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.542+0000 7f6a1660c640 1 -- 172.21.15.50:0/2168429441 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b18012020 con 0x55621fa356f0 2024-05-08T18:05:26.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.542+0000 7f6a1660c640 1 -- 172.21.15.50:0/2168429441 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1800e5a0 con 0x55621fa356f0 2024-05-08T18:05:26.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.542+0000 7f6a1660c640 1 -- 172.21.15.50:0/2168429441 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b1800e8a0 con 0x55621fa356f0 2024-05-08T18:05:26.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.543+0000 7f6b2a719800 1 -- 172.21.15.50:0/2168429441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa356f0 msgr2=0x55621fb26b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.543+0000 7f6b2a719800 1 --2- 172.21.15.50:0/2168429441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa356f0 0x55621fb26b10 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6b18001e40 tx=0x7f6b18001e70 comp rx=0 tx=0).stop 2024-05-08T18:05:26.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.543+0000 7f6b2a719800 1 -- 172.21.15.50:0/2168429441 shutdown_connections 2024-05-08T18:05:26.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.543+0000 7f6b2a719800 1 --2- 172.21.15.50:0/2168429441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa356f0 0x55621fb26b10 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.543+0000 7f6b2a719800 1 -- 172.21.15.50:0/2168429441 >> 172.21.15.50:0/2168429441 conn(0x55621fa30c70 msgr2=0x55621fb22430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.543+0000 7f6b2a719800 1 -- 172.21.15.50:0/2168429441 shutdown_connections 2024-05-08T18:05:26.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.543+0000 7f6b2a719800 1 -- 172.21.15.50:0/2168429441 wait complete. 2024-05-08T18:05:26.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.543+0000 7f6b2a719800 1 Processor -- start 2024-05-08T18:05:26.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.544+0000 7f6b2a719800 1 -- start start 2024-05-08T18:05:26.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.544+0000 7f6b2a719800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa356f0 0x55621f9aaee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.544+0000 7f6b2a719800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55621fa3c970 con 0x55621fa356f0 2024-05-08T18:05:26.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.544+0000 7f6b28211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa356f0 0x55621f9aaee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.544+0000 7f6b28211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa356f0 0x55621f9aaee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43784/0 (socket says 172.21.15.50:43784) 2024-05-08T18:05:26.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.544+0000 7f6b28211640 1 -- 172.21.15.50:0/2340587455 learned_addr learned my addr 172.21.15.50:0/2340587455 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.544+0000 7f6b28211640 1 -- 172.21.15.50:0/2340587455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55621f9a95d0 con 0x55621fa356f0 2024-05-08T18:05:26.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.544+0000 7f6b28211640 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa356f0 0x55621f9aaee0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6b18000c00 tx=0x7f6b18000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.544+0000 7f6a14e09640 1 -- 172.21.15.50:0/2340587455 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b18012030 con 0x55621fa356f0 2024-05-08T18:05:26.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.545+0000 7f6a14e09640 1 -- 172.21.15.50:0/2340587455 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1800ee50 con 0x55621fa356f0 2024-05-08T18:05:26.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.545+0000 7f6b2a719800 1 -- 172.21.15.50:0/2340587455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55621f9a9860 con 0x55621fa356f0 2024-05-08T18:05:26.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.545+0000 7f6a14e09640 1 -- 172.21.15.50:0/2340587455 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b1801a940 con 0x55621fa356f0 2024-05-08T18:05:26.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.545+0000 7f6b2a719800 1 -- 172.21.15.50:0/2340587455 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55621f9a9d80 con 0x55621fa356f0 2024-05-08T18:05:26.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.545+0000 7f6a14e09640 1 -- 172.21.15.50:0/2340587455 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6b18015020 con 0x55621fa356f0 2024-05-08T18:05:26.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.546+0000 7f6a14e09640 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a1003bba0 0x7f6a1003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.546+0000 7f6a14e09640 1 -- 172.21.15.50:0/2340587455 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f6b18020030 con 0x55621fa356f0 2024-05-08T18:05:26.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.546+0000 7f6b27a10640 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a1003bba0 0x7f6a1003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.546+0000 7f6b2a719800 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55621fa39ae0 0x55621fb37a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.546+0000 7f6b2a719800 1 -- 172.21.15.50:0/2340587455 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb37ff0 con 0x55621fa39ae0 2024-05-08T18:05:26.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.546+0000 7f6b28a12640 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55621fa39ae0 0x55621fb37a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.546+0000 7f6b27a10640 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a1003bba0 0x7f6a1003e060 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c054000 tx=0x7f6b1c04e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.546+0000 7f6b28a12640 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55621fa39ae0 0x55621fb37a60 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.547+0000 7f6b28a12640 1 -- 172.21.15.50:0/2340587455 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fa39ae0 2024-05-08T18:05:26.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.547+0000 7f6b2a719800 1 -- 172.21.15.50:0/2340587455 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb388d0 con 0x55621fa39ae0 2024-05-08T18:05:26.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.547+0000 7f6b28a12640 1 -- 172.21.15.50:0/2340587455 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fa39ae0 2024-05-08T18:05:26.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.547+0000 7f6b2a719800 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55621fb3a350 0x55621fb3c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.547+0000 7f6b2a719800 1 -- 172.21.15.50:0/2340587455 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb3cfa0 con 0x55621fb3a350 2024-05-08T18:05:26.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.547+0000 7f6b28211640 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55621fb3a350 0x55621fb3c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.548+0000 7f6b28211640 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55621fb3a350 0x55621fb3c810 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.548+0000 7f6b28211640 1 -- 172.21.15.50:0/2340587455 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb3a350 2024-05-08T18:05:26.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.548+0000 7f6b2a719800 1 -- 172.21.15.50:0/2340587455 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb39120 con 0x55621fa39ae0 2024-05-08T18:05:26.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.548+0000 7f6b28a12640 1 -- 172.21.15.50:0/2340587455 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fa39ae0 2024-05-08T18:05:26.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.548+0000 7f6b2a719800 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55621fb3e400 0x55621fb407f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.548+0000 7f6b2a719800 1 -- 172.21.15.50:0/2340587455 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb40ee0 con 0x55621fb3e400 2024-05-08T18:05:26.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.548+0000 7f6b27a10640 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55621fb3e400 0x55621fb407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.549+0000 7f6b27a10640 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55621fb3e400 0x55621fb407f0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.549+0000 7f6b27a10640 1 -- 172.21.15.50:0/2340587455 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f6b1c052940 con 0x55621fb3e400 2024-05-08T18:05:26.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.549+0000 7f6b2a719800 1 Processor -- start 2024-05-08T18:05:26.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.549+0000 7f6b2a719800 1 -- start start 2024-05-08T18:05:26.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.550+0000 7f6b2a719800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb4f250 0x55621fb4f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.550+0000 7f6b2a719800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55621f9ab420 con 0x55621fb4f250 2024-05-08T18:05:26.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.550+0000 7f6b28211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb4f250 0x55621fb4f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.550+0000 7f6b28211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb4f250 0x55621fb4f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43794/0 (socket says 172.21.15.50:43794) 2024-05-08T18:05:26.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.550+0000 7f6b28211640 1 -- 172.21.15.50:0/1560894199 learned_addr learned my addr 172.21.15.50:0/1560894199 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.550+0000 7f6b28211640 1 -- 172.21.15.50:0/1560894199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55621fb4fbb0 con 0x55621fb4f250 2024-05-08T18:05:26.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.550+0000 7f6b28211640 1 --2- 172.21.15.50:0/1560894199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb4f250 0x55621fb4f670 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6b18051180 tx=0x7f6b1800db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.550+0000 7f6a0dffb640 1 -- 172.21.15.50:0/1560894199 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b1803d990 con 0x55621fb4f250 2024-05-08T18:05:26.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.550+0000 7f6a0dffb640 1 -- 172.21.15.50:0/1560894199 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1803db30 con 0x55621fb4f250 2024-05-08T18:05:26.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.550+0000 7f6a0dffb640 1 -- 172.21.15.50:0/1560894199 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b1803de10 con 0x55621fb4f250 2024-05-08T18:05:26.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.550+0000 7f6b2a719800 1 -- 172.21.15.50:0/1560894199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55621fb50620 con 0x55621fb4f250 2024-05-08T18:05:26.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.550+0000 7f6b2a719800 1 -- 172.21.15.50:0/1560894199 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55621fb50b20 con 0x55621fb4f250 2024-05-08T18:05:26.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.551+0000 7f6a0dffb640 1 -- 172.21.15.50:0/1560894199 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6b1803d2e0 con 0x55621fb4f250 2024-05-08T18:05:26.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.551+0000 7f6a0dffb640 1 --2- 172.21.15.50:0/1560894199 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a0003bb50 0x7f6a0003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.551+0000 7f6a0dffb640 1 -- 172.21.15.50:0/1560894199 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f6b18025020 con 0x55621fb4f250 2024-05-08T18:05:26.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.551+0000 7f6b27a10640 1 --2- 172.21.15.50:0/1560894199 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a0003bb50 0x7f6a0003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.552+0000 7f6b2a719800 1 Processor -- start 2024-05-08T18:05:26.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.552+0000 7f6b2a719800 1 -- start start 2024-05-08T18:05:26.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.552+0000 7f6b27a10640 1 --2- 172.21.15.50:0/1560894199 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a0003bb50 0x7f6a0003e010 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55621f9aa2a0 tx=0x7f6b1c005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.552+0000 7f6b2a719800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb5c380 0x55621fb5c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.552+0000 7f6b2a719800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55621f927330 con 0x55621fb5c380 2024-05-08T18:05:26.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.552+0000 7f6b28211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb5c380 0x55621fb5c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.552+0000 7f6b28211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb5c380 0x55621fb5c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43808/0 (socket says 172.21.15.50:43808) 2024-05-08T18:05:26.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.552+0000 7f6b28211640 1 -- 172.21.15.50:0/2989615055 learned_addr learned my addr 172.21.15.50:0/2989615055 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.552+0000 7f6b28211640 1 -- 172.21.15.50:0/2989615055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55621fb5cd70 con 0x55621fb5c380 2024-05-08T18:05:26.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.552+0000 7f6b28211640 1 --2- 172.21.15.50:0/2989615055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb5c380 0x55621fb5c7a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6b18016620 tx=0x7f6b18051e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176fefdd0e19c693 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.553+0000 7f69ff7fe640 1 -- 172.21.15.50:0/2989615055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b18019d00 con 0x55621fb5c380 2024-05-08T18:05:26.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.553+0000 7f69ff7fe640 1 -- 172.21.15.50:0/2989615055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b18043cd0 con 0x55621fb5c380 2024-05-08T18:05:26.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.553+0000 7f69ff7fe640 1 -- 172.21.15.50:0/2989615055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b1804a280 con 0x55621fb5c380 2024-05-08T18:05:26.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.553+0000 7f6b2a719800 1 -- 172.21.15.50:0/2989615055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb5c380 msgr2=0x55621fb5c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.553+0000 7f6b2a719800 1 --2- 172.21.15.50:0/2989615055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb5c380 0x55621fb5c7a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6b18016620 tx=0x7f6b18051e40 comp rx=0 tx=0).stop 2024-05-08T18:05:26.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.553+0000 7f6b2a719800 1 -- 172.21.15.50:0/2989615055 shutdown_connections 2024-05-08T18:05:26.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.553+0000 7f6b2a719800 1 --2- 172.21.15.50:0/2989615055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb5c380 0x55621fb5c7a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.553+0000 7f6b2a719800 1 -- 172.21.15.50:0/2989615055 >> 172.21.15.50:0/2989615055 conn(0x55621fb586a0 msgr2=0x55621fb56700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.553+0000 7f6b2a719800 1 -- 172.21.15.50:0/2989615055 shutdown_connections 2024-05-08T18:05:26.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.553+0000 7f6b2a719800 1 -- 172.21.15.50:0/2989615055 wait complete. 2024-05-08T18:05:26.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.553+0000 7f6b2a719800 1 Processor -- start 2024-05-08T18:05:26.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.553+0000 7f6b2a719800 1 -- start start 2024-05-08T18:05:26.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.554+0000 7f6b2a719800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb5c380 0x55621fb636f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.554+0000 7f6b2a719800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55621fb5d310 con 0x55621fb5c380 2024-05-08T18:05:26.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.554+0000 7f6b28211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb5c380 0x55621fb636f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.554+0000 7f6b28211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb5c380 0x55621fb636f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43810/0 (socket says 172.21.15.50:43810) 2024-05-08T18:05:26.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.554+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 learned_addr learned my addr 172.21.15.50:0/3303510934 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.554+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55621fb63c30 con 0x55621fb5c380 2024-05-08T18:05:26.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.554+0000 7f6b28211640 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb5c380 0x55621fb636f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6b18024a00 tx=0x7f6b18012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.554+0000 7f69fdffb640 1 -- 172.21.15.50:0/3303510934 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b18024a70 con 0x55621fb5c380 2024-05-08T18:05:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.554+0000 7f69fdffb640 1 -- 172.21.15.50:0/3303510934 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b18024c10 con 0x55621fb5c380 2024-05-08T18:05:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.554+0000 7f69fdffb640 1 -- 172.21.15.50:0/3303510934 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b18016790 con 0x55621fb5c380 2024-05-08T18:05:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.554+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55621fb63c30 con 0x55621fb5c380 2024-05-08T18:05:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.554+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55621fb63c30 con 0x55621fb5c380 2024-05-08T18:05:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.555+0000 7f69fdffb640 1 -- 172.21.15.50:0/3303510934 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6b18070060 con 0x55621fb5c380 2024-05-08T18:05:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.555+0000 7f69fdffb640 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f69f003bb50 0x7f69f003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.555+0000 7f69fdffb640 1 -- 172.21.15.50:0/3303510934 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f6b1805a030 con 0x55621fb5c380 2024-05-08T18:05:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.556+0000 7f6b27a10640 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f69f003bb50 0x7f69f003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.556+0000 7f6b27a10640 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f69f003bb50 0x7f69f003e010 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c052fd0 tx=0x7f6b1c04f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.556+0000 7f6b2a719800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.556+0000 7f6b2a719800 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55621fb6ecc0 0x55621fb71180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.556+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb718d0 con 0x55621fb6ecc0 2024-05-08T18:05:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.556+0000 7f6b28a12640 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55621fb6ecc0 0x55621fb71180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.557+0000 7f6b28a12640 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55621fb6ecc0 0x55621fb71180 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.557+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fb6ecc0 2024-05-08T18:05:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.557+0000 7f6b2a719800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.557+0000 7f6b2a719800 20 realm 2024-05-08T18:05:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.557+0000 7f6b2a719800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.557+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb71ce0 con 0x55621fb6ecc0 2024-05-08T18:05:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.557+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fb6ecc0 2024-05-08T18:05:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.557+0000 7f6b2a719800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.557+0000 7f6b2a719800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.557+0000 7f6b2a719800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.557+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb725b0 con 0x55621fb6ecc0 2024-05-08T18:05:26.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.557+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fb6ecc0 2024-05-08T18:05:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.557+0000 7f6b2a719800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.557+0000 7f6b2a719800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.557+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb72950 con 0x55621fb6ecc0 2024-05-08T18:05:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.558+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fb6ecc0 2024-05-08T18:05:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.558+0000 7f6b2a719800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.558+0000 7f6b2a719800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.558+0000 7f6b2a719800 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55621fb736b0 0x55621fb75aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.558+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb760b0 con 0x55621fb736b0 2024-05-08T18:05:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.558+0000 7f6b28211640 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55621fb736b0 0x55621fb75aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.558+0000 7f6b28211640 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55621fb736b0 0x55621fb75aa0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.558+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.558+0000 7f6b2a719800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.558+0000 7f6b2a719800 20 searching for the correct realm 2024-05-08T18:05:26.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.558+0000 7f6b2a719800 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55621fb78060 0x55621fb7a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.558+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7abf0 con 0x55621fb78060 2024-05-08T18:05:26.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.559+0000 7f6b27a10640 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55621fb78060 0x55621fb7a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.559+0000 7f6b27a10640 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55621fb78060 0x55621fb7a520 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.559+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.559+0000 7f6b27a10640 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6b1c076040 0x7f6b1c083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.559+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7abf0 con 0x7f6b1c076040 2024-05-08T18:05:26.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.559+0000 7f6b28a12640 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6b1c076040 0x7f6b1c083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.559+0000 7f6b28a12640 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6b1c076040 0x7f6b1c083410 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.560+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b2000a4a0 con 0x7f6b1c076040 2024-05-08T18:05:26.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.560+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55621fb72950 con 0x55621fb736b0 2024-05-08T18:05:26.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.560+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.560+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55621fb72950 con 0x55621fb6ecc0 2024-05-08T18:05:26.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.560+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fb6ecc0 2024-05-08T18:05:26.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.560+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55621fb72950 con 0x55621fb6ecc0 2024-05-08T18:05:26.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.560+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fb6ecc0 2024-05-08T18:05:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.560+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55621fb72950 con 0x55621fb736b0 2024-05-08T18:05:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.561+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.561+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55621fb72950 con 0x55621fb736b0 2024-05-08T18:05:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.561+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.561+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55621fb72950 con 0x7f6b1c076040 2024-05-08T18:05:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.561+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b2000a4a0 con 0x7f6b1c076040 2024-05-08T18:05:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.561+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55621fb72950 con 0x55621fb78060 2024-05-08T18:05:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.562+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.562+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55621fb72950 con 0x55621fb6ecc0 2024-05-08T18:05:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.562+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fb6ecc0 2024-05-08T18:05:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.562+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55621fb72950 con 0x55621fb736b0 2024-05-08T18:05:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.562+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.562+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55621fb72950 con 0x55621fb6ecc0 2024-05-08T18:05:26.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.562+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fb6ecc0 2024-05-08T18:05:26.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.562+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55621fb72950 con 0x7f6b1c076040 2024-05-08T18:05:26.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.563+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f6b2000a4a0 con 0x7f6b1c076040 2024-05-08T18:05:26.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.563+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55621fb72950 con 0x55621fb78060 2024-05-08T18:05:26.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.563+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.563+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x55621fb78060 2024-05-08T18:05:26.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.563+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.563+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x55621fb736b0 2024-05-08T18:05:26.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.563+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.563+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x55621fb78060 2024-05-08T18:05:26.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.564+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.564+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x55621fb78060 2024-05-08T18:05:26.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.564+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.564+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x55621fb6ecc0 2024-05-08T18:05:26.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.564+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fb6ecc0 2024-05-08T18:05:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.564+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x55621fb6ecc0 2024-05-08T18:05:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.564+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fb6ecc0 2024-05-08T18:05:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.564+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x55621fb736b0 2024-05-08T18:05:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.565+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.565+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x55621fb78060 2024-05-08T18:05:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.565+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.565+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x7f6b1c076040 2024-05-08T18:05:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.565+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b2000a4a0 con 0x7f6b1c076040 2024-05-08T18:05:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.565+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x55621fb6ecc0 2024-05-08T18:05:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.565+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fb6ecc0 2024-05-08T18:05:26.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.565+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x55621fb736b0 2024-05-08T18:05:26.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.566+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.566+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x55621fb78060 2024-05-08T18:05:26.581 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.566+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.566+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x55621fb736b0 2024-05-08T18:05:26.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.566+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.566+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x7f6b1c076040 2024-05-08T18:05:26.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.566+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b2000a4a0 con 0x7f6b1c076040 2024-05-08T18:05:26.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.566+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x55621fb78060 2024-05-08T18:05:26.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.567+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.567+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x55621fb736b0 2024-05-08T18:05:26.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.567+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.567+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x55621fb6ecc0 2024-05-08T18:05:26.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.567+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fb6ecc0 2024-05-08T18:05:26.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.567+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f6b1c090040 con 0x7f6b1c076040 2024-05-08T18:05:26.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6b2000a4a0 con 0x7f6b1c076040 2024-05-08T18:05:26.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:26.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:26.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:26.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:26.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:26.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:26.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb76970 con 0x55621fb6ecc0 2024-05-08T18:05:26.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fb6ecc0 2024-05-08T18:05:26.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb76970 con 0x55621fb6ecc0 2024-05-08T18:05:26.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6b2000a4a0 con 0x55621fb6ecc0 2024-05-08T18:05:26.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:26.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb76970 con 0x55621fb78060 2024-05-08T18:05:26.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:26.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 20 zone default found 2024-05-08T18:05:26.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 4 Realm: () 2024-05-08T18:05:26.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:26.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:26.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:26.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 20 zonegroup default 2024-05-08T18:05:26.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.568+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb772a0 con 0x55621fb736b0 2024-05-08T18:05:26.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.569+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.569+0000 7f6b2a719800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.569+0000 7f6b2a719800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.569+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7cab0 con 0x55621fb78060 2024-05-08T18:05:26.584 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.569+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.569+0000 7f6b2a719800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.569+0000 7f6b2a719800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.569+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7cf40 con 0x55621fb78060 2024-05-08T18:05:26.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.569+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.569+0000 7f6b2a719800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.569+0000 7f6b2a719800 20 started sync module instance, tier type = 2024-05-08T18:05:26.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.569+0000 7f6b2a719800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:26.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.569+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7bef0 con 0x7f6b1c076040 2024-05-08T18:05:26.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.570+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b2000a4a0 con 0x7f6b1c076040 2024-05-08T18:05:26.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.570+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93879927292912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7bef0 con 0x7f6b1c076040 2024-05-08T18:05:26.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.570+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7da00 con 0x55621fb6ecc0 2024-05-08T18:05:26.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.571+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93879927292912] v27'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x7f6b1c076040 2024-05-08T18:05:26.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.572+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x55621fb6ecc0 2024-05-08T18:05:26.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.572+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93879927297296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7bef0 con 0x55621fb6ecc0 2024-05-08T18:05:26.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.572+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7e490 con 0x55621fb78060 2024-05-08T18:05:26.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.573+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93879927297296] v27'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x55621fb6ecc0 2024-05-08T18:05:26.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.573+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.573+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93879927290432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7be60 con 0x55621fb78060 2024-05-08T18:05:26.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.573+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7f350 con 0x7f6b1c076040 2024-05-08T18:05:26.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.574+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x7f6b1c076040 2024-05-08T18:05:26.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.574+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93879927304320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7fad0 con 0x7f6b1c076040 2024-05-08T18:05:26.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.574+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb805d0 con 0x55621fb6ecc0 2024-05-08T18:05:26.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.574+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93879927290432] v27'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.575+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x55621fb6ecc0 2024-05-08T18:05:26.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.575+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93879927308480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb80590 con 0x55621fb6ecc0 2024-05-08T18:05:26.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.575+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb81010 con 0x55621fb736b0 2024-05-08T18:05:26.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.575+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93879927304320] v27'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x7f6b1c076040 2024-05-08T18:05:26.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.576+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.576+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93879927311200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb81400 con 0x55621fb736b0 2024-05-08T18:05:26.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.576+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93879927308480] v27'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x55621fb6ecc0 2024-05-08T18:05:26.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.576+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7e880 con 0x55621fb736b0 2024-05-08T18:05:26.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.577+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93879927311200] v27'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.577+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.577+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93879927314704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb821e0 con 0x55621fb736b0 2024-05-08T18:05:26.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.577+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb82a30 con 0x55621fb736b0 2024-05-08T18:05:26.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.578+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93879927314704] v27'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.578+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.578+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93879927317888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb82e50 con 0x55621fb736b0 2024-05-08T18:05:26.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.578+0000 7f6b2a719800 20 add_watcher() i=0 2024-05-08T18:05:26.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.578+0000 7f6b2a719800 20 add_watcher() i=1 2024-05-08T18:05:26.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.578+0000 7f6b2a719800 20 add_watcher() i=2 2024-05-08T18:05:26.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.578+0000 7f6b2a719800 20 add_watcher() i=3 2024-05-08T18:05:26.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.578+0000 7f6b2a719800 20 add_watcher() i=4 2024-05-08T18:05:26.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.578+0000 7f6b2a719800 20 add_watcher() i=5 2024-05-08T18:05:26.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.578+0000 7f6b2a719800 20 add_watcher() i=6 2024-05-08T18:05:26.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.579+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93879927317888] v27'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.579+0000 7f6b2a719800 20 add_watcher() i=7 2024-05-08T18:05:26.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.579+0000 7f6b2a719800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:26.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.579+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fb86110 con 0x55621fb6ecc0 2024-05-08T18:05:26.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.580+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f6b20040040 con 0x55621fb6ecc0 2024-05-08T18:05:26.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.580+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93879927319312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb833e0 con 0x55621fb6ecc0 2024-05-08T18:05:26.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.581+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93879927319312] v27'173 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x55621fb6ecc0 2024-05-08T18:05:26.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.581+0000 7f69defdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:26.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.581+0000 7f6b2a719800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:26.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.581+0000 7f6b2a719800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:26.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.581+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb85940 con 0x55621fb6ecc0 2024-05-08T18:05:26.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.582+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'29 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x55621fb6ecc0 2024-05-08T18:05:26.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.582+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb8f220 con 0x55621fb736b0 2024-05-08T18:05:26.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.583+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'24 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.583+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb8f220 con 0x55621fb78060 2024-05-08T18:05:26.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.584+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'23 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.584+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb8f810 con 0x55621fb6ecc0 2024-05-08T18:05:26.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.585+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'174 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x55621fb6ecc0 2024-05-08T18:05:26.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.585+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb8fcc0 con 0x7f6b1c076040 2024-05-08T18:05:26.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.586+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'59 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x7f6b1c076040 2024-05-08T18:05:26.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.586+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90230 con 0x55621fb78060 2024-05-08T18:05:26.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.587+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'39 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.587+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb906e0 con 0x55621fb6ecc0 2024-05-08T18:05:26.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.588+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'25 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x55621fb6ecc0 2024-05-08T18:05:26.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.588+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb906e0 con 0x55621fb736b0 2024-05-08T18:05:26.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.589+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'39 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.589+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90790 con 0x7f6b1c076040 2024-05-08T18:05:26.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.590+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'23 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x7f6b1c076040 2024-05-08T18:05:26.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.590+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90790 con 0x55621fb78060 2024-05-08T18:05:26.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.591+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'29 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.591+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90ec0 con 0x7f6b1c076040 2024-05-08T18:05:26.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.592+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'60 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x7f6b1c076040 2024-05-08T18:05:26.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.592+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d90 con 0x55621fb736b0 2024-05-08T18:05:26.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.593+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'26 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.593+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d90 con 0x7f6b1c076040 2024-05-08T18:05:26.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.594+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'23 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x7f6b1c076040 2024-05-08T18:05:26.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.594+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d90 con 0x55621fb736b0 2024-05-08T18:05:26.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.595+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'45 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.595+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x55621fb6ecc0 2024-05-08T18:05:26.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.596+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'175 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x55621fb6ecc0 2024-05-08T18:05:26.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.596+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x55621fb78060 2024-05-08T18:05:26.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.597+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'54 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.597+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x7f6b1c076040 2024-05-08T18:05:26.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.598+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'61 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x7f6b1c076040 2024-05-08T18:05:26.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.598+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x55621fb78060 2024-05-08T18:05:26.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.599+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'23 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.599+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x55621fb6ecc0 2024-05-08T18:05:26.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.600+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'31 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x55621fb6ecc0 2024-05-08T18:05:26.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.600+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x55621fb6ecc0 2024-05-08T18:05:26.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.601+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'176 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x55621fb6ecc0 2024-05-08T18:05:26.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.601+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x55621fb78060 2024-05-08T18:05:26.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.602+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'55 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.602+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x7f6b1c076040 2024-05-08T18:05:26.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.603+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'33 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x7f6b1c076040 2024-05-08T18:05:26.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.603+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x55621fb78060 2024-05-08T18:05:26.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.604+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'25 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.604+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x55621fb78060 2024-05-08T18:05:26.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.605+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'29 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.605+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x55621fb78060 2024-05-08T18:05:26.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.606+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'21 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.606+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x55621fb78060 2024-05-08T18:05:26.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.607+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'40 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.607+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x55621fb736b0 2024-05-08T18:05:26.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.608+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'25 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.608+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x55621fb78060 2024-05-08T18:05:26.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.609+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'23 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.609+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x55621fb736b0 2024-05-08T18:05:26.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.610+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'40 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.610+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x55621fb78060 2024-05-08T18:05:26.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.611+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'56 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.612+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x55621fb736b0 2024-05-08T18:05:26.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.613+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'46 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.613+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb90d00 con 0x7f6b1c076040 2024-05-08T18:05:26.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.614+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'29 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x7f6b1c076040 2024-05-08T18:05:26.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.614+0000 7f69ddfdb640 20 reqs_thread_entry: start 2024-05-08T18:05:26.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.614+0000 7f6b2a719800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:26.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.614+0000 7f6b2a719800 20 Filter name: none 2024-05-08T18:05:26.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.614+0000 7f694f7fe640 20 reqs_thread_entry: start 2024-05-08T18:05:26.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.614+0000 7f6b2a719800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:26.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.614+0000 7f6b2a719800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.614+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fbbbd70 con 0x55621fb6ecc0 2024-05-08T18:05:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.615+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(94 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv13 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f6b20040040 con 0x55621fb6ecc0 2024-05-08T18:05:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.615+0000 7f6b2a719800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.615+0000 7f6b2a719800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.615+0000 7f6b2a719800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.615+0000 7f6b2a719800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.615+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:95 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55621fbbbd30 con 0x7f6b1c076040 2024-05-08T18:05:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.615+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(95 foo.buckets [call out=183b] v0'0 uv28 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f6b20040040 con 0x7f6b1c076040 2024-05-08T18:05:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-05-08T18:05:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.616+0000 7f69defdd640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:26.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.616+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:96 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93879927319312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb697f0 con 0x55621fb6ecc0 2024-05-08T18:05:26.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.617+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(96 data_loggenerations_metadata [watch unwatch cookie 93879927319312] v27'177 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x55621fb6ecc0 2024-05-08T18:05:26.615 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.617+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:97 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93879927292912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb82b10 con 0x7f6b1c076040 2024-05-08T18:05:26.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.618+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(97 notify.0 [watch unwatch cookie 93879927292912] v27'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x7f6b1c076040 2024-05-08T18:05:26.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.618+0000 7f6b2a719800 20 remove_watcher() i=0 2024-05-08T18:05:26.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.618+0000 7f6b2a719800 2 removed watcher, disabling cache 2024-05-08T18:05:26.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.618+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93879927297296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb82b10 con 0x55621fb6ecc0 2024-05-08T18:05:26.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.619+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(98 notify.1 [watch unwatch cookie 93879927297296] v27'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x55621fb6ecc0 2024-05-08T18:05:26.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.619+0000 7f6b2a719800 20 remove_watcher() i=1 2024-05-08T18:05:26.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.619+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:99 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93879927290432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7d510 con 0x55621fb78060 2024-05-08T18:05:26.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.620+0000 7f6b27a10640 1 -- 172.21.15.50:0/3303510934 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(99 notify.2 [watch unwatch cookie 93879927290432] v27'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b1c081070 con 0x55621fb78060 2024-05-08T18:05:26.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.620+0000 7f6b2a719800 20 remove_watcher() i=2 2024-05-08T18:05:26.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.620+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:100 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93879927304320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb82b10 con 0x7f6b1c076040 2024-05-08T18:05:26.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.621+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.2 v2:172.21.15.50:6811/3039646617 20 ==== osd_op_reply(100 notify.3 [watch unwatch cookie 93879927304320] v27'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x7f6b1c076040 2024-05-08T18:05:26.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.621+0000 7f6b2a719800 20 remove_watcher() i=3 2024-05-08T18:05:26.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.621+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93879927308480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7ba40 con 0x55621fb6ecc0 2024-05-08T18:05:26.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.622+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(101 notify.4 [watch unwatch cookie 93879927308480] v27'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b20040040 con 0x55621fb6ecc0 2024-05-08T18:05:26.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.622+0000 7f6b2a719800 20 remove_watcher() i=4 2024-05-08T18:05:26.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.622+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93879927311200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7d510 con 0x55621fb736b0 2024-05-08T18:05:26.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.623+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(102 notify.5 [watch unwatch cookie 93879927311200] v27'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.623+0000 7f6b2a719800 20 remove_watcher() i=5 2024-05-08T18:05:26.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.623+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:103 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93879927314704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb7f080 con 0x55621fb736b0 2024-05-08T18:05:26.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.624+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(103 notify.6 [watch unwatch cookie 93879927314704] v27'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.624+0000 7f6b2a719800 20 remove_watcher() i=6 2024-05-08T18:05:26.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.624+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:104 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93879927317888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55621fb82b10 con 0x55621fb736b0 2024-05-08T18:05:26.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b28211640 1 -- 172.21.15.50:0/3303510934 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(104 notify.7 [watch unwatch cookie 93879927317888] v27'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6b1801acc0 con 0x55621fb736b0 2024-05-08T18:05:26.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 20 remove_watcher() i=7 2024-05-08T18:05:26.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55621fb736b0 msgr2=0x55621fb75aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55621fb736b0 0x55621fb75aa0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55621fb6ecc0 msgr2=0x55621fb71180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55621fb6ecc0 0x55621fb71180 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6b1c076040 msgr2=0x7f6b1c083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6b1c076040 0x7f6b1c083410 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55621fb78060 msgr2=0x55621fb7a520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55621fb78060 0x55621fb7a520 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f69f003bb50 msgr2=0x7f69f003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f69f003bb50 0x7f69f003e010 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c052fd0 tx=0x7f6b1c04f000 comp rx=0 tx=0).stop 2024-05-08T18:05:26.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb5c380 msgr2=0x55621fb636f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 --2- 172.21.15.50:0/3303510934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb5c380 0x55621fb636f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6b18024a00 tx=0x7f6b18012ea0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b28a12640 1 -- 172.21.15.50:0/3303510934 reap_dead start 2024-05-08T18:05:26.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 shutdown_connections 2024-05-08T18:05:26.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 >> 172.21.15.50:0/3303510934 conn(0x55621fb586a0 msgr2=0x55621fb55d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 shutdown_connections 2024-05-08T18:05:26.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 -- 172.21.15.50:0/3303510934 wait complete. 2024-05-08T18:05:26.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 -- 172.21.15.50:0/1560894199 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a0003bb50 msgr2=0x7f6a0003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 --2- 172.21.15.50:0/1560894199 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a0003bb50 0x7f6a0003e010 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55621f9aa2a0 tx=0x7f6b1c005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:26.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 -- 172.21.15.50:0/1560894199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb4f250 msgr2=0x55621fb4f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 --2- 172.21.15.50:0/1560894199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb4f250 0x55621fb4f670 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6b18051180 tx=0x7f6b1800db20 comp rx=0 tx=0).stop 2024-05-08T18:05:26.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 -- 172.21.15.50:0/1560894199 shutdown_connections 2024-05-08T18:05:26.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 --2- 172.21.15.50:0/1560894199 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a0003bb50 0x7f6a0003e010 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 --2- 172.21.15.50:0/1560894199 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fb4f250 0x55621fb4f670 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 -- 172.21.15.50:0/1560894199 >> 172.21.15.50:0/1560894199 conn(0x55621fb432f0 msgr2=0x55621fb436f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.626+0000 7f6b2a719800 1 -- 172.21.15.50:0/1560894199 shutdown_connections 2024-05-08T18:05:26.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.627+0000 7f6b2a719800 1 -- 172.21.15.50:0/1560894199 wait complete. 2024-05-08T18:05:26.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.627+0000 7f6b2a719800 1 -- 172.21.15.50:0/2340587455 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55621fb3a350 msgr2=0x55621fb3c810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.627+0000 7f6b2a719800 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55621fb3a350 0x55621fb3c810 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.627+0000 7f6b2a719800 1 -- 172.21.15.50:0/2340587455 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55621fa39ae0 msgr2=0x55621fb37a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.627+0000 7f6b2a719800 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55621fa39ae0 0x55621fb37a60 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.627+0000 7f6b2a719800 1 -- 172.21.15.50:0/2340587455 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55621fb3e400 msgr2=0x55621fb407f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.627+0000 7f6b2a719800 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55621fb3e400 0x55621fb407f0 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.627+0000 7f6b2a719800 1 -- 172.21.15.50:0/2340587455 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a1003bba0 msgr2=0x7f6a1003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.627+0000 7f6b2a719800 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a1003bba0 0x7f6a1003e060 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c054000 tx=0x7f6b1c04e000 comp rx=0 tx=0).stop 2024-05-08T18:05:26.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.627+0000 7f6b2a719800 1 -- 172.21.15.50:0/2340587455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa356f0 msgr2=0x55621f9aaee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.627+0000 7f6b2a719800 1 --2- 172.21.15.50:0/2340587455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55621fa356f0 0x55621f9aaee0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6b18000c00 tx=0x7f6b18000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.627+0000 7f6b28a12640 1 -- 172.21.15.50:0/2340587455 reap_dead start 2024-05-08T18:05:26.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.627+0000 7f6b2a719800 1 -- 172.21.15.50:0/2340587455 shutdown_connections 2024-05-08T18:05:26.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.627+0000 7f6b2a719800 1 -- 172.21.15.50:0/2340587455 >> 172.21.15.50:0/2340587455 conn(0x55621fa30c70 msgr2=0x55621fb20c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.627+0000 7f6b2a719800 1 -- 172.21.15.50:0/2340587455 shutdown_connections 2024-05-08T18:05:26.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.627+0000 7f6b2a719800 1 -- 172.21.15.50:0/2340587455 wait complete. 2024-05-08T18:05:26.634 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-05-08T18:05:26.643 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=236 failed=True 2024-05-08T18:05:26.648 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:26.675 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:26.684 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:26.731 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2024-05-08T18:05:26.735 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:26.736 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-05-08T18:05:26.736 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-05-08T18:05:26.736 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2024-05-08T18:05:26.782 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:26.783 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:26.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.785+0000 7fa1d812b800 1 Processor -- start 2024-05-08T18:05:26.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.786+0000 7fa1d812b800 1 -- start start 2024-05-08T18:05:26.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.786+0000 7fa1d812b800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5580cd1f9fb0 0x5580cd1fa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.786+0000 7fa1d812b800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5580cd16e8c0 con 0x5580cd1fa910 2024-05-08T18:05:26.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.786+0000 7fa1d812b800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5580cd09be20 con 0x5580cd1f9fb0 2024-05-08T18:05:26.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.786+0000 7fa1d3d4b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5580cd1f9fb0 0x5580cd1fa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.786+0000 7fa1d3d4b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5580cd1f9fb0 0x5580cd1fa3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43816/0 (socket says 172.21.15.50:43816) 2024-05-08T18:05:26.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.786+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/5715411 learned_addr learned my addr 172.21.15.50:0/5715411 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.786+0000 7fa1d354a640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5580cd1fa910 0x5580cd1fadc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:60378/0 (socket says 172.21.15.50:60378) 2024-05-08T18:05:26.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.786+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/5715411 >> v1:172.21.15.46:6789/0 conn(0x5580cd1fa910 legacy=0x5580cd1fadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.786+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/5715411 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580cd1fb4d0 con 0x5580cd1f9fb0 2024-05-08T18:05:26.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.788+0000 7fa1d3d4b640 1 --2- 172.21.15.50:0/5715411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 0x5580cd1fa3d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400a520 tx=0x7fa1c401d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43caf30573075c3a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.789+0000 7fa1d2d49640 1 -- 172.21.15.50:0/5715411 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1c4057070 con 0x5580cd1f9fb0 2024-05-08T18:05:26.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.789+0000 7fa1d2d49640 1 -- 172.21.15.50:0/5715411 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c4052570 con 0x5580cd1f9fb0 2024-05-08T18:05:26.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.789+0000 7fa1d2d49640 1 -- 172.21.15.50:0/5715411 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1c404ae60 con 0x5580cd1f9fb0 2024-05-08T18:05:26.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.789+0000 7fa1d812b800 1 -- 172.21.15.50:0/5715411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 msgr2=0x5580cd1fa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.789+0000 7fa1d812b800 1 --2- 172.21.15.50:0/5715411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 0x5580cd1fa3d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400a520 tx=0x7fa1c401d010 comp rx=0 tx=0).stop 2024-05-08T18:05:26.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.789+0000 7fa1d812b800 1 -- 172.21.15.50:0/5715411 shutdown_connections 2024-05-08T18:05:26.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.789+0000 7fa1d812b800 1 --2- 172.21.15.50:0/5715411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 0x5580cd1fa3d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.789+0000 7fa1d812b800 1 -- 172.21.15.50:0/5715411 >> 172.21.15.50:0/5715411 conn(0x5580cd1f5b70 msgr2=0x5580cd1f7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.789+0000 7fa1d812b800 1 -- 172.21.15.50:0/5715411 shutdown_connections 2024-05-08T18:05:26.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.789+0000 7fa1d812b800 1 -- 172.21.15.50:0/5715411 wait complete. 2024-05-08T18:05:26.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.804+0000 7fa1d812b800 1 Processor -- start 2024-05-08T18:05:26.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.804+0000 7fa1d812b800 1 -- start start 2024-05-08T18:05:26.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.805+0000 7fa1d812b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 0x5580cd2eefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.805+0000 7fa1d812b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5580cd146a30 con 0x5580cd1f9fb0 2024-05-08T18:05:26.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.805+0000 7fa1d3d4b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 0x5580cd2eefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.805+0000 7fa1d3d4b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 0x5580cd2eefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43824/0 (socket says 172.21.15.50:43824) 2024-05-08T18:05:26.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.805+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/3133009843 learned_addr learned my addr 172.21.15.50:0/3133009843 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.805+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/3133009843 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580cd2ef5b0 con 0x5580cd1f9fb0 2024-05-08T18:05:26.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.805+0000 7fa1d3d4b640 1 --2- 172.21.15.50:0/3133009843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 0x5580cd2eefe0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4055040 tx=0x7fa1c404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc82c37be7d7c466 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.805+0000 7fa0c1e0b640 1 -- 172.21.15.50:0/3133009843 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1c4057040 con 0x5580cd1f9fb0 2024-05-08T18:05:26.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.806+0000 7fa0c1e0b640 1 -- 172.21.15.50:0/3133009843 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c4052720 con 0x5580cd1f9fb0 2024-05-08T18:05:26.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.806+0000 7fa0c1e0b640 1 -- 172.21.15.50:0/3133009843 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1c4052a40 con 0x5580cd1f9fb0 2024-05-08T18:05:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.806+0000 7fa1d812b800 1 -- 172.21.15.50:0/3133009843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 msgr2=0x5580cd2eefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.806+0000 7fa1d812b800 1 --2- 172.21.15.50:0/3133009843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 0x5580cd2eefe0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4055040 tx=0x7fa1c404c000 comp rx=0 tx=0).stop 2024-05-08T18:05:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.806+0000 7fa1d812b800 1 -- 172.21.15.50:0/3133009843 shutdown_connections 2024-05-08T18:05:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.806+0000 7fa1d812b800 1 --2- 172.21.15.50:0/3133009843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 0x5580cd2eefe0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.806+0000 7fa1d812b800 1 -- 172.21.15.50:0/3133009843 >> 172.21.15.50:0/3133009843 conn(0x5580cd1f5b70 msgr2=0x5580cd2ea900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.806+0000 7fa1d812b800 1 -- 172.21.15.50:0/3133009843 shutdown_connections 2024-05-08T18:05:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.806+0000 7fa1d812b800 1 -- 172.21.15.50:0/3133009843 wait complete. 2024-05-08T18:05:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.807+0000 7fa1d812b800 1 Processor -- start 2024-05-08T18:05:26.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.807+0000 7fa1d812b800 1 -- start start 2024-05-08T18:05:26.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.807+0000 7fa1d812b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 0x5580cd2f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.807+0000 7fa1d812b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5580cd1fba00 con 0x5580cd1f9fb0 2024-05-08T18:05:26.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.807+0000 7fa1d3d4b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 0x5580cd2f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.807+0000 7fa1d3d4b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 0x5580cd2f6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43836/0 (socket says 172.21.15.50:43836) 2024-05-08T18:05:26.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.807+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/1100739226 learned_addr learned my addr 172.21.15.50:0/1100739226 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.807+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/1100739226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580cd2f6fe0 con 0x5580cd1f9fb0 2024-05-08T18:05:26.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.807+0000 7fa1d3d4b640 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 0x5580cd2f6aa0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40524f0 tx=0x7fa1c4003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.808+0000 7fa0bbfff640 1 -- 172.21.15.50:0/1100739226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1c4073070 con 0x5580cd1f9fb0 2024-05-08T18:05:26.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.808+0000 7fa0bbfff640 1 -- 172.21.15.50:0/1100739226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c405d440 con 0x5580cd1f9fb0 2024-05-08T18:05:26.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.808+0000 7fa1d812b800 1 -- 172.21.15.50:0/1100739226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5580cd2f7270 con 0x5580cd1f9fb0 2024-05-08T18:05:26.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.808+0000 7fa0bbfff640 1 -- 172.21.15.50:0/1100739226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1c4057070 con 0x5580cd1f9fb0 2024-05-08T18:05:26.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.808+0000 7fa1d812b800 1 -- 172.21.15.50:0/1100739226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580cd2f9e80 con 0x5580cd1f9fb0 2024-05-08T18:05:26.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.808+0000 7fa0bbfff640 1 -- 172.21.15.50:0/1100739226 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa1c4076070 con 0x5580cd1f9fb0 2024-05-08T18:05:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.809+0000 7fa0bbfff640 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa0bc03bb50 0x7fa0bc03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.809+0000 7fa0bbfff640 1 -- 172.21.15.50:0/1100739226 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fa1c4094020 con 0x5580cd1f9fb0 2024-05-08T18:05:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.809+0000 7fa1d354a640 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa0bc03bb50 0x7fa0bc03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.809+0000 7fa1d812b800 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5580cd1fa910 0x5580cd2fdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.809+0000 7fa1d812b800 1 -- 172.21.15.50:0/1100739226 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd2fe210 con 0x5580cd1fa910 2024-05-08T18:05:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.809+0000 7fa1d454c640 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5580cd1fa910 0x5580cd2fdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.809+0000 7fa1d354a640 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa0bc03bb50 0x7fa0bc03e010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc001e30 tx=0x7fa1bc001390 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.809+0000 7fa1d454c640 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5580cd1fa910 0x5580cd2fdc40 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.810+0000 7fa1d454c640 1 -- 172.21.15.50:0/1100739226 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd1fa910 2024-05-08T18:05:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.810+0000 7fa1d812b800 1 -- 172.21.15.50:0/1100739226 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd2fea90 con 0x5580cd1fa910 2024-05-08T18:05:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.810+0000 7fa1d454c640 1 -- 172.21.15.50:0/1100739226 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd1fa910 2024-05-08T18:05:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.810+0000 7fa1d812b800 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5580cd3003a0 0x5580cd302830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.810+0000 7fa1d812b800 1 -- 172.21.15.50:0/1100739226 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd302fc0 con 0x5580cd3003a0 2024-05-08T18:05:26.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.810+0000 7fa1d3d4b640 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5580cd3003a0 0x5580cd302830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.811+0000 7fa1d3d4b640 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5580cd3003a0 0x5580cd302830 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.811+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/1100739226 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fa1c4075030 con 0x5580cd3003a0 2024-05-08T18:05:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.811+0000 7fa1d812b800 1 -- 172.21.15.50:0/1100739226 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd2ff2e0 con 0x5580cd1fa910 2024-05-08T18:05:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.811+0000 7fa1d454c640 1 -- 172.21.15.50:0/1100739226 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd1fa910 2024-05-08T18:05:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.811+0000 7fa1d812b800 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5580cd304420 0x5580cd306810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.811+0000 7fa1d812b800 1 -- 172.21.15.50:0/1100739226 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd306f00 con 0x5580cd304420 2024-05-08T18:05:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.811+0000 7fa1d354a640 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5580cd304420 0x5580cd306810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.812+0000 7fa1d354a640 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5580cd304420 0x5580cd306810 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.812+0000 7fa1d354a640 1 -- 172.21.15.50:0/1100739226 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fa1bc00bd50 con 0x5580cd304420 2024-05-08T18:05:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.812+0000 7fa1d812b800 1 Processor -- start 2024-05-08T18:05:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.812+0000 7fa1d812b800 1 -- start start 2024-05-08T18:05:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.812+0000 7fa1d812b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd315270 0x5580cd315690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.812+0000 7fa1d812b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5580cd0ed330 con 0x5580cd315270 2024-05-08T18:05:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.813+0000 7fa1d3d4b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd315270 0x5580cd315690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.813+0000 7fa1d3d4b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd315270 0x5580cd315690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43850/0 (socket says 172.21.15.50:43850) 2024-05-08T18:05:26.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.813+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2631106080 learned_addr learned my addr 172.21.15.50:0/2631106080 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.819 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.813+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2631106080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580cd315bd0 con 0x5580cd315270 2024-05-08T18:05:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.813+0000 7fa1d3d4b640 1 --2- 172.21.15.50:0/2631106080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd315270 0x5580cd315690 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa1c405da90 tx=0x7fa1c40666e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.813+0000 7fa0b97fa640 1 -- 172.21.15.50:0/2631106080 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1c405c920 con 0x5580cd315270 2024-05-08T18:05:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.813+0000 7fa0b97fa640 1 -- 172.21.15.50:0/2631106080 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c405cae0 con 0x5580cd315270 2024-05-08T18:05:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.813+0000 7fa1d812b800 1 -- 172.21.15.50:0/2631106080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5580cd316640 con 0x5580cd315270 2024-05-08T18:05:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.813+0000 7fa0b97fa640 1 -- 172.21.15.50:0/2631106080 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1c405ce00 con 0x5580cd315270 2024-05-08T18:05:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.814+0000 7fa1d812b800 1 -- 172.21.15.50:0/2631106080 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580cd316b40 con 0x5580cd315270 2024-05-08T18:05:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.814+0000 7fa0b97fa640 1 -- 172.21.15.50:0/2631106080 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa1c407f070 con 0x5580cd315270 2024-05-08T18:05:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.814+0000 7fa0b97fa640 1 --2- 172.21.15.50:0/2631106080 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa0ac03bba0 0x7fa0ac03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.814+0000 7fa0b97fa640 1 -- 172.21.15.50:0/2631106080 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fa1c4095c60 con 0x5580cd315270 2024-05-08T18:05:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.815+0000 7fa1d354a640 1 --2- 172.21.15.50:0/2631106080 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa0ac03bba0 0x7fa0ac03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.815+0000 7fa1d812b800 1 Processor -- start 2024-05-08T18:05:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.815+0000 7fa1d812b800 1 -- start start 2024-05-08T18:05:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.815+0000 7fa1d354a640 1 --2- 172.21.15.50:0/2631106080 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa0ac03bba0 0x7fa0ac03e060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x5580cd2fb130 tx=0x7fa1bc00f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.815+0000 7fa1d812b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd322420 0x5580cd322840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.815+0000 7fa1d812b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5580cd2ef870 con 0x5580cd322420 2024-05-08T18:05:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.815+0000 7fa1d3d4b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd322420 0x5580cd322840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.815+0000 7fa1d3d4b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd322420 0x5580cd322840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43860/0 (socket says 172.21.15.50:43860) 2024-05-08T18:05:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.815+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/1614172521 learned_addr learned my addr 172.21.15.50:0/1614172521 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.816+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/1614172521 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580cd322e10 con 0x5580cd322420 2024-05-08T18:05:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.816+0000 7fa1d3d4b640 1 --2- 172.21.15.50:0/1614172521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd322420 0x5580cd322840 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4006a90 tx=0x7fa1c40533f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1274dd012315527 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.816+0000 7fa0b2ffd640 1 -- 172.21.15.50:0/1614172521 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1c409f040 con 0x5580cd322420 2024-05-08T18:05:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.816+0000 7fa0b2ffd640 1 -- 172.21.15.50:0/1614172521 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c4076880 con 0x5580cd322420 2024-05-08T18:05:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.816+0000 7fa0b2ffd640 1 -- 172.21.15.50:0/1614172521 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1c408b0b0 con 0x5580cd322420 2024-05-08T18:05:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.816+0000 7fa1d812b800 1 -- 172.21.15.50:0/1614172521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd322420 msgr2=0x5580cd322840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.816+0000 7fa1d812b800 1 --2- 172.21.15.50:0/1614172521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd322420 0x5580cd322840 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4006a90 tx=0x7fa1c40533f0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.817+0000 7fa1d812b800 1 -- 172.21.15.50:0/1614172521 shutdown_connections 2024-05-08T18:05:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.817+0000 7fa1d812b800 1 --2- 172.21.15.50:0/1614172521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd322420 0x5580cd322840 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.817+0000 7fa1d812b800 1 -- 172.21.15.50:0/1614172521 >> 172.21.15.50:0/1614172521 conn(0x5580cd31e6f0 msgr2=0x5580cd31c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.817+0000 7fa1d812b800 1 -- 172.21.15.50:0/1614172521 shutdown_connections 2024-05-08T18:05:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.817+0000 7fa1d812b800 1 -- 172.21.15.50:0/1614172521 wait complete. 2024-05-08T18:05:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.817+0000 7fa1d812b800 1 Processor -- start 2024-05-08T18:05:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.817+0000 7fa1d812b800 1 -- start start 2024-05-08T18:05:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.817+0000 7fa1d812b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd322420 0x5580cd3296a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.817+0000 7fa1d812b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5580cd323470 con 0x5580cd322420 2024-05-08T18:05:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.817+0000 7fa1d3d4b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd322420 0x5580cd3296a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.817+0000 7fa1d3d4b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd322420 0x5580cd3296a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43874/0 (socket says 172.21.15.50:43874) 2024-05-08T18:05:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.817+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 learned_addr learned my addr 172.21.15.50:0/2641618634 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.818+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580cd329be0 con 0x5580cd322420 2024-05-08T18:05:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.818+0000 7fa1d3d4b640 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd322420 0x5580cd3296a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4006a90 tx=0x7fa1c4087fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.818+0000 7fa0b17fa640 1 -- 172.21.15.50:0/2641618634 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1c408b2e0 con 0x5580cd322420 2024-05-08T18:05:26.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.818+0000 7fa0b17fa640 1 -- 172.21.15.50:0/2641618634 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c4076360 con 0x5580cd322420 2024-05-08T18:05:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.818+0000 7fa0b17fa640 1 -- 172.21.15.50:0/2641618634 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa1c408b480 con 0x5580cd322420 2024-05-08T18:05:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.818+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5580cd329be0 con 0x5580cd322420 2024-05-08T18:05:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.818+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580cd329be0 con 0x5580cd322420 2024-05-08T18:05:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.819+0000 7fa0b17fa640 1 -- 172.21.15.50:0/2641618634 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa1c4054080 con 0x5580cd322420 2024-05-08T18:05:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.819+0000 7fa0b17fa640 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa09c03bba0 0x7fa09c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.819+0000 7fa0b17fa640 1 -- 172.21.15.50:0/2641618634 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fa1c40b2c90 con 0x5580cd322420 2024-05-08T18:05:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.819+0000 7fa1d354a640 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa09c03bba0 0x7fa09c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.820+0000 7fa1d354a640 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa09c03bba0 0x7fa09c03e060 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x5580cd307300 tx=0x7fa1bc010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.820+0000 7fa1d812b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.820+0000 7fa1d812b800 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5580cd334e70 0x5580cd337330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.820+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd337a80 con 0x5580cd334e70 2024-05-08T18:05:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.820+0000 7fa1d454c640 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5580cd334e70 0x5580cd337330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.820+0000 7fa1d454c640 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5580cd334e70 0x5580cd337330 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.820+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd334e70 2024-05-08T18:05:26.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.820+0000 7fa1d812b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.820+0000 7fa1d812b800 20 realm 2024-05-08T18:05:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.820+0000 7fa1d812b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.820+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd337e90 con 0x5580cd334e70 2024-05-08T18:05:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.821+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd334e70 2024-05-08T18:05:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.821+0000 7fa1d812b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.821+0000 7fa1d812b800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.821+0000 7fa1d812b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.821+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338760 con 0x5580cd334e70 2024-05-08T18:05:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.821+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd334e70 2024-05-08T18:05:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.821+0000 7fa1d812b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.821+0000 7fa1d812b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.821+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd334e70 2024-05-08T18:05:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.821+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd334e70 2024-05-08T18:05:26.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.821+0000 7fa1d812b800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.821+0000 7fa1d812b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.821+0000 7fa1d812b800 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5580cd339860 0x5580cd33bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.821+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd33c260 con 0x5580cd339860 2024-05-08T18:05:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.822+0000 7fa1d3d4b640 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5580cd339860 0x5580cd33bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.822+0000 7fa1d3d4b640 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5580cd339860 0x5580cd33bc50 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.822+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.822+0000 7fa1d812b800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:26.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.822+0000 7fa1d812b800 20 searching for the correct realm 2024-05-08T18:05:26.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.822+0000 7fa1d812b800 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5580cd33e210 0x5580cd3406d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.822+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd340da0 con 0x5580cd33e210 2024-05-08T18:05:26.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.822+0000 7fa1d354a640 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5580cd33e210 0x5580cd3406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.823+0000 7fa1d354a640 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5580cd33e210 0x5580cd3406d0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.823+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1bc037040 con 0x5580cd33e210 2024-05-08T18:05:26.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.823+0000 7fa1d354a640 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa1bc00f070 0x7fa1bc02e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.823+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd340da0 con 0x7fa1bc00f070 2024-05-08T18:05:26.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.823+0000 7fa1d454c640 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa1bc00f070 0x7fa1bc02e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.823+0000 7fa1d454c640 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa1bc00f070 0x7fa1bc02e910 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.824+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x7fa1bc00f070 2024-05-08T18:05:26.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.824+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd339860 2024-05-08T18:05:26.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.824+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.824+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd334e70 2024-05-08T18:05:26.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.824+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd334e70 2024-05-08T18:05:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.824+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd334e70 2024-05-08T18:05:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.824+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd334e70 2024-05-08T18:05:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.824+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd339860 2024-05-08T18:05:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.825+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.825+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd339860 2024-05-08T18:05:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.825+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.825+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x7fa1bc00f070 2024-05-08T18:05:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.825+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x7fa1bc00f070 2024-05-08T18:05:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.825+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd33e210 2024-05-08T18:05:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.826+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1bc037040 con 0x5580cd33e210 2024-05-08T18:05:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.826+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd334e70 2024-05-08T18:05:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.826+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd334e70 2024-05-08T18:05:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.826+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd339860 2024-05-08T18:05:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.826+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.826+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd334e70 2024-05-08T18:05:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.827+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd334e70 2024-05-08T18:05:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.827+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x7fa1bc00f070 2024-05-08T18:05:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.827+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x7fa1bc00f070 2024-05-08T18:05:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.827+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd33e210 2024-05-08T18:05:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.827+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1bc037040 con 0x5580cd33e210 2024-05-08T18:05:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.827+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd33e210 2024-05-08T18:05:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.827+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1bc037040 con 0x5580cd33e210 2024-05-08T18:05:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.827+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd339860 2024-05-08T18:05:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.828+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.828+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd33e210 2024-05-08T18:05:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.828+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1bc037040 con 0x5580cd33e210 2024-05-08T18:05:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.828+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd33e210 2024-05-08T18:05:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.828+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fa1bc037040 con 0x5580cd33e210 2024-05-08T18:05:26.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.828+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd334e70 2024-05-08T18:05:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.828+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd334e70 2024-05-08T18:05:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.828+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd334e70 2024-05-08T18:05:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.829+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd334e70 2024-05-08T18:05:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.829+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd339860 2024-05-08T18:05:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.829+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.829+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd33e210 2024-05-08T18:05:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.829+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1bc037040 con 0x5580cd33e210 2024-05-08T18:05:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.829+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x7fa1bc00f070 2024-05-08T18:05:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.829+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x7fa1bc00f070 2024-05-08T18:05:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.829+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd334e70 2024-05-08T18:05:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.830+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd334e70 2024-05-08T18:05:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.830+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd339860 2024-05-08T18:05:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.830+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.830+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd33e210 2024-05-08T18:05:26.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.830+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1bc037040 con 0x5580cd33e210 2024-05-08T18:05:26.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.830+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd339860 2024-05-08T18:05:26.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.831+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.831+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x7fa1bc00f070 2024-05-08T18:05:26.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.831+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x7fa1bc00f070 2024-05-08T18:05:26.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.831+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd33e210 2024-05-08T18:05:26.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.831+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1bc037040 con 0x5580cd33e210 2024-05-08T18:05:26.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.831+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd339860 2024-05-08T18:05:26.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x5580cd334e70 2024-05-08T18:05:26.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd334e70 2024-05-08T18:05:26.842 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5580cd338b00 con 0x7fa1bc00f070 2024-05-08T18:05:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x7fa1bc00f070 2024-05-08T18:05:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d812b800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d812b800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d812b800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d812b800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d812b800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d812b800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d812b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd33cb20 con 0x5580cd334e70 2024-05-08T18:05:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd334e70 2024-05-08T18:05:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d812b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d812b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.832+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd33cb20 con 0x5580cd334e70 2024-05-08T18:05:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x5580cd334e70 2024-05-08T18:05:26.843 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d812b800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d812b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd33cb20 con 0x5580cd33e210 2024-05-08T18:05:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fa1bc037040 con 0x5580cd33e210 2024-05-08T18:05:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d812b800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d812b800 20 zone default found 2024-05-08T18:05:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d812b800 4 Realm: () 2024-05-08T18:05:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d812b800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d812b800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d812b800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d812b800 20 zonegroup default 2024-05-08T18:05:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d812b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd33d490 con 0x5580cd339860 2024-05-08T18:05:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d812b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d812b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.833+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd342c60 con 0x5580cd33e210 2024-05-08T18:05:26.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.834+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa1bc037040 con 0x5580cd33e210 2024-05-08T18:05:26.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.834+0000 7fa1d812b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.834+0000 7fa1d812b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.834+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd3430c0 con 0x5580cd33e210 2024-05-08T18:05:26.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.834+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa1bc037040 con 0x5580cd33e210 2024-05-08T18:05:26.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.834+0000 7fa1d812b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.834+0000 7fa1d812b800 20 started sync module instance, tier type = 2024-05-08T18:05:26.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.834+0000 7fa1d812b800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:26.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.834+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd3420a0 con 0x7fa1bc00f070 2024-05-08T18:05:26.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.835+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1cc00a4a0 con 0x7fa1bc00f070 2024-05-08T18:05:26.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.835+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94011686921632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd3420a0 con 0x7fa1bc00f070 2024-05-08T18:05:26.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.835+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd343c00 con 0x5580cd334e70 2024-05-08T18:05:26.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.836+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1cc040040 con 0x5580cd334e70 2024-05-08T18:05:26.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.836+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94011686921632] v27'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1cc040040 con 0x7fa1bc00f070 2024-05-08T18:05:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.836+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94011686926064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd343c00 con 0x5580cd334e70 2024-05-08T18:05:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.836+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd341bf0 con 0x5580cd33e210 2024-05-08T18:05:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.837+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1bc037040 con 0x5580cd33e210 2024-05-08T18:05:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.837+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94011686930160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd344fc0 con 0x5580cd33e210 2024-05-08T18:05:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.837+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd345a10 con 0x7fa1bc00f070 2024-05-08T18:05:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.837+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94011686926064] v27'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1cc040040 con 0x5580cd334e70 2024-05-08T18:05:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.838+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94011686930160] v27'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1bc037040 con 0x5580cd33e210 2024-05-08T18:05:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.838+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1cc040040 con 0x7fa1bc00f070 2024-05-08T18:05:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.838+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94011686931216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd343fd0 con 0x7fa1bc00f070 2024-05-08T18:05:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.838+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd346420 con 0x5580cd334e70 2024-05-08T18:05:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.839+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94011686931216] v27'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1cc040040 con 0x7fa1bc00f070 2024-05-08T18:05:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.839+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1cc040040 con 0x5580cd334e70 2024-05-08T18:05:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.839+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94011686933760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd346420 con 0x5580cd334e70 2024-05-08T18:05:26.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.839+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd346cb0 con 0x5580cd339860 2024-05-08T18:05:26.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.840+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.840+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94011686938112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd347550 con 0x5580cd339860 2024-05-08T18:05:26.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.841+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd347fa0 con 0x5580cd339860 2024-05-08T18:05:26.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.841+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94011686933760] v27'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1cc040040 con 0x5580cd334e70 2024-05-08T18:05:26.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.841+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94011686938112] v27'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.842+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.842+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94011686943376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd348390 con 0x5580cd339860 2024-05-08T18:05:26.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.842+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd348be0 con 0x5580cd339860 2024-05-08T18:05:26.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.843+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94011686943376] v27'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.843+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.843+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94011686945680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd348c60 con 0x5580cd339860 2024-05-08T18:05:26.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.843+0000 7fa1d812b800 20 add_watcher() i=0 2024-05-08T18:05:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.843+0000 7fa1d812b800 20 add_watcher() i=1 2024-05-08T18:05:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.843+0000 7fa1d812b800 20 add_watcher() i=2 2024-05-08T18:05:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.843+0000 7fa1d812b800 20 add_watcher() i=3 2024-05-08T18:05:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.843+0000 7fa1d812b800 20 add_watcher() i=4 2024-05-08T18:05:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.843+0000 7fa1d812b800 20 add_watcher() i=5 2024-05-08T18:05:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.843+0000 7fa1d812b800 20 add_watcher() i=6 2024-05-08T18:05:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.844+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94011686945680] v27'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.844+0000 7fa1d812b800 20 add_watcher() i=7 2024-05-08T18:05:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.844+0000 7fa1d812b800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.844+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd34bf70 con 0x5580cd334e70 2024-05-08T18:05:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.844+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fa1cc040040 con 0x5580cd334e70 2024-05-08T18:05:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.844+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94011686947104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd3491f0 con 0x5580cd334e70 2024-05-08T18:05:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.845+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94011686947104] v27'178 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa1cc040040 con 0x5580cd334e70 2024-05-08T18:05:26.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.845+0000 7fa089fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.845+0000 7fa1d812b800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.845+0000 7fa1d812b800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.845+0000 7fa1d812b800 5 note: GC not initialized 2024-05-08T18:05:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.846+0000 7fa088fe1640 20 reqs_thread_entry: start 2024-05-08T18:05:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.846+0000 7fa1d812b800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.846+0000 7fa1d812b800 20 Filter name: none 2024-05-08T18:05:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.846+0000 7fa0027fc640 20 reqs_thread_entry: start 2024-05-08T18:05:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.846+0000 7fa1d812b800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-05-08T18:05:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.846+0000 7fa1d812b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.846+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd37fad0 con 0x7fa1bc00f070 2024-05-08T18:05:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.846+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv35 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fa1cc040040 con 0x7fa1bc00f070 2024-05-08T18:05:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.846+0000 7fa1d812b800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.846+0000 7fa1d812b800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-05-08T18:05:26.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.846+0000 7fa1d812b800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:26.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.846+0000 7fa1d812b800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 : miss 2024-05-08T18:05:26.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.846+0000 7fa1d812b800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.846+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.7 8:e716d2f9:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd381550 con 0x5580cd339860 2024-05-08T18:05:26.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.847+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv2 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.847+0000 7fa1d812b800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:26.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.847+0000 7fa1d812b800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 info.flags=0x17 2024-05-08T18:05:26.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.847+0000 7fa1d812b800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 to cache LRU end 2024-05-08T18:05:26.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.847+0000 7fa1d812b800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:26.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.847+0000 7fa1d812b800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:26.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.847+0000 7fa1d812b800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 2024-05-08T18:05:26.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.847+0000 7fa1d812b800 20 get_obj_state: octx=0x7fa1c4093f20 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x5580cd383c18 s->prefetch_data=0 2024-05-08T18:05:26.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.847+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 3.32s0 3:4d0cffb4:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5580cd384a40 con 0x5580cd33e210 2024-05-08T18:05:26.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.847+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=848b,stat out=16b] v0'0 uv2 ondisk = 0) ==== 262+0+864 (crc 0 0 0) 0x7fa1bc037040 con 0x5580cd33e210 2024-05-08T18:05:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.847+0000 7fa1d812b800 10 manifest: total_size = 5 2024-05-08T18:05:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.847+0000 7fa1d812b800 20 get_obj_state: setting s->obj_tag to _KwPrxSqvPMX1pC4goIdVCXETGIFe8Zk 2024-05-08T18:05:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.848+0000 7fa089fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.848+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94011686947104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd37d420 con 0x5580cd334e70 2024-05-08T18:05:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.849+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94011686947104] v27'179 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa1cc040040 con 0x5580cd334e70 2024-05-08T18:05:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.849+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94011686921632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd37d420 con 0x7fa1bc00f070 2024-05-08T18:05:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.850+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94011686921632] v27'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1cc040040 con 0x7fa1bc00f070 2024-05-08T18:05:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.850+0000 7fa1d812b800 20 remove_watcher() i=0 2024-05-08T18:05:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.850+0000 7fa1d812b800 2 removed watcher, disabling cache 2024-05-08T18:05:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.850+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94011686926064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd37d420 con 0x5580cd334e70 2024-05-08T18:05:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.851+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94011686926064] v27'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1cc040040 con 0x5580cd334e70 2024-05-08T18:05:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.851+0000 7fa1d812b800 20 remove_watcher() i=1 2024-05-08T18:05:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.851+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94011686930160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd37d420 con 0x5580cd33e210 2024-05-08T18:05:26.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.852+0000 7fa1d354a640 1 -- 172.21.15.50:0/2641618634 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94011686930160] v27'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1bc037040 con 0x5580cd33e210 2024-05-08T18:05:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.852+0000 7fa1d812b800 20 remove_watcher() i=2 2024-05-08T18:05:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.852+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94011686931216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd37d420 con 0x7fa1bc00f070 2024-05-08T18:05:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.853+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94011686931216] v27'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1cc040040 con 0x7fa1bc00f070 2024-05-08T18:05:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.853+0000 7fa1d812b800 20 remove_watcher() i=3 2024-05-08T18:05:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.853+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94011686933760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd37d420 con 0x5580cd334e70 2024-05-08T18:05:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.854+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94011686933760] v27'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1cc040040 con 0x5580cd334e70 2024-05-08T18:05:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.854+0000 7fa1d812b800 20 remove_watcher() i=4 2024-05-08T18:05:26.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.854+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94011686938112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd37d420 con 0x5580cd339860 2024-05-08T18:05:26.861 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.855+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94011686938112] v27'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.855+0000 7fa1d812b800 20 remove_watcher() i=5 2024-05-08T18:05:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.855+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94011686943376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd37d420 con 0x5580cd339860 2024-05-08T18:05:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.856+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94011686943376] v27'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.856+0000 7fa1d812b800 20 remove_watcher() i=6 2024-05-08T18:05:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.856+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94011686945680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5580cd37d420 con 0x5580cd339860 2024-05-08T18:05:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.857+0000 7fa1d3d4b640 1 -- 172.21.15.50:0/2641618634 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94011686945680] v27'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa1c40b7020 con 0x5580cd339860 2024-05-08T18:05:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.857+0000 7fa1d812b800 20 remove_watcher() i=7 2024-05-08T18:05:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.857+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5580cd339860 msgr2=0x5580cd33bc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.857+0000 7fa1d812b800 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5580cd339860 0x5580cd33bc50 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.857+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5580cd334e70 msgr2=0x5580cd337330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5580cd334e70 0x5580cd337330 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa1bc00f070 msgr2=0x7fa1bc02e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa1bc00f070 0x7fa1bc02e910 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5580cd33e210 msgr2=0x5580cd3406d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5580cd33e210 0x5580cd3406d0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa09c03bba0 msgr2=0x7fa09c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa09c03bba0 0x7fa09c03e060 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x5580cd307300 tx=0x7fa1bc010000 comp rx=0 tx=0).stop 2024-05-08T18:05:26.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd322420 msgr2=0x5580cd3296a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 --2- 172.21.15.50:0/2641618634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd322420 0x5580cd3296a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4006a90 tx=0x7fa1c4087fa0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d454c640 1 -- 172.21.15.50:0/2641618634 reap_dead start 2024-05-08T18:05:26.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 shutdown_connections 2024-05-08T18:05:26.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 >> 172.21.15.50:0/2641618634 conn(0x5580cd31e6f0 msgr2=0x5580cd31bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 shutdown_connections 2024-05-08T18:05:26.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 -- 172.21.15.50:0/2641618634 wait complete. 2024-05-08T18:05:26.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 -- 172.21.15.50:0/2631106080 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa0ac03bba0 msgr2=0x7fa0ac03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.863 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 --2- 172.21.15.50:0/2631106080 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa0ac03bba0 0x7fa0ac03e060 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x5580cd2fb130 tx=0x7fa1bc00f000 comp rx=0 tx=0).stop 2024-05-08T18:05:26.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 -- 172.21.15.50:0/2631106080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd315270 msgr2=0x5580cd315690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 --2- 172.21.15.50:0/2631106080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd315270 0x5580cd315690 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa1c405da90 tx=0x7fa1c40666e0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 -- 172.21.15.50:0/2631106080 shutdown_connections 2024-05-08T18:05:26.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 --2- 172.21.15.50:0/2631106080 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa0ac03bba0 0x7fa0ac03e060 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 --2- 172.21.15.50:0/2631106080 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd315270 0x5580cd315690 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 -- 172.21.15.50:0/2631106080 >> 172.21.15.50:0/2631106080 conn(0x5580cd309310 msgr2=0x5580cd309710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 -- 172.21.15.50:0/2631106080 shutdown_connections 2024-05-08T18:05:26.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.858+0000 7fa1d812b800 1 -- 172.21.15.50:0/2631106080 wait complete. 2024-05-08T18:05:26.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.859+0000 7fa1d812b800 1 -- 172.21.15.50:0/1100739226 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5580cd3003a0 msgr2=0x5580cd302830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.859+0000 7fa1d812b800 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5580cd3003a0 0x5580cd302830 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.859+0000 7fa1d812b800 1 -- 172.21.15.50:0/1100739226 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5580cd1fa910 msgr2=0x5580cd2fdc40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.859+0000 7fa1d812b800 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5580cd1fa910 0x5580cd2fdc40 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.859+0000 7fa1d812b800 1 -- 172.21.15.50:0/1100739226 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5580cd304420 msgr2=0x5580cd306810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.859+0000 7fa1d812b800 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5580cd304420 0x5580cd306810 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.859+0000 7fa1d812b800 1 -- 172.21.15.50:0/1100739226 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa0bc03bb50 msgr2=0x7fa0bc03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.859+0000 7fa1d812b800 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa0bc03bb50 0x7fa0bc03e010 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc001e30 tx=0x7fa1bc001390 comp rx=0 tx=0).stop 2024-05-08T18:05:26.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.859+0000 7fa1d812b800 1 -- 172.21.15.50:0/1100739226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 msgr2=0x5580cd2f6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.859+0000 7fa1d812b800 1 --2- 172.21.15.50:0/1100739226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5580cd1f9fb0 0x5580cd2f6aa0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40524f0 tx=0x7fa1c4003020 comp rx=0 tx=0).stop 2024-05-08T18:05:26.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.859+0000 7fa1d454c640 1 -- 172.21.15.50:0/1100739226 reap_dead start 2024-05-08T18:05:26.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.859+0000 7fa1d812b800 1 -- 172.21.15.50:0/1100739226 shutdown_connections 2024-05-08T18:05:26.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.859+0000 7fa1d812b800 1 -- 172.21.15.50:0/1100739226 >> 172.21.15.50:0/1100739226 conn(0x5580cd1f5b70 msgr2=0x5580cd2e9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.859+0000 7fa1d812b800 1 -- 172.21.15.50:0/1100739226 shutdown_connections 2024-05-08T18:05:26.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.859+0000 7fa1d812b800 1 -- 172.21.15.50:0/1100739226 wait complete. 2024-05-08T18:05:26.866 INFO:teuthology.orchestra.run.smithi050.stdout:fooFoofooFooFULL_CONTROL 2024-05-08T18:05:26.866 DEBUG:tasks.util.rgw: raw result: fooFoofooFooFULL_CONTROL 2024-05-08T18:05:26.868 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=417 failed=False 2024-05-08T18:05:26.873 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:26.873 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-05-08T18:05:26.873 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-05-08T18:05:26.873 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2024-05-08T18:05:26.920 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:26.920 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:26.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.923+0000 7f051013e800 1 Processor -- start 2024-05-08T18:05:26.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.923+0000 7f051013e800 1 -- start start 2024-05-08T18:05:26.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.923+0000 7f051013e800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x558aa5887c20 0x558aa5888040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.923+0000 7f051013e800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x558aa58906d0 con 0x558aa5888580 2024-05-08T18:05:26.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.923+0000 7f051013e800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x558aa5727e20 con 0x558aa5887c20 2024-05-08T18:05:26.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.923+0000 7f050dc36640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x558aa5887c20 0x558aa5888040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.924+0000 7f050dc36640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x558aa5887c20 0x558aa5888040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43876/0 (socket says 172.21.15.50:43876) 2024-05-08T18:05:26.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.924+0000 7f050dc36640 1 -- 172.21.15.50:0/1135367606 learned_addr learned my addr 172.21.15.50:0/1135367606 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.924+0000 7f050dc36640 1 -- 172.21.15.50:0/1135367606 >> v1:172.21.15.46:6789/0 conn(0x558aa5888580 legacy=0x558aa588ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.924+0000 7f050dc36640 1 -- 172.21.15.50:0/1135367606 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aa5891900 con 0x558aa5887c20 2024-05-08T18:05:26.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.926+0000 7f050dc36640 1 --2- 172.21.15.50:0/1135367606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 0x558aa5888040 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f050000a520 tx=0x7f050001d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132bab48e346336b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.926+0000 7f050cc34640 1 -- 172.21.15.50:0/1135367606 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0500057070 con 0x558aa5887c20 2024-05-08T18:05:26.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.926+0000 7f050cc34640 1 -- 172.21.15.50:0/1135367606 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0500052570 con 0x558aa5887c20 2024-05-08T18:05:26.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.926+0000 7f050cc34640 1 -- 172.21.15.50:0/1135367606 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f050004ae60 con 0x558aa5887c20 2024-05-08T18:05:26.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.927+0000 7f051013e800 1 -- 172.21.15.50:0/1135367606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 msgr2=0x558aa5888040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.927+0000 7f051013e800 1 --2- 172.21.15.50:0/1135367606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 0x558aa5888040 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f050000a520 tx=0x7f050001d010 comp rx=0 tx=0).stop 2024-05-08T18:05:26.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.927+0000 7f051013e800 1 -- 172.21.15.50:0/1135367606 shutdown_connections 2024-05-08T18:05:26.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.927+0000 7f051013e800 1 --2- 172.21.15.50:0/1135367606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 0x558aa5888040 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.927+0000 7f051013e800 1 -- 172.21.15.50:0/1135367606 >> 172.21.15.50:0/1135367606 conn(0x558aa58812f0 msgr2=0x558aa5883710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.927+0000 7f051013e800 1 -- 172.21.15.50:0/1135367606 shutdown_connections 2024-05-08T18:05:26.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.927+0000 7f051013e800 1 -- 172.21.15.50:0/1135367606 wait complete. 2024-05-08T18:05:26.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.942+0000 7f051013e800 1 Processor -- start 2024-05-08T18:05:26.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.942+0000 7f051013e800 1 -- start start 2024-05-08T18:05:26.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.942+0000 7f051013e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 0x558aa597adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.942+0000 7f051013e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558aa57d2a30 con 0x558aa5887c20 2024-05-08T18:05:26.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.943+0000 7f050dc36640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 0x558aa597adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.943+0000 7f050dc36640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 0x558aa597adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43878/0 (socket says 172.21.15.50:43878) 2024-05-08T18:05:26.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.943+0000 7f050dc36640 1 -- 172.21.15.50:0/3022269086 learned_addr learned my addr 172.21.15.50:0/3022269086 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.943+0000 7f050dc36640 1 -- 172.21.15.50:0/3022269086 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aa597b390 con 0x558aa5887c20 2024-05-08T18:05:26.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.943+0000 7f050dc36640 1 --2- 172.21.15.50:0/3022269086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 0x558aa597adc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0500055040 tx=0x7f050004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e34390bd44bfee1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.943+0000 7f03fbe07640 1 -- 172.21.15.50:0/3022269086 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0500057040 con 0x558aa5887c20 2024-05-08T18:05:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.943+0000 7f03fbe07640 1 -- 172.21.15.50:0/3022269086 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0500052720 con 0x558aa5887c20 2024-05-08T18:05:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.943+0000 7f03fbe07640 1 -- 172.21.15.50:0/3022269086 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0500052a40 con 0x558aa5887c20 2024-05-08T18:05:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.944+0000 7f051013e800 1 -- 172.21.15.50:0/3022269086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 msgr2=0x558aa597adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.944+0000 7f051013e800 1 --2- 172.21.15.50:0/3022269086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 0x558aa597adc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0500055040 tx=0x7f050004c000 comp rx=0 tx=0).stop 2024-05-08T18:05:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.944+0000 7f051013e800 1 -- 172.21.15.50:0/3022269086 shutdown_connections 2024-05-08T18:05:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.944+0000 7f051013e800 1 --2- 172.21.15.50:0/3022269086 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 0x558aa597adc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.944+0000 7f051013e800 1 -- 172.21.15.50:0/3022269086 >> 172.21.15.50:0/3022269086 conn(0x558aa58812f0 msgr2=0x558aa59766e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.944+0000 7f051013e800 1 -- 172.21.15.50:0/3022269086 shutdown_connections 2024-05-08T18:05:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.944+0000 7f051013e800 1 -- 172.21.15.50:0/3022269086 wait complete. 2024-05-08T18:05:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.944+0000 7f051013e800 1 Processor -- start 2024-05-08T18:05:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.944+0000 7f051013e800 1 -- start start 2024-05-08T18:05:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.945+0000 7f051013e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 0x558aa5982810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.945+0000 7f051013e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558aa5890b40 con 0x558aa5887c20 2024-05-08T18:05:26.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.945+0000 7f050dc36640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 0x558aa5982810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.945+0000 7f050dc36640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 0x558aa5982810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43894/0 (socket says 172.21.15.50:43894) 2024-05-08T18:05:26.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.945+0000 7f050dc36640 1 -- 172.21.15.50:0/1951298305 learned_addr learned my addr 172.21.15.50:0/1951298305 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.945+0000 7f050dc36640 1 -- 172.21.15.50:0/1951298305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aa5982d50 con 0x558aa5887c20 2024-05-08T18:05:26.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.945+0000 7f050dc36640 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 0x558aa5982810 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f05000524f0 tx=0x7f0500003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.945+0000 7f03fa604640 1 -- 172.21.15.50:0/1951298305 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0500073070 con 0x558aa5887c20 2024-05-08T18:05:26.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.945+0000 7f03fa604640 1 -- 172.21.15.50:0/1951298305 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f050005d440 con 0x558aa5887c20 2024-05-08T18:05:26.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.945+0000 7f051013e800 1 -- 172.21.15.50:0/1951298305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558aa5982fe0 con 0x558aa5887c20 2024-05-08T18:05:26.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.945+0000 7f03fa604640 1 -- 172.21.15.50:0/1951298305 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0500057070 con 0x558aa5887c20 2024-05-08T18:05:26.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.945+0000 7f051013e800 1 -- 172.21.15.50:0/1951298305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aa5985c90 con 0x558aa5887c20 2024-05-08T18:05:26.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.946+0000 7f03fa604640 1 -- 172.21.15.50:0/1951298305 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f0500076070 con 0x558aa5887c20 2024-05-08T18:05:26.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.946+0000 7f03fa604640 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f03f403bb50 0x7f03f403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.947+0000 7f03fa604640 1 -- 172.21.15.50:0/1951298305 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f05000940e0 con 0x558aa5887c20 2024-05-08T18:05:26.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.947+0000 7f050d435640 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f03f403bb50 0x7f03f403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.947+0000 7f051013e800 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558aa5888580 0x558aa5989a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.947+0000 7f051013e800 1 -- 172.21.15.50:0/1951298305 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa5989fe0 con 0x558aa5888580 2024-05-08T18:05:26.949 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.947+0000 7f050e437640 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558aa5888580 0x558aa5989a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.947+0000 7f050d435640 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f03f403bb50 0x7f03f403e010 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x558aa57eacb0 tx=0x7f04f80013d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.947+0000 7f050e437640 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558aa5888580 0x558aa5989a50 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.947+0000 7f050e437640 1 -- 172.21.15.50:0/1951298305 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa5888580 2024-05-08T18:05:26.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.947+0000 7f051013e800 1 -- 172.21.15.50:0/1951298305 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa598a8c0 con 0x558aa5888580 2024-05-08T18:05:26.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.948+0000 7f050e437640 1 -- 172.21.15.50:0/1951298305 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa5888580 2024-05-08T18:05:26.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.948+0000 7f051013e800 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558aa598c210 0x558aa598e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.948+0000 7f051013e800 1 -- 172.21.15.50:0/1951298305 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa598ee30 con 0x558aa598c210 2024-05-08T18:05:26.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.948+0000 7f050dc36640 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558aa598c210 0x558aa598e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.948+0000 7f050dc36640 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558aa598c210 0x558aa598e6a0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.949+0000 7f050dc36640 1 -- 172.21.15.50:0/1951298305 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f0500075030 con 0x558aa598c210 2024-05-08T18:05:26.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.949+0000 7f051013e800 1 -- 172.21.15.50:0/1951298305 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa598b110 con 0x558aa5888580 2024-05-08T18:05:26.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.949+0000 7f050e437640 1 -- 172.21.15.50:0/1951298305 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa5888580 2024-05-08T18:05:26.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.949+0000 7f051013e800 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558aa5990290 0x558aa5992680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.950 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.949+0000 7f051013e800 1 -- 172.21.15.50:0/1951298305 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa5992d70 con 0x558aa5990290 2024-05-08T18:05:26.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.949+0000 7f050d435640 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558aa5990290 0x558aa5992680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.949+0000 7f050d435640 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558aa5990290 0x558aa5992680 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.950+0000 7f050d435640 1 -- 172.21.15.50:0/1951298305 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f04f8010600 con 0x558aa5990290 2024-05-08T18:05:26.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.950+0000 7f051013e800 1 Processor -- start 2024-05-08T18:05:26.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.950+0000 7f051013e800 1 -- start start 2024-05-08T18:05:26.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.950+0000 7f051013e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59a10e0 0x558aa59a1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.950+0000 7f051013e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558aa5779330 con 0x558aa59a10e0 2024-05-08T18:05:26.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.950+0000 7f050dc36640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59a10e0 0x558aa59a1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.950+0000 7f050dc36640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59a10e0 0x558aa59a1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43904/0 (socket says 172.21.15.50:43904) 2024-05-08T18:05:26.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.950+0000 7f050dc36640 1 -- 172.21.15.50:0/1867607914 learned_addr learned my addr 172.21.15.50:0/1867607914 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.951+0000 7f050dc36640 1 -- 172.21.15.50:0/1867607914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aa59a1a40 con 0x558aa59a10e0 2024-05-08T18:05:26.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.951+0000 7f050dc36640 1 --2- 172.21.15.50:0/1867607914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59a10e0 0x558aa59a1500 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f050005c720 tx=0x7f0500006670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.951+0000 7f03f37fe640 1 -- 172.21.15.50:0/1867607914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f050008ca50 con 0x558aa59a10e0 2024-05-08T18:05:26.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.951+0000 7f03f37fe640 1 -- 172.21.15.50:0/1867607914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f050008cc10 con 0x558aa59a10e0 2024-05-08T18:05:26.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.951+0000 7f03f37fe640 1 -- 172.21.15.50:0/1867607914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f050005c920 con 0x558aa59a10e0 2024-05-08T18:05:26.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.951+0000 7f051013e800 1 -- 172.21.15.50:0/1867607914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558aa59a24b0 con 0x558aa59a10e0 2024-05-08T18:05:26.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.951+0000 7f051013e800 1 -- 172.21.15.50:0/1867607914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aa59a29b0 con 0x558aa59a10e0 2024-05-08T18:05:26.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.952+0000 7f03f37fe640 1 -- 172.21.15.50:0/1867607914 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f050007f070 con 0x558aa59a10e0 2024-05-08T18:05:26.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.952+0000 7f03f37fe640 1 --2- 172.21.15.50:0/1867607914 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f03e803bba0 0x7f03e803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.952+0000 7f03f37fe640 1 -- 172.21.15.50:0/1867607914 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f0500095c60 con 0x558aa59a10e0 2024-05-08T18:05:26.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.952+0000 7f050d435640 1 --2- 172.21.15.50:0/1867607914 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f03e803bba0 0x7f03e803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.952+0000 7f051013e800 1 Processor -- start 2024-05-08T18:05:26.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.952+0000 7f051013e800 1 -- start start 2024-05-08T18:05:26.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.952+0000 7f050d435640 1 --2- 172.21.15.50:0/1867607914 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f03e803bba0 0x7f03e803e060 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x558aa5987060 tx=0x7f04f8006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.952+0000 7f051013e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59ae1d0 0x558aa59ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.952+0000 7f051013e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558aa597b650 con 0x558aa59ae1d0 2024-05-08T18:05:26.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.953+0000 7f050dc36640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59ae1d0 0x558aa59ae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.953+0000 7f050dc36640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59ae1d0 0x558aa59ae5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43916/0 (socket says 172.21.15.50:43916) 2024-05-08T18:05:26.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.953+0000 7f050dc36640 1 -- 172.21.15.50:0/173416320 learned_addr learned my addr 172.21.15.50:0/173416320 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.953+0000 7f050dc36640 1 -- 172.21.15.50:0/173416320 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aa59aebc0 con 0x558aa59ae1d0 2024-05-08T18:05:26.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.953+0000 7f050dc36640 1 --2- 172.21.15.50:0/173416320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59ae1d0 0x558aa59ae5f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f05000067c0 tx=0x7f0500078d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2115872f1466c5d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.953+0000 7f03f17fa640 1 -- 172.21.15.50:0/173416320 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f050009f040 con 0x558aa59ae1d0 2024-05-08T18:05:26.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.953+0000 7f03f17fa640 1 -- 172.21.15.50:0/173416320 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f050005cac0 con 0x558aa59ae1d0 2024-05-08T18:05:26.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.953+0000 7f03f17fa640 1 -- 172.21.15.50:0/173416320 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0500093b70 con 0x558aa59ae1d0 2024-05-08T18:05:26.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.954+0000 7f051013e800 1 -- 172.21.15.50:0/173416320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59ae1d0 msgr2=0x558aa59ae5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.954+0000 7f051013e800 1 --2- 172.21.15.50:0/173416320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59ae1d0 0x558aa59ae5f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f05000067c0 tx=0x7f0500078d50 comp rx=0 tx=0).stop 2024-05-08T18:05:26.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.954+0000 7f051013e800 1 -- 172.21.15.50:0/173416320 shutdown_connections 2024-05-08T18:05:26.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.954+0000 7f051013e800 1 --2- 172.21.15.50:0/173416320 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59ae1d0 0x558aa59ae5f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.954+0000 7f051013e800 1 -- 172.21.15.50:0/173416320 >> 172.21.15.50:0/173416320 conn(0x558aa59aa530 msgr2=0x558aa59a8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:26.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.954+0000 7f051013e800 1 -- 172.21.15.50:0/173416320 shutdown_connections 2024-05-08T18:05:26.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.954+0000 7f051013e800 1 -- 172.21.15.50:0/173416320 wait complete. 2024-05-08T18:05:26.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.954+0000 7f051013e800 1 Processor -- start 2024-05-08T18:05:26.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.954+0000 7f051013e800 1 -- start start 2024-05-08T18:05:26.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.955+0000 7f051013e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59ae1d0 0x558aa59b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.955+0000 7f051013e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558aa59af270 con 0x558aa59ae1d0 2024-05-08T18:05:26.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.955+0000 7f050dc36640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59ae1d0 0x558aa59b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.955+0000 7f050dc36640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59ae1d0 0x558aa59b55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43926/0 (socket says 172.21.15.50:43926) 2024-05-08T18:05:26.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.955+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 learned_addr learned my addr 172.21.15.50:0/1699569594 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:26.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.955+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558aa59b5b30 con 0x558aa59ae1d0 2024-05-08T18:05:26.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.955+0000 7f050dc36640 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59ae1d0 0x558aa59b55f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f050008a420 tx=0x7f050004cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.955+0000 7f03df7fe640 1 -- 172.21.15.50:0/1699569594 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f050009e040 con 0x558aa59ae1d0 2024-05-08T18:05:26.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.955+0000 7f03df7fe640 1 -- 172.21.15.50:0/1699569594 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0500078a70 con 0x558aa59ae1d0 2024-05-08T18:05:26.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.955+0000 7f03df7fe640 1 -- 172.21.15.50:0/1699569594 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0500053240 con 0x558aa59ae1d0 2024-05-08T18:05:26.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.955+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558aa59b5b30 con 0x558aa59ae1d0 2024-05-08T18:05:26.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.955+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558aa59b5b30 con 0x558aa59ae1d0 2024-05-08T18:05:26.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.956+0000 7f03df7fe640 1 -- 172.21.15.50:0/1699569594 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f05000533e0 con 0x558aa59ae1d0 2024-05-08T18:05:26.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.956+0000 7f03df7fe640 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f03d803bb50 0x7f03d803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.956+0000 7f03df7fe640 1 -- 172.21.15.50:0/1699569594 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f05000afc90 con 0x558aa59ae1d0 2024-05-08T18:05:26.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.957+0000 7f050d435640 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f03d803bb50 0x7f03d803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.957+0000 7f050d435640 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f03d803bb50 0x7f03d803e010 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x558aa5993170 tx=0x7f04f8011000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.957+0000 7f051013e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.957+0000 7f051013e800 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558aa59c0d30 0x558aa59c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.957+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c3940 con 0x558aa59c0d30 2024-05-08T18:05:26.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.957+0000 7f050e437640 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558aa59c0d30 0x558aa59c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.957+0000 7f050e437640 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558aa59c0d30 0x558aa59c31f0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa59c0d30 2024-05-08T18:05:26.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f051013e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f051013e800 20 realm 2024-05-08T18:05:26.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f051013e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c3d50 con 0x558aa59c0d30 2024-05-08T18:05:26.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa59c0d30 2024-05-08T18:05:26.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f051013e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f051013e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:26.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f051013e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c4620 con 0x558aa59c0d30 2024-05-08T18:05:26.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa59c0d30 2024-05-08T18:05:26.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f051013e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f051013e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c0d30 2024-05-08T18:05:26.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa59c0d30 2024-05-08T18:05:26.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f051013e800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:26.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f051013e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f051013e800 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558aa59c5720 0x558aa59c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.958+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c8120 con 0x558aa59c5720 2024-05-08T18:05:26.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.959+0000 7f050dc36640 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558aa59c5720 0x558aa59c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.959+0000 7f050dc36640 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558aa59c5720 0x558aa59c7b10 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.959+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.959+0000 7f051013e800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:26.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.959+0000 7f051013e800 20 searching for the correct realm 2024-05-08T18:05:26.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.959+0000 7f051013e800 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558aa59ca0d0 0x558aa59cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.959+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59ccc60 con 0x558aa59ca0d0 2024-05-08T18:05:26.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.960+0000 7f050d435640 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558aa59ca0d0 0x558aa59cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.960+0000 7f050d435640 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558aa59ca0d0 0x558aa59cc590 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.960+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f04f8036040 con 0x558aa59ca0d0 2024-05-08T18:05:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.960+0000 7f050d435640 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f04f80393c0 0x7f04f80397c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.960+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59ccc60 con 0x7f04f80393c0 2024-05-08T18:05:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.960+0000 7f050e437640 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f04f80393c0 0x7f04f80397c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.961+0000 7f050e437640 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f04f80393c0 0x7f04f80397c0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.961+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f050800a4a0 con 0x7f04f80393c0 2024-05-08T18:05:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.961+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c5720 2024-05-08T18:05:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.961+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.961+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c0d30 2024-05-08T18:05:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.962+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa59c0d30 2024-05-08T18:05:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.962+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c0d30 2024-05-08T18:05:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.962+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa59c0d30 2024-05-08T18:05:26.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.962+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c5720 2024-05-08T18:05:26.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.962+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.962+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c5720 2024-05-08T18:05:26.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.962+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.962+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x7f04f80393c0 2024-05-08T18:05:26.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.962+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f050800a4a0 con 0x7f04f80393c0 2024-05-08T18:05:26.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.963+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59ca0d0 2024-05-08T18:05:26.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.963+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f04f8036040 con 0x558aa59ca0d0 2024-05-08T18:05:26.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.963+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c0d30 2024-05-08T18:05:26.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.963+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa59c0d30 2024-05-08T18:05:26.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.963+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c5720 2024-05-08T18:05:26.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.963+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.963+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c0d30 2024-05-08T18:05:26.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.963+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa59c0d30 2024-05-08T18:05:26.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.963+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x7f04f80393c0 2024-05-08T18:05:26.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.964+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f050800a4a0 con 0x7f04f80393c0 2024-05-08T18:05:26.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.964+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59ca0d0 2024-05-08T18:05:26.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.964+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f04f8036040 con 0x558aa59ca0d0 2024-05-08T18:05:26.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.964+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59ca0d0 2024-05-08T18:05:26.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.964+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f04f8036040 con 0x558aa59ca0d0 2024-05-08T18:05:26.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.964+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c5720 2024-05-08T18:05:26.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.965+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.965+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59ca0d0 2024-05-08T18:05:26.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.965+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f04f8036040 con 0x558aa59ca0d0 2024-05-08T18:05:26.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.965+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59ca0d0 2024-05-08T18:05:26.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.965+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f04f8036040 con 0x558aa59ca0d0 2024-05-08T18:05:26.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.965+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c0d30 2024-05-08T18:05:26.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.965+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa59c0d30 2024-05-08T18:05:26.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.965+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c0d30 2024-05-08T18:05:26.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.965+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa59c0d30 2024-05-08T18:05:26.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.965+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c5720 2024-05-08T18:05:26.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.966+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.966+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59ca0d0 2024-05-08T18:05:26.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.966+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f04f8036040 con 0x558aa59ca0d0 2024-05-08T18:05:26.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.966+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x7f04f80393c0 2024-05-08T18:05:26.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.966+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f050800a4a0 con 0x7f04f80393c0 2024-05-08T18:05:26.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.966+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c0d30 2024-05-08T18:05:26.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.966+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa59c0d30 2024-05-08T18:05:26.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.966+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c5720 2024-05-08T18:05:26.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.967+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.967+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59ca0d0 2024-05-08T18:05:26.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.967+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f04f8036040 con 0x558aa59ca0d0 2024-05-08T18:05:26.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.967+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c5720 2024-05-08T18:05:26.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.967+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.967+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x7f04f80393c0 2024-05-08T18:05:26.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.967+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f050800a4a0 con 0x7f04f80393c0 2024-05-08T18:05:26.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.968+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59ca0d0 2024-05-08T18:05:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.968+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f04f8036040 con 0x558aa59ca0d0 2024-05-08T18:05:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.968+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c5720 2024-05-08T18:05:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.968+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.968+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x558aa59c0d30 2024-05-08T18:05:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.968+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa59c0d30 2024-05-08T18:05:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.968+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c49c0 con 0x7f04f80393c0 2024-05-08T18:05:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f050800a4a0 con 0x7f04f80393c0 2024-05-08T18:05:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c89e0 con 0x558aa59c0d30 2024-05-08T18:05:26.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa59c0d30 2024-05-08T18:05:26.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c89e0 con 0x558aa59c0d30 2024-05-08T18:05:26.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f050800a4a0 con 0x558aa59c0d30 2024-05-08T18:05:26.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:26.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c89e0 con 0x558aa59ca0d0 2024-05-08T18:05:26.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f04f8036040 con 0x558aa59ca0d0 2024-05-08T18:05:26.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:26.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 20 zone default found 2024-05-08T18:05:26.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 4 Realm: () 2024-05-08T18:05:26.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:26.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:26.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:26.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 20 zonegroup default 2024-05-08T18:05:26.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.969+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c93b0 con 0x558aa59c5720 2024-05-08T18:05:26.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.970+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.970+0000 7f051013e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.970+0000 7f051013e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.970+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa59c8f10 con 0x558aa59ca0d0 2024-05-08T18:05:26.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.970+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f04f8036040 con 0x558aa59ca0d0 2024-05-08T18:05:26.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.970+0000 7f051013e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.970+0000 7f051013e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.970+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa59ceff0 con 0x558aa59ca0d0 2024-05-08T18:05:26.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.970+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f04f8036040 con 0x558aa59ca0d0 2024-05-08T18:05:26.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.970+0000 7f051013e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:26.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.970+0000 7f051013e800 20 started sync module instance, tier type = 2024-05-08T18:05:26.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.970+0000 7f051013e800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.970+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59ce400 con 0x7f04f80393c0 2024-05-08T18:05:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.971+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f050800a4a0 con 0x7f04f80393c0 2024-05-08T18:05:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.971+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94053972371744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59ce400 con 0x7f04f80393c0 2024-05-08T18:05:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.971+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59cfae0 con 0x558aa59c0d30 2024-05-08T18:05:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.972+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94053972371744] v27'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0508040040 con 0x7f04f80393c0 2024-05-08T18:05:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.973+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0508040040 con 0x558aa59c0d30 2024-05-08T18:05:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.973+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94053972374992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59ce400 con 0x558aa59c0d30 2024-05-08T18:05:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.973+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59d0560 con 0x558aa59ca0d0 2024-05-08T18:05:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.974+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94053972374992] v27'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0508040040 con 0x558aa59c0d30 2024-05-08T18:05:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.974+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f04f8036040 con 0x558aa59ca0d0 2024-05-08T18:05:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.974+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94053972369232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59ce370 con 0x558aa59ca0d0 2024-05-08T18:05:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.974+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59d1450 con 0x7f04f80393c0 2024-05-08T18:05:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.975+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94053972369232] v27'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f04f8036040 con 0x558aa59ca0d0 2024-05-08T18:05:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.975+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0508040040 con 0x7f04f80393c0 2024-05-08T18:05:26.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.975+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94053972381600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59d13c0 con 0x7f04f80393c0 2024-05-08T18:05:26.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.975+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59d21c0 con 0x558aa59c0d30 2024-05-08T18:05:26.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.976+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94053972381600] v27'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0508040040 con 0x7f04f80393c0 2024-05-08T18:05:26.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.976+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0508040040 con 0x558aa59c0d30 2024-05-08T18:05:26.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.976+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94053972383664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59d21c0 con 0x558aa59c0d30 2024-05-08T18:05:26.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.976+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59d2bd0 con 0x558aa59c5720 2024-05-08T18:05:26.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.977+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94053972383664] v27'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0508040040 con 0x558aa59c0d30 2024-05-08T18:05:26.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.977+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.977+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94053972387616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59d2fc0 con 0x558aa59c5720 2024-05-08T18:05:26.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.977+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59d3980 con 0x558aa59c5720 2024-05-08T18:05:26.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.978+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94053972387616] v27'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.978+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.978+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94053972384720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59d4220 con 0x558aa59c5720 2024-05-08T18:05:26.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.978+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59d4ab0 con 0x558aa59c5720 2024-05-08T18:05:26.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.979+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94053972384720] v27'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.979+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.979+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94053972395456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59d4ea0 con 0x558aa59c5720 2024-05-08T18:05:26.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.979+0000 7f051013e800 20 add_watcher() i=0 2024-05-08T18:05:26.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.979+0000 7f051013e800 20 add_watcher() i=1 2024-05-08T18:05:26.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.979+0000 7f051013e800 20 add_watcher() i=2 2024-05-08T18:05:26.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.979+0000 7f051013e800 20 add_watcher() i=3 2024-05-08T18:05:26.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.979+0000 7f051013e800 20 add_watcher() i=4 2024-05-08T18:05:26.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.979+0000 7f051013e800 20 add_watcher() i=5 2024-05-08T18:05:26.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.979+0000 7f051013e800 20 add_watcher() i=6 2024-05-08T18:05:26.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.980+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94053972395456] v27'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.980+0000 7f051013e800 20 add_watcher() i=7 2024-05-08T18:05:26.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.980+0000 7f051013e800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:26.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.980+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa59d82a0 con 0x558aa59c0d30 2024-05-08T18:05:26.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.980+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f0508040040 con 0x558aa59c0d30 2024-05-08T18:05:26.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.980+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94053972393248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa59d5760 con 0x558aa59c0d30 2024-05-08T18:05:26.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.981+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94053972393248] v27'180 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0508040040 con 0x558aa59c0d30 2024-05-08T18:05:26.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.982+0000 7f03bf7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:26.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.982+0000 7f051013e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:26.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.982+0000 7f051013e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:26.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.982+0000 7f051013e800 5 note: GC not initialized 2024-05-08T18:05:26.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.982+0000 7f03be7fc640 20 reqs_thread_entry: start 2024-05-08T18:05:26.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.982+0000 7f051013e800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:26.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.982+0000 7f051013e800 20 Filter name: none 2024-05-08T18:05:26.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.982+0000 7f0333fff640 20 reqs_thread_entry: start 2024-05-08T18:05:26.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.982+0000 7f051013e800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-05-08T18:05:26.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.982+0000 7f051013e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.982+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa5a0be40 con 0x7f04f80393c0 2024-05-08T18:05:26.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv35 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f0508040040 con 0x7f04f80393c0 2024-05-08T18:05:26.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f051013e800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:26.991 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f051013e800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-05-08T18:05:26.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f051013e800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:26.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f051013e800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 : miss 2024-05-08T18:05:26.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f051013e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:26.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.7 8:e716d2f9:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa5a0d8d0 con 0x558aa59c5720 2024-05-08T18:05:26.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv2 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f051013e800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:26.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f051013e800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 info.flags=0x17 2024-05-08T18:05:26.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f051013e800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 to cache LRU end 2024-05-08T18:05:26.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f051013e800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:26.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f051013e800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:26.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f051013e800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 2024-05-08T18:05:26.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f051013e800 20 get_obj_state: octx=0x7f0500058430 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x558aa5a0ff98 s->prefetch_data=0 2024-05-08T18:05:26.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 3.32s0 3:4d0cffb4:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x558aa5a10dc0 con 0x558aa59ca0d0 2024-05-08T18:05:26.992 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=910b,stat out=16b] v0'0 uv3 ondisk = 0) ==== 262+0+926 (crc 0 0 0) 0x7f04f8036040 con 0x558aa59ca0d0 2024-05-08T18:05:26.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f051013e800 10 manifest: total_size = 5 2024-05-08T18:05:26.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.983+0000 7f051013e800 20 get_obj_state: setting s->obj_tag to _b0UVA55X0eEjPlUkdwo15sz8j2DbmBH 2024-05-08T18:05:26.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.984+0000 7f03bf7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:26.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.984+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94053972393248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa5a09790 con 0x558aa59c0d30 2024-05-08T18:05:26.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.985+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94053972393248] v27'181 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0508040040 con 0x558aa59c0d30 2024-05-08T18:05:26.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.985+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94053972371744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa5a09790 con 0x7f04f80393c0 2024-05-08T18:05:26.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.986+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94053972371744] v27'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0508040040 con 0x7f04f80393c0 2024-05-08T18:05:26.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.986+0000 7f051013e800 20 remove_watcher() i=0 2024-05-08T18:05:26.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.986+0000 7f051013e800 2 removed watcher, disabling cache 2024-05-08T18:05:26.993 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.986+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94053972374992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa5a09790 con 0x558aa59c0d30 2024-05-08T18:05:26.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.987+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94053972374992] v27'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0508040040 con 0x558aa59c0d30 2024-05-08T18:05:26.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.987+0000 7f051013e800 20 remove_watcher() i=1 2024-05-08T18:05:26.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.987+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94053972369232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa5a09790 con 0x558aa59ca0d0 2024-05-08T18:05:26.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.988+0000 7f050d435640 1 -- 172.21.15.50:0/1699569594 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94053972369232] v27'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f04f8036040 con 0x558aa59ca0d0 2024-05-08T18:05:26.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.988+0000 7f051013e800 20 remove_watcher() i=2 2024-05-08T18:05:26.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.988+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94053972381600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa5a09790 con 0x7f04f80393c0 2024-05-08T18:05:26.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.989+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94053972381600] v27'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0508040040 con 0x7f04f80393c0 2024-05-08T18:05:26.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.989+0000 7f051013e800 20 remove_watcher() i=3 2024-05-08T18:05:26.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.989+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94053972383664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa5a09790 con 0x558aa59c0d30 2024-05-08T18:05:26.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.990+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94053972383664] v27'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0508040040 con 0x558aa59c0d30 2024-05-08T18:05:26.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.990+0000 7f051013e800 20 remove_watcher() i=4 2024-05-08T18:05:26.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.990+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94053972387616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa5a09790 con 0x558aa59c5720 2024-05-08T18:05:26.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.991+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94053972387616] v27'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.991+0000 7f051013e800 20 remove_watcher() i=5 2024-05-08T18:05:26.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.991+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94053972384720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa5a09790 con 0x558aa59c5720 2024-05-08T18:05:26.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.992+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94053972384720] v27'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.992+0000 7f051013e800 20 remove_watcher() i=6 2024-05-08T18:05:26.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.992+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94053972395456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x558aa5a09790 con 0x558aa59c5720 2024-05-08T18:05:26.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.993+0000 7f050dc36640 1 -- 172.21.15.50:0/1699569594 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94053972395456] v27'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0500075030 con 0x558aa59c5720 2024-05-08T18:05:26.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.993+0000 7f051013e800 20 remove_watcher() i=7 2024-05-08T18:05:26.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.993+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558aa59c5720 msgr2=0x558aa59c7b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.993+0000 7f051013e800 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558aa59c5720 0x558aa59c7b10 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.993+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558aa59c0d30 msgr2=0x558aa59c31f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.993+0000 7f051013e800 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558aa59c0d30 0x558aa59c31f0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.993+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f04f80393c0 msgr2=0x7f04f80397c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f04f80393c0 0x7f04f80397c0 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558aa59ca0d0 msgr2=0x558aa59cc590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558aa59ca0d0 0x558aa59cc590 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:26.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f03d803bb50 msgr2=0x7f03d803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f03d803bb50 0x7f03d803e010 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x558aa5993170 tx=0x7f04f8011000 comp rx=0 tx=0).stop 2024-05-08T18:05:26.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59ae1d0 msgr2=0x558aa59b55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:26.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 --2- 172.21.15.50:0/1699569594 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59ae1d0 0x558aa59b55f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f050008a420 tx=0x7f050004cf10 comp rx=0 tx=0).stop 2024-05-08T18:05:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 reap_dead start 2024-05-08T18:05:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f050e437640 1 -- 172.21.15.50:0/1699569594 reap_dead start 2024-05-08T18:05:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 shutdown_connections 2024-05-08T18:05:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 >> 172.21.15.50:0/1699569594 conn(0x558aa59aa530 msgr2=0x558aa59a7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 shutdown_connections 2024-05-08T18:05:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 -- 172.21.15.50:0/1699569594 wait complete. 2024-05-08T18:05:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 -- 172.21.15.50:0/1867607914 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f03e803bba0 msgr2=0x7f03e803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 --2- 172.21.15.50:0/1867607914 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f03e803bba0 0x7f03e803e060 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x558aa5987060 tx=0x7f04f8006000 comp rx=0 tx=0).stop 2024-05-08T18:05:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 -- 172.21.15.50:0/1867607914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59a10e0 msgr2=0x558aa59a1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 --2- 172.21.15.50:0/1867607914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59a10e0 0x558aa59a1500 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f050005c720 tx=0x7f0500006670 comp rx=0 tx=0).stop 2024-05-08T18:05:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 -- 172.21.15.50:0/1867607914 shutdown_connections 2024-05-08T18:05:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 --2- 172.21.15.50:0/1867607914 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f03e803bba0 0x7f03e803e060 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 --2- 172.21.15.50:0/1867607914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa59a10e0 0x558aa59a1500 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 -- 172.21.15.50:0/1867607914 >> 172.21.15.50:0/1867607914 conn(0x558aa5995180 msgr2=0x558aa5995580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 -- 172.21.15.50:0/1867607914 shutdown_connections 2024-05-08T18:05:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.994+0000 7f051013e800 1 -- 172.21.15.50:0/1867607914 wait complete. 2024-05-08T18:05:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.995+0000 7f051013e800 1 -- 172.21.15.50:0/1951298305 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558aa598c210 msgr2=0x558aa598e6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.995+0000 7f051013e800 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558aa598c210 0x558aa598e6a0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.995+0000 7f051013e800 1 -- 172.21.15.50:0/1951298305 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558aa5888580 msgr2=0x558aa5989a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.995+0000 7f051013e800 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558aa5888580 0x558aa5989a50 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.995+0000 7f051013e800 1 -- 172.21.15.50:0/1951298305 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558aa5990290 msgr2=0x558aa5992680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.995+0000 7f051013e800 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558aa5990290 0x558aa5992680 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.995+0000 7f051013e800 1 -- 172.21.15.50:0/1951298305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f03f403bb50 msgr2=0x7f03f403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.995+0000 7f051013e800 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f03f403bb50 0x7f03f403e010 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x558aa57eacb0 tx=0x7f04f80013d0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.995+0000 7f051013e800 1 -- 172.21.15.50:0/1951298305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 msgr2=0x558aa5982810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.995+0000 7f051013e800 1 --2- 172.21.15.50:0/1951298305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558aa5887c20 0x558aa5982810 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f05000524f0 tx=0x7f0500003020 comp rx=0 tx=0).stop 2024-05-08T18:05:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.995+0000 7f050e437640 1 -- 172.21.15.50:0/1951298305 reap_dead start 2024-05-08T18:05:27.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.995+0000 7f051013e800 1 -- 172.21.15.50:0/1951298305 shutdown_connections 2024-05-08T18:05:27.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.995+0000 7f051013e800 1 -- 172.21.15.50:0/1951298305 >> 172.21.15.50:0/1951298305 conn(0x558aa58812f0 msgr2=0x558aa5974ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.995+0000 7f051013e800 1 -- 172.21.15.50:0/1951298305 shutdown_connections 2024-05-08T18:05:27.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:26.995+0000 7f051013e800 1 -- 172.21.15.50:0/1951298305 wait complete. 2024-05-08T18:05:27.003 INFO:teuthology.orchestra.run.smithi050.stdout:fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2024-05-08T18:05:27.003 DEBUG:tasks.util.rgw: raw result: fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2024-05-08T18:05:27.005 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=609 failed=False 2024-05-08T18:05:27.018 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:27.075 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2024-05-08T18:05:27.075 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2024-05-08T18:05:27.075 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-05-08T18:05:27.075 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2024-05-08T18:05:27.076 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2024-05-08T18:05:27.076 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2024-05-08T18:05:27.076 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket rm --bucket myfoo --purge-objects 2024-05-08T18:05:27.122 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:27.122 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:27.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.125+0000 7f8264887800 1 Processor -- start 2024-05-08T18:05:27.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.125+0000 7f8264887800 1 -- start start 2024-05-08T18:05:27.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.125+0000 7f8264887800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5593d9235710 0x5593d9235b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.125+0000 7f8264887800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5593d92361b0 con 0x5593d92368d0 2024-05-08T18:05:27.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.125+0000 7f8264887800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5593d90d5e20 con 0x5593d9235710 2024-05-08T18:05:27.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.125+0000 7f826237f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5593d9235710 0x5593d9235b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.125+0000 7f8261b7e640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5593d92368d0 0x5593d9236d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:60396/0 (socket says 172.21.15.50:60396) 2024-05-08T18:05:27.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.125+0000 7f8261b7e640 1 -- 172.21.15.50:0/2420831523 learned_addr learned my addr 172.21.15.50:0/2420831523 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:27.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.125+0000 7f826237f640 1 -- 172.21.15.50:0/2420831523 >> v1:172.21.15.46:6789/0 conn(0x5593d92368d0 legacy=0x5593d9236d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.125+0000 7f826237f640 1 -- 172.21.15.50:0/2420831523 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593d9237490 con 0x5593d9235710 2024-05-08T18:05:27.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.127+0000 7f826237f640 1 --2- 172.21.15.50:0/2420831523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 0x5593d9235b30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f825400a470 tx=0x7f825401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9620582bf2b72cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.127+0000 7f826137d640 1 -- 172.21.15.50:0/2420831523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8254057070 con 0x5593d9235710 2024-05-08T18:05:27.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.128+0000 7f826137d640 1 -- 172.21.15.50:0/2420831523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825404adc0 con 0x5593d9235710 2024-05-08T18:05:27.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.128+0000 7f826137d640 1 -- 172.21.15.50:0/2420831523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8254052590 con 0x5593d9235710 2024-05-08T18:05:27.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.128+0000 7f8264887800 1 -- 172.21.15.50:0/2420831523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 msgr2=0x5593d9235b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.128+0000 7f8264887800 1 --2- 172.21.15.50:0/2420831523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 0x5593d9235b30 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f825400a470 tx=0x7f825401ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:27.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.128+0000 7f8264887800 1 -- 172.21.15.50:0/2420831523 shutdown_connections 2024-05-08T18:05:27.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.128+0000 7f8264887800 1 --2- 172.21.15.50:0/2420831523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 0x5593d9235b30 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.128+0000 7f8264887800 1 -- 172.21.15.50:0/2420831523 >> 172.21.15.50:0/2420831523 conn(0x5593d9230c90 msgr2=0x5593d92330d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.128+0000 7f8264887800 1 -- 172.21.15.50:0/2420831523 shutdown_connections 2024-05-08T18:05:27.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.128+0000 7f8264887800 1 -- 172.21.15.50:0/2420831523 wait complete. 2024-05-08T18:05:27.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.143+0000 7f8264887800 1 Processor -- start 2024-05-08T18:05:27.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.143+0000 7f8264887800 1 -- start start 2024-05-08T18:05:27.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.143+0000 7f8264887800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 0x5593d9326c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.143+0000 7f8264887800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5593d9180a30 con 0x5593d9235710 2024-05-08T18:05:27.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.144+0000 7f826237f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 0x5593d9326c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.144+0000 7f826237f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 0x5593d9326c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43950/0 (socket says 172.21.15.50:43950) 2024-05-08T18:05:27.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.144+0000 7f826237f640 1 -- 172.21.15.50:0/1394773503 learned_addr learned my addr 172.21.15.50:0/1394773503 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.144+0000 7f826237f640 1 -- 172.21.15.50:0/1394773503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593d9327240 con 0x5593d9235710 2024-05-08T18:05:27.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.144+0000 7f826237f640 1 --2- 172.21.15.50:0/1394773503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 0x5593d9326c70 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8254055040 tx=0x7f825404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13185bb27bf72e56 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.145+0000 7f8150608640 1 -- 172.21.15.50:0/1394773503 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8254057040 con 0x5593d9235710 2024-05-08T18:05:27.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.145+0000 7f8150608640 1 -- 172.21.15.50:0/1394773503 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8254062070 con 0x5593d9235710 2024-05-08T18:05:27.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.145+0000 7f8150608640 1 -- 172.21.15.50:0/1394773503 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f825405d470 con 0x5593d9235710 2024-05-08T18:05:27.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.145+0000 7f8264887800 1 -- 172.21.15.50:0/1394773503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 msgr2=0x5593d9326c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.145+0000 7f8264887800 1 --2- 172.21.15.50:0/1394773503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 0x5593d9326c70 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8254055040 tx=0x7f825404c000 comp rx=0 tx=0).stop 2024-05-08T18:05:27.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.145+0000 7f8264887800 1 -- 172.21.15.50:0/1394773503 shutdown_connections 2024-05-08T18:05:27.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.145+0000 7f8264887800 1 --2- 172.21.15.50:0/1394773503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 0x5593d9326c70 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.145+0000 7f8264887800 1 -- 172.21.15.50:0/1394773503 >> 172.21.15.50:0/1394773503 conn(0x5593d9230c90 msgr2=0x5593d9322590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.145+0000 7f8264887800 1 -- 172.21.15.50:0/1394773503 shutdown_connections 2024-05-08T18:05:27.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.145+0000 7f8264887800 1 -- 172.21.15.50:0/1394773503 wait complete. 2024-05-08T18:05:27.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.146+0000 7f8264887800 1 Processor -- start 2024-05-08T18:05:27.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.146+0000 7f8264887800 1 -- start start 2024-05-08T18:05:27.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.146+0000 7f8264887800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 0x5593d91aae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.146+0000 7f8264887800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5593d923eda0 con 0x5593d9235710 2024-05-08T18:05:27.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.146+0000 7f826237f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 0x5593d91aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.146+0000 7f826237f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 0x5593d91aae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43960/0 (socket says 172.21.15.50:43960) 2024-05-08T18:05:27.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.146+0000 7f826237f640 1 -- 172.21.15.50:0/1839521994 learned_addr learned my addr 172.21.15.50:0/1839521994 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.146+0000 7f826237f640 1 -- 172.21.15.50:0/1839521994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593d91ab380 con 0x5593d9235710 2024-05-08T18:05:27.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.147+0000 7f826237f640 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 0x5593d91aae40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f82540057d0 tx=0x7f8254002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.147+0000 7f814ee05640 1 -- 172.21.15.50:0/1839521994 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f825406b070 con 0x5593d9235710 2024-05-08T18:05:27.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.147+0000 7f814ee05640 1 -- 172.21.15.50:0/1839521994 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825405d9a0 con 0x5593d9235710 2024-05-08T18:05:27.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.147+0000 7f8264887800 1 -- 172.21.15.50:0/1839521994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5593d91a95d0 con 0x5593d9235710 2024-05-08T18:05:27.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.147+0000 7f814ee05640 1 -- 172.21.15.50:0/1839521994 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8254057070 con 0x5593d9235710 2024-05-08T18:05:27.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.147+0000 7f8264887800 1 -- 172.21.15.50:0/1839521994 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593d91a9b50 con 0x5593d9235710 2024-05-08T18:05:27.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.148+0000 7f814ee05640 1 -- 172.21.15.50:0/1839521994 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f825405dc80 con 0x5593d9235710 2024-05-08T18:05:27.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.148+0000 7f814ee05640 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f814803bba0 0x7f814803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.148+0000 7f814ee05640 1 -- 172.21.15.50:0/1839521994 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f8254094560 con 0x5593d9235710 2024-05-08T18:05:27.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.148+0000 7f8261b7e640 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f814803bba0 0x7f814803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.148+0000 7f8264887800 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5593d92368d0 0x5593d9337af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.148+0000 7f8264887800 1 -- 172.21.15.50:0/1839521994 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d9338030 con 0x5593d92368d0 2024-05-08T18:05:27.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.148+0000 7f8262b80640 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5593d92368d0 0x5593d9337af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.149+0000 7f8261b7e640 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f814803bba0 0x7f814803e060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8258004170 tx=0x7f8258001330 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.149+0000 7f8262b80640 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5593d92368d0 0x5593d9337af0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.149+0000 7f8262b80640 1 -- 172.21.15.50:0/1839521994 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d92368d0 2024-05-08T18:05:27.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.149+0000 7f8264887800 1 -- 172.21.15.50:0/1839521994 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d91a9ff0 con 0x5593d92368d0 2024-05-08T18:05:27.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.149+0000 7f8262b80640 1 -- 172.21.15.50:0/1839521994 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d92368d0 2024-05-08T18:05:27.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.149+0000 7f8264887800 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5593d933a300 0x5593d933c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.149+0000 7f8264887800 1 -- 172.21.15.50:0/1839521994 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d933cd50 con 0x5593d933a300 2024-05-08T18:05:27.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.150+0000 7f826237f640 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5593d933a300 0x5593d933c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.150+0000 7f826237f640 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5593d933a300 0x5593d933c6f0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.150+0000 7f826237f640 1 -- 172.21.15.50:0/1839521994 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f825405a040 con 0x5593d933a300 2024-05-08T18:05:27.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.150+0000 7f8264887800 1 -- 172.21.15.50:0/1839521994 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93391a0 con 0x5593d92368d0 2024-05-08T18:05:27.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.150+0000 7f8262b80640 1 -- 172.21.15.50:0/1839521994 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d92368d0 2024-05-08T18:05:27.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.150+0000 7f8264887800 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5593d933e1b0 0x5593d93405a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.150+0000 7f8264887800 1 -- 172.21.15.50:0/1839521994 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d9340ca0 con 0x5593d933e1b0 2024-05-08T18:05:27.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.151+0000 7f8261b7e640 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5593d933e1b0 0x5593d93405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.151+0000 7f8261b7e640 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5593d933e1b0 0x5593d93405a0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.151+0000 7f8261b7e640 1 -- 172.21.15.50:0/1839521994 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f8258009050 con 0x5593d933e1b0 2024-05-08T18:05:27.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.152+0000 7f8264887800 1 Processor -- start 2024-05-08T18:05:27.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.152+0000 7f8264887800 1 -- start start 2024-05-08T18:05:27.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.152+0000 7f8264887800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d934f010 0x5593d934f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.152+0000 7f8264887800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5593d9127330 con 0x5593d934f010 2024-05-08T18:05:27.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.152+0000 7f826237f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d934f010 0x5593d934f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.152+0000 7f826237f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d934f010 0x5593d934f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43974/0 (socket says 172.21.15.50:43974) 2024-05-08T18:05:27.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.152+0000 7f826237f640 1 -- 172.21.15.50:0/3590555564 learned_addr learned my addr 172.21.15.50:0/3590555564 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.152+0000 7f826237f640 1 -- 172.21.15.50:0/3590555564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593d934f970 con 0x5593d934f010 2024-05-08T18:05:27.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.152+0000 7f826237f640 1 --2- 172.21.15.50:0/3590555564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d934f010 0x5593d934f430 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8254092a10 tx=0x7f825405b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.153+0000 7f8147fff640 1 -- 172.21.15.50:0/3590555564 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8254078700 con 0x5593d934f010 2024-05-08T18:05:27.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.153+0000 7f8147fff640 1 -- 172.21.15.50:0/3590555564 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82540788a0 con 0x5593d934f010 2024-05-08T18:05:27.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.153+0000 7f8264887800 1 -- 172.21.15.50:0/3590555564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5593d93503e0 con 0x5593d934f010 2024-05-08T18:05:27.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.153+0000 7f8147fff640 1 -- 172.21.15.50:0/3590555564 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8254009aa0 con 0x5593d934f010 2024-05-08T18:05:27.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.153+0000 7f8264887800 1 -- 172.21.15.50:0/3590555564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593d93508e0 con 0x5593d934f010 2024-05-08T18:05:27.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.153+0000 7f8147fff640 1 -- 172.21.15.50:0/3590555564 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f8254009c40 con 0x5593d934f010 2024-05-08T18:05:27.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.153+0000 7f8147fff640 1 --2- 172.21.15.50:0/3590555564 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f813c03bba0 0x7f813c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.154+0000 7f8147fff640 1 -- 172.21.15.50:0/3590555564 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f8254082c80 con 0x5593d934f010 2024-05-08T18:05:27.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.154+0000 7f8261b7e640 1 --2- 172.21.15.50:0/3590555564 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f813c03bba0 0x7f813c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.154+0000 7f8264887800 1 Processor -- start 2024-05-08T18:05:27.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.154+0000 7f8264887800 1 -- start start 2024-05-08T18:05:27.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.154+0000 7f8261b7e640 1 --2- 172.21.15.50:0/3590555564 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f813c03bba0 0x7f813c03e060 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x5593d91a9f70 tx=0x7f825800f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.154+0000 7f8264887800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d935c1c0 0x5593d935c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.154+0000 7f8264887800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5593d9327500 con 0x5593d935c1c0 2024-05-08T18:05:27.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.154+0000 7f826237f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d935c1c0 0x5593d935c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.154+0000 7f826237f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d935c1c0 0x5593d935c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43990/0 (socket says 172.21.15.50:43990) 2024-05-08T18:05:27.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.154+0000 7f826237f640 1 -- 172.21.15.50:0/3234436775 learned_addr learned my addr 172.21.15.50:0/3234436775 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.155+0000 7f826237f640 1 -- 172.21.15.50:0/3234436775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593d935cbb0 con 0x5593d935c1c0 2024-05-08T18:05:27.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.155+0000 7f826237f640 1 --2- 172.21.15.50:0/3234436775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d935c1c0 0x5593d935c5e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8254001eb0 tx=0x7f82540099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae20153940250d46 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.155+0000 7f8145ffb640 1 -- 172.21.15.50:0/3234436775 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8254092bc0 con 0x5593d935c1c0 2024-05-08T18:05:27.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.155+0000 7f8145ffb640 1 -- 172.21.15.50:0/3234436775 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825405c750 con 0x5593d935c1c0 2024-05-08T18:05:27.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.155+0000 7f8145ffb640 1 -- 172.21.15.50:0/3234436775 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f825405c3d0 con 0x5593d935c1c0 2024-05-08T18:05:27.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.155+0000 7f8264887800 1 -- 172.21.15.50:0/3234436775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d935c1c0 msgr2=0x5593d935c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.155+0000 7f8264887800 1 --2- 172.21.15.50:0/3234436775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d935c1c0 0x5593d935c5e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8254001eb0 tx=0x7f82540099f0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.155+0000 7f8264887800 1 -- 172.21.15.50:0/3234436775 shutdown_connections 2024-05-08T18:05:27.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.155+0000 7f8264887800 1 --2- 172.21.15.50:0/3234436775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d935c1c0 0x5593d935c5e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.155+0000 7f8264887800 1 -- 172.21.15.50:0/3234436775 >> 172.21.15.50:0/3234436775 conn(0x5593d93584d0 msgr2=0x5593d9356530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.155+0000 7f8264887800 1 -- 172.21.15.50:0/3234436775 shutdown_connections 2024-05-08T18:05:27.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.155+0000 7f8264887800 1 -- 172.21.15.50:0/3234436775 wait complete. 2024-05-08T18:05:27.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.156+0000 7f8264887800 1 Processor -- start 2024-05-08T18:05:27.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.156+0000 7f8264887800 1 -- start start 2024-05-08T18:05:27.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.156+0000 7f8264887800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d935c1c0 0x5593d9363580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.156+0000 7f8264887800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5593d935d240 con 0x5593d935c1c0 2024-05-08T18:05:27.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.156+0000 7f826237f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d935c1c0 0x5593d9363580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.156+0000 7f826237f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d935c1c0 0x5593d9363580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:43992/0 (socket says 172.21.15.50:43992) 2024-05-08T18:05:27.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.156+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 learned_addr learned my addr 172.21.15.50:0/1307305089 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.156+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593d9363ac0 con 0x5593d935c1c0 2024-05-08T18:05:27.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.157+0000 7f826237f640 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d935c1c0 0x5593d9363580 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f825400a290 tx=0x7f825406df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.157+0000 7f8133fff640 1 -- 172.21.15.50:0/1307305089 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82540ae070 con 0x5593d935c1c0 2024-05-08T18:05:27.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.157+0000 7f8133fff640 1 -- 172.21.15.50:0/1307305089 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825408e890 con 0x5593d935c1c0 2024-05-08T18:05:27.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.157+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5593d9363ac0 con 0x5593d935c1c0 2024-05-08T18:05:27.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.157+0000 7f8133fff640 1 -- 172.21.15.50:0/1307305089 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f825408ebf0 con 0x5593d935c1c0 2024-05-08T18:05:27.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.157+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593d9363ac0 con 0x5593d935c1c0 2024-05-08T18:05:27.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.158+0000 7f8133fff640 1 -- 172.21.15.50:0/1307305089 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f82540b1070 con 0x5593d935c1c0 2024-05-08T18:05:27.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.158+0000 7f8133fff640 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f812c03bab0 0x7f812c03df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.158+0000 7f8133fff640 1 -- 172.21.15.50:0/1307305089 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f825409b070 con 0x5593d935c1c0 2024-05-08T18:05:27.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.158+0000 7f8261b7e640 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f812c03bab0 0x7f812c03df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.158+0000 7f8261b7e640 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f812c03bab0 0x7f812c03df70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5593d93410a0 tx=0x7f8258010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.159+0000 7f8264887800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.159+0000 7f8264887800 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5593d936ed30 0x5593d93711f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.159+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d9371940 con 0x5593d936ed30 2024-05-08T18:05:27.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.159+0000 7f8262b80640 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5593d936ed30 0x5593d93711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.159+0000 7f8262b80640 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5593d936ed30 0x5593d93711f0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.159+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d936ed30 2024-05-08T18:05:27.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.159+0000 7f8264887800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.159+0000 7f8264887800 20 realm 2024-05-08T18:05:27.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.159+0000 7f8264887800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.159+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d9371d50 con 0x5593d936ed30 2024-05-08T18:05:27.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.159+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d936ed30 2024-05-08T18:05:27.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.160+0000 7f8264887800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.160+0000 7f8264887800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:27.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.160+0000 7f8264887800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.160+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d9372620 con 0x5593d936ed30 2024-05-08T18:05:27.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.160+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d936ed30 2024-05-08T18:05:27.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.160+0000 7f8264887800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.160+0000 7f8264887800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.160+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d936ed30 2024-05-08T18:05:27.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.160+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d936ed30 2024-05-08T18:05:27.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.160+0000 7f8264887800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:27.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.160+0000 7f8264887800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.160+0000 7f8264887800 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5593d9373720 0x5593d9375b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.160+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d9376120 con 0x5593d9373720 2024-05-08T18:05:27.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.160+0000 7f826237f640 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5593d9373720 0x5593d9375b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.161+0000 7f826237f640 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5593d9373720 0x5593d9375b10 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.161+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.161+0000 7f8264887800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:27.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.161+0000 7f8264887800 20 searching for the correct realm 2024-05-08T18:05:27.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.161+0000 7f8264887800 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5593d93780d0 0x5593d937a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.161+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d937ac60 con 0x5593d93780d0 2024-05-08T18:05:27.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.161+0000 7f8261b7e640 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5593d93780d0 0x5593d937a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.162+0000 7f8261b7e640 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5593d93780d0 0x5593d937a590 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.162+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.162+0000 7f8261b7e640 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f8258037040 0x7f8258036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.162+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d937ac60 con 0x7f8258037040 2024-05-08T18:05:27.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.162+0000 7f8262b80640 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f8258037040 0x7f8258036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.162+0000 7f8262b80640 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f8258037040 0x7f8258036410 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.163+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f825c00a4a0 con 0x7f8258037040 2024-05-08T18:05:27.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.163+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d9373720 2024-05-08T18:05:27.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.163+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.163+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d936ed30 2024-05-08T18:05:27.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.163+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d936ed30 2024-05-08T18:05:27.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.163+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d936ed30 2024-05-08T18:05:27.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.163+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d936ed30 2024-05-08T18:05:27.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.163+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d9373720 2024-05-08T18:05:27.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.164+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.164+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d9373720 2024-05-08T18:05:27.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.164+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.164+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x7f8258037040 2024-05-08T18:05:27.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.164+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f825c00a4a0 con 0x7f8258037040 2024-05-08T18:05:27.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.164+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d93780d0 2024-05-08T18:05:27.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.164+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.164+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d936ed30 2024-05-08T18:05:27.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.164+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d936ed30 2024-05-08T18:05:27.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.165+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d9373720 2024-05-08T18:05:27.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.165+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.165+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d936ed30 2024-05-08T18:05:27.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.165+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d936ed30 2024-05-08T18:05:27.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.165+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x7f8258037040 2024-05-08T18:05:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.165+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f825c00a4a0 con 0x7f8258037040 2024-05-08T18:05:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.165+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d93780d0 2024-05-08T18:05:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.166+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.166+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d93780d0 2024-05-08T18:05:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.166+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.166+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d9373720 2024-05-08T18:05:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.166+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.166+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d93780d0 2024-05-08T18:05:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.166+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.166+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d93780d0 2024-05-08T18:05:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.167+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.167+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d936ed30 2024-05-08T18:05:27.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.167+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d936ed30 2024-05-08T18:05:27.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.167+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d936ed30 2024-05-08T18:05:27.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.167+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d936ed30 2024-05-08T18:05:27.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.167+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d9373720 2024-05-08T18:05:27.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.167+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.167+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d93780d0 2024-05-08T18:05:27.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.168+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.168+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x7f8258037040 2024-05-08T18:05:27.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.168+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f825c00a4a0 con 0x7f8258037040 2024-05-08T18:05:27.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.168+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d936ed30 2024-05-08T18:05:27.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.168+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d936ed30 2024-05-08T18:05:27.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.168+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d9373720 2024-05-08T18:05:27.182 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.168+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.168+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d93780d0 2024-05-08T18:05:27.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.169+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.169+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d9373720 2024-05-08T18:05:27.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.169+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.169+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x7f8258037040 2024-05-08T18:05:27.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.169+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f825c00a4a0 con 0x7f8258037040 2024-05-08T18:05:27.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.169+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d93780d0 2024-05-08T18:05:27.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.169+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.169+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d9373720 2024-05-08T18:05:27.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x5593d936ed30 2024-05-08T18:05:27.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d936ed30 2024-05-08T18:05:27.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5593d93729c0 con 0x7f8258037040 2024-05-08T18:05:27.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f825c00a4a0 con 0x7f8258037040 2024-05-08T18:05:27.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f8264887800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:27.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f8264887800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:27.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f8264887800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:27.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f8264887800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:27.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f8264887800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:27.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f8264887800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:27.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f8264887800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93769e0 con 0x5593d936ed30 2024-05-08T18:05:27.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d936ed30 2024-05-08T18:05:27.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f8264887800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f8264887800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.170+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93769e0 con 0x5593d936ed30 2024-05-08T18:05:27.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.171+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f825c00a4a0 con 0x5593d936ed30 2024-05-08T18:05:27.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.171+0000 7f8264887800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:27.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.171+0000 7f8264887800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.171+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93769e0 con 0x5593d93780d0 2024-05-08T18:05:27.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.171+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.171+0000 7f8264887800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:27.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.171+0000 7f8264887800 20 zone default found 2024-05-08T18:05:27.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.171+0000 7f8264887800 4 Realm: () 2024-05-08T18:05:27.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.171+0000 7f8264887800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:27.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.171+0000 7f8264887800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:27.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.171+0000 7f8264887800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:27.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.171+0000 7f8264887800 20 zonegroup default 2024-05-08T18:05:27.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.171+0000 7f8264887800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.171+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d9377320 con 0x5593d9373720 2024-05-08T18:05:27.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.171+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.171+0000 7f8264887800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.172+0000 7f8264887800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.172+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d937caf0 con 0x5593d93780d0 2024-05-08T18:05:27.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.172+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.172+0000 7f8264887800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.172+0000 7f8264887800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.172+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d937cfe0 con 0x5593d93780d0 2024-05-08T18:05:27.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.172+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.172+0000 7f8264887800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.172+0000 7f8264887800 20 started sync module instance, tier type = 2024-05-08T18:05:27.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.172+0000 7f8264887800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:27.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.172+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d937bf60 con 0x7f8258037040 2024-05-08T18:05:27.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.173+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f825c00a4a0 con 0x7f8258037040 2024-05-08T18:05:27.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.173+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94093492864096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d937bf60 con 0x7f8258037040 2024-05-08T18:05:27.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.173+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d937dae0 con 0x5593d936ed30 2024-05-08T18:05:27.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.174+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f825c040040 con 0x5593d936ed30 2024-05-08T18:05:27.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.174+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94093492864096] v27'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f825c040040 con 0x7f8258037040 2024-05-08T18:05:27.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.174+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94093492868528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d937dae0 con 0x5593d936ed30 2024-05-08T18:05:27.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.174+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d937e910 con 0x5593d93780d0 2024-05-08T18:05:27.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.175+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.175+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94093492872288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d937f160 con 0x5593d93780d0 2024-05-08T18:05:27.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.175+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d937f9b0 con 0x7f8258037040 2024-05-08T18:05:27.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.175+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94093492868528] v27'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f825c040040 con 0x5593d936ed30 2024-05-08T18:05:27.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.176+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94093492872288] v27'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.176+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f825c040040 con 0x7f8258037040 2024-05-08T18:05:27.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.176+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94093492870736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d937f9b0 con 0x7f8258037040 2024-05-08T18:05:27.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.176+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9380200 con 0x5593d936ed30 2024-05-08T18:05:27.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.177+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f825c040040 con 0x5593d936ed30 2024-05-08T18:05:27.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.177+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94093492878224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93805e0 con 0x5593d936ed30 2024-05-08T18:05:27.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.177+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9381100 con 0x5593d9373720 2024-05-08T18:05:27.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.177+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94093492870736] v27'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f825c040040 con 0x7f8258037040 2024-05-08T18:05:27.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.178+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94093492878224] v27'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f825c040040 con 0x5593d936ed30 2024-05-08T18:05:27.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.178+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.178+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94093492882384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9380030 con 0x5593d9373720 2024-05-08T18:05:27.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.178+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93819a0 con 0x5593d9373720 2024-05-08T18:05:27.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.179+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v27'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.179+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94093492886848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9382610 con 0x5593d9373720 2024-05-08T18:05:27.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.179+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9383030 con 0x5593d9373720 2024-05-08T18:05:27.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.179+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94093492882384] v27'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.180+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94093492886848] v27'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.180+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.180+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94093492890400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93821a0 con 0x5593d9373720 2024-05-08T18:05:27.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.180+0000 7f8264887800 20 add_watcher() i=0 2024-05-08T18:05:27.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.180+0000 7f8264887800 20 add_watcher() i=1 2024-05-08T18:05:27.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.180+0000 7f8264887800 20 add_watcher() i=2 2024-05-08T18:05:27.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.180+0000 7f8264887800 20 add_watcher() i=3 2024-05-08T18:05:27.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.180+0000 7f8264887800 20 add_watcher() i=4 2024-05-08T18:05:27.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.180+0000 7f8264887800 20 add_watcher() i=5 2024-05-08T18:05:27.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.180+0000 7f8264887800 20 add_watcher() i=6 2024-05-08T18:05:27.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.181+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94093492890400] v27'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.181+0000 7f8264887800 20 add_watcher() i=7 2024-05-08T18:05:27.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.181+0000 7f8264887800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:27.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.181+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d9386130 con 0x5593d936ed30 2024-05-08T18:05:27.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.182+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f825c040040 con 0x5593d936ed30 2024-05-08T18:05:27.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.182+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94093492876544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93835b0 con 0x5593d936ed30 2024-05-08T18:05:27.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.183+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94093492876544] v27'182 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f825c040040 con 0x5593d936ed30 2024-05-08T18:05:27.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.183+0000 7f8118fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:27.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.183+0000 7f8264887800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:27.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.183+0000 7f8264887800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:27.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.183+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d938e820 con 0x5593d936ed30 2024-05-08T18:05:27.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.184+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'30 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f825c040040 con 0x5593d936ed30 2024-05-08T18:05:27.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.184+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d938eef0 con 0x5593d9373720 2024-05-08T18:05:27.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.185+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'25 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.185+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d938eef0 con 0x5593d93780d0 2024-05-08T18:05:27.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.186+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'24 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.186+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d938f9b0 con 0x5593d936ed30 2024-05-08T18:05:27.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.187+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'183 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f825c040040 con 0x5593d936ed30 2024-05-08T18:05:27.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.187+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d938c7a0 con 0x7f8258037040 2024-05-08T18:05:27.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.188+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'62 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f825c040040 con 0x7f8258037040 2024-05-08T18:05:27.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.188+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d938c770 con 0x5593d93780d0 2024-05-08T18:05:27.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.189+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'41 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.189+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93906e0 con 0x5593d936ed30 2024-05-08T18:05:27.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.190+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'26 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f825c040040 con 0x5593d936ed30 2024-05-08T18:05:27.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.190+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d938c760 con 0x5593d9373720 2024-05-08T18:05:27.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.191+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'41 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.191+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d938c760 con 0x7f8258037040 2024-05-08T18:05:27.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.192+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'24 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f825c040040 con 0x7f8258037040 2024-05-08T18:05:27.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.192+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d938c760 con 0x5593d93780d0 2024-05-08T18:05:27.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.193+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'30 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.193+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390e00 con 0x7f8258037040 2024-05-08T18:05:27.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.194+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'63 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f825c040040 con 0x7f8258037040 2024-05-08T18:05:27.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.194+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390e00 con 0x5593d9373720 2024-05-08T18:05:27.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.195+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'27 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.195+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390e00 con 0x7f8258037040 2024-05-08T18:05:27.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.196+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'25 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f825c040040 con 0x7f8258037040 2024-05-08T18:05:27.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.196+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390e00 con 0x5593d9373720 2024-05-08T18:05:27.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.197+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'47 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.197+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390e00 con 0x5593d936ed30 2024-05-08T18:05:27.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.198+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'184 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f825c040040 con 0x5593d936ed30 2024-05-08T18:05:27.196 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.198+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390e00 con 0x5593d93780d0 2024-05-08T18:05:27.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.199+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'57 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.197 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.199+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390e90 con 0x7f8258037040 2024-05-08T18:05:27.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.200+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'64 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f825c040040 con 0x7f8258037040 2024-05-08T18:05:27.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.200+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390e90 con 0x5593d93780d0 2024-05-08T18:05:27.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.201+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'24 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.201+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390e90 con 0x5593d936ed30 2024-05-08T18:05:27.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.202+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'32 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f825c040040 con 0x5593d936ed30 2024-05-08T18:05:27.200 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.202+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390f20 con 0x5593d936ed30 2024-05-08T18:05:27.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.203+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'185 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f825c040040 con 0x5593d936ed30 2024-05-08T18:05:27.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.203+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390f20 con 0x5593d93780d0 2024-05-08T18:05:27.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.204+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'58 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.204+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390fb0 con 0x7f8258037040 2024-05-08T18:05:27.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.205+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'34 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f825c040040 con 0x7f8258037040 2024-05-08T18:05:27.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.205+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390fb0 con 0x5593d93780d0 2024-05-08T18:05:27.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.206+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'26 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.206+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390fb0 con 0x5593d93780d0 2024-05-08T18:05:27.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.207+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'30 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.207+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390fb0 con 0x5593d93780d0 2024-05-08T18:05:27.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.208+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'22 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.208+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390fb0 con 0x5593d93780d0 2024-05-08T18:05:27.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.209+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'42 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.207 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.209+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390ff0 con 0x5593d9373720 2024-05-08T18:05:27.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.210+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'26 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.208 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.210+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390fb0 con 0x5593d93780d0 2024-05-08T18:05:27.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.211+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'24 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.209 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.211+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390ff0 con 0x5593d9373720 2024-05-08T18:05:27.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.212+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'42 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.210 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.212+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390fb0 con 0x5593d93780d0 2024-05-08T18:05:27.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.213+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'59 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.211 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.213+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390fb0 con 0x5593d9373720 2024-05-08T18:05:27.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.214+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'48 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.212 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.214+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d9390ff0 con 0x7f8258037040 2024-05-08T18:05:27.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.215+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'30 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f825c040040 con 0x7f8258037040 2024-05-08T18:05:27.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.216+0000 7f80937fe640 20 reqs_thread_entry: start 2024-05-08T18:05:27.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.216+0000 7f8264887800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:27.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.216+0000 7f8264887800 20 Filter name: none 2024-05-08T18:05:27.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.216+0000 7f80917fa640 20 reqs_thread_entry: start 2024-05-08T18:05:27.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.216+0000 7f8264887800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-05-08T18:05:27.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.216+0000 7f8264887800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.216+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:94 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93bc5d0 con 0x7f8258037040 2024-05-08T18:05:27.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.216+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(94 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv35 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f825c040040 con 0x7f8258037040 2024-05-08T18:05:27.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.216+0000 7f8264887800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:05:27.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.216+0000 7f8264887800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-05-08T18:05:27.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.216+0000 7f8264887800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:05:27.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.216+0000 7f8264887800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 : miss 2024-05-08T18:05:27.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.216+0000 7f8264887800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.216+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:95 8.7 8:e716d2f9:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93bdd20 con 0x5593d9373720 2024-05-08T18:05:27.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.217+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(95 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv2 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.217+0000 7f8264887800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:05:27.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.217+0000 7f8264887800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 info.flags=0x17 2024-05-08T18:05:27.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.217+0000 7f8264887800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 to cache LRU end 2024-05-08T18:05:27.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.217+0000 7f8264887800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:27.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.217+0000 7f8264887800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:05:27.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.217+0000 7f8264887800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 2024-05-08T18:05:27.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.217+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:96 4.b 4:d16eb398:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d943b440 con 0x7f8258037040 2024-05-08T18:05:27.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.217+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(96 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f825c040040 con 0x7f8258037040 2024-05-08T18:05:27.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.217+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:97 4.15 4:aa9d0ef9:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d943b480 con 0x5593d93780d0 2024-05-08T18:05:27.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.218+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(97 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.218+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:98 4.26 4:658676bd:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d943b480 con 0x5593d9373720 2024-05-08T18:05:27.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.218+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(98 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.2 [call out=426b] v0'0 uv7 ondisk = 0) ==== 194+0+426 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.218+0000 7f8264887800 10 cls_bucket_list_unordered: got 9c8454ddf7aa50116496bac348d7550d 2024-05-08T18:05:27.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.218+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:99 4.9 4:927242f4:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d943b4c0 con 0x5593d936ed30 2024-05-08T18:05:27.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.219+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(99 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.3 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f825c040040 con 0x5593d936ed30 2024-05-08T18:05:27.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.219+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:100 4.2c 4:341c0785:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d943b4c0 con 0x5593d936ed30 2024-05-08T18:05:27.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.219+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(100 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f825c040040 con 0x5593d936ed30 2024-05-08T18:05:27.220 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.219+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:101 4.3c 4:3da0c2be:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d943b4c0 con 0x5593d936ed30 2024-05-08T18:05:27.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.219+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(101 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.5 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f825c040040 con 0x5593d936ed30 2024-05-08T18:05:27.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.219+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:102 4.3b 4:df5ac0d4:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d943b4c0 con 0x5593d93780d0 2024-05-08T18:05:27.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.220+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 28 ==== osd_op_reply(102 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.6 [call out=426b] v0'0 uv7 ondisk = 0) ==== 194+0+426 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.220+0000 7f8264887800 10 cls_bucket_list_unordered: got b1b9a972ccd8c962a473909b97007eb4 2024-05-08T18:05:27.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.220+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:103 4.1f 4:fb972dd1:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d943b4c0 con 0x5593d93780d0 2024-05-08T18:05:27.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.220+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 29 ==== osd_op_reply(103 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.7 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.220+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:104 4.8 4:11bc5294:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d943b4c0 con 0x5593d93780d0 2024-05-08T18:05:27.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.220+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 30 ==== osd_op_reply(104 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.8 [call out=426b] v0'0 uv3 ondisk = 0) ==== 194+0+426 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.220+0000 7f8264887800 10 cls_bucket_list_unordered: got c785e1ed2950e3e36b1e2ca01f299a54 2024-05-08T18:05:27.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.220+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:105 4.2f 4:f5b60246:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d943b4c0 con 0x5593d936ed30 2024-05-08T18:05:27.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.221+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(105 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f825c040040 con 0x5593d936ed30 2024-05-08T18:05:27.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.221+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:106 4.16 4:6bff304f:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d943b480 con 0x5593d93780d0 2024-05-08T18:05:27.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.221+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 31 ==== osd_op_reply(106 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10 [call out=698b] v0'0 uv11 ondisk = 0) ==== 195+0+698 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.221+0000 7f8264887800 10 cls_bucket_list_unordered: got 24d27c169c2c881eb09a065116f2aa5c 2024-05-08T18:05:27.221 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.221+0000 7f8264887800 10 cls_bucket_list_unordered: got bb3aec0fdcdbc2974890f805c585d432 2024-05-08T18:05:27.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.221+0000 7f8264887800 20 get_obj_state: octx=0x7f82540a2f20 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x5593d943a8d8 s->prefetch_data=0 2024-05-08T18:05:27.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.221+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:107 3.27s0 3:e5a949cc:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_9c8454ddf7aa50116496bac348d7550d:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d9438dd0 con 0x5593d9373720 2024-05-08T18:05:27.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.221+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(107 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_9c8454ddf7aa50116496bac348d7550d [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 262+0+895 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.221+0000 7f8264887800 10 manifest: total_size = 6 2024-05-08T18:05:27.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.221+0000 7f8264887800 20 get_obj_state: setting s->obj_tag to 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.14454311017509819901 2024-05-08T18:05:27.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.221+0000 7f8264887800 20 get_obj_state: octx=0x7f82540a2f20 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x5593d943a8d8 s->prefetch_data=0 2024-05-08T18:05:27.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.221+0000 7f8264887800 20 prepare_atomic_modification: state is not atomic. state=0x5593d943a8d8 2024-05-08T18:05:27.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.222+0000 7f8264887800 20 bucket index object: :.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.2 2024-05-08T18:05:27.222 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.222+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:108 4.26 4:658676bd:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c2e40 con 0x5593d9373720 2024-05-08T18:05:27.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.223+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(108 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.2 [stat,call,call] v27'8 uv8 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.223+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:109 3.27s0 3:e5a949cc:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_9c8454ddf7aa50116496bac348d7550d:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c2e40 con 0x5593d9373720 2024-05-08T18:05:27.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.224+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(109 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_9c8454ddf7aa50116496bac348d7550d [call] v27'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.224+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:110 4.26 4:658676bd:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c38d0 con 0x5593d9373720 2024-05-08T18:05:27.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.224+0000 7f8264887800 20 get_obj_state: octx=0x7f82540a2f20 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x5593d943a8d8 s->prefetch_data=0 2024-05-08T18:05:27.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.224+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:111 3.2as0 3:57efe377:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_b1b9a972ccd8c962a473909b97007eb4:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c38d0 con 0x5593d936ed30 2024-05-08T18:05:27.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.225+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(111 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_b1b9a972ccd8c962a473909b97007eb4 [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 262+0+895 (crc 0 0 0) 0x7f825c04d390 con 0x5593d936ed30 2024-05-08T18:05:27.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.225+0000 7f8264887800 10 manifest: total_size = 3 2024-05-08T18:05:27.224 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.225+0000 7f8264887800 20 get_obj_state: setting s->obj_tag to 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.16533754748163609326 2024-05-08T18:05:27.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.225+0000 7f8264887800 20 get_obj_state: octx=0x7f82540a2f20 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x5593d943a8d8 s->prefetch_data=0 2024-05-08T18:05:27.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.225+0000 7f8264887800 20 prepare_atomic_modification: state is not atomic. state=0x5593d943a8d8 2024-05-08T18:05:27.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.225+0000 7f8264887800 20 bucket index object: :.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.6 2024-05-08T18:05:27.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.225+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:112 4.3b 4:df5ac0d4:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c38d0 con 0x5593d93780d0 2024-05-08T18:05:27.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.225+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== osd_op_reply(110 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.2 [stat,call,call] v27'9 uv9 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.225+0000 7f8145ffb640 20 handle_completion(): completion ok for obj=9c8454ddf7aa50116496bac348d7550d 2024-05-08T18:05:27.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.226+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 32 ==== osd_op_reply(112 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.6 [stat,call,call] v27'8 uv8 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.226+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:113 3.2as0 3:57efe377:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_b1b9a972ccd8c962a473909b97007eb4:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c38d0 con 0x5593d936ed30 2024-05-08T18:05:27.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.227+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 32 ==== osd_op_reply(113 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_b1b9a972ccd8c962a473909b97007eb4 [call] v27'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f825c04d390 con 0x5593d936ed30 2024-05-08T18:05:27.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.227+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:114 4.3b 4:df5ac0d4:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c4540 con 0x5593d93780d0 2024-05-08T18:05:27.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.227+0000 7f8264887800 20 get_obj_state: octx=0x7f82540a2f20 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x5593d943a8d8 s->prefetch_data=0 2024-05-08T18:05:27.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.227+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:115 3.6s0 3:625b0886:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_c785e1ed2950e3e36b1e2ca01f299a54:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c4540 con 0x5593d936ed30 2024-05-08T18:05:27.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.227+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 33 ==== osd_op_reply(115 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_c785e1ed2950e3e36b1e2ca01f299a54 [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 262+0+895 (crc 0 0 0) 0x7f825c04d390 con 0x5593d936ed30 2024-05-08T18:05:27.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.228+0000 7f8264887800 10 manifest: total_size = 4 2024-05-08T18:05:27.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.228+0000 7f8264887800 20 get_obj_state: setting s->obj_tag to 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11604244899013991556 2024-05-08T18:05:27.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.228+0000 7f8264887800 20 get_obj_state: octx=0x7f82540a2f20 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x5593d943a8d8 s->prefetch_data=0 2024-05-08T18:05:27.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.228+0000 7f8264887800 20 prepare_atomic_modification: state is not atomic. state=0x5593d943a8d8 2024-05-08T18:05:27.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.228+0000 7f8264887800 20 bucket index object: :.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.8 2024-05-08T18:05:27.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.228+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:116 4.8 4:11bc5294:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c4540 con 0x5593d93780d0 2024-05-08T18:05:27.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.228+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 33 ==== osd_op_reply(114 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.6 [stat,call,call] v27'9 uv9 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.228+0000 7f81457fa640 20 handle_completion(): completion ok for obj=b1b9a972ccd8c962a473909b97007eb4 2024-05-08T18:05:27.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.229+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 34 ==== osd_op_reply(116 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.8 [stat,call,call] v27'4 uv4 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.229+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:117 3.6s0 3:625b0886:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_c785e1ed2950e3e36b1e2ca01f299a54:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c4540 con 0x5593d936ed30 2024-05-08T18:05:27.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.230+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 34 ==== osd_op_reply(117 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_c785e1ed2950e3e36b1e2ca01f299a54 [call] v27'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f825c04d390 con 0x5593d936ed30 2024-05-08T18:05:27.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.230+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:118 4.8 4:11bc5294:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c4ca0 con 0x5593d93780d0 2024-05-08T18:05:27.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.230+0000 7f8264887800 20 get_obj_state: octx=0x7f82540a2f20 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x5593d943a8d8 s->prefetch_data=0 2024-05-08T18:05:27.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.230+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:119 3.35s0 3:afa4ea37:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_24d27c169c2c881eb09a065116f2aa5c:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c4ca0 con 0x5593d936ed30 2024-05-08T18:05:27.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.230+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 35 ==== osd_op_reply(119 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_24d27c169c2c881eb09a065116f2aa5c [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 262+0+895 (crc 0 0 0) 0x7f825c04d390 con 0x5593d936ed30 2024-05-08T18:05:27.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.231+0000 7f8264887800 10 manifest: total_size = 5 2024-05-08T18:05:27.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.231+0000 7f8264887800 20 get_obj_state: setting s->obj_tag to 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.13868222138857627287 2024-05-08T18:05:27.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.231+0000 7f8264887800 20 get_obj_state: octx=0x7f82540a2f20 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x5593d943a8d8 s->prefetch_data=0 2024-05-08T18:05:27.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.231+0000 7f8264887800 20 prepare_atomic_modification: state is not atomic. state=0x5593d943a8d8 2024-05-08T18:05:27.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.231+0000 7f8264887800 20 bucket index object: :.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10 2024-05-08T18:05:27.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.231+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:120 4.16 4:6bff304f:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c4c40 con 0x5593d93780d0 2024-05-08T18:05:27.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.231+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 35 ==== osd_op_reply(118 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.8 [stat,call,call] v27'5 uv5 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.231+0000 7f8145ffb640 20 handle_completion(): completion ok for obj=c785e1ed2950e3e36b1e2ca01f299a54 2024-05-08T18:05:27.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.232+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 36 ==== osd_op_reply(120 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10 [stat,call,call] v27'12 uv12 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.232+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:121 3.35s0 3:afa4ea37:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_24d27c169c2c881eb09a065116f2aa5c:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c4c40 con 0x5593d936ed30 2024-05-08T18:05:27.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.233+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 36 ==== osd_op_reply(121 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_24d27c169c2c881eb09a065116f2aa5c [call] v27'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f825c04d390 con 0x5593d936ed30 2024-05-08T18:05:27.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.233+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:122 4.16 4:6bff304f:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c53a0 con 0x5593d93780d0 2024-05-08T18:05:27.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.233+0000 7f8264887800 20 get_obj_state: octx=0x7f82540a2f20 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x5593d943a8d8 s->prefetch_data=0 2024-05-08T18:05:27.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.233+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:123 3.32s0 3:4d0cffb4:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c53a0 con 0x5593d93780d0 2024-05-08T18:05:27.232 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.233+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 37 ==== osd_op_reply(123 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=910b,stat out=16b] v0'0 uv3 ondisk = 0) ==== 262+0+926 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.233+0000 7f8264887800 10 manifest: total_size = 5 2024-05-08T18:05:27.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.233+0000 7f8264887800 20 get_obj_state: setting s->obj_tag to _b0UVA55X0eEjPlUkdwo15sz8j2DbmBH 2024-05-08T18:05:27.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.233+0000 7f8264887800 20 get_obj_state: octx=0x7f82540a2f20 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x5593d943a8d8 s->prefetch_data=0 2024-05-08T18:05:27.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.233+0000 7f8264887800 20 prepare_atomic_modification: state is not atomic. state=0x5593d943a8d8 2024-05-08T18:05:27.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.234+0000 7f8264887800 20 bucket index object: :.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10 2024-05-08T18:05:27.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.234+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:124 4.16 4:6bff304f:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c53a0 con 0x5593d93780d0 2024-05-08T18:05:27.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.234+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 38 ==== osd_op_reply(122 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10 [stat,call,call] v27'13 uv13 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.234+0000 7f81457fa640 20 handle_completion(): completion ok for obj=24d27c169c2c881eb09a065116f2aa5c 2024-05-08T18:05:27.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.234+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 39 ==== osd_op_reply(124 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10 [stat,call,call] v27'14 uv14 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.233 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.235+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:125 3.32s0 3:4d0cffb4:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_bb3aec0fdcdbc2974890f805c585d432:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c53a0 con 0x5593d93780d0 2024-05-08T18:05:27.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.236+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 40 ==== osd_op_reply(125 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5_bb3aec0fdcdbc2974890f805c585d432 [call] v27'4 uv4 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.236+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:126 4.16 4:6bff304f:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c4460 con 0x5593d93780d0 2024-05-08T18:05:27.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.236+0000 7f8264887800 20 list_objects_ordered: starting attempt 1 2024-05-08T18:05:27.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.236+0000 7f8264887800 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-08T18:05:27.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.236+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:127 4.b 4:d16eb398:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c4460 con 0x7f8258037040 2024-05-08T18:05:27.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.236+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:128 4.15 4:aa9d0ef9:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93c4460 con 0x5593d93780d0 2024-05-08T18:05:27.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.236+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:129 4.26 4:658676bd:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d943aba0 con 0x5593d9373720 2024-05-08T18:05:27.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.236+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:130 4.9 4:927242f4:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d943aba0 con 0x5593d936ed30 2024-05-08T18:05:27.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.236+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:131 4.2c 4:341c0785:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d1310 con 0x5593d936ed30 2024-05-08T18:05:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.236+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:132 4.3c 4:3da0c2be:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d3de0 con 0x5593d936ed30 2024-05-08T18:05:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.236+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:133 4.3b 4:df5ac0d4:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d68b0 con 0x5593d93780d0 2024-05-08T18:05:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.236+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 20 ==== osd_op_reply(127 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f825c04d390 con 0x7f8258037040 2024-05-08T18:05:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.237+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 31 ==== osd_op_reply(129 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.2 [call out=124b] v0'0 uv9 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.237+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 41 ==== osd_op_reply(128 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.237+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 37 ==== osd_op_reply(131 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f825c04d390 con 0x5593d936ed30 2024-05-08T18:05:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.237+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 38 ==== osd_op_reply(130 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.3 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f825c04d390 con 0x5593d936ed30 2024-05-08T18:05:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.237+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:134 4.1f 4:fb972dd1:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d3de0 con 0x5593d93780d0 2024-05-08T18:05:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.237+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:135 4.8 4:11bc5294:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d9438e90 con 0x5593d93780d0 2024-05-08T18:05:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.237+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:136 4.2f 4:f5b60246:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0ed0 con 0x5593d936ed30 2024-05-08T18:05:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.237+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:137 4.16 4:6bff304f:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93cf400 con 0x5593d93780d0 2024-05-08T18:05:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.237+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 42 ==== osd_op_reply(126 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10 [stat,call,call] v27'15 uv15 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.237+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 39 ==== osd_op_reply(132 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.5 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f825c04d390 con 0x5593d936ed30 2024-05-08T18:05:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.237+0000 7f81457fa640 20 handle_completion(): completion ok for obj=bb3aec0fdcdbc2974890f805c585d432 2024-05-08T18:05:27.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.237+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 40 ==== osd_op_reply(136 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f825c04d390 con 0x5593d936ed30 2024-05-08T18:05:27.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.237+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 43 ==== osd_op_reply(133 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.6 [call out=124b] v0'0 uv9 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.237+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 44 ==== osd_op_reply(134 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.7 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 45 ==== osd_op_reply(137 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10 [call out=124b] v0'0 uv15 ondisk = 0) ==== 195+0+124 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 46 ==== osd_op_reply(135 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.8 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8264887800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-08T18:05:27.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8264887800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-08T18:05:27.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8264887800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-08T18:05:27.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8264887800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-08T18:05:27.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8264887800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5])"; uploads.size()=0; is_truncated=0 2024-05-08T18:05:27.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:138 4.b 4:d16eb398:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0ed0 con 0x7f8258037040 2024-05-08T18:05:27.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:139 4.15 4:aa9d0ef9:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93cf400 con 0x5593d93780d0 2024-05-08T18:05:27.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:140 4.26 4:658676bd:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d9438e90 con 0x5593d9373720 2024-05-08T18:05:27.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:141 4.9 4:927242f4:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d1310 con 0x5593d936ed30 2024-05-08T18:05:27.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:142 4.2c 4:341c0785:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d2c60 con 0x5593d936ed30 2024-05-08T18:05:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:143 4.3c 4:3da0c2be:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0ed0 con 0x5593d936ed30 2024-05-08T18:05:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:144 4.3b 4:df5ac0d4:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93cf400 con 0x5593d93780d0 2024-05-08T18:05:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:145 4.1f 4:fb972dd1:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0ed0 con 0x5593d93780d0 2024-05-08T18:05:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:146 4.8 4:11bc5294:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93cf400 con 0x5593d93780d0 2024-05-08T18:05:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:147 4.2f 4:f5b60246:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d2c60 con 0x5593d936ed30 2024-05-08T18:05:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:148 4.16 4:6bff304f:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0ed0 con 0x5593d93780d0 2024-05-08T18:05:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 21 ==== osd_op_reply(138 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f825c05b040 con 0x7f8258037040 2024-05-08T18:05:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 47 ==== osd_op_reply(139 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 32 ==== osd_op_reply(140 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.2 [call out=124b] v0'0 uv9 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 48 ==== osd_op_reply(144 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.6 [call out=124b] v0'0 uv9 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 41 ==== osd_op_reply(142 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f825c05b040 con 0x5593d936ed30 2024-05-08T18:05:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 49 ==== osd_op_reply(145 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.7 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 50 ==== osd_op_reply(146 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.8 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 42 ==== osd_op_reply(141 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.3 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f825c05b040 con 0x5593d936ed30 2024-05-08T18:05:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 43 ==== osd_op_reply(143 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.5 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f825c05b040 con 0x5593d936ed30 2024-05-08T18:05:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.238+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 44 ==== osd_op_reply(147 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f825c05b040 con 0x5593d936ed30 2024-05-08T18:05:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.239+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 51 ==== osd_op_reply(148 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10 [call out=124b] v0'0 uv15 ondisk = 0) ==== 195+0+124 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.239+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:149 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93cf400 con 0x7f8258037040 2024-05-08T18:05:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.240+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 22 ==== osd_op_reply(149 foo.buckets [call] v27'39 uv39 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f825c05b040 con 0x7f8258037040 2024-05-08T18:05:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.240+0000 7f8264887800 10 cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x11, cached=0x11) 2024-05-08T18:05:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.240+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:150 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0af0 con 0x7f8258037040 2024-05-08T18:05:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.241+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 23 ==== osd_op_reply(150 myfoo [call,call,delete] v27'40 uv40 ondisk = 0) ==== 233+0+0 (crc 0 0 0) 0x7f825c05b040 con 0x7f8258037040 2024-05-08T18:05:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.241+0000 7f8264887800 10 removing default.rgw.meta+root+myfoo from cache 2024-05-08T18:05:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.241+0000 7f8264887800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-05-08T18:05:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.241+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:151 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94093493211824 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0af0 con 0x5593d9373720 2024-05-08T18:05:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.241+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 33 ==== watch-notify(notify (1) cookie 94093492886848 notify 115964117017 ret 0) ==== 191+0+0 (crc 0 0 0) 0x5593d93bd230 con 0x5593d9373720 2024-05-08T18:05:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.241+0000 7f81457fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117017 cookie 94093492886848 notifier 4641 bl.length()=149 2024-05-08T18:05:27.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.241+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 34 ==== osd_op_reply(151 notify.6 [notify cookie 94093493211824 out=8b] v0'0 uv177 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.241+0000 7f81457fa640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:152 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f80980032c0 con 0x5593d9373720 2024-05-08T18:05:27.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.241+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 35 ==== osd_op_reply(152 notify.6 [notify-ack] v0'0 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.241+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 36 ==== watch-notify(notify_complete (2) cookie 94093493211824 notify 115964117017 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5593d934f970 con 0x5593d9373720 2024-05-08T18:05:27.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.241+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:153 8.7 8:e716d2f9:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0af0 con 0x5593d9373720 2024-05-08T18:05:27.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.242+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 37 ==== osd_op_reply(153 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 [call,call,delete] v27'3 uv3 ondisk = 0) ==== 290+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.242+0000 7f8264887800 10 removing default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 from cache 2024-05-08T18:05:27.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.242+0000 7f8264887800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5, ofs0, ns] 2024-05-08T18:05:27.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.242+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:154 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94093493211824 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0af0 con 0x5593d9373720 2024-05-08T18:05:27.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.243+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 38 ==== watch-notify(notify (1) cookie 94093492886848 notify 115964117018 ret 0) ==== 248+0+0 (crc 0 0 0) 0x5593d934f970 con 0x5593d9373720 2024-05-08T18:05:27.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.243+0000 7f81457fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117018 cookie 94093492886848 notifier 4641 bl.length()=206 2024-05-08T18:05:27.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.243+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 39 ==== osd_op_reply(154 notify.6 [notify cookie 94093493211824 out=8b] v0'0 uv177 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.243+0000 7f81457fa640 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:155 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f8098003260 con 0x5593d9373720 2024-05-08T18:05:27.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.243+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 40 ==== watch-notify(notify_complete (2) cookie 94093493211824 notify 115964117018 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5593d91a9b50 con 0x5593d9373720 2024-05-08T18:05:27.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.243+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 41 ==== osd_op_reply(155 notify.6 [notify-ack] v0'0 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.243+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:156 4.b 4:d16eb398:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0af0 con 0x7f8258037040 2024-05-08T18:05:27.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.243+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:157 4.15 4:aa9d0ef9:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93cf400 con 0x5593d93780d0 2024-05-08T18:05:27.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.243+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:158 4.26 4:658676bd:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d12b0 con 0x5593d9373720 2024-05-08T18:05:27.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.243+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:159 4.9 4:927242f4:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93bb5c0 con 0x5593d936ed30 2024-05-08T18:05:27.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.243+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:160 4.2c 4:341c0785:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93bc170 con 0x5593d936ed30 2024-05-08T18:05:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.243+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:161 4.3c 4:3da0c2be:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0af0 con 0x5593d936ed30 2024-05-08T18:05:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.243+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:162 4.3b 4:df5ac0d4:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93cf400 con 0x5593d93780d0 2024-05-08T18:05:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.243+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:163 4.1f 4:fb972dd1:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93bb5c0 con 0x5593d93780d0 2024-05-08T18:05:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.243+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:164 4.8 4:11bc5294:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d12b0 con 0x5593d93780d0 2024-05-08T18:05:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.244+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:165 4.2f 4:f5b60246:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93cf400 con 0x5593d936ed30 2024-05-08T18:05:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.244+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:166 4.16 4:6bff304f:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d12b0 con 0x5593d93780d0 2024-05-08T18:05:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.244+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 42 ==== osd_op_reply(158 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.2 [delete] v27'10 uv10 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.245+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 24 ==== osd_op_reply(156 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.0 [delete] v27'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f825c05b040 con 0x7f8258037040 2024-05-08T18:05:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.245+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 45 ==== osd_op_reply(159 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.3 [delete] v27'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f825c05b040 con 0x5593d936ed30 2024-05-08T18:05:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.245+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 46 ==== osd_op_reply(160 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.4 [delete] v27'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f825c05b040 con 0x5593d936ed30 2024-05-08T18:05:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.245+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 47 ==== osd_op_reply(161 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.5 [delete] v27'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f825c05b040 con 0x5593d936ed30 2024-05-08T18:05:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.245+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 52 ==== osd_op_reply(157 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.1 [delete] v27'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.245+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 53 ==== osd_op_reply(162 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.6 [delete] v27'10 uv10 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.245+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 54 ==== osd_op_reply(163 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.7 [delete] v27'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.245+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 55 ==== osd_op_reply(166 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.10 [delete] v27'16 uv16 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.245+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 48 ==== osd_op_reply(165 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.9 [delete] v27'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f825c05b040 con 0x5593d936ed30 2024-05-08T18:05:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.245+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 56 ==== osd_op_reply(164 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5.8 [delete] v27'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.246+0000 7f8264887800 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 : miss 2024-05-08T18:05:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.246+0000 7f8264887800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.246+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:167 6.e 6:71b37635:::pubsub..bucket.myfoo%2f1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0af0 con 0x5593d936ed30 2024-05-08T18:05:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.246+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 49 ==== osd_op_reply(167 pubsub..bucket.myfoo/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 208+0+0 (crc 0 0 0) 0x7f825c05b040 con 0x5593d936ed30 2024-05-08T18:05:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.246+0000 7f8264887800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.246+0000 7f8264887800 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 info.flags=0x0 2024-05-08T18:05:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.246+0000 7f8264887800 10 adding default.rgw.log++pubsub..bucket.myfoo/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 to cache LRU end 2024-05-08T18:05:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.246+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:168 6.e 6:71b37635:::pubsub..bucket.myfoo%2f1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93bc170 con 0x5593d936ed30 2024-05-08T18:05:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.247+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 50 ==== osd_op_reply(168 pubsub..bucket.myfoo/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.5 [delete] v27'27 uv23 ondisk = -2 ((2) No such file or directory)) ==== 208+0+0 (crc 0 0 0) 0x7f825c05b040 con 0x5593d936ed30 2024-05-08T18:05:27.250 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.247+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:169 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93bb5c0 con 0x7f8258037040 2024-05-08T18:05:27.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.248+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 25 ==== osd_op_reply(169 foo.buckets [call] v27'41 uv41 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f825c05b040 con 0x7f8258037040 2024-05-08T18:05:27.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.248+0000 7f8118fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:27.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.248+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:170 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94093492876544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0af0 con 0x5593d936ed30 2024-05-08T18:05:27.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.250+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 51 ==== osd_op_reply(170 data_loggenerations_metadata [watch unwatch cookie 94093492876544] v27'186 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f825c05b040 con 0x5593d936ed30 2024-05-08T18:05:27.251 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.250+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:171 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94093492864096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0af0 con 0x7f8258037040 2024-05-08T18:05:27.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.251+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 26 ==== osd_op_reply(171 notify.0 [watch unwatch cookie 94093492864096] v27'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f825c05b040 con 0x7f8258037040 2024-05-08T18:05:27.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.251+0000 7f8264887800 20 remove_watcher() i=0 2024-05-08T18:05:27.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.251+0000 7f8264887800 2 removed watcher, disabling cache 2024-05-08T18:05:27.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.251+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:172 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94093492868528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0af0 con 0x5593d936ed30 2024-05-08T18:05:27.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.252+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 52 ==== osd_op_reply(172 notify.1 [watch unwatch cookie 94093492868528] v27'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f825c05b040 con 0x5593d936ed30 2024-05-08T18:05:27.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.252+0000 7f8264887800 20 remove_watcher() i=1 2024-05-08T18:05:27.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.252+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:173 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94093492872288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0af0 con 0x5593d93780d0 2024-05-08T18:05:27.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.253+0000 7f8261b7e640 1 -- 172.21.15.50:0/1307305089 <== osd.3 v2:172.21.15.50:6800/3130492744 57 ==== osd_op_reply(173 notify.2 [watch unwatch cookie 94093492872288] v27'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8258009050 con 0x5593d93780d0 2024-05-08T18:05:27.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.253+0000 7f8264887800 20 remove_watcher() i=2 2024-05-08T18:05:27.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.253+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:174 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94093492870736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0af0 con 0x7f8258037040 2024-05-08T18:05:27.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.254+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.2 v2:172.21.15.50:6811/3039646617 27 ==== osd_op_reply(174 notify.3 [watch unwatch cookie 94093492870736] v27'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f825c05b040 con 0x7f8258037040 2024-05-08T18:05:27.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.254+0000 7f8264887800 20 remove_watcher() i=3 2024-05-08T18:05:27.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.254+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:175 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94093492878224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0af0 con 0x5593d936ed30 2024-05-08T18:05:27.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.255+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 <== osd.1 v2:172.21.15.50:6808/2141923372 53 ==== osd_op_reply(175 notify.4 [watch unwatch cookie 94093492878224] v27'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f825c05b040 con 0x5593d936ed30 2024-05-08T18:05:27.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.255+0000 7f8264887800 20 remove_watcher() i=4 2024-05-08T18:05:27.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.255+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:176 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94093492882384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0af0 con 0x5593d9373720 2024-05-08T18:05:27.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.256+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 43 ==== osd_op_reply(176 notify.5 [watch unwatch cookie 94093492882384] v27'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.256+0000 7f8264887800 20 remove_watcher() i=5 2024-05-08T18:05:27.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.256+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:177 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94093492886848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0af0 con 0x5593d9373720 2024-05-08T18:05:27.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.257+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 44 ==== osd_op_reply(177 notify.6 [watch unwatch cookie 94093492886848] v27'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.257+0000 7f8264887800 20 remove_watcher() i=6 2024-05-08T18:05:27.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.257+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:178 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94093492890400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5593d93d0af0 con 0x5593d9373720 2024-05-08T18:05:27.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f826237f640 1 -- 172.21.15.50:0/1307305089 <== osd.0 v2:172.21.15.46:6800/337143975 45 ==== osd_op_reply(178 notify.7 [watch unwatch cookie 94093492890400] v27'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f82540ba070 con 0x5593d9373720 2024-05-08T18:05:27.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 20 remove_watcher() i=7 2024-05-08T18:05:27.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5593d9373720 msgr2=0x5593d9375b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5593d9373720 0x5593d9375b10 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5593d936ed30 msgr2=0x5593d93711f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5593d936ed30 0x5593d93711f0 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f8258037040 msgr2=0x7f8258036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f8258037040 0x7f8258036410 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5593d93780d0 msgr2=0x5593d937a590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5593d93780d0 0x5593d937a590 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f812c03bab0 msgr2=0x7f812c03df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f812c03bab0 0x7f812c03df70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x5593d93410a0 tx=0x7f8258010000 comp rx=0 tx=0).stop 2024-05-08T18:05:27.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d935c1c0 msgr2=0x5593d9363580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 1 --2- 172.21.15.50:0/1307305089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d935c1c0 0x5593d9363580 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f825400a290 tx=0x7f825406df80 comp rx=0 tx=0).stop 2024-05-08T18:05:27.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 reap_dead start 2024-05-08T18:05:27.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8262b80640 1 -- 172.21.15.50:0/1307305089 reap_dead start 2024-05-08T18:05:27.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 shutdown_connections 2024-05-08T18:05:27.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 >> 172.21.15.50:0/1307305089 conn(0x5593d93584d0 msgr2=0x5593d9355b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 shutdown_connections 2024-05-08T18:05:27.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.258+0000 7f8264887800 1 -- 172.21.15.50:0/1307305089 wait complete. 2024-05-08T18:05:27.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 -- 172.21.15.50:0/3590555564 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f813c03bba0 msgr2=0x7f813c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 --2- 172.21.15.50:0/3590555564 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f813c03bba0 0x7f813c03e060 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x5593d91a9f70 tx=0x7f825800f000 comp rx=0 tx=0).stop 2024-05-08T18:05:27.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 -- 172.21.15.50:0/3590555564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d934f010 msgr2=0x5593d934f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 --2- 172.21.15.50:0/3590555564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d934f010 0x5593d934f430 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8254092a10 tx=0x7f825405b430 comp rx=0 tx=0).stop 2024-05-08T18:05:27.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 -- 172.21.15.50:0/3590555564 shutdown_connections 2024-05-08T18:05:27.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 --2- 172.21.15.50:0/3590555564 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f813c03bba0 0x7f813c03e060 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 --2- 172.21.15.50:0/3590555564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d934f010 0x5593d934f430 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 -- 172.21.15.50:0/3590555564 >> 172.21.15.50:0/3590555564 conn(0x5593d93430b0 msgr2=0x5593d93434b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 -- 172.21.15.50:0/3590555564 shutdown_connections 2024-05-08T18:05:27.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 -- 172.21.15.50:0/3590555564 wait complete. 2024-05-08T18:05:27.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 -- 172.21.15.50:0/1839521994 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5593d933a300 msgr2=0x5593d933c6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5593d933a300 0x5593d933c6f0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 -- 172.21.15.50:0/1839521994 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5593d92368d0 msgr2=0x5593d9337af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5593d92368d0 0x5593d9337af0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 -- 172.21.15.50:0/1839521994 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5593d933e1b0 msgr2=0x5593d93405a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5593d933e1b0 0x5593d93405a0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 -- 172.21.15.50:0/1839521994 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f814803bba0 msgr2=0x7f814803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f814803bba0 0x7f814803e060 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8258004170 tx=0x7f8258001330 comp rx=0 tx=0).stop 2024-05-08T18:05:27.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 -- 172.21.15.50:0/1839521994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 msgr2=0x5593d91aae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 --2- 172.21.15.50:0/1839521994 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5593d9235710 0x5593d91aae40 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f82540057d0 tx=0x7f8254002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8262b80640 1 -- 172.21.15.50:0/1839521994 reap_dead start 2024-05-08T18:05:27.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 -- 172.21.15.50:0/1839521994 shutdown_connections 2024-05-08T18:05:27.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.259+0000 7f8264887800 1 -- 172.21.15.50:0/1839521994 >> 172.21.15.50:0/1839521994 conn(0x5593d9230c90 msgr2=0x5593d9320d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.260+0000 7f8264887800 1 -- 172.21.15.50:0/1839521994 shutdown_connections 2024-05-08T18:05:27.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.260+0000 7f8264887800 1 -- 172.21.15.50:0/1839521994 wait complete. 2024-05-08T18:05:27.267 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2024-05-08T18:05:27.267 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2024-05-08T18:05:27.267 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid foo --caps user=read 2024-05-08T18:05:27.314 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:27.314 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:27.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.317+0000 7ff8c5d36800 1 Processor -- start 2024-05-08T18:05:27.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.317+0000 7ff8c5d36800 1 -- start start 2024-05-08T18:05:27.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.317+0000 7ff8c5d36800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x562d7cc8f710 0x562d7cc8fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.317+0000 7ff8c5d36800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562d7cc901b0 con 0x562d7cc908d0 2024-05-08T18:05:27.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.317+0000 7ff8c5d36800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x562d7cb2fe20 con 0x562d7cc8f710 2024-05-08T18:05:27.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.318+0000 7ff8c1ac5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x562d7cc8f710 0x562d7cc8fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.318+0000 7ff8c12c4640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x562d7cc908d0 0x562d7cc90d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:60404/0 (socket says 172.21.15.50:60404) 2024-05-08T18:05:27.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.318+0000 7ff8c12c4640 1 -- 172.21.15.50:0/1225616545 learned_addr learned my addr 172.21.15.50:0/1225616545 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:27.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.318+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/1225616545 >> v1:172.21.15.46:6789/0 conn(0x562d7cc908d0 legacy=0x562d7cc90d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.318+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/1225616545 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d7cc91490 con 0x562d7cc8f710 2024-05-08T18:05:27.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.320+0000 7ff8c1ac5640 1 --2- 172.21.15.50:0/1225616545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 0x562d7cc8fb30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400a470 tx=0x7ff8b401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb1269ec5f94aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.320+0000 7ff8c0ac3640 1 -- 172.21.15.50:0/1225616545 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b4057070 con 0x562d7cc8f710 2024-05-08T18:05:27.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.320+0000 7ff8c0ac3640 1 -- 172.21.15.50:0/1225616545 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b404adc0 con 0x562d7cc8f710 2024-05-08T18:05:27.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.320+0000 7ff8c0ac3640 1 -- 172.21.15.50:0/1225616545 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b4052590 con 0x562d7cc8f710 2024-05-08T18:05:27.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.321+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1225616545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 msgr2=0x562d7cc8fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.321+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/1225616545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 0x562d7cc8fb30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400a470 tx=0x7ff8b401ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:27.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.321+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1225616545 shutdown_connections 2024-05-08T18:05:27.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.321+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/1225616545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 0x562d7cc8fb30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.321+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1225616545 >> 172.21.15.50:0/1225616545 conn(0x562d7cc8ac90 msgr2=0x562d7cc8d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.321+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1225616545 shutdown_connections 2024-05-08T18:05:27.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.321+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1225616545 wait complete. 2024-05-08T18:05:27.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.336+0000 7ff8c5d36800 1 Processor -- start 2024-05-08T18:05:27.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.336+0000 7ff8c5d36800 1 -- start start 2024-05-08T18:05:27.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.336+0000 7ff8c5d36800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 0x562d7cc04f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.336+0000 7ff8c5d36800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d7cbdaa30 con 0x562d7cc8f710 2024-05-08T18:05:27.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.336+0000 7ff8c1ac5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 0x562d7cc04f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.336+0000 7ff8c1ac5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 0x562d7cc04f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:44000/0 (socket says 172.21.15.50:44000) 2024-05-08T18:05:27.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.336+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/1228018218 learned_addr learned my addr 172.21.15.50:0/1228018218 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.337+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/1228018218 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d7cc034c0 con 0x562d7cc8f710 2024-05-08T18:05:27.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.337+0000 7ff8c1ac5640 1 --2- 172.21.15.50:0/1228018218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 0x562d7cc04f00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4055040 tx=0x7ff8b404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8fc2504ffe10db server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.337+0000 7ff7afe07640 1 -- 172.21.15.50:0/1228018218 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b4057040 con 0x562d7cc8f710 2024-05-08T18:05:27.335 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.337+0000 7ff7afe07640 1 -- 172.21.15.50:0/1228018218 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b4062070 con 0x562d7cc8f710 2024-05-08T18:05:27.336 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.337+0000 7ff7afe07640 1 -- 172.21.15.50:0/1228018218 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b405d470 con 0x562d7cc8f710 2024-05-08T18:05:27.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.338+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1228018218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 msgr2=0x562d7cc04f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.338+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/1228018218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 0x562d7cc04f00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4055040 tx=0x7ff8b404c000 comp rx=0 tx=0).stop 2024-05-08T18:05:27.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.338+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1228018218 shutdown_connections 2024-05-08T18:05:27.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.338+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/1228018218 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 0x562d7cc04f00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.338+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1228018218 >> 172.21.15.50:0/1228018218 conn(0x562d7cc8ac90 msgr2=0x562d7cd7c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.338+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1228018218 shutdown_connections 2024-05-08T18:05:27.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.338+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1228018218 wait complete. 2024-05-08T18:05:27.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.338+0000 7ff8c5d36800 1 Processor -- start 2024-05-08T18:05:27.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.338+0000 7ff8c5d36800 1 -- start start 2024-05-08T18:05:27.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.338+0000 7ff8c5d36800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 0x562d7cd8a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.338+0000 7ff8c5d36800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d7cc98da0 con 0x562d7cc8f710 2024-05-08T18:05:27.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.339+0000 7ff8c1ac5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 0x562d7cd8a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.339+0000 7ff8c1ac5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 0x562d7cd8a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:44008/0 (socket says 172.21.15.50:44008) 2024-05-08T18:05:27.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.339+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/1015357448 learned_addr learned my addr 172.21.15.50:0/1015357448 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.339+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/1015357448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d7cd8adc0 con 0x562d7cc8f710 2024-05-08T18:05:27.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.339+0000 7ff8c1ac5640 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 0x562d7cd8a880 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff8b405f040 tx=0x7ff8b4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.339+0000 7ff7ae604640 1 -- 172.21.15.50:0/1015357448 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b406b070 con 0x562d7cc8f710 2024-05-08T18:05:27.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.339+0000 7ff7ae604640 1 -- 172.21.15.50:0/1015357448 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b405d940 con 0x562d7cc8f710 2024-05-08T18:05:27.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.339+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1015357448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562d7cd8b050 con 0x562d7cc8f710 2024-05-08T18:05:27.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.339+0000 7ff7ae604640 1 -- 172.21.15.50:0/1015357448 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b4057070 con 0x562d7cc8f710 2024-05-08T18:05:27.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.339+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1015357448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d7cd8dd00 con 0x562d7cc8f710 2024-05-08T18:05:27.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.340+0000 7ff7ae604640 1 -- 172.21.15.50:0/1015357448 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff8b405dc20 con 0x562d7cc8f710 2024-05-08T18:05:27.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.340+0000 7ff7ae604640 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff7a803bb50 0x7ff7a803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.340+0000 7ff7ae604640 1 -- 172.21.15.50:0/1015357448 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7ff8b40946a0 con 0x562d7cc8f710 2024-05-08T18:05:27.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.341+0000 7ff8c12c4640 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff7a803bb50 0x7ff7a803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.341+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562d7cc908d0 0x562d7cd91ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.341+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1015357448 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cd92040 con 0x562d7cc908d0 2024-05-08T18:05:27.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.341+0000 7ff8c22c6640 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562d7cc908d0 0x562d7cd91ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.341+0000 7ff8c12c4640 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff7a803bb50 0x7ff7a803e010 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8001d70 tx=0x7ff8b80012d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.341+0000 7ff8c22c6640 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562d7cc908d0 0x562d7cd91ac0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.341+0000 7ff8c22c6640 1 -- 172.21.15.50:0/1015357448 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cc908d0 2024-05-08T18:05:27.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.341+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1015357448 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cd92910 con 0x562d7cc908d0 2024-05-08T18:05:27.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.342+0000 7ff8c22c6640 1 -- 172.21.15.50:0/1015357448 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cc908d0 2024-05-08T18:05:27.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.342+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562d7cd94220 0x562d7cd966b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.342+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1015357448 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cd96e40 con 0x562d7cd94220 2024-05-08T18:05:27.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.342+0000 7ff8c1ac5640 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562d7cd94220 0x562d7cd966b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.342+0000 7ff8c1ac5640 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562d7cd94220 0x562d7cd966b0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.342+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/1015357448 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff8b405a040 con 0x562d7cd94220 2024-05-08T18:05:27.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.342+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1015357448 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cd93160 con 0x562d7cc908d0 2024-05-08T18:05:27.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.343+0000 7ff8c22c6640 1 -- 172.21.15.50:0/1015357448 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cc908d0 2024-05-08T18:05:27.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.343+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562d7cd982a0 0x562d7cd9a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.343+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1015357448 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cd9ad80 con 0x562d7cd982a0 2024-05-08T18:05:27.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.343+0000 7ff8c12c4640 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562d7cd982a0 0x562d7cd9a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.343+0000 7ff8c12c4640 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562d7cd982a0 0x562d7cd9a690 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.343+0000 7ff8c12c4640 1 -- 172.21.15.50:0/1015357448 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff8b8009050 con 0x562d7cd982a0 2024-05-08T18:05:27.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.344+0000 7ff8c5d36800 1 Processor -- start 2024-05-08T18:05:27.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.344+0000 7ff8c5d36800 1 -- start start 2024-05-08T18:05:27.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.344+0000 7ff8c5d36800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cda90f0 0x562d7cda9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.344+0000 7ff8c5d36800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d7cb81330 con 0x562d7cda90f0 2024-05-08T18:05:27.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.344+0000 7ff8c1ac5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cda90f0 0x562d7cda9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.344+0000 7ff8c1ac5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cda90f0 0x562d7cda9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:44014/0 (socket says 172.21.15.50:44014) 2024-05-08T18:05:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.344+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/3948618553 learned_addr learned my addr 172.21.15.50:0/3948618553 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.345+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/3948618553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d7cda9a50 con 0x562d7cda90f0 2024-05-08T18:05:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.345+0000 7ff8c1ac5640 1 --2- 172.21.15.50:0/3948618553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cda90f0 0x562d7cda9510 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4092a30 tx=0x7ff8b405b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.345+0000 7ff7a77fe640 1 -- 172.21.15.50:0/3948618553 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b4086070 con 0x562d7cda90f0 2024-05-08T18:05:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.345+0000 7ff7a77fe640 1 -- 172.21.15.50:0/3948618553 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b4091380 con 0x562d7cda90f0 2024-05-08T18:05:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.345+0000 7ff8c5d36800 1 -- 172.21.15.50:0/3948618553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562d7cdaa4c0 con 0x562d7cda90f0 2024-05-08T18:05:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.345+0000 7ff7a77fe640 1 -- 172.21.15.50:0/3948618553 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b4052620 con 0x562d7cda90f0 2024-05-08T18:05:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.345+0000 7ff8c5d36800 1 -- 172.21.15.50:0/3948618553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d7cdaa9c0 con 0x562d7cda90f0 2024-05-08T18:05:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.346+0000 7ff7a77fe640 1 -- 172.21.15.50:0/3948618553 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff8b4089070 con 0x562d7cda90f0 2024-05-08T18:05:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.346+0000 7ff7a77fe640 1 --2- 172.21.15.50:0/3948618553 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff79c03bba0 0x7ff79c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.346+0000 7ff7a77fe640 1 -- 172.21.15.50:0/3948618553 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7ff8b4096080 con 0x562d7cda90f0 2024-05-08T18:05:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.346+0000 7ff8c12c4640 1 --2- 172.21.15.50:0/3948618553 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff79c03bba0 0x7ff79c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.346+0000 7ff8c5d36800 1 Processor -- start 2024-05-08T18:05:27.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.346+0000 7ff8c5d36800 1 -- start start 2024-05-08T18:05:27.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.346+0000 7ff8c12c4640 1 --2- 172.21.15.50:0/3948618553 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff79c03bba0 0x7ff79c03e060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80016b0 tx=0x7ff8b800f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.346+0000 7ff8c5d36800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cdb6210 0x562d7cdb6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.346+0000 7ff8c5d36800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d7cc03780 con 0x562d7cdb6210 2024-05-08T18:05:27.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.347+0000 7ff8c1ac5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cdb6210 0x562d7cdb6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.347+0000 7ff8c1ac5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cdb6210 0x562d7cdb6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:44028/0 (socket says 172.21.15.50:44028) 2024-05-08T18:05:27.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.347+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/1450869228 learned_addr learned my addr 172.21.15.50:0/1450869228 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.347+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/1450869228 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d7cdb6c00 con 0x562d7cdb6210 2024-05-08T18:05:27.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.347+0000 7ff8c1ac5640 1 --2- 172.21.15.50:0/1450869228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cdb6210 0x562d7cdb6630 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff8b405d490 tx=0x7ff8b408b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2c357ff02ac328 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.347+0000 7ff7a57fa640 1 -- 172.21.15.50:0/1450869228 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b4081db0 con 0x562d7cdb6210 2024-05-08T18:05:27.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.347+0000 7ff7a57fa640 1 -- 172.21.15.50:0/1450869228 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b4081400 con 0x562d7cdb6210 2024-05-08T18:05:27.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.347+0000 7ff7a57fa640 1 -- 172.21.15.50:0/1450869228 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b4070460 con 0x562d7cdb6210 2024-05-08T18:05:27.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.348+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1450869228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cdb6210 msgr2=0x562d7cdb6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.348+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/1450869228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cdb6210 0x562d7cdb6630 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff8b405d490 tx=0x7ff8b408b810 comp rx=0 tx=0).stop 2024-05-08T18:05:27.350 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.348+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1450869228 shutdown_connections 2024-05-08T18:05:27.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.348+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/1450869228 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cdb6210 0x562d7cdb6630 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.348+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1450869228 >> 172.21.15.50:0/1450869228 conn(0x562d7cdb2520 msgr2=0x562d7cdb0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.348+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1450869228 shutdown_connections 2024-05-08T18:05:27.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.348+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1450869228 wait complete. 2024-05-08T18:05:27.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.348+0000 7ff8c5d36800 1 Processor -- start 2024-05-08T18:05:27.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.348+0000 7ff8c5d36800 1 -- start start 2024-05-08T18:05:27.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.348+0000 7ff8c5d36800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cdb6210 0x562d7cdbd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.348+0000 7ff8c5d36800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562d7cdb7290 con 0x562d7cdb6210 2024-05-08T18:05:27.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.348+0000 7ff8c1ac5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cdb6210 0x562d7cdbd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.348+0000 7ff8c1ac5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cdb6210 0x562d7cdbd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:44034/0 (socket says 172.21.15.50:44034) 2024-05-08T18:05:27.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.348+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 learned_addr learned my addr 172.21.15.50:0/4192247129 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.349+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d7cdbdaf0 con 0x562d7cdb6210 2024-05-08T18:05:27.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.349+0000 7ff8c1ac5640 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cdb6210 0x562d7cdbd5b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4070f90 tx=0x7ff8b4091730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.349+0000 7ff7937fe640 1 -- 172.21.15.50:0/4192247129 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b40b1070 con 0x562d7cdb6210 2024-05-08T18:05:27.351 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.349+0000 7ff7937fe640 1 -- 172.21.15.50:0/4192247129 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b40949c0 con 0x562d7cdb6210 2024-05-08T18:05:27.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.349+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562d7cdbdaf0 con 0x562d7cdb6210 2024-05-08T18:05:27.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.349+0000 7ff7937fe640 1 -- 172.21.15.50:0/4192247129 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8b4094ca0 con 0x562d7cdb6210 2024-05-08T18:05:27.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.349+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d7cdbdaf0 con 0x562d7cdb6210 2024-05-08T18:05:27.352 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.350+0000 7ff7937fe640 1 -- 172.21.15.50:0/4192247129 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff8b408b8c0 con 0x562d7cdb6210 2024-05-08T18:05:27.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.350+0000 7ff7937fe640 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff78c03bba0 0x7ff78c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.350+0000 7ff7937fe640 1 -- 172.21.15.50:0/4192247129 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7ff8b40b6c60 con 0x562d7cdb6210 2024-05-08T18:05:27.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.350+0000 7ff8c12c4640 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff78c03bba0 0x7ff78c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.351+0000 7ff8c12c4640 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff78c03bba0 0x7ff78c03e060 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800efd0 tx=0x7ff8b8010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.351+0000 7ff8c5d36800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.351+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562d7cdc8db0 0x562d7cdcb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.351+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcb9c0 con 0x562d7cdc8db0 2024-05-08T18:05:27.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.351+0000 7ff8c22c6640 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562d7cdc8db0 0x562d7cdcb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.351+0000 7ff8c22c6640 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562d7cdc8db0 0x562d7cdcb270 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cdc8db0 2024-05-08T18:05:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c5d36800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c5d36800 20 realm 2024-05-08T18:05:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c5d36800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcbdd0 con 0x562d7cdc8db0 2024-05-08T18:05:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cdc8db0 2024-05-08T18:05:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c5d36800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c5d36800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c5d36800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcc6a0 con 0x562d7cdc8db0 2024-05-08T18:05:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cdc8db0 2024-05-08T18:05:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c5d36800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c5d36800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdc8db0 2024-05-08T18:05:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cdc8db0 2024-05-08T18:05:27.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c5d36800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:27.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c5d36800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562d7cdcd7a0 0x562d7cdcfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.352+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd01a0 con 0x562d7cdcd7a0 2024-05-08T18:05:27.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.353+0000 7ff8c1ac5640 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562d7cdcd7a0 0x562d7cdcfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.353+0000 7ff8c1ac5640 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562d7cdcd7a0 0x562d7cdcfb90 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.353+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.353+0000 7ff8c5d36800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:27.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.353+0000 7ff8c5d36800 20 searching for the correct realm 2024-05-08T18:05:27.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.353+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562d7cdd2150 0x562d7cdd4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.353+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd4ce0 con 0x562d7cdd2150 2024-05-08T18:05:27.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.353+0000 7ff8c12c4640 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562d7cdd2150 0x562d7cdd4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.354+0000 7ff8c12c4640 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562d7cdd2150 0x562d7cdd4610 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.354+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8b8009050 con 0x562d7cdd2150 2024-05-08T18:05:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.354+0000 7ff8c12c4640 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff8b8037040 0x7ff8b8036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.354+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd4ce0 con 0x7ff8b8037040 2024-05-08T18:05:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.354+0000 7ff8c22c6640 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff8b8037040 0x7ff8b8036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.354+0000 7ff8c22c6640 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff8b8037040 0x7ff8b8036410 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.355+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x7ff8b8037040 2024-05-08T18:05:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.355+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdcd7a0 2024-05-08T18:05:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.355+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.355+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdc8db0 2024-05-08T18:05:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.355+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cdc8db0 2024-05-08T18:05:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.355+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdc8db0 2024-05-08T18:05:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.355+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cdc8db0 2024-05-08T18:05:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.355+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdcd7a0 2024-05-08T18:05:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.356+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.356+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdcd7a0 2024-05-08T18:05:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.356+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.356+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x7ff8b8037040 2024-05-08T18:05:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.356+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x7ff8b8037040 2024-05-08T18:05:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.356+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdd2150 2024-05-08T18:05:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.356+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8b8009050 con 0x562d7cdd2150 2024-05-08T18:05:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.356+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdc8db0 2024-05-08T18:05:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.357+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cdc8db0 2024-05-08T18:05:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.357+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdcd7a0 2024-05-08T18:05:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.357+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.357+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdc8db0 2024-05-08T18:05:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.357+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cdc8db0 2024-05-08T18:05:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.357+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x7ff8b8037040 2024-05-08T18:05:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.357+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x7ff8b8037040 2024-05-08T18:05:27.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.357+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdd2150 2024-05-08T18:05:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.358+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8b8009050 con 0x562d7cdd2150 2024-05-08T18:05:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.358+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdd2150 2024-05-08T18:05:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.358+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8b8009050 con 0x562d7cdd2150 2024-05-08T18:05:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.358+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdcd7a0 2024-05-08T18:05:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.358+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.358+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdd2150 2024-05-08T18:05:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.358+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8b8009050 con 0x562d7cdd2150 2024-05-08T18:05:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.359+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdd2150 2024-05-08T18:05:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.359+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ff8b8009050 con 0x562d7cdd2150 2024-05-08T18:05:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.359+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdc8db0 2024-05-08T18:05:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.359+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cdc8db0 2024-05-08T18:05:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.359+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdc8db0 2024-05-08T18:05:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.359+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cdc8db0 2024-05-08T18:05:27.369 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.359+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdcd7a0 2024-05-08T18:05:27.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.360+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.360+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdd2150 2024-05-08T18:05:27.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.360+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8b8009050 con 0x562d7cdd2150 2024-05-08T18:05:27.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.360+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x7ff8b8037040 2024-05-08T18:05:27.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.360+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x7ff8b8037040 2024-05-08T18:05:27.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.360+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdc8db0 2024-05-08T18:05:27.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.360+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cdc8db0 2024-05-08T18:05:27.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.360+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdcd7a0 2024-05-08T18:05:27.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.361+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.361+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdd2150 2024-05-08T18:05:27.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.361+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8b8009050 con 0x562d7cdd2150 2024-05-08T18:05:27.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.361+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdcd7a0 2024-05-08T18:05:27.370 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.361+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.361+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x7ff8b8037040 2024-05-08T18:05:27.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.361+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x7ff8b8037040 2024-05-08T18:05:27.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.362+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdd2150 2024-05-08T18:05:27.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.362+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8b8009050 con 0x562d7cdd2150 2024-05-08T18:05:27.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.362+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdcd7a0 2024-05-08T18:05:27.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.362+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.362+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x562d7cdc8db0 2024-05-08T18:05:27.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cdc8db0 2024-05-08T18:05:27.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdcca40 con 0x7ff8b8037040 2024-05-08T18:05:27.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x7ff8b8037040 2024-05-08T18:05:27.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c5d36800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:27.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c5d36800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:27.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c5d36800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:27.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c5d36800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:27.371 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c5d36800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:27.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c5d36800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:27.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c5d36800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd0a60 con 0x562d7cdc8db0 2024-05-08T18:05:27.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cdc8db0 2024-05-08T18:05:27.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c5d36800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c5d36800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd0a60 con 0x562d7cdc8db0 2024-05-08T18:05:27.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x562d7cdc8db0 2024-05-08T18:05:27.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c5d36800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:27.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c5d36800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.363+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd0a60 con 0x562d7cdd2150 2024-05-08T18:05:27.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff8b8009050 con 0x562d7cdd2150 2024-05-08T18:05:27.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:27.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 20 zone default found 2024-05-08T18:05:27.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 4 Realm: () 2024-05-08T18:05:27.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:27.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:27.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:27.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 20 zonegroup default 2024-05-08T18:05:27.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd1390 con 0x562d7cdcd7a0 2024-05-08T18:05:27.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd6ba0 con 0x562d7cdd2150 2024-05-08T18:05:27.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff8b8009050 con 0x562d7cdd2150 2024-05-08T18:05:27.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd70a0 con 0x562d7cdd2150 2024-05-08T18:05:27.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff8b8009050 con 0x562d7cdd2150 2024-05-08T18:05:27.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 20 started sync module instance, tier type = 2024-05-08T18:05:27.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.364+0000 7ff8c5d36800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:27.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.365+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd5fe0 con 0x7ff8b8037040 2024-05-08T18:05:27.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.366+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8bc00a4a0 con 0x7ff8b8037040 2024-05-08T18:05:27.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.366+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94753368401120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd5fe0 con 0x7ff8b8037040 2024-05-08T18:05:27.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.366+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd7b60 con 0x562d7cdc8db0 2024-05-08T18:05:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.367+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8bc040040 con 0x562d7cdc8db0 2024-05-08T18:05:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.367+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94753368401120] v27'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8bc040040 con 0x7ff8b8037040 2024-05-08T18:05:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.367+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94753368405616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd7b30 con 0x562d7cdc8db0 2024-05-08T18:05:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.367+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd5b70 con 0x562d7cdd2150 2024-05-08T18:05:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.368+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94753368405616] v27'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8bc040040 con 0x562d7cdc8db0 2024-05-08T18:05:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.368+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8b8009050 con 0x562d7cdd2150 2024-05-08T18:05:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.368+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94753368407888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd8ad0 con 0x562d7cdd2150 2024-05-08T18:05:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.368+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd9490 con 0x7ff8b8037040 2024-05-08T18:05:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.369+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94753368407888] v27'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8b8009050 con 0x562d7cdd2150 2024-05-08T18:05:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.369+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8bc040040 con 0x7ff8b8037040 2024-05-08T18:05:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.369+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94753368412128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd98b0 con 0x7ff8b8037040 2024-05-08T18:05:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.369+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdda270 con 0x562d7cdc8db0 2024-05-08T18:05:27.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.370+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8bc040040 con 0x562d7cdc8db0 2024-05-08T18:05:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.370+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94753368415680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdda690 con 0x562d7cdc8db0 2024-05-08T18:05:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.370+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cddb110 con 0x562d7cdcd7a0 2024-05-08T18:05:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.370+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94753368412128] v27'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8bc040040 con 0x7ff8b8037040 2024-05-08T18:05:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.371+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.371+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94753368419328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cddb500 con 0x562d7cdcd7a0 2024-05-08T18:05:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.371+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cddbec0 con 0x562d7cdcd7a0 2024-05-08T18:05:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.371+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94753368415680] v27'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8bc040040 con 0x562d7cdc8db0 2024-05-08T18:05:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.372+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94753368419328] v27'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.372+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.372+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94753368422928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cddc2b0 con 0x562d7cdcd7a0 2024-05-08T18:05:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.372+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cddcb00 con 0x562d7cdcd7a0 2024-05-08T18:05:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.373+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94753368422928] v27'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.373+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.373+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94753368425136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cddcb80 con 0x562d7cdcd7a0 2024-05-08T18:05:27.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.373+0000 7ff8c5d36800 20 add_watcher() i=0 2024-05-08T18:05:27.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.373+0000 7ff8c5d36800 20 add_watcher() i=1 2024-05-08T18:05:27.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.373+0000 7ff8c5d36800 20 add_watcher() i=2 2024-05-08T18:05:27.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.373+0000 7ff8c5d36800 20 add_watcher() i=3 2024-05-08T18:05:27.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.373+0000 7ff8c5d36800 20 add_watcher() i=4 2024-05-08T18:05:27.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.373+0000 7ff8c5d36800 20 add_watcher() i=5 2024-05-08T18:05:27.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.373+0000 7ff8c5d36800 20 add_watcher() i=6 2024-05-08T18:05:27.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.374+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94753368425136] v27'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.374+0000 7ff8c5d36800 20 add_watcher() i=7 2024-05-08T18:05:27.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.374+0000 7ff8c5d36800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:27.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.374+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7cddfe80 con 0x562d7cdc8db0 2024-05-08T18:05:27.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.375+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff8bc040040 con 0x562d7cdc8db0 2024-05-08T18:05:27.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.375+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94753368426560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cddd110 con 0x562d7cdc8db0 2024-05-08T18:05:27.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.376+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94753368426560] v27'187 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff8bc040040 con 0x562d7cdc8db0 2024-05-08T18:05:27.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.376+0000 7ff7737fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:27.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.376+0000 7ff8c5d36800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:27.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.376+0000 7ff8c5d36800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:27.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.376+0000 7ff8c5d36800 5 note: GC not initialized 2024-05-08T18:05:27.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.376+0000 7ff7727fc640 20 reqs_thread_entry: start 2024-05-08T18:05:27.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.376+0000 7ff8c5d36800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:27.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.376+0000 7ff8c5d36800 20 Filter name: none 2024-05-08T18:05:27.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.376+0000 7ff6e7fff640 20 reqs_thread_entry: start 2024-05-08T18:05:27.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.376+0000 7ff8c5d36800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:27.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.376+0000 7ff8c5d36800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.376+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7ce133f0 con 0x562d7cdc8db0 2024-05-08T18:05:27.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.377+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv13 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7ff8bc040040 con 0x562d7cdc8db0 2024-05-08T18:05:27.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.377+0000 7ff8c5d36800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:27.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.377+0000 7ff8c5d36800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:27.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.377+0000 7ff8c5d36800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:27.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.377+0000 7ff8c5d36800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:27.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.377+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~403 in=403b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7ce19660 con 0x562d7cdc8db0 2024-05-08T18:05:27.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.378+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~403] v27'14 uv14 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7ff8bc040040 con 0x562d7cdc8db0 2024-05-08T18:05:27.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.378+0000 7ff8c5d36800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:27.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.378+0000 7ff8c5d36800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:27.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.378+0000 7ff8c5d36800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-05-08T18:05:27.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.378+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94753368674336 in=591b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562d7ce14e70 con 0x562d7cdc8db0 2024-05-08T18:05:27.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.378+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== watch-notify(notify (1) cookie 94753368415680 notify 115964117014 ret 0) ==== 621+0+0 (crc 0 0 0) 0x7ff8bc038450 con 0x562d7cdc8db0 2024-05-08T18:05:27.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.378+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(64 notify.4 [notify cookie 94753368674336 out=8b] v0'0 uv180 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff8bc040040 con 0x562d7cdc8db0 2024-05-08T18:05:27.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.378+0000 7ff7a57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117014 cookie 94753368415680 notifier 4648 bl.length()=579 2024-05-08T18:05:27.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.378+0000 7ff7a57fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:27.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.378+0000 7ff7a57fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:27.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.378+0000 7ff7a57fa640 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7ff784003ea0 con 0x562d7cdc8db0 2024-05-08T18:05:27.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.379+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== watch-notify(notify_complete (2) cookie 94753368674336 notify 115964117014 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7ff8bc038620 con 0x562d7cdc8db0 2024-05-08T18:05:27.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.379+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8bc040040 con 0x562d7cdc8db0 2024-05-08T18:05:27.385 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[],"caps":[{"type":"user","perm":"read"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:27.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.380+0000 7ff7737fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:27.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.380+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94753368426560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cddf550 con 0x562d7cdc8db0 2024-05-08T18:05:27.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.381+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94753368426560] v27'188 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff8bc040040 con 0x562d7cdc8db0 2024-05-08T18:05:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.381+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94753368401120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cddc840 con 0x7ff8b8037040 2024-05-08T18:05:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.382+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94753368401120] v27'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8bc040040 con 0x7ff8b8037040 2024-05-08T18:05:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.382+0000 7ff8c5d36800 20 remove_watcher() i=0 2024-05-08T18:05:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.382+0000 7ff8c5d36800 2 removed watcher, disabling cache 2024-05-08T18:05:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.382+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94753368405616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cddc840 con 0x562d7cdc8db0 2024-05-08T18:05:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.383+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94753368405616] v27'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8bc040040 con 0x562d7cdc8db0 2024-05-08T18:05:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.383+0000 7ff8c5d36800 20 remove_watcher() i=1 2024-05-08T18:05:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.383+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94753368407888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cddc840 con 0x562d7cdd2150 2024-05-08T18:05:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.384+0000 7ff8c12c4640 1 -- 172.21.15.50:0/4192247129 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94753368407888] v27'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8b8009050 con 0x562d7cdd2150 2024-05-08T18:05:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.384+0000 7ff8c5d36800 20 remove_watcher() i=2 2024-05-08T18:05:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.384+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94753368412128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd64e0 con 0x7ff8b8037040 2024-05-08T18:05:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.385+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94753368412128] v27'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8bc040040 con 0x7ff8b8037040 2024-05-08T18:05:27.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.385+0000 7ff8c5d36800 20 remove_watcher() i=3 2024-05-08T18:05:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.385+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94753368415680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd64e0 con 0x562d7cdc8db0 2024-05-08T18:05:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.386+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94753368415680] v27'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8bc040040 con 0x562d7cdc8db0 2024-05-08T18:05:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.386+0000 7ff8c5d36800 20 remove_watcher() i=4 2024-05-08T18:05:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.386+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94753368419328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd9dc0 con 0x562d7cdcd7a0 2024-05-08T18:05:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.387+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94753368419328] v27'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.387+0000 7ff8c5d36800 20 remove_watcher() i=5 2024-05-08T18:05:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.387+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94753368422928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cdd64e0 con 0x562d7cdcd7a0 2024-05-08T18:05:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.388+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94753368422928] v27'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.388+0000 7ff8c5d36800 20 remove_watcher() i=6 2024-05-08T18:05:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.388+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94753368425136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562d7cddac00 con 0x562d7cdcd7a0 2024-05-08T18:05:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c1ac5640 1 -- 172.21.15.50:0/4192247129 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94753368425136] v27'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff8b40a0070 con 0x562d7cdcd7a0 2024-05-08T18:05:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 20 remove_watcher() i=7 2024-05-08T18:05:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562d7cdcd7a0 msgr2=0x562d7cdcfb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562d7cdcd7a0 0x562d7cdcfb90 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562d7cdc8db0 msgr2=0x562d7cdcb270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562d7cdc8db0 0x562d7cdcb270 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff8b8037040 msgr2=0x7ff8b8036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff8b8037040 0x7ff8b8036410 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562d7cdd2150 msgr2=0x562d7cdd4610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562d7cdd2150 0x562d7cdd4610 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff78c03bba0 msgr2=0x7ff78c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff78c03bba0 0x7ff78c03e060 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800efd0 tx=0x7ff8b8010000 comp rx=0 tx=0).stop 2024-05-08T18:05:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cdb6210 msgr2=0x562d7cdbd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c22c6640 1 -- 172.21.15.50:0/4192247129 reap_dead start 2024-05-08T18:05:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cdb6210 0x562d7cdbd5b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4070f90 tx=0x7ff8b4091730 comp rx=0 tx=0).stop 2024-05-08T18:05:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 shutdown_connections 2024-05-08T18:05:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/4192247129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cdb6210 0x562d7cdbd5b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 >> 172.21.15.50:0/4192247129 conn(0x562d7cdb2520 msgr2=0x562d7cdafb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 shutdown_connections 2024-05-08T18:05:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.389+0000 7ff8c5d36800 1 -- 172.21.15.50:0/4192247129 wait complete. 2024-05-08T18:05:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 -- 172.21.15.50:0/3948618553 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff79c03bba0 msgr2=0x7ff79c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/3948618553 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff79c03bba0 0x7ff79c03e060 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff8b80016b0 tx=0x7ff8b800f000 comp rx=0 tx=0).stop 2024-05-08T18:05:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 -- 172.21.15.50:0/3948618553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cda90f0 msgr2=0x562d7cda9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/3948618553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cda90f0 0x562d7cda9510 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4092a30 tx=0x7ff8b405b5a0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 -- 172.21.15.50:0/3948618553 shutdown_connections 2024-05-08T18:05:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/3948618553 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff79c03bba0 0x7ff79c03e060 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/3948618553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cda90f0 0x562d7cda9510 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 -- 172.21.15.50:0/3948618553 >> 172.21.15.50:0/3948618553 conn(0x562d7cd9d190 msgr2=0x562d7cd9d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 -- 172.21.15.50:0/3948618553 shutdown_connections 2024-05-08T18:05:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 -- 172.21.15.50:0/3948618553 wait complete. 2024-05-08T18:05:27.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1015357448 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562d7cd94220 msgr2=0x562d7cd966b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562d7cd94220 0x562d7cd966b0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1015357448 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562d7cc908d0 msgr2=0x562d7cd91ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562d7cc908d0 0x562d7cd91ac0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1015357448 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562d7cd982a0 msgr2=0x562d7cd9a690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562d7cd982a0 0x562d7cd9a690 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1015357448 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff7a803bb50 msgr2=0x7ff7a803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff7a803bb50 0x7ff7a803e010 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8001d70 tx=0x7ff8b80012d0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1015357448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 msgr2=0x562d7cd8a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 --2- 172.21.15.50:0/1015357448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562d7cc8f710 0x562d7cd8a880 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff8b405f040 tx=0x7ff8b4002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c22c6640 1 -- 172.21.15.50:0/1015357448 reap_dead start 2024-05-08T18:05:27.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1015357448 shutdown_connections 2024-05-08T18:05:27.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1015357448 >> 172.21.15.50:0/1015357448 conn(0x562d7cc8ac90 msgr2=0x562d7cc03ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1015357448 shutdown_connections 2024-05-08T18:05:27.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.390+0000 7ff8c5d36800 1 -- 172.21.15.50:0/1015357448 wait complete. 2024-05-08T18:05:27.397 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [], 'caps': [{'type': 'user', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:27.398 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2024-05-08T18:05:27.398 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2024-05-08T18:05:27.398 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid foo --caps user=read 2024-05-08T18:05:27.445 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:27.446 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:27.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.448+0000 7fba92f2e800 1 Processor -- start 2024-05-08T18:05:27.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.448+0000 7fba92f2e800 1 -- start start 2024-05-08T18:05:27.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.449+0000 7fba92f2e800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5624497c4960 0x5624497c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.449+0000 7fba92f2e800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5624497c53b0 con 0x5624497c7950 2024-05-08T18:05:27.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.449+0000 7fba92f2e800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x562449666e20 con 0x5624497c4960 2024-05-08T18:05:27.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.449+0000 7fba8ec84640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5624497c4960 0x5624497c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.449+0000 7fba86483640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5624497c7950 0x5624497c7e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:60414/0 (socket says 172.21.15.50:60414) 2024-05-08T18:05:27.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.449+0000 7fba86483640 1 -- 172.21.15.50:0/183632541 learned_addr learned my addr 172.21.15.50:0/183632541 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:27.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.449+0000 7fba8ec84640 1 -- 172.21.15.50:0/183632541 >> v1:172.21.15.46:6789/0 conn(0x5624497c7950 legacy=0x5624497c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.449+0000 7fba8ec84640 1 -- 172.21.15.50:0/183632541 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624497c8510 con 0x5624497c4960 2024-05-08T18:05:27.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.451+0000 7fba8ec84640 1 --2- 172.21.15.50:0/183632541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 0x5624497c4d80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fba8000a470 tx=0x7fba8001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2664921c92da16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.451+0000 7fba8e483640 1 -- 172.21.15.50:0/183632541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba80057070 con 0x5624497c4960 2024-05-08T18:05:27.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.451+0000 7fba8e483640 1 -- 172.21.15.50:0/183632541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8004adc0 con 0x5624497c4960 2024-05-08T18:05:27.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.451+0000 7fba8e483640 1 -- 172.21.15.50:0/183632541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba80052590 con 0x5624497c4960 2024-05-08T18:05:27.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.452+0000 7fba92f2e800 1 -- 172.21.15.50:0/183632541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 msgr2=0x5624497c4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.452+0000 7fba92f2e800 1 --2- 172.21.15.50:0/183632541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 0x5624497c4d80 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fba8000a470 tx=0x7fba8001ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:27.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.452+0000 7fba92f2e800 1 -- 172.21.15.50:0/183632541 shutdown_connections 2024-05-08T18:05:27.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.452+0000 7fba92f2e800 1 --2- 172.21.15.50:0/183632541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 0x5624497c4d80 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.452+0000 7fba92f2e800 1 -- 172.21.15.50:0/183632541 >> 172.21.15.50:0/183632541 conn(0x5624497c02f0 msgr2=0x5624497c2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.452+0000 7fba92f2e800 1 -- 172.21.15.50:0/183632541 shutdown_connections 2024-05-08T18:05:27.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.452+0000 7fba92f2e800 1 -- 172.21.15.50:0/183632541 wait complete. 2024-05-08T18:05:27.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.467+0000 7fba92f2e800 1 Processor -- start 2024-05-08T18:05:27.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.467+0000 7fba92f2e800 1 -- start start 2024-05-08T18:05:27.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.467+0000 7fba92f2e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 0x5624498b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.468+0000 7fba92f2e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562449711a30 con 0x5624497c4960 2024-05-08T18:05:27.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.468+0000 7fba8ec84640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 0x5624498b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.468+0000 7fba8ec84640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 0x5624498b7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:44060/0 (socket says 172.21.15.50:44060) 2024-05-08T18:05:27.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.468+0000 7fba8ec84640 1 -- 172.21.15.50:0/1041375944 learned_addr learned my addr 172.21.15.50:0/1041375944 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.468+0000 7fba8ec84640 1 -- 172.21.15.50:0/1041375944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624498b81e0 con 0x5624497c4960 2024-05-08T18:05:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.468+0000 7fba8ec84640 1 --2- 172.21.15.50:0/1041375944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 0x5624498b7c10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fba80055040 tx=0x7fba8004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad0535e9ae48347 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.468+0000 7fb97ce09640 1 -- 172.21.15.50:0/1041375944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba80057040 con 0x5624497c4960 2024-05-08T18:05:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.468+0000 7fb97ce09640 1 -- 172.21.15.50:0/1041375944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba80062070 con 0x5624497c4960 2024-05-08T18:05:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.468+0000 7fb97ce09640 1 -- 172.21.15.50:0/1041375944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba8005d470 con 0x5624497c4960 2024-05-08T18:05:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.469+0000 7fba92f2e800 1 -- 172.21.15.50:0/1041375944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 msgr2=0x5624498b7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.469+0000 7fba92f2e800 1 --2- 172.21.15.50:0/1041375944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 0x5624498b7c10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fba80055040 tx=0x7fba8004c000 comp rx=0 tx=0).stop 2024-05-08T18:05:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.469+0000 7fba92f2e800 1 -- 172.21.15.50:0/1041375944 shutdown_connections 2024-05-08T18:05:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.469+0000 7fba92f2e800 1 --2- 172.21.15.50:0/1041375944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 0x5624498b7c10 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.469+0000 7fba92f2e800 1 -- 172.21.15.50:0/1041375944 >> 172.21.15.50:0/1041375944 conn(0x5624497c02f0 msgr2=0x5624498b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.469+0000 7fba92f2e800 1 -- 172.21.15.50:0/1041375944 shutdown_connections 2024-05-08T18:05:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.469+0000 7fba92f2e800 1 -- 172.21.15.50:0/1041375944 wait complete. 2024-05-08T18:05:27.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.469+0000 7fba92f2e800 1 Processor -- start 2024-05-08T18:05:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.470+0000 7fba92f2e800 1 -- start start 2024-05-08T18:05:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.470+0000 7fba92f2e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 0x5624498bf620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.470+0000 7fba92f2e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5624497cfe20 con 0x5624497c4960 2024-05-08T18:05:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.470+0000 7fba8ec84640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 0x5624498bf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.470+0000 7fba8ec84640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 0x5624498bf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:44072/0 (socket says 172.21.15.50:44072) 2024-05-08T18:05:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.470+0000 7fba8ec84640 1 -- 172.21.15.50:0/1963337501 learned_addr learned my addr 172.21.15.50:0/1963337501 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.470+0000 7fba8ec84640 1 -- 172.21.15.50:0/1963337501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624498bfb60 con 0x5624497c4960 2024-05-08T18:05:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.470+0000 7fba8ec84640 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 0x5624498bf620 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fba800057d0 tx=0x7fba80002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.471+0000 7fb97b606640 1 -- 172.21.15.50:0/1963337501 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba8006b070 con 0x5624497c4960 2024-05-08T18:05:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.471+0000 7fb97b606640 1 -- 172.21.15.50:0/1963337501 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8005d9a0 con 0x5624497c4960 2024-05-08T18:05:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.471+0000 7fba92f2e800 1 -- 172.21.15.50:0/1963337501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5624498bfdf0 con 0x5624497c4960 2024-05-08T18:05:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.471+0000 7fb97b606640 1 -- 172.21.15.50:0/1963337501 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba80057070 con 0x5624497c4960 2024-05-08T18:05:27.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.471+0000 7fba92f2e800 1 -- 172.21.15.50:0/1963337501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624498c2aa0 con 0x5624497c4960 2024-05-08T18:05:27.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.471+0000 7fb97b606640 1 -- 172.21.15.50:0/1963337501 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fba8005dc80 con 0x5624497c4960 2024-05-08T18:05:27.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.472+0000 7fb97b606640 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb97403bba0 0x7fb97403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.472+0000 7fb97b606640 1 -- 172.21.15.50:0/1963337501 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fba80094620 con 0x5624497c4960 2024-05-08T18:05:27.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.472+0000 7fba86483640 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb97403bba0 0x7fb97403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.472+0000 7fba92f2e800 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5624497c7950 0x5624498c6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.472+0000 7fba92f2e800 1 -- 172.21.15.50:0/1963337501 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5624498c6da0 con 0x5624497c7950 2024-05-08T18:05:27.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.472+0000 7fba8f485640 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5624497c7950 0x5624498c6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.472+0000 7fba86483640 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb97403bba0 0x7fb97403e060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fba78001d70 tx=0x7fba780012d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.472+0000 7fba8f485640 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5624497c7950 0x5624498c6860 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.473+0000 7fba8f485640 1 -- 172.21.15.50:0/1963337501 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624497c7950 2024-05-08T18:05:27.474 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.473+0000 7fba92f2e800 1 -- 172.21.15.50:0/1963337501 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5624498c7670 con 0x5624497c7950 2024-05-08T18:05:27.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.473+0000 7fba8f485640 1 -- 172.21.15.50:0/1963337501 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624497c7950 2024-05-08T18:05:27.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.473+0000 7fba92f2e800 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5624498c8fc0 0x5624498cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.473+0000 7fba92f2e800 1 -- 172.21.15.50:0/1963337501 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5624498cbbe0 con 0x5624498c8fc0 2024-05-08T18:05:27.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.473+0000 7fba8ec84640 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5624498c8fc0 0x5624498cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.474+0000 7fba8ec84640 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5624498c8fc0 0x5624498cb450 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.474+0000 7fba8ec84640 1 -- 172.21.15.50:0/1963337501 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fba8005a040 con 0x5624498c8fc0 2024-05-08T18:05:27.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.474+0000 7fba92f2e800 1 -- 172.21.15.50:0/1963337501 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5624498c7ec0 con 0x5624497c7950 2024-05-08T18:05:27.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.474+0000 7fba8f485640 1 -- 172.21.15.50:0/1963337501 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624497c7950 2024-05-08T18:05:27.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.474+0000 7fba92f2e800 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5624498cd040 0x5624498cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.474+0000 7fba92f2e800 1 -- 172.21.15.50:0/1963337501 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5624498cfb20 con 0x5624498cd040 2024-05-08T18:05:27.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.475+0000 7fba86483640 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5624498cd040 0x5624498cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.475+0000 7fba86483640 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5624498cd040 0x5624498cf430 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.475+0000 7fba86483640 1 -- 172.21.15.50:0/1963337501 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fba78009050 con 0x5624498cd040 2024-05-08T18:05:27.475 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.476+0000 7fba92f2e800 1 Processor -- start 2024-05-08T18:05:27.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.476+0000 7fba92f2e800 1 -- start start 2024-05-08T18:05:27.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.476+0000 7fba92f2e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c74f0 0x5624497c5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.476+0000 7fba92f2e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5624496b8330 con 0x5624497c74f0 2024-05-08T18:05:27.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.476+0000 7fba8ec84640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c74f0 0x5624497c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.476+0000 7fba8ec84640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c74f0 0x5624497c5960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:44086/0 (socket says 172.21.15.50:44086) 2024-05-08T18:05:27.476 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.476+0000 7fba8ec84640 1 -- 172.21.15.50:0/2716762354 learned_addr learned my addr 172.21.15.50:0/2716762354 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.476+0000 7fba8ec84640 1 -- 172.21.15.50:0/2716762354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624497c5ea0 con 0x5624497c74f0 2024-05-08T18:05:27.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.477+0000 7fba8ec84640 1 --2- 172.21.15.50:0/2716762354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c74f0 0x5624497c5960 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fba8005f040 tx=0x7fba8005bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.477+0000 7fb978e01640 1 -- 172.21.15.50:0/2716762354 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba800783a0 con 0x5624497c74f0 2024-05-08T18:05:27.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.477+0000 7fb978e01640 1 -- 172.21.15.50:0/2716762354 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba80094050 con 0x5624497c74f0 2024-05-08T18:05:27.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.477+0000 7fba92f2e800 1 -- 172.21.15.50:0/2716762354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5624497c60d0 con 0x5624497c74f0 2024-05-08T18:05:27.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.477+0000 7fb978e01640 1 -- 172.21.15.50:0/2716762354 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba80078540 con 0x5624497c74f0 2024-05-08T18:05:27.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.477+0000 7fba92f2e800 1 -- 172.21.15.50:0/2716762354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624497c65d0 con 0x5624497c74f0 2024-05-08T18:05:27.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.477+0000 7fb978e01640 1 -- 172.21.15.50:0/2716762354 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fba80088060 con 0x5624497c74f0 2024-05-08T18:05:27.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.478+0000 7fb978e01640 1 --2- 172.21.15.50:0/2716762354 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb96803bba0 0x7fb96803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.478+0000 7fb978e01640 1 -- 172.21.15.50:0/2716762354 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fba80082c70 con 0x5624497c74f0 2024-05-08T18:05:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.478+0000 7fba86483640 1 --2- 172.21.15.50:0/2716762354 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb96803bba0 0x7fb96803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.478+0000 7fba92f2e800 1 Processor -- start 2024-05-08T18:05:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.478+0000 7fba92f2e800 1 -- start start 2024-05-08T18:05:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.478+0000 7fba86483640 1 --2- 172.21.15.50:0/2716762354 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb96803bba0 0x7fb96803e060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fba780016b0 tx=0x7fba7800f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.478+0000 7fba92f2e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624498ed1a0 0x5624498ed5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.478+0000 7fba92f2e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5624498b84a0 con 0x5624498ed1a0 2024-05-08T18:05:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.479+0000 7fba8ec84640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624498ed1a0 0x5624498ed5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.479+0000 7fba8ec84640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624498ed1a0 0x5624498ed5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:44100/0 (socket says 172.21.15.50:44100) 2024-05-08T18:05:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.479+0000 7fba8ec84640 1 -- 172.21.15.50:0/116937269 learned_addr learned my addr 172.21.15.50:0/116937269 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.479+0000 7fba8ec84640 1 -- 172.21.15.50:0/116937269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624498edb90 con 0x5624498ed1a0 2024-05-08T18:05:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.479+0000 7fba8ec84640 1 --2- 172.21.15.50:0/116937269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624498ed1a0 0x5624498ed5c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fba80072040 tx=0x7fba80083440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b3a0bc512d5450 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.479+0000 7fb9727fc640 1 -- 172.21.15.50:0/116937269 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba800a3070 con 0x5624498ed1a0 2024-05-08T18:05:27.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.479+0000 7fb9727fc640 1 -- 172.21.15.50:0/116937269 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba8005c950 con 0x5624498ed1a0 2024-05-08T18:05:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.479+0000 7fb9727fc640 1 -- 172.21.15.50:0/116937269 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba80091530 con 0x5624498ed1a0 2024-05-08T18:05:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.480+0000 7fba92f2e800 1 -- 172.21.15.50:0/116937269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624498ed1a0 msgr2=0x5624498ed5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.480+0000 7fba92f2e800 1 --2- 172.21.15.50:0/116937269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624498ed1a0 0x5624498ed5c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fba80072040 tx=0x7fba80083440 comp rx=0 tx=0).stop 2024-05-08T18:05:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.480+0000 7fba92f2e800 1 -- 172.21.15.50:0/116937269 shutdown_connections 2024-05-08T18:05:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.480+0000 7fba92f2e800 1 --2- 172.21.15.50:0/116937269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624498ed1a0 0x5624498ed5c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.480+0000 7fba92f2e800 1 -- 172.21.15.50:0/116937269 >> 172.21.15.50:0/116937269 conn(0x5624498e94b0 msgr2=0x5624498e7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.480+0000 7fba92f2e800 1 -- 172.21.15.50:0/116937269 shutdown_connections 2024-05-08T18:05:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.480+0000 7fba92f2e800 1 -- 172.21.15.50:0/116937269 wait complete. 2024-05-08T18:05:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.480+0000 7fba92f2e800 1 Processor -- start 2024-05-08T18:05:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.480+0000 7fba92f2e800 1 -- start start 2024-05-08T18:05:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.480+0000 7fba92f2e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624498ed1a0 0x5624498f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.480+0000 7fba92f2e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5624498ee220 con 0x5624498ed1a0 2024-05-08T18:05:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.480+0000 7fba8ec84640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624498ed1a0 0x5624498f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.480+0000 7fba8ec84640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624498ed1a0 0x5624498f45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:44116/0 (socket says 172.21.15.50:44116) 2024-05-08T18:05:27.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.480+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 learned_addr learned my addr 172.21.15.50:0/2284379180 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.481+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624498f4af0 con 0x5624498ed1a0 2024-05-08T18:05:27.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.481+0000 7fba8ec84640 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624498ed1a0 0x5624498f45b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fba8000a290 tx=0x7fba80090ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.481+0000 7fb970ff9640 1 -- 172.21.15.50:0/2284379180 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba800ae020 con 0x5624498ed1a0 2024-05-08T18:05:27.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.481+0000 7fb970ff9640 1 -- 172.21.15.50:0/2284379180 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba80000ed0 con 0x5624498ed1a0 2024-05-08T18:05:27.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.481+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5624498f4af0 con 0x5624498ed1a0 2024-05-08T18:05:27.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.481+0000 7fb970ff9640 1 -- 172.21.15.50:0/2284379180 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba800a3030 con 0x5624498ed1a0 2024-05-08T18:05:27.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.481+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624498f4af0 con 0x5624498ed1a0 2024-05-08T18:05:27.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.482+0000 7fb970ff9640 1 -- 172.21.15.50:0/2284379180 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fba800b1020 con 0x5624498ed1a0 2024-05-08T18:05:27.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.482+0000 7fb970ff9640 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb95803bba0 0x7fb95803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.482+0000 7fb970ff9640 1 -- 172.21.15.50:0/2284379180 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7fba800a4070 con 0x5624498ed1a0 2024-05-08T18:05:27.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.482+0000 7fba86483640 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb95803bba0 0x7fb95803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.483+0000 7fba86483640 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb95803bba0 0x7fb95803e060 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fba7800efd0 tx=0x7fba78010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.483+0000 7fba92f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.483+0000 7fba92f2e800 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5624498ffd00 0x5624499021c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.483+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562449902910 con 0x5624498ffd00 2024-05-08T18:05:27.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.483+0000 7fba8f485640 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5624498ffd00 0x5624499021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.483+0000 7fba8f485640 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5624498ffd00 0x5624499021c0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.483+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624498ffd00 2024-05-08T18:05:27.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.483+0000 7fba92f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.483+0000 7fba92f2e800 20 realm 2024-05-08T18:05:27.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.483+0000 7fba92f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.483+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562449902d20 con 0x5624498ffd00 2024-05-08T18:05:27.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.484+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624498ffd00 2024-05-08T18:05:27.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.484+0000 7fba92f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.484+0000 7fba92f2e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:27.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.484+0000 7fba92f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.484+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5624499035f0 con 0x5624498ffd00 2024-05-08T18:05:27.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.484+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624498ffd00 2024-05-08T18:05:27.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.484+0000 7fba92f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.484+0000 7fba92f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.484+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624498ffd00 2024-05-08T18:05:27.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.484+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624498ffd00 2024-05-08T18:05:27.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.484+0000 7fba92f2e800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:27.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.484+0000 7fba92f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.484+0000 7fba92f2e800 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5624499046f0 0x562449906ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.484+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5624499070f0 con 0x5624499046f0 2024-05-08T18:05:27.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.484+0000 7fba8ec84640 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5624499046f0 0x562449906ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.485+0000 7fba8ec84640 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5624499046f0 0x562449906ae0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.485+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.485+0000 7fba92f2e800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:27.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.485+0000 7fba92f2e800 20 searching for the correct realm 2024-05-08T18:05:27.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.485+0000 7fba92f2e800 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562449909080 0x56244990b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.485+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56244990bc10 con 0x562449909080 2024-05-08T18:05:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.485+0000 7fba86483640 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562449909080 0x56244990b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.485+0000 7fba86483640 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562449909080 0x56244990b540 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.486+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba78009050 con 0x562449909080 2024-05-08T18:05:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.486+0000 7fba86483640 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fba78037040 0x7fba78036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.486+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56244990bc10 con 0x7fba78037040 2024-05-08T18:05:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.486+0000 7fba8f485640 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fba78037040 0x7fba78036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.486+0000 7fba8f485640 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fba78037040 0x7fba78036410 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.486+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x7fba78037040 2024-05-08T18:05:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.486+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624499046f0 2024-05-08T18:05:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.487+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.487+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624498ffd00 2024-05-08T18:05:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.487+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624498ffd00 2024-05-08T18:05:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.487+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624498ffd00 2024-05-08T18:05:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.487+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624498ffd00 2024-05-08T18:05:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.487+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624499046f0 2024-05-08T18:05:27.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.488+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.488+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624499046f0 2024-05-08T18:05:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.488+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.488+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x7fba78037040 2024-05-08T18:05:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.488+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x7fba78037040 2024-05-08T18:05:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.488+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x562449909080 2024-05-08T18:05:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.488+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba78009050 con 0x562449909080 2024-05-08T18:05:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.488+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624498ffd00 2024-05-08T18:05:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.489+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624498ffd00 2024-05-08T18:05:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.489+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624499046f0 2024-05-08T18:05:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.489+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.489+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624498ffd00 2024-05-08T18:05:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.489+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624498ffd00 2024-05-08T18:05:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.489+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x7fba78037040 2024-05-08T18:05:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.489+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fba8800a4a0 con 0x7fba78037040 2024-05-08T18:05:27.501 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.489+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x562449909080 2024-05-08T18:05:27.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.490+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba78009050 con 0x562449909080 2024-05-08T18:05:27.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.490+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x562449909080 2024-05-08T18:05:27.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.490+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba78009050 con 0x562449909080 2024-05-08T18:05:27.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.490+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624499046f0 2024-05-08T18:05:27.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.490+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.490+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x562449909080 2024-05-08T18:05:27.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.490+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba78009050 con 0x562449909080 2024-05-08T18:05:27.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.490+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x562449909080 2024-05-08T18:05:27.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.491+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fba78009050 con 0x562449909080 2024-05-08T18:05:27.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.491+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624498ffd00 2024-05-08T18:05:27.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.491+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624498ffd00 2024-05-08T18:05:27.502 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.491+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624498ffd00 2024-05-08T18:05:27.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.491+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624498ffd00 2024-05-08T18:05:27.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.491+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624499046f0 2024-05-08T18:05:27.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.492+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.492+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x562449909080 2024-05-08T18:05:27.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.492+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba78009050 con 0x562449909080 2024-05-08T18:05:27.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.492+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x7fba78037040 2024-05-08T18:05:27.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.492+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x7fba78037040 2024-05-08T18:05:27.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.492+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624498ffd00 2024-05-08T18:05:27.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.492+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624498ffd00 2024-05-08T18:05:27.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.492+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624499046f0 2024-05-08T18:05:27.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.493+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.493+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x562449909080 2024-05-08T18:05:27.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.493+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba78009050 con 0x562449909080 2024-05-08T18:05:27.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.493+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624499046f0 2024-05-08T18:05:27.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.493+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.503 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.493+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x7fba78037040 2024-05-08T18:05:27.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.494+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x7fba78037040 2024-05-08T18:05:27.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.494+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x562449909080 2024-05-08T18:05:27.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.494+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba78009050 con 0x562449909080 2024-05-08T18:05:27.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.494+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624499046f0 2024-05-08T18:05:27.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.494+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.494+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x5624498ffd00 2024-05-08T18:05:27.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.494+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624498ffd00 2024-05-08T18:05:27.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.494+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562449903990 con 0x7fba78037040 2024-05-08T18:05:27.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x7fba78037040 2024-05-08T18:05:27.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:27.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:27.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:27.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:27.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:27.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:27.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5624499079b0 con 0x5624498ffd00 2024-05-08T18:05:27.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624498ffd00 2024-05-08T18:05:27.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5624499079b0 con 0x5624498ffd00 2024-05-08T18:05:27.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fba8800a4a0 con 0x5624498ffd00 2024-05-08T18:05:27.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:27.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5624499079b0 con 0x562449909080 2024-05-08T18:05:27.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fba78009050 con 0x562449909080 2024-05-08T18:05:27.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:27.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 20 zone default found 2024-05-08T18:05:27.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 4 Realm: () 2024-05-08T18:05:27.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:27.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:27.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:27.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 20 zonegroup default 2024-05-08T18:05:27.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.495+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562449908360 con 0x5624499046f0 2024-05-08T18:05:27.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.496+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.496+0000 7fba92f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.496+0000 7fba92f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.496+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562449907ec0 con 0x562449909080 2024-05-08T18:05:27.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.496+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fba78009050 con 0x562449909080 2024-05-08T18:05:27.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.496+0000 7fba92f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.496+0000 7fba92f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.496+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56244990dfe0 con 0x562449909080 2024-05-08T18:05:27.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.496+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fba78009050 con 0x562449909080 2024-05-08T18:05:27.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.496+0000 7fba92f2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.496+0000 7fba92f2e800 20 started sync module instance, tier type = 2024-05-08T18:05:27.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.496+0000 7fba92f2e800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:27.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.496+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56244990d3b0 con 0x7fba78037040 2024-05-08T18:05:27.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.497+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba8800a4a0 con 0x7fba78037040 2024-05-08T18:05:27.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.497+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94713853040816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56244990d3b0 con 0x7fba78037040 2024-05-08T18:05:27.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.497+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56244990eb10 con 0x5624498ffd00 2024-05-08T18:05:27.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.498+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94713853040816] v27'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88040040 con 0x7fba78037040 2024-05-08T18:05:27.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.498+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88040040 con 0x5624498ffd00 2024-05-08T18:05:27.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.498+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94713853044224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56244990d3b0 con 0x5624498ffd00 2024-05-08T18:05:27.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.498+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56244990f590 con 0x562449909080 2024-05-08T18:05:27.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.499+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba78009050 con 0x562449909080 2024-05-08T18:05:27.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.499+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94713853046944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56244990fe60 con 0x562449909080 2024-05-08T18:05:27.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.499+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562449910930 con 0x7fba78037040 2024-05-08T18:05:27.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.500+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94713853044224] v27'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88040040 con 0x5624498ffd00 2024-05-08T18:05:27.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.500+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94713853046944] v27'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba78009050 con 0x562449909080 2024-05-08T18:05:27.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.501+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88040040 con 0x7fba78037040 2024-05-08T18:05:27.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.501+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94713853049264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56244990cf00 con 0x7fba78037040 2024-05-08T18:05:27.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.501+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562449911200 con 0x5624498ffd00 2024-05-08T18:05:27.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.502+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88040040 con 0x5624498ffd00 2024-05-08T18:05:27.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.502+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94713853054208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562449911120 con 0x5624498ffd00 2024-05-08T18:05:27.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.502+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94713853049264] v27'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88040040 con 0x7fba78037040 2024-05-08T18:05:27.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.502+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562449911f50 con 0x5624499046f0 2024-05-08T18:05:27.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.503+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94713853054208] v27'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88040040 con 0x5624498ffd00 2024-05-08T18:05:27.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.503+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.503+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94713853056496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562449911120 con 0x5624499046f0 2024-05-08T18:05:27.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.503+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562449912950 con 0x5624499046f0 2024-05-08T18:05:27.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.504+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94713853056496] v27'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.504+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.504+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94713853057552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562449912d40 con 0x5624499046f0 2024-05-08T18:05:27.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.504+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562449913740 con 0x5624499046f0 2024-05-08T18:05:27.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.505+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94713853057552] v27'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.505+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(58 notify.7 [create] v27'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.505+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94713853060192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562449913700 con 0x5624499046f0 2024-05-08T18:05:27.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.505+0000 7fba92f2e800 20 add_watcher() i=0 2024-05-08T18:05:27.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.505+0000 7fba92f2e800 20 add_watcher() i=1 2024-05-08T18:05:27.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.505+0000 7fba92f2e800 20 add_watcher() i=2 2024-05-08T18:05:27.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.505+0000 7fba92f2e800 20 add_watcher() i=3 2024-05-08T18:05:27.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.505+0000 7fba92f2e800 20 add_watcher() i=4 2024-05-08T18:05:27.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.505+0000 7fba92f2e800 20 add_watcher() i=5 2024-05-08T18:05:27.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.505+0000 7fba92f2e800 20 add_watcher() i=6 2024-05-08T18:05:27.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.506+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94713853060192] v27'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.506+0000 7fba92f2e800 20 add_watcher() i=7 2024-05-08T18:05:27.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.506+0000 7fba92f2e800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:27.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.506+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562449916bf0 con 0x5624498ffd00 2024-05-08T18:05:27.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.506+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fba88040040 con 0x5624498ffd00 2024-05-08T18:05:27.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.506+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94713853064224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5624499140a0 con 0x5624498ffd00 2024-05-08T18:05:27.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.507+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94713853064224] v27'189 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fba88040040 con 0x5624498ffd00 2024-05-08T18:05:27.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.507+0000 7fb9457e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:27.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.507+0000 7fba92f2e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:27.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.507+0000 7fba92f2e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:27.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.507+0000 7fba92f2e800 5 note: GC not initialized 2024-05-08T18:05:27.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.508+0000 7fb8bffff640 20 reqs_thread_entry: start 2024-05-08T18:05:27.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.508+0000 7fba92f2e800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:27.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.508+0000 7fba92f2e800 20 Filter name: none 2024-05-08T18:05:27.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.508+0000 7fb8bdffb640 20 reqs_thread_entry: start 2024-05-08T18:05:27.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.508+0000 7fba92f2e800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:27.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.508+0000 7fba92f2e800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.508+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56244994a0c0 con 0x5624498ffd00 2024-05-08T18:05:27.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.508+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 foo [call out=48b,read 0~403 out=403b,getxattrs out=75b] v0'0 uv14 ondisk = 0) ==== 231+0+526 (crc 0 0 0) 0x7fba88040040 con 0x5624498ffd00 2024-05-08T18:05:27.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.508+0000 7fba92f2e800 20 rados_obj.operate() r=0 bl.length=403 2024-05-08T18:05:27.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.508+0000 7fba92f2e800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:27.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.508+0000 7fba92f2e800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:27.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.508+0000 7fba92f2e800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:27.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.508+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562449950360 con 0x5624498ffd00 2024-05-08T18:05:27.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.509+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~391] v27'15 uv15 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fba88040040 con 0x5624498ffd00 2024-05-08T18:05:27.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.510+0000 7fba92f2e800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:27.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.510+0000 7fba92f2e800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:27.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.510+0000 7fba92f2e800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-05-08T18:05:27.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.510+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94713853312288 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56244994bb40 con 0x5624498ffd00 2024-05-08T18:05:27.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.510+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== watch-notify(notify (1) cookie 94713853054208 notify 115964117015 ret 0) ==== 609+0+0 (crc 0 0 0) 0x7fba88018a30 con 0x5624498ffd00 2024-05-08T18:05:27.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.510+0000 7fb9727fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117015 cookie 94713853054208 notifier 4655 bl.length()=567 2024-05-08T18:05:27.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.510+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(64 notify.4 [notify cookie 94713853312288 out=8b] v0'0 uv183 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fba88040040 con 0x5624498ffd00 2024-05-08T18:05:27.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.510+0000 7fb9727fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-05-08T18:05:27.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.510+0000 7fb9727fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:27.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.510+0000 7fb9727fc640 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fb954003ea0 con 0x5624498ffd00 2024-05-08T18:05:27.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.510+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88040040 con 0x5624498ffd00 2024-05-08T18:05:27.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.510+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== watch-notify(notify_complete (2) cookie 94713853312288 notify 115964117015 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fba88018c00 con 0x5624498ffd00 2024-05-08T18:05:27.517 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-05-08T18:05:03.910065Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:05:03.910050Z","tags":[],"group_ids":[]} 2024-05-08T18:05:27.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.511+0000 7fb9457e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:27.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.511+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94713853064224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56244994f510 con 0x5624498ffd00 2024-05-08T18:05:27.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.512+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94713853064224] v27'190 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fba88040040 con 0x5624498ffd00 2024-05-08T18:05:27.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.512+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94713853040816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56244994f510 con 0x7fba78037040 2024-05-08T18:05:27.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.513+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94713853040816] v27'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88040040 con 0x7fba78037040 2024-05-08T18:05:27.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.513+0000 7fba92f2e800 20 remove_watcher() i=0 2024-05-08T18:05:27.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.513+0000 7fba92f2e800 2 removed watcher, disabling cache 2024-05-08T18:05:27.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.513+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94713853044224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56244990d8b0 con 0x5624498ffd00 2024-05-08T18:05:27.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.514+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94713853044224] v27'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88040040 con 0x5624498ffd00 2024-05-08T18:05:27.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.514+0000 7fba92f2e800 20 remove_watcher() i=1 2024-05-08T18:05:27.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.514+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94713853046944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56244990d8b0 con 0x562449909080 2024-05-08T18:05:27.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.515+0000 7fba86483640 1 -- 172.21.15.50:0/2284379180 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94713853046944] v27'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba78009050 con 0x562449909080 2024-05-08T18:05:27.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.515+0000 7fba92f2e800 20 remove_watcher() i=2 2024-05-08T18:05:27.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.515+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94713853049264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56244990e600 con 0x7fba78037040 2024-05-08T18:05:27.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.516+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94713853049264] v27'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88040040 con 0x7fba78037040 2024-05-08T18:05:27.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.516+0000 7fba92f2e800 20 remove_watcher() i=3 2024-05-08T18:05:27.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.516+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94713853054208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56244990e600 con 0x5624498ffd00 2024-05-08T18:05:27.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.517+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94713853054208] v27'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88040040 con 0x5624498ffd00 2024-05-08T18:05:27.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.517+0000 7fba92f2e800 20 remove_watcher() i=4 2024-05-08T18:05:27.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.517+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94713853056496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56244990e600 con 0x5624499046f0 2024-05-08T18:05:27.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.518+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94713853056496] v27'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.518+0000 7fba92f2e800 20 remove_watcher() i=5 2024-05-08T18:05:27.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.518+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94713853057552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5624499115f0 con 0x5624499046f0 2024-05-08T18:05:27.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.519+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94713853057552] v27'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.519+0000 7fba92f2e800 20 remove_watcher() i=6 2024-05-08T18:05:27.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.519+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94713853060192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56244990e600 con 0x5624499046f0 2024-05-08T18:05:27.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.520+0000 7fba8ec84640 1 -- 172.21.15.50:0/2284379180 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94713853060192] v27'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba8005a040 con 0x5624499046f0 2024-05-08T18:05:27.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.520+0000 7fba92f2e800 20 remove_watcher() i=7 2024-05-08T18:05:27.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.520+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5624499046f0 msgr2=0x562449906ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.520+0000 7fba92f2e800 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5624499046f0 0x562449906ae0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.520+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5624498ffd00 msgr2=0x5624499021c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.520+0000 7fba92f2e800 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5624498ffd00 0x5624499021c0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.520+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fba78037040 msgr2=0x7fba78036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.520+0000 7fba92f2e800 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fba78037040 0x7fba78036410 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.520+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562449909080 msgr2=0x56244990b540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.520+0000 7fba92f2e800 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562449909080 0x56244990b540 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.520+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb95803bba0 msgr2=0x7fb95803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.520+0000 7fba92f2e800 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb95803bba0 0x7fb95803e060 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fba7800efd0 tx=0x7fba78010000 comp rx=0 tx=0).stop 2024-05-08T18:05:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.520+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624498ed1a0 msgr2=0x5624498f45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.520+0000 7fba92f2e800 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624498ed1a0 0x5624498f45b0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fba8000a290 tx=0x7fba80090ef0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.520+0000 7fba8f485640 1 -- 172.21.15.50:0/2284379180 reap_dead start 2024-05-08T18:05:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 shutdown_connections 2024-05-08T18:05:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 --2- 172.21.15.50:0/2284379180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624498ed1a0 0x5624498f45b0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 >> 172.21.15.50:0/2284379180 conn(0x5624498e94b0 msgr2=0x5624498e6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 shutdown_connections 2024-05-08T18:05:27.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 -- 172.21.15.50:0/2284379180 wait complete. 2024-05-08T18:05:27.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 -- 172.21.15.50:0/2716762354 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb96803bba0 msgr2=0x7fb96803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 --2- 172.21.15.50:0/2716762354 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb96803bba0 0x7fb96803e060 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fba780016b0 tx=0x7fba7800f000 comp rx=0 tx=0).stop 2024-05-08T18:05:27.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 -- 172.21.15.50:0/2716762354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c74f0 msgr2=0x5624497c5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 --2- 172.21.15.50:0/2716762354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c74f0 0x5624497c5960 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fba8005f040 tx=0x7fba8005bf70 comp rx=0 tx=0).stop 2024-05-08T18:05:27.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 -- 172.21.15.50:0/2716762354 shutdown_connections 2024-05-08T18:05:27.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 --2- 172.21.15.50:0/2716762354 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb96803bba0 0x7fb96803e060 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 --2- 172.21.15.50:0/2716762354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c74f0 0x5624497c5960 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 -- 172.21.15.50:0/2716762354 >> 172.21.15.50:0/2716762354 conn(0x5624498d1f30 msgr2=0x5624498d2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 -- 172.21.15.50:0/2716762354 shutdown_connections 2024-05-08T18:05:27.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 -- 172.21.15.50:0/2716762354 wait complete. 2024-05-08T18:05:27.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 -- 172.21.15.50:0/1963337501 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5624498c8fc0 msgr2=0x5624498cb450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5624498c8fc0 0x5624498cb450 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 -- 172.21.15.50:0/1963337501 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5624497c7950 msgr2=0x5624498c6860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5624497c7950 0x5624498c6860 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 -- 172.21.15.50:0/1963337501 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5624498cd040 msgr2=0x5624498cf430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.521+0000 7fba92f2e800 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5624498cd040 0x5624498cf430 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.522+0000 7fba92f2e800 1 -- 172.21.15.50:0/1963337501 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb97403bba0 msgr2=0x7fb97403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.522+0000 7fba92f2e800 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb97403bba0 0x7fb97403e060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fba78001d70 tx=0x7fba780012d0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.522+0000 7fba92f2e800 1 -- 172.21.15.50:0/1963337501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 msgr2=0x5624498bf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.522+0000 7fba92f2e800 1 --2- 172.21.15.50:0/1963337501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5624497c4960 0x5624498bf620 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fba800057d0 tx=0x7fba80002fb0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.522+0000 7fba8f485640 1 -- 172.21.15.50:0/1963337501 reap_dead start 2024-05-08T18:05:27.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.522+0000 7fba92f2e800 1 -- 172.21.15.50:0/1963337501 shutdown_connections 2024-05-08T18:05:27.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.522+0000 7fba92f2e800 1 -- 172.21.15.50:0/1963337501 >> 172.21.15.50:0/1963337501 conn(0x5624497c02f0 msgr2=0x5624498b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.522+0000 7fba92f2e800 1 -- 172.21.15.50:0/1963337501 shutdown_connections 2024-05-08T18:05:27.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.522+0000 7fba92f2e800 1 -- 172.21.15.50:0/1963337501 wait complete. 2024-05-08T18:05:27.529 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-05-08T18:05:03.910065Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:05:03.910050Z', 'tags': [], 'group_ids': []} 2024-05-08T18:05:27.539 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:27.539 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-05-08T18:05:27.539 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-05-08T18:05:27.539 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-05-08T18:05:27.584 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:05:27.585 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:05:27.585 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.587+0000 7f8c0283d800 1 Processor -- start 2024-05-08T18:05:27.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.588+0000 7f8c0283d800 1 -- start start 2024-05-08T18:05:27.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.588+0000 7f8c0283d800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5614a4bab9e0 0x5614a4baddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.588+0000 7f8c0283d800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5614a4bae3a0 con 0x5614a4ba7550 2024-05-08T18:05:27.586 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.588+0000 7f8c0283d800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5614a4a48e20 con 0x5614a4bab9e0 2024-05-08T18:05:27.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.588+0000 7f8bf7b34640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5614a4bab9e0 0x5614a4baddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.588+0000 7f8c00335640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5614a4ba7550 0x5614a4ba7970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:60416/0 (socket says 172.21.15.50:60416) 2024-05-08T18:05:27.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.588+0000 7f8bf7b34640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5614a4bab9e0 0x5614a4baddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:44126/0 (socket says 172.21.15.50:44126) 2024-05-08T18:05:27.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.588+0000 7f8c00335640 1 -- 172.21.15.50:0/1913454711 learned_addr learned my addr 172.21.15.50:0/1913454711 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:05:27.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.588+0000 7f8bf7b34640 1 -- 172.21.15.50:0/1913454711 >> v1:172.21.15.46:6789/0 conn(0x5614a4ba7550 legacy=0x5614a4ba7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.588+0000 7f8bf7b34640 1 -- 172.21.15.50:0/1913454711 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614a4baf5d0 con 0x5614a4bab9e0 2024-05-08T18:05:27.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.590+0000 7f8bf7b34640 1 --2- 172.21.15.50:0/1913454711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4bab9e0 0x5614a4baddd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8be800a470 tx=0x7f8be801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4a942f436694a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.590+0000 7f8bffb34640 1 -- 172.21.15.50:0/1913454711 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8be8057070 con 0x5614a4bab9e0 2024-05-08T18:05:27.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.590+0000 7f8bffb34640 1 -- 172.21.15.50:0/1913454711 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8be8052570 con 0x5614a4bab9e0 2024-05-08T18:05:27.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.590+0000 7f8bffb34640 1 -- 172.21.15.50:0/1913454711 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8be8052870 con 0x5614a4bab9e0 2024-05-08T18:05:27.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.591+0000 7f8c0283d800 1 -- 172.21.15.50:0/1913454711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4bab9e0 msgr2=0x5614a4baddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.591+0000 7f8c0283d800 1 --2- 172.21.15.50:0/1913454711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4bab9e0 0x5614a4baddd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8be800a470 tx=0x7f8be801ce90 comp rx=0 tx=0).stop 2024-05-08T18:05:27.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.591+0000 7f8c0283d800 1 -- 172.21.15.50:0/1913454711 shutdown_connections 2024-05-08T18:05:27.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.591+0000 7f8c0283d800 1 --2- 172.21.15.50:0/1913454711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4bab9e0 0x5614a4baddd0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.591+0000 7f8c0283d800 1 -- 172.21.15.50:0/1913454711 >> 172.21.15.50:0/1913454711 conn(0x5614a4ba2b50 msgr2=0x5614a4ba4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.591+0000 7f8c0283d800 1 -- 172.21.15.50:0/1913454711 shutdown_connections 2024-05-08T18:05:27.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.591+0000 7f8c0283d800 1 -- 172.21.15.50:0/1913454711 wait complete. 2024-05-08T18:05:27.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.606+0000 7f8c0283d800 1 Processor -- start 2024-05-08T18:05:27.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.606+0000 7f8c0283d800 1 -- start start 2024-05-08T18:05:27.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.606+0000 7f8c0283d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ba7550 0x5614a4b1b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.607+0000 7f8c0283d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5614a4af3a10 con 0x5614a4ba7550 2024-05-08T18:05:27.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.607+0000 7f8c00335640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ba7550 0x5614a4b1b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.607+0000 7f8c00335640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ba7550 0x5614a4b1b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:44142/0 (socket says 172.21.15.50:44142) 2024-05-08T18:05:27.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.607+0000 7f8c00335640 1 -- 172.21.15.50:0/164667673 learned_addr learned my addr 172.21.15.50:0/164667673 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.607+0000 7f8c00335640 1 -- 172.21.15.50:0/164667673 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614a4c9a600 con 0x5614a4ba7550 2024-05-08T18:05:27.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.607+0000 7f8c00335640 1 --2- 172.21.15.50:0/164667673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ba7550 0x5614a4b1b530 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0001e40 tx=0x7f8bf0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ce52a201ed72e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.607+0000 7f8aee60c640 1 -- 172.21.15.50:0/164667673 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bf0012020 con 0x5614a4ba7550 2024-05-08T18:05:27.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.607+0000 7f8aee60c640 1 -- 172.21.15.50:0/164667673 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf000e5a0 con 0x5614a4ba7550 2024-05-08T18:05:27.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.607+0000 7f8aee60c640 1 -- 172.21.15.50:0/164667673 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bf000e8a0 con 0x5614a4ba7550 2024-05-08T18:05:27.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.608+0000 7f8c0283d800 1 -- 172.21.15.50:0/164667673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ba7550 msgr2=0x5614a4b1b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.608+0000 7f8c0283d800 1 --2- 172.21.15.50:0/164667673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ba7550 0x5614a4b1b530 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0001e40 tx=0x7f8bf0001e70 comp rx=0 tx=0).stop 2024-05-08T18:05:27.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.608+0000 7f8c0283d800 1 -- 172.21.15.50:0/164667673 shutdown_connections 2024-05-08T18:05:27.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.608+0000 7f8c0283d800 1 --2- 172.21.15.50:0/164667673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ba7550 0x5614a4b1b530 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.608+0000 7f8c0283d800 1 -- 172.21.15.50:0/164667673 >> 172.21.15.50:0/164667673 conn(0x5614a4ba2b50 msgr2=0x5614a4c94300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.608+0000 7f8c0283d800 1 -- 172.21.15.50:0/164667673 shutdown_connections 2024-05-08T18:05:27.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.608+0000 7f8c0283d800 1 -- 172.21.15.50:0/164667673 wait complete. 2024-05-08T18:05:27.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.608+0000 7f8c0283d800 1 Processor -- start 2024-05-08T18:05:27.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.609+0000 7f8c0283d800 1 -- start start 2024-05-08T18:05:27.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.609+0000 7f8c0283d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ba7550 0x5614a4ca3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.609+0000 7f8c0283d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5614a4bae870 con 0x5614a4ba7550 2024-05-08T18:05:27.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.609+0000 7f8c00335640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ba7550 0x5614a4ca3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.609+0000 7f8c00335640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ba7550 0x5614a4ca3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:44150/0 (socket says 172.21.15.50:44150) 2024-05-08T18:05:27.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.609+0000 7f8c00335640 1 -- 172.21.15.50:0/119863861 learned_addr learned my addr 172.21.15.50:0/119863861 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.609+0000 7f8c00335640 1 -- 172.21.15.50:0/119863861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614a4ca3dd0 con 0x5614a4ba7550 2024-05-08T18:05:27.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.609+0000 7f8c00335640 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ba7550 0x5614a4ca3890 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0000c00 tx=0x7f8bf0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.610+0000 7f8aece09640 1 -- 172.21.15.50:0/119863861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bf0012030 con 0x5614a4ba7550 2024-05-08T18:05:27.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.610+0000 7f8aece09640 1 -- 172.21.15.50:0/119863861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf000ee50 con 0x5614a4ba7550 2024-05-08T18:05:27.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.610+0000 7f8c0283d800 1 -- 172.21.15.50:0/119863861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5614a4ca4060 con 0x5614a4ba7550 2024-05-08T18:05:27.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.610+0000 7f8aece09640 1 -- 172.21.15.50:0/119863861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bf001a940 con 0x5614a4ba7550 2024-05-08T18:05:27.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.610+0000 7f8c0283d800 1 -- 172.21.15.50:0/119863861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614a4ca6d10 con 0x5614a4ba7550 2024-05-08T18:05:27.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.611+0000 7f8aece09640 1 -- 172.21.15.50:0/119863861 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f8bf0015020 con 0x5614a4ba7550 2024-05-08T18:05:27.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.611+0000 7f8aece09640 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f8ae803bb50 0x7f8ae803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.611+0000 7f8aece09640 1 -- 172.21.15.50:0/119863861 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f8bf0020030 con 0x5614a4ba7550 2024-05-08T18:05:27.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.611+0000 7f8bf7b34640 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f8ae803bb50 0x7f8ae803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.611+0000 7f8c0283d800 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5614a4bab9e0 0x5614a4caaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.611+0000 7f8c0283d800 1 -- 172.21.15.50:0/119863861 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cab0a0 con 0x5614a4bab9e0 2024-05-08T18:05:27.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.611+0000 7f8c00b36640 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5614a4bab9e0 0x5614a4caaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.611+0000 7f8bf7b34640 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f8ae803bb50 0x7f8ae803e010 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8be8054000 tx=0x7f8be804e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.611+0000 7f8c00b36640 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5614a4bab9e0 0x5614a4caaad0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.612+0000 7f8c00b36640 1 -- 172.21.15.50:0/119863861 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4bab9e0 2024-05-08T18:05:27.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.612+0000 7f8c0283d800 1 -- 172.21.15.50:0/119863861 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ca80a0 con 0x5614a4bab9e0 2024-05-08T18:05:27.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.612+0000 7f8c00b36640 1 -- 172.21.15.50:0/119863861 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4bab9e0 2024-05-08T18:05:27.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.612+0000 7f8c0283d800 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5614a4cad3b0 0x5614a4caf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.612+0000 7f8c0283d800 1 -- 172.21.15.50:0/119863861 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cafe40 con 0x5614a4cad3b0 2024-05-08T18:05:27.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.612+0000 7f8c00335640 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5614a4cad3b0 0x5614a4caf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.613+0000 7f8c00335640 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5614a4cad3b0 0x5614a4caf7a0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.613+0000 7f8c00335640 1 -- 172.21.15.50:0/119863861 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f8bf003fd70 con 0x5614a4cad3b0 2024-05-08T18:05:27.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.613+0000 7f8c0283d800 1 -- 172.21.15.50:0/119863861 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cac250 con 0x5614a4bab9e0 2024-05-08T18:05:27.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.613+0000 7f8c00b36640 1 -- 172.21.15.50:0/119863861 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4bab9e0 2024-05-08T18:05:27.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.613+0000 7f8c0283d800 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5614a4cb12a0 0x5614a4cb3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.613+0000 7f8c0283d800 1 -- 172.21.15.50:0/119863861 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cb3d90 con 0x5614a4cb12a0 2024-05-08T18:05:27.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.614+0000 7f8bf7b34640 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5614a4cb12a0 0x5614a4cb3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.614+0000 7f8bf7b34640 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5614a4cb12a0 0x5614a4cb3690 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.614+0000 7f8bf7b34640 1 -- 172.21.15.50:0/119863861 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f8be8052940 con 0x5614a4cb12a0 2024-05-08T18:05:27.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.615+0000 7f8c0283d800 1 Processor -- start 2024-05-08T18:05:27.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.615+0000 7f8c0283d800 1 -- start start 2024-05-08T18:05:27.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.615+0000 7f8c0283d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4cc2100 0x5614a4cc2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.615+0000 7f8c0283d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5614a4a9a330 con 0x5614a4cc2100 2024-05-08T18:05:27.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.615+0000 7f8c00335640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4cc2100 0x5614a4cc2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.615+0000 7f8c00335640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4cc2100 0x5614a4cc2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:44152/0 (socket says 172.21.15.50:44152) 2024-05-08T18:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.615+0000 7f8c00335640 1 -- 172.21.15.50:0/3504231450 learned_addr learned my addr 172.21.15.50:0/3504231450 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.615+0000 7f8c00335640 1 -- 172.21.15.50:0/3504231450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614a4cc2a60 con 0x5614a4cc2100 2024-05-08T18:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.616+0000 7f8c00335640 1 --2- 172.21.15.50:0/3504231450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4cc2100 0x5614a4cc2520 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00510b0 tx=0x7f8bf0043500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.616+0000 7f8ae5ffb640 1 -- 172.21.15.50:0/3504231450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bf0018c90 con 0x5614a4cc2100 2024-05-08T18:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.616+0000 7f8ae5ffb640 1 -- 172.21.15.50:0/3504231450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf0047070 con 0x5614a4cc2100 2024-05-08T18:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.616+0000 7f8c0283d800 1 -- 172.21.15.50:0/3504231450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5614a4cc34d0 con 0x5614a4cc2100 2024-05-08T18:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.616+0000 7f8ae5ffb640 1 -- 172.21.15.50:0/3504231450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bf003daf0 con 0x5614a4cc2100 2024-05-08T18:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.616+0000 7f8c0283d800 1 -- 172.21.15.50:0/3504231450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614a4cc34d0 con 0x5614a4cc2100 2024-05-08T18:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.617+0000 7f8ae5ffb640 1 -- 172.21.15.50:0/3504231450 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f8bf003dc90 con 0x5614a4cc2100 2024-05-08T18:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.617+0000 7f8ae5ffb640 1 --2- 172.21.15.50:0/3504231450 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f8ad803bba0 0x7f8ad803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.617+0000 7f8ae5ffb640 1 -- 172.21.15.50:0/3504231450 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f8bf0038cd0 con 0x5614a4cc2100 2024-05-08T18:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.617+0000 7f8bf7b34640 1 --2- 172.21.15.50:0/3504231450 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f8ad803bba0 0x7f8ad803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.617+0000 7f8c0283d800 1 Processor -- start 2024-05-08T18:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.617+0000 7f8c0283d800 1 -- start start 2024-05-08T18:05:27.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.617+0000 7f8bf7b34640 1 --2- 172.21.15.50:0/3504231450 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f8ad803bba0 0x7f8ad803e060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5614a4ca7fc0 tx=0x7f8be8005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.617+0000 7f8c0283d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ccf030 0x5614a4ccf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.617+0000 7f8c0283d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5614a4c9ed90 con 0x5614a4ccf030 2024-05-08T18:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.618+0000 7f8c00335640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ccf030 0x5614a4ccf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.618+0000 7f8c00335640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ccf030 0x5614a4ccf450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:44166/0 (socket says 172.21.15.50:44166) 2024-05-08T18:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.618+0000 7f8c00335640 1 -- 172.21.15.50:0/3342348935 learned_addr learned my addr 172.21.15.50:0/3342348935 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.618+0000 7f8c00335640 1 -- 172.21.15.50:0/3342348935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614a4ccfa20 con 0x5614a4ccf030 2024-05-08T18:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.618+0000 7f8c00335640 1 --2- 172.21.15.50:0/3342348935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ccf030 0x5614a4ccf450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0016fd0 tx=0x7f8bf00529f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4976b86a7cb1203d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.618+0000 7f8adf7fe640 1 -- 172.21.15.50:0/3342348935 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bf0043e30 con 0x5614a4ccf030 2024-05-08T18:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.618+0000 7f8adf7fe640 1 -- 172.21.15.50:0/3342348935 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf004a350 con 0x5614a4ccf030 2024-05-08T18:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.618+0000 7f8adf7fe640 1 -- 172.21.15.50:0/3342348935 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bf00522a0 con 0x5614a4ccf030 2024-05-08T18:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.618+0000 7f8c0283d800 1 -- 172.21.15.50:0/3342348935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ccf030 msgr2=0x5614a4ccf450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.618+0000 7f8c0283d800 1 --2- 172.21.15.50:0/3342348935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ccf030 0x5614a4ccf450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0016fd0 tx=0x7f8bf00529f0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.618+0000 7f8c0283d800 1 -- 172.21.15.50:0/3342348935 shutdown_connections 2024-05-08T18:05:27.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.619+0000 7f8c0283d800 1 --2- 172.21.15.50:0/3342348935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ccf030 0x5614a4ccf450 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.619+0000 7f8c0283d800 1 -- 172.21.15.50:0/3342348935 >> 172.21.15.50:0/3342348935 conn(0x5614a4ccb390 msgr2=0x5614a4cc93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.619+0000 7f8c0283d800 1 -- 172.21.15.50:0/3342348935 shutdown_connections 2024-05-08T18:05:27.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.619+0000 7f8c0283d800 1 -- 172.21.15.50:0/3342348935 wait complete. 2024-05-08T18:05:27.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.619+0000 7f8c0283d800 1 Processor -- start 2024-05-08T18:05:27.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.619+0000 7f8c0283d800 1 -- start start 2024-05-08T18:05:27.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.619+0000 7f8c0283d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ccf030 0x5614a4cd64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.619+0000 7f8c0283d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5614a4cd00d0 con 0x5614a4ccf030 2024-05-08T18:05:27.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.619+0000 7f8c00335640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ccf030 0x5614a4cd64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.619+0000 7f8c00335640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ccf030 0x5614a4cd64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:44174/0 (socket says 172.21.15.50:44174) 2024-05-08T18:05:27.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.619+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 learned_addr learned my addr 172.21.15.50:0/2878544332 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:05:27.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.620+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614a4cd6a00 con 0x5614a4ccf030 2024-05-08T18:05:27.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.620+0000 7f8c00335640 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ccf030 0x5614a4cd64c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0016fd0 tx=0x7f8bf000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.620+0000 7f8addffb640 1 -- 172.21.15.50:0/2878544332 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bf006f070 con 0x5614a4ccf030 2024-05-08T18:05:27.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.620+0000 7f8addffb640 1 -- 172.21.15.50:0/2878544332 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bf000f9b0 con 0x5614a4ccf030 2024-05-08T18:05:27.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.620+0000 7f8addffb640 1 -- 172.21.15.50:0/2878544332 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bf00227c0 con 0x5614a4ccf030 2024-05-08T18:05:27.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.620+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5614a4cd6a00 con 0x5614a4ccf030 2024-05-08T18:05:27.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.620+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614a4cd6a00 con 0x5614a4ccf030 2024-05-08T18:05:27.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.621+0000 7f8addffb640 1 -- 172.21.15.50:0/2878544332 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f8bf0072070 con 0x5614a4ccf030 2024-05-08T18:05:27.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.621+0000 7f8addffb640 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f8ad003bb50 0x7f8ad003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.621+0000 7f8addffb640 1 -- 172.21.15.50:0/2878544332 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6133+0+0 (secure 0 0 0) 0x7f8bf005a020 con 0x5614a4ccf030 2024-05-08T18:05:27.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.621+0000 7f8bf7b34640 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f8ad003bb50 0x7f8ad003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.621+0000 7f8bf7b34640 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f8ad003bb50 0x7f8ad003e010 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8be8052fd0 tx=0x7f8be804f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.622+0000 7f8c0283d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.622+0000 7f8c0283d800 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5614a4ce1b60 0x5614a4ce4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.622+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce4770 con 0x5614a4ce1b60 2024-05-08T18:05:27.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.622+0000 7f8c00b36640 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5614a4ce1b60 0x5614a4ce4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.622+0000 7f8c00b36640 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5614a4ce1b60 0x5614a4ce4020 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.622+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4ce1b60 2024-05-08T18:05:27.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.622+0000 7f8c0283d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.622+0000 7f8c0283d800 20 realm 2024-05-08T18:05:27.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.622+0000 7f8c0283d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.622+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce4b80 con 0x5614a4ce1b60 2024-05-08T18:05:27.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.623+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4ce1b60 2024-05-08T18:05:27.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.623+0000 7f8c0283d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.623+0000 7f8c0283d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:05:27.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.623+0000 7f8c0283d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.623+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce5450 con 0x5614a4ce1b60 2024-05-08T18:05:27.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.623+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4ce1b60 2024-05-08T18:05:27.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.623+0000 7f8c0283d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.623+0000 7f8c0283d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.623+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce57f0 con 0x5614a4ce1b60 2024-05-08T18:05:27.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.623+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4ce1b60 2024-05-08T18:05:27.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.623+0000 7f8c0283d800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:27.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.623+0000 7f8c0283d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.623+0000 7f8c0283d800 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5614a4ce6550 0x5614a4ce8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.623+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce8f50 con 0x5614a4ce6550 2024-05-08T18:05:27.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.623+0000 7f8c00335640 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5614a4ce6550 0x5614a4ce8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.624+0000 7f8c00335640 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5614a4ce6550 0x5614a4ce8940 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.624+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.624+0000 7f8c0283d800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:05:27.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.624+0000 7f8c0283d800 20 searching for the correct realm 2024-05-08T18:05:27.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.624+0000 7f8c0283d800 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5614a4ceaf00 0x5614a4ced3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.624+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ceda90 con 0x5614a4ceaf00 2024-05-08T18:05:27.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.624+0000 7f8bf7b34640 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5614a4ceaf00 0x5614a4ced3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.624+0000 7f8bf7b34640 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5614a4ceaf00 0x5614a4ced3c0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.625+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.625+0000 7f8bf7b34640 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f8be8076040 0x7f8be8083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:05:27.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.625+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ceda90 con 0x7f8be8076040 2024-05-08T18:05:27.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.625+0000 7f8c00b36640 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f8be8076040 0x7f8be8083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:05:27.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.625+0000 7f8c00b36640 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f8be8076040 0x7f8be8083410 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:05:27.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.625+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf800a4a0 con 0x7f8be8076040 2024-05-08T18:05:27.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.625+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce57f0 con 0x5614a4ce6550 2024-05-08T18:05:27.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.626+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.626+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce57f0 con 0x5614a4ce1b60 2024-05-08T18:05:27.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.626+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4ce1b60 2024-05-08T18:05:27.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.626+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce57f0 con 0x5614a4ce1b60 2024-05-08T18:05:27.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.626+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4ce1b60 2024-05-08T18:05:27.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.626+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce57f0 con 0x5614a4ce6550 2024-05-08T18:05:27.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.626+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.626+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce57f0 con 0x5614a4ce6550 2024-05-08T18:05:27.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.627+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(12 [pgnls start_epoch 27 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.627+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce57f0 con 0x7f8be8076040 2024-05-08T18:05:27.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.627+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf800a4a0 con 0x7f8be8076040 2024-05-08T18:05:27.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.627+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce57f0 con 0x5614a4ceaf00 2024-05-08T18:05:27.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.627+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.627+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce57f0 con 0x5614a4ce1b60 2024-05-08T18:05:27.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.627+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4ce1b60 2024-05-08T18:05:27.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.627+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce57f0 con 0x5614a4ce6550 2024-05-08T18:05:27.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.628+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.628+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce57f0 con 0x5614a4ce1b60 2024-05-08T18:05:27.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.628+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4ce1b60 2024-05-08T18:05:27.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.628+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce57f0 con 0x7f8be8076040 2024-05-08T18:05:27.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.628+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f8bf800a4a0 con 0x7f8be8076040 2024-05-08T18:05:27.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.628+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce57f0 con 0x5614a4ceaf00 2024-05-08T18:05:27.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.628+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.628+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x5614a4ceaf00 2024-05-08T18:05:27.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.628+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.628+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x5614a4ce6550 2024-05-08T18:05:27.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.629+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.629+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x5614a4ceaf00 2024-05-08T18:05:27.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.629+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.629+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x5614a4ceaf00 2024-05-08T18:05:27.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.629+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 27 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.629+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x5614a4ce1b60 2024-05-08T18:05:27.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.629+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4ce1b60 2024-05-08T18:05:27.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.630+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x5614a4ce1b60 2024-05-08T18:05:27.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.630+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4ce1b60 2024-05-08T18:05:27.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.630+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x5614a4ce6550 2024-05-08T18:05:27.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.630+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.630+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x5614a4ceaf00 2024-05-08T18:05:27.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.630+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.630+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x7f8be8076040 2024-05-08T18:05:27.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.630+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf800a4a0 con 0x7f8be8076040 2024-05-08T18:05:27.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.630+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x5614a4ce1b60 2024-05-08T18:05:27.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.631+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4ce1b60 2024-05-08T18:05:27.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.631+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x5614a4ce6550 2024-05-08T18:05:27.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.631+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.631+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x5614a4ceaf00 2024-05-08T18:05:27.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.631+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.631+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x5614a4ce6550 2024-05-08T18:05:27.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.631+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.631+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x7f8be8076040 2024-05-08T18:05:27.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.632+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf800a4a0 con 0x7f8be8076040 2024-05-08T18:05:27.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.632+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x5614a4ceaf00 2024-05-08T18:05:27.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.632+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.632+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x5614a4ce6550 2024-05-08T18:05:27.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.632+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.632+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x5614a4ce1b60 2024-05-08T18:05:27.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4ce1b60 2024-05-08T18:05:27.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f8be8090040 con 0x7f8be8076040 2024-05-08T18:05:27.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8bf800a4a0 con 0x7f8be8076040 2024-05-08T18:05:27.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c0283d800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:05:27.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c0283d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:05:27.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c0283d800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:05:27.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c0283d800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:05:27.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c0283d800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:05:27.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c0283d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:05:27.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c0283d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce9810 con 0x5614a4ce1b60 2024-05-08T18:05:27.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4ce1b60 2024-05-08T18:05:27.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c0283d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c0283d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce9810 con 0x5614a4ce1b60 2024-05-08T18:05:27.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8bf800a4a0 con 0x5614a4ce1b60 2024-05-08T18:05:27.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c0283d800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:05:27.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c0283d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.633+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ce9810 con 0x5614a4ceaf00 2024-05-08T18:05:27.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8c0283d800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:05:27.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8c0283d800 20 zone default found 2024-05-08T18:05:27.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8c0283d800 4 Realm: () 2024-05-08T18:05:27.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8c0283d800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:05:27.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8c0283d800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:05:27.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8c0283d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:05:27.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8c0283d800 20 zonegroup default 2024-05-08T18:05:27.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8c0283d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cea0d0 con 0x5614a4ce6550 2024-05-08T18:05:27.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8c0283d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8c0283d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cef8f0 con 0x5614a4ceaf00 2024-05-08T18:05:27.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8c0283d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8c0283d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cefd20 con 0x5614a4ceaf00 2024-05-08T18:05:27.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.634+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.635+0000 7f8c0283d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:05:27.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.635+0000 7f8c0283d800 20 started sync module instance, tier type = 2024-05-08T18:05:27.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.635+0000 7f8c0283d800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:05:27.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.635+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ceed90 con 0x7f8be8076040 2024-05-08T18:05:27.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.636+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v27'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf800a4a0 con 0x7f8be8076040 2024-05-08T18:05:27.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.636+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94646664360592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ceed90 con 0x7f8be8076040 2024-05-08T18:05:27.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.636+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf0810 con 0x5614a4ce1b60 2024-05-08T18:05:27.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.637+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94646664360592] v27'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x7f8be8076040 2024-05-08T18:05:27.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.637+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(46 notify.1 [create] v27'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x5614a4ce1b60 2024-05-08T18:05:27.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.637+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94646664364832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ceed90 con 0x5614a4ce1b60 2024-05-08T18:05:27.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.637+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf12e0 con 0x5614a4ceaf00 2024-05-08T18:05:27.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.638+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94646664364832] v27'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x5614a4ce1b60 2024-05-08T18:05:27.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.638+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v27'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.638+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94646664358112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4ceed00 con 0x5614a4ceaf00 2024-05-08T18:05:27.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.638+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf21a0 con 0x7f8be8076040 2024-05-08T18:05:27.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.639+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94646664358112] v27'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.639+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v27'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x7f8be8076040 2024-05-08T18:05:27.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.639+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94646664371440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf25c0 con 0x7f8be8076040 2024-05-08T18:05:27.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.639+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf2e90 con 0x5614a4ce1b60 2024-05-08T18:05:27.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.640+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(52 notify.4 [create] v27'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x5614a4ce1b60 2024-05-08T18:05:27.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.640+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94646664371440] v27'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x7f8be8076040 2024-05-08T18:05:27.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.640+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94646664373696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf29e0 con 0x5614a4ce1b60 2024-05-08T18:05:27.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.640+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf2130 con 0x5614a4ce6550 2024-05-08T18:05:27.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.641+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(54 notify.5 [create] v27'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.641+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94646664373696] v27'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x5614a4ce1b60 2024-05-08T18:05:27.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.641+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94646664378704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf4220 con 0x5614a4ce6550 2024-05-08T18:05:27.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.641+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf3280 con 0x5614a4ce6550 2024-05-08T18:05:27.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.642+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94646664378704] v27'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.642+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(56 notify.6 [create] v27'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.642+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94646664382256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf5000 con 0x5614a4ce6550 2024-05-08T18:05:27.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.642+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf5850 con 0x5614a4ce6550 2024-05-08T18:05:27.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.643+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v27'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.643+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94646664385024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf5c90 con 0x5614a4ce6550 2024-05-08T18:05:27.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.643+0000 7f8c0283d800 20 add_watcher() i=0 2024-05-08T18:05:27.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.643+0000 7f8c0283d800 20 add_watcher() i=1 2024-05-08T18:05:27.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.643+0000 7f8c0283d800 20 add_watcher() i=2 2024-05-08T18:05:27.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.643+0000 7f8c0283d800 20 add_watcher() i=3 2024-05-08T18:05:27.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.643+0000 7f8c0283d800 20 add_watcher() i=4 2024-05-08T18:05:27.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.643+0000 7f8c0283d800 20 add_watcher() i=5 2024-05-08T18:05:27.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.643+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94646664382256] v27'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.643+0000 7f8c0283d800 20 add_watcher() i=6 2024-05-08T18:05:27.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.644+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94646664385024] v27'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.644+0000 7f8c0283d800 20 add_watcher() i=7 2024-05-08T18:05:27.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.644+0000 7f8c0283d800 2 all 8 watchers are set, enabling cache 2024-05-08T18:05:27.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.644+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf4b50 con 0x5614a4ce1b60 2024-05-08T18:05:27.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.645+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f8bf8040040 con 0x5614a4ce1b60 2024-05-08T18:05:27.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.645+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94646664386336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf4b50 con 0x5614a4ce1b60 2024-05-08T18:05:27.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.646+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94646664386336] v27'191 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x5614a4ce1b60 2024-05-08T18:05:27.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.646+0000 7f8ab67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:27.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.646+0000 7f8c0283d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:05:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.646+0000 7f8c0283d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:05:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.646+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d016a0 con 0x5614a4ce1b60 2024-05-08T18:05:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.647+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'31 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x5614a4ce1b60 2024-05-08T18:05:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.647+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d01d70 con 0x5614a4ce6550 2024-05-08T18:05:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.648+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'26 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.648+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d01d70 con 0x5614a4ceaf00 2024-05-08T18:05:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.649+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'25 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.649+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d021f0 con 0x5614a4ce1b60 2024-05-08T18:05:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.650+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'192 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x5614a4ce1b60 2024-05-08T18:05:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.650+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d02760 con 0x7f8be8076040 2024-05-08T18:05:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.651+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'65 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x7f8be8076040 2024-05-08T18:05:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.651+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d02cd0 con 0x5614a4ceaf00 2024-05-08T18:05:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.652+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'43 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.652+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03180 con 0x5614a4ce1b60 2024-05-08T18:05:27.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.653+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'28 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x5614a4ce1b60 2024-05-08T18:05:27.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.653+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03180 con 0x5614a4ce6550 2024-05-08T18:05:27.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.654+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'43 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.654+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03280 con 0x7f8be8076040 2024-05-08T18:05:27.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.655+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'25 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x7f8be8076040 2024-05-08T18:05:27.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.655+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03280 con 0x5614a4ceaf00 2024-05-08T18:05:27.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.656+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'31 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.656+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x7f8be8076040 2024-05-08T18:05:27.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.657+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'66 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x7f8be8076040 2024-05-08T18:05:27.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.657+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x5614a4ce6550 2024-05-08T18:05:27.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.658+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'28 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.658+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x7f8be8076040 2024-05-08T18:05:27.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.659+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'26 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x7f8be8076040 2024-05-08T18:05:27.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.659+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x5614a4ce6550 2024-05-08T18:05:27.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.660+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'49 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.660+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x5614a4ce1b60 2024-05-08T18:05:27.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.661+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'193 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x5614a4ce1b60 2024-05-08T18:05:27.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.661+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x5614a4ceaf00 2024-05-08T18:05:27.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.663+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'60 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.663+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03850 con 0x7f8be8076040 2024-05-08T18:05:27.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.664+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'67 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x7f8be8076040 2024-05-08T18:05:27.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.664+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03850 con 0x5614a4ceaf00 2024-05-08T18:05:27.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.664+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'25 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.664+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03820 con 0x5614a4ce1b60 2024-05-08T18:05:27.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.665+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'33 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x5614a4ce1b60 2024-05-08T18:05:27.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.666+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03820 con 0x5614a4ce1b60 2024-05-08T18:05:27.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.666+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'194 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x5614a4ce1b60 2024-05-08T18:05:27.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.667+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03820 con 0x5614a4ceaf00 2024-05-08T18:05:27.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.667+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'61 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.667+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x7f8be8076040 2024-05-08T18:05:27.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.668+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'35 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x7f8be8076040 2024-05-08T18:05:27.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.668+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x5614a4ceaf00 2024-05-08T18:05:27.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.669+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'27 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.669+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x5614a4ceaf00 2024-05-08T18:05:27.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.670+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'31 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.670+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x5614a4ceaf00 2024-05-08T18:05:27.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.671+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'23 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.669 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.671+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x5614a4ceaf00 2024-05-08T18:05:27.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.672+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'44 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.672+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x5614a4ce6550 2024-05-08T18:05:27.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.673+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'27 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.673+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x5614a4ceaf00 2024-05-08T18:05:27.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.674+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'25 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.674+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x5614a4ce6550 2024-05-08T18:05:27.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.675+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'44 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.675+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x5614a4ceaf00 2024-05-08T18:05:27.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.676+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'62 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.676+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x5614a4ce6550 2024-05-08T18:05:27.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.677+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'50 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.677+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d03790 con 0x7f8be8076040 2024-05-08T18:05:27.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.678+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'31 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x7f8be8076040 2024-05-08T18:05:27.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.678+0000 7f8ab57e2640 20 reqs_thread_entry: start 2024-05-08T18:05:27.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.678+0000 7f8c0283d800 20 init_complete bucket index max shards: 11 2024-05-08T18:05:27.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.678+0000 7f8c0283d800 20 Filter name: none 2024-05-08T18:05:27.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.678+0000 7f8a26ffd640 20 reqs_thread_entry: start 2024-05-08T18:05:27.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.678+0000 7f8c0283d800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:05:27.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.678+0000 7f8c0283d800 20 rados->read ofs=0 len=0 2024-05-08T18:05:27.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.678+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d2e900 con 0x5614a4ce1b60 2024-05-08T18:05:27.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.679+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(94 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f8bf8040040 con 0x5614a4ce1b60 2024-05-08T18:05:27.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.679+0000 7f8c0283d800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:05:27.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.679+0000 7f8c0283d800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:05:27.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.679+0000 7f8c0283d800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:05:27.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.679+0000 7f8c0283d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:05:27.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.679+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:95 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d2e880 con 0x7f8be8076040 2024-05-08T18:05:27.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.679+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(95 foo.buckets [call out=183b] v0'0 uv43 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f8bf8040040 con 0x7f8be8076040 2024-05-08T18:05:27.678 INFO:teuthology.orchestra.run.smithi050.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-05-08T18:05:27.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.680+0000 7f8ab67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:05:27.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.680+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:96 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94646664386336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4d00490 con 0x5614a4ce1b60 2024-05-08T18:05:27.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.681+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(96 data_loggenerations_metadata [watch unwatch cookie 94646664386336] v27'195 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x5614a4ce1b60 2024-05-08T18:05:27.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.681+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:97 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94646664360592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf56e0 con 0x7f8be8076040 2024-05-08T18:05:27.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.682+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(97 notify.0 [watch unwatch cookie 94646664360592] v27'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x7f8be8076040 2024-05-08T18:05:27.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.682+0000 7f8c0283d800 20 remove_watcher() i=0 2024-05-08T18:05:27.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.682+0000 7f8c0283d800 2 removed watcher, disabling cache 2024-05-08T18:05:27.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.682+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94646664364832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cef290 con 0x5614a4ce1b60 2024-05-08T18:05:27.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.683+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(98 notify.1 [watch unwatch cookie 94646664364832] v27'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x5614a4ce1b60 2024-05-08T18:05:27.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.683+0000 7f8c0283d800 20 remove_watcher() i=1 2024-05-08T18:05:27.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.683+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:99 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94646664358112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cef290 con 0x5614a4ceaf00 2024-05-08T18:05:27.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.684+0000 7f8bf7b34640 1 -- 172.21.15.50:0/2878544332 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(99 notify.2 [watch unwatch cookie 94646664358112] v27'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8be8081070 con 0x5614a4ceaf00 2024-05-08T18:05:27.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.684+0000 7f8c0283d800 20 remove_watcher() i=2 2024-05-08T18:05:27.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.684+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:100 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94646664371440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cee8e0 con 0x7f8be8076040 2024-05-08T18:05:27.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.685+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.2 v2:172.21.15.50:6811/3039646617 20 ==== osd_op_reply(100 notify.3 [watch unwatch cookie 94646664371440] v27'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x7f8be8076040 2024-05-08T18:05:27.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.685+0000 7f8c0283d800 20 remove_watcher() i=3 2024-05-08T18:05:27.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.685+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94646664373696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf1cf0 con 0x5614a4ce1b60 2024-05-08T18:05:27.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.686+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(101 notify.4 [watch unwatch cookie 94646664373696] v27'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf8040040 con 0x5614a4ce1b60 2024-05-08T18:05:27.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.686+0000 7f8c0283d800 20 remove_watcher() i=4 2024-05-08T18:05:27.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.686+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94646664378704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cee8e0 con 0x5614a4ce6550 2024-05-08T18:05:27.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.687+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(102 notify.5 [watch unwatch cookie 94646664378704] v27'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.687+0000 7f8c0283d800 20 remove_watcher() i=5 2024-05-08T18:05:27.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.687+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:103 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94646664382256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf25c0 con 0x5614a4ce6550 2024-05-08T18:05:27.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.688+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(103 notify.6 [watch unwatch cookie 94646664382256] v27'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.688+0000 7f8c0283d800 20 remove_watcher() i=6 2024-05-08T18:05:27.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.688+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:104 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94646664385024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5614a4cf1cf0 con 0x5614a4ce6550 2024-05-08T18:05:27.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c00335640 1 -- 172.21.15.50:0/2878544332 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(104 notify.7 [watch unwatch cookie 94646664385024] v27'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8bf0071030 con 0x5614a4ce6550 2024-05-08T18:05:27.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c0283d800 20 remove_watcher() i=7 2024-05-08T18:05:27.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5614a4ce6550 msgr2=0x5614a4ce8940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c0283d800 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5614a4ce6550 0x5614a4ce8940 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5614a4ce1b60 msgr2=0x5614a4ce4020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c0283d800 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5614a4ce1b60 0x5614a4ce4020 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f8be8076040 msgr2=0x7f8be8083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c0283d800 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f8be8076040 0x7f8be8083410 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5614a4ceaf00 msgr2=0x5614a4ced3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c0283d800 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5614a4ceaf00 0x5614a4ced3c0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f8ad003bb50 msgr2=0x7f8ad003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c0283d800 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f8ad003bb50 0x7f8ad003e010 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8be8052fd0 tx=0x7f8be804f000 comp rx=0 tx=0).stop 2024-05-08T18:05:27.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ccf030 msgr2=0x5614a4cd64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c0283d800 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ccf030 0x5614a4cd64c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0016fd0 tx=0x7f8bf000a340 comp rx=0 tx=0).stop 2024-05-08T18:05:27.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c00b36640 1 -- 172.21.15.50:0/2878544332 reap_dead start 2024-05-08T18:05:27.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 shutdown_connections 2024-05-08T18:05:27.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c0283d800 1 --2- 172.21.15.50:0/2878544332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ccf030 0x5614a4cd64c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.689+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 >> 172.21.15.50:0/2878544332 conn(0x5614a4ccb390 msgr2=0x5614a4cc89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 shutdown_connections 2024-05-08T18:05:27.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 -- 172.21.15.50:0/2878544332 wait complete. 2024-05-08T18:05:27.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 -- 172.21.15.50:0/3504231450 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f8ad803bba0 msgr2=0x7f8ad803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 --2- 172.21.15.50:0/3504231450 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f8ad803bba0 0x7f8ad803e060 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5614a4ca7fc0 tx=0x7f8be8005a00 comp rx=0 tx=0).stop 2024-05-08T18:05:27.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 -- 172.21.15.50:0/3504231450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4cc2100 msgr2=0x5614a4cc2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 --2- 172.21.15.50:0/3504231450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4cc2100 0x5614a4cc2520 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00510b0 tx=0x7f8bf0043500 comp rx=0 tx=0).stop 2024-05-08T18:05:27.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 -- 172.21.15.50:0/3504231450 shutdown_connections 2024-05-08T18:05:27.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 --2- 172.21.15.50:0/3504231450 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f8ad803bba0 0x7f8ad803e060 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 --2- 172.21.15.50:0/3504231450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4cc2100 0x5614a4cc2520 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 -- 172.21.15.50:0/3504231450 >> 172.21.15.50:0/3504231450 conn(0x5614a4cb61a0 msgr2=0x5614a4cb65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 -- 172.21.15.50:0/3504231450 shutdown_connections 2024-05-08T18:05:27.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 -- 172.21.15.50:0/3504231450 wait complete. 2024-05-08T18:05:27.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 -- 172.21.15.50:0/119863861 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5614a4cad3b0 msgr2=0x5614a4caf7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5614a4cad3b0 0x5614a4caf7a0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 -- 172.21.15.50:0/119863861 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5614a4bab9e0 msgr2=0x5614a4caaad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5614a4bab9e0 0x5614a4caaad0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 -- 172.21.15.50:0/119863861 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5614a4cb12a0 msgr2=0x5614a4cb3690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5614a4cb12a0 0x5614a4cb3690 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 -- 172.21.15.50:0/119863861 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f8ae803bb50 msgr2=0x7f8ae803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f8ae803bb50 0x7f8ae803e010 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8be8054000 tx=0x7f8be804e000 comp rx=0 tx=0).stop 2024-05-08T18:05:27.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 -- 172.21.15.50:0/119863861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ba7550 msgr2=0x5614a4ca3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:05:27.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c0283d800 1 --2- 172.21.15.50:0/119863861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614a4ba7550 0x5614a4ca3890 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0000c00 tx=0x7f8bf0000ed0 comp rx=0 tx=0).stop 2024-05-08T18:05:27.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.690+0000 7f8c00b36640 1 -- 172.21.15.50:0/119863861 reap_dead start 2024-05-08T18:05:27.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.691+0000 7f8c0283d800 1 -- 172.21.15.50:0/119863861 shutdown_connections 2024-05-08T18:05:27.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.691+0000 7f8c0283d800 1 -- 172.21.15.50:0/119863861 >> 172.21.15.50:0/119863861 conn(0x5614a4ba2b50 msgr2=0x5614a4c92b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:05:27.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.691+0000 7f8c0283d800 1 -- 172.21.15.50:0/119863861 shutdown_connections 2024-05-08T18:05:27.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:05:27.691+0000 7f8c0283d800 1 -- 172.21.15.50:0/119863861 wait complete. 2024-05-08T18:05:27.697 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-05-08T18:05:27.711 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:05:27.754 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2024-05-08T18:06:02.756 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--categories', 'delete_obj'] 2024-05-08T18:06:02.756 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--categories', 'delete_obj'] 2024-05-08T18:06:02.756 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --categories delete_obj 2024-05-08T18:06:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:02.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.812+0000 7f881a175800 1 Processor -- start 2024-05-08T18:06:02.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.812+0000 7f881a175800 1 -- start start 2024-05-08T18:06:02.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.812+0000 7f881a175800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55cfec1ebc90 0x55cfec1ee080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.812+0000 7f881a175800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55cfec1ee5c0 con 0x55cfec1e8180 2024-05-08T18:06:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.812+0000 7f881a175800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55cfec087e20 con 0x55cfec1ebc90 2024-05-08T18:06:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.813+0000 7f880ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55cfec1ebc90 0x55cfec1ee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.813+0000 7f8817c6d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55cfec1e8180 0x55cfec1ea570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:54508/0 (socket says 172.21.15.50:54508) 2024-05-08T18:06:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.813+0000 7f8817c6d640 1 -- 172.21.15.50:0/3371736638 learned_addr learned my addr 172.21.15.50:0/3371736638 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.813+0000 7f880ffff640 1 -- 172.21.15.50:0/3371736638 >> v1:172.21.15.46:6789/0 conn(0x55cfec1e8180 legacy=0x55cfec1ea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.813+0000 7f880ffff640 1 -- 172.21.15.50:0/3371736638 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfec1ef7a0 con 0x55cfec1ebc90 2024-05-08T18:06:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.815+0000 7f880ffff640 1 --2- 172.21.15.50:0/3371736638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1ebc90 0x55cfec1ee080 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f880000a470 tx=0x7f880001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98b48d9108a48718 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.815+0000 7f881746c640 1 -- 172.21.15.50:0/3371736638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8800057070 con 0x55cfec1ebc90 2024-05-08T18:06:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.815+0000 7f881746c640 1 -- 172.21.15.50:0/3371736638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8800052570 con 0x55cfec1ebc90 2024-05-08T18:06:02.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.815+0000 7f881746c640 1 -- 172.21.15.50:0/3371736638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8800052870 con 0x55cfec1ebc90 2024-05-08T18:06:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.816+0000 7f881a175800 1 -- 172.21.15.50:0/3371736638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1ebc90 msgr2=0x55cfec1ee080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.816+0000 7f881a175800 1 --2- 172.21.15.50:0/3371736638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1ebc90 0x55cfec1ee080 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f880000a470 tx=0x7f880001ce90 comp rx=0 tx=0).stop 2024-05-08T18:06:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.816+0000 7f881a175800 1 -- 172.21.15.50:0/3371736638 shutdown_connections 2024-05-08T18:06:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.816+0000 7f881a175800 1 --2- 172.21.15.50:0/3371736638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1ebc90 0x55cfec1ee080 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.816+0000 7f881a175800 1 -- 172.21.15.50:0/3371736638 >> 172.21.15.50:0/3371736638 conn(0x55cfec1e1b50 msgr2=0x55cfec1e3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:02.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.816+0000 7f881a175800 1 -- 172.21.15.50:0/3371736638 shutdown_connections 2024-05-08T18:06:02.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.816+0000 7f881a175800 1 -- 172.21.15.50:0/3371736638 wait complete. 2024-05-08T18:06:02.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.833+0000 7f881a175800 1 Processor -- start 2024-05-08T18:06:02.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.833+0000 7f881a175800 1 -- start start 2024-05-08T18:06:02.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.833+0000 7f881a175800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1e8180 0x55cfec1e7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.833+0000 7f881a175800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cfec132a10 con 0x55cfec1e8180 2024-05-08T18:06:02.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.834+0000 7f8817c6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1e8180 0x55cfec1e7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.834+0000 7f8817c6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1e8180 0x55cfec1e7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48562/0 (socket says 172.21.15.50:48562) 2024-05-08T18:06:02.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.834+0000 7f8817c6d640 1 -- 172.21.15.50:0/2532218629 learned_addr learned my addr 172.21.15.50:0/2532218629 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:02.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.834+0000 7f8817c6d640 1 -- 172.21.15.50:0/2532218629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfec1e6080 con 0x55cfec1e8180 2024-05-08T18:06:02.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.834+0000 7f8817c6d640 1 --2- 172.21.15.50:0/2532218629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1e8180 0x55cfec1e7ac0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8808001dc0 tx=0x7f880800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ea36613b8a105d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.834+0000 7f8705e0b640 1 -- 172.21.15.50:0/2532218629 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8808012020 con 0x55cfec1e8180 2024-05-08T18:06:02.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.835+0000 7f8705e0b640 1 -- 172.21.15.50:0/2532218629 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880800e470 con 0x55cfec1e8180 2024-05-08T18:06:02.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.835+0000 7f8705e0b640 1 -- 172.21.15.50:0/2532218629 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f880800e7b0 con 0x55cfec1e8180 2024-05-08T18:06:02.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.835+0000 7f881a175800 1 -- 172.21.15.50:0/2532218629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1e8180 msgr2=0x55cfec1e7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.835+0000 7f881a175800 1 --2- 172.21.15.50:0/2532218629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1e8180 0x55cfec1e7ac0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8808001dc0 tx=0x7f880800b6c0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.835+0000 7f881a175800 1 -- 172.21.15.50:0/2532218629 shutdown_connections 2024-05-08T18:06:02.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.835+0000 7f881a175800 1 --2- 172.21.15.50:0/2532218629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1e8180 0x55cfec1e7ac0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.835+0000 7f881a175800 1 -- 172.21.15.50:0/2532218629 >> 172.21.15.50:0/2532218629 conn(0x55cfec1e1b50 msgr2=0x55cfec2d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:02.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.835+0000 7f881a175800 1 -- 172.21.15.50:0/2532218629 shutdown_connections 2024-05-08T18:06:02.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.835+0000 7f881a175800 1 -- 172.21.15.50:0/2532218629 wait complete. 2024-05-08T18:06:02.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.836+0000 7f881a175800 1 Processor -- start 2024-05-08T18:06:02.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.836+0000 7f881a175800 1 -- start start 2024-05-08T18:06:02.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.836+0000 7f881a175800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1e8180 0x55cfec2e28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.836+0000 7f881a175800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cfec1eea90 con 0x55cfec1e8180 2024-05-08T18:06:02.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.836+0000 7f8817c6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1e8180 0x55cfec2e28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.836+0000 7f8817c6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1e8180 0x55cfec2e28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48566/0 (socket says 172.21.15.50:48566) 2024-05-08T18:06:02.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.836+0000 7f8817c6d640 1 -- 172.21.15.50:0/79927911 learned_addr learned my addr 172.21.15.50:0/79927911 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:02.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.837+0000 7f8817c6d640 1 -- 172.21.15.50:0/79927911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfec2e2e00 con 0x55cfec1e8180 2024-05-08T18:06:02.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.837+0000 7f8817c6d640 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1e8180 0x55cfec2e28c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f880800b700 tx=0x7f8808000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.837+0000 7f86fffff640 1 -- 172.21.15.50:0/79927911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8808012030 con 0x55cfec1e8180 2024-05-08T18:06:02.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.837+0000 7f86fffff640 1 -- 172.21.15.50:0/79927911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8808001420 con 0x55cfec1e8180 2024-05-08T18:06:02.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.837+0000 7f881a175800 1 -- 172.21.15.50:0/79927911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cfec2e3090 con 0x55cfec1e8180 2024-05-08T18:06:02.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.837+0000 7f86fffff640 1 -- 172.21.15.50:0/79927911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f880801a7a0 con 0x55cfec1e8180 2024-05-08T18:06:02.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.837+0000 7f881a175800 1 -- 172.21.15.50:0/79927911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfec2e5d40 con 0x55cfec1e8180 2024-05-08T18:06:02.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.838+0000 7f86fffff640 1 -- 172.21.15.50:0/79927911 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f8808015020 con 0x55cfec1e8180 2024-05-08T18:06:02.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.838+0000 7f86fffff640 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f870003bb50 0x7f870003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.838+0000 7f86fffff640 1 -- 172.21.15.50:0/79927911 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f8808020030 con 0x55cfec1e8180 2024-05-08T18:06:02.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.838+0000 7f880ffff640 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f870003bb50 0x7f870003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.839+0000 7f881a175800 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfec1ebc90 0x55cfec2e9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.839+0000 7f881a175800 1 -- 172.21.15.50:0/79927911 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec2ea040 con 0x55cfec1ebc90 2024-05-08T18:06:02.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.839+0000 7f881846e640 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfec1ebc90 0x55cfec2e9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.839+0000 7f880ffff640 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f870003bb50 0x7f870003e010 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8800054000 tx=0x7f880004e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.839+0000 7f881846e640 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfec1ebc90 0x55cfec2e9b00 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.839+0000 7f881846e640 1 -- 172.21.15.50:0/79927911 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f881000a4a0 con 0x55cfec1ebc90 2024-05-08T18:06:02.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.839+0000 7f881a175800 1 -- 172.21.15.50:0/79927911 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec2e70d0 con 0x55cfec1ebc90 2024-05-08T18:06:02.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.840+0000 7f881846e640 1 -- 172.21.15.50:0/79927911 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f881000a4a0 con 0x55cfec1ebc90 2024-05-08T18:06:02.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.840+0000 7f881a175800 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfec2ec350 0x55cfec2ee740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.840+0000 7f881a175800 1 -- 172.21.15.50:0/79927911 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec2eee30 con 0x55cfec2ec350 2024-05-08T18:06:02.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.840+0000 7f8817c6d640 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfec2ec350 0x55cfec2ee740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.840+0000 7f8817c6d640 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfec2ec350 0x55cfec2ee740 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.841+0000 7f8817c6d640 1 -- 172.21.15.50:0/79927911 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f8808043130 con 0x55cfec2ec350 2024-05-08T18:06:02.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.841+0000 7f881a175800 1 -- 172.21.15.50:0/79927911 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec2eb1f0 con 0x55cfec1ebc90 2024-05-08T18:06:02.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.841+0000 7f881846e640 1 -- 172.21.15.50:0/79927911 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f881000a4a0 con 0x55cfec1ebc90 2024-05-08T18:06:02.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.841+0000 7f881a175800 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfec2f0290 0x55cfec2f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.841+0000 7f881a175800 1 -- 172.21.15.50:0/79927911 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec2f2d80 con 0x55cfec2f0290 2024-05-08T18:06:02.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.841+0000 7f880ffff640 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfec2f0290 0x55cfec2f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.841 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.842+0000 7f880ffff640 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfec2f0290 0x55cfec2f2680 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.842+0000 7f880ffff640 1 -- 172.21.15.50:0/79927911 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f8800052a40 con 0x55cfec2f0290 2024-05-08T18:06:02.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.842+0000 7f881a175800 1 Processor -- start 2024-05-08T18:06:02.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.842+0000 7f881a175800 1 -- start start 2024-05-08T18:06:02.844 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.843+0000 7f881a175800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec3010f0 0x55cfec301510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.843+0000 7f881a175800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cfec0d9330 con 0x55cfec3010f0 2024-05-08T18:06:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.843+0000 7f8817c6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec3010f0 0x55cfec301510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.843+0000 7f8817c6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec3010f0 0x55cfec301510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48570/0 (socket says 172.21.15.50:48570) 2024-05-08T18:06:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.843+0000 7f8817c6d640 1 -- 172.21.15.50:0/2406602800 learned_addr learned my addr 172.21.15.50:0/2406602800 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.843+0000 7f8817c6d640 1 -- 172.21.15.50:0/2406602800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfec301a50 con 0x55cfec3010f0 2024-05-08T18:06:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.843+0000 7f8817c6d640 1 --2- 172.21.15.50:0/2406602800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec3010f0 0x55cfec301510 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8808050eb0 tx=0x7f880804a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.844+0000 7f86fd7fa640 1 -- 172.21.15.50:0/2406602800 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f880800cc10 con 0x55cfec3010f0 2024-05-08T18:06:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.844+0000 7f86fd7fa640 1 -- 172.21.15.50:0/2406602800 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8808047020 con 0x55cfec3010f0 2024-05-08T18:06:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.844+0000 7f881a175800 1 -- 172.21.15.50:0/2406602800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cfec3024c0 con 0x55cfec3010f0 2024-05-08T18:06:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.844+0000 7f86fd7fa640 1 -- 172.21.15.50:0/2406602800 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f880800cdb0 con 0x55cfec3010f0 2024-05-08T18:06:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.844+0000 7f881a175800 1 -- 172.21.15.50:0/2406602800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfec3024c0 con 0x55cfec3010f0 2024-05-08T18:06:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.844+0000 7f86fd7fa640 1 -- 172.21.15.50:0/2406602800 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f88080165c0 con 0x55cfec3010f0 2024-05-08T18:06:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.845+0000 7f86fd7fa640 1 --2- 172.21.15.50:0/2406602800 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f86f003bba0 0x7f86f003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.845+0000 7f86fd7fa640 1 -- 172.21.15.50:0/2406602800 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f8808048030 con 0x55cfec3010f0 2024-05-08T18:06:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.845+0000 7f880ffff640 1 --2- 172.21.15.50:0/2406602800 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f86f003bba0 0x7f86f003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.845+0000 7f881a175800 1 Processor -- start 2024-05-08T18:06:02.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.845+0000 7f881a175800 1 -- start start 2024-05-08T18:06:02.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.845+0000 7f880ffff640 1 --2- 172.21.15.50:0/2406602800 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f86f003bba0 0x7f86f003e060 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55cfec2ea900 tx=0x7f8800005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.845+0000 7f881a175800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec30e100 0x55cfec30e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.845+0000 7f881a175800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cfec1e6400 con 0x55cfec30e100 2024-05-08T18:06:02.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.846+0000 7f8817c6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec30e100 0x55cfec30e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.846+0000 7f8817c6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec30e100 0x55cfec30e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48584/0 (socket says 172.21.15.50:48584) 2024-05-08T18:06:02.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.846+0000 7f8817c6d640 1 -- 172.21.15.50:0/3244836210 learned_addr learned my addr 172.21.15.50:0/3244836210 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:02.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.846+0000 7f8817c6d640 1 -- 172.21.15.50:0/3244836210 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfec30eaf0 con 0x55cfec30e100 2024-05-08T18:06:02.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.846+0000 7f8817c6d640 1 --2- 172.21.15.50:0/3244836210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec30e100 0x55cfec30e520 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8808020000 tx=0x7f88080496a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc592d5a53ffde2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.846+0000 7f86f6ffd640 1 -- 172.21.15.50:0/3244836210 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8808061070 con 0x55cfec30e100 2024-05-08T18:06:02.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.846+0000 7f86f6ffd640 1 -- 172.21.15.50:0/3244836210 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f880803d7c0 con 0x55cfec30e100 2024-05-08T18:06:02.846 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.846+0000 7f86f6ffd640 1 -- 172.21.15.50:0/3244836210 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88080168b0 con 0x55cfec30e100 2024-05-08T18:06:02.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.847+0000 7f881a175800 1 -- 172.21.15.50:0/3244836210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec30e100 msgr2=0x55cfec30e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.847+0000 7f881a175800 1 --2- 172.21.15.50:0/3244836210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec30e100 0x55cfec30e520 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8808020000 tx=0x7f88080496a0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.847+0000 7f881a175800 1 -- 172.21.15.50:0/3244836210 shutdown_connections 2024-05-08T18:06:02.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.847+0000 7f881a175800 1 --2- 172.21.15.50:0/3244836210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec30e100 0x55cfec30e520 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.847+0000 7f881a175800 1 -- 172.21.15.50:0/3244836210 >> 172.21.15.50:0/3244836210 conn(0x55cfec30a380 msgr2=0x55cfec3083e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:02.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.847+0000 7f881a175800 1 -- 172.21.15.50:0/3244836210 shutdown_connections 2024-05-08T18:06:02.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.847+0000 7f881a175800 1 -- 172.21.15.50:0/3244836210 wait complete. 2024-05-08T18:06:02.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.847+0000 7f881a175800 1 Processor -- start 2024-05-08T18:06:02.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.848+0000 7f881a175800 1 -- start start 2024-05-08T18:06:02.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.848+0000 7f881a175800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec30e100 0x55cfec315410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.848+0000 7f881a175800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cfec30f160 con 0x55cfec30e100 2024-05-08T18:06:02.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.848+0000 7f8817c6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec30e100 0x55cfec315410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.848+0000 7f8817c6d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec30e100 0x55cfec315410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48594/0 (socket says 172.21.15.50:48594) 2024-05-08T18:06:02.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.848+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 learned_addr learned my addr 172.21.15.50:0/3945948265 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:02.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.848+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfec315950 con 0x55cfec30e100 2024-05-08T18:06:02.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.848+0000 7f8817c6d640 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec30e100 0x55cfec315410 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f880803da70 tx=0x7f8808050410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.849+0000 7f86f57fa640 1 -- 172.21.15.50:0/3945948265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f880806a070 con 0x55cfec30e100 2024-05-08T18:06:02.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.849+0000 7f86f57fa640 1 -- 172.21.15.50:0/3945948265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8808043420 con 0x55cfec30e100 2024-05-08T18:06:02.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.849+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cfec315950 con 0x55cfec30e100 2024-05-08T18:06:02.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.849+0000 7f86f57fa640 1 -- 172.21.15.50:0/3945948265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8808061040 con 0x55cfec30e100 2024-05-08T18:06:02.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.849+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfec315950 con 0x55cfec30e100 2024-05-08T18:06:02.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.850+0000 7f86f57fa640 1 -- 172.21.15.50:0/3945948265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f880806d070 con 0x55cfec30e100 2024-05-08T18:06:02.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.850+0000 7f86f57fa640 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f86e003bb50 0x7f86e003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.850+0000 7f86f57fa640 1 -- 172.21.15.50:0/3945948265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f8808059070 con 0x55cfec30e100 2024-05-08T18:06:02.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.850+0000 7f880ffff640 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f86e003bb50 0x7f86e003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.850+0000 7f881a175800 20 rados->read ofs=0 len=0 2024-05-08T18:06:02.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.850+0000 7f881a175800 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfec320b40 0x55cfec323000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.850+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec323750 con 0x55cfec320b40 2024-05-08T18:06:02.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.850+0000 7f881846e640 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfec320b40 0x55cfec323000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.851+0000 7f880ffff640 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f86e003bb50 0x7f86e003e010 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8800052fd0 tx=0x7f880004f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.851+0000 7f881846e640 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfec320b40 0x55cfec323000 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.851+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f881000a4a0 con 0x55cfec320b40 2024-05-08T18:06:02.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.851+0000 7f881a175800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:02.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.851+0000 7f881a175800 20 realm 2024-05-08T18:06:02.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.851+0000 7f881a175800 20 rados->read ofs=0 len=0 2024-05-08T18:06:02.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.851+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec323b60 con 0x55cfec320b40 2024-05-08T18:06:02.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.851+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f881000a4a0 con 0x55cfec320b40 2024-05-08T18:06:02.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.851+0000 7f881a175800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:02.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.851+0000 7f881a175800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:02.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.851+0000 7f881a175800 20 rados->read ofs=0 len=0 2024-05-08T18:06:02.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.851+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec324430 con 0x55cfec320b40 2024-05-08T18:06:02.855 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.852+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f881000a4a0 con 0x55cfec320b40 2024-05-08T18:06:02.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.852+0000 7f881a175800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:02.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.852+0000 7f881a175800 20 rados->read ofs=0 len=0 2024-05-08T18:06:02.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.852+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3247d0 con 0x55cfec320b40 2024-05-08T18:06:02.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.852+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f881000a4a0 con 0x55cfec320b40 2024-05-08T18:06:02.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.852+0000 7f881a175800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:02.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.852+0000 7f881a175800 20 rados->read ofs=0 len=0 2024-05-08T18:06:02.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.852+0000 7f881a175800 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfec3252c0 0x55cfec3276b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.852+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec327e10 con 0x55cfec3252c0 2024-05-08T18:06:02.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.852+0000 7f8817c6d640 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfec3252c0 0x55cfec3276b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.853+0000 7f8817c6d640 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfec3252c0 0x55cfec3276b0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.853+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.853+0000 7f881a175800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:02.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.853+0000 7f881a175800 20 searching for the correct realm 2024-05-08T18:06:02.856 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.853+0000 7f881a175800 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfec329ec0 0x55cfec32c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.853+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfec32ca50 con 0x55cfec329ec0 2024-05-08T18:06:02.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.853+0000 7f880ffff640 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfec329ec0 0x55cfec32c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.854+0000 7f880ffff640 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfec329ec0 0x55cfec32c380 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.854+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.854+0000 7f880ffff640 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f8800076040 0x7f8800083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.854+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfec32ca50 con 0x7f8800076040 2024-05-08T18:06:02.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.854+0000 7f881846e640 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f8800076040 0x7f8800083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.855+0000 7f881846e640 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f8800076040 0x7f8800083410 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.855+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f881000a4a0 con 0x7f8800076040 2024-05-08T18:06:02.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.855+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3247d0 con 0x55cfec3252c0 2024-05-08T18:06:02.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.855+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.855+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3247d0 con 0x55cfec320b40 2024-05-08T18:06:02.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.856+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f881000a4a0 con 0x55cfec320b40 2024-05-08T18:06:02.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.856+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3247d0 con 0x55cfec320b40 2024-05-08T18:06:02.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.856+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f881000a4a0 con 0x55cfec320b40 2024-05-08T18:06:02.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.856+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3247d0 con 0x55cfec320b40 2024-05-08T18:06:02.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.856+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f881000a4a0 con 0x55cfec320b40 2024-05-08T18:06:02.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.856+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3247d0 con 0x55cfec3252c0 2024-05-08T18:06:02.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.857+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.857+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3247d0 con 0x7f8800076040 2024-05-08T18:06:02.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.857+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f881000a4a0 con 0x7f8800076040 2024-05-08T18:06:02.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.857+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3247d0 con 0x55cfec329ec0 2024-05-08T18:06:02.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.857+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.857+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3247d0 con 0x55cfec320b40 2024-05-08T18:06:02.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.857+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f881000a4a0 con 0x55cfec320b40 2024-05-08T18:06:02.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.857+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3247d0 con 0x55cfec3252c0 2024-05-08T18:06:02.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.858+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.858+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3247d0 con 0x55cfec320b40 2024-05-08T18:06:02.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.858+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f881000a4a0 con 0x55cfec320b40 2024-05-08T18:06:02.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.858+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3247d0 con 0x7f8800076040 2024-05-08T18:06:02.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.858+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f8810032040 con 0x7f8800076040 2024-05-08T18:06:02.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.858+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3247d0 con 0x55cfec329ec0 2024-05-08T18:06:02.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.859+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.859+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x55cfec329ec0 2024-05-08T18:06:02.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.859+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.859+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x55cfec3252c0 2024-05-08T18:06:02.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.859+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.859+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x55cfec329ec0 2024-05-08T18:06:02.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.859+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.859+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x55cfec329ec0 2024-05-08T18:06:02.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.860+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.860+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x55cfec320b40 2024-05-08T18:06:02.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.860+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.860+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x55cfec320b40 2024-05-08T18:06:02.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.860+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.860+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x55cfec3252c0 2024-05-08T18:06:02.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.861+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.869 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.861+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x55cfec329ec0 2024-05-08T18:06:02.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.861+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.861+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x7f8800076040 2024-05-08T18:06:02.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.861+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8810032040 con 0x7f8800076040 2024-05-08T18:06:02.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.861+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x55cfec320b40 2024-05-08T18:06:02.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.861+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.861+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x55cfec3252c0 2024-05-08T18:06:02.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.862+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.862+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x55cfec329ec0 2024-05-08T18:06:02.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.862+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.862+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x55cfec3252c0 2024-05-08T18:06:02.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.862+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.862+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x7f8800076040 2024-05-08T18:06:02.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.863+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8810032040 con 0x7f8800076040 2024-05-08T18:06:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.863+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x55cfec329ec0 2024-05-08T18:06:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.863+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.863+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x55cfec3252c0 2024-05-08T18:06:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.863+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.863+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x55cfec320b40 2024-05-08T18:06:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.864+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.864+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f8800090040 con 0x7f8800076040 2024-05-08T18:06:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.864+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8810032040 con 0x7f8800076040 2024-05-08T18:06:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.864+0000 7f881a175800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.864+0000 7f881a175800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.864+0000 7f881a175800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:02.871 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.864+0000 7f881a175800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.864+0000 7f881a175800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.864+0000 7f881a175800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.864+0000 7f881a175800 20 rados->read ofs=0 len=0 2024-05-08T18:06:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.864+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3287c0 con 0x55cfec320b40 2024-05-08T18:06:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.864+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.864+0000 7f881a175800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.864+0000 7f881a175800 20 rados->read ofs=0 len=0 2024-05-08T18:06:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.864+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3287c0 con 0x55cfec320b40 2024-05-08T18:06:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f881a175800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f881a175800 20 rados->read ofs=0 len=0 2024-05-08T18:06:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec328790 con 0x55cfec329ec0 2024-05-08T18:06:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.872 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f881a175800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f881a175800 20 zone default found 2024-05-08T18:06:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f881a175800 4 Realm: () 2024-05-08T18:06:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f881a175800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f881a175800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f881a175800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f881a175800 20 zonegroup default 2024-05-08T18:06:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f881a175800 20 rados->read ofs=0 len=0 2024-05-08T18:06:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec329060 con 0x55cfec3252c0 2024-05-08T18:06:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f881a175800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f881a175800 20 rados->read ofs=0 len=0 2024-05-08T18:06:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.865+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec32e8b0 con 0x55cfec329ec0 2024-05-08T18:06:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.866+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.873 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.866+0000 7f881a175800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.866+0000 7f881a175800 20 rados->read ofs=0 len=0 2024-05-08T18:06:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.866+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec32ec50 con 0x55cfec329ec0 2024-05-08T18:06:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.866+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.866+0000 7f881a175800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.866+0000 7f881a175800 20 started sync module instance, tier type = 2024-05-08T18:06:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.866+0000 7f881a175800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.866+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec32dd50 con 0x7f8800076040 2024-05-08T18:06:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.867+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x7f8800076040 2024-05-08T18:06:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.867+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94351509348944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec32dd50 con 0x7f8800076040 2024-05-08T18:06:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.867+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec32f610 con 0x55cfec320b40 2024-05-08T18:06:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.868+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94351509348944] v29'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x7f8800076040 2024-05-08T18:06:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.869+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.869+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94351509353280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec32dd50 con 0x55cfec320b40 2024-05-08T18:06:02.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.869+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec330090 con 0x55cfec329ec0 2024-05-08T18:06:02.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.870+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94351509353280] v29'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.870+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.870+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94351509346464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec32dcc0 con 0x55cfec329ec0 2024-05-08T18:06:02.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.870+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec330f50 con 0x7f8800076040 2024-05-08T18:06:02.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.871+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x7f8800076040 2024-05-08T18:06:02.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.871+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94351509346464] v29'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.871+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94351509359776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec331340 con 0x7f8800076040 2024-05-08T18:06:02.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.871+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec331d30 con 0x55cfec320b40 2024-05-08T18:06:02.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.873+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94351509359776] v29'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x7f8800076040 2024-05-08T18:06:02.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.873+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.873+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94351509357696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec331d30 con 0x55cfec320b40 2024-05-08T18:06:02.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.873+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec332740 con 0x55cfec3252c0 2024-05-08T18:06:02.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.874+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94351509357696] v29'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.874+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.874+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94351509362816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec331ca0 con 0x55cfec3252c0 2024-05-08T18:06:02.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.874+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3334e0 con 0x55cfec3252c0 2024-05-08T18:06:02.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.875+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94351509362816] v29'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.875+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.875+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94351509367600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec333d30 con 0x55cfec3252c0 2024-05-08T18:06:02.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.876+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3346f0 con 0x55cfec3252c0 2024-05-08T18:06:02.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.877+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94351509367600] v29'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.877+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.877+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94351509371520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3346f0 con 0x55cfec3252c0 2024-05-08T18:06:02.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.877+0000 7f881a175800 20 add_watcher() i=0 2024-05-08T18:06:02.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.877+0000 7f881a175800 20 add_watcher() i=1 2024-05-08T18:06:02.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.877+0000 7f881a175800 20 add_watcher() i=2 2024-05-08T18:06:02.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.877+0000 7f881a175800 20 add_watcher() i=3 2024-05-08T18:06:02.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.877+0000 7f881a175800 20 add_watcher() i=4 2024-05-08T18:06:02.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.877+0000 7f881a175800 20 add_watcher() i=5 2024-05-08T18:06:02.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.877+0000 7f881a175800 20 add_watcher() i=6 2024-05-08T18:06:02.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.878+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94351509371520] v29'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.878+0000 7f881a175800 20 add_watcher() i=7 2024-05-08T18:06:02.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.878+0000 7f881a175800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:02.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.878+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec337a40 con 0x55cfec320b40 2024-05-08T18:06:02.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.878+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.879+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94351509373872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec334c80 con 0x55cfec320b40 2024-05-08T18:06:02.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.880+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94351509373872] v29'196 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.880+0000 7f86cdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:02.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.880+0000 7f881a175800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:02.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.880+0000 7f881a175800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:02.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.880+0000 7f881a175800 5 note: GC not initialized 2024-05-08T18:06:02.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.880+0000 7f86ccfe1640 20 reqs_thread_entry: start 2024-05-08T18:06:02.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.880+0000 7f881a175800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:02.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.880+0000 7f881a175800 20 Filter name: none 2024-05-08T18:06:02.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.880+0000 7f86467fc640 20 reqs_thread_entry: start 2024-05-08T18:06:02.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.880+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36b570 con 0x55cfec320b40 2024-05-08T18:06:02.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.881+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.881+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36b570 con 0x7f8800076040 2024-05-08T18:06:02.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.881+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f8810032040 con 0x7f8800076040 2024-05-08T18:06:02.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.881+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36b530 con 0x55cfec320b40 2024-05-08T18:06:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.881+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.881+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36b530 con 0x55cfec329ec0 2024-05-08T18:06:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.881+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.882+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36ba10 con 0x55cfec329ec0 2024-05-08T18:06:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.882+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.882+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36ba10 con 0x55cfec329ec0 2024-05-08T18:06:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.882+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.882+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36ba10 con 0x55cfec320b40 2024-05-08T18:06:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.882+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.882+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36ba10 con 0x55cfec320b40 2024-05-08T18:06:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.883+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.883+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36b990 con 0x55cfec329ec0 2024-05-08T18:06:02.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.883+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.883+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36b990 con 0x55cfec329ec0 2024-05-08T18:06:02.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.883+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.883+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36b990 con 0x55cfec3252c0 2024-05-08T18:06:02.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.884+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(72 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.884+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec320b40 2024-05-08T18:06:02.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.884+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.885+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x7f8800076040 2024-05-08T18:06:02.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.885+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x7f8800076040 2024-05-08T18:06:02.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.885+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec329ec0 2024-05-08T18:06:02.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.885+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.885+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec3252c0 2024-05-08T18:06:02.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.885+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.885+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec3252c0 2024-05-08T18:06:02.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.886+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.886+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec329ec0 2024-05-08T18:06:02.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.886+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.886+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec3252c0 2024-05-08T18:06:02.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.886+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(79 usage.17 [call out=185b] v0'0 uv6 ondisk = 0) ==== 152+0+185 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.886+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec320b40 2024-05-08T18:06:02.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.887+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.887+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec329ec0 2024-05-08T18:06:02.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.887+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.887+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec3252c0 2024-05-08T18:06:02.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.887+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.887+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x7f8800076040 2024-05-08T18:06:02.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.887+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x7f8800076040 2024-05-08T18:06:02.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.887+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec3252c0 2024-05-08T18:06:02.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.888+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.888+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec320b40 2024-05-08T18:06:02.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.888+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.888+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec320b40 2024-05-08T18:06:02.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.888+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.888+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec329ec0 2024-05-08T18:06:02.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.888+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.889+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec320b40 2024-05-08T18:06:02.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.889+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.889+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec320b40 2024-05-08T18:06:02.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.889+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.889+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec329ec0 2024-05-08T18:06:02.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.889+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.889+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec329ec0 2024-05-08T18:06:02.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.889+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.890+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec329ec0 2024-05-08T18:06:02.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.890+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.890+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfec36d0e0 con 0x55cfec3252c0 2024-05-08T18:06:02.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.890+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.891+0000 7f86cdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:02.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.891+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94351509373872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3367c0 con 0x55cfec320b40 2024-05-08T18:06:02.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.892+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 32 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94351509373872] v29'197 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.892+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94351509348944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3343b0 con 0x7f8800076040 2024-05-08T18:06:02.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.893+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94351509348944] v29'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x7f8800076040 2024-05-08T18:06:02.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.893+0000 7f881a175800 20 remove_watcher() i=0 2024-05-08T18:06:02.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.893+0000 7f881a175800 2 removed watcher, disabling cache 2024-05-08T18:06:02.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.893+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94351509353280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3343b0 con 0x55cfec320b40 2024-05-08T18:06:02.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.895+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 33 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94351509353280] v29'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.895+0000 7f881a175800 20 remove_watcher() i=1 2024-05-08T18:06:02.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.895+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94351509346464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3343b0 con 0x55cfec329ec0 2024-05-08T18:06:02.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.896+0000 7f880ffff640 1 -- 172.21.15.50:0/3945948265 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94351509346464] v29'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8800081070 con 0x55cfec329ec0 2024-05-08T18:06:02.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.896+0000 7f881a175800 20 remove_watcher() i=2 2024-05-08T18:06:02.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.896+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94351509359776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3343b0 con 0x7f8800076040 2024-05-08T18:06:02.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.897+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94351509359776] v29'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x7f8800076040 2024-05-08T18:06:02.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.897+0000 7f881a175800 20 remove_watcher() i=3 2024-05-08T18:06:02.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.897+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94351509357696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec330ca0 con 0x55cfec320b40 2024-05-08T18:06:02.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.898+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 <== osd.1 v2:172.21.15.50:6808/2141923372 34 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94351509357696] v29'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8810032040 con 0x55cfec320b40 2024-05-08T18:06:02.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.898+0000 7f881a175800 20 remove_watcher() i=4 2024-05-08T18:06:02.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.898+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94351509362816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec3343b0 con 0x55cfec3252c0 2024-05-08T18:06:02.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.900+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94351509362816] v29'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.900+0000 7f881a175800 20 remove_watcher() i=5 2024-05-08T18:06:02.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.900+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94351509367600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec330ca0 con 0x55cfec3252c0 2024-05-08T18:06:02.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.901+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94351509367600] v29'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.901+0000 7f881a175800 20 remove_watcher() i=6 2024-05-08T18:06:02.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.901+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94351509371520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfec331880 con 0x55cfec3252c0 2024-05-08T18:06:02.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f8817c6d640 1 -- 172.21.15.50:0/3945948265 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94351509371520] v29'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8808076070 con 0x55cfec3252c0 2024-05-08T18:06:02.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 20 remove_watcher() i=7 2024-05-08T18:06:02.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfec3252c0 msgr2=0x55cfec3276b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfec3252c0 0x55cfec3276b0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfec320b40 msgr2=0x55cfec323000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfec320b40 0x55cfec323000 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f8800076040 msgr2=0x7f8800083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f8800076040 0x7f8800083410 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfec329ec0 msgr2=0x55cfec32c380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfec329ec0 0x55cfec32c380 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f86e003bb50 msgr2=0x7f86e003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f86e003bb50 0x7f86e003e010 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8800052fd0 tx=0x7f880004f000 comp rx=0 tx=0).stop 2024-05-08T18:06:02.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec30e100 msgr2=0x55cfec315410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 --2- 172.21.15.50:0/3945948265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec30e100 0x55cfec315410 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f880803da70 tx=0x7f8808050410 comp rx=0 tx=0).stop 2024-05-08T18:06:02.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 reap_dead start 2024-05-08T18:06:02.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881846e640 1 -- 172.21.15.50:0/3945948265 reap_dead start 2024-05-08T18:06:02.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 shutdown_connections 2024-05-08T18:06:02.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 >> 172.21.15.50:0/3945948265 conn(0x55cfec30a380 msgr2=0x55cfec3079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:02.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 shutdown_connections 2024-05-08T18:06:02.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 -- 172.21.15.50:0/3945948265 wait complete. 2024-05-08T18:06:02.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 -- 172.21.15.50:0/2406602800 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f86f003bba0 msgr2=0x7f86f003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 --2- 172.21.15.50:0/2406602800 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f86f003bba0 0x7f86f003e060 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x55cfec2ea900 tx=0x7f8800005a00 comp rx=0 tx=0).stop 2024-05-08T18:06:02.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 -- 172.21.15.50:0/2406602800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec3010f0 msgr2=0x55cfec301510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.902+0000 7f881a175800 1 --2- 172.21.15.50:0/2406602800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec3010f0 0x55cfec301510 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8808050eb0 tx=0x7f880804a1f0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 -- 172.21.15.50:0/2406602800 shutdown_connections 2024-05-08T18:06:02.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 --2- 172.21.15.50:0/2406602800 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f86f003bba0 0x7f86f003e060 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 --2- 172.21.15.50:0/2406602800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec3010f0 0x55cfec301510 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 -- 172.21.15.50:0/2406602800 >> 172.21.15.50:0/2406602800 conn(0x55cfec2f5190 msgr2=0x55cfec2f5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:02.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 -- 172.21.15.50:0/2406602800 shutdown_connections 2024-05-08T18:06:02.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 -- 172.21.15.50:0/2406602800 wait complete. 2024-05-08T18:06:02.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 -- 172.21.15.50:0/79927911 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfec2ec350 msgr2=0x55cfec2ee740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfec2ec350 0x55cfec2ee740 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 -- 172.21.15.50:0/79927911 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfec1ebc90 msgr2=0x55cfec2e9b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfec1ebc90 0x55cfec2e9b00 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 -- 172.21.15.50:0/79927911 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfec2f0290 msgr2=0x55cfec2f2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfec2f0290 0x55cfec2f2680 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 -- 172.21.15.50:0/79927911 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f870003bb50 msgr2=0x7f870003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f870003bb50 0x7f870003e010 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8800054000 tx=0x7f880004e000 comp rx=0 tx=0).stop 2024-05-08T18:06:02.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 -- 172.21.15.50:0/79927911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1e8180 msgr2=0x55cfec2e28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 --2- 172.21.15.50:0/79927911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfec1e8180 0x55cfec2e28c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f880800b700 tx=0x7f8808000bc0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881846e640 1 -- 172.21.15.50:0/79927911 reap_dead start 2024-05-08T18:06:02.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 -- 172.21.15.50:0/79927911 shutdown_connections 2024-05-08T18:06:02.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 -- 172.21.15.50:0/79927911 >> 172.21.15.50:0/79927911 conn(0x55cfec1e1b50 msgr2=0x55cfec1e6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:02.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 -- 172.21.15.50:0/79927911 shutdown_connections 2024-05-08T18:06:02.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.903+0000 7f881a175800 1 -- 172.21.15.50:0/79927911 wait complete. 2024-05-08T18:06:02.910 INFO:teuthology.orchestra.run.smithi050.stdout:{"entries":[{"user":"anonymous","buckets":[{"bucket":"","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"anonymous","categories":[],"s3select":{}}]},{"user":"foo","buckets":[{"bucket":"","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}],"s3select":{}},{"bucket":"myfoo2","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo3","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo4","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[],"s3select":{}}]}],"summary":[{"user":"anonymous","categories":[],"total":{"bytes_sent":0,"bytes_received":0,"ops":0,"successful_ops":0,"bytes_processed":0,"bytes_returned":0}},{"user":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}],"total":{"bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4,"bytes_processed":0,"bytes_returned":0}}]} 2024-05-08T18:06:02.911 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'anonymous', 'categories': [], 's3select': {}}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 's3select': {}}, {'bucket': 'myfoo2', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo3', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo4', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [], 's3select': {}}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-05-08T18:06:02.911 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2024-05-08T18:06:02.911 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2024-05-08T18:06:02.911 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2024-05-08T18:06:02.955 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:02.955 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:02.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.958+0000 7f6eb03af800 1 Processor -- start 2024-05-08T18:06:02.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.958+0000 7f6eb03af800 1 -- start start 2024-05-08T18:06:02.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.958+0000 7f6eb03af800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5630f55e20d0 0x5630f55e44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.958+0000 7f6eb03af800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5630f55e8510 con 0x5630f55e4a00 2024-05-08T18:06:02.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.958+0000 7f6eb03af800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5630f5481e20 con 0x5630f55e20d0 2024-05-08T18:06:02.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.958+0000 7f6eadeae640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5630f55e20d0 0x5630f55e44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.958+0000 7f6eadeae640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5630f55e20d0 0x5630f55e44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48610/0 (socket says 172.21.15.50:48610) 2024-05-08T18:06:02.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.958+0000 7f6ead6ad640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5630f55e4a00 0x5630f55e6df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:54520/0 (socket says 172.21.15.50:54520) 2024-05-08T18:06:02.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.958+0000 7f6ead6ad640 1 -- 172.21.15.50:0/1343074906 learned_addr learned my addr 172.21.15.50:0/1343074906 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:02.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.959+0000 7f6eadeae640 1 -- 172.21.15.50:0/1343074906 >> v1:172.21.15.46:6789/0 conn(0x5630f55e4a00 legacy=0x5630f55e6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.959+0000 7f6eadeae640 1 -- 172.21.15.50:0/1343074906 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630f55e96f0 con 0x5630f55e20d0 2024-05-08T18:06:02.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.960+0000 7f6eadeae640 1 --2- 172.21.15.50:0/1343074906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 0x5630f55e44c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000a470 tx=0x7f6ea001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0b0119b2c065f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.961+0000 7f6eaceac640 1 -- 172.21.15.50:0/1343074906 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ea0057070 con 0x5630f55e20d0 2024-05-08T18:06:02.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.961+0000 7f6eaceac640 1 -- 172.21.15.50:0/1343074906 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea0052570 con 0x5630f55e20d0 2024-05-08T18:06:02.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.961+0000 7f6eaceac640 1 -- 172.21.15.50:0/1343074906 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ea0052870 con 0x5630f55e20d0 2024-05-08T18:06:02.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.961+0000 7f6eb03af800 1 -- 172.21.15.50:0/1343074906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 msgr2=0x5630f55e44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.961+0000 7f6eb03af800 1 --2- 172.21.15.50:0/1343074906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 0x5630f55e44c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6ea000a470 tx=0x7f6ea001ce90 comp rx=0 tx=0).stop 2024-05-08T18:06:02.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.961+0000 7f6eb03af800 1 -- 172.21.15.50:0/1343074906 shutdown_connections 2024-05-08T18:06:02.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.961+0000 7f6eb03af800 1 --2- 172.21.15.50:0/1343074906 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 0x5630f55e44c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.961+0000 7f6eb03af800 1 -- 172.21.15.50:0/1343074906 >> 172.21.15.50:0/1343074906 conn(0x5630f55dbb40 msgr2=0x5630f55ddf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:02.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.961+0000 7f6eb03af800 1 -- 172.21.15.50:0/1343074906 shutdown_connections 2024-05-08T18:06:02.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.961+0000 7f6eb03af800 1 -- 172.21.15.50:0/1343074906 wait complete. 2024-05-08T18:06:02.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.977+0000 7f6eb03af800 1 Processor -- start 2024-05-08T18:06:02.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.977+0000 7f6eb03af800 1 -- start start 2024-05-08T18:06:02.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.977+0000 7f6eb03af800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 0x5630f56d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.977+0000 7f6eb03af800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5630f552ca00 con 0x5630f55e20d0 2024-05-08T18:06:02.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.977+0000 7f6eadeae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 0x5630f56d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.977+0000 7f6eadeae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 0x5630f56d2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48620/0 (socket says 172.21.15.50:48620) 2024-05-08T18:06:02.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.977+0000 7f6eadeae640 1 -- 172.21.15.50:0/1173733029 learned_addr learned my addr 172.21.15.50:0/1173733029 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:02.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.978+0000 7f6eadeae640 1 -- 172.21.15.50:0/1173733029 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630f56d30d0 con 0x5630f55e20d0 2024-05-08T18:06:02.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.978+0000 7f6eadeae640 1 --2- 172.21.15.50:0/1173733029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 0x5630f56d2b00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0055040 tx=0x7f6ea004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf27224fc7e1ac64 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.978+0000 7f6d9be07640 1 -- 172.21.15.50:0/1173733029 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ea0057070 con 0x5630f55e20d0 2024-05-08T18:06:02.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.978+0000 7f6d9be07640 1 -- 172.21.15.50:0/1173733029 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea0063070 con 0x5630f55e20d0 2024-05-08T18:06:02.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.978+0000 7f6d9be07640 1 -- 172.21.15.50:0/1173733029 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ea0052720 con 0x5630f55e20d0 2024-05-08T18:06:02.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.979+0000 7f6eb03af800 1 -- 172.21.15.50:0/1173733029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 msgr2=0x5630f56d2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.979+0000 7f6eb03af800 1 --2- 172.21.15.50:0/1173733029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 0x5630f56d2b00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0055040 tx=0x7f6ea004c000 comp rx=0 tx=0).stop 2024-05-08T18:06:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.979+0000 7f6eb03af800 1 -- 172.21.15.50:0/1173733029 shutdown_connections 2024-05-08T18:06:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.979+0000 7f6eb03af800 1 --2- 172.21.15.50:0/1173733029 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 0x5630f56d2b00 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.979+0000 7f6eb03af800 1 -- 172.21.15.50:0/1173733029 >> 172.21.15.50:0/1173733029 conn(0x5630f55dbb40 msgr2=0x5630f56ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.979+0000 7f6eb03af800 1 -- 172.21.15.50:0/1173733029 shutdown_connections 2024-05-08T18:06:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.979+0000 7f6eb03af800 1 -- 172.21.15.50:0/1173733029 wait complete. 2024-05-08T18:06:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.979+0000 7f6eb03af800 1 Processor -- start 2024-05-08T18:06:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.979+0000 7f6eb03af800 1 -- start start 2024-05-08T18:06:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.979+0000 7f6eb03af800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 0x5630f56da580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.979+0000 7f6eb03af800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5630f55e8980 con 0x5630f55e20d0 2024-05-08T18:06:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.979+0000 7f6eadeae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 0x5630f56da580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.979+0000 7f6eadeae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 0x5630f56da580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48628/0 (socket says 172.21.15.50:48628) 2024-05-08T18:06:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.979+0000 7f6eadeae640 1 -- 172.21.15.50:0/604906017 learned_addr learned my addr 172.21.15.50:0/604906017 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.980+0000 7f6eadeae640 1 -- 172.21.15.50:0/604906017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630f56daac0 con 0x5630f55e20d0 2024-05-08T18:06:02.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.980+0000 7f6eadeae640 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 0x5630f56da580 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0060040 tx=0x7f6ea0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.980+0000 7f6d9a604640 1 -- 172.21.15.50:0/604906017 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ea0052e30 con 0x5630f55e20d0 2024-05-08T18:06:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.980+0000 7f6d9a604640 1 -- 172.21.15.50:0/604906017 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea006c070 con 0x5630f55e20d0 2024-05-08T18:06:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.980+0000 7f6eb03af800 1 -- 172.21.15.50:0/604906017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5630f56dad50 con 0x5630f55e20d0 2024-05-08T18:06:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.980+0000 7f6d9a604640 1 -- 172.21.15.50:0/604906017 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ea005e5f0 con 0x5630f55e20d0 2024-05-08T18:06:02.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.980+0000 7f6eb03af800 1 -- 172.21.15.50:0/604906017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630f56dda00 con 0x5630f55e20d0 2024-05-08T18:06:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.981+0000 7f6d9a604640 1 -- 172.21.15.50:0/604906017 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6ea005e790 con 0x5630f55e20d0 2024-05-08T18:06:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.981+0000 7f6d9a604640 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6d9403bb50 0x7f6d9403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.981+0000 7f6d9a604640 1 -- 172.21.15.50:0/604906017 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f6ea0094e00 con 0x5630f55e20d0 2024-05-08T18:06:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.981+0000 7f6ead6ad640 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6d9403bb50 0x7f6d9403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.981+0000 7f6eb03af800 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5630f55e4a00 0x5630f56e17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.981+0000 7f6eb03af800 1 -- 172.21.15.50:0/604906017 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f56e1d40 con 0x5630f55e4a00 2024-05-08T18:06:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.982+0000 7f6eae6af640 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5630f55e4a00 0x5630f56e17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.982+0000 7f6ead6ad640 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6d9403bb50 0x7f6d9403e010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4001d70 tx=0x7f6ea40012d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.982+0000 7f6eae6af640 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5630f55e4a00 0x5630f56e17c0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.982+0000 7f6eae6af640 1 -- 172.21.15.50:0/604906017 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6ea800a4a0 con 0x5630f55e4a00 2024-05-08T18:06:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.982+0000 7f6eb03af800 1 -- 172.21.15.50:0/604906017 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f56ded70 con 0x5630f55e4a00 2024-05-08T18:06:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.982+0000 7f6eae6af640 1 -- 172.21.15.50:0/604906017 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6ea800a4a0 con 0x5630f55e4a00 2024-05-08T18:06:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.982+0000 7f6eb03af800 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5630f56e4070 0x5630f56e6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.982+0000 7f6eb03af800 1 -- 172.21.15.50:0/604906017 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f56e6b50 con 0x5630f56e4070 2024-05-08T18:06:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.983+0000 7f6eadeae640 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5630f56e4070 0x5630f56e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.983+0000 7f6eadeae640 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5630f56e4070 0x5630f56e6460 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.984+0000 7f6eadeae640 1 -- 172.21.15.50:0/604906017 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f56e4070 2024-05-08T18:06:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.984+0000 7f6eb03af800 1 -- 172.21.15.50:0/604906017 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f56e2ee0 con 0x5630f55e4a00 2024-05-08T18:06:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.984+0000 7f6eae6af640 1 -- 172.21.15.50:0/604906017 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6ea800a4a0 con 0x5630f55e4a00 2024-05-08T18:06:02.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.984+0000 7f6eb03af800 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5630f56e7fb0 0x5630f56ea3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.984+0000 7f6eb03af800 1 -- 172.21.15.50:0/604906017 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f56eaa90 con 0x5630f56e7fb0 2024-05-08T18:06:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.984+0000 7f6ead6ad640 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5630f56e7fb0 0x5630f56ea3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.984+0000 7f6ead6ad640 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5630f56e7fb0 0x5630f56ea3a0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.985+0000 7f6ead6ad640 1 -- 172.21.15.50:0/604906017 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f56e7fb0 2024-05-08T18:06:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.985+0000 7f6eb03af800 1 Processor -- start 2024-05-08T18:06:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.985+0000 7f6eb03af800 1 -- start start 2024-05-08T18:06:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.985+0000 7f6eb03af800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f56f8dc0 0x5630f56f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.985+0000 7f6eb03af800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5630f54d3330 con 0x5630f56f8dc0 2024-05-08T18:06:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.985+0000 7f6eadeae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f56f8dc0 0x5630f56f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.985+0000 7f6eadeae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f56f8dc0 0x5630f56f91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48630/0 (socket says 172.21.15.50:48630) 2024-05-08T18:06:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.985+0000 7f6eadeae640 1 -- 172.21.15.50:0/728785759 learned_addr learned my addr 172.21.15.50:0/728785759 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.986+0000 7f6eadeae640 1 -- 172.21.15.50:0/728785759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630f56f9720 con 0x5630f56f8dc0 2024-05-08T18:06:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.986+0000 7f6eadeae640 1 --2- 172.21.15.50:0/728785759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f56f8dc0 0x5630f56f91e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0093180 tx=0x7f6ea005ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:02.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.986+0000 7f6d937fe640 1 -- 172.21.15.50:0/728785759 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ea008b070 con 0x5630f56f8dc0 2024-05-08T18:06:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.986+0000 7f6d937fe640 1 -- 172.21.15.50:0/728785759 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea007f070 con 0x5630f56f8dc0 2024-05-08T18:06:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.986+0000 7f6d937fe640 1 -- 172.21.15.50:0/728785759 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ea0009830 con 0x5630f56f8dc0 2024-05-08T18:06:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.986+0000 7f6eb03af800 1 -- 172.21.15.50:0/728785759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5630f56fa190 con 0x5630f56f8dc0 2024-05-08T18:06:02.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.986+0000 7f6eb03af800 1 -- 172.21.15.50:0/728785759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630f56fa690 con 0x5630f56f8dc0 2024-05-08T18:06:02.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.987+0000 7f6d937fe640 1 -- 172.21.15.50:0/728785759 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6ea008c260 con 0x5630f56f8dc0 2024-05-08T18:06:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.987+0000 7f6d937fe640 1 --2- 172.21.15.50:0/728785759 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6d8803bb50 0x7f6d8803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.987+0000 7f6d937fe640 1 -- 172.21.15.50:0/728785759 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f6ea009bd30 con 0x5630f56f8dc0 2024-05-08T18:06:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.987+0000 7f6ead6ad640 1 --2- 172.21.15.50:0/728785759 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6d8803bb50 0x7f6d8803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.987+0000 7f6eb03af800 1 Processor -- start 2024-05-08T18:06:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.988+0000 7f6eb03af800 1 -- start start 2024-05-08T18:06:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.988+0000 7f6ead6ad640 1 --2- 172.21.15.50:0/728785759 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6d8803bb50 0x7f6d8803e010 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400bf40 tx=0x7f6ea400f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.988+0000 7f6eb03af800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f5705ee0 0x5630f5706300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.988+0000 7f6eb03af800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5630f56d3390 con 0x5630f5705ee0 2024-05-08T18:06:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.988+0000 7f6eadeae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f5705ee0 0x5630f5706300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.988+0000 7f6eadeae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f5705ee0 0x5630f5706300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48646/0 (socket says 172.21.15.50:48646) 2024-05-08T18:06:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.988+0000 7f6eadeae640 1 -- 172.21.15.50:0/4238125278 learned_addr learned my addr 172.21.15.50:0/4238125278 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.988+0000 7f6eadeae640 1 -- 172.21.15.50:0/4238125278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630f57068d0 con 0x5630f5705ee0 2024-05-08T18:06:03.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.988+0000 7f6eadeae640 1 --2- 172.21.15.50:0/4238125278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f5705ee0 0x5630f5706300 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6ea007a7e0 tx=0x7f6ea0086f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccf5f4979b459d05 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.988+0000 7f6d917fa640 1 -- 172.21.15.50:0/4238125278 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ea00a9020 con 0x5630f5705ee0 2024-05-08T18:06:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.988+0000 7f6d917fa640 1 -- 172.21.15.50:0/4238125278 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea0058040 con 0x5630f5705ee0 2024-05-08T18:06:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.988+0000 7f6d917fa640 1 -- 172.21.15.50:0/4238125278 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ea0071df0 con 0x5630f5705ee0 2024-05-08T18:06:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.989+0000 7f6eb03af800 1 -- 172.21.15.50:0/4238125278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f5705ee0 msgr2=0x5630f5706300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.989+0000 7f6eb03af800 1 --2- 172.21.15.50:0/4238125278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f5705ee0 0x5630f5706300 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6ea007a7e0 tx=0x7f6ea0086f60 comp rx=0 tx=0).stop 2024-05-08T18:06:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.989+0000 7f6eb03af800 1 -- 172.21.15.50:0/4238125278 shutdown_connections 2024-05-08T18:06:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.989+0000 7f6eb03af800 1 --2- 172.21.15.50:0/4238125278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f5705ee0 0x5630f5706300 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.989+0000 7f6eb03af800 1 -- 172.21.15.50:0/4238125278 >> 172.21.15.50:0/4238125278 conn(0x5630f57021f0 msgr2=0x5630f5700250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.989+0000 7f6eb03af800 1 -- 172.21.15.50:0/4238125278 shutdown_connections 2024-05-08T18:06:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.989+0000 7f6eb03af800 1 -- 172.21.15.50:0/4238125278 wait complete. 2024-05-08T18:06:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.989+0000 7f6eb03af800 1 Processor -- start 2024-05-08T18:06:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.989+0000 7f6eb03af800 1 -- start start 2024-05-08T18:06:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.990+0000 7f6eb03af800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f5705ee0 0x5630f570d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.990+0000 7f6eb03af800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5630f5706f60 con 0x5630f5705ee0 2024-05-08T18:06:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.990+0000 7f6eadeae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f5705ee0 0x5630f570d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.990+0000 7f6eadeae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f5705ee0 0x5630f570d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48654/0 (socket says 172.21.15.50:48654) 2024-05-08T18:06:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.990+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 learned_addr learned my addr 172.21.15.50:0/303358253 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.990+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630f570d860 con 0x5630f5705ee0 2024-05-08T18:06:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.990+0000 7f6eadeae640 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f5705ee0 0x5630f570d320 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6ea005bd20 tx=0x7f6ea005bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.990+0000 7f6d7f7fe640 1 -- 172.21.15.50:0/303358253 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ea008cb60 con 0x5630f5705ee0 2024-05-08T18:06:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.990+0000 7f6d7f7fe640 1 -- 172.21.15.50:0/303358253 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ea008cd00 con 0x5630f5705ee0 2024-05-08T18:06:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.990+0000 7f6d7f7fe640 1 -- 172.21.15.50:0/303358253 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ea0086480 con 0x5630f5705ee0 2024-05-08T18:06:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.991+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5630f570d860 con 0x5630f5705ee0 2024-05-08T18:06:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.991+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630f570d860 con 0x5630f5705ee0 2024-05-08T18:06:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.991+0000 7f6d7f7fe640 1 -- 172.21.15.50:0/303358253 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6ea00ac020 con 0x5630f5705ee0 2024-05-08T18:06:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.992+0000 7f6d7f7fe640 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6d7803bb50 0x7f6d7803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.992+0000 7f6d7f7fe640 1 -- 172.21.15.50:0/303358253 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f6ea00b5050 con 0x5630f5705ee0 2024-05-08T18:06:03.002 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.992+0000 7f6ead6ad640 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6d7803bb50 0x7f6d7803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.992+0000 7f6eb03af800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.992+0000 7f6eb03af800 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5630f5718a40 0x5630f571af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.992+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f571b650 con 0x5630f5718a40 2024-05-08T18:06:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.992+0000 7f6eae6af640 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5630f5718a40 0x5630f571af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.992+0000 7f6ead6ad640 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6d7803bb50 0x7f6d7803e010 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400c010 tx=0x7f6ea4010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.992+0000 7f6eae6af640 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5630f5718a40 0x5630f571af00 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.993+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6ea800a4a0 con 0x5630f5718a40 2024-05-08T18:06:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.993+0000 7f6eb03af800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.993+0000 7f6eb03af800 20 realm 2024-05-08T18:06:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.993+0000 7f6eb03af800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.993+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f571ba60 con 0x5630f5718a40 2024-05-08T18:06:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.993+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6ea800a4a0 con 0x5630f5718a40 2024-05-08T18:06:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.993+0000 7f6eb03af800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.993+0000 7f6eb03af800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.993+0000 7f6eb03af800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.993+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c330 con 0x5630f5718a40 2024-05-08T18:06:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.993+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6ea800a4a0 con 0x5630f5718a40 2024-05-08T18:06:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.993+0000 7f6eb03af800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.993+0000 7f6eb03af800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.993+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5718a40 2024-05-08T18:06:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.993+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6ea800a4a0 con 0x5630f5718a40 2024-05-08T18:06:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.994+0000 7f6eb03af800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.994+0000 7f6eb03af800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.994+0000 7f6eb03af800 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5630f571d1c0 0x5630f571f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.994+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f571fd10 con 0x5630f571d1c0 2024-05-08T18:06:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.994+0000 7f6eadeae640 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5630f571d1c0 0x5630f571f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.994+0000 7f6eadeae640 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5630f571d1c0 0x5630f571f5b0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.994+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.004 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.994+0000 7f6eb03af800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.994+0000 7f6eb03af800 20 searching for the correct realm 2024-05-08T18:06:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.994+0000 7f6eb03af800 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5630f5721dc0 0x5630f5724280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.994+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f5724950 con 0x5630f5721dc0 2024-05-08T18:06:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.995+0000 7f6ead6ad640 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5630f5721dc0 0x5630f5724280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.995+0000 7f6ead6ad640 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5630f5721dc0 0x5630f5724280 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.995+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.995+0000 7f6ead6ad640 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6ea4037040 0x7f6ea4036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.995+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f5724950 con 0x7f6ea4037040 2024-05-08T18:06:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.995+0000 7f6eae6af640 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6ea4037040 0x7f6ea4036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.996+0000 7f6eae6af640 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6ea4037040 0x7f6ea4036410 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.996+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea800a4a0 con 0x7f6ea4037040 2024-05-08T18:06:03.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.996+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f571d1c0 2024-05-08T18:06:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.996+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.996+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5718a40 2024-05-08T18:06:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.996+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea800a4a0 con 0x5630f5718a40 2024-05-08T18:06:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.996+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5718a40 2024-05-08T18:06:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.997+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea800a4a0 con 0x5630f5718a40 2024-05-08T18:06:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.997+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5718a40 2024-05-08T18:06:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.997+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea800a4a0 con 0x5630f5718a40 2024-05-08T18:06:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.997+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f571d1c0 2024-05-08T18:06:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.997+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.997+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x7f6ea4037040 2024-05-08T18:06:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.997+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea800a4a0 con 0x7f6ea4037040 2024-05-08T18:06:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.997+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5721dc0 2024-05-08T18:06:03.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.998+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.998+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5718a40 2024-05-08T18:06:03.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.998+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea800a4a0 con 0x5630f5718a40 2024-05-08T18:06:03.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.998+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f571d1c0 2024-05-08T18:06:03.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.998+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.998+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5718a40 2024-05-08T18:06:03.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.998+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea800a4a0 con 0x5630f5718a40 2024-05-08T18:06:03.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.998+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x7f6ea4037040 2024-05-08T18:06:03.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.999+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f6ea8032040 con 0x7f6ea4037040 2024-05-08T18:06:03.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.999+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5721dc0 2024-05-08T18:06:03.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.999+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.999+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5721dc0 2024-05-08T18:06:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.999+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:02.999+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f571d1c0 2024-05-08T18:06:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.000+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.000+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5721dc0 2024-05-08T18:06:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.000+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.000+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5721dc0 2024-05-08T18:06:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.000+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.000+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5718a40 2024-05-08T18:06:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.000+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.000+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5718a40 2024-05-08T18:06:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.001+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.001+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f571d1c0 2024-05-08T18:06:03.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.001+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.001+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5721dc0 2024-05-08T18:06:03.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.001+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.001+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x7f6ea4037040 2024-05-08T18:06:03.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.002+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea8032040 con 0x7f6ea4037040 2024-05-08T18:06:03.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.002+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5718a40 2024-05-08T18:06:03.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.002+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.002+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f571d1c0 2024-05-08T18:06:03.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.002+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.002+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5721dc0 2024-05-08T18:06:03.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.002+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.002+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f571d1c0 2024-05-08T18:06:03.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.003+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.003+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x7f6ea4037040 2024-05-08T18:06:03.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.003+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea8032040 con 0x7f6ea4037040 2024-05-08T18:06:03.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.003+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5721dc0 2024-05-08T18:06:03.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.003+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.003+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f571d1c0 2024-05-08T18:06:03.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x5630f5718a40 2024-05-08T18:06:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5630f571c6d0 con 0x7f6ea4037040 2024-05-08T18:06:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ea8032040 con 0x7f6ea4037040 2024-05-08T18:06:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eb03af800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eb03af800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eb03af800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eb03af800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eb03af800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eb03af800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eb03af800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f57206c0 con 0x5630f5718a40 2024-05-08T18:06:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eb03af800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eb03af800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.004+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f57206c0 con 0x5630f5718a40 2024-05-08T18:06:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eb03af800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eb03af800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5720690 con 0x5630f5721dc0 2024-05-08T18:06:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eb03af800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eb03af800 20 zone default found 2024-05-08T18:06:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eb03af800 4 Realm: () 2024-05-08T18:06:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eb03af800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eb03af800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eb03af800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:03.027 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eb03af800 20 zonegroup default 2024-05-08T18:06:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eb03af800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5720f60 con 0x5630f571d1c0 2024-05-08T18:06:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eb03af800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eb03af800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.005+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f57267b0 con 0x5630f5721dc0 2024-05-08T18:06:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.006+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.006+0000 7f6eb03af800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.006+0000 7f6eb03af800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.006+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5726b50 con 0x5630f5721dc0 2024-05-08T18:06:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.006+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.006+0000 7f6eb03af800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.006+0000 7f6eb03af800 20 started sync module instance, tier type = 2024-05-08T18:06:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.006+0000 7f6eb03af800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:03.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.006+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f5725c50 con 0x7f6ea4037040 2024-05-08T18:06:03.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.007+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x7f6ea4037040 2024-05-08T18:06:03.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.007+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94768276332928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f5725c50 con 0x7f6ea4037040 2024-05-08T18:06:03.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.007+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f57275a0 con 0x5630f5718a40 2024-05-08T18:06:03.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.008+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94768276332928] v29'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x7f6ea4037040 2024-05-08T18:06:03.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.008+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.008+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94768276336816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f5725c50 con 0x5630f5718a40 2024-05-08T18:06:03.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.008+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f5728070 con 0x5630f5721dc0 2024-05-08T18:06:03.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.009+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.009+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94768276339552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f5728910 con 0x5630f5721dc0 2024-05-08T18:06:03.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.010+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f5729410 con 0x7f6ea4037040 2024-05-08T18:06:03.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.010+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94768276336816] v29'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.010+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x7f6ea4037040 2024-05-08T18:06:03.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.010+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94768276344576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f5729800 con 0x7f6ea4037040 2024-05-08T18:06:03.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.011+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f572a1c0 con 0x5630f5718a40 2024-05-08T18:06:03.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.011+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94768276339552] v29'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.011+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.012+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94768276341856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f572a1c0 con 0x5630f5718a40 2024-05-08T18:06:03.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.012+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f572abd0 con 0x5630f571d1c0 2024-05-08T18:06:03.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.012+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94768276344576] v29'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x7f6ea4037040 2024-05-08T18:06:03.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.012+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94768276341856] v29'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.013+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.013+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94768276330400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f572a1c0 con 0x5630f571d1c0 2024-05-08T18:06:03.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.013+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f572b470 con 0x5630f571d1c0 2024-05-08T18:06:03.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.014+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(56 notify.6 [create] v29'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.014+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94768276355088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f572c0e0 con 0x5630f571d1c0 2024-05-08T18:06:03.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.014+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f572cb20 con 0x5630f571d1c0 2024-05-08T18:06:03.030 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.014+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94768276330400] v29'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.015+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94768276355088] v29'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.015+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.015+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94768276358640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f572bc70 con 0x5630f571d1c0 2024-05-08T18:06:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.015+0000 7f6eb03af800 20 add_watcher() i=0 2024-05-08T18:06:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.015+0000 7f6eb03af800 20 add_watcher() i=1 2024-05-08T18:06:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.015+0000 7f6eb03af800 20 add_watcher() i=2 2024-05-08T18:06:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.015+0000 7f6eb03af800 20 add_watcher() i=3 2024-05-08T18:06:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.015+0000 7f6eb03af800 20 add_watcher() i=4 2024-05-08T18:06:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.015+0000 7f6eb03af800 20 add_watcher() i=5 2024-05-08T18:06:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.015+0000 7f6eb03af800 20 add_watcher() i=6 2024-05-08T18:06:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.016+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94768276358640] v29'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.016+0000 7f6eb03af800 20 add_watcher() i=7 2024-05-08T18:06:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.016+0000 7f6eb03af800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:03.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.016+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f572fc20 con 0x5630f5718a40 2024-05-08T18:06:03.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.016+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.016+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94768276348176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f572d0a0 con 0x5630f5718a40 2024-05-08T18:06:03.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.017+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94768276348176] v29'198 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.018+0000 7f6d5f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:03.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.018+0000 7f6eb03af800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:03.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.018+0000 7f6eb03af800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:03.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.018+0000 7f6eb03af800 5 note: GC not initialized 2024-05-08T18:06:03.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.018+0000 7f6d5e7fc640 20 reqs_thread_entry: start 2024-05-08T18:06:03.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.018+0000 7f6eb03af800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:03.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.018+0000 7f6eb03af800 20 Filter name: none 2024-05-08T18:06:03.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.018+0000 7f6ccffff640 20 reqs_thread_entry: start 2024-05-08T18:06:03.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.018+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5763380 con 0x5630f5718a40 2024-05-08T18:06:03.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.018+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.019+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5763380 con 0x7f6ea4037040 2024-05-08T18:06:03.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.019+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x7f6ea4037040 2024-05-08T18:06:03.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.019+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5763340 con 0x5630f5718a40 2024-05-08T18:06:03.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.019+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.019+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5763340 con 0x5630f5721dc0 2024-05-08T18:06:03.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.019+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.019+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5763820 con 0x5630f5721dc0 2024-05-08T18:06:03.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.019+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.019+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5763820 con 0x5630f5721dc0 2024-05-08T18:06:03.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.020+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.020+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5763820 con 0x5630f5718a40 2024-05-08T18:06:03.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.020+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.020+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5763820 con 0x5630f5718a40 2024-05-08T18:06:03.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.020+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.020+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f57637d0 con 0x5630f5721dc0 2024-05-08T18:06:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.020+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.020+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f57637d0 con 0x5630f5721dc0 2024-05-08T18:06:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.021+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.021+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f57637d0 con 0x5630f571d1c0 2024-05-08T18:06:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.021+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(72 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.021+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765250 con 0x5630f5718a40 2024-05-08T18:06:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.021+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.021+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765250 con 0x7f6ea4037040 2024-05-08T18:06:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.022+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x7f6ea4037040 2024-05-08T18:06:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.022+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765250 con 0x5630f5721dc0 2024-05-08T18:06:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.022+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.022+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765250 con 0x5630f571d1c0 2024-05-08T18:06:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.022+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.022+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765250 con 0x5630f571d1c0 2024-05-08T18:06:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.022+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.022+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f57651d0 con 0x5630f5721dc0 2024-05-08T18:06:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.023+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.023+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f57651d0 con 0x5630f571d1c0 2024-05-08T18:06:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.023+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(79 usage.17 [call out=185b] v0'0 uv6 ondisk = 0) ==== 152+0+185 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.023+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f57651d0 con 0x5630f5718a40 2024-05-08T18:06:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.023+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.023+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f57651d0 con 0x5630f5721dc0 2024-05-08T18:06:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.024+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.024+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765180 con 0x5630f571d1c0 2024-05-08T18:06:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.024+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.024+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765180 con 0x7f6ea4037040 2024-05-08T18:06:03.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.024+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x7f6ea4037040 2024-05-08T18:06:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.024+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765180 con 0x5630f571d1c0 2024-05-08T18:06:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.024+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.024+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765180 con 0x5630f5718a40 2024-05-08T18:06:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.025+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.025+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765180 con 0x5630f5718a40 2024-05-08T18:06:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.025+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.025+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765180 con 0x5630f5721dc0 2024-05-08T18:06:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.025+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.025+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765180 con 0x5630f5718a40 2024-05-08T18:06:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.025+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.025+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765180 con 0x5630f5718a40 2024-05-08T18:06:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.026+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.026+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765180 con 0x5630f5721dc0 2024-05-08T18:06:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.026+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.026+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765180 con 0x5630f5721dc0 2024-05-08T18:06:03.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.026+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.026+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765180 con 0x5630f5721dc0 2024-05-08T18:06:03.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.026+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.026+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5630f5765180 con 0x5630f571d1c0 2024-05-08T18:06:03.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.027+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.028+0000 7f6d5f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:03.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.028+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94768276348176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f572cbf0 con 0x5630f5718a40 2024-05-08T18:06:03.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.029+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 32 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94768276348176] v29'199 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.029+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94768276332928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f5729d10 con 0x7f6ea4037040 2024-05-08T18:06:03.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.030+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94768276332928] v29'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x7f6ea4037040 2024-05-08T18:06:03.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.030+0000 7f6eb03af800 20 remove_watcher() i=0 2024-05-08T18:06:03.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.030+0000 7f6eb03af800 2 removed watcher, disabling cache 2024-05-08T18:06:03.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.030+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94768276336816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f5729d10 con 0x5630f5718a40 2024-05-08T18:06:03.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.031+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 33 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94768276336816] v29'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.031+0000 7f6eb03af800 20 remove_watcher() i=1 2024-05-08T18:06:03.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.031+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94768276339552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f5729d10 con 0x5630f5721dc0 2024-05-08T18:06:03.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.032+0000 7f6ead6ad640 1 -- 172.21.15.50:0/303358253 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94768276339552] v29'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea4009050 con 0x5630f5721dc0 2024-05-08T18:06:03.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.032+0000 7f6eb03af800 20 remove_watcher() i=2 2024-05-08T18:06:03.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.032+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94768276344576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f5729d10 con 0x7f6ea4037040 2024-05-08T18:06:03.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.033+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94768276344576] v29'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x7f6ea4037040 2024-05-08T18:06:03.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.033+0000 7f6eb03af800 20 remove_watcher() i=3 2024-05-08T18:06:03.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.033+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94768276341856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f5728f00 con 0x5630f5718a40 2024-05-08T18:06:03.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.034+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 <== osd.1 v2:172.21.15.50:6808/2141923372 34 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94768276341856] v29'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea8032040 con 0x5630f5718a40 2024-05-08T18:06:03.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.034+0000 7f6eb03af800 20 remove_watcher() i=4 2024-05-08T18:06:03.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.034+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94768276330400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f5729d10 con 0x5630f571d1c0 2024-05-08T18:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.036+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94768276330400] v29'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.036+0000 7f6eb03af800 20 remove_watcher() i=5 2024-05-08T18:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.036+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94768276355088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f57257a0 con 0x5630f571d1c0 2024-05-08T18:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.037+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94768276355088] v29'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.037+0000 7f6eb03af800 20 remove_watcher() i=6 2024-05-08T18:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.037+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94768276358640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5630f5729d10 con 0x5630f571d1c0 2024-05-08T18:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eadeae640 1 -- 172.21.15.50:0/303358253 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94768276358640] v29'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ea0009bc0 con 0x5630f571d1c0 2024-05-08T18:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 20 remove_watcher() i=7 2024-05-08T18:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5630f571d1c0 msgr2=0x5630f571f5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5630f571d1c0 0x5630f571f5b0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5630f5718a40 msgr2=0x5630f571af00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5630f5718a40 0x5630f571af00 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6ea4037040 msgr2=0x7f6ea4036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f6ea4037040 0x7f6ea4036410 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5630f5721dc0 msgr2=0x5630f5724280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5630f5721dc0 0x5630f5724280 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6d7803bb50 msgr2=0x7f6d7803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6d7803bb50 0x7f6d7803e010 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400c010 tx=0x7f6ea4010000 comp rx=0 tx=0).stop 2024-05-08T18:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f5705ee0 msgr2=0x5630f570d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f5705ee0 0x5630f570d320 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6ea005bd20 tx=0x7f6ea005bd70 comp rx=0 tx=0).stop 2024-05-08T18:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eae6af640 1 -- 172.21.15.50:0/303358253 reap_dead start 2024-05-08T18:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 shutdown_connections 2024-05-08T18:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 --2- 172.21.15.50:0/303358253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f5705ee0 0x5630f570d320 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 >> 172.21.15.50:0/303358253 conn(0x5630f57021f0 msgr2=0x5630f56ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 shutdown_connections 2024-05-08T18:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 -- 172.21.15.50:0/303358253 wait complete. 2024-05-08T18:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 -- 172.21.15.50:0/728785759 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6d8803bb50 msgr2=0x7f6d8803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 --2- 172.21.15.50:0/728785759 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6d8803bb50 0x7f6d8803e010 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400bf40 tx=0x7f6ea400f000 comp rx=0 tx=0).stop 2024-05-08T18:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 -- 172.21.15.50:0/728785759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f56f8dc0 msgr2=0x5630f56f91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 --2- 172.21.15.50:0/728785759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f56f8dc0 0x5630f56f91e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0093180 tx=0x7f6ea005ed20 comp rx=0 tx=0).stop 2024-05-08T18:06:03.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 -- 172.21.15.50:0/728785759 shutdown_connections 2024-05-08T18:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 --2- 172.21.15.50:0/728785759 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6d8803bb50 0x7f6d8803e010 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 --2- 172.21.15.50:0/728785759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f56f8dc0 0x5630f56f91e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 -- 172.21.15.50:0/728785759 >> 172.21.15.50:0/728785759 conn(0x5630f56ece60 msgr2=0x5630f56ed260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.038+0000 7f6eb03af800 1 -- 172.21.15.50:0/728785759 shutdown_connections 2024-05-08T18:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.039+0000 7f6eb03af800 1 -- 172.21.15.50:0/728785759 wait complete. 2024-05-08T18:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.039+0000 7f6eb03af800 1 -- 172.21.15.50:0/604906017 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5630f56e4070 msgr2=0x5630f56e6460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.039+0000 7f6eb03af800 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5630f56e4070 0x5630f56e6460 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.039+0000 7f6eb03af800 1 -- 172.21.15.50:0/604906017 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5630f55e4a00 msgr2=0x5630f56e17c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.039+0000 7f6eb03af800 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5630f55e4a00 0x5630f56e17c0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.039+0000 7f6eb03af800 1 -- 172.21.15.50:0/604906017 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5630f56e7fb0 msgr2=0x5630f56ea3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.039+0000 7f6eb03af800 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5630f56e7fb0 0x5630f56ea3a0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.039+0000 7f6eb03af800 1 -- 172.21.15.50:0/604906017 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6d9403bb50 msgr2=0x7f6d9403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.039+0000 7f6eb03af800 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6d9403bb50 0x7f6d9403e010 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4001d70 tx=0x7f6ea40012d0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.039+0000 7f6eb03af800 1 -- 172.21.15.50:0/604906017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 msgr2=0x5630f56da580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.039+0000 7f6eb03af800 1 --2- 172.21.15.50:0/604906017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630f55e20d0 0x5630f56da580 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0060040 tx=0x7f6ea0002fb0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.039+0000 7f6eae6af640 1 -- 172.21.15.50:0/604906017 reap_dead start 2024-05-08T18:06:03.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.039+0000 7f6eb03af800 1 -- 172.21.15.50:0/604906017 shutdown_connections 2024-05-08T18:06:03.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.039+0000 7f6eb03af800 1 -- 172.21.15.50:0/604906017 >> 172.21.15.50:0/604906017 conn(0x5630f55dbb40 msgr2=0x5630f56ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.039+0000 7f6eb03af800 1 -- 172.21.15.50:0/604906017 shutdown_connections 2024-05-08T18:06:03.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.039+0000 7f6eb03af800 1 -- 172.21.15.50:0/604906017 wait complete. 2024-05-08T18:06:03.048 INFO:teuthology.orchestra.run.smithi050.stdout:{"entries":[{"user":"anonymous","buckets":[{"bucket":"","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"anonymous","categories":[{"category":"list_buckets","bytes_sent":204,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}}]},{"user":"foo","buckets":[{"bucket":"","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":5,"successful_ops":5},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":2,"successful_ops":1},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo2","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo3","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo4","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}}]}],"summary":[{"user":"anonymous","categories":[{"category":"list_buckets","bytes_sent":204,"bytes_received":0,"ops":1,"successful_ops":1}],"total":{"bytes_sent":204,"bytes_received":0,"ops":1,"successful_ops":1,"bytes_processed":0,"bytes_returned":0}},{"user":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"total":{"bytes_sent":2103,"bytes_received":30000042,"ops":35,"successful_ops":34,"bytes_processed":0,"bytes_returned":0}}]} 2024-05-08T18:06:03.048 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo2', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo3', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo4', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2103, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-05-08T18:06:03.048 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2024-05-08T18:06:03.048 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2024-05-08T18:06:03.048 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2024-05-08T18:06:03.094 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:03.094 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:03.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.097+0000 7f850ac98800 1 Processor -- start 2024-05-08T18:06:03.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.097+0000 7f850ac98800 1 -- start start 2024-05-08T18:06:03.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.097+0000 7f850ac98800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ec20597a00 0x55ec20599df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.097+0000 7f850ac98800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55ec2059a3c0 con 0x55ec20593a50 2024-05-08T18:06:03.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.097+0000 7f850ac98800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55ec20434e20 con 0x55ec20597a00 2024-05-08T18:06:03.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.097+0000 7f8507f8f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ec20597a00 0x55ec20599df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.097+0000 7f8507f8f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ec20597a00 0x55ec20599df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48670/0 (socket says 172.21.15.50:48670) 2024-05-08T18:06:03.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.097+0000 7f8507f8f640 1 -- 172.21.15.50:0/556393018 learned_addr learned my addr 172.21.15.50:0/556393018 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.097+0000 7f8508790640 1 --1- 172.21.15.50:0/556393018 >> v1:172.21.15.46:6789/0 conn(0x55ec20593a50 0x55ec20593e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:54530/0 (socket says 172.21.15.50:54530) 2024-05-08T18:06:03.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.098+0000 7f8507f8f640 1 -- 172.21.15.50:0/556393018 >> v1:172.21.15.46:6789/0 conn(0x55ec20593a50 legacy=0x55ec20593e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.098+0000 7f8507f8f640 1 -- 172.21.15.50:0/556393018 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec2059b5a0 con 0x55ec20597a00 2024-05-08T18:06:03.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.099+0000 7f8507f8f640 1 --2- 172.21.15.50:0/556393018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20597a00 0x55ec20599df0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00a520 tx=0x7f84fc01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25206ab6d1fae5a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.100+0000 7f850778e640 1 -- 172.21.15.50:0/556393018 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84fc057070 con 0x55ec20597a00 2024-05-08T18:06:03.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.100+0000 7f850778e640 1 -- 172.21.15.50:0/556393018 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84fc052570 con 0x55ec20597a00 2024-05-08T18:06:03.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.100+0000 7f850778e640 1 -- 172.21.15.50:0/556393018 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84fc04ae60 con 0x55ec20597a00 2024-05-08T18:06:03.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.100+0000 7f850ac98800 1 -- 172.21.15.50:0/556393018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20597a00 msgr2=0x55ec20599df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.100+0000 7f850ac98800 1 --2- 172.21.15.50:0/556393018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20597a00 0x55ec20599df0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00a520 tx=0x7f84fc01d010 comp rx=0 tx=0).stop 2024-05-08T18:06:03.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.100+0000 7f850ac98800 1 -- 172.21.15.50:0/556393018 shutdown_connections 2024-05-08T18:06:03.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.100+0000 7f850ac98800 1 --2- 172.21.15.50:0/556393018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20597a00 0x55ec20599df0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.100+0000 7f850ac98800 1 -- 172.21.15.50:0/556393018 >> 172.21.15.50:0/556393018 conn(0x55ec2058e2d0 msgr2=0x55ec205906f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.100+0000 7f850ac98800 1 -- 172.21.15.50:0/556393018 shutdown_connections 2024-05-08T18:06:03.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.100+0000 7f850ac98800 1 -- 172.21.15.50:0/556393018 wait complete. 2024-05-08T18:06:03.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.115+0000 7f850ac98800 1 Processor -- start 2024-05-08T18:06:03.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.115+0000 7f850ac98800 1 -- start start 2024-05-08T18:06:03.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.115+0000 7f850ac98800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20593a50 0x55ec20684a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.115+0000 7f850ac98800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec204dfa10 con 0x55ec20593a50 2024-05-08T18:06:03.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.116+0000 7f8508790640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20593a50 0x55ec20684a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.116+0000 7f8508790640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20593a50 0x55ec20684a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48672/0 (socket says 172.21.15.50:48672) 2024-05-08T18:06:03.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.116+0000 7f8508790640 1 -- 172.21.15.50:0/3935283162 learned_addr learned my addr 172.21.15.50:0/3935283162 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.116+0000 7f8508790640 1 -- 172.21.15.50:0/3935283162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec20684fd0 con 0x55ec20593a50 2024-05-08T18:06:03.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.116+0000 7f8508790640 1 --2- 172.21.15.50:0/3935283162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20593a50 0x55ec20684a00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f84f800b260 tx=0x7f84f800b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec9d8dbf052074c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.116+0000 7f83f660c640 1 -- 172.21.15.50:0/3935283162 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f800e5c0 con 0x55ec20593a50 2024-05-08T18:06:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.117+0000 7f83f660c640 1 -- 172.21.15.50:0/3935283162 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f8006070 con 0x55ec20593a50 2024-05-08T18:06:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.117+0000 7f83f660c640 1 -- 172.21.15.50:0/3935283162 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f800e890 con 0x55ec20593a50 2024-05-08T18:06:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.117+0000 7f850ac98800 1 -- 172.21.15.50:0/3935283162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20593a50 msgr2=0x55ec20684a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.117+0000 7f850ac98800 1 --2- 172.21.15.50:0/3935283162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20593a50 0x55ec20684a00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f84f800b260 tx=0x7f84f800b720 comp rx=0 tx=0).stop 2024-05-08T18:06:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.117+0000 7f850ac98800 1 -- 172.21.15.50:0/3935283162 shutdown_connections 2024-05-08T18:06:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.117+0000 7f850ac98800 1 --2- 172.21.15.50:0/3935283162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20593a50 0x55ec20684a00 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.117+0000 7f850ac98800 1 -- 172.21.15.50:0/3935283162 >> 172.21.15.50:0/3935283162 conn(0x55ec2058e2d0 msgr2=0x55ec20680320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.117+0000 7f850ac98800 1 -- 172.21.15.50:0/3935283162 shutdown_connections 2024-05-08T18:06:03.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.117+0000 7f850ac98800 1 -- 172.21.15.50:0/3935283162 wait complete. 2024-05-08T18:06:03.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.118+0000 7f850ac98800 1 Processor -- start 2024-05-08T18:06:03.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.118+0000 7f850ac98800 1 -- start start 2024-05-08T18:06:03.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.118+0000 7f850ac98800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20593a50 0x55ec20690290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.118+0000 7f850ac98800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec2059a890 con 0x55ec20593a50 2024-05-08T18:06:03.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.118+0000 7f8508790640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20593a50 0x55ec20690290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.118+0000 7f8508790640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20593a50 0x55ec20690290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48678/0 (socket says 172.21.15.50:48678) 2024-05-08T18:06:03.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.118+0000 7f8508790640 1 -- 172.21.15.50:0/3284099190 learned_addr learned my addr 172.21.15.50:0/3284099190 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.119+0000 7f8508790640 1 -- 172.21.15.50:0/3284099190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec20593200 con 0x55ec20593a50 2024-05-08T18:06:03.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.119+0000 7f8508790640 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20593a50 0x55ec20690290 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f84f8000f60 tx=0x7f84f8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.119+0000 7f83f4e09640 1 -- 172.21.15.50:0/3284099190 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f801a7c0 con 0x55ec20593a50 2024-05-08T18:06:03.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.119+0000 7f83f4e09640 1 -- 172.21.15.50:0/3284099190 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f801a960 con 0x55ec20593a50 2024-05-08T18:06:03.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.119+0000 7f83f4e09640 1 -- 172.21.15.50:0/3284099190 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f801ac40 con 0x55ec20593a50 2024-05-08T18:06:03.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.119+0000 7f850ac98800 1 -- 172.21.15.50:0/3284099190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec20593490 con 0x55ec20593a50 2024-05-08T18:06:03.119 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.119+0000 7f850ac98800 1 -- 172.21.15.50:0/3284099190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec206909e0 con 0x55ec20593a50 2024-05-08T18:06:03.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.120+0000 7f83f4e09640 1 -- 172.21.15.50:0/3284099190 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f84f8017070 con 0x55ec20593a50 2024-05-08T18:06:03.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.120+0000 7f83f4e09640 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f83f003bba0 0x7f83f003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.120+0000 7f83f4e09640 1 -- 172.21.15.50:0/3284099190 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f84f8015020 con 0x55ec20593a50 2024-05-08T18:06:03.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.120+0000 7f8507f8f640 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f83f003bba0 0x7f83f003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.120+0000 7f850ac98800 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec20597a00 0x55ec20694800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.120+0000 7f850ac98800 1 -- 172.21.15.50:0/3284099190 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec20694e20 con 0x55ec20597a00 2024-05-08T18:06:03.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.120+0000 7f8508f91640 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec20597a00 0x55ec20694800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.121+0000 7f8507f8f640 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f83f003bba0 0x7f83f003e060 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f84fc054000 tx=0x7f84fc04e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.121+0000 7f8508f91640 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec20597a00 0x55ec20694800 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.121+0000 7f8508f91640 1 -- 172.21.15.50:0/3284099190 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f850000a4a0 con 0x55ec20597a00 2024-05-08T18:06:03.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.121+0000 7f850ac98800 1 -- 172.21.15.50:0/3284099190 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec20691dd0 con 0x55ec20597a00 2024-05-08T18:06:03.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.121+0000 7f8508f91640 1 -- 172.21.15.50:0/3284099190 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f850000a4a0 con 0x55ec20597a00 2024-05-08T18:06:03.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.121+0000 7f850ac98800 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec20697170 0x55ec20699560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.121+0000 7f850ac98800 1 -- 172.21.15.50:0/3284099190 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec20699c50 con 0x55ec20697170 2024-05-08T18:06:03.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.122+0000 7f8508790640 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec20697170 0x55ec20699560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.122+0000 7f8508790640 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec20697170 0x55ec20699560 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.122+0000 7f8508790640 1 -- 172.21.15.50:0/3284099190 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f84f802e070 con 0x55ec20697170 2024-05-08T18:06:03.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.122+0000 7f850ac98800 1 -- 172.21.15.50:0/3284099190 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec20696010 con 0x55ec20597a00 2024-05-08T18:06:03.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.123+0000 7f8508f91640 1 -- 172.21.15.50:0/3284099190 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f850000a4a0 con 0x55ec20597a00 2024-05-08T18:06:03.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.123+0000 7f850ac98800 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec2069b0b0 0x55ec2069d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.123+0000 7f850ac98800 1 -- 172.21.15.50:0/3284099190 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec2069dba0 con 0x55ec2069b0b0 2024-05-08T18:06:03.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.123+0000 7f8507f8f640 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec2069b0b0 0x55ec2069d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.123+0000 7f8507f8f640 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec2069b0b0 0x55ec2069d4a0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.123+0000 7f8507f8f640 1 -- 172.21.15.50:0/3284099190 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f84fc05a040 con 0x55ec2069b0b0 2024-05-08T18:06:03.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.124+0000 7f850ac98800 1 Processor -- start 2024-05-08T18:06:03.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.124+0000 7f850ac98800 1 -- start start 2024-05-08T18:06:03.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.124+0000 7f850ac98800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206abf10 0x55ec206ac330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.124+0000 7f850ac98800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec20486330 con 0x55ec206abf10 2024-05-08T18:06:03.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.124+0000 7f8508790640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206abf10 0x55ec206ac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.124+0000 7f8508790640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206abf10 0x55ec206ac330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48690/0 (socket says 172.21.15.50:48690) 2024-05-08T18:06:03.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.124+0000 7f8508790640 1 -- 172.21.15.50:0/528963479 learned_addr learned my addr 172.21.15.50:0/528963479 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.124+0000 7f8508790640 1 -- 172.21.15.50:0/528963479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec206ac870 con 0x55ec206abf10 2024-05-08T18:06:03.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.125+0000 7f8508790640 1 --2- 172.21.15.50:0/528963479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206abf10 0x55ec206ac330 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f84f8051800 tx=0x7f84f8050510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.125+0000 7f83ee7fc640 1 -- 172.21.15.50:0/528963479 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f8017b70 con 0x55ec206abf10 2024-05-08T18:06:03.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.125+0000 7f83ee7fc640 1 -- 172.21.15.50:0/528963479 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f8017d10 con 0x55ec206abf10 2024-05-08T18:06:03.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.125+0000 7f83ee7fc640 1 -- 172.21.15.50:0/528963479 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f801b590 con 0x55ec206abf10 2024-05-08T18:06:03.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.125+0000 7f850ac98800 1 -- 172.21.15.50:0/528963479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec206ad2e0 con 0x55ec206abf10 2024-05-08T18:06:03.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.125+0000 7f850ac98800 1 -- 172.21.15.50:0/528963479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec206ad7e0 con 0x55ec206abf10 2024-05-08T18:06:03.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.126+0000 7f83ee7fc640 1 -- 172.21.15.50:0/528963479 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f84f801b750 con 0x55ec206abf10 2024-05-08T18:06:03.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.126+0000 7f83ee7fc640 1 --2- 172.21.15.50:0/528963479 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f83e003bba0 0x7f83e003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.126+0000 7f83ee7fc640 1 -- 172.21.15.50:0/528963479 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f84f803bc40 con 0x55ec206abf10 2024-05-08T18:06:03.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.126+0000 7f8507f8f640 1 --2- 172.21.15.50:0/528963479 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f83e003bba0 0x7f83e003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.126+0000 7f850ac98800 1 Processor -- start 2024-05-08T18:06:03.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.126+0000 7f850ac98800 1 -- start start 2024-05-08T18:06:03.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.126+0000 7f8507f8f640 1 --2- 172.21.15.50:0/528963479 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f83e003bba0 0x7f83e003e060 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55ec20695720 tx=0x7f84fc005ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.126+0000 7f850ac98800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206b8fd0 0x55ec206b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.126+0000 7f850ac98800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec20685350 con 0x55ec206b8fd0 2024-05-08T18:06:03.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.127+0000 7f8508790640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206b8fd0 0x55ec206b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.137 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.127+0000 7f8508790640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206b8fd0 0x55ec206b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48696/0 (socket says 172.21.15.50:48696) 2024-05-08T18:06:03.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.127+0000 7f8508790640 1 -- 172.21.15.50:0/2531352360 learned_addr learned my addr 172.21.15.50:0/2531352360 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.127+0000 7f8508790640 1 -- 172.21.15.50:0/2531352360 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec206b99c0 con 0x55ec206b8fd0 2024-05-08T18:06:03.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.127+0000 7f8508790640 1 --2- 172.21.15.50:0/2531352360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206b8fd0 0x55ec206b93f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f84f803b8d0 tx=0x7f84f8043100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e370b8ae71bf5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.127+0000 7f83e77fe640 1 -- 172.21.15.50:0/2531352360 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f8027430 con 0x55ec206b8fd0 2024-05-08T18:06:03.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.127+0000 7f83e77fe640 1 -- 172.21.15.50:0/2531352360 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f8019860 con 0x55ec206b8fd0 2024-05-08T18:06:03.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.127+0000 7f83e77fe640 1 -- 172.21.15.50:0/2531352360 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f804a670 con 0x55ec206b8fd0 2024-05-08T18:06:03.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.128+0000 7f850ac98800 1 -- 172.21.15.50:0/2531352360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206b8fd0 msgr2=0x55ec206b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.128+0000 7f850ac98800 1 --2- 172.21.15.50:0/2531352360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206b8fd0 0x55ec206b93f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f84f803b8d0 tx=0x7f84f8043100 comp rx=0 tx=0).stop 2024-05-08T18:06:03.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.128+0000 7f850ac98800 1 -- 172.21.15.50:0/2531352360 shutdown_connections 2024-05-08T18:06:03.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.128+0000 7f850ac98800 1 --2- 172.21.15.50:0/2531352360 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206b8fd0 0x55ec206b93f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.128+0000 7f850ac98800 1 -- 172.21.15.50:0/2531352360 >> 172.21.15.50:0/2531352360 conn(0x55ec206b5330 msgr2=0x55ec206b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.128+0000 7f850ac98800 1 -- 172.21.15.50:0/2531352360 shutdown_connections 2024-05-08T18:06:03.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.128+0000 7f850ac98800 1 -- 172.21.15.50:0/2531352360 wait complete. 2024-05-08T18:06:03.138 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.128+0000 7f850ac98800 1 Processor -- start 2024-05-08T18:06:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.128+0000 7f850ac98800 1 -- start start 2024-05-08T18:06:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.128+0000 7f850ac98800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206b8fd0 0x55ec206c03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.128+0000 7f850ac98800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec206ba070 con 0x55ec206b8fd0 2024-05-08T18:06:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.128+0000 7f8508790640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206b8fd0 0x55ec206c03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.128+0000 7f8508790640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206b8fd0 0x55ec206c03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48704/0 (socket says 172.21.15.50:48704) 2024-05-08T18:06:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.128+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 learned_addr learned my addr 172.21.15.50:0/4270235224 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.129+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec206c0920 con 0x55ec206b8fd0 2024-05-08T18:06:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.129+0000 7f8508790640 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206b8fd0 0x55ec206c03e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f84f801be20 tx=0x7f84f801ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.129+0000 7f83e5ffb640 1 -- 172.21.15.50:0/4270235224 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f8027430 con 0x55ec206b8fd0 2024-05-08T18:06:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.129+0000 7f83e5ffb640 1 -- 172.21.15.50:0/4270235224 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84f803c0c0 con 0x55ec206b8fd0 2024-05-08T18:06:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.129+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec206c0920 con 0x55ec206b8fd0 2024-05-08T18:06:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.129+0000 7f83e5ffb640 1 -- 172.21.15.50:0/4270235224 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84f803c3a0 con 0x55ec206b8fd0 2024-05-08T18:06:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.129+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec206c0dc0 con 0x55ec206b8fd0 2024-05-08T18:06:03.139 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.130+0000 7f83e5ffb640 1 -- 172.21.15.50:0/4270235224 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f84f80434a0 con 0x55ec206b8fd0 2024-05-08T18:06:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.130+0000 7f83e5ffb640 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f83d003bba0 0x7f83d003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.130+0000 7f83e5ffb640 1 -- 172.21.15.50:0/4270235224 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f84f805b030 con 0x55ec206b8fd0 2024-05-08T18:06:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.130+0000 7f850ac98800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.130+0000 7f8507f8f640 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f83d003bba0 0x7f83d003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.130+0000 7f850ac98800 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec206cb980 0x55ec206cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.130+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec206ce590 con 0x55ec206cb980 2024-05-08T18:06:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.131+0000 7f8508f91640 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec206cb980 0x55ec206cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.131+0000 7f8508f91640 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec206cb980 0x55ec206cde40 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.131+0000 7f8507f8f640 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f83d003bba0 0x7f83d003e060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55ec2069dfa0 tx=0x7f84fc04f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.131+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f850000a4a0 con 0x55ec206cb980 2024-05-08T18:06:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.131+0000 7f850ac98800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.131+0000 7f850ac98800 20 realm 2024-05-08T18:06:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.131+0000 7f850ac98800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.131+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec206ce9a0 con 0x55ec206cb980 2024-05-08T18:06:03.140 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.131+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f850000a4a0 con 0x55ec206cb980 2024-05-08T18:06:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.131+0000 7f850ac98800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.131+0000 7f850ac98800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.131+0000 7f850ac98800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.131+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec206cf240 con 0x55ec206cb980 2024-05-08T18:06:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.132+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f850000a4a0 con 0x55ec206cb980 2024-05-08T18:06:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.132+0000 7f850ac98800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.132+0000 7f850ac98800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.132+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec206cf5e0 con 0x55ec206cb980 2024-05-08T18:06:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.132+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f850000a4a0 con 0x55ec206cb980 2024-05-08T18:06:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.132+0000 7f850ac98800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.132+0000 7f850ac98800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.132+0000 7f850ac98800 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec206d0340 0x55ec206d2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.132+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec206d2cb0 con 0x55ec206d0340 2024-05-08T18:06:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.132+0000 7f8508790640 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec206d0340 0x55ec206d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.141 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.133+0000 7f8508790640 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec206d0340 0x55ec206d2730 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.133+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.133+0000 7f850ac98800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:03.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.133+0000 7f850ac98800 20 searching for the correct realm 2024-05-08T18:06:03.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.133+0000 7f850ac98800 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec206d4d00 0x55ec206d71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.133+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ec206d7890 con 0x55ec206d4d00 2024-05-08T18:06:03.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.133+0000 7f8507f8f640 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec206d4d00 0x55ec206d71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.133+0000 7f8507f8f640 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec206d4d00 0x55ec206d71c0 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.134+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f84fc07f070 con 0x55ec206d4d00 2024-05-08T18:06:03.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.134+0000 7f8507f8f640 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f84fc074040 0x7f84fc081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.134+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ec206d7890 con 0x7f84fc074040 2024-05-08T18:06:03.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.134+0000 7f8508f91640 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f84fc074040 0x7f84fc081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.134+0000 7f8508f91640 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f84fc074040 0x7f84fc081410 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.142 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.134+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f850000a4a0 con 0x7f84fc074040 2024-05-08T18:06:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.134+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ec206cf5e0 con 0x55ec206d0340 2024-05-08T18:06:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.135+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.135+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ec206cf5e0 con 0x55ec206cb980 2024-05-08T18:06:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.135+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f850000a4a0 con 0x55ec206cb980 2024-05-08T18:06:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.135+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ec206cf5e0 con 0x55ec206cb980 2024-05-08T18:06:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.135+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f850000a4a0 con 0x55ec206cb980 2024-05-08T18:06:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.135+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ec206cf5e0 con 0x55ec206cb980 2024-05-08T18:06:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.135+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f850000a4a0 con 0x55ec206cb980 2024-05-08T18:06:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.135+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ec206cf5e0 con 0x55ec206d0340 2024-05-08T18:06:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.136+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.136+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ec206cf5e0 con 0x7f84fc074040 2024-05-08T18:06:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.136+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f850000a4a0 con 0x7f84fc074040 2024-05-08T18:06:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.136+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ec206cf5e0 con 0x55ec206d4d00 2024-05-08T18:06:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.136+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f84fc07f070 con 0x55ec206d4d00 2024-05-08T18:06:03.143 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.136+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ec206cf5e0 con 0x55ec206cb980 2024-05-08T18:06:03.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.136+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f850000a4a0 con 0x55ec206cb980 2024-05-08T18:06:03.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.136+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ec206cf5e0 con 0x55ec206d0340 2024-05-08T18:06:03.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.137+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.137+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ec206cf5e0 con 0x55ec206cb980 2024-05-08T18:06:03.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.137+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f850000a4a0 con 0x55ec206cb980 2024-05-08T18:06:03.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.137+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ec206cf5e0 con 0x7f84fc074040 2024-05-08T18:06:03.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.137+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f8500032040 con 0x7f84fc074040 2024-05-08T18:06:03.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.137+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ec206cf5e0 con 0x55ec206d4d00 2024-05-08T18:06:03.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.137+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f84fc07f070 con 0x55ec206d4d00 2024-05-08T18:06:03.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.137+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x55ec206d4d00 2024-05-08T18:06:03.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.138+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f84fc07f070 con 0x55ec206d4d00 2024-05-08T18:06:03.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.138+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x55ec206d0340 2024-05-08T18:06:03.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.138+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.138+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x55ec206d4d00 2024-05-08T18:06:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.138+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f84fc07f070 con 0x55ec206d4d00 2024-05-08T18:06:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.138+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x55ec206d4d00 2024-05-08T18:06:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.138+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f84fc07f070 con 0x55ec206d4d00 2024-05-08T18:06:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.138+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x55ec206cb980 2024-05-08T18:06:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.139+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8500032040 con 0x55ec206cb980 2024-05-08T18:06:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.139+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x55ec206cb980 2024-05-08T18:06:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.139+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f8500032040 con 0x55ec206cb980 2024-05-08T18:06:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.139+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x55ec206d0340 2024-05-08T18:06:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.139+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.139+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x55ec206d4d00 2024-05-08T18:06:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.139+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f84fc07f070 con 0x55ec206d4d00 2024-05-08T18:06:03.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.139+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x7f84fc074040 2024-05-08T18:06:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.140+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8500032040 con 0x7f84fc074040 2024-05-08T18:06:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.140+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x55ec206cb980 2024-05-08T18:06:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.140+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8500032040 con 0x55ec206cb980 2024-05-08T18:06:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.140+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x55ec206d0340 2024-05-08T18:06:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.140+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.140+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x55ec206d4d00 2024-05-08T18:06:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.141+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f84fc07f070 con 0x55ec206d4d00 2024-05-08T18:06:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.141+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x55ec206d0340 2024-05-08T18:06:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.141+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.141+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x7f84fc074040 2024-05-08T18:06:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.141+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8500032040 con 0x7f84fc074040 2024-05-08T18:06:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.141+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x55ec206d4d00 2024-05-08T18:06:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.141+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f84fc07f070 con 0x55ec206d4d00 2024-05-08T18:06:03.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.141+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x55ec206d0340 2024-05-08T18:06:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x55ec206cb980 2024-05-08T18:06:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f8500032040 con 0x55ec206cb980 2024-05-08T18:06:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f84fc08e040 con 0x7f84fc074040 2024-05-08T18:06:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8500032040 con 0x7f84fc074040 2024-05-08T18:06:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f850ac98800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f850ac98800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f850ac98800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f850ac98800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f850ac98800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f850ac98800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:03.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f850ac98800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec206d3600 con 0x55ec206cb980 2024-05-08T18:06:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8500032040 con 0x55ec206cb980 2024-05-08T18:06:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f850ac98800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f850ac98800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.142+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec206d3600 con 0x55ec206cb980 2024-05-08T18:06:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8500032040 con 0x55ec206cb980 2024-05-08T18:06:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f850ac98800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f850ac98800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec206d35d0 con 0x55ec206d4d00 2024-05-08T18:06:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f84fc07f070 con 0x55ec206d4d00 2024-05-08T18:06:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f850ac98800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f850ac98800 20 zone default found 2024-05-08T18:06:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f850ac98800 4 Realm: () 2024-05-08T18:06:03.157 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f850ac98800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f850ac98800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f850ac98800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f850ac98800 20 zonegroup default 2024-05-08T18:06:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f850ac98800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec206d3ed0 con 0x55ec206d0340 2024-05-08T18:06:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f850ac98800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f850ac98800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec206d96f0 con 0x55ec206d4d00 2024-05-08T18:06:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.143+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f84fc07f070 con 0x55ec206d4d00 2024-05-08T18:06:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.144+0000 7f850ac98800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.144+0000 7f850ac98800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.144+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec206d9a90 con 0x55ec206d4d00 2024-05-08T18:06:03.158 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.144+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f84fc07f070 con 0x55ec206d4d00 2024-05-08T18:06:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.144+0000 7f850ac98800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.144+0000 7f850ac98800 20 started sync module instance, tier type = 2024-05-08T18:06:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.144+0000 7f850ac98800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.144+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206d8b90 con 0x7f84fc074040 2024-05-08T18:06:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.145+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8500032040 con 0x7f84fc074040 2024-05-08T18:06:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.145+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94472644694208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206d8b90 con 0x7f84fc074040 2024-05-08T18:06:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.145+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206da450 con 0x55ec206cb980 2024-05-08T18:06:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.146+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94472644694208] v29'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8500032040 con 0x7f84fc074040 2024-05-08T18:06:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.146+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8500032040 con 0x55ec206cb980 2024-05-08T18:06:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.146+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94472644692160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206da450 con 0x55ec206cb980 2024-05-08T18:06:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.146+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206dae70 con 0x55ec206d4d00 2024-05-08T18:06:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.147+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f84fc07f070 con 0x55ec206d4d00 2024-05-08T18:06:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.147+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94472644700512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206db710 con 0x55ec206d4d00 2024-05-08T18:06:03.159 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.147+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206dc1f0 con 0x7f84fc074040 2024-05-08T18:06:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.148+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94472644692160] v29'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8500032040 con 0x55ec206cb980 2024-05-08T18:06:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.148+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94472644700512] v29'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f84fc07f070 con 0x55ec206d4d00 2024-05-08T18:06:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.148+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8500032040 con 0x7f84fc074040 2024-05-08T18:06:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.148+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94472644702816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206d9fa0 con 0x7f84fc074040 2024-05-08T18:06:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.148+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206dca90 con 0x55ec206cb980 2024-05-08T18:06:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.149+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8500032040 con 0x55ec206cb980 2024-05-08T18:06:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.149+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94472644707728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206dc9b0 con 0x55ec206cb980 2024-05-08T18:06:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.149+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206dd990 con 0x55ec206d0340 2024-05-08T18:06:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.149+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94472644702816] v29'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8500032040 con 0x7f84fc074040 2024-05-08T18:06:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.150+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.151+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94472644707728] v29'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8500032040 con 0x55ec206cb980 2024-05-08T18:06:03.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.151+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94472644705536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206ddd80 con 0x55ec206d0340 2024-05-08T18:06:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.151+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206dce80 con 0x55ec206d0340 2024-05-08T18:06:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.151+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94472644705536] v29'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.152+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.152+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94472644715152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206deb60 con 0x55ec206d0340 2024-05-08T18:06:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.152+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206df3b0 con 0x55ec206d0340 2024-05-08T18:06:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.153+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94472644715152] v29'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.153+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.153+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94472644718336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206df320 con 0x55ec206d0340 2024-05-08T18:06:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.153+0000 7f850ac98800 20 add_watcher() i=0 2024-05-08T18:06:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.153+0000 7f850ac98800 20 add_watcher() i=1 2024-05-08T18:06:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.153+0000 7f850ac98800 20 add_watcher() i=2 2024-05-08T18:06:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.153+0000 7f850ac98800 20 add_watcher() i=3 2024-05-08T18:06:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.153+0000 7f850ac98800 20 add_watcher() i=4 2024-05-08T18:06:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.153+0000 7f850ac98800 20 add_watcher() i=5 2024-05-08T18:06:03.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.153+0000 7f850ac98800 20 add_watcher() i=6 2024-05-08T18:06:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.154+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94472644718336] v29'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.154+0000 7f850ac98800 20 add_watcher() i=7 2024-05-08T18:06:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.154+0000 7f850ac98800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.154+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec206e2c80 con 0x55ec206cb980 2024-05-08T18:06:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.154+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f8500032040 con 0x55ec206cb980 2024-05-08T18:06:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.154+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94472644720544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206e00b0 con 0x55ec206cb980 2024-05-08T18:06:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.155+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94472644720544] v29'200 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8500032040 con 0x55ec206cb980 2024-05-08T18:06:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.155+0000 7f83be7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.155+0000 7f850ac98800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.155+0000 7f850ac98800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.155+0000 7f850ac98800 5 note: GC not initialized 2024-05-08T18:06:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.155+0000 7f83bd7e2640 20 reqs_thread_entry: start 2024-05-08T18:06:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.156+0000 7f850ac98800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:03.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.156+0000 7f850ac98800 20 Filter name: none 2024-05-08T18:06:03.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.156+0000 7f832effd640 20 reqs_thread_entry: start 2024-05-08T18:06:03.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.156+0000 7f850ac98800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.156+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec20715c00 con 0x55ec206cb980 2024-05-08T18:06:03.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.156+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f8500032040 con 0x55ec206cb980 2024-05-08T18:06:03.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.156+0000 7f850ac98800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:06:03.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.156+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ec20717150 con 0x55ec206d0340 2024-05-08T18:06:03.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.157+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(63 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.158+0000 7f83be7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:03.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.158+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94472644720544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206c58f0 con 0x55ec206cb980 2024-05-08T18:06:03.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.159+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94472644720544] v29'201 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8500032040 con 0x55ec206cb980 2024-05-08T18:06:03.168 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.159+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94472644694208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206df7a0 con 0x7f84fc074040 2024-05-08T18:06:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.160+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94472644694208] v29'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8500032040 con 0x7f84fc074040 2024-05-08T18:06:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.160+0000 7f850ac98800 20 remove_watcher() i=0 2024-05-08T18:06:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.160+0000 7f850ac98800 2 removed watcher, disabling cache 2024-05-08T18:06:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.160+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94472644692160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206d90c0 con 0x55ec206cb980 2024-05-08T18:06:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.161+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94472644692160] v29'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8500032040 con 0x55ec206cb980 2024-05-08T18:06:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.161+0000 7f850ac98800 20 remove_watcher() i=1 2024-05-08T18:06:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.161+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94472644700512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206d90c0 con 0x55ec206d4d00 2024-05-08T18:06:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.162+0000 7f8507f8f640 1 -- 172.21.15.50:0/4270235224 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94472644700512] v29'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f84fc07f070 con 0x55ec206d4d00 2024-05-08T18:06:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.162+0000 7f850ac98800 20 remove_watcher() i=2 2024-05-08T18:06:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.162+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94472644702816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206d90c0 con 0x7f84fc074040 2024-05-08T18:06:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.163+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94472644702816] v29'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8500032040 con 0x7f84fc074040 2024-05-08T18:06:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.163+0000 7f850ac98800 20 remove_watcher() i=3 2024-05-08T18:06:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.163+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94472644707728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206db260 con 0x55ec206cb980 2024-05-08T18:06:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.164+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94472644707728] v29'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8500032040 con 0x55ec206cb980 2024-05-08T18:06:03.169 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.164+0000 7f850ac98800 20 remove_watcher() i=4 2024-05-08T18:06:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.164+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94472644705536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206d90c0 con 0x55ec206d0340 2024-05-08T18:06:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.165+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94472644705536] v29'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.166+0000 7f850ac98800 20 remove_watcher() i=5 2024-05-08T18:06:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.166+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94472644715152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206dc590 con 0x55ec206d0340 2024-05-08T18:06:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.166+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94472644715152] v29'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.166+0000 7f850ac98800 20 remove_watcher() i=6 2024-05-08T18:06:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.166+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94472644718336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ec206dbd00 con 0x55ec206d0340 2024-05-08T18:06:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.167+0000 7f8508790640 1 -- 172.21.15.50:0/4270235224 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94472644718336] v29'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f84f8037020 con 0x55ec206d0340 2024-05-08T18:06:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.167+0000 7f850ac98800 20 remove_watcher() i=7 2024-05-08T18:06:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.167+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec206d0340 msgr2=0x55ec206d2730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.167+0000 7f850ac98800 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec206d0340 0x55ec206d2730 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.167+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec206cb980 msgr2=0x55ec206cde40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.167+0000 7f850ac98800 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec206cb980 0x55ec206cde40 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.167+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f84fc074040 msgr2=0x7f84fc081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.167+0000 7f850ac98800 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f84fc074040 0x7f84fc081410 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.167+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec206d4d00 msgr2=0x55ec206d71c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.167+0000 7f850ac98800 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec206d4d00 0x55ec206d71c0 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f83d003bba0 msgr2=0x7f83d003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f83d003bba0 0x7f83d003e060 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55ec2069dfa0 tx=0x7f84fc04f000 comp rx=0 tx=0).stop 2024-05-08T18:06:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206b8fd0 msgr2=0x55ec206c03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 --2- 172.21.15.50:0/4270235224 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206b8fd0 0x55ec206c03e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f84f801be20 tx=0x7f84f801ba40 comp rx=0 tx=0).stop 2024-05-08T18:06:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 reap_dead start 2024-05-08T18:06:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f8508f91640 1 -- 172.21.15.50:0/4270235224 reap_dead start 2024-05-08T18:06:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 shutdown_connections 2024-05-08T18:06:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 >> 172.21.15.50:0/4270235224 conn(0x55ec206b5330 msgr2=0x55ec206b2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 shutdown_connections 2024-05-08T18:06:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 -- 172.21.15.50:0/4270235224 wait complete. 2024-05-08T18:06:03.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 -- 172.21.15.50:0/528963479 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f83e003bba0 msgr2=0x7f83e003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 --2- 172.21.15.50:0/528963479 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f83e003bba0 0x7f83e003e060 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55ec20695720 tx=0x7f84fc005ac0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 -- 172.21.15.50:0/528963479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206abf10 msgr2=0x55ec206ac330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 --2- 172.21.15.50:0/528963479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206abf10 0x55ec206ac330 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f84f8051800 tx=0x7f84f8050510 comp rx=0 tx=0).stop 2024-05-08T18:06:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 -- 172.21.15.50:0/528963479 shutdown_connections 2024-05-08T18:06:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 --2- 172.21.15.50:0/528963479 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f83e003bba0 0x7f83e003e060 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 --2- 172.21.15.50:0/528963479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec206abf10 0x55ec206ac330 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 -- 172.21.15.50:0/528963479 >> 172.21.15.50:0/528963479 conn(0x55ec2069ffb0 msgr2=0x55ec206a03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 -- 172.21.15.50:0/528963479 shutdown_connections 2024-05-08T18:06:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.168+0000 7f850ac98800 1 -- 172.21.15.50:0/528963479 wait complete. 2024-05-08T18:06:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.169+0000 7f850ac98800 1 -- 172.21.15.50:0/3284099190 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec20697170 msgr2=0x55ec20699560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.169+0000 7f850ac98800 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ec20697170 0x55ec20699560 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.169+0000 7f850ac98800 1 -- 172.21.15.50:0/3284099190 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec20597a00 msgr2=0x55ec20694800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.169+0000 7f850ac98800 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ec20597a00 0x55ec20694800 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.169+0000 7f850ac98800 1 -- 172.21.15.50:0/3284099190 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec2069b0b0 msgr2=0x55ec2069d4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.169+0000 7f850ac98800 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ec2069b0b0 0x55ec2069d4a0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.169+0000 7f850ac98800 1 -- 172.21.15.50:0/3284099190 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f83f003bba0 msgr2=0x7f83f003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.169+0000 7f850ac98800 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f83f003bba0 0x7f83f003e060 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f84fc054000 tx=0x7f84fc04e000 comp rx=0 tx=0).stop 2024-05-08T18:06:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.169+0000 7f850ac98800 1 -- 172.21.15.50:0/3284099190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20593a50 msgr2=0x55ec20690290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.169+0000 7f850ac98800 1 --2- 172.21.15.50:0/3284099190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec20593a50 0x55ec20690290 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f84f8000f60 tx=0x7f84f8000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.169+0000 7f8508f91640 1 -- 172.21.15.50:0/3284099190 reap_dead start 2024-05-08T18:06:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.169+0000 7f850ac98800 1 -- 172.21.15.50:0/3284099190 shutdown_connections 2024-05-08T18:06:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.169+0000 7f850ac98800 1 -- 172.21.15.50:0/3284099190 >> 172.21.15.50:0/3284099190 conn(0x55ec2058e2d0 msgr2=0x55ec2067eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.169+0000 7f850ac98800 1 -- 172.21.15.50:0/3284099190 shutdown_connections 2024-05-08T18:06:03.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.169+0000 7f850ac98800 1 -- 172.21.15.50:0/3284099190 wait complete. 2024-05-08T18:06:03.176 INFO:teuthology.orchestra.run.smithi050.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":5,"successful_ops":5},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":2,"successful_ops":1},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo2","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo3","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}},{"bucket":"myfoo4","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{"bytes_processed":0,"bytes_returned":0}}]}],"summary":[{"user":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"total":{"bytes_sent":2103,"bytes_received":30000042,"ops":35,"successful_ops":34,"bytes_processed":0,"bytes_returned":0}}]} 2024-05-08T18:06:03.176 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo2', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo3', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo4', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2103, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-05-08T18:06:03.176 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2024-05-08T18:06:03.176 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2024-05-08T18:06:03.177 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories create_bucket 2024-05-08T18:06:03.224 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:03.224 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:03.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.227+0000 7fd1f991f800 1 Processor -- start 2024-05-08T18:06:03.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.227+0000 7fd1f991f800 1 -- start start 2024-05-08T18:06:03.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.227+0000 7fd1f991f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x555ab0985b10 0x555ab098e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.227+0000 7fd1f991f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x555ab08f9010 con 0x555ab0983fd0 2024-05-08T18:06:03.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.227+0000 7fd1f991f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x555ab0825e20 con 0x555ab0985b10 2024-05-08T18:06:03.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.227+0000 7fd1effff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x555ab0985b10 0x555ab098e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.227+0000 7fd1f5652640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x555ab0983fd0 0x555ab09843f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:54536/0 (socket says 172.21.15.50:54536) 2024-05-08T18:06:03.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.227+0000 7fd1f5652640 1 -- 172.21.15.50:0/953780677 learned_addr learned my addr 172.21.15.50:0/953780677 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:03.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.227+0000 7fd1effff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x555ab0985b10 0x555ab098e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48720/0 (socket says 172.21.15.50:48720) 2024-05-08T18:06:03.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.228+0000 7fd1effff640 1 -- 172.21.15.50:0/953780677 >> v1:172.21.15.46:6789/0 conn(0x555ab0983fd0 legacy=0x555ab09843f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.228+0000 7fd1f4e51640 1 -- 172.21.15.50:0/953780677 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 613667274 0 0) 0x555ab08f9010 con 0x555ab0983fd0 2024-05-08T18:06:03.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.228+0000 7fd1effff640 1 -- 172.21.15.50:0/953780677 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ab08f91e0 con 0x555ab0985b10 2024-05-08T18:06:03.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.230+0000 7fd1effff640 1 --2- 172.21.15.50:0/953780677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0985b10 0x555ab098e610 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000a470 tx=0x7fd1e001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ecd17a7ba6f67f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.230+0000 7fd1f4e51640 1 -- 172.21.15.50:0/953780677 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e0057070 con 0x555ab0985b10 2024-05-08T18:06:03.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.230+0000 7fd1f4e51640 1 -- 172.21.15.50:0/953780677 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e0052570 con 0x555ab0985b10 2024-05-08T18:06:03.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.230+0000 7fd1f4e51640 1 -- 172.21.15.50:0/953780677 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e0052870 con 0x555ab0985b10 2024-05-08T18:06:03.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.230+0000 7fd1f991f800 1 -- 172.21.15.50:0/953780677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0985b10 msgr2=0x555ab098e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.230+0000 7fd1f991f800 1 --2- 172.21.15.50:0/953780677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0985b10 0x555ab098e610 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd1e000a470 tx=0x7fd1e001ce90 comp rx=0 tx=0).stop 2024-05-08T18:06:03.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.230+0000 7fd1f991f800 1 -- 172.21.15.50:0/953780677 shutdown_connections 2024-05-08T18:06:03.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.230+0000 7fd1f991f800 1 --2- 172.21.15.50:0/953780677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0985b10 0x555ab098e610 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.230+0000 7fd1f991f800 1 -- 172.21.15.50:0/953780677 >> 172.21.15.50:0/953780677 conn(0x555ab097faf0 msgr2=0x555ab0981f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.230+0000 7fd1f991f800 1 -- 172.21.15.50:0/953780677 shutdown_connections 2024-05-08T18:06:03.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.230+0000 7fd1f991f800 1 -- 172.21.15.50:0/953780677 wait complete. 2024-05-08T18:06:03.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.245+0000 7fd1f991f800 1 Processor -- start 2024-05-08T18:06:03.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.245+0000 7fd1f991f800 1 -- start start 2024-05-08T18:06:03.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.246+0000 7fd1f991f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0983fd0 0x555ab0a78b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.246+0000 7fd1f991f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555ab08d0a30 con 0x555ab0983fd0 2024-05-08T18:06:03.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.246+0000 7fd1f5652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0983fd0 0x555ab0a78b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.246+0000 7fd1f5652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0983fd0 0x555ab0a78b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48736/0 (socket says 172.21.15.50:48736) 2024-05-08T18:06:03.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.246+0000 7fd1f5652640 1 -- 172.21.15.50:0/364014390 learned_addr learned my addr 172.21.15.50:0/364014390 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.246+0000 7fd1f5652640 1 -- 172.21.15.50:0/364014390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ab0a79140 con 0x555ab0983fd0 2024-05-08T18:06:03.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.247+0000 7fd1f5652640 1 --2- 172.21.15.50:0/364014390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0983fd0 0x555ab0a78b70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8001e70 tx=0x7fd1e800b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21a75df75f773437 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.247+0000 7fd0e3606640 1 -- 172.21.15.50:0/364014390 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e8008070 con 0x555ab0983fd0 2024-05-08T18:06:03.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.247+0000 7fd0e3606640 1 -- 172.21.15.50:0/364014390 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e800d500 con 0x555ab0983fd0 2024-05-08T18:06:03.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.247+0000 7fd0e3606640 1 -- 172.21.15.50:0/364014390 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e800d8a0 con 0x555ab0983fd0 2024-05-08T18:06:03.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.247+0000 7fd1f991f800 1 -- 172.21.15.50:0/364014390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0983fd0 msgr2=0x555ab0a78b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.247+0000 7fd1f991f800 1 --2- 172.21.15.50:0/364014390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0983fd0 0x555ab0a78b70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8001e70 tx=0x7fd1e800b6a0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.247+0000 7fd1f991f800 1 -- 172.21.15.50:0/364014390 shutdown_connections 2024-05-08T18:06:03.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.247+0000 7fd1f991f800 1 --2- 172.21.15.50:0/364014390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0983fd0 0x555ab0a78b70 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.247+0000 7fd1f991f800 1 -- 172.21.15.50:0/364014390 >> 172.21.15.50:0/364014390 conn(0x555ab097faf0 msgr2=0x555ab0a74490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.247+0000 7fd1f991f800 1 -- 172.21.15.50:0/364014390 shutdown_connections 2024-05-08T18:06:03.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.247+0000 7fd1f991f800 1 -- 172.21.15.50:0/364014390 wait complete. 2024-05-08T18:06:03.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.248+0000 7fd1f991f800 1 Processor -- start 2024-05-08T18:06:03.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.248+0000 7fd1f991f800 1 -- start start 2024-05-08T18:06:03.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.248+0000 7fd1f991f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0983fd0 0x555ab0a80750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.248+0000 7fd1f991f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555ab098ee20 con 0x555ab0983fd0 2024-05-08T18:06:03.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.248+0000 7fd1f5652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0983fd0 0x555ab0a80750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.248+0000 7fd1f5652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0983fd0 0x555ab0a80750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48750/0 (socket says 172.21.15.50:48750) 2024-05-08T18:06:03.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.248+0000 7fd1f5652640 1 -- 172.21.15.50:0/4092989047 learned_addr learned my addr 172.21.15.50:0/4092989047 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.249+0000 7fd1f5652640 1 -- 172.21.15.50:0/4092989047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ab0a80c90 con 0x555ab0983fd0 2024-05-08T18:06:03.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.249+0000 7fd1f5652640 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0983fd0 0x555ab0a80750 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x555ab0a79900 tx=0x7fd1e8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.249+0000 7fd0e1e03640 1 -- 172.21.15.50:0/4092989047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e800f980 con 0x555ab0983fd0 2024-05-08T18:06:03.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.249+0000 7fd0e1e03640 1 -- 172.21.15.50:0/4092989047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e800fb20 con 0x555ab0983fd0 2024-05-08T18:06:03.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.249+0000 7fd0e1e03640 1 -- 172.21.15.50:0/4092989047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e800fe00 con 0x555ab0983fd0 2024-05-08T18:06:03.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.249+0000 7fd1f991f800 1 -- 172.21.15.50:0/4092989047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555ab0a80f20 con 0x555ab0983fd0 2024-05-08T18:06:03.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.249+0000 7fd1f991f800 1 -- 172.21.15.50:0/4092989047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ab0a83b80 con 0x555ab0983fd0 2024-05-08T18:06:03.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.250+0000 7fd0e1e03640 1 -- 172.21.15.50:0/4092989047 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fd1e8017040 con 0x555ab0983fd0 2024-05-08T18:06:03.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.250+0000 7fd0e1e03640 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd0dc03bba0 0x7fd0dc03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.250+0000 7fd0e1e03640 1 -- 172.21.15.50:0/4092989047 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fd1e8015020 con 0x555ab0983fd0 2024-05-08T18:06:03.252 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.250+0000 7fd1effff640 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd0dc03bba0 0x7fd0dc03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.250+0000 7fd1f991f800 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555ab0985b10 0x555ab0a87940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.250+0000 7fd1f991f800 1 -- 172.21.15.50:0/4092989047 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0a87e80 con 0x555ab0985b10 2024-05-08T18:06:03.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.251+0000 7fd1f5e53640 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555ab0985b10 0x555ab0a87940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.251+0000 7fd1effff640 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd0dc03bba0 0x7fd0dc03e060 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0054000 tx=0x7fd1e004e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.251+0000 7fd1f5e53640 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555ab0985b10 0x555ab0a87940 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.251+0000 7fd1f5e53640 1 -- 172.21.15.50:0/4092989047 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd1f000a4a0 con 0x555ab0985b10 2024-05-08T18:06:03.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.251+0000 7fd1f991f800 1 -- 172.21.15.50:0/4092989047 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0a84f10 con 0x555ab0985b10 2024-05-08T18:06:03.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.251+0000 7fd1f5e53640 1 -- 172.21.15.50:0/4092989047 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd1f000a4a0 con 0x555ab0985b10 2024-05-08T18:06:03.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.251+0000 7fd1f991f800 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555ab0a8a190 0x555ab0a8c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.251+0000 7fd1f991f800 1 -- 172.21.15.50:0/4092989047 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0a8cc70 con 0x555ab0a8a190 2024-05-08T18:06:03.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.252+0000 7fd1f5652640 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555ab0a8a190 0x555ab0a8c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.252+0000 7fd1f5652640 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555ab0a8a190 0x555ab0a8c580 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.252+0000 7fd1f5652640 1 -- 172.21.15.50:0/4092989047 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fd1e802d070 con 0x555ab0a8a190 2024-05-08T18:06:03.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.252+0000 7fd1f991f800 1 -- 172.21.15.50:0/4092989047 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0a89030 con 0x555ab0985b10 2024-05-08T18:06:03.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.253+0000 7fd1f5e53640 1 -- 172.21.15.50:0/4092989047 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd1f000a4a0 con 0x555ab0985b10 2024-05-08T18:06:03.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.253+0000 7fd1f991f800 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555ab0a8e0d0 0x555ab0a904c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.253+0000 7fd1f991f800 1 -- 172.21.15.50:0/4092989047 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0a90bc0 con 0x555ab0a8e0d0 2024-05-08T18:06:03.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.253+0000 7fd1effff640 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555ab0a8e0d0 0x555ab0a904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.253+0000 7fd1effff640 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555ab0a8e0d0 0x555ab0a904c0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.253+0000 7fd1effff640 1 -- 172.21.15.50:0/4092989047 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fd1e00528e0 con 0x555ab0a8e0d0 2024-05-08T18:06:03.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.254+0000 7fd1f991f800 1 Processor -- start 2024-05-08T18:06:03.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.254+0000 7fd1f991f800 1 -- start start 2024-05-08T18:06:03.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.254+0000 7fd1f991f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0a9eef0 0x555ab0a9f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.254+0000 7fd1f991f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555ab0877330 con 0x555ab0a9eef0 2024-05-08T18:06:03.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.254+0000 7fd1f5652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0a9eef0 0x555ab0a9f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.254+0000 7fd1f5652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0a9eef0 0x555ab0a9f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48758/0 (socket says 172.21.15.50:48758) 2024-05-08T18:06:03.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.254+0000 7fd1f5652640 1 -- 172.21.15.50:0/1567997072 learned_addr learned my addr 172.21.15.50:0/1567997072 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.255+0000 7fd1f5652640 1 -- 172.21.15.50:0/1567997072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ab0a9f850 con 0x555ab0a9eef0 2024-05-08T18:06:03.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.255+0000 7fd1f5652640 1 --2- 172.21.15.50:0/1567997072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0a9eef0 0x555ab0a9f310 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800ecf0 tx=0x7fd1e801ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.255+0000 7fd0daffd640 1 -- 172.21.15.50:0/1567997072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e804a450 con 0x555ab0a9eef0 2024-05-08T18:06:03.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.255+0000 7fd0daffd640 1 -- 172.21.15.50:0/1567997072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e8047020 con 0x555ab0a9eef0 2024-05-08T18:06:03.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.255+0000 7fd1f991f800 1 -- 172.21.15.50:0/1567997072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555ab0aa02c0 con 0x555ab0a9eef0 2024-05-08T18:06:03.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.255+0000 7fd0daffd640 1 -- 172.21.15.50:0/1567997072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e8053200 con 0x555ab0a9eef0 2024-05-08T18:06:03.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.255+0000 7fd1f991f800 1 -- 172.21.15.50:0/1567997072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ab0aa07c0 con 0x555ab0a9eef0 2024-05-08T18:06:03.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.256+0000 7fd0daffd640 1 -- 172.21.15.50:0/1567997072 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fd1e80183d0 con 0x555ab0a9eef0 2024-05-08T18:06:03.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.256+0000 7fd0daffd640 1 --2- 172.21.15.50:0/1567997072 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd0d003bb50 0x7fd0d003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.256+0000 7fd0daffd640 1 -- 172.21.15.50:0/1567997072 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fd1e8048030 con 0x555ab0a9eef0 2024-05-08T18:06:03.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.256+0000 7fd1effff640 1 --2- 172.21.15.50:0/1567997072 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd0d003bb50 0x7fd0d003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.256+0000 7fd1f991f800 1 Processor -- start 2024-05-08T18:06:03.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.256+0000 7fd1f991f800 1 -- start start 2024-05-08T18:06:03.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.257+0000 7fd1f991f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0aac2e0 0x555ab0aac700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.257+0000 7fd1f991f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555ab0a794c0 con 0x555ab0aac2e0 2024-05-08T18:06:03.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.257+0000 7fd1effff640 1 --2- 172.21.15.50:0/1567997072 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd0d003bb50 0x7fd0d003e010 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x555ab0a88740 tx=0x7fd1e0005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.257+0000 7fd1f5652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0aac2e0 0x555ab0aac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.257+0000 7fd1f5652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0aac2e0 0x555ab0aac700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48768/0 (socket says 172.21.15.50:48768) 2024-05-08T18:06:03.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.257+0000 7fd1f5652640 1 -- 172.21.15.50:0/3221636627 learned_addr learned my addr 172.21.15.50:0/3221636627 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.257+0000 7fd1f5652640 1 -- 172.21.15.50:0/3221636627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ab0aaccd0 con 0x555ab0aac2e0 2024-05-08T18:06:03.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.257+0000 7fd1f5652640 1 --2- 172.21.15.50:0/3221636627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0aac2e0 0x555ab0aac700 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd1e801ac70 tx=0x7fd1e8051a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8edf6e19a217c27 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.257+0000 7fd0d8ff9640 1 -- 172.21.15.50:0/3221636627 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e802f3e0 con 0x555ab0aac2e0 2024-05-08T18:06:03.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.257+0000 7fd0d8ff9640 1 -- 172.21.15.50:0/3221636627 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e802fe70 con 0x555ab0aac2e0 2024-05-08T18:06:03.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.257+0000 7fd0d8ff9640 1 -- 172.21.15.50:0/3221636627 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e801d040 con 0x555ab0aac2e0 2024-05-08T18:06:03.262 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.258+0000 7fd1f991f800 1 -- 172.21.15.50:0/3221636627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0aac2e0 msgr2=0x555ab0aac700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.258+0000 7fd1f991f800 1 --2- 172.21.15.50:0/3221636627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0aac2e0 0x555ab0aac700 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd1e801ac70 tx=0x7fd1e8051a40 comp rx=0 tx=0).stop 2024-05-08T18:06:03.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.258+0000 7fd1f991f800 1 -- 172.21.15.50:0/3221636627 shutdown_connections 2024-05-08T18:06:03.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.258+0000 7fd1f991f800 1 --2- 172.21.15.50:0/3221636627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0aac2e0 0x555ab0aac700 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.258+0000 7fd1f991f800 1 -- 172.21.15.50:0/3221636627 >> 172.21.15.50:0/3221636627 conn(0x555ab0aa85b0 msgr2=0x555ab0aa6610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.258+0000 7fd1f991f800 1 -- 172.21.15.50:0/3221636627 shutdown_connections 2024-05-08T18:06:03.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.258+0000 7fd1f991f800 1 -- 172.21.15.50:0/3221636627 wait complete. 2024-05-08T18:06:03.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.258+0000 7fd1f991f800 1 Processor -- start 2024-05-08T18:06:03.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.258+0000 7fd1f991f800 1 -- start start 2024-05-08T18:06:03.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.258+0000 7fd1f991f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0aac2e0 0x555ab0ab36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.258+0000 7fd1f991f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555ab0aad3a0 con 0x555ab0aac2e0 2024-05-08T18:06:03.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.259+0000 7fd1f5652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0aac2e0 0x555ab0ab36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.259+0000 7fd1f5652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0aac2e0 0x555ab0ab36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48774/0 (socket says 172.21.15.50:48774) 2024-05-08T18:06:03.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.259+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 learned_addr learned my addr 172.21.15.50:0/3258234774 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.259+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ab0ab3bf0 con 0x555ab0aac2e0 2024-05-08T18:06:03.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.259+0000 7fd1f5652640 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0aac2e0 0x555ab0ab36b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8050650 tx=0x7fd1e8049f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.259+0000 7fd0c6ffd640 1 -- 172.21.15.50:0/3258234774 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e806e070 con 0x555ab0aac2e0 2024-05-08T18:06:03.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.259+0000 7fd0c6ffd640 1 -- 172.21.15.50:0/3258234774 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e801c9f0 con 0x555ab0aac2e0 2024-05-08T18:06:03.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.259+0000 7fd0c6ffd640 1 -- 172.21.15.50:0/3258234774 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1e801ccd0 con 0x555ab0aac2e0 2024-05-08T18:06:03.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.259+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555ab0ab3bf0 con 0x555ab0aac2e0 2024-05-08T18:06:03.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.260+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ab0ab3bf0 con 0x555ab0aac2e0 2024-05-08T18:06:03.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.260+0000 7fd0c6ffd640 1 -- 172.21.15.50:0/3258234774 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fd1e8071070 con 0x555ab0aac2e0 2024-05-08T18:06:03.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.260+0000 7fd0c6ffd640 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd0c003bb50 0x7fd0c003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.261+0000 7fd0c6ffd640 1 -- 172.21.15.50:0/3258234774 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fd1e805a030 con 0x555ab0aac2e0 2024-05-08T18:06:03.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.261+0000 7fd1effff640 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd0c003bb50 0x7fd0c003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.261+0000 7fd1f991f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.261+0000 7fd1f991f800 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555ab0abedd0 0x555ab0ac1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.261+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac19e0 con 0x555ab0abedd0 2024-05-08T18:06:03.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.261+0000 7fd1f5e53640 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555ab0abedd0 0x555ab0ac1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.261+0000 7fd1effff640 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd0c003bb50 0x7fd0c003e010 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0052fd0 tx=0x7fd1e004f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.261+0000 7fd1f5e53640 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555ab0abedd0 0x555ab0ac1290 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd1f000a4a0 con 0x555ab0abedd0 2024-05-08T18:06:03.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f991f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f991f800 20 realm 2024-05-08T18:06:03.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f991f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac1df0 con 0x555ab0abedd0 2024-05-08T18:06:03.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd1f000a4a0 con 0x555ab0abedd0 2024-05-08T18:06:03.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f991f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f991f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:03.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f991f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac26c0 con 0x555ab0abedd0 2024-05-08T18:06:03.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd1f000a4a0 con 0x555ab0abedd0 2024-05-08T18:06:03.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f991f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f991f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac2a60 con 0x555ab0abedd0 2024-05-08T18:06:03.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd1f000a4a0 con 0x555ab0abedd0 2024-05-08T18:06:03.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f991f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:03.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f991f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f991f800 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555ab0ac3550 0x555ab0ac5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.262+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac60a0 con 0x555ab0ac3550 2024-05-08T18:06:03.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.263+0000 7fd1f5652640 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555ab0ac3550 0x555ab0ac5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.263+0000 7fd1f5652640 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555ab0ac3550 0x555ab0ac5940 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.263+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.263+0000 7fd1f991f800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:03.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.263+0000 7fd1f991f800 20 searching for the correct realm 2024-05-08T18:06:03.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.263+0000 7fd1f991f800 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555ab0ac8150 0x555ab0aca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.263+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x555ab0acace0 con 0x555ab0ac8150 2024-05-08T18:06:03.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.263+0000 7fd1effff640 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555ab0ac8150 0x555ab0aca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.264+0000 7fd1effff640 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555ab0ac8150 0x555ab0aca610 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.264+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e0081070 con 0x555ab0ac8150 2024-05-08T18:06:03.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.264+0000 7fd1effff640 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fd1e0076040 0x7fd1e005e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.264+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x555ab0acace0 con 0x7fd1e0076040 2024-05-08T18:06:03.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.264+0000 7fd1f5e53640 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fd1e0076040 0x7fd1e005e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.265+0000 7fd1f5e53640 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fd1e0076040 0x7fd1e005e480 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.265+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1f000a4a0 con 0x7fd1e0076040 2024-05-08T18:06:03.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.265+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac2a60 con 0x555ab0ac3550 2024-05-08T18:06:03.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.265+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.265+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac2a60 con 0x555ab0abedd0 2024-05-08T18:06:03.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.265+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1f000a4a0 con 0x555ab0abedd0 2024-05-08T18:06:03.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.265+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac2a60 con 0x555ab0abedd0 2024-05-08T18:06:03.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.266+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1f000a4a0 con 0x555ab0abedd0 2024-05-08T18:06:03.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.266+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac2a60 con 0x555ab0abedd0 2024-05-08T18:06:03.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.266+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1f000a4a0 con 0x555ab0abedd0 2024-05-08T18:06:03.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.266+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac2a60 con 0x555ab0ac3550 2024-05-08T18:06:03.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.266+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.266+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac2a60 con 0x7fd1e0076040 2024-05-08T18:06:03.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.267+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1f000a4a0 con 0x7fd1e0076040 2024-05-08T18:06:03.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.267+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac2a60 con 0x555ab0ac8150 2024-05-08T18:06:03.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.267+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e0081070 con 0x555ab0ac8150 2024-05-08T18:06:03.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.267+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac2a60 con 0x555ab0abedd0 2024-05-08T18:06:03.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.267+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1f000a4a0 con 0x555ab0abedd0 2024-05-08T18:06:03.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.267+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac2a60 con 0x555ab0ac3550 2024-05-08T18:06:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.267+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.267+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac2a60 con 0x555ab0abedd0 2024-05-08T18:06:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.268+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1f000a4a0 con 0x555ab0abedd0 2024-05-08T18:06:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.268+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac2a60 con 0x7fd1e0076040 2024-05-08T18:06:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.268+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fd1f0032040 con 0x7fd1e0076040 2024-05-08T18:06:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.268+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac2a60 con 0x555ab0ac8150 2024-05-08T18:06:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.268+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e0081070 con 0x555ab0ac8150 2024-05-08T18:06:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.268+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x555ab0ac8150 2024-05-08T18:06:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.268+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e0081070 con 0x555ab0ac8150 2024-05-08T18:06:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.268+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x555ab0ac3550 2024-05-08T18:06:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.269+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.269+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x555ab0ac8150 2024-05-08T18:06:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.269+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e0081070 con 0x555ab0ac8150 2024-05-08T18:06:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.269+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x555ab0ac8150 2024-05-08T18:06:03.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.269+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fd1e0081070 con 0x555ab0ac8150 2024-05-08T18:06:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.269+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x555ab0abedd0 2024-05-08T18:06:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.269+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1f0032040 con 0x555ab0abedd0 2024-05-08T18:06:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.269+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x555ab0abedd0 2024-05-08T18:06:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.270+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fd1f0032040 con 0x555ab0abedd0 2024-05-08T18:06:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.270+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x555ab0ac3550 2024-05-08T18:06:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.270+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.270+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x555ab0ac8150 2024-05-08T18:06:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.270+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e0081070 con 0x555ab0ac8150 2024-05-08T18:06:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.270+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x7fd1e0076040 2024-05-08T18:06:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.270+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1f0032040 con 0x7fd1e0076040 2024-05-08T18:06:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.270+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x555ab0abedd0 2024-05-08T18:06:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.271+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1f0032040 con 0x555ab0abedd0 2024-05-08T18:06:03.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.271+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x555ab0ac3550 2024-05-08T18:06:03.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.271+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.271+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x555ab0ac8150 2024-05-08T18:06:03.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.271+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e0081070 con 0x555ab0ac8150 2024-05-08T18:06:03.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.271+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x555ab0ac3550 2024-05-08T18:06:03.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.271+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.272+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x7fd1e0076040 2024-05-08T18:06:03.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.272+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1f0032040 con 0x7fd1e0076040 2024-05-08T18:06:03.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.272+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x555ab0ac8150 2024-05-08T18:06:03.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.272+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e0081070 con 0x555ab0ac8150 2024-05-08T18:06:03.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.272+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x555ab0ac3550 2024-05-08T18:06:03.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.272+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.272+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x555ab0abedd0 2024-05-08T18:06:03.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fd1f0032040 con 0x555ab0abedd0 2024-05-08T18:06:03.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fd1e0090040 con 0x7fd1e0076040 2024-05-08T18:06:03.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd1f0032040 con 0x7fd1e0076040 2024-05-08T18:06:03.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f991f800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:03.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f991f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:03.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f991f800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:03.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f991f800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:03.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f991f800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:03.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f991f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:03.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f991f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac6a50 con 0x555ab0abedd0 2024-05-08T18:06:03.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd1f0032040 con 0x555ab0abedd0 2024-05-08T18:06:03.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f991f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f991f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac6a50 con 0x555ab0abedd0 2024-05-08T18:06:03.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd1f0032040 con 0x555ab0abedd0 2024-05-08T18:06:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f991f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f991f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.273+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac6a20 con 0x555ab0ac8150 2024-05-08T18:06:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fd1e0081070 con 0x555ab0ac8150 2024-05-08T18:06:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1f991f800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1f991f800 20 zone default found 2024-05-08T18:06:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1f991f800 4 Realm: () 2024-05-08T18:06:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1f991f800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1f991f800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1f991f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1f991f800 20 zonegroup default 2024-05-08T18:06:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1f991f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ac72f0 con 0x555ab0ac3550 2024-05-08T18:06:03.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1f991f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1f991f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0accb40 con 0x555ab0ac8150 2024-05-08T18:06:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd1e0081070 con 0x555ab0ac8150 2024-05-08T18:06:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1f991f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1f991f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0accee0 con 0x555ab0ac8150 2024-05-08T18:06:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.274+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd1e0081070 con 0x555ab0ac8150 2024-05-08T18:06:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.275+0000 7fd1f991f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.275+0000 7fd1f991f800 20 started sync module instance, tier type = 2024-05-08T18:06:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.275+0000 7fd1f991f800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.275+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0acbfe0 con 0x7fd1e0076040 2024-05-08T18:06:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.276+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1f0032040 con 0x7fd1e0076040 2024-05-08T18:06:03.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.276+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93847999530256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0acbfe0 con 0x7fd1e0076040 2024-05-08T18:06:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.276+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0acd8e0 con 0x555ab0abedd0 2024-05-08T18:06:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.277+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93847999530256] v29'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1f0032040 con 0x7fd1e0076040 2024-05-08T18:06:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.277+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1f0032040 con 0x555ab0abedd0 2024-05-08T18:06:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.277+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93847999534064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0acbfe0 con 0x555ab0abedd0 2024-05-08T18:06:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.277+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ace350 con 0x555ab0ac8150 2024-05-08T18:06:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.278+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93847999534064] v29'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1f0032040 con 0x555ab0abedd0 2024-05-08T18:06:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.278+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e0081070 con 0x555ab0ac8150 2024-05-08T18:06:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.278+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93847999527728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0acbf50 con 0x555ab0ac8150 2024-05-08T18:06:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.278+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0acf210 con 0x7fd1e0076040 2024-05-08T18:06:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.279+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93847999527728] v29'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e0081070 con 0x555ab0ac8150 2024-05-08T18:06:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.279+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1f0032040 con 0x7fd1e0076040 2024-05-08T18:06:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.279+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93847999540576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0acf630 con 0x7fd1e0076040 2024-05-08T18:06:03.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.279+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0acfff0 con 0x555ab0abedd0 2024-05-08T18:06:03.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.280+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1f0032040 con 0x555ab0abedd0 2024-05-08T18:06:03.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.280+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93847999544128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ad0410 con 0x555ab0abedd0 2024-05-08T18:06:03.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.281+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ad0eb0 con 0x555ab0ac3550 2024-05-08T18:06:03.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.281+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93847999540576] v29'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1f0032040 con 0x7fd1e0076040 2024-05-08T18:06:03.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.282+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.282+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93847999547776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ad12a0 con 0x555ab0ac3550 2024-05-08T18:06:03.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.282+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93847999544128] v29'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1f0032040 con 0x555ab0abedd0 2024-05-08T18:06:03.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.282+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0acff60 con 0x555ab0ac3550 2024-05-08T18:06:03.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.283+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(56 notify.6 [create] v29'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.283+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93847999551888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ad2320 con 0x555ab0ac3550 2024-05-08T18:06:03.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.283+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ad2e10 con 0x555ab0ac3550 2024-05-08T18:06:03.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.283+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93847999547776] v29'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.283+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93847999551888] v29'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.284+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.284+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93847999555808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ad2dd0 con 0x555ab0ac3550 2024-05-08T18:06:03.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.284+0000 7fd1f991f800 20 add_watcher() i=0 2024-05-08T18:06:03.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.284+0000 7fd1f991f800 20 add_watcher() i=1 2024-05-08T18:06:03.292 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.284+0000 7fd1f991f800 20 add_watcher() i=2 2024-05-08T18:06:03.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.284+0000 7fd1f991f800 20 add_watcher() i=3 2024-05-08T18:06:03.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.284+0000 7fd1f991f800 20 add_watcher() i=4 2024-05-08T18:06:03.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.284+0000 7fd1f991f800 20 add_watcher() i=5 2024-05-08T18:06:03.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.284+0000 7fd1f991f800 20 add_watcher() i=6 2024-05-08T18:06:03.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.285+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93847999555808] v29'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.285+0000 7fd1f991f800 20 add_watcher() i=7 2024-05-08T18:06:03.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.285+0000 7fd1f991f800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:03.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.285+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ad62b0 con 0x555ab0abedd0 2024-05-08T18:06:03.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.285+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fd1f0032040 con 0x555ab0abedd0 2024-05-08T18:06:03.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.285+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93847999557072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0ad36a0 con 0x555ab0abedd0 2024-05-08T18:06:03.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.286+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93847999557072] v29'202 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd1f0032040 con 0x555ab0abedd0 2024-05-08T18:06:03.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.286+0000 7fd0ab7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:03.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.286+0000 7fd1f991f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:03.293 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.286+0000 7fd1f991f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:03.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.286+0000 7fd1f991f800 5 note: GC not initialized 2024-05-08T18:06:03.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.286+0000 7fd0aa7e4640 20 reqs_thread_entry: start 2024-05-08T18:06:03.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.286+0000 7fd1f991f800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:03.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.287+0000 7fd1f991f800 20 Filter name: none 2024-05-08T18:06:03.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.287+0000 7fd01bfff640 20 reqs_thread_entry: start 2024-05-08T18:06:03.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.287+0000 7fd1f991f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.287+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0b091a0 con 0x555ab0abedd0 2024-05-08T18:06:03.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.287+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fd1f0032040 con 0x555ab0abedd0 2024-05-08T18:06:03.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.287+0000 7fd1f991f800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:06:03.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.287+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x555ab0b0a6f0 con 0x555ab0ac3550 2024-05-08T18:06:03.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.288+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(63 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.289+0000 7fd0ab7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:03.294 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.289+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93847999557072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0b07db0 con 0x555ab0abedd0 2024-05-08T18:06:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.290+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 93847999557072] v29'203 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd1f0032040 con 0x555ab0abedd0 2024-05-08T18:06:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.290+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93847999530256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0b07db0 con 0x7fd1e0076040 2024-05-08T18:06:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.291+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 93847999530256] v29'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1f0032040 con 0x7fd1e0076040 2024-05-08T18:06:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.291+0000 7fd1f991f800 20 remove_watcher() i=0 2024-05-08T18:06:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.291+0000 7fd1f991f800 2 removed watcher, disabling cache 2024-05-08T18:06:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.291+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93847999534064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0acc510 con 0x555ab0abedd0 2024-05-08T18:06:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.292+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 93847999534064] v29'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1f0032040 con 0x555ab0abedd0 2024-05-08T18:06:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.292+0000 7fd1f991f800 20 remove_watcher() i=1 2024-05-08T18:06:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.292+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93847999527728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0acd3f0 con 0x555ab0ac8150 2024-05-08T18:06:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.293+0000 7fd1effff640 1 -- 172.21.15.50:0/3258234774 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 93847999527728] v29'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e0081070 con 0x555ab0ac8150 2024-05-08T18:06:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.293+0000 7fd1f991f800 20 remove_watcher() i=2 2024-05-08T18:06:03.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.293+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93847999540576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0acd3f0 con 0x7fd1e0076040 2024-05-08T18:06:03.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.294+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 93847999540576] v29'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1f0032040 con 0x7fd1e0076040 2024-05-08T18:06:03.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.294+0000 7fd1f991f800 20 remove_watcher() i=3 2024-05-08T18:06:03.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.294+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93847999544128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0aced60 con 0x555ab0abedd0 2024-05-08T18:06:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.295+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 93847999544128] v29'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1f0032040 con 0x555ab0abedd0 2024-05-08T18:06:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.295+0000 7fd1f991f800 20 remove_watcher() i=4 2024-05-08T18:06:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.295+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93847999547776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0acd3f0 con 0x555ab0ac3550 2024-05-08T18:06:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.296+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 93847999547776] v29'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.296+0000 7fd1f991f800 20 remove_watcher() i=5 2024-05-08T18:06:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.296+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93847999551888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0acfb40 con 0x555ab0ac3550 2024-05-08T18:06:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.297+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 93847999551888] v29'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.297+0000 7fd1f991f800 20 remove_watcher() i=6 2024-05-08T18:06:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.297+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93847999555808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x555ab0aced60 con 0x555ab0ac3550 2024-05-08T18:06:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f5652640 1 -- 172.21.15.50:0/3258234774 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 93847999555808] v29'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd1e8070030 con 0x555ab0ac3550 2024-05-08T18:06:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f991f800 20 remove_watcher() i=7 2024-05-08T18:06:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555ab0ac3550 msgr2=0x555ab0ac5940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f991f800 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555ab0ac3550 0x555ab0ac5940 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555ab0abedd0 msgr2=0x555ab0ac1290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f991f800 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555ab0abedd0 0x555ab0ac1290 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fd1e0076040 msgr2=0x7fd1e005e480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f991f800 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fd1e0076040 0x7fd1e005e480 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555ab0ac8150 msgr2=0x555ab0aca610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f991f800 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555ab0ac8150 0x555ab0aca610 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd0c003bb50 msgr2=0x7fd0c003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f991f800 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd0c003bb50 0x7fd0c003e010 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0052fd0 tx=0x7fd1e004f000 comp rx=0 tx=0).stop 2024-05-08T18:06:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0aac2e0 msgr2=0x555ab0ab36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f5e53640 1 -- 172.21.15.50:0/3258234774 reap_dead start 2024-05-08T18:06:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f991f800 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0aac2e0 0x555ab0ab36b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd1e8050650 tx=0x7fd1e8049f40 comp rx=0 tx=0).stop 2024-05-08T18:06:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 shutdown_connections 2024-05-08T18:06:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f991f800 1 --2- 172.21.15.50:0/3258234774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0aac2e0 0x555ab0ab36b0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.298+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 >> 172.21.15.50:0/3258234774 conn(0x555ab0aa85b0 msgr2=0x555ab0aa5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 shutdown_connections 2024-05-08T18:06:03.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 -- 172.21.15.50:0/3258234774 wait complete. 2024-05-08T18:06:03.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 -- 172.21.15.50:0/1567997072 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd0d003bb50 msgr2=0x7fd0d003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 --2- 172.21.15.50:0/1567997072 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd0d003bb50 0x7fd0d003e010 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x555ab0a88740 tx=0x7fd1e0005a00 comp rx=0 tx=0).stop 2024-05-08T18:06:03.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 -- 172.21.15.50:0/1567997072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0a9eef0 msgr2=0x555ab0a9f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 --2- 172.21.15.50:0/1567997072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0a9eef0 0x555ab0a9f310 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800ecf0 tx=0x7fd1e801ab20 comp rx=0 tx=0).stop 2024-05-08T18:06:03.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 -- 172.21.15.50:0/1567997072 shutdown_connections 2024-05-08T18:06:03.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 --2- 172.21.15.50:0/1567997072 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd0d003bb50 0x7fd0d003e010 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 --2- 172.21.15.50:0/1567997072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0a9eef0 0x555ab0a9f310 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 -- 172.21.15.50:0/1567997072 >> 172.21.15.50:0/1567997072 conn(0x555ab0a92f90 msgr2=0x555ab0a93390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 -- 172.21.15.50:0/1567997072 shutdown_connections 2024-05-08T18:06:03.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 -- 172.21.15.50:0/1567997072 wait complete. 2024-05-08T18:06:03.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 -- 172.21.15.50:0/4092989047 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555ab0a8a190 msgr2=0x555ab0a8c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x555ab0a8a190 0x555ab0a8c580 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 -- 172.21.15.50:0/4092989047 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555ab0985b10 msgr2=0x555ab0a87940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x555ab0985b10 0x555ab0a87940 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 -- 172.21.15.50:0/4092989047 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555ab0a8e0d0 msgr2=0x555ab0a904c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x555ab0a8e0d0 0x555ab0a904c0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 -- 172.21.15.50:0/4092989047 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd0dc03bba0 msgr2=0x7fd0dc03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fd0dc03bba0 0x7fd0dc03e060 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0054000 tx=0x7fd1e004e000 comp rx=0 tx=0).stop 2024-05-08T18:06:03.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 -- 172.21.15.50:0/4092989047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0983fd0 msgr2=0x555ab0a80750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f991f800 1 --2- 172.21.15.50:0/4092989047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555ab0983fd0 0x555ab0a80750 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x555ab0a79900 tx=0x7fd1e8000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.299+0000 7fd1f5e53640 1 -- 172.21.15.50:0/4092989047 reap_dead start 2024-05-08T18:06:03.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.300+0000 7fd1f991f800 1 -- 172.21.15.50:0/4092989047 shutdown_connections 2024-05-08T18:06:03.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.300+0000 7fd1f991f800 1 -- 172.21.15.50:0/4092989047 >> 172.21.15.50:0/4092989047 conn(0x555ab097faf0 msgr2=0x555ab0a72c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.300+0000 7fd1f991f800 1 -- 172.21.15.50:0/4092989047 shutdown_connections 2024-05-08T18:06:03.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.300+0000 7fd1f991f800 1 -- 172.21.15.50:0/4092989047 wait complete. 2024-05-08T18:06:03.306 INFO:teuthology.orchestra.run.smithi050.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":5,"successful_ops":5}],"s3select":{}},{"bucket":"myfoo2","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}},{"bucket":"myfoo3","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}},{"bucket":"myfoo4","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}}]}],"summary":[{"user":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8}],"total":{"bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8,"bytes_processed":0,"bytes_returned":0}}]} 2024-05-08T18:06:03.307 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}], 's3select': {}}, {'bucket': 'myfoo2', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'myfoo3', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'myfoo4', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-05-08T18:06:03.307 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2024-05-08T18:06:03.307 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2024-05-08T18:06:03.307 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories put_obj 2024-05-08T18:06:03.355 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:03.355 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.358+0000 7f72bd12d800 1 Processor -- start 2024-05-08T18:06:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.358+0000 7f72bd12d800 1 -- start start 2024-05-08T18:06:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.358+0000 7f72bd12d800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5603a0915960 0x5603a0915d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.358+0000 7f72bd12d800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5603a09163b0 con 0x5603a0918950 2024-05-08T18:06:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.358+0000 7f72bd12d800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5603a07b7e20 con 0x5603a0915960 2024-05-08T18:06:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.359+0000 7f72b8d71640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5603a0915960 0x5603a0915d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.359+0000 7f72b8d71640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5603a0915960 0x5603a0915d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48784/0 (socket says 172.21.15.50:48784) 2024-05-08T18:06:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.359+0000 7f72b8d71640 1 -- 172.21.15.50:0/279334225 learned_addr learned my addr 172.21.15.50:0/279334225 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.359+0000 7f72b3fff640 1 --1- 172.21.15.50:0/279334225 >> v1:172.21.15.46:6789/0 conn(0x5603a0918950 0x5603a0918e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:54550/0 (socket says 172.21.15.50:54550) 2024-05-08T18:06:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.359+0000 7f72b8d71640 1 -- 172.21.15.50:0/279334225 >> v1:172.21.15.46:6789/0 conn(0x5603a0918950 legacy=0x5603a0918e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.359+0000 7f72b37fe640 1 -- 172.21.15.50:0/279334225 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3253049476 0 0) 0x5603a09163b0 con 0x5603a0918950 2024-05-08T18:06:03.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.359+0000 7f72b8d71640 1 -- 172.21.15.50:0/279334225 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a0919510 con 0x5603a0915960 2024-05-08T18:06:03.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.361+0000 7f72b8d71640 1 --2- 172.21.15.50:0/279334225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 0x5603a0915d80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00a520 tx=0x7f72ac01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c348584e7837f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.361+0000 7f72b37fe640 1 -- 172.21.15.50:0/279334225 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72ac057070 con 0x5603a0915960 2024-05-08T18:06:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.361+0000 7f72b37fe640 1 -- 172.21.15.50:0/279334225 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ac052570 con 0x5603a0915960 2024-05-08T18:06:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.361+0000 7f72b37fe640 1 -- 172.21.15.50:0/279334225 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72ac052870 con 0x5603a0915960 2024-05-08T18:06:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.361+0000 7f72bd12d800 1 -- 172.21.15.50:0/279334225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 msgr2=0x5603a0915d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.361+0000 7f72bd12d800 1 --2- 172.21.15.50:0/279334225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 0x5603a0915d80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f72ac00a520 tx=0x7f72ac01d010 comp rx=0 tx=0).stop 2024-05-08T18:06:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.362+0000 7f72bd12d800 1 -- 172.21.15.50:0/279334225 shutdown_connections 2024-05-08T18:06:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.362+0000 7f72bd12d800 1 --2- 172.21.15.50:0/279334225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 0x5603a0915d80 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.362+0000 7f72bd12d800 1 -- 172.21.15.50:0/279334225 >> 172.21.15.50:0/279334225 conn(0x5603a09112f0 msgr2=0x5603a0913710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.362+0000 7f72bd12d800 1 -- 172.21.15.50:0/279334225 shutdown_connections 2024-05-08T18:06:03.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.362+0000 7f72bd12d800 1 -- 172.21.15.50:0/279334225 wait complete. 2024-05-08T18:06:03.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.377+0000 7f72bd12d800 1 Processor -- start 2024-05-08T18:06:03.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.377+0000 7f72bd12d800 1 -- start start 2024-05-08T18:06:03.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.377+0000 7f72bd12d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 0x5603a0a08a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.377+0000 7f72bd12d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5603a0862a30 con 0x5603a0915960 2024-05-08T18:06:03.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.377+0000 7f72b8d71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 0x5603a0a08a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.377+0000 7f72b8d71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 0x5603a0a08a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48796/0 (socket says 172.21.15.50:48796) 2024-05-08T18:06:03.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.377+0000 7f72b8d71640 1 -- 172.21.15.50:0/651113393 learned_addr learned my addr 172.21.15.50:0/651113393 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.378+0000 7f72b8d71640 1 -- 172.21.15.50:0/651113393 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a0a08fd0 con 0x5603a0915960 2024-05-08T18:06:03.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.378+0000 7f72b8d71640 1 --2- 172.21.15.50:0/651113393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 0x5603a0a08a00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f72ac055040 tx=0x7f72ac04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb31ea6547764e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.378+0000 7f71a6e05640 1 -- 172.21.15.50:0/651113393 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72ac062070 con 0x5603a0915960 2024-05-08T18:06:03.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.378+0000 7f71a6e05640 1 -- 172.21.15.50:0/651113393 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ac052720 con 0x5603a0915960 2024-05-08T18:06:03.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.378+0000 7f71a6e05640 1 -- 172.21.15.50:0/651113393 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72ac057040 con 0x5603a0915960 2024-05-08T18:06:03.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.379+0000 7f72bd12d800 1 -- 172.21.15.50:0/651113393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 msgr2=0x5603a0a08a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.379+0000 7f72bd12d800 1 --2- 172.21.15.50:0/651113393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 0x5603a0a08a00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f72ac055040 tx=0x7f72ac04c000 comp rx=0 tx=0).stop 2024-05-08T18:06:03.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.379+0000 7f72bd12d800 1 -- 172.21.15.50:0/651113393 shutdown_connections 2024-05-08T18:06:03.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.379+0000 7f72bd12d800 1 --2- 172.21.15.50:0/651113393 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 0x5603a0a08a00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.379+0000 7f72bd12d800 1 -- 172.21.15.50:0/651113393 >> 172.21.15.50:0/651113393 conn(0x5603a09112f0 msgr2=0x5603a0a04320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.379+0000 7f72bd12d800 1 -- 172.21.15.50:0/651113393 shutdown_connections 2024-05-08T18:06:03.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.379+0000 7f72bd12d800 1 -- 172.21.15.50:0/651113393 wait complete. 2024-05-08T18:06:03.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.379+0000 7f72bd12d800 1 Processor -- start 2024-05-08T18:06:03.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.379+0000 7f72bd12d800 1 -- start start 2024-05-08T18:06:03.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.379+0000 7f72bd12d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 0x5603a0a104e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.379+0000 7f72bd12d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5603a0920e20 con 0x5603a0915960 2024-05-08T18:06:03.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.379+0000 7f72b8d71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 0x5603a0a104e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.379+0000 7f72b8d71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 0x5603a0a104e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48808/0 (socket says 172.21.15.50:48808) 2024-05-08T18:06:03.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.379+0000 7f72b8d71640 1 -- 172.21.15.50:0/3274465427 learned_addr learned my addr 172.21.15.50:0/3274465427 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.380+0000 7f72b8d71640 1 -- 172.21.15.50:0/3274465427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a0a10a20 con 0x5603a0915960 2024-05-08T18:06:03.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.380+0000 7f72b8d71640 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 0x5603a0a104e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f72ac000c00 tx=0x7f72ac002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.380+0000 7f71a5602640 1 -- 172.21.15.50:0/3274465427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72ac057050 con 0x5603a0915960 2024-05-08T18:06:03.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.380+0000 7f71a5602640 1 -- 172.21.15.50:0/3274465427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ac067580 con 0x5603a0915960 2024-05-08T18:06:03.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.380+0000 7f72bd12d800 1 -- 172.21.15.50:0/3274465427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5603a0a10cb0 con 0x5603a0915960 2024-05-08T18:06:03.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.380+0000 7f71a5602640 1 -- 172.21.15.50:0/3274465427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72ac062040 con 0x5603a0915960 2024-05-08T18:06:03.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.380+0000 7f72bd12d800 1 -- 172.21.15.50:0/3274465427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603a0a13960 con 0x5603a0915960 2024-05-08T18:06:03.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.381+0000 7f71a5602640 1 -- 172.21.15.50:0/3274465427 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f72ac067860 con 0x5603a0915960 2024-05-08T18:06:03.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.381+0000 7f71a5602640 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f71a003bba0 0x7f71a003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.381+0000 7f71a5602640 1 -- 172.21.15.50:0/3274465427 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f72ac0947a0 con 0x5603a0915960 2024-05-08T18:06:03.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.381+0000 7f72b3fff640 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f71a003bba0 0x7f71a003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.381+0000 7f72bd12d800 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5603a0918950 0x5603a0a17720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.381+0000 7f72bd12d800 1 -- 172.21.15.50:0/3274465427 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a17c60 con 0x5603a0918950 2024-05-08T18:06:03.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.381+0000 7f72b9572640 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5603a0918950 0x5603a0a17720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.382+0000 7f72b3fff640 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f71a003bba0 0x7f71a003e060 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f72a4000c00 tx=0x7f72a40013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.382+0000 7f72b9572640 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5603a0918950 0x5603a0a17720 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.382+0000 7f72b9572640 1 -- 172.21.15.50:0/3274465427 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f72b400a4a0 con 0x5603a0918950 2024-05-08T18:06:03.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.382+0000 7f72bd12d800 1 -- 172.21.15.50:0/3274465427 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a14cd0 con 0x5603a0918950 2024-05-08T18:06:03.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.382+0000 7f72b9572640 1 -- 172.21.15.50:0/3274465427 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f72b400a4a0 con 0x5603a0918950 2024-05-08T18:06:03.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.382+0000 7f72bd12d800 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5603a0a19f80 0x5603a0a1c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.382+0000 7f72bd12d800 1 -- 172.21.15.50:0/3274465427 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a1ca60 con 0x5603a0a19f80 2024-05-08T18:06:03.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.383+0000 7f72b8d71640 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5603a0a19f80 0x5603a0a1c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.383+0000 7f72b8d71640 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5603a0a19f80 0x5603a0a1c370 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.383+0000 7f72b8d71640 1 -- 172.21.15.50:0/3274465427 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a19f80 2024-05-08T18:06:03.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.383+0000 7f72bd12d800 1 -- 172.21.15.50:0/3274465427 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a18e20 con 0x5603a0918950 2024-05-08T18:06:03.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.383+0000 7f72b9572640 1 -- 172.21.15.50:0/3274465427 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f72b400a4a0 con 0x5603a0918950 2024-05-08T18:06:03.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.384+0000 7f72bd12d800 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5603a0a1dec0 0x5603a0a202b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.384+0000 7f72bd12d800 1 -- 172.21.15.50:0/3274465427 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a209b0 con 0x5603a0a1dec0 2024-05-08T18:06:03.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.384+0000 7f72b3fff640 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5603a0a1dec0 0x5603a0a202b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.384+0000 7f72b3fff640 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5603a0a1dec0 0x5603a0a202b0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.384+0000 7f72b3fff640 1 -- 172.21.15.50:0/3274465427 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f72a400f670 con 0x5603a0a1dec0 2024-05-08T18:06:03.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.385+0000 7f72bd12d800 1 Processor -- start 2024-05-08T18:06:03.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.385+0000 7f72bd12d800 1 -- start start 2024-05-08T18:06:03.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.385+0000 7f72bd12d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a09184f0 0x5603a0916960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.385+0000 7f72bd12d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5603a0809330 con 0x5603a09184f0 2024-05-08T18:06:03.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.385+0000 7f72b8d71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a09184f0 0x5603a0916960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.385+0000 7f72b8d71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a09184f0 0x5603a0916960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48822/0 (socket says 172.21.15.50:48822) 2024-05-08T18:06:03.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.385+0000 7f72b8d71640 1 -- 172.21.15.50:0/1445629105 learned_addr learned my addr 172.21.15.50:0/1445629105 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.385+0000 7f72b8d71640 1 -- 172.21.15.50:0/1445629105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a0916ea0 con 0x5603a09184f0 2024-05-08T18:06:03.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.386+0000 7f72b8d71640 1 --2- 172.21.15.50:0/1445629105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a09184f0 0x5603a0916960 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f72ac094010 tx=0x7f72ac05d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.386+0000 7f719e7fc640 1 -- 172.21.15.50:0/1445629105 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72ac05d960 con 0x5603a09184f0 2024-05-08T18:06:03.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.386+0000 7f719e7fc640 1 -- 172.21.15.50:0/1445629105 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ac05db00 con 0x5603a09184f0 2024-05-08T18:06:03.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.386+0000 7f719e7fc640 1 -- 172.21.15.50:0/1445629105 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72ac0940b0 con 0x5603a09184f0 2024-05-08T18:06:03.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.386+0000 7f72bd12d800 1 -- 172.21.15.50:0/1445629105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5603a0917130 con 0x5603a09184f0 2024-05-08T18:06:03.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.386+0000 7f72bd12d800 1 -- 172.21.15.50:0/1445629105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603a0917630 con 0x5603a09184f0 2024-05-08T18:06:03.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.387+0000 7f719e7fc640 1 -- 172.21.15.50:0/1445629105 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f72ac094250 con 0x5603a09184f0 2024-05-08T18:06:03.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.387+0000 7f719e7fc640 1 --2- 172.21.15.50:0/1445629105 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f719003bb00 0x7f719003dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.387+0000 7f719e7fc640 1 -- 172.21.15.50:0/1445629105 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f72ac07f040 con 0x5603a09184f0 2024-05-08T18:06:03.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.387+0000 7f72b3fff640 1 --2- 172.21.15.50:0/1445629105 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f719003bb00 0x7f719003dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.387+0000 7f72bd12d800 1 Processor -- start 2024-05-08T18:06:03.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.387+0000 7f72bd12d800 1 -- start start 2024-05-08T18:06:03.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.387+0000 7f72b3fff640 1 --2- 172.21.15.50:0/1445629105 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f719003bb00 0x7f719003dfc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5603a0a18530 tx=0x7f72a4006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.387+0000 7f72bd12d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0a3e110 0x5603a0a3e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.387+0000 7f72bd12d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5603a0a09290 con 0x5603a0a3e110 2024-05-08T18:06:03.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.388+0000 7f72b8d71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0a3e110 0x5603a0a3e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.388+0000 7f72b8d71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0a3e110 0x5603a0a3e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48832/0 (socket says 172.21.15.50:48832) 2024-05-08T18:06:03.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.388+0000 7f72b8d71640 1 -- 172.21.15.50:0/2462802701 learned_addr learned my addr 172.21.15.50:0/2462802701 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.388+0000 7f72b8d71640 1 -- 172.21.15.50:0/2462802701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a0a3eb00 con 0x5603a0a3e110 2024-05-08T18:06:03.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.388+0000 7f72b8d71640 1 --2- 172.21.15.50:0/2462802701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0a3e110 0x5603a0a3e530 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f72ac08dfd0 tx=0x7f72ac094540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d052ee4027e704ad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.388+0000 7f718ffff640 1 -- 172.21.15.50:0/2462802701 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72ac08ed50 con 0x5603a0a3e110 2024-05-08T18:06:03.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.388+0000 7f718ffff640 1 -- 172.21.15.50:0/2462802701 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ac0a9070 con 0x5603a0a3e110 2024-05-08T18:06:03.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.388+0000 7f718ffff640 1 -- 172.21.15.50:0/2462802701 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72ac06f5c0 con 0x5603a0a3e110 2024-05-08T18:06:03.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.389+0000 7f72bd12d800 1 -- 172.21.15.50:0/2462802701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0a3e110 msgr2=0x5603a0a3e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.389+0000 7f72bd12d800 1 --2- 172.21.15.50:0/2462802701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0a3e110 0x5603a0a3e530 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f72ac08dfd0 tx=0x7f72ac094540 comp rx=0 tx=0).stop 2024-05-08T18:06:03.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.389+0000 7f72bd12d800 1 -- 172.21.15.50:0/2462802701 shutdown_connections 2024-05-08T18:06:03.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.389+0000 7f72bd12d800 1 --2- 172.21.15.50:0/2462802701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0a3e110 0x5603a0a3e530 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.389+0000 7f72bd12d800 1 -- 172.21.15.50:0/2462802701 >> 172.21.15.50:0/2462802701 conn(0x5603a0a3a420 msgr2=0x5603a0a38480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.389+0000 7f72bd12d800 1 -- 172.21.15.50:0/2462802701 shutdown_connections 2024-05-08T18:06:03.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.389+0000 7f72bd12d800 1 -- 172.21.15.50:0/2462802701 wait complete. 2024-05-08T18:06:03.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.389+0000 7f72bd12d800 1 Processor -- start 2024-05-08T18:06:03.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.389+0000 7f72bd12d800 1 -- start start 2024-05-08T18:06:03.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.389+0000 7f72bd12d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0a3e110 0x5603a0a45520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.389+0000 7f72bd12d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5603a0a3f190 con 0x5603a0a3e110 2024-05-08T18:06:03.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.389+0000 7f72b8d71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0a3e110 0x5603a0a45520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.389+0000 7f72b8d71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0a3e110 0x5603a0a45520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48840/0 (socket says 172.21.15.50:48840) 2024-05-08T18:06:03.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.389+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 learned_addr learned my addr 172.21.15.50:0/2416106182 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.390+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603a0a45a60 con 0x5603a0a3e110 2024-05-08T18:06:03.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.390+0000 7f72b8d71640 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0a3e110 0x5603a0a45520 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0a6040 tx=0x7f72ac063f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.390+0000 7f718e7fc640 1 -- 172.21.15.50:0/2416106182 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72ac07ee30 con 0x5603a0a3e110 2024-05-08T18:06:03.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.390+0000 7f718e7fc640 1 -- 172.21.15.50:0/2416106182 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ac06f5c0 con 0x5603a0a3e110 2024-05-08T18:06:03.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.390+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5603a0a45a60 con 0x5603a0a3e110 2024-05-08T18:06:03.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.390+0000 7f718e7fc640 1 -- 172.21.15.50:0/2416106182 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72ac08b360 con 0x5603a0a3e110 2024-05-08T18:06:03.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.390+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603a0a45a60 con 0x5603a0a3e110 2024-05-08T18:06:03.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.391+0000 7f718e7fc640 1 -- 172.21.15.50:0/2416106182 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f72ac08b500 con 0x5603a0a3e110 2024-05-08T18:06:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.391+0000 7f72bd12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.392+0000 7f718e7fc640 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f718403bb50 0x7f718403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.392+0000 7f718e7fc640 1 -- 172.21.15.50:0/2416106182 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f72ac06a1b0 con 0x5603a0a3e110 2024-05-08T18:06:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.392+0000 7f72b3fff640 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f718403bb50 0x7f718403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.392+0000 7f72bd12d800 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5603a0a50ba0 0x5603a0a53060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.392+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a537b0 con 0x5603a0a50ba0 2024-05-08T18:06:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.392+0000 7f72b9572640 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5603a0a50ba0 0x5603a0a53060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.392+0000 7f72b3fff640 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f718403bb50 0x7f718403e010 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x5603a0a20db0 tx=0x7f72a4023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.392+0000 7f72b9572640 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5603a0a50ba0 0x5603a0a53060 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f72b400a4a0 con 0x5603a0a50ba0 2024-05-08T18:06:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72bd12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72bd12d800 20 realm 2024-05-08T18:06:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72bd12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a53bc0 con 0x5603a0a50ba0 2024-05-08T18:06:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f72b400a4a0 con 0x5603a0a50ba0 2024-05-08T18:06:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72bd12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72bd12d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72bd12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54460 con 0x5603a0a50ba0 2024-05-08T18:06:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f72b400a4a0 con 0x5603a0a50ba0 2024-05-08T18:06:03.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72bd12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72bd12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a50ba0 2024-05-08T18:06:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f72b400a4a0 con 0x5603a0a50ba0 2024-05-08T18:06:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72bd12d800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72bd12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72bd12d800 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5603a0a55560 0x5603a0a57950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.393+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a57ed0 con 0x5603a0a55560 2024-05-08T18:06:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.394+0000 7f72b8d71640 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5603a0a55560 0x5603a0a57950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.394+0000 7f72b8d71640 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5603a0a55560 0x5603a0a57950 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.394+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.394+0000 7f72bd12d800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.394+0000 7f72bd12d800 20 searching for the correct realm 2024-05-08T18:06:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.394+0000 7f72bd12d800 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5603a0a59f20 0x5603a0a5c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.394+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a5cab0 con 0x5603a0a59f20 2024-05-08T18:06:03.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.394+0000 7f72b3fff640 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5603a0a59f20 0x5603a0a5c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.395+0000 7f72b3fff640 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5603a0a59f20 0x5603a0a5c3e0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.395+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72a4037040 con 0x5603a0a59f20 2024-05-08T18:06:03.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.395+0000 7f72b3fff640 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f72a402ea80 0x7f72a403a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.395+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a5cab0 con 0x7f72a402ea80 2024-05-08T18:06:03.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.395+0000 7f72b9572640 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f72a402ea80 0x7f72a403a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.395+0000 7f72b9572640 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f72a402ea80 0x7f72a403a3c0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.396+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72b400a4a0 con 0x7f72a402ea80 2024-05-08T18:06:03.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.396+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a55560 2024-05-08T18:06:03.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.396+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.396+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a50ba0 2024-05-08T18:06:03.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.396+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72b400a4a0 con 0x5603a0a50ba0 2024-05-08T18:06:03.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.396+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a50ba0 2024-05-08T18:06:03.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.396+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72b400a4a0 con 0x5603a0a50ba0 2024-05-08T18:06:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.396+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a50ba0 2024-05-08T18:06:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.397+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72b400a4a0 con 0x5603a0a50ba0 2024-05-08T18:06:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.397+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a55560 2024-05-08T18:06:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.397+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.397+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x7f72a402ea80 2024-05-08T18:06:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.397+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72b400a4a0 con 0x7f72a402ea80 2024-05-08T18:06:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.397+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a59f20 2024-05-08T18:06:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.397+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72a4037040 con 0x5603a0a59f20 2024-05-08T18:06:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.397+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a50ba0 2024-05-08T18:06:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.398+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72b400a4a0 con 0x5603a0a50ba0 2024-05-08T18:06:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.398+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a55560 2024-05-08T18:06:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.398+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.398+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a50ba0 2024-05-08T18:06:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.398+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72b400a4a0 con 0x5603a0a50ba0 2024-05-08T18:06:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.398+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x7f72a402ea80 2024-05-08T18:06:03.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.398+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f72b4032040 con 0x7f72a402ea80 2024-05-08T18:06:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.398+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a59f20 2024-05-08T18:06:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.399+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72a4037040 con 0x5603a0a59f20 2024-05-08T18:06:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.399+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a59f20 2024-05-08T18:06:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.399+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72a4037040 con 0x5603a0a59f20 2024-05-08T18:06:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.399+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a55560 2024-05-08T18:06:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.399+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.399+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a59f20 2024-05-08T18:06:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.399+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72a4037040 con 0x5603a0a59f20 2024-05-08T18:06:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.399+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a59f20 2024-05-08T18:06:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.400+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f72a4037040 con 0x5603a0a59f20 2024-05-08T18:06:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.400+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a50ba0 2024-05-08T18:06:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.400+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72b4032040 con 0x5603a0a50ba0 2024-05-08T18:06:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.400+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a50ba0 2024-05-08T18:06:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.400+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f72b4032040 con 0x5603a0a50ba0 2024-05-08T18:06:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.400+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a55560 2024-05-08T18:06:03.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.400+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.401+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a59f20 2024-05-08T18:06:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.401+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72a4037040 con 0x5603a0a59f20 2024-05-08T18:06:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.401+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x7f72a402ea80 2024-05-08T18:06:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.401+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72b4032040 con 0x7f72a402ea80 2024-05-08T18:06:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.401+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a50ba0 2024-05-08T18:06:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.401+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72b4032040 con 0x5603a0a50ba0 2024-05-08T18:06:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.401+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a55560 2024-05-08T18:06:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.402+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.402+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a59f20 2024-05-08T18:06:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.402+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72a4037040 con 0x5603a0a59f20 2024-05-08T18:06:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.402+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a55560 2024-05-08T18:06:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.402+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.402+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x7f72a402ea80 2024-05-08T18:06:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.402+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72b4032040 con 0x7f72a402ea80 2024-05-08T18:06:03.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.402+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a59f20 2024-05-08T18:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.402+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72a4037040 con 0x5603a0a59f20 2024-05-08T18:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.403+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a55560 2024-05-08T18:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.403+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.403+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x5603a0a50ba0 2024-05-08T18:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.403+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f72b4032040 con 0x5603a0a50ba0 2024-05-08T18:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.403+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a54800 con 0x7f72a402ea80 2024-05-08T18:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.403+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f72b4032040 con 0x7f72a402ea80 2024-05-08T18:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.403+0000 7f72bd12d800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.403+0000 7f72bd12d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.403+0000 7f72bd12d800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.403+0000 7f72bd12d800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.403+0000 7f72bd12d800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.403+0000 7f72bd12d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:03.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.403+0000 7f72bd12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.403+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a58820 con 0x5603a0a50ba0 2024-05-08T18:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f72b4032040 con 0x5603a0a50ba0 2024-05-08T18:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a58820 con 0x5603a0a50ba0 2024-05-08T18:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f72b4032040 con 0x5603a0a50ba0 2024-05-08T18:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a587f0 con 0x5603a0a59f20 2024-05-08T18:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f72a4037040 con 0x5603a0a59f20 2024-05-08T18:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 20 zone default found 2024-05-08T18:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 4 Realm: () 2024-05-08T18:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:03.411 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 20 zonegroup default 2024-05-08T18:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a590c0 con 0x5603a0a55560 2024-05-08T18:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.404+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a5e910 con 0x5603a0a59f20 2024-05-08T18:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.405+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f72a4037040 con 0x5603a0a59f20 2024-05-08T18:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.405+0000 7f72bd12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.405+0000 7f72bd12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.405+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a5ecb0 con 0x5603a0a59f20 2024-05-08T18:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.405+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f72a4037040 con 0x5603a0a59f20 2024-05-08T18:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.405+0000 7f72bd12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.405+0000 7f72bd12d800 20 started sync module instance, tier type = 2024-05-08T18:06:03.412 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.405+0000 7f72bd12d800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:03.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.405+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a5ddb0 con 0x7f72a402ea80 2024-05-08T18:06:03.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.406+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72b4032040 con 0x7f72a402ea80 2024-05-08T18:06:03.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.406+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94573580116704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a5ddb0 con 0x7f72a402ea80 2024-05-08T18:06:03.417 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.406+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a5f6f0 con 0x5603a0a50ba0 2024-05-08T18:06:03.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.407+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94573580116704] v29'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72b4032040 con 0x7f72a402ea80 2024-05-08T18:06:03.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.407+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72b4032040 con 0x5603a0a50ba0 2024-05-08T18:06:03.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.407+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94573580120512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a5ddb0 con 0x5603a0a50ba0 2024-05-08T18:06:03.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.407+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a60190 con 0x5603a0a59f20 2024-05-08T18:06:03.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.408+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72a4037040 con 0x5603a0a59f20 2024-05-08T18:06:03.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.408+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94573580123264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a60a30 con 0x5603a0a59f20 2024-05-08T18:06:03.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.408+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a61520 con 0x7f72a402ea80 2024-05-08T18:06:03.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.409+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94573580120512] v29'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72b4032040 con 0x5603a0a50ba0 2024-05-08T18:06:03.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.409+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94573580123264] v29'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72a4037040 con 0x5603a0a59f20 2024-05-08T18:06:03.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.410+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72b4032040 con 0x7f72a402ea80 2024-05-08T18:06:03.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.410+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94573580125568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a5d900 con 0x7f72a402ea80 2024-05-08T18:06:03.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.410+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a61dc0 con 0x5603a0a50ba0 2024-05-08T18:06:03.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.411+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72b4032040 con 0x5603a0a50ba0 2024-05-08T18:06:03.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.411+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94573580130448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a621a0 con 0x5603a0a50ba0 2024-05-08T18:06:03.418 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.411+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a62ca0 con 0x5603a0a55560 2024-05-08T18:06:03.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.411+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94573580125568] v29'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72b4032040 con 0x7f72a402ea80 2024-05-08T18:06:03.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.412+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.412+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94573580134288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a63090 con 0x5603a0a55560 2024-05-08T18:06:03.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.412+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94573580130448] v29'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72b4032040 con 0x5603a0a50ba0 2024-05-08T18:06:03.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.412+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a63a90 con 0x5603a0a55560 2024-05-08T18:06:03.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.413+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94573580134288] v29'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.413+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.413+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94573580128240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a63600 con 0x5603a0a55560 2024-05-08T18:06:03.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.413+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a64310 con 0x5603a0a55560 2024-05-08T18:06:03.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.414+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94573580128240] v29'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.414+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.414+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94573580140000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a642d0 con 0x5603a0a55560 2024-05-08T18:06:03.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.414+0000 7f72bd12d800 20 add_watcher() i=0 2024-05-08T18:06:03.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.414+0000 7f72bd12d800 20 add_watcher() i=1 2024-05-08T18:06:03.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.414+0000 7f72bd12d800 20 add_watcher() i=2 2024-05-08T18:06:03.419 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.414+0000 7f72bd12d800 20 add_watcher() i=3 2024-05-08T18:06:03.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.414+0000 7f72bd12d800 20 add_watcher() i=4 2024-05-08T18:06:03.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.414+0000 7f72bd12d800 20 add_watcher() i=5 2024-05-08T18:06:03.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.414+0000 7f72bd12d800 20 add_watcher() i=6 2024-05-08T18:06:03.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.415+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94573580140000] v29'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.415+0000 7f72bd12d800 20 add_watcher() i=7 2024-05-08T18:06:03.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.415+0000 7f72bd12d800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:03.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.415+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a67760 con 0x5603a0a50ba0 2024-05-08T18:06:03.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.415+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f72b4032040 con 0x5603a0a50ba0 2024-05-08T18:06:03.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.415+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94573580131504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a64c80 con 0x5603a0a50ba0 2024-05-08T18:06:03.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.416+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94573580131504] v29'204 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f72b4032040 con 0x5603a0a50ba0 2024-05-08T18:06:03.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.416+0000 7f716efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:03.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.416+0000 7f72bd12d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:03.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.416+0000 7f72bd12d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:03.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.416+0000 7f72bd12d800 5 note: GC not initialized 2024-05-08T18:06:03.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.417+0000 7f716dfe3640 20 reqs_thread_entry: start 2024-05-08T18:06:03.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.417+0000 7f72bd12d800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:03.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.417+0000 7f72bd12d800 20 Filter name: none 2024-05-08T18:06:03.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.417+0000 7f70df7fe640 20 reqs_thread_entry: start 2024-05-08T18:06:03.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.417+0000 7f72bd12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.417+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a9a830 con 0x5603a0a50ba0 2024-05-08T18:06:03.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.417+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f72b4032040 con 0x5603a0a50ba0 2024-05-08T18:06:03.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.417+0000 7f72bd12d800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:06:03.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.417+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a9bd80 con 0x5603a0a55560 2024-05-08T18:06:03.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.418+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(63 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.419+0000 7f716efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:03.421 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.419+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94573580131504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a66f60 con 0x5603a0a50ba0 2024-05-08T18:06:03.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.420+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94573580131504] v29'205 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f72b4032040 con 0x5603a0a50ba0 2024-05-08T18:06:03.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.420+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94573580116704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a66ab0 con 0x7f72a402ea80 2024-05-08T18:06:03.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.421+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94573580116704] v29'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72b4032040 con 0x7f72a402ea80 2024-05-08T18:06:03.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.421+0000 7f72bd12d800 20 remove_watcher() i=0 2024-05-08T18:06:03.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.421+0000 7f72bd12d800 2 removed watcher, disabling cache 2024-05-08T18:06:03.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.421+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94573580120512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a5e2e0 con 0x5603a0a50ba0 2024-05-08T18:06:03.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.422+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94573580120512] v29'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72b4032040 con 0x5603a0a50ba0 2024-05-08T18:06:03.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.422+0000 7f72bd12d800 20 remove_watcher() i=1 2024-05-08T18:06:03.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.422+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94573580123264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a5e2e0 con 0x5603a0a59f20 2024-05-08T18:06:03.423 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.423+0000 7f72b3fff640 1 -- 172.21.15.50:0/2416106182 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94573580123264] v29'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72a4037040 con 0x5603a0a59f20 2024-05-08T18:06:03.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.423+0000 7f72bd12d800 20 remove_watcher() i=2 2024-05-08T18:06:03.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.423+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94573580125568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a5fc80 con 0x7f72a402ea80 2024-05-08T18:06:03.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.424+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94573580125568] v29'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72b4032040 con 0x7f72a402ea80 2024-05-08T18:06:03.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.424+0000 7f72bd12d800 20 remove_watcher() i=3 2024-05-08T18:06:03.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.424+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94573580130448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a60580 con 0x5603a0a50ba0 2024-05-08T18:06:03.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.426+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94573580130448] v29'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72b4032040 con 0x5603a0a50ba0 2024-05-08T18:06:03.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.426+0000 7f72bd12d800 20 remove_watcher() i=4 2024-05-08T18:06:03.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.426+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94573580134288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a5fc80 con 0x5603a0a55560 2024-05-08T18:06:03.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.427+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94573580134288] v29'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.427+0000 7f72bd12d800 20 remove_watcher() i=5 2024-05-08T18:06:03.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.427+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94573580128240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a60580 con 0x5603a0a55560 2024-05-08T18:06:03.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.428+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94573580128240] v29'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.428+0000 7f72bd12d800 20 remove_watcher() i=6 2024-05-08T18:06:03.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.428+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94573580140000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5603a0a5fc80 con 0x5603a0a55560 2024-05-08T18:06:03.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72b8d71640 1 -- 172.21.15.50:0/2416106182 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94573580140000] v29'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f72ac064070 con 0x5603a0a55560 2024-05-08T18:06:03.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 20 remove_watcher() i=7 2024-05-08T18:06:03.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5603a0a55560 msgr2=0x5603a0a57950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5603a0a55560 0x5603a0a57950 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5603a0a50ba0 msgr2=0x5603a0a53060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5603a0a50ba0 0x5603a0a53060 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f72a402ea80 msgr2=0x7f72a403a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f72a402ea80 0x7f72a403a3c0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5603a0a59f20 msgr2=0x5603a0a5c3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5603a0a59f20 0x5603a0a5c3e0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f718403bb50 msgr2=0x7f718403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f718403bb50 0x7f718403e010 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x5603a0a20db0 tx=0x7f72a4023040 comp rx=0 tx=0).stop 2024-05-08T18:06:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0a3e110 msgr2=0x5603a0a45520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 --2- 172.21.15.50:0/2416106182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0a3e110 0x5603a0a45520 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f72ac0a6040 tx=0x7f72ac063f70 comp rx=0 tx=0).stop 2024-05-08T18:06:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 reap_dead start 2024-05-08T18:06:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72b9572640 1 -- 172.21.15.50:0/2416106182 reap_dead start 2024-05-08T18:06:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 shutdown_connections 2024-05-08T18:06:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 >> 172.21.15.50:0/2416106182 conn(0x5603a0a3a420 msgr2=0x5603a0a37a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 shutdown_connections 2024-05-08T18:06:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 -- 172.21.15.50:0/2416106182 wait complete. 2024-05-08T18:06:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 -- 172.21.15.50:0/1445629105 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f719003bb00 msgr2=0x7f719003dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 --2- 172.21.15.50:0/1445629105 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f719003bb00 0x7f719003dfc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5603a0a18530 tx=0x7f72a4006000 comp rx=0 tx=0).stop 2024-05-08T18:06:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 -- 172.21.15.50:0/1445629105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a09184f0 msgr2=0x5603a0916960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 --2- 172.21.15.50:0/1445629105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a09184f0 0x5603a0916960 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f72ac094010 tx=0x7f72ac05d710 comp rx=0 tx=0).stop 2024-05-08T18:06:03.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 -- 172.21.15.50:0/1445629105 shutdown_connections 2024-05-08T18:06:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 --2- 172.21.15.50:0/1445629105 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f719003bb00 0x7f719003dfc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 --2- 172.21.15.50:0/1445629105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a09184f0 0x5603a0916960 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.429+0000 7f72bd12d800 1 -- 172.21.15.50:0/1445629105 >> 172.21.15.50:0/1445629105 conn(0x5603a0a22dc0 msgr2=0x5603a0a231c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72bd12d800 1 -- 172.21.15.50:0/1445629105 shutdown_connections 2024-05-08T18:06:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72bd12d800 1 -- 172.21.15.50:0/1445629105 wait complete. 2024-05-08T18:06:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72bd12d800 1 -- 172.21.15.50:0/3274465427 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5603a0a19f80 msgr2=0x5603a0a1c370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72bd12d800 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5603a0a19f80 0x5603a0a1c370 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72bd12d800 1 -- 172.21.15.50:0/3274465427 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5603a0918950 msgr2=0x5603a0a17720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72bd12d800 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5603a0918950 0x5603a0a17720 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72bd12d800 1 -- 172.21.15.50:0/3274465427 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5603a0a1dec0 msgr2=0x5603a0a202b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72bd12d800 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5603a0a1dec0 0x5603a0a202b0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72bd12d800 1 -- 172.21.15.50:0/3274465427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f71a003bba0 msgr2=0x7f71a003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72bd12d800 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f71a003bba0 0x7f71a003e060 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f72a4000c00 tx=0x7f72a40013b0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72bd12d800 1 -- 172.21.15.50:0/3274465427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 msgr2=0x5603a0a104e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72bd12d800 1 --2- 172.21.15.50:0/3274465427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603a0915960 0x5603a0a104e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f72ac000c00 tx=0x7f72ac002fb0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72b9572640 1 -- 172.21.15.50:0/3274465427 reap_dead start 2024-05-08T18:06:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72bd12d800 1 -- 172.21.15.50:0/3274465427 shutdown_connections 2024-05-08T18:06:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72bd12d800 1 -- 172.21.15.50:0/3274465427 >> 172.21.15.50:0/3274465427 conn(0x5603a09112f0 msgr2=0x5603a0a02b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72bd12d800 1 -- 172.21.15.50:0/3274465427 shutdown_connections 2024-05-08T18:06:03.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.430+0000 7f72bd12d800 1 -- 172.21.15.50:0/3274465427 wait complete. 2024-05-08T18:06:03.438 INFO:teuthology.orchestra.run.smithi050.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"s3select":{}},{"bucket":"myfoo2","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo3","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo4","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[],"s3select":{}}]}],"summary":[{"user":"foo","categories":[{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"total":{"bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11,"bytes_processed":0,"bytes_returned":0}}]} 2024-05-08T18:06:03.438 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 's3select': {}}, {'bucket': 'myfoo2', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo3', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo4', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-05-08T18:06:03.438 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2024-05-08T18:06:03.438 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2024-05-08T18:06:03.438 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_obj 2024-05-08T18:06:03.486 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:03.486 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:03.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.489+0000 7fbe89208800 1 Processor -- start 2024-05-08T18:06:03.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.489+0000 7fbe89208800 1 -- start start 2024-05-08T18:06:03.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.489+0000 7fbe89208800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ebbe88dfb0 0x55ebbe88e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.489+0000 7fbe89208800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55ebbe8028c0 con 0x55ebbe88e910 2024-05-08T18:06:03.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.489+0000 7fbe89208800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55ebbe72fe20 con 0x55ebbe88dfb0 2024-05-08T18:06:03.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.489+0000 7fbe86d00640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55ebbe88dfb0 0x55ebbe88e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.489+0000 7fbe7ffff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55ebbe88e910 0x55ebbe88edc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:54562/0 (socket says 172.21.15.50:54562) 2024-05-08T18:06:03.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.490+0000 7fbe7ffff640 1 -- 172.21.15.50:0/346902808 learned_addr learned my addr 172.21.15.50:0/346902808 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:03.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.490+0000 7fbe86d00640 1 -- 172.21.15.50:0/346902808 >> v1:172.21.15.46:6789/0 conn(0x55ebbe88e910 legacy=0x55ebbe88edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.490+0000 7fbe864ff640 1 -- 172.21.15.50:0/346902808 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1117887772 0 0) 0x55ebbe8028c0 con 0x55ebbe88e910 2024-05-08T18:06:03.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.490+0000 7fbe86d00640 1 -- 172.21.15.50:0/346902808 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbe88f4d0 con 0x55ebbe88dfb0 2024-05-08T18:06:03.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.492+0000 7fbe86d00640 1 --2- 172.21.15.50:0/346902808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 0x55ebbe88e3d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800a470 tx=0x7fbe7801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef61d63842ea9d84 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.492+0000 7fbe864ff640 1 -- 172.21.15.50:0/346902808 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe78057070 con 0x55ebbe88dfb0 2024-05-08T18:06:03.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.492+0000 7fbe864ff640 1 -- 172.21.15.50:0/346902808 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe78052570 con 0x55ebbe88dfb0 2024-05-08T18:06:03.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.492+0000 7fbe864ff640 1 -- 172.21.15.50:0/346902808 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe78052870 con 0x55ebbe88dfb0 2024-05-08T18:06:03.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.492+0000 7fbe89208800 1 -- 172.21.15.50:0/346902808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 msgr2=0x55ebbe88e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.492+0000 7fbe89208800 1 --2- 172.21.15.50:0/346902808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 0x55ebbe88e3d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fbe7800a470 tx=0x7fbe7801ce90 comp rx=0 tx=0).stop 2024-05-08T18:06:03.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.492+0000 7fbe89208800 1 -- 172.21.15.50:0/346902808 shutdown_connections 2024-05-08T18:06:03.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.492+0000 7fbe89208800 1 --2- 172.21.15.50:0/346902808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 0x55ebbe88e3d0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.492+0000 7fbe89208800 1 -- 172.21.15.50:0/346902808 >> 172.21.15.50:0/346902808 conn(0x55ebbe889b70 msgr2=0x55ebbe88bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.492+0000 7fbe89208800 1 -- 172.21.15.50:0/346902808 shutdown_connections 2024-05-08T18:06:03.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.492+0000 7fbe89208800 1 -- 172.21.15.50:0/346902808 wait complete. 2024-05-08T18:06:03.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.507+0000 7fbe89208800 1 Processor -- start 2024-05-08T18:06:03.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.508+0000 7fbe89208800 1 -- start start 2024-05-08T18:06:03.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.508+0000 7fbe89208800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 0x55ebbe982d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.508+0000 7fbe89208800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ebbe7daa30 con 0x55ebbe88dfb0 2024-05-08T18:06:03.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.508+0000 7fbe86d00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 0x55ebbe982d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.508+0000 7fbe86d00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 0x55ebbe982d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48864/0 (socket says 172.21.15.50:48864) 2024-05-08T18:06:03.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.508+0000 7fbe86d00640 1 -- 172.21.15.50:0/400146143 learned_addr learned my addr 172.21.15.50:0/400146143 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.508+0000 7fbe86d00640 1 -- 172.21.15.50:0/400146143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbe983350 con 0x55ebbe88dfb0 2024-05-08T18:06:03.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.509+0000 7fbe86d00640 1 --2- 172.21.15.50:0/400146143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 0x55ebbe982d80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fbe78055040 tx=0x7fbe7804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf92d329a282abe7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.509+0000 7fbd74e09640 1 -- 172.21.15.50:0/400146143 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe78063070 con 0x55ebbe88dfb0 2024-05-08T18:06:03.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.509+0000 7fbd74e09640 1 -- 172.21.15.50:0/400146143 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe78058040 con 0x55ebbe88dfb0 2024-05-08T18:06:03.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.509+0000 7fbd74e09640 1 -- 172.21.15.50:0/400146143 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe78057070 con 0x55ebbe88dfb0 2024-05-08T18:06:03.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.509+0000 7fbe89208800 1 -- 172.21.15.50:0/400146143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 msgr2=0x55ebbe982d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.509+0000 7fbe89208800 1 --2- 172.21.15.50:0/400146143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 0x55ebbe982d80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fbe78055040 tx=0x7fbe7804c000 comp rx=0 tx=0).stop 2024-05-08T18:06:03.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.510+0000 7fbe89208800 1 -- 172.21.15.50:0/400146143 shutdown_connections 2024-05-08T18:06:03.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.510+0000 7fbe89208800 1 --2- 172.21.15.50:0/400146143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 0x55ebbe982d80 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.510+0000 7fbe89208800 1 -- 172.21.15.50:0/400146143 >> 172.21.15.50:0/400146143 conn(0x55ebbe889b70 msgr2=0x55ebbe97e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.510+0000 7fbe89208800 1 -- 172.21.15.50:0/400146143 shutdown_connections 2024-05-08T18:06:03.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.510+0000 7fbe89208800 1 -- 172.21.15.50:0/400146143 wait complete. 2024-05-08T18:06:03.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.510+0000 7fbe89208800 1 Processor -- start 2024-05-08T18:06:03.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.510+0000 7fbe89208800 1 -- start start 2024-05-08T18:06:03.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.510+0000 7fbe89208800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 0x55ebbe98a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.510+0000 7fbe89208800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ebbe88fa60 con 0x55ebbe88dfb0 2024-05-08T18:06:03.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.510+0000 7fbe86d00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 0x55ebbe98a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.510+0000 7fbe86d00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 0x55ebbe98a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48880/0 (socket says 172.21.15.50:48880) 2024-05-08T18:06:03.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.510+0000 7fbe86d00640 1 -- 172.21.15.50:0/3711153108 learned_addr learned my addr 172.21.15.50:0/3711153108 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.511+0000 7fbe86d00640 1 -- 172.21.15.50:0/3711153108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbe98add0 con 0x55ebbe88dfb0 2024-05-08T18:06:03.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.511+0000 7fbe86d00640 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 0x55ebbe98a890 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbe78061040 tx=0x7fbe78002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.511+0000 7fbd73606640 1 -- 172.21.15.50:0/3711153108 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe78063030 con 0x55ebbe88dfb0 2024-05-08T18:06:03.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.511+0000 7fbd73606640 1 -- 172.21.15.50:0/3711153108 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe78052d90 con 0x55ebbe88dfb0 2024-05-08T18:06:03.510 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.511+0000 7fbe89208800 1 -- 172.21.15.50:0/3711153108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ebbe98b060 con 0x55ebbe88dfb0 2024-05-08T18:06:03.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.511+0000 7fbd73606640 1 -- 172.21.15.50:0/3711153108 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe7805e5b0 con 0x55ebbe88dfb0 2024-05-08T18:06:03.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.511+0000 7fbe89208800 1 -- 172.21.15.50:0/3711153108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebbe98dd10 con 0x55ebbe88dfb0 2024-05-08T18:06:03.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.512+0000 7fbd73606640 1 -- 172.21.15.50:0/3711153108 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fbe7805e750 con 0x55ebbe88dfb0 2024-05-08T18:06:03.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.512+0000 7fbd73606640 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbd6c03bba0 0x7fbd6c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.512+0000 7fbd73606640 1 -- 172.21.15.50:0/3711153108 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fbe78064070 con 0x55ebbe88dfb0 2024-05-08T18:06:03.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.512+0000 7fbe7ffff640 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbd6c03bba0 0x7fbd6c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.512+0000 7fbe89208800 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ebbe88e910 0x55ebbe991ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.512+0000 7fbe89208800 1 -- 172.21.15.50:0/3711153108 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe992010 con 0x55ebbe88e910 2024-05-08T18:06:03.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.512+0000 7fbe87501640 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ebbe88e910 0x55ebbe991ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.513+0000 7fbe7ffff640 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbd6c03bba0 0x7fbd6c03e060 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbe70000c00 tx=0x7fbe700013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.513+0000 7fbe87501640 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ebbe88e910 0x55ebbe991ad0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.513+0000 7fbe87501640 1 -- 172.21.15.50:0/3711153108 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbe8000a4a0 con 0x55ebbe88e910 2024-05-08T18:06:03.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.513+0000 7fbe89208800 1 -- 172.21.15.50:0/3711153108 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe98f080 con 0x55ebbe88e910 2024-05-08T18:06:03.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.513+0000 7fbe87501640 1 -- 172.21.15.50:0/3711153108 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbe8000a4a0 con 0x55ebbe88e910 2024-05-08T18:06:03.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.513+0000 7fbe89208800 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ebbe994350 0x55ebbe996740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.513+0000 7fbe89208800 1 -- 172.21.15.50:0/3711153108 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe996de0 con 0x55ebbe994350 2024-05-08T18:06:03.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.514+0000 7fbe86d00640 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ebbe994350 0x55ebbe996740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.514+0000 7fbe86d00640 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ebbe994350 0x55ebbe996740 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.514+0000 7fbe86d00640 1 -- 172.21.15.50:0/3711153108 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fbe78070020 con 0x55ebbe994350 2024-05-08T18:06:03.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.514+0000 7fbe89208800 1 -- 172.21.15.50:0/3711153108 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9931f0 con 0x55ebbe88e910 2024-05-08T18:06:03.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.515+0000 7fbe87501640 1 -- 172.21.15.50:0/3711153108 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbe8000a4a0 con 0x55ebbe88e910 2024-05-08T18:06:03.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.515+0000 7fbe89208800 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ebbe998240 0x55ebbe99a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.515+0000 7fbe89208800 1 -- 172.21.15.50:0/3711153108 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe99ad70 con 0x55ebbe998240 2024-05-08T18:06:03.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.515+0000 7fbe7ffff640 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ebbe998240 0x55ebbe99a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.515+0000 7fbe7ffff640 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ebbe998240 0x55ebbe99a630 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.515+0000 7fbe7ffff640 1 -- 172.21.15.50:0/3711153108 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fbe7000f670 con 0x55ebbe998240 2024-05-08T18:06:03.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.516+0000 7fbe89208800 1 Processor -- start 2024-05-08T18:06:03.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.516+0000 7fbe89208800 1 -- start start 2024-05-08T18:06:03.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.516+0000 7fbe89208800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9a9070 0x55ebbe9a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.516+0000 7fbe89208800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ebbe781330 con 0x55ebbe9a9070 2024-05-08T18:06:03.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.516+0000 7fbe86d00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9a9070 0x55ebbe9a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.516+0000 7fbe86d00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9a9070 0x55ebbe9a9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48892/0 (socket says 172.21.15.50:48892) 2024-05-08T18:06:03.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.516+0000 7fbe86d00640 1 -- 172.21.15.50:0/1757017601 learned_addr learned my addr 172.21.15.50:0/1757017601 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.516+0000 7fbe86d00640 1 -- 172.21.15.50:0/1757017601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbe9a99d0 con 0x55ebbe9a9070 2024-05-08T18:06:03.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.517+0000 7fbe86d00640 1 --2- 172.21.15.50:0/1757017601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9a9070 0x55ebbe9a9490 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbe78093120 tx=0x7fbe78069390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.517+0000 7fbd70e01640 1 -- 172.21.15.50:0/1757017601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe7805c910 con 0x55ebbe9a9070 2024-05-08T18:06:03.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.517+0000 7fbd70e01640 1 -- 172.21.15.50:0/1757017601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe7806a950 con 0x55ebbe9a9070 2024-05-08T18:06:03.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.517+0000 7fbe89208800 1 -- 172.21.15.50:0/1757017601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ebbe9aa440 con 0x55ebbe9a9070 2024-05-08T18:06:03.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.517+0000 7fbd70e01640 1 -- 172.21.15.50:0/1757017601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe7806ac30 con 0x55ebbe9a9070 2024-05-08T18:06:03.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.517+0000 7fbe89208800 1 -- 172.21.15.50:0/1757017601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebbe9aa440 con 0x55ebbe9a9070 2024-05-08T18:06:03.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.518+0000 7fbd70e01640 1 -- 172.21.15.50:0/1757017601 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fbe7808bf90 con 0x55ebbe9a9070 2024-05-08T18:06:03.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.518+0000 7fbd70e01640 1 --2- 172.21.15.50:0/1757017601 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbd6003bab0 0x7fbd6003df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.518+0000 7fbd70e01640 1 -- 172.21.15.50:0/1757017601 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fbe7809ed10 con 0x55ebbe9a9070 2024-05-08T18:06:03.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.518+0000 7fbe7ffff640 1 --2- 172.21.15.50:0/1757017601 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbd6003bab0 0x7fbd6003df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.518+0000 7fbe89208800 1 Processor -- start 2024-05-08T18:06:03.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.518+0000 7fbe89208800 1 -- start start 2024-05-08T18:06:03.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.518+0000 7fbe7ffff640 1 --2- 172.21.15.50:0/1757017601 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbd6003bab0 0x7fbd6003df70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55ebbe98efc0 tx=0x7fbe70006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.518+0000 7fbe89208800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9b60d0 0x55ebbe9b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.518+0000 7fbe89208800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ebbe983610 con 0x55ebbe9b60d0 2024-05-08T18:06:03.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.519+0000 7fbe86d00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9b60d0 0x55ebbe9b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.519+0000 7fbe86d00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9b60d0 0x55ebbe9b64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48894/0 (socket says 172.21.15.50:48894) 2024-05-08T18:06:03.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.519+0000 7fbe86d00640 1 -- 172.21.15.50:0/2661018659 learned_addr learned my addr 172.21.15.50:0/2661018659 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.519+0000 7fbe86d00640 1 -- 172.21.15.50:0/2661018659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbe9b6ac0 con 0x55ebbe9b60d0 2024-05-08T18:06:03.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.519+0000 7fbe86d00640 1 --2- 172.21.15.50:0/2661018659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9b60d0 0x55ebbe9b64f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbe78005820 tx=0x7fbe78006540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706b211350e645b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.519+0000 7fbd6a7fc640 1 -- 172.21.15.50:0/2661018659 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe7808cda0 con 0x55ebbe9b60d0 2024-05-08T18:06:03.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.519+0000 7fbd6a7fc640 1 -- 172.21.15.50:0/2661018659 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe780695b0 con 0x55ebbe9b60d0 2024-05-08T18:06:03.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.519+0000 7fbd6a7fc640 1 -- 172.21.15.50:0/2661018659 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe780727d0 con 0x55ebbe9b60d0 2024-05-08T18:06:03.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.520+0000 7fbe89208800 1 -- 172.21.15.50:0/2661018659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9b60d0 msgr2=0x55ebbe9b64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.520+0000 7fbe89208800 1 --2- 172.21.15.50:0/2661018659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9b60d0 0x55ebbe9b64f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbe78005820 tx=0x7fbe78006540 comp rx=0 tx=0).stop 2024-05-08T18:06:03.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.520+0000 7fbe89208800 1 -- 172.21.15.50:0/2661018659 shutdown_connections 2024-05-08T18:06:03.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.520+0000 7fbe89208800 1 --2- 172.21.15.50:0/2661018659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9b60d0 0x55ebbe9b64f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.520+0000 7fbe89208800 1 -- 172.21.15.50:0/2661018659 >> 172.21.15.50:0/2661018659 conn(0x55ebbe9b23e0 msgr2=0x55ebbe9b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.520+0000 7fbe89208800 1 -- 172.21.15.50:0/2661018659 shutdown_connections 2024-05-08T18:06:03.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.520+0000 7fbe89208800 1 -- 172.21.15.50:0/2661018659 wait complete. 2024-05-08T18:06:03.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.520+0000 7fbe89208800 1 Processor -- start 2024-05-08T18:06:03.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.520+0000 7fbe89208800 1 -- start start 2024-05-08T18:06:03.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.520+0000 7fbe89208800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9b60d0 0x55ebbe9bd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.525 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.520+0000 7fbe89208800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ebbe9b6fd0 con 0x55ebbe9b60d0 2024-05-08T18:06:03.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.520+0000 7fbe86d00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9b60d0 0x55ebbe9bd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.520+0000 7fbe86d00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9b60d0 0x55ebbe9bd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48902/0 (socket says 172.21.15.50:48902) 2024-05-08T18:06:03.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.520+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 learned_addr learned my addr 172.21.15.50:0/1336198148 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.521+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbe9bd810 con 0x55ebbe9b60d0 2024-05-08T18:06:03.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.521+0000 7fbe86d00640 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9b60d0 0x55ebbe9bd2d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbe78006580 tx=0x7fbe78096ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.521+0000 7fbd68ff9640 1 -- 172.21.15.50:0/1336198148 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe7807f500 con 0x55ebbe9b60d0 2024-05-08T18:06:03.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.521+0000 7fbd68ff9640 1 -- 172.21.15.50:0/1336198148 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe78002030 con 0x55ebbe9b60d0 2024-05-08T18:06:03.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.521+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ebbe9bd810 con 0x55ebbe9b60d0 2024-05-08T18:06:03.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.521+0000 7fbd68ff9640 1 -- 172.21.15.50:0/1336198148 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbe7805d760 con 0x55ebbe9b60d0 2024-05-08T18:06:03.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.521+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebbe9bd810 con 0x55ebbe9b60d0 2024-05-08T18:06:03.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.522+0000 7fbd68ff9640 1 -- 172.21.15.50:0/1336198148 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fbe780a9070 con 0x55ebbe9b60d0 2024-05-08T18:06:03.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.522+0000 7fbd68ff9640 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbd5003bba0 0x7fbd5003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.522+0000 7fbd68ff9640 1 -- 172.21.15.50:0/1336198148 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fbe780b91b0 con 0x55ebbe9b60d0 2024-05-08T18:06:03.526 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.522+0000 7fbe7ffff640 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbd5003bba0 0x7fbd5003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.522+0000 7fbe89208800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.522+0000 7fbe89208800 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ebbe9c87a0 0x55ebbe9cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.522+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cb3b0 con 0x55ebbe9c87a0 2024-05-08T18:06:03.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.523+0000 7fbe87501640 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ebbe9c87a0 0x55ebbe9cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.523+0000 7fbe7ffff640 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbd5003bba0 0x7fbd5003e060 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55ebbe993590 tx=0x7fbe70023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.523+0000 7fbe87501640 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ebbe9c87a0 0x55ebbe9cac60 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.523+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbe8000a4a0 con 0x55ebbe9c87a0 2024-05-08T18:06:03.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.523+0000 7fbe89208800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.523+0000 7fbe89208800 20 realm 2024-05-08T18:06:03.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.523+0000 7fbe89208800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.523+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cb7c0 con 0x55ebbe9c87a0 2024-05-08T18:06:03.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.523+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbe8000a4a0 con 0x55ebbe9c87a0 2024-05-08T18:06:03.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.523+0000 7fbe89208800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.523+0000 7fbe89208800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:03.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.523+0000 7fbe89208800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.523+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc090 con 0x55ebbe9c87a0 2024-05-08T18:06:03.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.524+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbe8000a4a0 con 0x55ebbe9c87a0 2024-05-08T18:06:03.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.524+0000 7fbe89208800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.524+0000 7fbe89208800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.524+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9c87a0 2024-05-08T18:06:03.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.524+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbe8000a4a0 con 0x55ebbe9c87a0 2024-05-08T18:06:03.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.524+0000 7fbe89208800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:03.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.524+0000 7fbe89208800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.524+0000 7fbe89208800 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ebbe9ccf60 0x55ebbe9cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.524+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cfa90 con 0x55ebbe9ccf60 2024-05-08T18:06:03.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.524+0000 7fbe86d00640 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ebbe9ccf60 0x55ebbe9cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.525+0000 7fbe86d00640 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ebbe9ccf60 0x55ebbe9cf350 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.525+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.525+0000 7fbe89208800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:03.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.525+0000 7fbe89208800 20 searching for the correct realm 2024-05-08T18:06:03.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.525+0000 7fbe89208800 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ebbe9d1af0 0x55ebbe9d3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.525+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d4660 con 0x55ebbe9d1af0 2024-05-08T18:06:03.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.525+0000 7fbe7ffff640 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ebbe9d1af0 0x55ebbe9d3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.525+0000 7fbe7ffff640 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ebbe9d1af0 0x55ebbe9d3f90 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.526+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe70037040 con 0x55ebbe9d1af0 2024-05-08T18:06:03.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.526+0000 7fbe7ffff640 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fbe7002ea80 0x7fbe7003a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.526+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d4660 con 0x7fbe7002ea80 2024-05-08T18:06:03.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.526+0000 7fbe87501640 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fbe7002ea80 0x7fbe7003a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.526+0000 7fbe87501640 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fbe7002ea80 0x7fbe7003a3c0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.526+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe8000a4a0 con 0x7fbe7002ea80 2024-05-08T18:06:03.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.526+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9ccf60 2024-05-08T18:06:03.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.527+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.527+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9c87a0 2024-05-08T18:06:03.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.527+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe8000a4a0 con 0x55ebbe9c87a0 2024-05-08T18:06:03.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.527+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9c87a0 2024-05-08T18:06:03.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.527+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe8000a4a0 con 0x55ebbe9c87a0 2024-05-08T18:06:03.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.527+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9c87a0 2024-05-08T18:06:03.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.527+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe8000a4a0 con 0x55ebbe9c87a0 2024-05-08T18:06:03.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.527+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9ccf60 2024-05-08T18:06:03.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.528+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.528+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x7fbe7002ea80 2024-05-08T18:06:03.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.528+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe8000a4a0 con 0x7fbe7002ea80 2024-05-08T18:06:03.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.528+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9d1af0 2024-05-08T18:06:03.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.528+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe70037040 con 0x55ebbe9d1af0 2024-05-08T18:06:03.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.528+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9c87a0 2024-05-08T18:06:03.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.528+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe8000a4a0 con 0x55ebbe9c87a0 2024-05-08T18:06:03.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.528+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9ccf60 2024-05-08T18:06:03.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.529+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.529+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9c87a0 2024-05-08T18:06:03.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.529+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe8000a4a0 con 0x55ebbe9c87a0 2024-05-08T18:06:03.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.529+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x7fbe7002ea80 2024-05-08T18:06:03.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.529+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fbe80032040 con 0x7fbe7002ea80 2024-05-08T18:06:03.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.529+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9d1af0 2024-05-08T18:06:03.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.529+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe70037040 con 0x55ebbe9d1af0 2024-05-08T18:06:03.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.529+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9d1af0 2024-05-08T18:06:03.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.530+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe70037040 con 0x55ebbe9d1af0 2024-05-08T18:06:03.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.530+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9ccf60 2024-05-08T18:06:03.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.530+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.530+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9d1af0 2024-05-08T18:06:03.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.530+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe70037040 con 0x55ebbe9d1af0 2024-05-08T18:06:03.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.530+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9d1af0 2024-05-08T18:06:03.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.530+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fbe70037040 con 0x55ebbe9d1af0 2024-05-08T18:06:03.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.530+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9c87a0 2024-05-08T18:06:03.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.531+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe80032040 con 0x55ebbe9c87a0 2024-05-08T18:06:03.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.531+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9c87a0 2024-05-08T18:06:03.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.531+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fbe80032040 con 0x55ebbe9c87a0 2024-05-08T18:06:03.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.531+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9ccf60 2024-05-08T18:06:03.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.531+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.531+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9d1af0 2024-05-08T18:06:03.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.532+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe70037040 con 0x55ebbe9d1af0 2024-05-08T18:06:03.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.532+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x7fbe7002ea80 2024-05-08T18:06:03.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.532+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe80032040 con 0x7fbe7002ea80 2024-05-08T18:06:03.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.532+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9c87a0 2024-05-08T18:06:03.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.532+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe80032040 con 0x55ebbe9c87a0 2024-05-08T18:06:03.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.532+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9ccf60 2024-05-08T18:06:03.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.532+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.532+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9d1af0 2024-05-08T18:06:03.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.533+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe70037040 con 0x55ebbe9d1af0 2024-05-08T18:06:03.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.533+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9ccf60 2024-05-08T18:06:03.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.533+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.533+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x7fbe7002ea80 2024-05-08T18:06:03.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.533+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe80032040 con 0x7fbe7002ea80 2024-05-08T18:06:03.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.533+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9d1af0 2024-05-08T18:06:03.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.533+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe70037040 con 0x55ebbe9d1af0 2024-05-08T18:06:03.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.533+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9ccf60 2024-05-08T18:06:03.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.534+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.534+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x55ebbe9c87a0 2024-05-08T18:06:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.534+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fbe80032040 con 0x55ebbe9c87a0 2024-05-08T18:06:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.534+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9cc430 con 0x7fbe7002ea80 2024-05-08T18:06:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.534+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbe80032040 con 0x7fbe7002ea80 2024-05-08T18:06:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.534+0000 7fbe89208800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.534+0000 7fbe89208800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.534+0000 7fbe89208800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.534+0000 7fbe89208800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.534+0000 7fbe89208800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.534+0000 7fbe89208800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.534+0000 7fbe89208800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.534+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d0410 con 0x55ebbe9c87a0 2024-05-08T18:06:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.534+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbe80032040 con 0x55ebbe9c87a0 2024-05-08T18:06:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.534+0000 7fbe89208800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d0410 con 0x55ebbe9c87a0 2024-05-08T18:06:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbe80032040 con 0x55ebbe9c87a0 2024-05-08T18:06:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d0410 con 0x55ebbe9d1af0 2024-05-08T18:06:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fbe70037040 con 0x55ebbe9d1af0 2024-05-08T18:06:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 20 zone default found 2024-05-08T18:06:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 4 Realm: () 2024-05-08T18:06:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:03.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 20 zonegroup default 2024-05-08T18:06:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d0cb0 con 0x55ebbe9ccf60 2024-05-08T18:06:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.535+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d64c0 con 0x55ebbe9d1af0 2024-05-08T18:06:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.536+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbe70037040 con 0x55ebbe9d1af0 2024-05-08T18:06:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.536+0000 7fbe89208800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.536+0000 7fbe89208800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.536+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d6860 con 0x55ebbe9d1af0 2024-05-08T18:06:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.536+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbe70037040 con 0x55ebbe9d1af0 2024-05-08T18:06:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.536+0000 7fbe89208800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.536+0000 7fbe89208800 20 started sync module instance, tier type = 2024-05-08T18:06:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.536+0000 7fbe89208800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.536+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d5960 con 0x7fbe7002ea80 2024-05-08T18:06:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.537+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe80032040 con 0x7fbe7002ea80 2024-05-08T18:06:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.537+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94471003659920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d5960 con 0x7fbe7002ea80 2024-05-08T18:06:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.537+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d7310 con 0x55ebbe9c87a0 2024-05-08T18:06:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.538+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94471003659920] v29'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe80032040 con 0x7fbe7002ea80 2024-05-08T18:06:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.539+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe80032040 con 0x55ebbe9c87a0 2024-05-08T18:06:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.539+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94471003663856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d5960 con 0x55ebbe9c87a0 2024-05-08T18:06:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.539+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d7da0 con 0x55ebbe9d1af0 2024-05-08T18:06:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.540+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe70037040 con 0x55ebbe9d1af0 2024-05-08T18:06:03.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.540+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94471003666544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d8640 con 0x55ebbe9d1af0 2024-05-08T18:06:03.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.540+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d9110 con 0x7fbe7002ea80 2024-05-08T18:06:03.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.540+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94471003663856] v29'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe80032040 con 0x55ebbe9c87a0 2024-05-08T18:06:03.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.541+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94471003666544] v29'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe70037040 con 0x55ebbe9d1af0 2024-05-08T18:06:03.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.541+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe80032040 con 0x7fbe7002ea80 2024-05-08T18:06:03.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.541+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94471003668880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d54b0 con 0x7fbe7002ea80 2024-05-08T18:06:03.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.541+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d99b0 con 0x55ebbe9c87a0 2024-05-08T18:06:03.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.542+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe80032040 con 0x55ebbe9c87a0 2024-05-08T18:06:03.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.542+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94471003673776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d98d0 con 0x55ebbe9c87a0 2024-05-08T18:06:03.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.542+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9da8f0 con 0x55ebbe9ccf60 2024-05-08T18:06:03.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.542+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94471003668880] v29'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe80032040 con 0x7fbe7002ea80 2024-05-08T18:06:03.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.543+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.543+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94471003677632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9dace0 con 0x55ebbe9ccf60 2024-05-08T18:06:03.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.543+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9db6e0 con 0x55ebbe9ccf60 2024-05-08T18:06:03.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.543+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94471003673776] v29'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe80032040 con 0x55ebbe9c87a0 2024-05-08T18:06:03.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.544+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94471003677632] v29'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.544+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.544+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94471003676064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9dba80 con 0x55ebbe9ccf60 2024-05-08T18:06:03.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.544+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9dc310 con 0x55ebbe9ccf60 2024-05-08T18:06:03.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.545+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(58 notify.7 [create] v29'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.545+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94471003684384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9dcb00 con 0x55ebbe9ccf60 2024-05-08T18:06:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.545+0000 7fbe89208800 20 add_watcher() i=0 2024-05-08T18:06:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.545+0000 7fbe89208800 20 add_watcher() i=1 2024-05-08T18:06:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.545+0000 7fbe89208800 20 add_watcher() i=2 2024-05-08T18:06:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.545+0000 7fbe89208800 20 add_watcher() i=3 2024-05-08T18:06:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.545+0000 7fbe89208800 20 add_watcher() i=4 2024-05-08T18:06:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.545+0000 7fbe89208800 20 add_watcher() i=5 2024-05-08T18:06:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.545+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94471003676064] v29'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.545+0000 7fbe89208800 20 add_watcher() i=6 2024-05-08T18:06:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.546+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94471003684384] v29'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.546+0000 7fbe89208800 20 add_watcher() i=7 2024-05-08T18:06:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.546+0000 7fbe89208800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.546+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9db1f0 con 0x55ebbe9c87a0 2024-05-08T18:06:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.546+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fbe80032040 con 0x55ebbe9c87a0 2024-05-08T18:06:03.555 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.546+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94471003686288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9db1f0 con 0x55ebbe9c87a0 2024-05-08T18:06:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.547+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94471003686288] v29'206 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbe80032040 con 0x55ebbe9c87a0 2024-05-08T18:06:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.547+0000 7fbd3d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.547+0000 7fbe89208800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.547+0000 7fbe89208800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.547+0000 7fbe89208800 5 note: GC not initialized 2024-05-08T18:06:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.548+0000 7fbcb7fff640 20 reqs_thread_entry: start 2024-05-08T18:06:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.548+0000 7fbe89208800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.548+0000 7fbe89208800 20 Filter name: none 2024-05-08T18:06:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.548+0000 7fbcb5ffb640 20 reqs_thread_entry: start 2024-05-08T18:06:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.548+0000 7fbe89208800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.548+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbea12c40 con 0x55ebbe9c87a0 2024-05-08T18:06:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.548+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fbe80032040 con 0x55ebbe9c87a0 2024-05-08T18:06:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.548+0000 7fbe89208800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:06:03.556 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.548+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55ebbea14190 con 0x55ebbe9ccf60 2024-05-08T18:06:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.549+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(63 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.550+0000 7fbd3d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.550+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94471003686288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9e91b0 con 0x55ebbe9c87a0 2024-05-08T18:06:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.551+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94471003686288] v29'207 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbe80032040 con 0x55ebbe9c87a0 2024-05-08T18:06:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.551+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94471003659920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9dc550 con 0x7fbe7002ea80 2024-05-08T18:06:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.552+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94471003659920] v29'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe80032040 con 0x7fbe7002ea80 2024-05-08T18:06:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.552+0000 7fbe89208800 20 remove_watcher() i=0 2024-05-08T18:06:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.552+0000 7fbe89208800 2 removed watcher, disabling cache 2024-05-08T18:06:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.552+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94471003663856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d5e90 con 0x55ebbe9c87a0 2024-05-08T18:06:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.553+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94471003663856] v29'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe80032040 con 0x55ebbe9c87a0 2024-05-08T18:06:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.553+0000 7fbe89208800 20 remove_watcher() i=1 2024-05-08T18:06:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.553+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94471003666544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d5e90 con 0x55ebbe9d1af0 2024-05-08T18:06:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.554+0000 7fbe7ffff640 1 -- 172.21.15.50:0/1336198148 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94471003666544] v29'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe70037040 con 0x55ebbe9d1af0 2024-05-08T18:06:03.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.554+0000 7fbe89208800 20 remove_watcher() i=2 2024-05-08T18:06:03.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.554+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94471003668880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d7870 con 0x7fbe7002ea80 2024-05-08T18:06:03.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.555+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94471003668880] v29'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe80032040 con 0x7fbe7002ea80 2024-05-08T18:06:03.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.555+0000 7fbe89208800 20 remove_watcher() i=3 2024-05-08T18:06:03.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.555+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94471003673776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d8190 con 0x55ebbe9c87a0 2024-05-08T18:06:03.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.556+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94471003673776] v29'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe80032040 con 0x55ebbe9c87a0 2024-05-08T18:06:03.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.556+0000 7fbe89208800 20 remove_watcher() i=4 2024-05-08T18:06:03.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.556+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94471003677632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d7870 con 0x55ebbe9ccf60 2024-05-08T18:06:03.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.557+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94471003677632] v29'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.557+0000 7fbe89208800 20 remove_watcher() i=5 2024-05-08T18:06:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.557+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94471003676064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d94b0 con 0x55ebbe9ccf60 2024-05-08T18:06:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.558+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94471003676064] v29'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.558+0000 7fbe89208800 20 remove_watcher() i=6 2024-05-08T18:06:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.558+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94471003684384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55ebbe9d8190 con 0x55ebbe9ccf60 2024-05-08T18:06:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.559+0000 7fbe86d00640 1 -- 172.21.15.50:0/1336198148 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94471003684384] v29'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbe780a8030 con 0x55ebbe9ccf60 2024-05-08T18:06:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.559+0000 7fbe89208800 20 remove_watcher() i=7 2024-05-08T18:06:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ebbe9ccf60 msgr2=0x55ebbe9cf350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ebbe9ccf60 0x55ebbe9cf350 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ebbe9c87a0 msgr2=0x55ebbe9cac60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ebbe9c87a0 0x55ebbe9cac60 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fbe7002ea80 msgr2=0x7fbe7003a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fbe7002ea80 0x7fbe7003a3c0 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ebbe9d1af0 msgr2=0x55ebbe9d3f90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ebbe9d1af0 0x55ebbe9d3f90 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbd5003bba0 msgr2=0x7fbd5003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbd5003bba0 0x7fbd5003e060 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55ebbe993590 tx=0x7fbe70023040 comp rx=0 tx=0).stop 2024-05-08T18:06:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9b60d0 msgr2=0x55ebbe9bd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 --2- 172.21.15.50:0/1336198148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9b60d0 0x55ebbe9bd2d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbe78006580 tx=0x7fbe78096ef0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 reap_dead start 2024-05-08T18:06:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe87501640 1 -- 172.21.15.50:0/1336198148 reap_dead start 2024-05-08T18:06:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 shutdown_connections 2024-05-08T18:06:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 >> 172.21.15.50:0/1336198148 conn(0x55ebbe9b23e0 msgr2=0x55ebbe9afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 shutdown_connections 2024-05-08T18:06:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 -- 172.21.15.50:0/1336198148 wait complete. 2024-05-08T18:06:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 -- 172.21.15.50:0/1757017601 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbd6003bab0 msgr2=0x7fbd6003df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 --2- 172.21.15.50:0/1757017601 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbd6003bab0 0x7fbd6003df70 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55ebbe98efc0 tx=0x7fbe70006000 comp rx=0 tx=0).stop 2024-05-08T18:06:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 -- 172.21.15.50:0/1757017601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9a9070 msgr2=0x55ebbe9a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 --2- 172.21.15.50:0/1757017601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9a9070 0x55ebbe9a9490 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbe78093120 tx=0x7fbe78069390 comp rx=0 tx=0).stop 2024-05-08T18:06:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 -- 172.21.15.50:0/1757017601 shutdown_connections 2024-05-08T18:06:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 --2- 172.21.15.50:0/1757017601 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbd6003bab0 0x7fbd6003df70 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 --2- 172.21.15.50:0/1757017601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe9a9070 0x55ebbe9a9490 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 -- 172.21.15.50:0/1757017601 >> 172.21.15.50:0/1757017601 conn(0x55ebbe99d110 msgr2=0x55ebbe99d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 -- 172.21.15.50:0/1757017601 shutdown_connections 2024-05-08T18:06:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.560+0000 7fbe89208800 1 -- 172.21.15.50:0/1757017601 wait complete. 2024-05-08T18:06:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.561+0000 7fbe89208800 1 -- 172.21.15.50:0/3711153108 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ebbe994350 msgr2=0x55ebbe996740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.561+0000 7fbe89208800 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55ebbe994350 0x55ebbe996740 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.561+0000 7fbe89208800 1 -- 172.21.15.50:0/3711153108 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ebbe88e910 msgr2=0x55ebbe991ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.561+0000 7fbe89208800 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55ebbe88e910 0x55ebbe991ad0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.561+0000 7fbe89208800 1 -- 172.21.15.50:0/3711153108 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ebbe998240 msgr2=0x55ebbe99a630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.561+0000 7fbe89208800 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55ebbe998240 0x55ebbe99a630 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.561+0000 7fbe89208800 1 -- 172.21.15.50:0/3711153108 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbd6c03bba0 msgr2=0x7fbd6c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.561+0000 7fbe89208800 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbd6c03bba0 0x7fbd6c03e060 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fbe70000c00 tx=0x7fbe700013b0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.561+0000 7fbe89208800 1 -- 172.21.15.50:0/3711153108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 msgr2=0x55ebbe98a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.561+0000 7fbe89208800 1 --2- 172.21.15.50:0/3711153108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ebbe88dfb0 0x55ebbe98a890 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbe78061040 tx=0x7fbe78002fb0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.561+0000 7fbe87501640 1 -- 172.21.15.50:0/3711153108 reap_dead start 2024-05-08T18:06:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.561+0000 7fbe89208800 1 -- 172.21.15.50:0/3711153108 shutdown_connections 2024-05-08T18:06:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.561+0000 7fbe89208800 1 -- 172.21.15.50:0/3711153108 >> 172.21.15.50:0/3711153108 conn(0x55ebbe889b70 msgr2=0x55ebbe97cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.561+0000 7fbe89208800 1 -- 172.21.15.50:0/3711153108 shutdown_connections 2024-05-08T18:06:03.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.561+0000 7fbe89208800 1 -- 172.21.15.50:0/3711153108 wait complete. 2024-05-08T18:06:03.568 INFO:teuthology.orchestra.run.smithi050.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}],"s3select":{}},{"bucket":"myfoo2","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo3","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo4","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[],"s3select":{}}]}],"summary":[{"user":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}],"total":{"bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4,"bytes_processed":0,"bytes_returned":0}}]} 2024-05-08T18:06:03.568 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 's3select': {}}, {'bucket': 'myfoo2', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo3', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo4', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-05-08T18:06:03.568 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2024-05-08T18:06:03.569 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2024-05-08T18:06:03.569 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_bucket 2024-05-08T18:06:03.616 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:03.617 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:03.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.619+0000 7fb5b3f19800 1 Processor -- start 2024-05-08T18:06:03.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.619+0000 7fb5b3f19800 1 -- start start 2024-05-08T18:06:03.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.620+0000 7fb5b3f19800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5652e01f5a00 0x5652e01f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.620+0000 7fb5b3f19800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5652e01f83c0 con 0x5652e01f1570 2024-05-08T18:06:03.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.620+0000 7fb5b3f19800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5652e0092e20 con 0x5652e01f5a00 2024-05-08T18:06:03.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.620+0000 7fb5b1217640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5652e01f5a00 0x5652e01f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.620+0000 7fb5b1217640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5652e01f5a00 0x5652e01f7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48918/0 (socket says 172.21.15.50:48918) 2024-05-08T18:06:03.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.620+0000 7fb5b1a18640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5652e01f1570 0x5652e01f1990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:54568/0 (socket says 172.21.15.50:54568) 2024-05-08T18:06:03.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.620+0000 7fb5b1217640 1 -- 172.21.15.50:0/4090760452 learned_addr learned my addr 172.21.15.50:0/4090760452 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.620+0000 7fb5b1217640 1 -- 172.21.15.50:0/4090760452 >> v1:172.21.15.46:6789/0 conn(0x5652e01f1570 legacy=0x5652e01f1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.620+0000 7fb5b1217640 1 -- 172.21.15.50:0/4090760452 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652e01f95f0 con 0x5652e01f5a00 2024-05-08T18:06:03.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.622+0000 7fb5b1217640 1 --2- 172.21.15.50:0/4090760452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f5a00 0x5652e01f7df0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800a520 tx=0x7fb5a801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe9b5a9d186f0b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.622+0000 7fb5b0a16640 1 -- 172.21.15.50:0/4090760452 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a8057070 con 0x5652e01f5a00 2024-05-08T18:06:03.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.622+0000 7fb5b0a16640 1 -- 172.21.15.50:0/4090760452 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a8052570 con 0x5652e01f5a00 2024-05-08T18:06:03.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.622+0000 7fb5b0a16640 1 -- 172.21.15.50:0/4090760452 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a804ae60 con 0x5652e01f5a00 2024-05-08T18:06:03.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.623+0000 7fb5b3f19800 1 -- 172.21.15.50:0/4090760452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f5a00 msgr2=0x5652e01f7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.623+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/4090760452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f5a00 0x5652e01f7df0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb5a800a520 tx=0x7fb5a801d010 comp rx=0 tx=0).stop 2024-05-08T18:06:03.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.623+0000 7fb5b3f19800 1 -- 172.21.15.50:0/4090760452 shutdown_connections 2024-05-08T18:06:03.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.623+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/4090760452 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f5a00 0x5652e01f7df0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.623+0000 7fb5b3f19800 1 -- 172.21.15.50:0/4090760452 >> 172.21.15.50:0/4090760452 conn(0x5652e01ecb70 msgr2=0x5652e01eef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.623+0000 7fb5b3f19800 1 -- 172.21.15.50:0/4090760452 shutdown_connections 2024-05-08T18:06:03.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.623+0000 7fb5b3f19800 1 -- 172.21.15.50:0/4090760452 wait complete. 2024-05-08T18:06:03.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.638+0000 7fb5b3f19800 1 Processor -- start 2024-05-08T18:06:03.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.638+0000 7fb5b3f19800 1 -- start start 2024-05-08T18:06:03.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.638+0000 7fb5b3f19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f1570 0x5652e0165550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.638+0000 7fb5b3f19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5652e013da30 con 0x5652e01f1570 2024-05-08T18:06:03.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.639+0000 7fb5b1a18640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f1570 0x5652e0165550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.639+0000 7fb5b1a18640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f1570 0x5652e0165550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48930/0 (socket says 172.21.15.50:48930) 2024-05-08T18:06:03.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.639+0000 7fb5b1a18640 1 -- 172.21.15.50:0/2291198630 learned_addr learned my addr 172.21.15.50:0/2291198630 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.639+0000 7fb5b1a18640 1 -- 172.21.15.50:0/2291198630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652e02e6880 con 0x5652e01f1570 2024-05-08T18:06:03.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.639+0000 7fb5b1a18640 1 --2- 172.21.15.50:0/2291198630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f1570 0x5652e0165550 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400b260 tx=0x7fb5a400b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc390ea39b3659c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.639+0000 7fb49fe07640 1 -- 172.21.15.50:0/2291198630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a400be60 con 0x5652e01f1570 2024-05-08T18:06:03.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.639+0000 7fb49fe07640 1 -- 172.21.15.50:0/2291198630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a4006070 con 0x5652e01f1570 2024-05-08T18:06:03.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.639+0000 7fb49fe07640 1 -- 172.21.15.50:0/2291198630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a400e710 con 0x5652e01f1570 2024-05-08T18:06:03.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.640+0000 7fb5b3f19800 1 -- 172.21.15.50:0/2291198630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f1570 msgr2=0x5652e0165550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.640+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/2291198630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f1570 0x5652e0165550 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400b260 tx=0x7fb5a400b660 comp rx=0 tx=0).stop 2024-05-08T18:06:03.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.640+0000 7fb5b3f19800 1 -- 172.21.15.50:0/2291198630 shutdown_connections 2024-05-08T18:06:03.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.640+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/2291198630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f1570 0x5652e0165550 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.640+0000 7fb5b3f19800 1 -- 172.21.15.50:0/2291198630 >> 172.21.15.50:0/2291198630 conn(0x5652e01ecb70 msgr2=0x5652e02de370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.640+0000 7fb5b3f19800 1 -- 172.21.15.50:0/2291198630 shutdown_connections 2024-05-08T18:06:03.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.640+0000 7fb5b3f19800 1 -- 172.21.15.50:0/2291198630 wait complete. 2024-05-08T18:06:03.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.640+0000 7fb5b3f19800 1 Processor -- start 2024-05-08T18:06:03.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.640+0000 7fb5b3f19800 1 -- start start 2024-05-08T18:06:03.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.641+0000 7fb5b3f19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f1570 0x5652e02e41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.641+0000 7fb5b3f19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5652e01f8890 con 0x5652e01f1570 2024-05-08T18:06:03.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.641+0000 7fb5b1a18640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f1570 0x5652e02e41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.641+0000 7fb5b1a18640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f1570 0x5652e02e41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48932/0 (socket says 172.21.15.50:48932) 2024-05-08T18:06:03.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.641+0000 7fb5b1a18640 1 -- 172.21.15.50:0/1074465088 learned_addr learned my addr 172.21.15.50:0/1074465088 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.641+0000 7fb5b1a18640 1 -- 172.21.15.50:0/1074465088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652e02e28a0 con 0x5652e01f1570 2024-05-08T18:06:03.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.641+0000 7fb5b1a18640 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f1570 0x5652e02e41b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x5652e0164b80 tx=0x7fb5a4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.641+0000 7fb49e604640 1 -- 172.21.15.50:0/1074465088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a401a7c0 con 0x5652e01f1570 2024-05-08T18:06:03.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.641+0000 7fb49e604640 1 -- 172.21.15.50:0/1074465088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a401a980 con 0x5652e01f1570 2024-05-08T18:06:03.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.641+0000 7fb5b3f19800 1 -- 172.21.15.50:0/1074465088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5652e02e2b30 con 0x5652e01f1570 2024-05-08T18:06:03.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.641+0000 7fb49e604640 1 -- 172.21.15.50:0/1074465088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a401aca0 con 0x5652e01f1570 2024-05-08T18:06:03.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.642+0000 7fb5b3f19800 1 -- 172.21.15.50:0/1074465088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652e02e3050 con 0x5652e01f1570 2024-05-08T18:06:03.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.642+0000 7fb49e604640 1 -- 172.21.15.50:0/1074465088 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fb5a4017070 con 0x5652e01f1570 2024-05-08T18:06:03.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.643+0000 7fb49e604640 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb49803bba0 0x7fb49803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.643+0000 7fb49e604640 1 -- 172.21.15.50:0/1074465088 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fb5a4015020 con 0x5652e01f1570 2024-05-08T18:06:03.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.643+0000 7fb5b1217640 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb49803bba0 0x7fb49803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.643+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5652e01f5a00 0x5652e02f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.643+0000 7fb5b3f19800 1 -- 172.21.15.50:0/1074465088 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e02f50c0 con 0x5652e01f5a00 2024-05-08T18:06:03.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.643+0000 7fb5b2219640 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5652e01f5a00 0x5652e02f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.643+0000 7fb5b1217640 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb49803bba0 0x7fb49803e060 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8054000 tx=0x7fb5a804e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.643+0000 7fb5b2219640 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5652e01f5a00 0x5652e02f4ae0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.644+0000 7fb5b2219640 1 -- 172.21.15.50:0/1074465088 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb5ac00a4a0 con 0x5652e01f5a00 2024-05-08T18:06:03.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.644+0000 7fb5b3f19800 1 -- 172.21.15.50:0/1074465088 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e02f59a0 con 0x5652e01f5a00 2024-05-08T18:06:03.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.644+0000 7fb5b2219640 1 -- 172.21.15.50:0/1074465088 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fb5ac00a4a0 con 0x5652e01f5a00 2024-05-08T18:06:03.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.644+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5652e02f7420 0x5652e02f98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.644+0000 7fb5b3f19800 1 -- 172.21.15.50:0/1074465088 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e02fa070 con 0x5652e02f7420 2024-05-08T18:06:03.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.644+0000 7fb5b1a18640 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5652e02f7420 0x5652e02f98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.644+0000 7fb5b1a18640 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5652e02f7420 0x5652e02f98e0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.645+0000 7fb5b1a18640 1 -- 172.21.15.50:0/1074465088 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fb5a402e070 con 0x5652e02f7420 2024-05-08T18:06:03.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.645+0000 7fb5b3f19800 1 -- 172.21.15.50:0/1074465088 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e02f61f0 con 0x5652e01f5a00 2024-05-08T18:06:03.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.645+0000 7fb5b2219640 1 -- 172.21.15.50:0/1074465088 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fb5ac00a4a0 con 0x5652e01f5a00 2024-05-08T18:06:03.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.645+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5652e02fb4d0 0x5652e02fd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.645+0000 7fb5b3f19800 1 -- 172.21.15.50:0/1074465088 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e02fdfb0 con 0x5652e02fb4d0 2024-05-08T18:06:03.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.645+0000 7fb5b1217640 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5652e02fb4d0 0x5652e02fd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.645+0000 7fb5b1217640 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5652e02fb4d0 0x5652e02fd8c0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.646+0000 7fb5b1217640 1 -- 172.21.15.50:0/1074465088 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fb5a805a040 con 0x5652e02fb4d0 2024-05-08T18:06:03.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.646+0000 7fb5b3f19800 1 Processor -- start 2024-05-08T18:06:03.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.646+0000 7fb5b3f19800 1 -- start start 2024-05-08T18:06:03.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.646+0000 7fb5b3f19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e030c320 0x5652e030c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.646+0000 7fb5b3f19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5652e02e46f0 con 0x5652e030c320 2024-05-08T18:06:03.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.646+0000 7fb5b1a18640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e030c320 0x5652e030c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.646+0000 7fb5b1a18640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e030c320 0x5652e030c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48940/0 (socket says 172.21.15.50:48940) 2024-05-08T18:06:03.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.646+0000 7fb5b1a18640 1 -- 172.21.15.50:0/3064631436 learned_addr learned my addr 172.21.15.50:0/3064631436 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.647+0000 7fb5b1a18640 1 -- 172.21.15.50:0/3064631436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652e030cc80 con 0x5652e030c320 2024-05-08T18:06:03.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.647+0000 7fb5b1a18640 1 --2- 172.21.15.50:0/3064631436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e030c320 0x5652e030c740 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4001840 tx=0x7fb5a401d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.647+0000 7fb4977fe640 1 -- 172.21.15.50:0/3064631436 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a404a3d0 con 0x5652e030c320 2024-05-08T18:06:03.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.647+0000 7fb4977fe640 1 -- 172.21.15.50:0/3064631436 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a404ae50 con 0x5652e030c320 2024-05-08T18:06:03.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.647+0000 7fb5b3f19800 1 -- 172.21.15.50:0/3064631436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5652e030d6f0 con 0x5652e030c320 2024-05-08T18:06:03.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.647+0000 7fb4977fe640 1 -- 172.21.15.50:0/3064631436 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a40531e0 con 0x5652e030c320 2024-05-08T18:06:03.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.647+0000 7fb5b3f19800 1 -- 172.21.15.50:0/3064631436 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652e030dbf0 con 0x5652e030c320 2024-05-08T18:06:03.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.648+0000 7fb4977fe640 1 -- 172.21.15.50:0/3064631436 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fb5a401b590 con 0x5652e030c320 2024-05-08T18:06:03.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.648+0000 7fb4977fe640 1 --2- 172.21.15.50:0/3064631436 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb48c03bb50 0x7fb48c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.648+0000 7fb4977fe640 1 -- 172.21.15.50:0/3064631436 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fb5a403d9d0 con 0x5652e030c320 2024-05-08T18:06:03.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.648+0000 7fb5b1217640 1 --2- 172.21.15.50:0/3064631436 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb48c03bb50 0x7fb48c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.648+0000 7fb5b3f19800 1 Processor -- start 2024-05-08T18:06:03.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.648+0000 7fb5b3f19800 1 -- start start 2024-05-08T18:06:03.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.648+0000 7fb5b1217640 1 --2- 172.21.15.50:0/3064631436 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb48c03bb50 0x7fb48c03e010 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x5652e02e3570 tx=0x7fb5a8005ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.649+0000 7fb5b3f19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e03193b0 0x5652e03197d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.649+0000 7fb5b3f19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5652e00e4330 con 0x5652e03193b0 2024-05-08T18:06:03.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.649+0000 7fb5b1a18640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e03193b0 0x5652e03197d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.649+0000 7fb5b1a18640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e03193b0 0x5652e03197d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48950/0 (socket says 172.21.15.50:48950) 2024-05-08T18:06:03.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.649+0000 7fb5b1a18640 1 -- 172.21.15.50:0/2675565710 learned_addr learned my addr 172.21.15.50:0/2675565710 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.649+0000 7fb5b1a18640 1 -- 172.21.15.50:0/2675565710 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652e0319da0 con 0x5652e03193b0 2024-05-08T18:06:03.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.649+0000 7fb5b1a18640 1 --2- 172.21.15.50:0/2675565710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e03193b0 0x5652e03197d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb5a401df40 tx=0x7fb5a400fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5eecd5dca2452d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.649+0000 7fb4957fa640 1 -- 172.21.15.50:0/2675565710 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a40315b0 con 0x5652e03193b0 2024-05-08T18:06:03.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.649+0000 7fb4957fa640 1 -- 172.21.15.50:0/2675565710 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a404a6f0 con 0x5652e03193b0 2024-05-08T18:06:03.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.649+0000 7fb4957fa640 1 -- 172.21.15.50:0/2675565710 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a402ccf0 con 0x5652e03193b0 2024-05-08T18:06:03.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.650+0000 7fb5b3f19800 1 -- 172.21.15.50:0/2675565710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e03193b0 msgr2=0x5652e03197d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.650+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/2675565710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e03193b0 0x5652e03197d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fb5a401df40 tx=0x7fb5a400fec0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.650+0000 7fb5b3f19800 1 -- 172.21.15.50:0/2675565710 shutdown_connections 2024-05-08T18:06:03.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.650+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/2675565710 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e03193b0 0x5652e03197d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.650+0000 7fb5b3f19800 1 -- 172.21.15.50:0/2675565710 >> 172.21.15.50:0/2675565710 conn(0x5652e0315710 msgr2=0x5652e0313770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.650+0000 7fb5b3f19800 1 -- 172.21.15.50:0/2675565710 shutdown_connections 2024-05-08T18:06:03.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.650+0000 7fb5b3f19800 1 -- 172.21.15.50:0/2675565710 wait complete. 2024-05-08T18:06:03.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.650+0000 7fb5b3f19800 1 Processor -- start 2024-05-08T18:06:03.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.650+0000 7fb5b3f19800 1 -- start start 2024-05-08T18:06:03.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.650+0000 7fb5b3f19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e03193b0 0x5652e0320650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.650+0000 7fb5b3f19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5652e031a2d0 con 0x5652e03193b0 2024-05-08T18:06:03.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.650+0000 7fb5b1a18640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e03193b0 0x5652e0320650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.650+0000 7fb5b1a18640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e03193b0 0x5652e0320650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48958/0 (socket says 172.21.15.50:48958) 2024-05-08T18:06:03.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.650+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 learned_addr learned my addr 172.21.15.50:0/148811137 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.651+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652e0320b90 con 0x5652e03193b0 2024-05-08T18:06:03.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.651+0000 7fb5b1a18640 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e03193b0 0x5652e0320650 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb5a403a740 tx=0x7fb5a404cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.651+0000 7fb4837fe640 1 -- 172.21.15.50:0/148811137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a403a7b0 con 0x5652e03193b0 2024-05-08T18:06:03.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.651+0000 7fb4837fe640 1 -- 172.21.15.50:0/148811137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5a406a070 con 0x5652e03193b0 2024-05-08T18:06:03.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.651+0000 7fb4837fe640 1 -- 172.21.15.50:0/148811137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb5a4019a70 con 0x5652e03193b0 2024-05-08T18:06:03.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.651+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5652e0320b90 con 0x5652e03193b0 2024-05-08T18:06:03.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.651+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652e0320b90 con 0x5652e03193b0 2024-05-08T18:06:03.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.652+0000 7fb4837fe640 1 -- 172.21.15.50:0/148811137 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fb5a4019c10 con 0x5652e03193b0 2024-05-08T18:06:03.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.652+0000 7fb4837fe640 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb47c03bba0 0x7fb47c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.652+0000 7fb4837fe640 1 -- 172.21.15.50:0/148811137 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fb5a4055110 con 0x5652e03193b0 2024-05-08T18:06:03.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.652+0000 7fb5b1217640 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb47c03bba0 0x7fb47c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.652+0000 7fb5b3f19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.652+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5652e032bd40 0x5652e032e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.653+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e032e950 con 0x5652e032bd40 2024-05-08T18:06:03.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.653+0000 7fb5b2219640 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5652e032bd40 0x5652e032e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.653+0000 7fb5b1217640 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb47c03bba0 0x7fb47c03e060 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8000c00 tx=0x7fb5a804f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.653+0000 7fb5b2219640 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5652e032bd40 0x5652e032e200 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.653+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb5ac00a4a0 con 0x5652e032bd40 2024-05-08T18:06:03.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.653+0000 7fb5b3f19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.653+0000 7fb5b3f19800 20 realm 2024-05-08T18:06:03.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.653+0000 7fb5b3f19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.653+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e032ed60 con 0x5652e032bd40 2024-05-08T18:06:03.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.653+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb5ac00a4a0 con 0x5652e032bd40 2024-05-08T18:06:03.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.653+0000 7fb5b3f19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.654+0000 7fb5b3f19800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:03.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.654+0000 7fb5b3f19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.654+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e032f630 con 0x5652e032bd40 2024-05-08T18:06:03.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.654+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb5ac00a4a0 con 0x5652e032bd40 2024-05-08T18:06:03.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.654+0000 7fb5b3f19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.654+0000 7fb5b3f19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.654+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e032f9d0 con 0x5652e032bd40 2024-05-08T18:06:03.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.654+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fb5ac00a4a0 con 0x5652e032bd40 2024-05-08T18:06:03.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.654+0000 7fb5b3f19800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:03.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.654+0000 7fb5b3f19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.654+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5652e03304c0 0x5652e03328b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.654+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e0333010 con 0x5652e03304c0 2024-05-08T18:06:03.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.654+0000 7fb5b1a18640 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5652e03304c0 0x5652e03328b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.655+0000 7fb5b1a18640 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5652e03304c0 0x5652e03328b0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.655+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.655+0000 7fb5b3f19800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:03.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.655+0000 7fb5b3f19800 20 searching for the correct realm 2024-05-08T18:06:03.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.655+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5652e03350b0 0x5652e0337570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.655+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5652e0337c40 con 0x5652e03350b0 2024-05-08T18:06:03.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.655+0000 7fb5b1217640 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5652e03350b0 0x5652e0337570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.656+0000 7fb5b1217640 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5652e03350b0 0x5652e0337570 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.656+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a807f070 con 0x5652e03350b0 2024-05-08T18:06:03.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.656+0000 7fb5b1217640 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fb5a8074040 0x7fb5a8081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.656+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5652e0337c40 con 0x7fb5a8074040 2024-05-08T18:06:03.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.656+0000 7fb5b2219640 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fb5a8074040 0x7fb5a8081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.656+0000 7fb5b2219640 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fb5a8074040 0x7fb5a8081410 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.657+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac00a4a0 con 0x7fb5a8074040 2024-05-08T18:06:03.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.657+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5652e032f9d0 con 0x5652e03304c0 2024-05-08T18:06:03.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.657+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.657+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5652e032f9d0 con 0x5652e032bd40 2024-05-08T18:06:03.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.657+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac00a4a0 con 0x5652e032bd40 2024-05-08T18:06:03.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.657+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5652e032f9d0 con 0x5652e032bd40 2024-05-08T18:06:03.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.657+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac00a4a0 con 0x5652e032bd40 2024-05-08T18:06:03.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.657+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5652e032f9d0 con 0x5652e032bd40 2024-05-08T18:06:03.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.658+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac00a4a0 con 0x5652e032bd40 2024-05-08T18:06:03.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.658+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5652e032f9d0 con 0x5652e03304c0 2024-05-08T18:06:03.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.658+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.658+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5652e032f9d0 con 0x7fb5a8074040 2024-05-08T18:06:03.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.658+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac00a4a0 con 0x7fb5a8074040 2024-05-08T18:06:03.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.658+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5652e032f9d0 con 0x5652e03350b0 2024-05-08T18:06:03.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.658+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a807f070 con 0x5652e03350b0 2024-05-08T18:06:03.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.658+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5652e032f9d0 con 0x5652e032bd40 2024-05-08T18:06:03.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.659+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac00a4a0 con 0x5652e032bd40 2024-05-08T18:06:03.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.659+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5652e032f9d0 con 0x5652e03304c0 2024-05-08T18:06:03.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.659+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.659+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5652e032f9d0 con 0x5652e032bd40 2024-05-08T18:06:03.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.659+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac00a4a0 con 0x5652e032bd40 2024-05-08T18:06:03.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.659+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5652e032f9d0 con 0x7fb5a8074040 2024-05-08T18:06:03.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.659+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fb5ac032040 con 0x7fb5a8074040 2024-05-08T18:06:03.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.659+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5652e032f9d0 con 0x5652e03350b0 2024-05-08T18:06:03.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.660+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a807f070 con 0x5652e03350b0 2024-05-08T18:06:03.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.660+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x5652e03350b0 2024-05-08T18:06:03.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.660+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a807f070 con 0x5652e03350b0 2024-05-08T18:06:03.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.660+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x5652e03304c0 2024-05-08T18:06:03.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.660+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.660+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x5652e03350b0 2024-05-08T18:06:03.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.660+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a807f070 con 0x5652e03350b0 2024-05-08T18:06:03.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.660+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x5652e03350b0 2024-05-08T18:06:03.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.660+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fb5a807f070 con 0x5652e03350b0 2024-05-08T18:06:03.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.661+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x5652e032bd40 2024-05-08T18:06:03.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.661+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac032040 con 0x5652e032bd40 2024-05-08T18:06:03.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.661+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x5652e032bd40 2024-05-08T18:06:03.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.661+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fb5ac032040 con 0x5652e032bd40 2024-05-08T18:06:03.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.661+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x5652e03304c0 2024-05-08T18:06:03.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.661+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.661+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x5652e03350b0 2024-05-08T18:06:03.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.661+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a807f070 con 0x5652e03350b0 2024-05-08T18:06:03.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.661+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x7fb5a8074040 2024-05-08T18:06:03.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.662+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac032040 con 0x7fb5a8074040 2024-05-08T18:06:03.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.662+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x5652e032bd40 2024-05-08T18:06:03.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.662+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac032040 con 0x5652e032bd40 2024-05-08T18:06:03.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.662+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x5652e03304c0 2024-05-08T18:06:03.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.662+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.662+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x5652e03350b0 2024-05-08T18:06:03.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.662+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a807f070 con 0x5652e03350b0 2024-05-08T18:06:03.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.663+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x5652e03304c0 2024-05-08T18:06:03.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.663+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.663+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x7fb5a8074040 2024-05-08T18:06:03.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.663+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac032040 con 0x7fb5a8074040 2024-05-08T18:06:03.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.663+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x5652e03350b0 2024-05-08T18:06:03.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.663+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a807f070 con 0x5652e03350b0 2024-05-08T18:06:03.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.663+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x5652e03304c0 2024-05-08T18:06:03.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x5652e032bd40 2024-05-08T18:06:03.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fb5ac032040 con 0x5652e032bd40 2024-05-08T18:06:03.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fb5a808e040 con 0x7fb5a8074040 2024-05-08T18:06:03.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb5ac032040 con 0x7fb5a8074040 2024-05-08T18:06:03.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b3f19800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:03.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b3f19800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:03.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b3f19800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:03.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b3f19800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:03.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b3f19800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:03.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b3f19800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:03.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b3f19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e03339b0 con 0x5652e032bd40 2024-05-08T18:06:03.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb5ac032040 con 0x5652e032bd40 2024-05-08T18:06:03.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b3f19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b3f19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.664+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e03339b0 con 0x5652e032bd40 2024-05-08T18:06:03.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fb5ac032040 con 0x5652e032bd40 2024-05-08T18:06:03.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b3f19800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:03.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b3f19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e0333960 con 0x5652e03350b0 2024-05-08T18:06:03.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fb5a807f070 con 0x5652e03350b0 2024-05-08T18:06:03.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b3f19800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:03.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b3f19800 20 zone default found 2024-05-08T18:06:03.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b3f19800 4 Realm: () 2024-05-08T18:06:03.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b3f19800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:03.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b3f19800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:03.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b3f19800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:03.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b3f19800 20 zonegroup default 2024-05-08T18:06:03.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b3f19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e0334250 con 0x5652e03304c0 2024-05-08T18:06:03.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b3f19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b3f19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.665+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e0339aa0 con 0x5652e03350b0 2024-05-08T18:06:03.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.666+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fb5a807f070 con 0x5652e03350b0 2024-05-08T18:06:03.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.666+0000 7fb5b3f19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.666+0000 7fb5b3f19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.666+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e0339e40 con 0x5652e03350b0 2024-05-08T18:06:03.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.666+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fb5a807f070 con 0x5652e03350b0 2024-05-08T18:06:03.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.666+0000 7fb5b3f19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.666+0000 7fb5b3f19800 20 started sync module instance, tier type = 2024-05-08T18:06:03.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.666+0000 7fb5b3f19800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:03.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.666+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e0338f40 con 0x7fb5a8074040 2024-05-08T18:06:03.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.667+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5ac032040 con 0x7fb5a8074040 2024-05-08T18:06:03.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.667+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94913948783712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e0338f40 con 0x7fb5a8074040 2024-05-08T18:06:03.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.667+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033a8b0 con 0x5652e032bd40 2024-05-08T18:06:03.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.668+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94913948783712] v29'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5ac032040 con 0x7fb5a8074040 2024-05-08T18:06:03.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.668+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5ac032040 con 0x5652e032bd40 2024-05-08T18:06:03.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.668+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94913948787616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e0338f40 con 0x5652e032bd40 2024-05-08T18:06:03.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.668+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033b360 con 0x5652e03350b0 2024-05-08T18:06:03.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.669+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a807f070 con 0x5652e03350b0 2024-05-08T18:06:03.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.669+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94913948790336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033bc00 con 0x5652e03350b0 2024-05-08T18:06:03.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.669+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033c6e0 con 0x7fb5a8074040 2024-05-08T18:06:03.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.669+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94913948787616] v29'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5ac032040 con 0x5652e032bd40 2024-05-08T18:06:03.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.670+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94913948790336] v29'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a807f070 con 0x5652e03350b0 2024-05-08T18:06:03.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.670+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5ac032040 con 0x7fb5a8074040 2024-05-08T18:06:03.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.670+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94913948792656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e0338a90 con 0x7fb5a8074040 2024-05-08T18:06:03.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.671+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033cf80 con 0x5652e032bd40 2024-05-08T18:06:03.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.672+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94913948792656] v29'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5ac032040 con 0x7fb5a8074040 2024-05-08T18:06:03.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.672+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5ac032040 con 0x5652e032bd40 2024-05-08T18:06:03.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.672+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94913948795376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033cf00 con 0x5652e032bd40 2024-05-08T18:06:03.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.672+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033d990 con 0x5652e03304c0 2024-05-08T18:06:03.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.673+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94913948795376] v29'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5ac032040 con 0x5652e032bd40 2024-05-08T18:06:03.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.673+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.673+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94913948797520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033ce70 con 0x5652e03304c0 2024-05-08T18:06:03.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.673+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033e6b0 con 0x5652e03304c0 2024-05-08T18:06:03.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.674+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94913948797520] v29'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.674+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.674+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94913948802832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033ef00 con 0x5652e03304c0 2024-05-08T18:06:03.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.674+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033f8c0 con 0x5652e03304c0 2024-05-08T18:06:03.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.675+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94913948802832] v29'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.675+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.675+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94913948806192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033f8c0 con 0x5652e03304c0 2024-05-08T18:06:03.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.675+0000 7fb5b3f19800 20 add_watcher() i=0 2024-05-08T18:06:03.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.675+0000 7fb5b3f19800 20 add_watcher() i=1 2024-05-08T18:06:03.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.675+0000 7fb5b3f19800 20 add_watcher() i=2 2024-05-08T18:06:03.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.675+0000 7fb5b3f19800 20 add_watcher() i=3 2024-05-08T18:06:03.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.675+0000 7fb5b3f19800 20 add_watcher() i=4 2024-05-08T18:06:03.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.675+0000 7fb5b3f19800 20 add_watcher() i=5 2024-05-08T18:06:03.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.675+0000 7fb5b3f19800 20 add_watcher() i=6 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.676+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94913948806192] v29'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.676+0000 7fb5b3f19800 20 add_watcher() i=7 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.676+0000 7fb5b3f19800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.676+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e0342bd0 con 0x5652e032bd40 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.676+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fb5ac032040 con 0x5652e032bd40 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.676+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94913948808576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033fe50 con 0x5652e032bd40 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.677+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94913948808576] v29'208 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fb5ac032040 con 0x5652e032bd40 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.677+0000 7fb4637fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.677+0000 7fb5b3f19800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.677+0000 7fb5b3f19800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.677+0000 7fb5b3f19800 5 note: GC not initialized 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.678+0000 7fb4627fc640 20 reqs_thread_entry: start 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.678+0000 7fb5b3f19800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.678+0000 7fb5b3f19800 20 Filter name: none 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.678+0000 7fb3d7fff640 20 reqs_thread_entry: start 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.678+0000 7fb5b3f19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.678+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e0375cc0 con 0x5652e032bd40 2024-05-08T18:06:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.678+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fb5ac032040 con 0x5652e032bd40 2024-05-08T18:06:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.678+0000 7fb5b3f19800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:06:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.678+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5652e0377210 con 0x5652e03304c0 2024-05-08T18:06:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.679+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(63 usage.10 [call out=1378b] v0'0 uv29 ondisk = 0) ==== 152+0+1378 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.680+0000 7fb4637fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.680+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94913948808576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e0341990 con 0x5652e032bd40 2024-05-08T18:06:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.681+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94913948808576] v29'209 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fb5ac032040 con 0x5652e032bd40 2024-05-08T18:06:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.681+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94913948783712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033f580 con 0x7fb5a8074040 2024-05-08T18:06:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.682+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94913948783712] v29'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5ac032040 con 0x7fb5a8074040 2024-05-08T18:06:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.682+0000 7fb5b3f19800 20 remove_watcher() i=0 2024-05-08T18:06:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.682+0000 7fb5b3f19800 2 removed watcher, disabling cache 2024-05-08T18:06:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.682+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94913948787616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033f580 con 0x5652e032bd40 2024-05-08T18:06:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.683+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94913948787616] v29'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5ac032040 con 0x5652e032bd40 2024-05-08T18:06:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.683+0000 7fb5b3f19800 20 remove_watcher() i=1 2024-05-08T18:06:03.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.683+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94913948790336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033a3a0 con 0x5652e03350b0 2024-05-08T18:06:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.684+0000 7fb5b1217640 1 -- 172.21.15.50:0/148811137 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94913948790336] v29'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a807f070 con 0x5652e03350b0 2024-05-08T18:06:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.684+0000 7fb5b3f19800 20 remove_watcher() i=2 2024-05-08T18:06:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.684+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94913948792656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033f580 con 0x7fb5a8074040 2024-05-08T18:06:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.685+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94913948792656] v29'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5ac032040 con 0x7fb5a8074040 2024-05-08T18:06:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.685+0000 7fb5b3f19800 20 remove_watcher() i=3 2024-05-08T18:06:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.685+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94913948795376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033ae40 con 0x5652e032bd40 2024-05-08T18:06:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.686+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94913948795376] v29'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5ac032040 con 0x5652e032bd40 2024-05-08T18:06:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.686+0000 7fb5b3f19800 20 remove_watcher() i=4 2024-05-08T18:06:03.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.686+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94913948797520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033a3a0 con 0x5652e03304c0 2024-05-08T18:06:03.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.687+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94913948797520] v29'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.688+0000 7fb5b3f19800 20 remove_watcher() i=5 2024-05-08T18:06:03.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.688+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94913948802832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033f580 con 0x5652e03304c0 2024-05-08T18:06:03.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.688+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94913948802832] v29'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.688+0000 7fb5b3f19800 20 remove_watcher() i=6 2024-05-08T18:06:03.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.688+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94913948806192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5652e033ae40 con 0x5652e03304c0 2024-05-08T18:06:03.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b1a18640 1 -- 172.21.15.50:0/148811137 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94913948806192] v29'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb5a405c070 con 0x5652e03304c0 2024-05-08T18:06:03.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 20 remove_watcher() i=7 2024-05-08T18:06:03.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5652e03304c0 msgr2=0x5652e03328b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5652e03304c0 0x5652e03328b0 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5652e032bd40 msgr2=0x5652e032e200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5652e032bd40 0x5652e032e200 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fb5a8074040 msgr2=0x7fb5a8081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fb5a8074040 0x7fb5a8081410 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5652e03350b0 msgr2=0x5652e0337570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5652e03350b0 0x5652e0337570 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb47c03bba0 msgr2=0x7fb47c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb47c03bba0 0x7fb47c03e060 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8000c00 tx=0x7fb5a804f000 comp rx=0 tx=0).stop 2024-05-08T18:06:03.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e03193b0 msgr2=0x5652e0320650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b2219640 1 -- 172.21.15.50:0/148811137 reap_dead start 2024-05-08T18:06:03.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e03193b0 0x5652e0320650 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fb5a403a740 tx=0x7fb5a404cea0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 shutdown_connections 2024-05-08T18:06:03.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/148811137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e03193b0 0x5652e0320650 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 >> 172.21.15.50:0/148811137 conn(0x5652e0315710 msgr2=0x5652e0312d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 shutdown_connections 2024-05-08T18:06:03.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.690+0000 7fb5b3f19800 1 -- 172.21.15.50:0/148811137 wait complete. 2024-05-08T18:06:03.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 -- 172.21.15.50:0/3064631436 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb48c03bb50 msgr2=0x7fb48c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/3064631436 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb48c03bb50 0x7fb48c03e010 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x5652e02e3570 tx=0x7fb5a8005ac0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 -- 172.21.15.50:0/3064631436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e030c320 msgr2=0x5652e030c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/3064631436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e030c320 0x5652e030c740 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fb5a4001840 tx=0x7fb5a401d7b0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 -- 172.21.15.50:0/3064631436 shutdown_connections 2024-05-08T18:06:03.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/3064631436 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb48c03bb50 0x7fb48c03e010 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/3064631436 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e030c320 0x5652e030c740 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 -- 172.21.15.50:0/3064631436 >> 172.21.15.50:0/3064631436 conn(0x5652e03003c0 msgr2=0x5652e03007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 -- 172.21.15.50:0/3064631436 shutdown_connections 2024-05-08T18:06:03.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 -- 172.21.15.50:0/3064631436 wait complete. 2024-05-08T18:06:03.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 -- 172.21.15.50:0/1074465088 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5652e02f7420 msgr2=0x5652e02f98e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5652e02f7420 0x5652e02f98e0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 -- 172.21.15.50:0/1074465088 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5652e01f5a00 msgr2=0x5652e02f4ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5652e01f5a00 0x5652e02f4ae0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 -- 172.21.15.50:0/1074465088 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5652e02fb4d0 msgr2=0x5652e02fd8c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5652e02fb4d0 0x5652e02fd8c0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 -- 172.21.15.50:0/1074465088 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb49803bba0 msgr2=0x7fb49803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb49803bba0 0x7fb49803e060 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8054000 tx=0x7fb5a804e000 comp rx=0 tx=0).stop 2024-05-08T18:06:03.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 -- 172.21.15.50:0/1074465088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f1570 msgr2=0x5652e02e41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 --2- 172.21.15.50:0/1074465088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5652e01f1570 0x5652e02e41b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x5652e0164b80 tx=0x7fb5a4000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b2219640 1 -- 172.21.15.50:0/1074465088 reap_dead start 2024-05-08T18:06:03.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 -- 172.21.15.50:0/1074465088 shutdown_connections 2024-05-08T18:06:03.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 -- 172.21.15.50:0/1074465088 >> 172.21.15.50:0/1074465088 conn(0x5652e01ecb70 msgr2=0x5652e02dcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 -- 172.21.15.50:0/1074465088 shutdown_connections 2024-05-08T18:06:03.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.691+0000 7fb5b3f19800 1 -- 172.21.15.50:0/1074465088 wait complete. 2024-05-08T18:06:03.698 INFO:teuthology.orchestra.run.smithi050.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[],"s3select":{}},{"bucket":"myfoo","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":2,"successful_ops":1}],"s3select":{}},{"bucket":"myfoo2","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}},{"bucket":"myfoo3","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}},{"bucket":"myfoo4","time":"2024-05-08T18:00:00.000000Z","epoch":1715191200,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}],"s3select":{}}]}],"summary":[{"user":"foo","categories":[{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4}],"total":{"bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4,"bytes_processed":0,"bytes_returned":0}}]} 2024-05-08T18:06:03.699 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'myfoo', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'myfoo2', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'myfoo3', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'myfoo4', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}], 'total': {'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-05-08T18:06:03.699 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-05-08T18:06:03.699 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-05-08T18:06:03.699 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid bar --display-name Bar --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --max-buckets 4 2024-05-08T18:06:03.753 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:03.754 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:03.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.756+0000 7f46fc719800 1 Processor -- start 2024-05-08T18:06:03.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.756+0000 7f46fc719800 1 -- start start 2024-05-08T18:06:03.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.757+0000 7f46fc719800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55d1f5478730 0x55d1f5478b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.757+0000 7f46fc719800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d1f54791d0 con 0x55d1f54798f0 2024-05-08T18:06:03.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.757+0000 7f46fc719800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55d1f5318e20 con 0x55d1f5478730 2024-05-08T18:06:03.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.757+0000 7f46fa211640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55d1f5478730 0x55d1f5478b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.757+0000 7f46fa211640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55d1f5478730 0x55d1f5478b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48966/0 (socket says 172.21.15.50:48966) 2024-05-08T18:06:03.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.757+0000 7f46fa211640 1 -- 172.21.15.50:0/3760915296 learned_addr learned my addr 172.21.15.50:0/3760915296 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.757+0000 7f46f9a10640 1 --1- 172.21.15.50:0/3760915296 >> v1:172.21.15.46:6789/0 conn(0x55d1f54798f0 0x55d1f5479d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:54570/0 (socket says 172.21.15.50:54570) 2024-05-08T18:06:03.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.757+0000 7f46fa211640 1 -- 172.21.15.50:0/3760915296 >> v1:172.21.15.46:6789/0 conn(0x55d1f54798f0 legacy=0x55d1f5479d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.757+0000 7f46fa211640 1 -- 172.21.15.50:0/3760915296 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1f547a4b0 con 0x55d1f5478730 2024-05-08T18:06:03.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.759+0000 7f46fa211640 1 --2- 172.21.15.50:0/3760915296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 0x55d1f5478b50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00a520 tx=0x7f46ec01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197e1d7267867666 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.759+0000 7f46f920f640 1 -- 172.21.15.50:0/3760915296 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ec057070 con 0x55d1f5478730 2024-05-08T18:06:03.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.759+0000 7f46f920f640 1 -- 172.21.15.50:0/3760915296 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec052570 con 0x55d1f5478730 2024-05-08T18:06:03.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.759+0000 7f46f920f640 1 -- 172.21.15.50:0/3760915296 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ec052870 con 0x55d1f5478730 2024-05-08T18:06:03.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.760+0000 7f46fc719800 1 -- 172.21.15.50:0/3760915296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 msgr2=0x55d1f5478b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.760+0000 7f46fc719800 1 --2- 172.21.15.50:0/3760915296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 0x55d1f5478b50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00a520 tx=0x7f46ec01d010 comp rx=0 tx=0).stop 2024-05-08T18:06:03.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.760+0000 7f46fc719800 1 -- 172.21.15.50:0/3760915296 shutdown_connections 2024-05-08T18:06:03.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.760+0000 7f46fc719800 1 --2- 172.21.15.50:0/3760915296 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 0x55d1f5478b50 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.760+0000 7f46fc719800 1 -- 172.21.15.50:0/3760915296 >> 172.21.15.50:0/3760915296 conn(0x55d1f5473cb0 msgr2=0x55d1f54760f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.760+0000 7f46fc719800 1 -- 172.21.15.50:0/3760915296 shutdown_connections 2024-05-08T18:06:03.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.760+0000 7f46fc719800 1 -- 172.21.15.50:0/3760915296 wait complete. 2024-05-08T18:06:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.775+0000 7f46fc719800 1 Processor -- start 2024-05-08T18:06:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.775+0000 7f46fc719800 1 -- start start 2024-05-08T18:06:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.775+0000 7f46fc719800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 0x55d1f5569d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.775+0000 7f46fc719800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d1f53c3a50 con 0x55d1f5478730 2024-05-08T18:06:03.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.776+0000 7f46fa211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 0x55d1f5569d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.776+0000 7f46fa211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 0x55d1f5569d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48974/0 (socket says 172.21.15.50:48974) 2024-05-08T18:06:03.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.776+0000 7f46fa211640 1 -- 172.21.15.50:0/4112527603 learned_addr learned my addr 172.21.15.50:0/4112527603 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.776+0000 7f46fa211640 1 -- 172.21.15.50:0/4112527603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1f556a320 con 0x55d1f5478730 2024-05-08T18:06:03.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.776+0000 7f46fa211640 1 --2- 172.21.15.50:0/4112527603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 0x55d1f5569d50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f46ec055040 tx=0x7f46ec04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e1e3c292d9c084 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.776+0000 7f45e8608640 1 -- 172.21.15.50:0/4112527603 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ec057040 con 0x55d1f5478730 2024-05-08T18:06:03.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.776+0000 7f45e8608640 1 -- 172.21.15.50:0/4112527603 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec052870 con 0x55d1f5478730 2024-05-08T18:06:03.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.776+0000 7f45e8608640 1 -- 172.21.15.50:0/4112527603 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ec052b90 con 0x55d1f5478730 2024-05-08T18:06:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.777+0000 7f46fc719800 1 -- 172.21.15.50:0/4112527603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 msgr2=0x55d1f5569d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.777+0000 7f46fc719800 1 --2- 172.21.15.50:0/4112527603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 0x55d1f5569d50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f46ec055040 tx=0x7f46ec04c000 comp rx=0 tx=0).stop 2024-05-08T18:06:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.777+0000 7f46fc719800 1 -- 172.21.15.50:0/4112527603 shutdown_connections 2024-05-08T18:06:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.777+0000 7f46fc719800 1 --2- 172.21.15.50:0/4112527603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 0x55d1f5569d50 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.777+0000 7f46fc719800 1 -- 172.21.15.50:0/4112527603 >> 172.21.15.50:0/4112527603 conn(0x55d1f5473cb0 msgr2=0x55d1f5565670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.777+0000 7f46fc719800 1 -- 172.21.15.50:0/4112527603 shutdown_connections 2024-05-08T18:06:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.777+0000 7f46fc719800 1 -- 172.21.15.50:0/4112527603 wait complete. 2024-05-08T18:06:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.777+0000 7f46fc719800 1 Processor -- start 2024-05-08T18:06:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.777+0000 7f46fc719800 1 -- start start 2024-05-08T18:06:03.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.778+0000 7f46fc719800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 0x55d1f53edf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.778+0000 7f46fc719800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d1f5481dc0 con 0x55d1f5478730 2024-05-08T18:06:03.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.778+0000 7f46fa211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 0x55d1f53edf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.778+0000 7f46fa211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 0x55d1f53edf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48982/0 (socket says 172.21.15.50:48982) 2024-05-08T18:06:03.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.778+0000 7f46fa211640 1 -- 172.21.15.50:0/3078458260 learned_addr learned my addr 172.21.15.50:0/3078458260 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.778+0000 7f46fa211640 1 -- 172.21.15.50:0/3078458260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1f53ec610 con 0x55d1f5478730 2024-05-08T18:06:03.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.778+0000 7f46fa211640 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 0x55d1f53edf20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0524f0 tx=0x7f46ec003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.778+0000 7f45e6e05640 1 -- 172.21.15.50:0/3078458260 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ec073070 con 0x55d1f5478730 2024-05-08T18:06:03.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.778+0000 7f45e6e05640 1 -- 172.21.15.50:0/3078458260 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec05d570 con 0x55d1f5478730 2024-05-08T18:06:03.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.778+0000 7f46fc719800 1 -- 172.21.15.50:0/3078458260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d1f53ec8a0 con 0x55d1f5478730 2024-05-08T18:06:03.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.778+0000 7f45e6e05640 1 -- 172.21.15.50:0/3078458260 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ec057070 con 0x55d1f5478730 2024-05-08T18:06:03.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.778+0000 7f46fc719800 1 -- 172.21.15.50:0/3078458260 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1f53ecdc0 con 0x55d1f5478730 2024-05-08T18:06:03.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.779+0000 7f45e6e05640 1 -- 172.21.15.50:0/3078458260 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f46ec076070 con 0x55d1f5478730 2024-05-08T18:06:03.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.779+0000 7f45e6e05640 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f45e003bb50 0x7f45e003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.780+0000 7f45e6e05640 1 -- 172.21.15.50:0/3078458260 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f46ec094350 con 0x55d1f5478730 2024-05-08T18:06:03.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.780+0000 7f46f9a10640 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f45e003bb50 0x7f45e003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.780+0000 7f46fc719800 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d1f54798f0 0x55d1f557ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.780+0000 7f46fc719800 1 -- 172.21.15.50:0/3078458260 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f557b160 con 0x55d1f54798f0 2024-05-08T18:06:03.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.780+0000 7f46faa12640 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d1f54798f0 0x55d1f557ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.780+0000 7f46f9a10640 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f45e003bb50 0x7f45e003e010 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f46f0001e30 tx=0x7f46f0001390 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.780+0000 7f46faa12640 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d1f54798f0 0x55d1f557ab90 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.780+0000 7f46faa12640 1 -- 172.21.15.50:0/3078458260 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f46f400a4a0 con 0x55d1f54798f0 2024-05-08T18:06:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.780+0000 7f46fc719800 1 -- 172.21.15.50:0/3078458260 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f557ba20 con 0x55d1f54798f0 2024-05-08T18:06:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.781+0000 7f46faa12640 1 -- 172.21.15.50:0/3078458260 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f46f400a4a0 con 0x55d1f54798f0 2024-05-08T18:06:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.781+0000 7f46fc719800 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d1f557d4a0 0x55d1f557f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.781+0000 7f46fc719800 1 -- 172.21.15.50:0/3078458260 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55800d0 con 0x55d1f557d4a0 2024-05-08T18:06:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.781+0000 7f46fa211640 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d1f557d4a0 0x55d1f557f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.781+0000 7f46fa211640 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d1f557d4a0 0x55d1f557f940 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.782+0000 7f46fa211640 1 -- 172.21.15.50:0/3078458260 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f46ec075030 con 0x55d1f557d4a0 2024-05-08T18:06:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.782+0000 7f46fc719800 1 -- 172.21.15.50:0/3078458260 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f557c270 con 0x55d1f54798f0 2024-05-08T18:06:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.782+0000 7f46faa12640 1 -- 172.21.15.50:0/3078458260 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f46f400a4a0 con 0x55d1f54798f0 2024-05-08T18:06:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.782+0000 7f46fc719800 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d1f5581530 0x55d1f5583920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.782+0000 7f46fc719800 1 -- 172.21.15.50:0/3078458260 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f5584060 con 0x55d1f5581530 2024-05-08T18:06:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.782+0000 7f46f9a10640 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d1f5581530 0x55d1f5583920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.782+0000 7f46f9a10640 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d1f5581530 0x55d1f5583920 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.783+0000 7f46f9a10640 1 -- 172.21.15.50:0/3078458260 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f46f000bd50 con 0x55d1f5581530 2024-05-08T18:06:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.783+0000 7f46fc719800 1 Processor -- start 2024-05-08T18:06:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.783+0000 7f46fc719800 1 -- start start 2024-05-08T18:06:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.783+0000 7f46fc719800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5592390 0x55d1f55927b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.783+0000 7f46fc719800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d1f53ee460 con 0x55d1f5592390 2024-05-08T18:06:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.783+0000 7f46fa211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5592390 0x55d1f55927b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.783+0000 7f46fa211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5592390 0x55d1f55927b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:48990/0 (socket says 172.21.15.50:48990) 2024-05-08T18:06:03.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.783+0000 7f46fa211640 1 -- 172.21.15.50:0/2313229737 learned_addr learned my addr 172.21.15.50:0/2313229737 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.784+0000 7f46fa211640 1 -- 172.21.15.50:0/2313229737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1f5592cf0 con 0x55d1f5592390 2024-05-08T18:06:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.784+0000 7f46fa211640 1 --2- 172.21.15.50:0/2313229737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5592390 0x55d1f55927b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f46ec05ca90 tx=0x7f46ec076440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.784+0000 7f45dffff640 1 -- 172.21.15.50:0/2313229737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ec05ddb0 con 0x55d1f5592390 2024-05-08T18:06:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.784+0000 7f45dffff640 1 -- 172.21.15.50:0/2313229737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec0929f0 con 0x55d1f5592390 2024-05-08T18:06:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.784+0000 7f46fc719800 1 -- 172.21.15.50:0/2313229737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d1f5593760 con 0x55d1f5592390 2024-05-08T18:06:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.784+0000 7f45dffff640 1 -- 172.21.15.50:0/2313229737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ec092d30 con 0x55d1f5592390 2024-05-08T18:06:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.784+0000 7f46fc719800 1 -- 172.21.15.50:0/2313229737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1f5593c60 con 0x55d1f5592390 2024-05-08T18:06:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.785+0000 7f45dffff640 1 -- 172.21.15.50:0/2313229737 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f46ec092260 con 0x55d1f5592390 2024-05-08T18:06:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.785+0000 7f45dffff640 1 --2- 172.21.15.50:0/2313229737 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f45d403bba0 0x7f45d403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.785+0000 7f45dffff640 1 -- 172.21.15.50:0/2313229737 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f46ec089070 con 0x55d1f5592390 2024-05-08T18:06:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.785+0000 7f46f9a10640 1 --2- 172.21.15.50:0/2313229737 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f45d403bba0 0x7f45d403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.785+0000 7f46fc719800 1 Processor -- start 2024-05-08T18:06:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.786+0000 7f46fc719800 1 -- start start 2024-05-08T18:06:03.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.786+0000 7f46f9a10640 1 --2- 172.21.15.50:0/2313229737 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f45d403bba0 0x7f45d403e060 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55d1f53ed420 tx=0x7f46f000f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.786+0000 7f46fc719800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f559f480 0x55d1f559f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.786+0000 7f46fc719800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d1f536a330 con 0x55d1f559f480 2024-05-08T18:06:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.786+0000 7f46fa211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f559f480 0x55d1f559f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.786+0000 7f46fa211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f559f480 0x55d1f559f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:49000/0 (socket says 172.21.15.50:49000) 2024-05-08T18:06:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.786+0000 7f46fa211640 1 -- 172.21.15.50:0/3157232630 learned_addr learned my addr 172.21.15.50:0/3157232630 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.786+0000 7f46fa211640 1 -- 172.21.15.50:0/3157232630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1f559fe70 con 0x55d1f559f480 2024-05-08T18:06:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.786+0000 7f46fa211640 1 --2- 172.21.15.50:0/3157232630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f559f480 0x55d1f559f8a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f46ec05b500 tx=0x7f46ec08ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744064df0e3d04f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.786+0000 7f45ddffb640 1 -- 172.21.15.50:0/3157232630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ec06ecd0 con 0x55d1f559f480 2024-05-08T18:06:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.786+0000 7f45ddffb640 1 -- 172.21.15.50:0/3157232630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec078d10 con 0x55d1f559f480 2024-05-08T18:06:03.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.786+0000 7f45ddffb640 1 -- 172.21.15.50:0/3157232630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ec06e400 con 0x55d1f559f480 2024-05-08T18:06:03.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.787+0000 7f46fc719800 1 -- 172.21.15.50:0/3157232630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f559f480 msgr2=0x55d1f559f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.787+0000 7f46fc719800 1 --2- 172.21.15.50:0/3157232630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f559f480 0x55d1f559f8a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f46ec05b500 tx=0x7f46ec08ffa0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.787+0000 7f46fc719800 1 -- 172.21.15.50:0/3157232630 shutdown_connections 2024-05-08T18:06:03.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.787+0000 7f46fc719800 1 --2- 172.21.15.50:0/3157232630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f559f480 0x55d1f559f8a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.787+0000 7f46fc719800 1 -- 172.21.15.50:0/3157232630 >> 172.21.15.50:0/3157232630 conn(0x55d1f559b7e0 msgr2=0x55d1f5599840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.787+0000 7f46fc719800 1 -- 172.21.15.50:0/3157232630 shutdown_connections 2024-05-08T18:06:03.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.787+0000 7f46fc719800 1 -- 172.21.15.50:0/3157232630 wait complete. 2024-05-08T18:06:03.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.787+0000 7f46fc719800 1 Processor -- start 2024-05-08T18:06:03.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.787+0000 7f46fc719800 1 -- start start 2024-05-08T18:06:03.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.787+0000 7f46fc719800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f559f480 0x55d1f55a6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.787+0000 7f46fc719800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d1f55a03a0 con 0x55d1f559f480 2024-05-08T18:06:03.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.787+0000 7f46fa211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f559f480 0x55d1f55a6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.788+0000 7f46fa211640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f559f480 0x55d1f55a6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:49004/0 (socket says 172.21.15.50:49004) 2024-05-08T18:06:03.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.788+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 learned_addr learned my addr 172.21.15.50:0/744285349 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.788+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1f55a6c40 con 0x55d1f559f480 2024-05-08T18:06:03.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.788+0000 7f46fa211640 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f559f480 0x55d1f55a6700 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f46ec05b910 tx=0x7f46ec072fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.788+0000 7f45cbfff640 1 -- 172.21.15.50:0/744285349 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ec0b8070 con 0x55d1f559f480 2024-05-08T18:06:03.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.788+0000 7f45cbfff640 1 -- 172.21.15.50:0/744285349 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec05be40 con 0x55d1f559f480 2024-05-08T18:06:03.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.788+0000 7f45cbfff640 1 -- 172.21.15.50:0/744285349 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ec06eaf0 con 0x55d1f559f480 2024-05-08T18:06:03.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.788+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d1f55a6c40 con 0x55d1f559f480 2024-05-08T18:06:03.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.788+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1f55a6c40 con 0x55d1f559f480 2024-05-08T18:06:03.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.789+0000 7f45cbfff640 1 -- 172.21.15.50:0/744285349 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f46ec0bb070 con 0x55d1f559f480 2024-05-08T18:06:03.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.789+0000 7f45cbfff640 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f45c403bb50 0x7f45c403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.789+0000 7f45cbfff640 1 -- 172.21.15.50:0/744285349 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f46ec0bed60 con 0x55d1f559f480 2024-05-08T18:06:03.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.789+0000 7f46f9a10640 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f45c403bb50 0x7f45c403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.790+0000 7f46f9a10640 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f45c403bb50 0x7f45c403e010 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55d1f5584440 tx=0x7f46f0010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.790+0000 7f46fc719800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.790+0000 7f46fc719800 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d1f55b1eb0 0x55d1f55b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.790+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b4ac0 con 0x55d1f55b1eb0 2024-05-08T18:06:03.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.790+0000 7f46faa12640 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d1f55b1eb0 0x55d1f55b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.790+0000 7f46faa12640 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d1f55b1eb0 0x55d1f55b4370 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f46f400a4a0 con 0x55d1f55b1eb0 2024-05-08T18:06:03.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46fc719800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46fc719800 20 realm 2024-05-08T18:06:03.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46fc719800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b4ed0 con 0x55d1f55b1eb0 2024-05-08T18:06:03.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f46f400a4a0 con 0x55d1f55b1eb0 2024-05-08T18:06:03.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46fc719800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46fc719800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:03.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46fc719800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b57a0 con 0x55d1f55b1eb0 2024-05-08T18:06:03.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f46f400a4a0 con 0x55d1f55b1eb0 2024-05-08T18:06:03.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46fc719800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46fc719800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b1eb0 2024-05-08T18:06:03.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f46f400a4a0 con 0x55d1f55b1eb0 2024-05-08T18:06:03.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46fc719800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:03.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46fc719800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46fc719800 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d1f55b68a0 0x55d1f55b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.791+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b9250 con 0x55d1f55b68a0 2024-05-08T18:06:03.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.792+0000 7f46fa211640 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d1f55b68a0 0x55d1f55b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.792+0000 7f46fa211640 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d1f55b68a0 0x55d1f55b8c90 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.792+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.792+0000 7f46fc719800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:03.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.792+0000 7f46fc719800 20 searching for the correct realm 2024-05-08T18:06:03.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.792+0000 7f46fc719800 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d1f55bb200 0x55d1f55bd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.792+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55bdd90 con 0x55d1f55bb200 2024-05-08T18:06:03.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.792+0000 7f46f9a10640 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d1f55bb200 0x55d1f55bd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.793+0000 7f46f9a10640 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d1f55bb200 0x55d1f55bd6c0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.793+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f0037040 con 0x55d1f55bb200 2024-05-08T18:06:03.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.793+0000 7f46f9a10640 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f46f000f070 0x7f46f002e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.793+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55bdd90 con 0x7f46f000f070 2024-05-08T18:06:03.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.793+0000 7f46faa12640 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f46f000f070 0x7f46f002e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.793+0000 7f46faa12640 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f46f000f070 0x7f46f002e910 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.794+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f400a4a0 con 0x7f46f000f070 2024-05-08T18:06:03.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.794+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b68a0 2024-05-08T18:06:03.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.794+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.794+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b1eb0 2024-05-08T18:06:03.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.794+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f400a4a0 con 0x55d1f55b1eb0 2024-05-08T18:06:03.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.794+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b1eb0 2024-05-08T18:06:03.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.794+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f400a4a0 con 0x55d1f55b1eb0 2024-05-08T18:06:03.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.794+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b1eb0 2024-05-08T18:06:03.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.795+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f400a4a0 con 0x55d1f55b1eb0 2024-05-08T18:06:03.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.795+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b68a0 2024-05-08T18:06:03.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.795+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.795+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x7f46f000f070 2024-05-08T18:06:03.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.795+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f400a4a0 con 0x7f46f000f070 2024-05-08T18:06:03.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.795+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55bb200 2024-05-08T18:06:03.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.795+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f0037040 con 0x55d1f55bb200 2024-05-08T18:06:03.806 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.795+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b1eb0 2024-05-08T18:06:03.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.796+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f400a4a0 con 0x55d1f55b1eb0 2024-05-08T18:06:03.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.796+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b68a0 2024-05-08T18:06:03.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.796+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.796+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b1eb0 2024-05-08T18:06:03.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.796+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f400a4a0 con 0x55d1f55b1eb0 2024-05-08T18:06:03.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.796+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x7f46f000f070 2024-05-08T18:06:03.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.796+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f46f4032040 con 0x7f46f000f070 2024-05-08T18:06:03.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.796+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55bb200 2024-05-08T18:06:03.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.797+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f0037040 con 0x55d1f55bb200 2024-05-08T18:06:03.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.797+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55bb200 2024-05-08T18:06:03.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.797+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f0037040 con 0x55d1f55bb200 2024-05-08T18:06:03.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.797+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b68a0 2024-05-08T18:06:03.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.797+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.807 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.797+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55bb200 2024-05-08T18:06:03.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.797+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f0037040 con 0x55d1f55bb200 2024-05-08T18:06:03.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.797+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55bb200 2024-05-08T18:06:03.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.797+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f46f0037040 con 0x55d1f55bb200 2024-05-08T18:06:03.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.797+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b1eb0 2024-05-08T18:06:03.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.798+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.798+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b1eb0 2024-05-08T18:06:03.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.798+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.798+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b68a0 2024-05-08T18:06:03.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.798+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.798+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55bb200 2024-05-08T18:06:03.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.798+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f0037040 con 0x55d1f55bb200 2024-05-08T18:06:03.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.798+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x7f46f000f070 2024-05-08T18:06:03.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.799+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f4032040 con 0x7f46f000f070 2024-05-08T18:06:03.808 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.799+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b1eb0 2024-05-08T18:06:03.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.799+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.799+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b68a0 2024-05-08T18:06:03.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.799+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.799+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55bb200 2024-05-08T18:06:03.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.799+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f0037040 con 0x55d1f55bb200 2024-05-08T18:06:03.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.799+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b68a0 2024-05-08T18:06:03.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.800+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.800+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x7f46f000f070 2024-05-08T18:06:03.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.800+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f4032040 con 0x7f46f000f070 2024-05-08T18:06:03.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.800+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55bb200 2024-05-08T18:06:03.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.800+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f0037040 con 0x55d1f55bb200 2024-05-08T18:06:03.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.800+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b68a0 2024-05-08T18:06:03.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.800+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.800+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x55d1f55b1eb0 2024-05-08T18:06:03.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b5b40 con 0x7f46f000f070 2024-05-08T18:06:03.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f46f4032040 con 0x7f46f000f070 2024-05-08T18:06:03.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46fc719800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:03.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46fc719800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:03.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46fc719800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:03.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46fc719800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:03.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46fc719800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:03.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46fc719800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:03.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46fc719800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b9b10 con 0x55d1f55b1eb0 2024-05-08T18:06:03.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46fc719800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46fc719800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b9b10 con 0x55d1f55b1eb0 2024-05-08T18:06:03.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46fc719800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:03.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46fc719800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.801+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55b9b10 con 0x55d1f55bb200 2024-05-08T18:06:03.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f46f0037040 con 0x55d1f55bb200 2024-05-08T18:06:03.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46fc719800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:03.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46fc719800 20 zone default found 2024-05-08T18:06:03.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46fc719800 4 Realm: () 2024-05-08T18:06:03.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46fc719800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:03.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46fc719800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:03.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46fc719800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:03.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46fc719800 20 zonegroup default 2024-05-08T18:06:03.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46fc719800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55ba480 con 0x55d1f55b68a0 2024-05-08T18:06:03.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46fc719800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46fc719800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55bfc80 con 0x55d1f55bb200 2024-05-08T18:06:03.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f46f0037040 con 0x55d1f55bb200 2024-05-08T18:06:03.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46fc719800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46fc719800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.802+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c0110 con 0x55d1f55bb200 2024-05-08T18:06:03.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.803+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f46f0037040 con 0x55d1f55bb200 2024-05-08T18:06:03.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.803+0000 7f46fc719800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.803+0000 7f46fc719800 20 started sync module instance, tier type = 2024-05-08T18:06:03.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.803+0000 7f46fc719800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:03.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.803+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55bf090 con 0x7f46f000f070 2024-05-08T18:06:03.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.804+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x7f46f000f070 2024-05-08T18:06:03.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.804+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94360252970416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55bf090 con 0x7f46f000f070 2024-05-08T18:06:03.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.804+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c0b90 con 0x55d1f55b1eb0 2024-05-08T18:06:03.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.805+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94360252970416] v29'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x7f46f000f070 2024-05-08T18:06:03.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.805+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.805+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94360252975296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55bf090 con 0x55d1f55b1eb0 2024-05-08T18:06:03.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.805+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c15e0 con 0x55d1f55bb200 2024-05-08T18:06:03.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.806+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94360252975296] v29'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.806+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f0037040 con 0x55d1f55bb200 2024-05-08T18:06:03.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.806+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94360252967904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55bf000 con 0x55d1f55bb200 2024-05-08T18:06:03.820 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.806+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c24d0 con 0x7f46f000f070 2024-05-08T18:06:03.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.807+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94360252967904] v29'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f0037040 con 0x55d1f55bb200 2024-05-08T18:06:03.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.807+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x7f46f000f070 2024-05-08T18:06:03.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.807+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94360252981280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c2440 con 0x7f46f000f070 2024-05-08T18:06:03.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.807+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c3390 con 0x55d1f55b1eb0 2024-05-08T18:06:03.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.808+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.808+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94360252985056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c37b0 con 0x55d1f55b1eb0 2024-05-08T18:06:03.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.809+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c42a0 con 0x55d1f55b68a0 2024-05-08T18:06:03.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.809+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94360252981280] v29'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x7f46f000f070 2024-05-08T18:06:03.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.810+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.810+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94360252988784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c4690 con 0x55d1f55b68a0 2024-05-08T18:06:03.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.810+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94360252985056] v29'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.810+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c50b0 con 0x55d1f55b68a0 2024-05-08T18:06:03.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.811+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94360252988784] v29'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.811+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.811+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94360252992416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c28c0 con 0x55d1f55b68a0 2024-05-08T18:06:03.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.811+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c5690 con 0x55d1f55b68a0 2024-05-08T18:06:03.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.812+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94360252992416] v29'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.812+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.812+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94360252986112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c5690 con 0x55d1f55b68a0 2024-05-08T18:06:03.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.812+0000 7f46fc719800 20 add_watcher() i=0 2024-05-08T18:06:03.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.812+0000 7f46fc719800 20 add_watcher() i=1 2024-05-08T18:06:03.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.812+0000 7f46fc719800 20 add_watcher() i=2 2024-05-08T18:06:03.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.812+0000 7f46fc719800 20 add_watcher() i=3 2024-05-08T18:06:03.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.812+0000 7f46fc719800 20 add_watcher() i=4 2024-05-08T18:06:03.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.812+0000 7f46fc719800 20 add_watcher() i=5 2024-05-08T18:06:03.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.812+0000 7f46fc719800 20 add_watcher() i=6 2024-05-08T18:06:03.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.813+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94360252986112] v29'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.813+0000 7f46fc719800 20 add_watcher() i=7 2024-05-08T18:06:03.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.813+0000 7f46fc719800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:03.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.813+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c8960 con 0x55d1f55b1eb0 2024-05-08T18:06:03.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.813+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.813+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94360252994384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c5c20 con 0x55d1f55b1eb0 2024-05-08T18:06:03.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.814+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94360252994384] v29'210 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.814+0000 7f45b0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:03.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.814+0000 7f46fc719800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:03.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.814+0000 7f46fc719800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:03.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.814+0000 7f46fc719800 5 note: GC not initialized 2024-05-08T18:06:03.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.815+0000 7f452b7fe640 20 reqs_thread_entry: start 2024-05-08T18:06:03.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.815+0000 7f46fc719800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:03.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.815+0000 7f46fc719800 20 Filter name: none 2024-05-08T18:06:03.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.815+0000 7f45297fa640 20 reqs_thread_entry: start 2024-05-08T18:06:03.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.815+0000 7f46fc719800 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-05-08T18:06:03.823 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.815+0000 7f46fc719800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.815+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55fbf80 con 0x55d1f55b68a0 2024-05-08T18:06:03.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.815+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(62 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.815+0000 7f46fc719800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.815+0000 7f46fc719800 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-05-08T18:06:03.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.815+0000 7f46fc719800 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-05-08T18:06:03.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.816+0000 7f46fc719800 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2024-05-08T18:06:03.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.816+0000 7f46fc719800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.816+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55fa720 con 0x55d1f55b1eb0 2024-05-08T18:06:03.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.816+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 NX5QOQKC6BH2IDN8HC7A [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.816+0000 7f46fc719800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.816+0000 7f46fc719800 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x0 2024-05-08T18:06:03.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.816+0000 7f46fc719800 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-05-08T18:06:03.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.816+0000 7f46fc719800 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-05-08T18:06:03.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.816+0000 7f46fc719800 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-05-08T18:06:03.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.816+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:64 8.e 8:718f6a54:users.uid::bar:head [delete,create,call version.set in=58b,writefull 0~380 in=380b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55fb010 con 0x55d1f55b68a0 2024-05-08T18:06:03.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.817+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(64 bar [delete,create,call,writefull 0~380] v29'3 uv3 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.817+0000 7f46fc719800 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-05-08T18:06:03.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.817+0000 7f46fc719800 10 moving default.rgw.meta+users.uid+bar to cache LRU end 2024-05-08T18:06:03.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.817+0000 7f46fc719800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-05-08T18:06:03.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.817+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:65 7.9 7:93e5b521:::notify.7:head [notify cookie 94360253213536 in=568b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55ff570 con 0x55d1f55b68a0 2024-05-08T18:06:03.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.818+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== watch-notify(notify (1) cookie 94360252986112 notify 124554051615 ret 0) ==== 598+0+0 (crc 0 0 0) 0x55d1f55fb9d0 con 0x55d1f55b68a0 2024-05-08T18:06:03.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.818+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(65 notify.7 [notify cookie 94360253213536 out=8b] v0'0 uv210 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.818+0000 7f45ddffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051615 cookie 94360252986112 notifier 4718 bl.length()=556 2024-05-08T18:06:03.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.818+0000 7f45ddffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-05-08T18:06:03.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.818+0000 7f45ddffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar to cache LRU end 2024-05-08T18:06:03.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.818+0000 7f45ddffb640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:66 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f45c0003ea0 con 0x55d1f55b68a0 2024-05-08T18:06:03.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.818+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(66 notify.7 [notify-ack] v0'0 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.818+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== watch-notify(notify_complete (2) cookie 94360253213536 notify 124554051615 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d1f5592cf0 con 0x55d1f55b68a0 2024-05-08T18:06:03.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.818+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55fb010 con 0x55d1f55b1eb0 2024-05-08T18:06:03.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.819+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(67 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v29'1 uv1 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.819+0000 7f46fc719800 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-05-08T18:06:03.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.819+0000 7f46fc719800 10 moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-05-08T18:06:03.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.819+0000 7f46fc719800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-05-08T18:06:03.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.819+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify cookie 94360253213856 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55ff9a0 con 0x55d1f55b68a0 2024-05-08T18:06:03.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.819+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== watch-notify(notify (1) cookie 94360252988784 notify 124554051616 ret 0) ==== 219+0+0 (crc 0 0 0) 0x55d1f5592cf0 con 0x55d1f55b68a0 2024-05-08T18:06:03.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.819+0000 7f45ddffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051616 cookie 94360252988784 notifier 4718 bl.length()=177 2024-05-08T18:06:03.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.819+0000 7f45ddffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-05-08T18:06:03.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.819+0000 7f45ddffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-05-08T18:06:03.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.819+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(68 notify.5 [notify cookie 94360253213856 out=8b] v0'0 uv210 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.819+0000 7f45ddffb640 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f45c0003ea0 con 0x55d1f55b68a0 2024-05-08T18:06:03.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.820+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== watch-notify(notify_complete (2) cookie 94360253213856 notify 124554051616 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d1f53ecdc0 con 0x55d1f55b68a0 2024-05-08T18:06:03.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.820+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(69 notify.5 [notify-ack] v0'0 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.827 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"bar","display_name":"Bar","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"bar","access_key":"NX5QOQKC6BH2IDN8HC7A","secret_key":"LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn","active":true,"create_date":"2024-05-08T18:06:03.816982Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:06:03.816968Z","tags":[],"group_ids":[]} 2024-05-08T18:06:03.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.821+0000 7f45b0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:03.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.821+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94360252994384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c8000 con 0x55d1f55b1eb0 2024-05-08T18:06:03.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.822+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94360252994384] v29'211 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.832 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.822+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94360252970416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c5350 con 0x7f46f000f070 2024-05-08T18:06:03.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.823+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94360252970416] v29'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x7f46f000f070 2024-05-08T18:06:03.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.823+0000 7f46fc719800 20 remove_watcher() i=0 2024-05-08T18:06:03.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.823+0000 7f46fc719800 2 removed watcher, disabling cache 2024-05-08T18:06:03.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.823+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94360252975296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c5350 con 0x55d1f55b1eb0 2024-05-08T18:06:03.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.824+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94360252975296] v29'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.824+0000 7f46fc719800 20 remove_watcher() i=1 2024-05-08T18:06:03.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.824+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94360252967904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c5350 con 0x55d1f55bb200 2024-05-08T18:06:03.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.825+0000 7f46f9a10640 1 -- 172.21.15.50:0/744285349 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94360252967904] v29'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f0037040 con 0x55d1f55bb200 2024-05-08T18:06:03.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.825+0000 7f46fc719800 20 remove_watcher() i=2 2024-05-08T18:06:03.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.825+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94360252981280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55bebe0 con 0x7f46f000f070 2024-05-08T18:06:03.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.826+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94360252981280] v29'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x7f46f000f070 2024-05-08T18:06:03.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.826+0000 7f46fc719800 20 remove_watcher() i=3 2024-05-08T18:06:03.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.826+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94360252985056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55bebe0 con 0x55d1f55b1eb0 2024-05-08T18:06:03.833 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.827+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94360252985056] v29'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46f4032040 con 0x55d1f55b1eb0 2024-05-08T18:06:03.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.827+0000 7f46fc719800 20 remove_watcher() i=4 2024-05-08T18:06:03.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.827+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94360252988784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c2ee0 con 0x55d1f55b68a0 2024-05-08T18:06:03.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.828+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94360252988784] v29'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.828+0000 7f46fc719800 20 remove_watcher() i=5 2024-05-08T18:06:03.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.828+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94360252992416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55bebe0 con 0x55d1f55b68a0 2024-05-08T18:06:03.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.829+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94360252992416] v29'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.829+0000 7f46fc719800 20 remove_watcher() i=6 2024-05-08T18:06:03.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.829+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94360252986112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d1f55c2ee0 con 0x55d1f55b68a0 2024-05-08T18:06:03.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.830+0000 7f46fa211640 1 -- 172.21.15.50:0/744285349 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94360252986112] v29'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f46ec0ba030 con 0x55d1f55b68a0 2024-05-08T18:06:03.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.830+0000 7f46fc719800 20 remove_watcher() i=7 2024-05-08T18:06:03.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d1f55b68a0 msgr2=0x55d1f55b8c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d1f55b68a0 0x55d1f55b8c90 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d1f55b1eb0 msgr2=0x55d1f55b4370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.834 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d1f55b1eb0 0x55d1f55b4370 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f46f000f070 msgr2=0x7f46f002e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f46f000f070 0x7f46f002e910 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d1f55bb200 msgr2=0x55d1f55bd6c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d1f55bb200 0x55d1f55bd6c0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f45c403bb50 msgr2=0x7f45c403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f45c403bb50 0x7f45c403e010 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55d1f5584440 tx=0x7f46f0010000 comp rx=0 tx=0).stop 2024-05-08T18:06:03.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f559f480 msgr2=0x55d1f55a6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f559f480 0x55d1f55a6700 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f46ec05b910 tx=0x7f46ec072fa0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46faa12640 1 -- 172.21.15.50:0/744285349 reap_dead start 2024-05-08T18:06:03.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 shutdown_connections 2024-05-08T18:06:03.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 --2- 172.21.15.50:0/744285349 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f559f480 0x55d1f55a6700 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 >> 172.21.15.50:0/744285349 conn(0x55d1f559b7e0 msgr2=0x55d1f5598e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 shutdown_connections 2024-05-08T18:06:03.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 -- 172.21.15.50:0/744285349 wait complete. 2024-05-08T18:06:03.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 -- 172.21.15.50:0/2313229737 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f45d403bba0 msgr2=0x7f45d403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 --2- 172.21.15.50:0/2313229737 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f45d403bba0 0x7f45d403e060 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55d1f53ed420 tx=0x7f46f000f000 comp rx=0 tx=0).stop 2024-05-08T18:06:03.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 -- 172.21.15.50:0/2313229737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5592390 msgr2=0x55d1f55927b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 --2- 172.21.15.50:0/2313229737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5592390 0x55d1f55927b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f46ec05ca90 tx=0x7f46ec076440 comp rx=0 tx=0).stop 2024-05-08T18:06:03.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 -- 172.21.15.50:0/2313229737 shutdown_connections 2024-05-08T18:06:03.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 --2- 172.21.15.50:0/2313229737 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f45d403bba0 0x7f45d403e060 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 --2- 172.21.15.50:0/2313229737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5592390 0x55d1f55927b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.831+0000 7f46fc719800 1 -- 172.21.15.50:0/2313229737 >> 172.21.15.50:0/2313229737 conn(0x55d1f5586430 msgr2=0x55d1f5586830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46fc719800 1 -- 172.21.15.50:0/2313229737 shutdown_connections 2024-05-08T18:06:03.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46fc719800 1 -- 172.21.15.50:0/2313229737 wait complete. 2024-05-08T18:06:03.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46fc719800 1 -- 172.21.15.50:0/3078458260 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d1f557d4a0 msgr2=0x55d1f557f940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46fc719800 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d1f557d4a0 0x55d1f557f940 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46fc719800 1 -- 172.21.15.50:0/3078458260 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d1f54798f0 msgr2=0x55d1f557ab90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46fc719800 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d1f54798f0 0x55d1f557ab90 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46fc719800 1 -- 172.21.15.50:0/3078458260 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d1f5581530 msgr2=0x55d1f5583920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46fc719800 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d1f5581530 0x55d1f5583920 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46fc719800 1 -- 172.21.15.50:0/3078458260 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f45e003bb50 msgr2=0x7f45e003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46fc719800 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f45e003bb50 0x7f45e003e010 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f46f0001e30 tx=0x7f46f0001390 comp rx=0 tx=0).stop 2024-05-08T18:06:03.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46fc719800 1 -- 172.21.15.50:0/3078458260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 msgr2=0x55d1f53edf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46fc719800 1 --2- 172.21.15.50:0/3078458260 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d1f5478730 0x55d1f53edf20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0524f0 tx=0x7f46ec003020 comp rx=0 tx=0).stop 2024-05-08T18:06:03.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46faa12640 1 -- 172.21.15.50:0/3078458260 reap_dead start 2024-05-08T18:06:03.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46fc719800 1 -- 172.21.15.50:0/3078458260 shutdown_connections 2024-05-08T18:06:03.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46fc719800 1 -- 172.21.15.50:0/3078458260 >> 172.21.15.50:0/3078458260 conn(0x55d1f5473cb0 msgr2=0x55d1f5563e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46fc719800 1 -- 172.21.15.50:0/3078458260 shutdown_connections 2024-05-08T18:06:03.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.832+0000 7f46fc719800 1 -- 172.21.15.50:0/3078458260 wait complete. 2024-05-08T18:06:03.842 DEBUG:tasks.util.rgw: json result: {'user_id': 'bar', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bar', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', 'active': True, 'create_date': '2024-05-08T18:06:03.816982Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:03.816968Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:03.854 INFO:tasks.radosgw_admin:RL: create_bucket myfoo6 bar bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:06:03.859 INFO:tasks.radosgw_admin:RL: put_obj myfoo6 bar bytes_out=8 bytes_in=0 failed=False 2024-05-08T18:06:03.859 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2024-05-08T18:06:03.859 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2024-05-08T18:06:03.859 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bar --new-uid bud 2024-05-08T18:06:03.906 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:03.906 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.909+0000 7f9c26b25800 1 Processor -- start 2024-05-08T18:06:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.909+0000 7f9c26b25800 1 -- start start 2024-05-08T18:06:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.909+0000 7f9c26b25800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5596102f4710 0x5596102f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.909+0000 7f9c26b25800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5596102f51b0 con 0x5596102f58d0 2024-05-08T18:06:03.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.909+0000 7f9c26b25800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x559610194e20 con 0x5596102f4710 2024-05-08T18:06:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.910+0000 7f9c22852640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5596102f4710 0x5596102f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.910+0000 7f9c22852640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5596102f4710 0x5596102f4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:49014/0 (socket says 172.21.15.50:49014) 2024-05-08T18:06:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.910+0000 7f9c22852640 1 -- 172.21.15.50:0/3565075470 learned_addr learned my addr 172.21.15.50:0/3565075470 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.910+0000 7f9c22051640 1 --1- 172.21.15.50:0/3565075470 >> v1:172.21.15.46:6789/0 conn(0x5596102f58d0 0x5596102f5d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:54576/0 (socket says 172.21.15.50:54576) 2024-05-08T18:06:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.910+0000 7f9c22852640 1 -- 172.21.15.50:0/3565075470 >> v1:172.21.15.46:6789/0 conn(0x5596102f58d0 legacy=0x5596102f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.910+0000 7f9c22852640 1 -- 172.21.15.50:0/3565075470 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596102f6490 con 0x5596102f4710 2024-05-08T18:06:03.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.912+0000 7f9c22852640 1 --2- 172.21.15.50:0/3565075470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 0x5596102f4b30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400a520 tx=0x7f9c1401d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca80dfab7bb8ca48 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.912+0000 7f9c21850640 1 -- 172.21.15.50:0/3565075470 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c14057070 con 0x5596102f4710 2024-05-08T18:06:03.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.912+0000 7f9c21850640 1 -- 172.21.15.50:0/3565075470 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c14052570 con 0x5596102f4710 2024-05-08T18:06:03.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.912+0000 7f9c21850640 1 -- 172.21.15.50:0/3565075470 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c14052870 con 0x5596102f4710 2024-05-08T18:06:03.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.912+0000 7f9c26b25800 1 -- 172.21.15.50:0/3565075470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 msgr2=0x5596102f4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.912+0000 7f9c26b25800 1 --2- 172.21.15.50:0/3565075470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 0x5596102f4b30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400a520 tx=0x7f9c1401d010 comp rx=0 tx=0).stop 2024-05-08T18:06:03.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.913+0000 7f9c26b25800 1 -- 172.21.15.50:0/3565075470 shutdown_connections 2024-05-08T18:06:03.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.913+0000 7f9c26b25800 1 --2- 172.21.15.50:0/3565075470 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 0x5596102f4b30 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.913+0000 7f9c26b25800 1 -- 172.21.15.50:0/3565075470 >> 172.21.15.50:0/3565075470 conn(0x5596102efc90 msgr2=0x5596102f20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.913+0000 7f9c26b25800 1 -- 172.21.15.50:0/3565075470 shutdown_connections 2024-05-08T18:06:03.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.913+0000 7f9c26b25800 1 -- 172.21.15.50:0/3565075470 wait complete. 2024-05-08T18:06:03.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.928+0000 7f9c26b25800 1 Processor -- start 2024-05-08T18:06:03.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.928+0000 7f9c26b25800 1 -- start start 2024-05-08T18:06:03.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.928+0000 7f9c26b25800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 0x5596103e5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.928+0000 7f9c26b25800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55961023fa30 con 0x5596102f4710 2024-05-08T18:06:03.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.928+0000 7f9c22852640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 0x5596103e5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.928+0000 7f9c22852640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 0x5596103e5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:49022/0 (socket says 172.21.15.50:49022) 2024-05-08T18:06:03.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.928+0000 7f9c22852640 1 -- 172.21.15.50:0/4133012496 learned_addr learned my addr 172.21.15.50:0/4133012496 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.929+0000 7f9c22852640 1 -- 172.21.15.50:0/4133012496 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596103e6240 con 0x5596102f4710 2024-05-08T18:06:03.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.929+0000 7f9c22852640 1 --2- 172.21.15.50:0/4133012496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 0x5596103e5c70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9c14055040 tx=0x7f9c1404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11696449f6f2fba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.929+0000 7f9b10e09640 1 -- 172.21.15.50:0/4133012496 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c14057040 con 0x5596102f4710 2024-05-08T18:06:03.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.929+0000 7f9b10e09640 1 -- 172.21.15.50:0/4133012496 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c14052870 con 0x5596102f4710 2024-05-08T18:06:03.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.929+0000 7f9b10e09640 1 -- 172.21.15.50:0/4133012496 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c14052b90 con 0x5596102f4710 2024-05-08T18:06:03.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.930+0000 7f9c26b25800 1 -- 172.21.15.50:0/4133012496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 msgr2=0x5596103e5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.930+0000 7f9c26b25800 1 --2- 172.21.15.50:0/4133012496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 0x5596103e5c70 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9c14055040 tx=0x7f9c1404c000 comp rx=0 tx=0).stop 2024-05-08T18:06:03.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.930+0000 7f9c26b25800 1 -- 172.21.15.50:0/4133012496 shutdown_connections 2024-05-08T18:06:03.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.930+0000 7f9c26b25800 1 --2- 172.21.15.50:0/4133012496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 0x5596103e5c70 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.930+0000 7f9c26b25800 1 -- 172.21.15.50:0/4133012496 >> 172.21.15.50:0/4133012496 conn(0x5596102efc90 msgr2=0x5596103e1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.930+0000 7f9c26b25800 1 -- 172.21.15.50:0/4133012496 shutdown_connections 2024-05-08T18:06:03.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.930+0000 7f9c26b25800 1 -- 172.21.15.50:0/4133012496 wait complete. 2024-05-08T18:06:03.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.930+0000 7f9c26b25800 1 Processor -- start 2024-05-08T18:06:03.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.930+0000 7f9c26b25800 1 -- start start 2024-05-08T18:06:03.931 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.930+0000 7f9c26b25800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 0x559610269eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.930+0000 7f9c26b25800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5596102fdda0 con 0x5596102f4710 2024-05-08T18:06:03.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.931+0000 7f9c22852640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 0x559610269eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.931+0000 7f9c22852640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 0x559610269eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:49034/0 (socket says 172.21.15.50:49034) 2024-05-08T18:06:03.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.931+0000 7f9c22852640 1 -- 172.21.15.50:0/3026582948 learned_addr learned my addr 172.21.15.50:0/3026582948 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.931+0000 7f9c22852640 1 -- 172.21.15.50:0/3026582948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55961026a3f0 con 0x5596102f4710 2024-05-08T18:06:03.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.931+0000 7f9c22852640 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 0x559610269eb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9c14052540 tx=0x7f9c14003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.931+0000 7f9b0f606640 1 -- 172.21.15.50:0/3026582948 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c14073070 con 0x5596102f4710 2024-05-08T18:06:03.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.931+0000 7f9b0f606640 1 -- 172.21.15.50:0/3026582948 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c1405d570 con 0x5596102f4710 2024-05-08T18:06:03.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.931+0000 7f9c26b25800 1 -- 172.21.15.50:0/3026582948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5596102685f0 con 0x5596102f4710 2024-05-08T18:06:03.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.931+0000 7f9b0f606640 1 -- 172.21.15.50:0/3026582948 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c14057070 con 0x5596102f4710 2024-05-08T18:06:03.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.931+0000 7f9c26b25800 1 -- 172.21.15.50:0/3026582948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559610268b10 con 0x5596102f4710 2024-05-08T18:06:03.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.932+0000 7f9b0f606640 1 -- 172.21.15.50:0/3026582948 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9c14076070 con 0x5596102f4710 2024-05-08T18:06:03.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.932+0000 7f9b0f606640 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9b0803bba0 0x7f9b0803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.932+0000 7f9b0f606640 1 -- 172.21.15.50:0/3026582948 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f9c14094110 con 0x5596102f4710 2024-05-08T18:06:03.932 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.932+0000 7f9c22051640 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9b0803bba0 0x7f9b0803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.932+0000 7f9c26b25800 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5596102f58d0 0x5596103f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.932+0000 7f9c26b25800 1 -- 172.21.15.50:0/3026582948 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5596103f70b0 con 0x5596102f58d0 2024-05-08T18:06:03.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.933+0000 7f9c23053640 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5596102f58d0 0x5596103f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.933+0000 7f9c22051640 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9b0803bba0 0x7f9b0803e060 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9c18001e30 tx=0x7f9c18001390 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.933+0000 7f9c23053640 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5596102f58d0 0x5596103f6b70 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.933+0000 7f9c23053640 1 -- 172.21.15.50:0/3026582948 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9c1c00a4a0 con 0x5596102f58d0 2024-05-08T18:06:03.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.933+0000 7f9c26b25800 1 -- 172.21.15.50:0/3026582948 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x559610268ff0 con 0x5596102f58d0 2024-05-08T18:06:03.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.933+0000 7f9c23053640 1 -- 172.21.15.50:0/3026582948 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9c1c00a4a0 con 0x5596102f58d0 2024-05-08T18:06:03.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.933+0000 7f9c26b25800 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5596103f9380 0x5596103fb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.933+0000 7f9c26b25800 1 -- 172.21.15.50:0/3026582948 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5596103fbe40 con 0x5596103f9380 2024-05-08T18:06:03.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.934+0000 7f9c22852640 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5596103f9380 0x5596103fb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.934+0000 7f9c22852640 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5596103f9380 0x5596103fb770 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.934+0000 7f9c22852640 1 -- 172.21.15.50:0/3026582948 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f9c14075030 con 0x5596103f9380 2024-05-08T18:06:03.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.934+0000 7f9c26b25800 1 -- 172.21.15.50:0/3026582948 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5596103f8220 con 0x5596102f58d0 2024-05-08T18:06:03.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.934+0000 7f9c23053640 1 -- 172.21.15.50:0/3026582948 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9c1c00a4a0 con 0x5596102f58d0 2024-05-08T18:06:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.934+0000 7f9c26b25800 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5596103fd280 0x5596103ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.935+0000 7f9c26b25800 1 -- 172.21.15.50:0/3026582948 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5596103ffde0 con 0x5596103fd280 2024-05-08T18:06:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.935+0000 7f9c22051640 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5596103fd280 0x5596103ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.935+0000 7f9c22051640 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5596103fd280 0x5596103ff670 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.935+0000 7f9c22051640 1 -- 172.21.15.50:0/3026582948 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9c1800bd50 con 0x5596103fd280 2024-05-08T18:06:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.936+0000 7f9c26b25800 1 Processor -- start 2024-05-08T18:06:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.936+0000 7f9c26b25800 1 -- start start 2024-05-08T18:06:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.936+0000 7f9c26b25800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961040e110 0x55961040e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.936+0000 7f9c26b25800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5596101e6330 con 0x55961040e110 2024-05-08T18:06:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.936+0000 7f9c22852640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961040e110 0x55961040e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.936+0000 7f9c22852640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961040e110 0x55961040e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:49040/0 (socket says 172.21.15.50:49040) 2024-05-08T18:06:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.936+0000 7f9c22852640 1 -- 172.21.15.50:0/1876192180 learned_addr learned my addr 172.21.15.50:0/1876192180 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.937+0000 7f9c22852640 1 -- 172.21.15.50:0/1876192180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55961040ea70 con 0x55961040e110 2024-05-08T18:06:03.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.937+0000 7f9c22852640 1 --2- 172.21.15.50:0/1876192180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961040e110 0x55961040e530 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9c14007000 tx=0x7f9c1408bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.937+0000 7f9b0ce01640 1 -- 172.21.15.50:0/1876192180 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c14076de0 con 0x55961040e110 2024-05-08T18:06:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.937+0000 7f9b0ce01640 1 -- 172.21.15.50:0/1876192180 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c1405cb30 con 0x55961040e110 2024-05-08T18:06:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.937+0000 7f9c26b25800 1 -- 172.21.15.50:0/1876192180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55961040f4e0 con 0x55961040e110 2024-05-08T18:06:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.937+0000 7f9b0ce01640 1 -- 172.21.15.50:0/1876192180 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c1405ce50 con 0x55961040e110 2024-05-08T18:06:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.937+0000 7f9c26b25800 1 -- 172.21.15.50:0/1876192180 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55961040f9e0 con 0x55961040e110 2024-05-08T18:06:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.938+0000 7f9b0ce01640 1 -- 172.21.15.50:0/1876192180 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9c14076360 con 0x55961040e110 2024-05-08T18:06:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.938+0000 7f9b0ce01640 1 --2- 172.21.15.50:0/1876192180 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9afc03bcf0 0x7f9afc03e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.938+0000 7f9b0ce01640 1 -- 172.21.15.50:0/1876192180 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f9c14086350 con 0x55961040e110 2024-05-08T18:06:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.938+0000 7f9c22051640 1 --2- 172.21.15.50:0/1876192180 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9afc03bcf0 0x7f9afc03e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.938+0000 7f9c26b25800 1 Processor -- start 2024-05-08T18:06:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.938+0000 7f9c26b25800 1 -- start start 2024-05-08T18:06:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.938+0000 7f9c22051640 1 --2- 172.21.15.50:0/1876192180 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9afc03bcf0 0x7f9afc03e1b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5596104001e0 tx=0x7f9c1800f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.938+0000 7f9c26b25800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961041b2a0 0x55961041b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.938+0000 7f9c26b25800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5596103e6500 con 0x55961041b2a0 2024-05-08T18:06:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.939+0000 7f9c22852640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961041b2a0 0x55961041b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.939+0000 7f9c22852640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961041b2a0 0x55961041b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:49046/0 (socket says 172.21.15.50:49046) 2024-05-08T18:06:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.939+0000 7f9c22852640 1 -- 172.21.15.50:0/2914228151 learned_addr learned my addr 172.21.15.50:0/2914228151 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.939+0000 7f9c22852640 1 -- 172.21.15.50:0/2914228151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55961041bc90 con 0x55961041b2a0 2024-05-08T18:06:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.939+0000 7f9c22852640 1 --2- 172.21.15.50:0/2914228151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961041b2a0 0x55961041b6c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9c14006800 tx=0x7f9c1406e4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5782e35cc62fb03f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.939+0000 7f9b067fc640 1 -- 172.21.15.50:0/2914228151 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c140a4070 con 0x55961041b2a0 2024-05-08T18:06:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.939+0000 7f9b067fc640 1 -- 172.21.15.50:0/2914228151 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c140773b0 con 0x55961041b2a0 2024-05-08T18:06:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.939+0000 7f9b067fc640 1 -- 172.21.15.50:0/2914228151 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c1408b330 con 0x55961041b2a0 2024-05-08T18:06:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.940+0000 7f9c26b25800 1 -- 172.21.15.50:0/2914228151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961041b2a0 msgr2=0x55961041b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.940+0000 7f9c26b25800 1 --2- 172.21.15.50:0/2914228151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961041b2a0 0x55961041b6c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9c14006800 tx=0x7f9c1406e4c0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.940+0000 7f9c26b25800 1 -- 172.21.15.50:0/2914228151 shutdown_connections 2024-05-08T18:06:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.940+0000 7f9c26b25800 1 --2- 172.21.15.50:0/2914228151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961041b2a0 0x55961041b6c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.940+0000 7f9c26b25800 1 -- 172.21.15.50:0/2914228151 >> 172.21.15.50:0/2914228151 conn(0x559610417570 msgr2=0x5596104155d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:03.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.940+0000 7f9c26b25800 1 -- 172.21.15.50:0/2914228151 shutdown_connections 2024-05-08T18:06:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.940+0000 7f9c26b25800 1 -- 172.21.15.50:0/2914228151 wait complete. 2024-05-08T18:06:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.940+0000 7f9c26b25800 1 Processor -- start 2024-05-08T18:06:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.940+0000 7f9c26b25800 1 -- start start 2024-05-08T18:06:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.940+0000 7f9c26b25800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961041b2a0 0x5596104225e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.940+0000 7f9c26b25800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55961041c2f0 con 0x55961041b2a0 2024-05-08T18:06:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.940+0000 7f9c22852640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961041b2a0 0x5596104225e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.940+0000 7f9c22852640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961041b2a0 0x5596104225e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:49060/0 (socket says 172.21.15.50:49060) 2024-05-08T18:06:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.940+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 learned_addr learned my addr 172.21.15.50:0/1683869150 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.941+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559610422b20 con 0x55961041b2a0 2024-05-08T18:06:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.941+0000 7f9c22852640 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961041b2a0 0x5596104225e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9c14006800 tx=0x7f9c14086080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.941+0000 7f9b04ff9640 1 -- 172.21.15.50:0/1683869150 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c140b9070 con 0x55961041b2a0 2024-05-08T18:06:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.941+0000 7f9b04ff9640 1 -- 172.21.15.50:0/1683869150 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c1406c7b0 con 0x55961041b2a0 2024-05-08T18:06:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.941+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559610422b20 con 0x55961041b2a0 2024-05-08T18:06:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.941+0000 7f9b04ff9640 1 -- 172.21.15.50:0/1683869150 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c1406ea50 con 0x55961041b2a0 2024-05-08T18:06:03.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.941+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559610422fc0 con 0x55961041b2a0 2024-05-08T18:06:03.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.942+0000 7f9b04ff9640 1 -- 172.21.15.50:0/1683869150 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9c140a7070 con 0x55961041b2a0 2024-05-08T18:06:03.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.942+0000 7f9b04ff9640 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9aec03bba0 0x7f9aec03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.942+0000 7f9b04ff9640 1 -- 172.21.15.50:0/1683869150 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f9c140bbb80 con 0x55961041b2a0 2024-05-08T18:06:03.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.942+0000 7f9c22051640 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9aec03bba0 0x7f9aec03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.943+0000 7f9c22051640 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9aec03bba0 0x7f9aec03e060 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800ffd0 tx=0x7f9c18010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.943+0000 7f9c26b25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.943+0000 7f9c26b25800 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55961042dbf0 0x5596104300b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.943+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x559610430800 con 0x55961042dbf0 2024-05-08T18:06:03.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.943+0000 7f9c23053640 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55961042dbf0 0x5596104300b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.943+0000 7f9c23053640 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55961042dbf0 0x5596104300b0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9c1c00a4a0 con 0x55961042dbf0 2024-05-08T18:06:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c26b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c26b25800 20 realm 2024-05-08T18:06:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c26b25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x559610430c10 con 0x55961042dbf0 2024-05-08T18:06:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9c1c00a4a0 con 0x55961042dbf0 2024-05-08T18:06:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c26b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c26b25800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c26b25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5596104314e0 con 0x55961042dbf0 2024-05-08T18:06:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9c1c00a4a0 con 0x55961042dbf0 2024-05-08T18:06:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c26b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c26b25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x55961042dbf0 2024-05-08T18:06:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9c1c00a4a0 con 0x55961042dbf0 2024-05-08T18:06:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c26b25800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c26b25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c26b25800 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5596104325e0 0x5596104349d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.944+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x559610434f90 con 0x5596104325e0 2024-05-08T18:06:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.945+0000 7f9c22852640 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5596104325e0 0x5596104349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.945+0000 7f9c22852640 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5596104325e0 0x5596104349d0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.945+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.945+0000 7f9c26b25800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.945+0000 7f9c26b25800 20 searching for the correct realm 2024-05-08T18:06:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.945+0000 7f9c26b25800 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559610436f20 0x5596104393e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.945+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610439ab0 con 0x559610436f20 2024-05-08T18:06:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.945+0000 7f9c22051640 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559610436f20 0x5596104393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.946+0000 7f9c22051640 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559610436f20 0x5596104393e0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.946+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.946+0000 7f9c22051640 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9c1800f070 0x7f9c1803a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.946+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610439ab0 con 0x7f9c1800f070 2024-05-08T18:06:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.946+0000 7f9c23053640 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9c1800f070 0x7f9c1803a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.946+0000 7f9c23053640 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9c1800f070 0x7f9c1803a3c0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.947+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c1c00a4a0 con 0x7f9c1800f070 2024-05-08T18:06:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.947+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x5596104325e0 2024-05-08T18:06:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.947+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.947+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x55961042dbf0 2024-05-08T18:06:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.947+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c1c00a4a0 con 0x55961042dbf0 2024-05-08T18:06:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.947+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x55961042dbf0 2024-05-08T18:06:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.947+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c1c00a4a0 con 0x55961042dbf0 2024-05-08T18:06:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.947+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x55961042dbf0 2024-05-08T18:06:03.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.948+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c1c00a4a0 con 0x55961042dbf0 2024-05-08T18:06:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.948+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x5596104325e0 2024-05-08T18:06:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.948+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.948+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x7f9c1800f070 2024-05-08T18:06:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.948+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c1c00a4a0 con 0x7f9c1800f070 2024-05-08T18:06:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.948+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x559610436f20 2024-05-08T18:06:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.948+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.948+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x55961042dbf0 2024-05-08T18:06:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.949+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c1c00a4a0 con 0x55961042dbf0 2024-05-08T18:06:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.949+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x5596104325e0 2024-05-08T18:06:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.949+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.949+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x55961042dbf0 2024-05-08T18:06:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.949+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c1c00a4a0 con 0x55961042dbf0 2024-05-08T18:06:03.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.949+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x7f9c1800f070 2024-05-08T18:06:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.949+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.949+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x559610436f20 2024-05-08T18:06:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.950+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.950+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x559610436f20 2024-05-08T18:06:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.950+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.950+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x5596104325e0 2024-05-08T18:06:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.950+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.950+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x559610436f20 2024-05-08T18:06:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.950+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.950+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x559610436f20 2024-05-08T18:06:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.950+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.950+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x55961042dbf0 2024-05-08T18:06:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.951+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.951+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x55961042dbf0 2024-05-08T18:06:03.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.951+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.951+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x5596104325e0 2024-05-08T18:06:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.951+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.951+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x559610436f20 2024-05-08T18:06:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.951+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.951+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x7f9c1800f070 2024-05-08T18:06:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.952+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.952+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x55961042dbf0 2024-05-08T18:06:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.952+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.952+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x5596104325e0 2024-05-08T18:06:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.952+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.952+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x559610436f20 2024-05-08T18:06:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.952+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.952+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x5596104325e0 2024-05-08T18:06:03.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.953+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.953+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x7f9c1800f070 2024-05-08T18:06:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.953+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.953+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x559610436f20 2024-05-08T18:06:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.953+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.953+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x5596104325e0 2024-05-08T18:06:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x55961042dbf0 2024-05-08T18:06:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x559610431880 con 0x7f9c1800f070 2024-05-08T18:06:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c26b25800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c26b25800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c26b25800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c26b25800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:03.965 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c26b25800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:03.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c26b25800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:03.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c26b25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x559610435850 con 0x55961042dbf0 2024-05-08T18:06:03.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c26b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c26b25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.954+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x559610435850 con 0x55961042dbf0 2024-05-08T18:06:03.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c26b25800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:03.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c26b25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x559610435850 con 0x559610436f20 2024-05-08T18:06:03.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c26b25800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:03.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c26b25800 20 zone default found 2024-05-08T18:06:03.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c26b25800 4 Realm: () 2024-05-08T18:06:03.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c26b25800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:03.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c26b25800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:03.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c26b25800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:03.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c26b25800 20 zonegroup default 2024-05-08T18:06:03.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c26b25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x559610436220 con 0x5596104325e0 2024-05-08T18:06:03.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c26b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c26b25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.955+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x559610435dc0 con 0x559610436f20 2024-05-08T18:06:03.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.956+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.956+0000 7f9c26b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.956+0000 7f9c26b25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.956+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961043bec0 con 0x559610436f20 2024-05-08T18:06:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.956+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.956+0000 7f9c26b25800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.956+0000 7f9c26b25800 20 started sync module instance, tier type = 2024-05-08T18:06:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.956+0000 7f9c26b25800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.956+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55961043b250 con 0x7f9c1800f070 2024-05-08T18:06:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.957+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.957+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94103006328656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55961043b250 con 0x7f9c1800f070 2024-05-08T18:06:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.957+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55961043c9f0 con 0x55961042dbf0 2024-05-08T18:06:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.958+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94103006328656] v29'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.959+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.959+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94103006332096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55961043b250 con 0x55961042dbf0 2024-05-08T18:06:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.959+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55961043d4a0 con 0x559610436f20 2024-05-08T18:06:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.959+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94103006332096] v29'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.960+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.960+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94103006326176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55961043b1c0 con 0x559610436f20 2024-05-08T18:06:03.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.960+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55961043e360 con 0x7f9c1800f070 2024-05-08T18:06:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.961+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94103006326176] v29'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.961+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.961+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94103006338736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55961043e2d0 con 0x7f9c1800f070 2024-05-08T18:06:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.961+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55961043f0f0 con 0x55961042dbf0 2024-05-08T18:06:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.962+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.962+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94103006338736] v29'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:03.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.962+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94103006342144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55961043f0f0 con 0x55961042dbf0 2024-05-08T18:06:03.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.962+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55961043e790 con 0x5596104325e0 2024-05-08T18:06:03.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.963+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.963+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94103006346032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610440400 con 0x5596104325e0 2024-05-08T18:06:03.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.963+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610440e20 con 0x5596104325e0 2024-05-08T18:06:03.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.963+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94103006342144] v29'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.964+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94103006346032] v29'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.964+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.964+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94103006349584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610441210 con 0x5596104325e0 2024-05-08T18:06:03.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.964+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610441a60 con 0x5596104325e0 2024-05-08T18:06:03.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.965+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94103006349584] v29'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.965+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.965+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94103006352816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610441e80 con 0x5596104325e0 2024-05-08T18:06:03.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.965+0000 7f9c26b25800 20 add_watcher() i=0 2024-05-08T18:06:03.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.965+0000 7f9c26b25800 20 add_watcher() i=1 2024-05-08T18:06:03.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.965+0000 7f9c26b25800 20 add_watcher() i=2 2024-05-08T18:06:03.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.965+0000 7f9c26b25800 20 add_watcher() i=3 2024-05-08T18:06:03.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.965+0000 7f9c26b25800 20 add_watcher() i=4 2024-05-08T18:06:03.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.965+0000 7f9c26b25800 20 add_watcher() i=5 2024-05-08T18:06:03.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.965+0000 7f9c26b25800 20 add_watcher() i=6 2024-05-08T18:06:03.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.966+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94103006352816] v29'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.966+0000 7f9c26b25800 20 add_watcher() i=7 2024-05-08T18:06:03.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.966+0000 7f9c26b25800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:03.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.966+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x559610445190 con 0x55961042dbf0 2024-05-08T18:06:03.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.967+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.967+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94103006354240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610442410 con 0x55961042dbf0 2024-05-08T18:06:03.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.968+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94103006354240] v29'212 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.968+0000 7f9ad97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:03.981 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.968+0000 7f9c26b25800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:03.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.968+0000 7f9c26b25800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:03.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.968+0000 7f9c26b25800 5 note: GC not initialized 2024-05-08T18:06:03.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.968+0000 7f9a53fff640 20 reqs_thread_entry: start 2024-05-08T18:06:03.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.968+0000 7f9c26b25800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:03.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.968+0000 7f9c26b25800 20 Filter name: none 2024-05-08T18:06:03.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.968+0000 7f9a51ffb640 20 reqs_thread_entry: start 2024-05-08T18:06:03.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.968+0000 7f9c26b25800 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-05-08T18:06:03.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.968+0000 7f9c26b25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.968+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x559610478710 con 0x5596104325e0 2024-05-08T18:06:03.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.969+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(62 bar [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.969+0000 7f9c26b25800 20 rados_obj.operate() r=0 bl.length=380 2024-05-08T18:06:03.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.969+0000 7f9c26b25800 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x13 2024-05-08T18:06:03.982 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.969+0000 7f9c26b25800 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-05-08T18:06:03.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.969+0000 7f9c26b25800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:03.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.969+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:63 8.4 8:267425a8:users.uid::bud:head [create,call version.set in=58b,writefull 0~206 in=206b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55961047cab0 con 0x7f9c1800f070 2024-05-08T18:06:03.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.970+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(63 bud [create,call,writefull 0~206] v29'1 uv1 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:03.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.970+0000 7f9c26b25800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-05-08T18:06:03.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.970+0000 7f9c26b25800 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-05-08T18:06:03.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.970+0000 7f9c26b25800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-05-08T18:06:03.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.970+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:64 7.3 7:c609908c:::notify.5:head [notify cookie 94103006594432 in=394b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047d700 con 0x5596104325e0 2024-05-08T18:06:03.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.971+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== watch-notify(notify (1) cookie 94103006346032 notify 124554051617 ret 0) ==== 424+0+0 (crc 0 0 0) 0x55961040f4e0 con 0x5596104325e0 2024-05-08T18:06:03.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.971+0000 7f9b067fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051617 cookie 94103006346032 notifier 4725 bl.length()=382 2024-05-08T18:06:03.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.971+0000 7f9b067fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-05-08T18:06:03.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.971+0000 7f9b067fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2024-05-08T18:06:03.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.971+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(64 notify.5 [notify cookie 94103006594432 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.971+0000 7f9b067fc640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f9ae8003ea0 con 0x5596104325e0 2024-05-08T18:06:03.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.971+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(65 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.971+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== watch-notify(notify_complete (2) cookie 94103006594432 notify 124554051617 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55961040ea70 con 0x5596104325e0 2024-05-08T18:06:03.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.971+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 8.5 8:a00000ee:users.uid::bar.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047c580 con 0x55961042dbf0 2024-05-08T18:06:03.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.971+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(66 bar.buckets [call out=184b] v0'0 uv3 ondisk = 0) ==== 155+0+184 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.971+0000 7f9c26b25800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 : miss 2024-05-08T18:06:03.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.971+0000 7f9c26b25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:03.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.971+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 8.19 8:98c2a5ec:root::.bucket.meta.myfoo6%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047a8b0 con 0x55961042dbf0 2024-05-08T18:06:03.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.972+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(67 .bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 [call out=48b,stat out=16b,read 0~380 out=380b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 333+0+662 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.972+0000 7f9c26b25800 20 rados_obj.operate() r=0 bl.length=380 2024-05-08T18:06:03.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.972+0000 7f9c26b25800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 info.flags=0x17 2024-05-08T18:06:03.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.972+0000 7f9c26b25800 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 to cache LRU end 2024-05-08T18:06:03.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.972+0000 7f9c26b25800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:03.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.972+0000 7f9c26b25800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:06:03.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.972+0000 7f9c26b25800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 2024-05-08T18:06:03.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.972+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 8.19 8:98c2a5ec:root::.bucket.meta.myfoo6%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~380 in=380b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610484740 con 0x55961042dbf0 2024-05-08T18:06:03.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.973+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 .bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 [delete,create,call,call,writefull 0~380,setxattr (117)] v29'4 uv4 ondisk = 0) ==== 417+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.973+0000 7f9c26b25800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 info.flags=0x17 2024-05-08T18:06:03.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.973+0000 7f9c26b25800 10 moving default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 to cache LRU end 2024-05-08T18:06:03.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.973+0000 7f9c26b25800 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-05-08T18:06:03.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.973+0000 7f9c26b25800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9, ofs0, ns] 2024-05-08T18:06:03.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.973+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [notify cookie 94103006581744 in=760b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047eb40 con 0x55961042dbf0 2024-05-08T18:06:03.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.973+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== watch-notify(notify (1) cookie 94103006332096 notify 124554051609 ret 0) ==== 790+0+0 (crc 0 0 0) 0x55961047aca0 con 0x55961042dbf0 2024-05-08T18:06:03.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.973+0000 7f9b067fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051609 cookie 94103006332096 notifier 4725 bl.length()=748 2024-05-08T18:06:03.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.973+0000 7f9b067fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 info.flags=0x17 2024-05-08T18:06:03.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.973+0000 7f9b067fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 to cache LRU end 2024-05-08T18:06:03.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.973+0000 7f9b067fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2024-05-08T18:06:03.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.973+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(69 notify.1 [notify cookie 94103006581744 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.973+0000 7f9b067fc640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f9ae8003ea0 con 0x55961042dbf0 2024-05-08T18:06:03.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.974+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== watch-notify(notify_complete (2) cookie 94103006581744 notify 124554051609 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55961047d8f0 con 0x55961042dbf0 2024-05-08T18:06:03.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.974+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(70 notify.1 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.974+0000 7f9c26b25800 20 handle_bi_update(): bucket=:myfoo6[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9]): orig_sources= new_sources= 2024-05-08T18:06:03.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.974+0000 7f9c26b25800 20 handle_bi_update(): bucket=:myfoo6[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9]): potential sources added=[] removed=[] 2024-05-08T18:06:03.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.974+0000 7f9c26b25800 20 handle_bi_update(): bucket=:myfoo6[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9]): orig_dests= new_dests= 2024-05-08T18:06:03.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.974+0000 7f9c26b25800 20 handle_bi_update(): bucket=:myfoo6[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9]): potential dests added=[] removed=[] 2024-05-08T18:06:03.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.974+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.remove_bucket in=145b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5596104793f0 con 0x55961042dbf0 2024-05-08T18:06:03.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.975+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(71 bud.buckets [call] v29'5 uv0 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.975+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610483f10 con 0x55961042dbf0 2024-05-08T18:06:03.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.976+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(72 bud.buckets [call] v29'6 uv6 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.976+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:73 8.18 8:1944a257:root::myfoo6:head [delete,create,call version.inc in=38b,writefull 0~166 in=166b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610483f10 con 0x55961042dbf0 2024-05-08T18:06:03.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.977+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(73 myfoo6 [delete,create,call,writefull 0~166] v29'17 uv17 ondisk = 0) ==== 276+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.977+0000 7f9c26b25800 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x7 2024-05-08T18:06:03.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.977+0000 7f9c26b25800 10 adding default.rgw.meta+root+myfoo6 to cache LRU end 2024-05-08T18:06:03.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.977+0000 7f9c26b25800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-05-08T18:06:03.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.977+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [notify cookie 94103006581744 in=328b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x559610483f10 con 0x7f9c1800f070 2024-05-08T18:06:03.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.977+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== watch-notify(notify (1) cookie 94103006338736 notify 124554051599 ret 0) ==== 358+0+0 (crc 0 0 0) 0x55961047aa70 con 0x7f9c1800f070 2024-05-08T18:06:03.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.977+0000 7f9b05ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051599 cookie 94103006338736 notifier 4725 bl.length()=316 2024-05-08T18:06:03.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.977+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(74 notify.3 [notify cookie 94103006581744 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:03.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.977+0000 7f9b05ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x7 2024-05-08T18:06:03.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.977+0000 7f9b05ffb640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo6 to cache LRU end 2024-05-08T18:06:03.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.977+0000 7f9b05ffb640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:75 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f9a58003400 con 0x7f9c1800f070 2024-05-08T18:06:03.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.977+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== watch-notify(notify_complete (2) cookie 94103006581744 notify 124554051599 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55961047d8f0 con 0x7f9c1800f070 2024-05-08T18:06:03.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.977+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(75 notify.3 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:03.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.978+0000 7f9c26b25800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 : hit (requested=0x17, cached=0x17) 2024-05-08T18:06:03.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.978+0000 7f9c26b25800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 2024-05-08T18:06:03.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.978+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 8.19 8:98c2a5ec:root::.bucket.meta.myfoo6%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~380 in=380b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610482350 con 0x55961042dbf0 2024-05-08T18:06:03.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 32 ==== osd_op_reply(76 .bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 [delete,create,call,call,writefull 0~380,setxattr (117)] v29'7 uv7 ondisk = 0) ==== 417+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9c26b25800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 info.flags=0x17 2024-05-08T18:06:03.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9c26b25800 10 moving default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 to cache LRU end 2024-05-08T18:06:03.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9c26b25800 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-05-08T18:06:03.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9c26b25800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9, ofs0, ns] 2024-05-08T18:06:03.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:77 7.0 7:05bf5b68:::notify.1:head [notify cookie 94103006585856 in=760b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047db30 con 0x55961042dbf0 2024-05-08T18:06:03.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 33 ==== watch-notify(notify (1) cookie 94103006332096 notify 124554051610 ret 0) ==== 790+0+0 (crc 0 0 0) 0x55961047d8f0 con 0x55961042dbf0 2024-05-08T18:06:03.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9b05ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051610 cookie 94103006332096 notifier 4725 bl.length()=748 2024-05-08T18:06:03.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9b05ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 info.flags=0x17 2024-05-08T18:06:03.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9b05ffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 to cache LRU end 2024-05-08T18:06:03.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 34 ==== osd_op_reply(77 notify.1 [notify cookie 94103006585856 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9b05ffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2024-05-08T18:06:03.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9b05ffb640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:78 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f9a580034d0 con 0x55961042dbf0 2024-05-08T18:06:03.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 35 ==== osd_op_reply(78 notify.1 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 36 ==== watch-notify(notify_complete (2) cookie 94103006585856 notify 124554051610 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f9c1c03d040 con 0x55961042dbf0 2024-05-08T18:06:03.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9c26b25800 20 handle_bi_update(): bucket=:myfoo6[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9]): orig_sources= new_sources= 2024-05-08T18:06:03.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9c26b25800 20 handle_bi_update(): bucket=:myfoo6[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9]): potential sources added=[] removed=[] 2024-05-08T18:06:03.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9c26b25800 20 handle_bi_update(): bucket=:myfoo6[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9]): orig_dests= new_dests= 2024-05-08T18:06:03.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9c26b25800 20 handle_bi_update(): bucket=:myfoo6[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9]): potential dests added=[] removed=[] 2024-05-08T18:06:03.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.979+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:79 4.1b 4:d9ac9af1:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047a400 con 0x7f9c1800f070 2024-05-08T18:06:03.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.980+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(79 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:03.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.980+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:80 4.13 4:c87f6b5c:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047a400 con 0x7f9c1800f070 2024-05-08T18:06:03.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.980+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(80 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.1 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:03.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.980+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:81 4.3a 4:5f00aa60:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047a400 con 0x559610436f20 2024-05-08T18:06:03.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.980+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(81 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.2 [call out=85b] v0'0 uv4 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.980+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:82 4.12 4:48cbad65:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047db30 con 0x55961042dbf0 2024-05-08T18:06:03.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.981+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 37 ==== osd_op_reply(82 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.3 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.981+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:83 4.2f 4:f4b659df:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047db30 con 0x55961042dbf0 2024-05-08T18:06:03.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.981+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 38 ==== osd_op_reply(83 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.4 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.981+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:84 4.e 4:70417d9c:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047db30 con 0x7f9c1800f070 2024-05-08T18:06:03.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.981+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(84 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.5 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:03.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.981+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 4.25 4:a4bd2273:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047db30 con 0x559610436f20 2024-05-08T18:06:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.982+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(85 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.6 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.982+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:86 4.26 4:652575e1:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047db30 con 0x5596104325e0 2024-05-08T18:06:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.982+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(86 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.7 [call out=85b] v0'0 uv14 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.982+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:87 4.28 4:162ce8db:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047db30 con 0x55961042dbf0 2024-05-08T18:06:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.983+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 39 ==== osd_op_reply(87 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.983+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 4.4 4:215f588e:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047db30 con 0x5596104325e0 2024-05-08T18:06:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.983+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(88 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9 [call out=378b] v0'0 uv3 ondisk = 0) ==== 194+0+378 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.983+0000 7f9c26b25800 10 cls_bucket_list_unordered: got thirteen 2024-05-08T18:06:03.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.983+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 4.39 4:9c486660:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047db30 con 0x559610436f20 2024-05-08T18:06:03.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.983+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(89 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.983+0000 7f9c26b25800 20 get_obj_state: octx=0x55961047c5d0 obj=myfoo6:thirteen state=0x5596104795d8 s->prefetch_data=0 2024-05-08T18:06:03.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.983+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:90 3.7s0 3:e145b1ed:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9_thirteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55961047db30 con 0x559610436f20 2024-05-08T18:06:03.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.984+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(90 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9_thirteen [getxattrs out=856b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 238+0+872 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.984+0000 7f9c26b25800 10 manifest: total_size = 8 2024-05-08T18:06:03.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.984+0000 7f9c26b25800 20 get_obj_state: setting s->obj_tag to 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.13556758686930028009 2024-05-08T18:06:03.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.984+0000 7f9c26b25800 20 Read xattr rgw_rados: user.rgw.acl 2024-05-08T18:06:03.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.984+0000 7f9c26b25800 20 Read xattr rgw_rados: user.rgw.content_type 2024-05-08T18:06:03.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.984+0000 7f9c26b25800 20 Read xattr rgw_rados: user.rgw.etag 2024-05-08T18:06:03.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.984+0000 7f9c26b25800 20 Read xattr rgw_rados: user.rgw.idtag 2024-05-08T18:06:03.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.984+0000 7f9c26b25800 20 Read xattr rgw_rados: user.rgw.manifest 2024-05-08T18:06:03.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.984+0000 7f9c26b25800 20 Read xattr rgw_rados: user.rgw.pg_ver 2024-05-08T18:06:03.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.984+0000 7f9c26b25800 20 Read xattr rgw_rados: user.rgw.source_zone 2024-05-08T18:06:03.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.984+0000 7f9c26b25800 20 Read xattr rgw_rados: user.rgw.tail_tag 2024-05-08T18:06:03.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.984+0000 7f9c26b25800 20 get_obj_state: octx=0x55961047c5d0 obj=myfoo6:thirteen state=0x5596104795d8 s->prefetch_data=0 2024-05-08T18:06:03.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.984+0000 7f9c26b25800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 : hit (requested=0x17, cached=0x17) 2024-05-08T18:06:03.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.984+0000 7f9c26b25800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 2024-05-08T18:06:03.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.984+0000 7f9c26b25800 20 bucket index object: :.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9 2024-05-08T18:06:03.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.984+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:91 4.4 4:215f588e:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610480f00 con 0x5596104325e0 2024-05-08T18:06:03.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.985+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(91 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9 [stat,call,call] v29'4 uv4 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.985+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:92 3.7s0 3:e145b1ed:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9_thirteen:head [cmpxattr user.rgw.idtag (63) op 1 mode 1 in=77b,setxattr user.rgw.acl (117) in=129b,setxattr user.rgw.idtag (33) in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610480180 con 0x559610436f20 2024-05-08T18:06:03.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.986+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(92 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9_thirteen [cmpxattr (63) op 1 mode 1,setxattr (117),setxattr (33)] v29'2 uv2 ondisk = 0) ==== 280+0+0 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:03.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.986+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:93 4.4 4:215f588e:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=267b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610487e10 con 0x5596104325e0 2024-05-08T18:06:03.997 INFO:teuthology.orchestra.run.smithi050.stderr:1 objects processed in :myfoo6[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9]). Next marker thirteen 2024-05-08T18:06:03.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.986+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:94 8.4 8:267425a8:users.uid::bud:head [delete,create,call version.set in=58b,writefull 0~380 in=380b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55961047d130 con 0x7f9c1800f070 2024-05-08T18:06:03.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.987+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(93 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9 [stat,call,call] v29'5 uv5 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.987+0000 7f9b05ffb640 20 handle_completion(): completion ok for obj=thirteen 2024-05-08T18:06:03.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.988+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(94 bud [delete,create,call,writefull 0~380] v29'2 uv2 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:03.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.988+0000 7f9c26b25800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-05-08T18:06:03.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.988+0000 7f9c26b25800 10 moving default.rgw.meta+users.uid+bud to cache LRU end 2024-05-08T18:06:03.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.988+0000 7f9c26b25800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-05-08T18:06:03.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.988+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [notify cookie 94103006624064 in=568b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5596104793f0 con 0x5596104325e0 2024-05-08T18:06:03.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.988+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== watch-notify(notify (1) cookie 94103006346032 notify 124554051618 ret 0) ==== 598+0+0 (crc 0 0 0) 0x55961040ea70 con 0x5596104325e0 2024-05-08T18:06:03.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.988+0000 7f9b067fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051618 cookie 94103006346032 notifier 4725 bl.length()=556 2024-05-08T18:06:03.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.988+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(95 notify.5 [notify cookie 94103006624064 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.988+0000 7f9b067fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-05-08T18:06:03.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.988+0000 7f9b067fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2024-05-08T18:06:03.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.988+0000 7f9b067fc640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:96 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f9ae8003ea0 con 0x5596104325e0 2024-05-08T18:06:03.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.988+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(96 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.988+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== watch-notify(notify_complete (2) cookie 94103006624064 notify 124554051618 ret 0) ==== 42+0+48 (crc 0 0 0) 0x559610268b10 con 0x5596104325e0 2024-05-08T18:06:03.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.988+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:97 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610482f00 con 0x55961042dbf0 2024-05-08T18:06:03.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.989+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 40 ==== osd_op_reply(97 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v29'2 uv2 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:03.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.989+0000 7f9c26b25800 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-05-08T18:06:03.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.989+0000 7f9c26b25800 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-05-08T18:06:03.998 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.989+0000 7f9c26b25800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.989+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:98 7.3 7:c609908c:::notify.5:head [notify cookie 94103006624064 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x559610482f00 con 0x5596104325e0 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.990+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== watch-notify(notify (1) cookie 94103006346032 notify 124554051619 ret 0) ==== 219+0+0 (crc 0 0 0) 0x559610268b10 con 0x5596104325e0 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.990+0000 7f9b067fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051619 cookie 94103006346032 notifier 4725 bl.length()=177 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.990+0000 7f9b067fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.990+0000 7f9b067fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.990+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 31 ==== osd_op_reply(98 notify.5 [notify cookie 94103006624064 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.990+0000 7f9b067fc640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:99 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f9ae8003ea0 con 0x5596104325e0 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.990+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 32 ==== osd_op_reply(99 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.990+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 33 ==== watch-notify(notify_complete (2) cookie 94103006624064 notify 124554051619 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5596102685f0 con 0x5596104325e0 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.990+0000 7f9c26b25800 10 removing user index: bar 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.990+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:100 8.e 8:718f6a54:users.uid::bar:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610482f00 con 0x5596104325e0 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.991+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 34 ==== osd_op_reply(100 bar [delete] v29'4 uv4 ondisk = 0) ==== 147+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.991+0000 7f9c26b25800 10 removing default.rgw.meta+users.uid+bar from cache 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.991+0000 7f9c26b25800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.991+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:101 7.9 7:93e5b521:::notify.7:head [notify cookie 94103006624064 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x559610482f00 con 0x5596104325e0 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.991+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 35 ==== watch-notify(notify (1) cookie 94103006352816 notify 124554051620 ret 0) ==== 194+0+0 (crc 0 0 0) 0x5596102685f0 con 0x5596104325e0 2024-05-08T18:06:03.999 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.991+0000 7f9b067fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051620 cookie 94103006352816 notifier 4725 bl.length()=152 2024-05-08T18:06:04.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.991+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 36 ==== osd_op_reply(101 notify.7 [notify cookie 94103006624064 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:04.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.991+0000 7f9b067fc640 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f9ae8003ea0 con 0x5596104325e0 2024-05-08T18:06:04.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.992+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 37 ==== watch-notify(notify_complete (2) cookie 94103006624064 notify 124554051620 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55961026a3f0 con 0x5596104325e0 2024-05-08T18:06:04.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.992+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 38 ==== osd_op_reply(102 notify.7 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:04.000 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"bud","display_name":"Bar","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"bud","access_key":"NX5QOQKC6BH2IDN8HC7A","secret_key":"LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn","active":true,"create_date":"2024-05-08T18:06:03.816982Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:06:03.816968Z","tags":[],"group_ids":[]} 2024-05-08T18:06:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.993+0000 7f9ad97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.993+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:103 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94103006354240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610482f00 con 0x55961042dbf0 2024-05-08T18:06:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.994+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 41 ==== osd_op_reply(103 data_loggenerations_metadata [watch unwatch cookie 94103006354240] v29'213 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.994+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94103006328656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610482f00 con 0x7f9c1800f070 2024-05-08T18:06:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.995+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 20 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 94103006328656] v29'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.995+0000 7f9c26b25800 20 remove_watcher() i=0 2024-05-08T18:06:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.995+0000 7f9c26b25800 2 removed watcher, disabling cache 2024-05-08T18:06:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.995+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94103006332096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610482f00 con 0x55961042dbf0 2024-05-08T18:06:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.996+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 42 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 94103006332096] v29'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:04.005 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.996+0000 7f9c26b25800 20 remove_watcher() i=1 2024-05-08T18:06:04.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.996+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94103006326176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610482f00 con 0x559610436f20 2024-05-08T18:06:04.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.997+0000 7f9c22051640 1 -- 172.21.15.50:0/1683869150 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 94103006326176] v29'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c18037040 con 0x559610436f20 2024-05-08T18:06:04.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.997+0000 7f9c26b25800 20 remove_watcher() i=2 2024-05-08T18:06:04.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.997+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94103006338736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610482f00 con 0x7f9c1800f070 2024-05-08T18:06:04.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.998+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.2 v2:172.21.15.50:6811/3039646617 21 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 94103006338736] v29'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x7f9c1800f070 2024-05-08T18:06:04.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.998+0000 7f9c26b25800 20 remove_watcher() i=3 2024-05-08T18:06:04.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.998+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94103006342144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610482f00 con 0x55961042dbf0 2024-05-08T18:06:04.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:03.999+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 <== osd.1 v2:172.21.15.50:6808/2141923372 43 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 94103006342144] v29'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c1c032040 con 0x55961042dbf0 2024-05-08T18:06:04.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.000+0000 7f9c26b25800 20 remove_watcher() i=4 2024-05-08T18:06:04.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.000+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94103006346032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610482f00 con 0x5596104325e0 2024-05-08T18:06:04.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.000+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 39 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 94103006346032] v29'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:04.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.001+0000 7f9c26b25800 20 remove_watcher() i=5 2024-05-08T18:06:04.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.001+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94103006349584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610482f00 con 0x5596104325e0 2024-05-08T18:06:04.006 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.001+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 40 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 94103006349584] v29'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:04.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.001+0000 7f9c26b25800 20 remove_watcher() i=6 2024-05-08T18:06:04.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.001+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94103006352816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x559610482f00 con 0x5596104325e0 2024-05-08T18:06:04.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.002+0000 7f9c22852640 1 -- 172.21.15.50:0/1683869150 <== osd.0 v2:172.21.15.46:6800/337143975 41 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 94103006352816] v29'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9c140b2070 con 0x5596104325e0 2024-05-08T18:06:04.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 20 remove_watcher() i=7 2024-05-08T18:06:04.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5596104325e0 msgr2=0x5596104349d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:04.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5596104325e0 0x5596104349d0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:04.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55961042dbf0 msgr2=0x5596104300b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:04.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55961042dbf0 0x5596104300b0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:04.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9c1800f070 msgr2=0x7f9c1803a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:04.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f9c1800f070 0x7f9c1803a3c0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:04.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559610436f20 msgr2=0x5596104393e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:04.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x559610436f20 0x5596104393e0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:04.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9aec03bba0 msgr2=0x7f9aec03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:04.007 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9aec03bba0 0x7f9aec03e060 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800ffd0 tx=0x7f9c18010000 comp rx=0 tx=0).stop 2024-05-08T18:06:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961041b2a0 msgr2=0x5596104225e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c23053640 1 -- 172.21.15.50:0/1683869150 reap_dead start 2024-05-08T18:06:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961041b2a0 0x5596104225e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f9c14006800 tx=0x7f9c14086080 comp rx=0 tx=0).stop 2024-05-08T18:06:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 shutdown_connections 2024-05-08T18:06:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 --2- 172.21.15.50:0/1683869150 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961041b2a0 0x5596104225e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 >> 172.21.15.50:0/1683869150 conn(0x559610417570 msgr2=0x559610414bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 shutdown_connections 2024-05-08T18:06:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 -- 172.21.15.50:0/1683869150 wait complete. 2024-05-08T18:06:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 -- 172.21.15.50:0/1876192180 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9afc03bcf0 msgr2=0x7f9afc03e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 --2- 172.21.15.50:0/1876192180 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9afc03bcf0 0x7f9afc03e1b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x5596104001e0 tx=0x7f9c1800f000 comp rx=0 tx=0).stop 2024-05-08T18:06:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 -- 172.21.15.50:0/1876192180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961040e110 msgr2=0x55961040e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 --2- 172.21.15.50:0/1876192180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961040e110 0x55961040e530 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9c14007000 tx=0x7f9c1408bd80 comp rx=0 tx=0).stop 2024-05-08T18:06:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 -- 172.21.15.50:0/1876192180 shutdown_connections 2024-05-08T18:06:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 --2- 172.21.15.50:0/1876192180 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9afc03bcf0 0x7f9afc03e1b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:04.008 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 --2- 172.21.15.50:0/1876192180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55961040e110 0x55961040e530 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 -- 172.21.15.50:0/1876192180 >> 172.21.15.50:0/1876192180 conn(0x5596104021b0 msgr2=0x5596104025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 -- 172.21.15.50:0/1876192180 shutdown_connections 2024-05-08T18:06:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.003+0000 7f9c26b25800 1 -- 172.21.15.50:0/1876192180 wait complete. 2024-05-08T18:06:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.004+0000 7f9c26b25800 1 -- 172.21.15.50:0/3026582948 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5596103f9380 msgr2=0x5596103fb770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.004+0000 7f9c26b25800 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5596103f9380 0x5596103fb770 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.004+0000 7f9c26b25800 1 -- 172.21.15.50:0/3026582948 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5596102f58d0 msgr2=0x5596103f6b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.004+0000 7f9c26b25800 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5596102f58d0 0x5596103f6b70 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.004+0000 7f9c26b25800 1 -- 172.21.15.50:0/3026582948 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5596103fd280 msgr2=0x5596103ff670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.004+0000 7f9c26b25800 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5596103fd280 0x5596103ff670 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.004+0000 7f9c26b25800 1 -- 172.21.15.50:0/3026582948 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9b0803bba0 msgr2=0x7f9b0803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.004+0000 7f9c26b25800 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9b0803bba0 0x7f9b0803e060 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9c18001e30 tx=0x7f9c18001390 comp rx=0 tx=0).stop 2024-05-08T18:06:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.004+0000 7f9c26b25800 1 -- 172.21.15.50:0/3026582948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 msgr2=0x559610269eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.004+0000 7f9c26b25800 1 --2- 172.21.15.50:0/3026582948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596102f4710 0x559610269eb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9c14052540 tx=0x7f9c14003020 comp rx=0 tx=0).stop 2024-05-08T18:06:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.004+0000 7f9c23053640 1 -- 172.21.15.50:0/3026582948 reap_dead start 2024-05-08T18:06:04.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.004+0000 7f9c26b25800 1 -- 172.21.15.50:0/3026582948 shutdown_connections 2024-05-08T18:06:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.004+0000 7f9c26b25800 1 -- 172.21.15.50:0/3026582948 >> 172.21.15.50:0/3026582948 conn(0x5596102efc90 msgr2=0x5596103dfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.004+0000 7f9c26b25800 1 -- 172.21.15.50:0/3026582948 shutdown_connections 2024-05-08T18:06:04.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:04.004+0000 7f9c26b25800 1 -- 172.21.15.50:0/3026582948 wait complete. 2024-05-08T18:06:04.011 DEBUG:tasks.util.rgw: json result: {'user_id': 'bud', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bud', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', 'active': True, 'create_date': '2024-05-08T18:06:03.816982Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:03.816968Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:09.019 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:06:09.019 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2024-05-08T18:06:09.020 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-05-08T18:06:09.020 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-05-08T18:06:09.020 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 4 2024-05-08T18:06:09.081 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:09.081 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:09.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.084+0000 7f12a9733800 1 Processor -- start 2024-05-08T18:06:09.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.084+0000 7f12a9733800 1 -- start start 2024-05-08T18:06:09.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.084+0000 7f12a9733800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56133ea11590 0x56133ea119b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.084+0000 7f12a9733800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56133ea12030 con 0x56133ea127f0 2024-05-08T18:06:09.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.084+0000 7f12a9733800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x56133e8b2e20 con 0x56133ea11590 2024-05-08T18:06:09.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.085+0000 7f12a54eb640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56133ea11590 0x56133ea119b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.085+0000 7f12a4cea640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x56133ea127f0 0x56133ea12c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:55694/0 (socket says 172.21.15.50:55694) 2024-05-08T18:06:09.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.085+0000 7f12a54eb640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56133ea11590 0x56133ea119b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50516/0 (socket says 172.21.15.50:50516) 2024-05-08T18:06:09.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.085+0000 7f12a4cea640 1 -- 172.21.15.50:0/3287514057 learned_addr learned my addr 172.21.15.50:0/3287514057 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:09.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.085+0000 7f12a54eb640 1 -- 172.21.15.50:0/3287514057 >> v1:172.21.15.46:6789/0 conn(0x56133ea127f0 legacy=0x56133ea12c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.085+0000 7f12a54eb640 1 -- 172.21.15.50:0/3287514057 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56133ea133b0 con 0x56133ea11590 2024-05-08T18:06:09.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.087+0000 7f12a54eb640 1 --2- 172.21.15.50:0/3287514057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 0x56133ea119b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f129800a470 tx=0x7f129801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ddbc2ecf646f5f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.087+0000 7f1297fff640 1 -- 172.21.15.50:0/3287514057 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1298057070 con 0x56133ea11590 2024-05-08T18:06:09.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.087+0000 7f1297fff640 1 -- 172.21.15.50:0/3287514057 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129804adc0 con 0x56133ea11590 2024-05-08T18:06:09.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.087+0000 7f1297fff640 1 -- 172.21.15.50:0/3287514057 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1298052590 con 0x56133ea11590 2024-05-08T18:06:09.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.088+0000 7f12a9733800 1 -- 172.21.15.50:0/3287514057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 msgr2=0x56133ea119b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.088+0000 7f12a9733800 1 --2- 172.21.15.50:0/3287514057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 0x56133ea119b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f129800a470 tx=0x7f129801ce90 comp rx=0 tx=0).stop 2024-05-08T18:06:09.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.088+0000 7f12a9733800 1 -- 172.21.15.50:0/3287514057 shutdown_connections 2024-05-08T18:06:09.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.088+0000 7f12a9733800 1 --2- 172.21.15.50:0/3287514057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 0x56133ea119b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.088+0000 7f12a9733800 1 -- 172.21.15.50:0/3287514057 >> 172.21.15.50:0/3287514057 conn(0x56133ea0cb90 msgr2=0x56133ea0efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.088+0000 7f12a9733800 1 -- 172.21.15.50:0/3287514057 shutdown_connections 2024-05-08T18:06:09.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.088+0000 7f12a9733800 1 -- 172.21.15.50:0/3287514057 wait complete. 2024-05-08T18:06:09.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.105+0000 7f12a9733800 1 Processor -- start 2024-05-08T18:06:09.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.105+0000 7f12a9733800 1 -- start start 2024-05-08T18:06:09.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.105+0000 7f12a9733800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 0x56133eaffab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.105+0000 7f12a9733800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56133e95da50 con 0x56133ea11590 2024-05-08T18:06:09.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.106+0000 7f12a54eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 0x56133eaffab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.106+0000 7f12a54eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 0x56133eaffab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50532/0 (socket says 172.21.15.50:50532) 2024-05-08T18:06:09.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.106+0000 7f12a54eb640 1 -- 172.21.15.50:0/2561780423 learned_addr learned my addr 172.21.15.50:0/2561780423 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.106+0000 7f12a54eb640 1 -- 172.21.15.50:0/2561780423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56133eafe070 con 0x56133ea11590 2024-05-08T18:06:09.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.106+0000 7f12a54eb640 1 --2- 172.21.15.50:0/2561780423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 0x56133eaffab0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1298055040 tx=0x7f129804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e560eb98e6b399a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.106+0000 7f1193606640 1 -- 172.21.15.50:0/2561780423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1298057040 con 0x56133ea11590 2024-05-08T18:06:09.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.106+0000 7f1193606640 1 -- 172.21.15.50:0/2561780423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1298062070 con 0x56133ea11590 2024-05-08T18:06:09.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.106+0000 7f1193606640 1 -- 172.21.15.50:0/2561780423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f129805d470 con 0x56133ea11590 2024-05-08T18:06:09.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.107+0000 7f12a9733800 1 -- 172.21.15.50:0/2561780423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 msgr2=0x56133eaffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.107+0000 7f12a9733800 1 --2- 172.21.15.50:0/2561780423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 0x56133eaffab0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f1298055040 tx=0x7f129804c000 comp rx=0 tx=0).stop 2024-05-08T18:06:09.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.107+0000 7f12a9733800 1 -- 172.21.15.50:0/2561780423 shutdown_connections 2024-05-08T18:06:09.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.107+0000 7f12a9733800 1 --2- 172.21.15.50:0/2561780423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 0x56133eaffab0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.107+0000 7f12a9733800 1 -- 172.21.15.50:0/2561780423 >> 172.21.15.50:0/2561780423 conn(0x56133ea0cb90 msgr2=0x56133eb02140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.109 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.107+0000 7f12a9733800 1 -- 172.21.15.50:0/2561780423 shutdown_connections 2024-05-08T18:06:09.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.107+0000 7f12a9733800 1 -- 172.21.15.50:0/2561780423 wait complete. 2024-05-08T18:06:09.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.108+0000 7f12a9733800 1 Processor -- start 2024-05-08T18:06:09.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.108+0000 7f12a9733800 1 -- start start 2024-05-08T18:06:09.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.108+0000 7f12a9733800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 0x56133eb0da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.108+0000 7f12a9733800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56133ea1acc0 con 0x56133ea11590 2024-05-08T18:06:09.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.108+0000 7f12a54eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 0x56133eb0da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.108+0000 7f12a54eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 0x56133eb0da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50536/0 (socket says 172.21.15.50:50536) 2024-05-08T18:06:09.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.108+0000 7f12a54eb640 1 -- 172.21.15.50:0/2867683090 learned_addr learned my addr 172.21.15.50:0/2867683090 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.109+0000 7f12a54eb640 1 -- 172.21.15.50:0/2867683090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56133eb0df40 con 0x56133ea11590 2024-05-08T18:06:09.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.109+0000 7f12a54eb640 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 0x56133eb0da00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f129801ce60 tx=0x7f1298002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.109+0000 7f1191e03640 1 -- 172.21.15.50:0/2867683090 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f129806b070 con 0x56133ea11590 2024-05-08T18:06:09.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.109+0000 7f1191e03640 1 -- 172.21.15.50:0/2867683090 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129805d970 con 0x56133ea11590 2024-05-08T18:06:09.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.109+0000 7f12a9733800 1 -- 172.21.15.50:0/2867683090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56133eb0e1d0 con 0x56133ea11590 2024-05-08T18:06:09.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.109+0000 7f1191e03640 1 -- 172.21.15.50:0/2867683090 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1298057070 con 0x56133ea11590 2024-05-08T18:06:09.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.109+0000 7f12a9733800 1 -- 172.21.15.50:0/2867683090 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56133eb10e80 con 0x56133ea11590 2024-05-08T18:06:09.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.110+0000 7f1191e03640 1 -- 172.21.15.50:0/2867683090 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f129805dc90 con 0x56133ea11590 2024-05-08T18:06:09.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.110+0000 7f1191e03640 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f118c03bba0 0x7f118c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.110+0000 7f1191e03640 1 -- 172.21.15.50:0/2867683090 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f1298094750 con 0x56133ea11590 2024-05-08T18:06:09.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.111+0000 7f12a4cea640 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f118c03bba0 0x7f118c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.111+0000 7f12a9733800 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56133ea127f0 0x56133eb14c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.111+0000 7f12a9733800 1 -- 172.21.15.50:0/2867683090 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb15180 con 0x56133ea127f0 2024-05-08T18:06:09.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.111+0000 7f12a5cec640 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56133ea127f0 0x56133eb14c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.111+0000 7f12a4cea640 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f118c03bba0 0x7f118c03e060 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f129c001d70 tx=0x7f129c0012d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.111+0000 7f12a5cec640 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56133ea127f0 0x56133eb14c40 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.111+0000 7f12a5cec640 1 -- 172.21.15.50:0/2867683090 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f12a000a4a0 con 0x56133ea127f0 2024-05-08T18:06:09.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.111+0000 7f12a9733800 1 -- 172.21.15.50:0/2867683090 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb121f0 con 0x56133ea127f0 2024-05-08T18:06:09.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.112+0000 7f12a5cec640 1 -- 172.21.15.50:0/2867683090 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f12a000a4a0 con 0x56133ea127f0 2024-05-08T18:06:09.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.112+0000 7f12a9733800 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56133eb174b0 0x56133eb198a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.112+0000 7f12a9733800 1 -- 172.21.15.50:0/2867683090 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb19f90 con 0x56133eb174b0 2024-05-08T18:06:09.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.112+0000 7f12a54eb640 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56133eb174b0 0x56133eb198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.112+0000 7f12a54eb640 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56133eb174b0 0x56133eb198a0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.113+0000 7f12a54eb640 1 -- 172.21.15.50:0/2867683090 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f129805a040 con 0x56133eb174b0 2024-05-08T18:06:09.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.113+0000 7f12a9733800 1 -- 172.21.15.50:0/2867683090 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb16320 con 0x56133ea127f0 2024-05-08T18:06:09.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.113+0000 7f12a5cec640 1 -- 172.21.15.50:0/2867683090 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f12a000a4a0 con 0x56133ea127f0 2024-05-08T18:06:09.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.113+0000 7f12a9733800 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56133eb1b3f0 0x56133eb1d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.113+0000 7f12a9733800 1 -- 172.21.15.50:0/2867683090 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb1ded0 con 0x56133eb1b3f0 2024-05-08T18:06:09.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.113+0000 7f12a4cea640 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56133eb1b3f0 0x56133eb1d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.113+0000 7f12a4cea640 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56133eb1b3f0 0x56133eb1d7e0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.114+0000 7f12a4cea640 1 -- 172.21.15.50:0/2867683090 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f129c009050 con 0x56133eb1b3f0 2024-05-08T18:06:09.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.114+0000 7f12a9733800 1 Processor -- start 2024-05-08T18:06:09.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.114+0000 7f12a9733800 1 -- start start 2024-05-08T18:06:09.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.114+0000 7f12a9733800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb2c200 0x56133eb2c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.114+0000 7f12a9733800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56133e904330 con 0x56133eb2c200 2024-05-08T18:06:09.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.115+0000 7f12a54eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb2c200 0x56133eb2c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.115+0000 7f12a54eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb2c200 0x56133eb2c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50538/0 (socket says 172.21.15.50:50538) 2024-05-08T18:06:09.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.115+0000 7f12a54eb640 1 -- 172.21.15.50:0/1945367867 learned_addr learned my addr 172.21.15.50:0/1945367867 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.115+0000 7f12a54eb640 1 -- 172.21.15.50:0/1945367867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56133eb2cb60 con 0x56133eb2c200 2024-05-08T18:06:09.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.115+0000 7f12a54eb640 1 --2- 172.21.15.50:0/1945367867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb2c200 0x56133eb2c620 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f129805f040 tx=0x7f129804bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.115+0000 7f118affd640 1 -- 172.21.15.50:0/1945367867 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1298009e60 con 0x56133eb2c200 2024-05-08T18:06:09.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.115+0000 7f118affd640 1 -- 172.21.15.50:0/1945367867 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1298052440 con 0x56133eb2c200 2024-05-08T18:06:09.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.115+0000 7f12a9733800 1 -- 172.21.15.50:0/1945367867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56133eb2d5d0 con 0x56133eb2c200 2024-05-08T18:06:09.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.115+0000 7f118affd640 1 -- 172.21.15.50:0/1945367867 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1298009410 con 0x56133eb2c200 2024-05-08T18:06:09.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.115+0000 7f12a9733800 1 -- 172.21.15.50:0/1945367867 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56133eb2dad0 con 0x56133eb2c200 2024-05-08T18:06:09.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.116+0000 7f118affd640 1 -- 172.21.15.50:0/1945367867 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f129806ea20 con 0x56133eb2c200 2024-05-08T18:06:09.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.117+0000 7f118affd640 1 --2- 172.21.15.50:0/1945367867 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f117c03bb50 0x7f117c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.117+0000 7f118affd640 1 -- 172.21.15.50:0/1945367867 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f1298083070 con 0x56133eb2c200 2024-05-08T18:06:09.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.117+0000 7f12a4cea640 1 --2- 172.21.15.50:0/1945367867 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f117c03bb50 0x7f117c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.117+0000 7f12a9733800 1 Processor -- start 2024-05-08T18:06:09.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.117+0000 7f12a9733800 1 -- start start 2024-05-08T18:06:09.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.117+0000 7f12a4cea640 1 --2- 172.21.15.50:0/1945367867 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f117c03bb50 0x7f117c03e010 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f129c00bf40 tx=0x7f129c00f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.117+0000 7f12a9733800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb39360 0x56133eb39780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.117+0000 7f12a9733800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56133eafe330 con 0x56133eb39360 2024-05-08T18:06:09.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.117+0000 7f12a54eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb39360 0x56133eb39780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.117+0000 7f12a54eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb39360 0x56133eb39780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50546/0 (socket says 172.21.15.50:50546) 2024-05-08T18:06:09.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.117+0000 7f12a54eb640 1 -- 172.21.15.50:0/1305403372 learned_addr learned my addr 172.21.15.50:0/1305403372 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.118+0000 7f12a54eb640 1 -- 172.21.15.50:0/1305403372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56133eb39d50 con 0x56133eb39360 2024-05-08T18:06:09.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.118+0000 7f12a54eb640 1 --2- 172.21.15.50:0/1305403372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb39360 0x56133eb39780 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f129805b6c0 tx=0x7f129805c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec66add9e0a893a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.118+0000 7f1188ff9640 1 -- 172.21.15.50:0/1305403372 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f129804f2a0 con 0x56133eb39360 2024-05-08T18:06:09.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.118+0000 7f1188ff9640 1 -- 172.21.15.50:0/1305403372 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129806ed60 con 0x56133eb39360 2024-05-08T18:06:09.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.118+0000 7f1188ff9640 1 -- 172.21.15.50:0/1305403372 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f129806e490 con 0x56133eb39360 2024-05-08T18:06:09.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.119+0000 7f12a9733800 1 -- 172.21.15.50:0/1305403372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb39360 msgr2=0x56133eb39780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.119+0000 7f12a9733800 1 --2- 172.21.15.50:0/1305403372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb39360 0x56133eb39780 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f129805b6c0 tx=0x7f129805c6c0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.119+0000 7f12a9733800 1 -- 172.21.15.50:0/1305403372 shutdown_connections 2024-05-08T18:06:09.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.119+0000 7f12a9733800 1 --2- 172.21.15.50:0/1305403372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb39360 0x56133eb39780 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.119+0000 7f12a9733800 1 -- 172.21.15.50:0/1305403372 >> 172.21.15.50:0/1305403372 conn(0x56133eb35630 msgr2=0x56133eb33690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.119+0000 7f12a9733800 1 -- 172.21.15.50:0/1305403372 shutdown_connections 2024-05-08T18:06:09.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.119+0000 7f12a9733800 1 -- 172.21.15.50:0/1305403372 wait complete. 2024-05-08T18:06:09.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.119+0000 7f12a9733800 1 Processor -- start 2024-05-08T18:06:09.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.119+0000 7f12a9733800 1 -- start start 2024-05-08T18:06:09.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.119+0000 7f12a9733800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb39360 0x56133eb40640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.119+0000 7f12a9733800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56133eb3a3b0 con 0x56133eb39360 2024-05-08T18:06:09.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.119+0000 7f12a54eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb39360 0x56133eb40640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.119+0000 7f12a54eb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb39360 0x56133eb40640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50562/0 (socket says 172.21.15.50:50562) 2024-05-08T18:06:09.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.119+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 learned_addr learned my addr 172.21.15.50:0/4211024071 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.120+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56133eb40b80 con 0x56133eb39360 2024-05-08T18:06:09.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.120+0000 7f12a54eb640 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb39360 0x56133eb40640 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f129805c590 tx=0x7f1298050ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.120+0000 7f117affd640 1 -- 172.21.15.50:0/4211024071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f12980b5020 con 0x56133eb39360 2024-05-08T18:06:09.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.120+0000 7f117affd640 1 -- 172.21.15.50:0/4211024071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f129804f2a0 con 0x56133eb39360 2024-05-08T18:06:09.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.120+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56133eb40b80 con 0x56133eb39360 2024-05-08T18:06:09.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.120+0000 7f117affd640 1 -- 172.21.15.50:0/4211024071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1298094b20 con 0x56133eb39360 2024-05-08T18:06:09.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.120+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56133eb40b80 con 0x56133eb39360 2024-05-08T18:06:09.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.121+0000 7f117affd640 1 -- 172.21.15.50:0/4211024071 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f12980b8020 con 0x56133eb39360 2024-05-08T18:06:09.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.121+0000 7f117affd640 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f117003bb50 0x7f117003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.121+0000 7f117affd640 1 -- 172.21.15.50:0/4211024071 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f12980ac040 con 0x56133eb39360 2024-05-08T18:06:09.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.121+0000 7f12a4cea640 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f117003bb50 0x7f117003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.122+0000 7f12a4cea640 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f117003bb50 0x7f117003e010 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f129c00c010 tx=0x7f129c010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.122+0000 7f12a9733800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.122+0000 7f12a9733800 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56133eb4be30 0x56133eb4e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.122+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4ea40 con 0x56133eb4be30 2024-05-08T18:06:09.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.122+0000 7f12a5cec640 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56133eb4be30 0x56133eb4e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.122+0000 7f12a5cec640 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56133eb4be30 0x56133eb4e2f0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.123+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f12a000a4a0 con 0x56133eb4be30 2024-05-08T18:06:09.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.123+0000 7f12a9733800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.123+0000 7f12a9733800 20 realm 2024-05-08T18:06:09.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.123+0000 7f12a9733800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.123+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4ee50 con 0x56133eb4be30 2024-05-08T18:06:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.123+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f12a000a4a0 con 0x56133eb4be30 2024-05-08T18:06:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.123+0000 7f12a9733800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.123+0000 7f12a9733800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.123+0000 7f12a9733800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.123+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4f720 con 0x56133eb4be30 2024-05-08T18:06:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.123+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f12a000a4a0 con 0x56133eb4be30 2024-05-08T18:06:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.123+0000 7f12a9733800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.123+0000 7f12a9733800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.123+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb4be30 2024-05-08T18:06:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.124+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f12a000a4a0 con 0x56133eb4be30 2024-05-08T18:06:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.124+0000 7f12a9733800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:09.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.124+0000 7f12a9733800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.124+0000 7f12a9733800 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56133eb50820 0x56133eb52c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.124+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb53220 con 0x56133eb50820 2024-05-08T18:06:09.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.124+0000 7f12a54eb640 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56133eb50820 0x56133eb52c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.124+0000 7f12a54eb640 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56133eb50820 0x56133eb52c10 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.125+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.125+0000 7f12a9733800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:09.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.125+0000 7f12a9733800 20 searching for the correct realm 2024-05-08T18:06:09.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.125+0000 7f12a9733800 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56133eb551f0 0x56133eb57680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.125+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb57d50 con 0x56133eb551f0 2024-05-08T18:06:09.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.125+0000 7f12a4cea640 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56133eb551f0 0x56133eb57680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.125+0000 7f12a4cea640 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56133eb551f0 0x56133eb57680 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.125+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.125+0000 7f12a4cea640 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f129c037040 0x7f129c036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.125+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb57d50 con 0x7f129c037040 2024-05-08T18:06:09.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.126+0000 7f12a5cec640 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f129c037040 0x7f129c036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.126+0000 7f12a5cec640 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f129c037040 0x7f129c036410 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.126+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f12a000a4a0 con 0x7f129c037040 2024-05-08T18:06:09.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.126+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb50820 2024-05-08T18:06:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.127+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.127+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb4be30 2024-05-08T18:06:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.127+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f12a000a4a0 con 0x56133eb4be30 2024-05-08T18:06:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.127+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb4be30 2024-05-08T18:06:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.127+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f12a000a4a0 con 0x56133eb4be30 2024-05-08T18:06:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.127+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb4be30 2024-05-08T18:06:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.128+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f12a000a4a0 con 0x56133eb4be30 2024-05-08T18:06:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.128+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb50820 2024-05-08T18:06:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.128+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.128+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x7f129c037040 2024-05-08T18:06:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.128+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f12a000a4a0 con 0x7f129c037040 2024-05-08T18:06:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.128+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb551f0 2024-05-08T18:06:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.129+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.129+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb4be30 2024-05-08T18:06:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.129+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f12a000a4a0 con 0x56133eb4be30 2024-05-08T18:06:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.129+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb50820 2024-05-08T18:06:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.129+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.129+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb4be30 2024-05-08T18:06:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.130+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f12a000a4a0 con 0x56133eb4be30 2024-05-08T18:06:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.130+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x7f129c037040 2024-05-08T18:06:09.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.130+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f12a0032040 con 0x7f129c037040 2024-05-08T18:06:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.130+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb551f0 2024-05-08T18:06:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.130+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.130+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb551f0 2024-05-08T18:06:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.130+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.130+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb50820 2024-05-08T18:06:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.131+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.131+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb551f0 2024-05-08T18:06:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.131+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.131+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb551f0 2024-05-08T18:06:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.132+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.132+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb4be30 2024-05-08T18:06:09.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.132+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f12a0032040 con 0x56133eb4be30 2024-05-08T18:06:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.132+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb4be30 2024-05-08T18:06:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.132+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f12a0032040 con 0x56133eb4be30 2024-05-08T18:06:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.132+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb50820 2024-05-08T18:06:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.133+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.133+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb551f0 2024-05-08T18:06:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.133+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.133+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x7f129c037040 2024-05-08T18:06:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.133+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f12a0032040 con 0x7f129c037040 2024-05-08T18:06:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.133+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb4be30 2024-05-08T18:06:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.133+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f12a0032040 con 0x56133eb4be30 2024-05-08T18:06:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.133+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb50820 2024-05-08T18:06:09.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.134+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.134+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb551f0 2024-05-08T18:06:09.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.134+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.134+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb50820 2024-05-08T18:06:09.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.134+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.134+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x7f129c037040 2024-05-08T18:06:09.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.135+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f12a0032040 con 0x7f129c037040 2024-05-08T18:06:09.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.135+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb551f0 2024-05-08T18:06:09.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.135+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.135+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb50820 2024-05-08T18:06:09.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.135+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.135+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x56133eb4be30 2024-05-08T18:06:09.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f12a0032040 con 0x56133eb4be30 2024-05-08T18:06:09.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56133eb4fac0 con 0x7f129c037040 2024-05-08T18:06:09.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f12a0032040 con 0x7f129c037040 2024-05-08T18:06:09.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a9733800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:09.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a9733800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:09.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a9733800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:09.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a9733800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:09.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a9733800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:09.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a9733800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:09.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a9733800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb53ae0 con 0x56133eb4be30 2024-05-08T18:06:09.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f12a0032040 con 0x56133eb4be30 2024-05-08T18:06:09.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a9733800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a9733800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb53ae0 con 0x56133eb4be30 2024-05-08T18:06:09.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f12a0032040 con 0x56133eb4be30 2024-05-08T18:06:09.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a9733800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:09.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a9733800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.136+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb53ae0 con 0x56133eb551f0 2024-05-08T18:06:09.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.137+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.137+0000 7f12a9733800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:09.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.137+0000 7f12a9733800 20 zone default found 2024-05-08T18:06:09.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.137+0000 7f12a9733800 4 Realm: () 2024-05-08T18:06:09.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.137+0000 7f12a9733800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:09.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.137+0000 7f12a9733800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:09.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.137+0000 7f12a9733800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:09.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.137+0000 7f12a9733800 20 zonegroup default 2024-05-08T18:06:09.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.137+0000 7f12a9733800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.137+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb54410 con 0x56133eb50820 2024-05-08T18:06:09.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.137+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.137+0000 7f12a9733800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.138+0000 7f12a9733800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.138+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb59c10 con 0x56133eb551f0 2024-05-08T18:06:09.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.138+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.138+0000 7f12a9733800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.138+0000 7f12a9733800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.153 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.138+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb5a120 con 0x56133eb551f0 2024-05-08T18:06:09.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.138+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.138+0000 7f12a9733800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.138+0000 7f12a9733800 20 started sync module instance, tier type = 2024-05-08T18:06:09.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.138+0000 7f12a9733800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:09.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.138+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb59050 con 0x7f129c037040 2024-05-08T18:06:09.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.139+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f12a0032040 con 0x7f129c037040 2024-05-08T18:06:09.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.139+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94640656454992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb59050 con 0x7f129c037040 2024-05-08T18:06:09.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.139+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb5aba0 con 0x56133eb4be30 2024-05-08T18:06:09.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.140+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94640656454992] v29'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f12a0032040 con 0x7f129c037040 2024-05-08T18:06:09.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.141+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f12a0032040 con 0x56133eb4be30 2024-05-08T18:06:09.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.141+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94640656459984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb59050 con 0x56133eb4be30 2024-05-08T18:06:09.154 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.141+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb5b620 con 0x56133eb551f0 2024-05-08T18:06:09.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.142+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94640656459984] v29'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f12a0032040 con 0x56133eb4be30 2024-05-08T18:06:09.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.142+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.142+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94640656452512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb58fc0 con 0x56133eb551f0 2024-05-08T18:06:09.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.142+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb5c410 con 0x7f129c037040 2024-05-08T18:06:09.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.143+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f12a0032040 con 0x7f129c037040 2024-05-08T18:06:09.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.143+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94640656465648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb5ccb0 con 0x7f129c037040 2024-05-08T18:06:09.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.143+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb5d730 con 0x56133eb4be30 2024-05-08T18:06:09.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.144+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94640656452512] v29'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.145+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94640656465648] v29'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f12a0032040 con 0x7f129c037040 2024-05-08T18:06:09.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.145+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f12a0032040 con 0x56133eb4be30 2024-05-08T18:06:09.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.145+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94640656470560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb5c7b0 con 0x56133eb4be30 2024-05-08T18:06:09.155 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.145+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb5dfd0 con 0x56133eb50820 2024-05-08T18:06:09.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.146+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94640656470560] v29'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f12a0032040 con 0x56133eb4be30 2024-05-08T18:06:09.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.146+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.146+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94640656472864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb5c7b0 con 0x56133eb50820 2024-05-08T18:06:09.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.146+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb5e9d0 con 0x56133eb50820 2024-05-08T18:06:09.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.147+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(56 notify.6 [create] v29'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.147+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94640656475360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb5f1e0 con 0x56133eb50820 2024-05-08T18:06:09.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.147+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb5fd00 con 0x56133eb50820 2024-05-08T18:06:09.156 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.147+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94640656472864] v29'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.148+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94640656475360] v29'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.148+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.148+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94640656480208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb5ec30 con 0x56133eb50820 2024-05-08T18:06:09.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.148+0000 7f12a9733800 20 add_watcher() i=0 2024-05-08T18:06:09.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.148+0000 7f12a9733800 20 add_watcher() i=1 2024-05-08T18:06:09.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.148+0000 7f12a9733800 20 add_watcher() i=2 2024-05-08T18:06:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.148+0000 7f12a9733800 20 add_watcher() i=3 2024-05-08T18:06:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.148+0000 7f12a9733800 20 add_watcher() i=4 2024-05-08T18:06:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.148+0000 7f12a9733800 20 add_watcher() i=5 2024-05-08T18:06:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.148+0000 7f12a9733800 20 add_watcher() i=6 2024-05-08T18:06:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.149+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94640656480208] v29'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.149+0000 7f12a9733800 20 add_watcher() i=7 2024-05-08T18:06:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.149+0000 7f12a9733800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.150+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb62cf0 con 0x56133eb4be30 2024-05-08T18:06:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.150+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f12a0032040 con 0x56133eb4be30 2024-05-08T18:06:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.150+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94640656464448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb601a0 con 0x56133eb4be30 2024-05-08T18:06:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.151+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94640656464448] v29'214 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f12a0032040 con 0x56133eb4be30 2024-05-08T18:06:09.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.151+0000 7f115b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.151+0000 7f12a9733800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.151+0000 7f12a9733800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.151+0000 7f12a9733800 5 note: GC not initialized 2024-05-08T18:06:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.151+0000 7f115a7e4640 20 reqs_thread_entry: start 2024-05-08T18:06:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.151+0000 7f12a9733800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.152+0000 7f12a9733800 20 Filter name: none 2024-05-08T18:06:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.152+0000 7f10cbfff640 20 reqs_thread_entry: start 2024-05-08T18:06:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.152+0000 7f12a9733800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-05-08T18:06:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.152+0000 7f12a9733800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.152+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb96210 con 0x56133eb50820 2024-05-08T18:06:09.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.152+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.152+0000 7f12a9733800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.152+0000 7f12a9733800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-05-08T18:06:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.152+0000 7f12a9733800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-05-08T18:06:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.152+0000 7f12a9733800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-05-08T18:06:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.152+0000 7f12a9733800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.152+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb949b0 con 0x56133eb50820 2024-05-08T18:06:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.153+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.153+0000 7f12a9733800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.153+0000 7f12a9733800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-05-08T18:06:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.153+0000 7f12a9733800 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-05-08T18:06:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.153+0000 7f12a9733800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-05-08T18:06:09.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.153+0000 7f12a9733800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-05-08T18:06:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.153+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~380 in=380b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb952a0 con 0x56133eb50820 2024-05-08T18:06:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.154+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~380] v29'5 uv5 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.154+0000 7f12a9733800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-05-08T18:06:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.154+0000 7f12a9733800 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-05-08T18:06:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.154+0000 7f12a9733800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-05-08T18:06:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.154+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 94640656698864 in=568b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb99800 con 0x56133eb50820 2024-05-08T18:06:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.154+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== watch-notify(notify (1) cookie 94640656472864 notify 124554051621 ret 0) ==== 598+0+0 (crc 0 0 0) 0x56133eb95c60 con 0x56133eb50820 2024-05-08T18:06:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.154+0000 7f1188ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051621 cookie 94640656472864 notifier 4732 bl.length()=556 2024-05-08T18:06:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.154+0000 7f1188ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-05-08T18:06:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.154+0000 7f1188ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-05-08T18:06:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.154+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(65 notify.5 [notify cookie 94640656698864 out=8b] v0'0 uv216 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.154+0000 7f1188ff9640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f1168003ea0 con 0x56133eb50820 2024-05-08T18:06:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.155+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.155+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== watch-notify(notify_complete (2) cookie 94640656698864 notify 124554051621 ret 0) ==== 42+0+48 (crc 0 0 0) 0x56133eb2cb60 con 0x56133eb50820 2024-05-08T18:06:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.155+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb95220 con 0x56133eb50820 2024-05-08T18:06:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.156+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v29'9 uv9 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.156+0000 7f12a9733800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-05-08T18:06:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.156+0000 7f12a9733800 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-05-08T18:06:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.156+0000 7f12a9733800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-05-08T18:06:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.156+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94640656698736 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56133eb95190 con 0x56133eb551f0 2024-05-08T18:06:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.156+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== watch-notify(notify (1) cookie 94640656452512 notify 124554051590 ret 0) ==== 219+0+0 (crc 0 0 0) 0x7f129c041040 con 0x56133eb551f0 2024-05-08T18:06:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.156+0000 7f1188ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051590 cookie 94640656452512 notifier 4732 bl.length()=177 2024-05-08T18:06:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.156+0000 7f1188ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-05-08T18:06:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.156+0000 7f1188ff9640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-05-08T18:06:09.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.156+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(68 notify.2 [notify cookie 94640656698736 out=8b] v0'0 uv216 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.156+0000 7f1188ff9640 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f1168003ea0 con 0x56133eb551f0 2024-05-08T18:06:09.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.157+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== watch-notify(notify_complete (2) cookie 94640656698736 notify 124554051590 ret 0) ==== 42+0+48 (crc 0 0 0) 0x56133ea12030 con 0x56133eb551f0 2024-05-08T18:06:09.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.157+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.167 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"fud","display_name":"Fud","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"fud","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-05-08T18:06:09.153812Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:06:09.153799Z","tags":[],"group_ids":[]} 2024-05-08T18:06:09.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.158+0000 7f115b7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:09.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.158+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94640656464448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb48030 con 0x56133eb4be30 2024-05-08T18:06:09.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.159+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94640656464448] v29'215 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f12a0032040 con 0x56133eb4be30 2024-05-08T18:06:09.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.159+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94640656454992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb47b80 con 0x7f129c037040 2024-05-08T18:06:09.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.160+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94640656454992] v29'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f12a0032040 con 0x7f129c037040 2024-05-08T18:06:09.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.160+0000 7f12a9733800 20 remove_watcher() i=0 2024-05-08T18:06:09.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.160+0000 7f12a9733800 2 removed watcher, disabling cache 2024-05-08T18:06:09.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.160+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94640656459984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb59550 con 0x56133eb4be30 2024-05-08T18:06:09.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.161+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94640656459984] v29'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f12a0032040 con 0x56133eb4be30 2024-05-08T18:06:09.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.161+0000 7f12a9733800 20 remove_watcher() i=1 2024-05-08T18:06:09.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.161+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94640656452512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb59550 con 0x56133eb551f0 2024-05-08T18:06:09.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.162+0000 7f12a4cea640 1 -- 172.21.15.50:0/4211024071 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94640656452512] v29'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f129c009050 con 0x56133eb551f0 2024-05-08T18:06:09.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.162+0000 7f12a9733800 20 remove_watcher() i=2 2024-05-08T18:06:09.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.162+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94640656465648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb58ba0 con 0x7f129c037040 2024-05-08T18:06:09.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.163+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94640656465648] v29'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f12a0032040 con 0x7f129c037040 2024-05-08T18:06:09.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.163+0000 7f12a9733800 20 remove_watcher() i=3 2024-05-08T18:06:09.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.163+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94640656470560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb58ba0 con 0x56133eb4be30 2024-05-08T18:06:09.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.164+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94640656470560] v29'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f12a0032040 con 0x56133eb4be30 2024-05-08T18:06:09.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.164+0000 7f12a9733800 20 remove_watcher() i=4 2024-05-08T18:06:09.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.164+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94640656472864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb58ba0 con 0x56133eb50820 2024-05-08T18:06:09.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.165+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94640656472864] v29'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.165+0000 7f12a9733800 20 remove_watcher() i=5 2024-05-08T18:06:09.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.165+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94640656475360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb58ba0 con 0x56133eb50820 2024-05-08T18:06:09.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.166+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94640656475360] v29'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.166+0000 7f12a9733800 20 remove_watcher() i=6 2024-05-08T18:06:09.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.166+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94640656480208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56133eb58ba0 con 0x56133eb50820 2024-05-08T18:06:09.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a54eb640 1 -- 172.21.15.50:0/4211024071 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94640656480208] v29'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f129805a040 con 0x56133eb50820 2024-05-08T18:06:09.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 20 remove_watcher() i=7 2024-05-08T18:06:09.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56133eb50820 msgr2=0x56133eb52c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56133eb50820 0x56133eb52c10 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56133eb4be30 msgr2=0x56133eb4e2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56133eb4be30 0x56133eb4e2f0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f129c037040 msgr2=0x7f129c036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.175 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f129c037040 0x7f129c036410 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56133eb551f0 msgr2=0x56133eb57680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56133eb551f0 0x56133eb57680 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f117003bb50 msgr2=0x7f117003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f117003bb50 0x7f117003e010 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f129c00c010 tx=0x7f129c010000 comp rx=0 tx=0).stop 2024-05-08T18:06:09.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a5cec640 1 -- 172.21.15.50:0/4211024071 reap_dead start 2024-05-08T18:06:09.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb39360 msgr2=0x56133eb40640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb39360 0x56133eb40640 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f129805c590 tx=0x7f1298050ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 shutdown_connections 2024-05-08T18:06:09.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 --2- 172.21.15.50:0/4211024071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb39360 0x56133eb40640 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 >> 172.21.15.50:0/4211024071 conn(0x56133eb35630 msgr2=0x56133eb32c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 shutdown_connections 2024-05-08T18:06:09.176 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 -- 172.21.15.50:0/4211024071 wait complete. 2024-05-08T18:06:09.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 -- 172.21.15.50:0/1945367867 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f117c03bb50 msgr2=0x7f117c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 --2- 172.21.15.50:0/1945367867 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f117c03bb50 0x7f117c03e010 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f129c00bf40 tx=0x7f129c00f000 comp rx=0 tx=0).stop 2024-05-08T18:06:09.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 -- 172.21.15.50:0/1945367867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb2c200 msgr2=0x56133eb2c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 --2- 172.21.15.50:0/1945367867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb2c200 0x56133eb2c620 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f129805f040 tx=0x7f129804bf70 comp rx=0 tx=0).stop 2024-05-08T18:06:09.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 -- 172.21.15.50:0/1945367867 shutdown_connections 2024-05-08T18:06:09.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 --2- 172.21.15.50:0/1945367867 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f117c03bb50 0x7f117c03e010 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 --2- 172.21.15.50:0/1945367867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133eb2c200 0x56133eb2c620 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 -- 172.21.15.50:0/1945367867 >> 172.21.15.50:0/1945367867 conn(0x56133eb202a0 msgr2=0x56133eb206a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.168+0000 7f12a9733800 1 -- 172.21.15.50:0/1945367867 shutdown_connections 2024-05-08T18:06:09.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.169+0000 7f12a9733800 1 -- 172.21.15.50:0/1945367867 wait complete. 2024-05-08T18:06:09.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.169+0000 7f12a9733800 1 -- 172.21.15.50:0/2867683090 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56133eb174b0 msgr2=0x56133eb198a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.169+0000 7f12a9733800 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56133eb174b0 0x56133eb198a0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.169+0000 7f12a9733800 1 -- 172.21.15.50:0/2867683090 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56133ea127f0 msgr2=0x56133eb14c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.177 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.169+0000 7f12a9733800 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56133ea127f0 0x56133eb14c40 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.169+0000 7f12a9733800 1 -- 172.21.15.50:0/2867683090 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56133eb1b3f0 msgr2=0x56133eb1d7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.169+0000 7f12a9733800 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56133eb1b3f0 0x56133eb1d7e0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.169+0000 7f12a9733800 1 -- 172.21.15.50:0/2867683090 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f118c03bba0 msgr2=0x7f118c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.169+0000 7f12a9733800 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f118c03bba0 0x7f118c03e060 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f129c001d70 tx=0x7f129c0012d0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.169+0000 7f12a9733800 1 -- 172.21.15.50:0/2867683090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 msgr2=0x56133eb0da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.169+0000 7f12a9733800 1 --2- 172.21.15.50:0/2867683090 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56133ea11590 0x56133eb0da00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f129801ce60 tx=0x7f1298002fb0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.169+0000 7f12a5cec640 1 -- 172.21.15.50:0/2867683090 reap_dead start 2024-05-08T18:06:09.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.169+0000 7f12a9733800 1 -- 172.21.15.50:0/2867683090 shutdown_connections 2024-05-08T18:06:09.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.169+0000 7f12a9733800 1 -- 172.21.15.50:0/2867683090 >> 172.21.15.50:0/2867683090 conn(0x56133ea0cb90 msgr2=0x56133eafd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.169+0000 7f12a9733800 1 -- 172.21.15.50:0/2867683090 shutdown_connections 2024-05-08T18:06:09.178 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.169+0000 7f12a9733800 1 -- 172.21.15.50:0/2867683090 wait complete. 2024-05-08T18:06:09.179 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-05-08T18:06:09.153812Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:09.153799Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:09.192 INFO:tasks.radosgw_admin:RL: create_bucket myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:06:09.196 INFO:tasks.radosgw_admin:RL: put_obj myfoo7 fud bytes_out=8 bytes_in=0 failed=False 2024-05-08T18:06:09.196 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2024-05-08T18:06:09.197 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2024-05-08T18:06:09.197 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bud --new-uid fud 2024-05-08T18:06:09.244 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:09.244 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:09.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.247+0000 7f4aa5a19800 1 Processor -- start 2024-05-08T18:06:09.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.247+0000 7f4aa5a19800 1 -- start start 2024-05-08T18:06:09.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.247+0000 7f4aa5a19800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55db5e0f41c0 0x55db5e0f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.247+0000 7f4aa5a19800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55db5e0fa600 con 0x55db5e0f6af0 2024-05-08T18:06:09.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.247+0000 7f4aa5a19800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55db5df93e20 con 0x55db5e0f41c0 2024-05-08T18:06:09.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.248+0000 7f4aa3511640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55db5e0f41c0 0x55db5e0f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.248+0000 7f4aa3511640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55db5e0f41c0 0x55db5e0f65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50576/0 (socket says 172.21.15.50:50576) 2024-05-08T18:06:09.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.248+0000 7f4aa3511640 1 -- 172.21.15.50:0/27370727 learned_addr learned my addr 172.21.15.50:0/27370727 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.248+0000 7f4aa3511640 1 -- 172.21.15.50:0/27370727 >> v1:172.21.15.46:6789/0 conn(0x55db5e0f6af0 legacy=0x55db5e0f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.248+0000 7f4aa3511640 1 -- 172.21.15.50:0/27370727 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db5e0fb7e0 con 0x55db5e0f41c0 2024-05-08T18:06:09.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.250+0000 7f4aa3511640 1 --2- 172.21.15.50:0/27370727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 0x55db5e0f65b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400a520 tx=0x7f4a9401d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22b755f1da23dff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.250+0000 7f4aa250f640 1 -- 172.21.15.50:0/27370727 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a94057070 con 0x55db5e0f41c0 2024-05-08T18:06:09.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.250+0000 7f4aa250f640 1 -- 172.21.15.50:0/27370727 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a94052570 con 0x55db5e0f41c0 2024-05-08T18:06:09.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.250+0000 7f4aa250f640 1 -- 172.21.15.50:0/27370727 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a9404ae60 con 0x55db5e0f41c0 2024-05-08T18:06:09.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.251+0000 7f4aa5a19800 1 -- 172.21.15.50:0/27370727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 msgr2=0x55db5e0f65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.251+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/27370727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 0x55db5e0f65b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4a9400a520 tx=0x7f4a9401d010 comp rx=0 tx=0).stop 2024-05-08T18:06:09.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.251+0000 7f4aa5a19800 1 -- 172.21.15.50:0/27370727 shutdown_connections 2024-05-08T18:06:09.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.251+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/27370727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 0x55db5e0f65b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.251+0000 7f4aa5a19800 1 -- 172.21.15.50:0/27370727 >> 172.21.15.50:0/27370727 conn(0x55db5e0edaf0 msgr2=0x55db5e0eff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.251+0000 7f4aa5a19800 1 -- 172.21.15.50:0/27370727 shutdown_connections 2024-05-08T18:06:09.249 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.251+0000 7f4aa5a19800 1 -- 172.21.15.50:0/27370727 wait complete. 2024-05-08T18:06:09.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.266+0000 7f4aa5a19800 1 Processor -- start 2024-05-08T18:06:09.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.266+0000 7f4aa5a19800 1 -- start start 2024-05-08T18:06:09.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.266+0000 7f4aa5a19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 0x55db5e0f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.266+0000 7f4aa5a19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55db5e03ea30 con 0x55db5e0f41c0 2024-05-08T18:06:09.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.267+0000 7f4aa3511640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 0x55db5e0f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.267+0000 7f4aa3511640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 0x55db5e0f3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50590/0 (socket says 172.21.15.50:50590) 2024-05-08T18:06:09.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.267+0000 7f4aa3511640 1 -- 172.21.15.50:0/3038845679 learned_addr learned my addr 172.21.15.50:0/3038845679 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.267+0000 7f4aa3511640 1 -- 172.21.15.50:0/3038845679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db5e0f20c0 con 0x55db5e0f41c0 2024-05-08T18:06:09.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.267+0000 7f4aa3511640 1 --2- 172.21.15.50:0/3038845679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 0x55db5e0f3b00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4a94055040 tx=0x7f4a9404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bbe343d79b74272 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.267+0000 7f499160a640 1 -- 172.21.15.50:0/3038845679 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a94057040 con 0x55db5e0f41c0 2024-05-08T18:06:09.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.267+0000 7f499160a640 1 -- 172.21.15.50:0/3038845679 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a94052720 con 0x55db5e0f41c0 2024-05-08T18:06:09.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.267+0000 7f499160a640 1 -- 172.21.15.50:0/3038845679 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a94052a40 con 0x55db5e0f41c0 2024-05-08T18:06:09.274 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.268+0000 7f4aa5a19800 1 -- 172.21.15.50:0/3038845679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 msgr2=0x55db5e0f3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.268+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/3038845679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 0x55db5e0f3b00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4a94055040 tx=0x7f4a9404c000 comp rx=0 tx=0).stop 2024-05-08T18:06:09.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.268+0000 7f4aa5a19800 1 -- 172.21.15.50:0/3038845679 shutdown_connections 2024-05-08T18:06:09.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.268+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/3038845679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 0x55db5e0f3b00 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.268+0000 7f4aa5a19800 1 -- 172.21.15.50:0/3038845679 >> 172.21.15.50:0/3038845679 conn(0x55db5e0edaf0 msgr2=0x55db5e1e0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.268+0000 7f4aa5a19800 1 -- 172.21.15.50:0/3038845679 shutdown_connections 2024-05-08T18:06:09.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.268+0000 7f4aa5a19800 1 -- 172.21.15.50:0/3038845679 wait complete. 2024-05-08T18:06:09.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.268+0000 7f4aa5a19800 1 Processor -- start 2024-05-08T18:06:09.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.268+0000 7f4aa5a19800 1 -- start start 2024-05-08T18:06:09.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.269+0000 7f4aa5a19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 0x55db5e1ee810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.269+0000 7f4aa5a19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55db5e0faa70 con 0x55db5e0f41c0 2024-05-08T18:06:09.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.269+0000 7f4aa3511640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 0x55db5e1ee810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.269+0000 7f4aa3511640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 0x55db5e1ee810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50598/0 (socket says 172.21.15.50:50598) 2024-05-08T18:06:09.275 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.269+0000 7f4aa3511640 1 -- 172.21.15.50:0/102434307 learned_addr learned my addr 172.21.15.50:0/102434307 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.269+0000 7f4aa3511640 1 -- 172.21.15.50:0/102434307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db5e1eed50 con 0x55db5e0f41c0 2024-05-08T18:06:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.269+0000 7f4aa3511640 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 0x55db5e1ee810 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4a94052540 tx=0x7f4a94003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.270+0000 7f498fe07640 1 -- 172.21.15.50:0/102434307 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a94073070 con 0x55db5e0f41c0 2024-05-08T18:06:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.270+0000 7f498fe07640 1 -- 172.21.15.50:0/102434307 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a9405d440 con 0x55db5e0f41c0 2024-05-08T18:06:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.270+0000 7f4aa5a19800 1 -- 172.21.15.50:0/102434307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55db5e1eefe0 con 0x55db5e0f41c0 2024-05-08T18:06:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.270+0000 7f498fe07640 1 -- 172.21.15.50:0/102434307 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a94057070 con 0x55db5e0f41c0 2024-05-08T18:06:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.270+0000 7f4aa5a19800 1 -- 172.21.15.50:0/102434307 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db5e1f1c90 con 0x55db5e0f41c0 2024-05-08T18:06:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.271+0000 7f498fe07640 1 -- 172.21.15.50:0/102434307 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f4a94076070 con 0x55db5e0f41c0 2024-05-08T18:06:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.271+0000 7f498fe07640 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f498803bb50 0x7f498803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.271+0000 7f498fe07640 1 -- 172.21.15.50:0/102434307 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f4a940940e0 con 0x55db5e0f41c0 2024-05-08T18:06:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.271+0000 7f4aa2d10640 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f498803bb50 0x7f498803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.271+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55db5e0f6af0 0x55db5e1f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.271+0000 7f4aa5a19800 1 -- 172.21.15.50:0/102434307 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e1f5fd0 con 0x55db5e0f6af0 2024-05-08T18:06:09.276 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.271+0000 7f4aa3d12640 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55db5e0f6af0 0x55db5e1f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.271+0000 7f4aa2d10640 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f498803bb50 0x7f498803e010 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4a98001e50 tx=0x7f4a980013d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.272+0000 7f4aa3d12640 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55db5e0f6af0 0x55db5e1f5a50 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.272+0000 7f4aa3d12640 1 -- 172.21.15.50:0/102434307 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4a9c00a4a0 con 0x55db5e0f6af0 2024-05-08T18:06:09.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.272+0000 7f4aa5a19800 1 -- 172.21.15.50:0/102434307 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e1f68c0 con 0x55db5e0f6af0 2024-05-08T18:06:09.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.272+0000 7f4aa3d12640 1 -- 172.21.15.50:0/102434307 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4a9c00a4a0 con 0x55db5e0f6af0 2024-05-08T18:06:09.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.272+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55db5e1f81d0 0x55db5e1fa660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.272+0000 7f4aa5a19800 1 -- 172.21.15.50:0/102434307 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e1fadf0 con 0x55db5e1f81d0 2024-05-08T18:06:09.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.273+0000 7f4aa3511640 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55db5e1f81d0 0x55db5e1fa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.273+0000 7f4aa3511640 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55db5e1f81d0 0x55db5e1fa660 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.273+0000 7f4aa3511640 1 -- 172.21.15.50:0/102434307 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f4a94075030 con 0x55db5e1f81d0 2024-05-08T18:06:09.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.273+0000 7f4aa5a19800 1 -- 172.21.15.50:0/102434307 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e1f7110 con 0x55db5e0f6af0 2024-05-08T18:06:09.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.274+0000 7f4aa3d12640 1 -- 172.21.15.50:0/102434307 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4a9c00a4a0 con 0x55db5e0f6af0 2024-05-08T18:06:09.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.274+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55db5e1fc250 0x55db5e1fe640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.274+0000 7f4aa5a19800 1 -- 172.21.15.50:0/102434307 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e1fed30 con 0x55db5e1fc250 2024-05-08T18:06:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.274+0000 7f4aa2d10640 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55db5e1fc250 0x55db5e1fe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.274+0000 7f4aa2d10640 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55db5e1fc250 0x55db5e1fe640 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.274+0000 7f4aa2d10640 1 -- 172.21.15.50:0/102434307 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4a98010600 con 0x55db5e1fc250 2024-05-08T18:06:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.275+0000 7f4aa5a19800 1 Processor -- start 2024-05-08T18:06:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.275+0000 7f4aa5a19800 1 -- start start 2024-05-08T18:06:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.275+0000 7f4aa5a19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e20d0a0 0x55db5e20d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.275+0000 7f4aa5a19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55db5dfe5330 con 0x55db5e20d0a0 2024-05-08T18:06:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.275+0000 7f4aa3511640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e20d0a0 0x55db5e20d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.275+0000 7f4aa3511640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e20d0a0 0x55db5e20d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50614/0 (socket says 172.21.15.50:50614) 2024-05-08T18:06:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.275+0000 7f4aa3511640 1 -- 172.21.15.50:0/2074652684 learned_addr learned my addr 172.21.15.50:0/2074652684 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.276+0000 7f4aa3511640 1 -- 172.21.15.50:0/2074652684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db5e20da00 con 0x55db5e20d0a0 2024-05-08T18:06:09.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.276+0000 7f4aa3511640 1 --2- 172.21.15.50:0/2074652684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e20d0a0 0x55db5e20d4c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4a940923d0 tx=0x7f4a9406e6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.276+0000 7f498d602640 1 -- 172.21.15.50:0/2074652684 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a94083040 con 0x55db5e20d0a0 2024-05-08T18:06:09.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.276+0000 7f498d602640 1 -- 172.21.15.50:0/2074652684 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a94076860 con 0x55db5e20d0a0 2024-05-08T18:06:09.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.276+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2074652684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55db5e20e470 con 0x55db5e20d0a0 2024-05-08T18:06:09.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.276+0000 7f498d602640 1 -- 172.21.15.50:0/2074652684 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a94066a90 con 0x55db5e20d0a0 2024-05-08T18:06:09.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.276+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2074652684 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db5e20e970 con 0x55db5e20d0a0 2024-05-08T18:06:09.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.277+0000 7f498d602640 1 -- 172.21.15.50:0/2074652684 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f4a94066c30 con 0x55db5e20d0a0 2024-05-08T18:06:09.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.277+0000 7f498d602640 1 --2- 172.21.15.50:0/2074652684 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f497c03bb50 0x7f497c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.277+0000 7f498d602640 1 -- 172.21.15.50:0/2074652684 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f4a94095d20 con 0x55db5e20d0a0 2024-05-08T18:06:09.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.277+0000 7f4aa2d10640 1 --2- 172.21.15.50:0/2074652684 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f497c03bb50 0x7f497c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.278+0000 7f4aa5a19800 1 Processor -- start 2024-05-08T18:06:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.278+0000 7f4aa5a19800 1 -- start start 2024-05-08T18:06:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.278+0000 7f4aa2d10640 1 --2- 172.21.15.50:0/2074652684 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f497c03bb50 0x7f497c03e010 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55db5e1f2f40 tx=0x7f4a98006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.278+0000 7f4aa5a19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e21a230 0x55db5e21a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.278+0000 7f4aa5a19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55db5e0f2380 con 0x55db5e21a230 2024-05-08T18:06:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.278+0000 7f4aa3511640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e21a230 0x55db5e21a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.278+0000 7f4aa3511640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e21a230 0x55db5e21a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50624/0 (socket says 172.21.15.50:50624) 2024-05-08T18:06:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.278+0000 7f4aa3511640 1 -- 172.21.15.50:0/960726640 learned_addr learned my addr 172.21.15.50:0/960726640 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.278+0000 7f4aa3511640 1 -- 172.21.15.50:0/960726640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db5e21ac20 con 0x55db5e21a230 2024-05-08T18:06:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.278+0000 7f4aa3511640 1 --2- 172.21.15.50:0/960726640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e21a230 0x55db5e21a650 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4a94008340 tx=0x7f4a94078810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc64868aa5eb77bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.279+0000 7f4986ffd640 1 -- 172.21.15.50:0/960726640 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a940774e0 con 0x55db5e21a230 2024-05-08T18:06:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.279+0000 7f4986ffd640 1 -- 172.21.15.50:0/960726640 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a94077730 con 0x55db5e21a230 2024-05-08T18:06:09.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.279+0000 7f4986ffd640 1 -- 172.21.15.50:0/960726640 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a94098170 con 0x55db5e21a230 2024-05-08T18:06:09.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.279+0000 7f4aa5a19800 1 -- 172.21.15.50:0/960726640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e21a230 msgr2=0x55db5e21a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.279+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/960726640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e21a230 0x55db5e21a650 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4a94008340 tx=0x7f4a94078810 comp rx=0 tx=0).stop 2024-05-08T18:06:09.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.279+0000 7f4aa5a19800 1 -- 172.21.15.50:0/960726640 shutdown_connections 2024-05-08T18:06:09.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.279+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/960726640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e21a230 0x55db5e21a650 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.279+0000 7f4aa5a19800 1 -- 172.21.15.50:0/960726640 >> 172.21.15.50:0/960726640 conn(0x55db5e216540 msgr2=0x55db5e2145a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.279+0000 7f4aa5a19800 1 -- 172.21.15.50:0/960726640 shutdown_connections 2024-05-08T18:06:09.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.279+0000 7f4aa5a19800 1 -- 172.21.15.50:0/960726640 wait complete. 2024-05-08T18:06:09.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.279+0000 7f4aa5a19800 1 Processor -- start 2024-05-08T18:06:09.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.280+0000 7f4aa5a19800 1 -- start start 2024-05-08T18:06:09.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.280+0000 7f4aa5a19800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e21a230 0x55db5e221630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.280+0000 7f4aa5a19800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55db5e21b2b0 con 0x55db5e21a230 2024-05-08T18:06:09.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.280+0000 7f4aa3511640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e21a230 0x55db5e221630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.280+0000 7f4aa3511640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e21a230 0x55db5e221630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50632/0 (socket says 172.21.15.50:50632) 2024-05-08T18:06:09.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.280+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 learned_addr learned my addr 172.21.15.50:0/2296335280 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.280+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db5e221b70 con 0x55db5e21a230 2024-05-08T18:06:09.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.280+0000 7f4aa3511640 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e21a230 0x55db5e221630 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4a940082a0 tx=0x7f4a94053260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.281+0000 7f49857fa640 1 -- 172.21.15.50:0/2296335280 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a94088b30 con 0x55db5e21a230 2024-05-08T18:06:09.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.281+0000 7f49857fa640 1 -- 172.21.15.50:0/2296335280 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a940b7070 con 0x55db5e21a230 2024-05-08T18:06:09.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.281+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55db5e221b70 con 0x55db5e21a230 2024-05-08T18:06:09.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.281+0000 7f49857fa640 1 -- 172.21.15.50:0/2296335280 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a940896e0 con 0x55db5e21a230 2024-05-08T18:06:09.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.281+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db5e221b70 con 0x55db5e21a230 2024-05-08T18:06:09.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.281+0000 7f49857fa640 1 -- 172.21.15.50:0/2296335280 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f4a940788c0 con 0x55db5e21a230 2024-05-08T18:06:09.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.282+0000 7f49857fa640 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f496c03bb50 0x7f496c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.282+0000 7f49857fa640 1 -- 172.21.15.50:0/2296335280 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f4a940ad140 con 0x55db5e21a230 2024-05-08T18:06:09.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.282+0000 7f4aa2d10640 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f496c03bb50 0x7f496c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.282+0000 7f4aa5a19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.282+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55db5e22cd70 0x55db5e22f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.282+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e22f980 con 0x55db5e22cd70 2024-05-08T18:06:09.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.282+0000 7f4aa3d12640 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55db5e22cd70 0x55db5e22f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.282+0000 7f4aa2d10640 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f496c03bb50 0x7f496c03e010 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800bf70 tx=0x7f4a98011000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.283+0000 7f4aa3d12640 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55db5e22cd70 0x55db5e22f230 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.283+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4a9c00a4a0 con 0x55db5e22cd70 2024-05-08T18:06:09.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.283+0000 7f4aa5a19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.283+0000 7f4aa5a19800 20 realm 2024-05-08T18:06:09.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.283+0000 7f4aa5a19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.283+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e22fd90 con 0x55db5e22cd70 2024-05-08T18:06:09.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.283+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4a9c00a4a0 con 0x55db5e22cd70 2024-05-08T18:06:09.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.283+0000 7f4aa5a19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.283+0000 7f4aa5a19800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:09.300 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.283+0000 7f4aa5a19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.283+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230660 con 0x55db5e22cd70 2024-05-08T18:06:09.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.283+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4a9c00a4a0 con 0x55db5e22cd70 2024-05-08T18:06:09.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.283+0000 7f4aa5a19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.283+0000 7f4aa5a19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.283+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e22cd70 2024-05-08T18:06:09.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.284+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4a9c00a4a0 con 0x55db5e22cd70 2024-05-08T18:06:09.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.284+0000 7f4aa5a19800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:09.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.284+0000 7f4aa5a19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.284+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55db5e231760 0x55db5e233b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.284+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e234160 con 0x55db5e231760 2024-05-08T18:06:09.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.284+0000 7f4aa3511640 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55db5e231760 0x55db5e233b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.284+0000 7f4aa3511640 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55db5e231760 0x55db5e233b50 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.301 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.285+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.285+0000 7f4aa5a19800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:09.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.285+0000 7f4aa5a19800 20 searching for the correct realm 2024-05-08T18:06:09.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.285+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55db5e236130 0x55db5e2385f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.285+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e238cc0 con 0x55db5e236130 2024-05-08T18:06:09.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.285+0000 7f4aa2d10640 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55db5e236130 0x55db5e2385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.285+0000 7f4aa2d10640 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55db5e236130 0x55db5e2385f0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.286+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a98036040 con 0x55db5e236130 2024-05-08T18:06:09.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.286+0000 7f4aa2d10640 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f4a980393c0 0x7f4a98039820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.286+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e238cc0 con 0x7f4a980393c0 2024-05-08T18:06:09.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.286+0000 7f4aa3d12640 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f4a980393c0 0x7f4a98039820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.286+0000 7f4aa3d12640 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f4a980393c0 0x7f4a98039820 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.286+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9c00a4a0 con 0x7f4a980393c0 2024-05-08T18:06:09.302 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.286+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e231760 2024-05-08T18:06:09.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.286+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.286+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e22cd70 2024-05-08T18:06:09.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.287+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9c00a4a0 con 0x55db5e22cd70 2024-05-08T18:06:09.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.287+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e22cd70 2024-05-08T18:06:09.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.287+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9c00a4a0 con 0x55db5e22cd70 2024-05-08T18:06:09.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.287+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e22cd70 2024-05-08T18:06:09.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.287+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9c00a4a0 con 0x55db5e22cd70 2024-05-08T18:06:09.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.287+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e231760 2024-05-08T18:06:09.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.288+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.288+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x7f4a980393c0 2024-05-08T18:06:09.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.288+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9c00a4a0 con 0x7f4a980393c0 2024-05-08T18:06:09.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.288+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e236130 2024-05-08T18:06:09.303 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.288+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a98036040 con 0x55db5e236130 2024-05-08T18:06:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.288+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e22cd70 2024-05-08T18:06:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.288+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9c00a4a0 con 0x55db5e22cd70 2024-05-08T18:06:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.288+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e231760 2024-05-08T18:06:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.289+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.289+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e22cd70 2024-05-08T18:06:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.289+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9c00a4a0 con 0x55db5e22cd70 2024-05-08T18:06:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.289+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x7f4a980393c0 2024-05-08T18:06:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.289+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f4a9c032040 con 0x7f4a980393c0 2024-05-08T18:06:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.289+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e236130 2024-05-08T18:06:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.289+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a98036040 con 0x55db5e236130 2024-05-08T18:06:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.289+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e236130 2024-05-08T18:06:09.304 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.290+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a98036040 con 0x55db5e236130 2024-05-08T18:06:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.290+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e231760 2024-05-08T18:06:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.290+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.290+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e236130 2024-05-08T18:06:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.290+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a98036040 con 0x55db5e236130 2024-05-08T18:06:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.290+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e236130 2024-05-08T18:06:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.290+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f4a98036040 con 0x55db5e236130 2024-05-08T18:06:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.290+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e22cd70 2024-05-08T18:06:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.291+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9c032040 con 0x55db5e22cd70 2024-05-08T18:06:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.291+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e22cd70 2024-05-08T18:06:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.291+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f4a9c032040 con 0x55db5e22cd70 2024-05-08T18:06:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.291+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e231760 2024-05-08T18:06:09.305 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.291+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.291+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e236130 2024-05-08T18:06:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.291+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a98036040 con 0x55db5e236130 2024-05-08T18:06:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.291+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x7f4a980393c0 2024-05-08T18:06:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.292+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9c032040 con 0x7f4a980393c0 2024-05-08T18:06:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.292+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e22cd70 2024-05-08T18:06:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.292+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9c032040 con 0x55db5e22cd70 2024-05-08T18:06:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.292+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e231760 2024-05-08T18:06:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.292+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.292+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e236130 2024-05-08T18:06:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.292+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a98036040 con 0x55db5e236130 2024-05-08T18:06:09.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.292+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e231760 2024-05-08T18:06:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.293+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.293+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x7f4a980393c0 2024-05-08T18:06:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.293+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9c032040 con 0x7f4a980393c0 2024-05-08T18:06:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.293+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e236130 2024-05-08T18:06:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.293+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a98036040 con 0x55db5e236130 2024-05-08T18:06:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.293+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e231760 2024-05-08T18:06:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x55db5e22cd70 2024-05-08T18:06:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f4a9c032040 con 0x55db5e22cd70 2024-05-08T18:06:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55db5e230a00 con 0x7f4a980393c0 2024-05-08T18:06:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4a9c032040 con 0x7f4a980393c0 2024-05-08T18:06:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa5a19800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa5a19800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:09.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa5a19800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:09.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa5a19800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:09.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa5a19800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:09.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa5a19800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:09.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa5a19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e234a20 con 0x55db5e22cd70 2024-05-08T18:06:09.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4a9c032040 con 0x55db5e22cd70 2024-05-08T18:06:09.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa5a19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa5a19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.294+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e234a20 con 0x55db5e22cd70 2024-05-08T18:06:09.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4a9c032040 con 0x55db5e22cd70 2024-05-08T18:06:09.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa5a19800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:09.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa5a19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e234a20 con 0x55db5e236130 2024-05-08T18:06:09.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f4a98036040 con 0x55db5e236130 2024-05-08T18:06:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa5a19800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa5a19800 20 zone default found 2024-05-08T18:06:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa5a19800 4 Realm: () 2024-05-08T18:06:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa5a19800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa5a19800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa5a19800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa5a19800 20 zonegroup default 2024-05-08T18:06:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa5a19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e2353f0 con 0x55db5e231760 2024-05-08T18:06:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa5a19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa5a19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.295+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e234f90 con 0x55db5e236130 2024-05-08T18:06:09.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.296+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4a98036040 con 0x55db5e236130 2024-05-08T18:06:09.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.296+0000 7f4aa5a19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.296+0000 7f4aa5a19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.296+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23b0d0 con 0x55db5e236130 2024-05-08T18:06:09.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.296+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4a98036040 con 0x55db5e236130 2024-05-08T18:06:09.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.296+0000 7f4aa5a19800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.296+0000 7f4aa5a19800 20 started sync module instance, tier type = 2024-05-08T18:06:09.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.296+0000 7f4aa5a19800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:09.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.296+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23a460 con 0x7f4a980393c0 2024-05-08T18:06:09.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.297+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9c032040 con 0x7f4a980393c0 2024-05-08T18:06:09.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.297+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94400665594208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23a460 con 0x7f4a980393c0 2024-05-08T18:06:09.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.297+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23bb90 con 0x55db5e22cd70 2024-05-08T18:06:09.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.298+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94400665594208] v29'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9c032040 con 0x7f4a980393c0 2024-05-08T18:06:09.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.298+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9c032040 con 0x55db5e22cd70 2024-05-08T18:06:09.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.298+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94400665597600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23a460 con 0x55db5e22cd70 2024-05-08T18:06:09.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.298+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23c610 con 0x55db5e236130 2024-05-08T18:06:09.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.299+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94400665597600] v29'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9c032040 con 0x55db5e22cd70 2024-05-08T18:06:09.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.300+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a98036040 con 0x55db5e236130 2024-05-08T18:06:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.300+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94400665591728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23a3d0 con 0x55db5e236130 2024-05-08T18:06:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.300+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23d500 con 0x7f4a980393c0 2024-05-08T18:06:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.301+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94400665591728] v29'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a98036040 con 0x55db5e236130 2024-05-08T18:06:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.301+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9c032040 con 0x7f4a980393c0 2024-05-08T18:06:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.301+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94400665604176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23d470 con 0x7f4a980393c0 2024-05-08T18:06:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.301+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23e3c0 con 0x55db5e22cd70 2024-05-08T18:06:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.302+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94400665604176] v29'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9c032040 con 0x7f4a980393c0 2024-05-08T18:06:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.302+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9c032040 con 0x55db5e22cd70 2024-05-08T18:06:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.302+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94400665606304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23e3c0 con 0x55db5e22cd70 2024-05-08T18:06:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.302+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23ed80 con 0x55db5e231760 2024-05-08T18:06:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.303+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.303+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94400665610448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23f5d0 con 0x55db5e231760 2024-05-08T18:06:09.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.303+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e240050 con 0x55db5e231760 2024-05-08T18:06:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.303+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94400665606304] v29'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9c032040 con 0x55db5e22cd70 2024-05-08T18:06:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.304+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(56 notify.6 [create] v29'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.304+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94400665610448] v29'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.304+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94400665607952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e2408a0 con 0x55db5e231760 2024-05-08T18:06:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.304+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23f120 con 0x55db5e231760 2024-05-08T18:06:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.305+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94400665607952] v29'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.305+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.305+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94400665617392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e241260 con 0x55db5e231760 2024-05-08T18:06:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.305+0000 7f4aa5a19800 20 add_watcher() i=0 2024-05-08T18:06:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.305+0000 7f4aa5a19800 20 add_watcher() i=1 2024-05-08T18:06:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.305+0000 7f4aa5a19800 20 add_watcher() i=2 2024-05-08T18:06:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.305+0000 7f4aa5a19800 20 add_watcher() i=3 2024-05-08T18:06:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.305+0000 7f4aa5a19800 20 add_watcher() i=4 2024-05-08T18:06:09.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.305+0000 7f4aa5a19800 20 add_watcher() i=5 2024-05-08T18:06:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.305+0000 7f4aa5a19800 20 add_watcher() i=6 2024-05-08T18:06:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.306+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94400665617392] v29'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.306+0000 7f4aa5a19800 20 add_watcher() i=7 2024-05-08T18:06:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.306+0000 7f4aa5a19800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.306+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e2443d0 con 0x55db5e22cd70 2024-05-08T18:06:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.307+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f4a9c032040 con 0x55db5e22cd70 2024-05-08T18:06:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.307+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94400665619888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e241680 con 0x55db5e22cd70 2024-05-08T18:06:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.308+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94400665619888] v29'216 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4a9c032040 con 0x55db5e22cd70 2024-05-08T18:06:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.308+0000 7f4959fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.308+0000 7f4aa5a19800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.308+0000 7f4aa5a19800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.308+0000 7f4aa5a19800 5 note: GC not initialized 2024-05-08T18:06:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.308+0000 7f4958fe1640 20 reqs_thread_entry: start 2024-05-08T18:06:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.308+0000 7f4aa5a19800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:09.324 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.308+0000 7f4aa5a19800 20 Filter name: none 2024-05-08T18:06:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.308+0000 7f48d27fc640 20 reqs_thread_entry: start 2024-05-08T18:06:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.308+0000 7f4aa5a19800 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-05-08T18:06:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.308+0000 7f4aa5a19800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.308+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:62 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55db5e277950 con 0x7f4a980393c0 2024-05-08T18:06:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.309+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(62 bud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f4a9c032040 con 0x7f4a980393c0 2024-05-08T18:06:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.309+0000 7f4aa5a19800 20 rados_obj.operate() r=0 bl.length=380 2024-05-08T18:06:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.309+0000 7f4aa5a19800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2024-05-08T18:06:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.309+0000 7f4aa5a19800 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-05-08T18:06:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.309+0000 7f4aa5a19800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.309+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.e 8:75f48285:users.uid::fud:head [create,call version.set in=58b,writefull 0~206 in=206b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e27bce0 con 0x55db5e231760 2024-05-08T18:06:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.310+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(63 fud [create,call,writefull 0~206] v29'6 uv5 ondisk = -17 ((17) File exists)) ==== 231+0+0 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:could not rename user: unable to rename user, user name given by --new-uid already exists. to overwrite this user, add --yes-i-really-mean-it 2024-05-08T18:06:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.311+0000 7f4959fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:09.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.311+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94400665619888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e22a400 con 0x55db5e22cd70 2024-05-08T18:06:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.312+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94400665619888] v29'217 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4a9c032040 con 0x55db5e22cd70 2024-05-08T18:06:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.312+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94400665594208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e240db0 con 0x7f4a980393c0 2024-05-08T18:06:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.313+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94400665594208] v29'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9c032040 con 0x7f4a980393c0 2024-05-08T18:06:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.313+0000 7f4aa5a19800 20 remove_watcher() i=0 2024-05-08T18:06:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.313+0000 7f4aa5a19800 2 removed watcher, disabling cache 2024-05-08T18:06:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.313+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94400665597600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e240db0 con 0x55db5e22cd70 2024-05-08T18:06:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.314+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94400665597600] v29'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9c032040 con 0x55db5e22cd70 2024-05-08T18:06:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.314+0000 7f4aa5a19800 20 remove_watcher() i=1 2024-05-08T18:06:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.314+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94400665591728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23b6a0 con 0x55db5e236130 2024-05-08T18:06:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.315+0000 7f4aa2d10640 1 -- 172.21.15.50:0/2296335280 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94400665591728] v29'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a98036040 con 0x55db5e236130 2024-05-08T18:06:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.315+0000 7f4aa5a19800 20 remove_watcher() i=2 2024-05-08T18:06:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.315+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94400665604176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e240db0 con 0x7f4a980393c0 2024-05-08T18:06:09.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.316+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94400665604176] v29'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9c032040 con 0x7f4a980393c0 2024-05-08T18:06:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.316+0000 7f4aa5a19800 20 remove_watcher() i=3 2024-05-08T18:06:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.316+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94400665606304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e239fb0 con 0x55db5e22cd70 2024-05-08T18:06:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.317+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94400665606304] v29'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9c032040 con 0x55db5e22cd70 2024-05-08T18:06:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.317+0000 7f4aa5a19800 20 remove_watcher() i=4 2024-05-08T18:06:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.318+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94400665610448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23b6a0 con 0x55db5e231760 2024-05-08T18:06:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.319+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94400665610448] v29'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.319+0000 7f4aa5a19800 20 remove_watcher() i=5 2024-05-08T18:06:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.319+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94400665607952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e23d050 con 0x55db5e231760 2024-05-08T18:06:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.320+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94400665607952] v29'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.320+0000 7f4aa5a19800 20 remove_watcher() i=6 2024-05-08T18:06:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.320+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94400665617392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55db5e240db0 con 0x55db5e231760 2024-05-08T18:06:09.327 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa3511640 1 -- 172.21.15.50:0/2296335280 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94400665617392] v29'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4a9409b070 con 0x55db5e231760 2024-05-08T18:06:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 20 remove_watcher() i=7 2024-05-08T18:06:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55db5e231760 msgr2=0x55db5e233b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55db5e231760 0x55db5e233b50 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55db5e22cd70 msgr2=0x55db5e22f230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55db5e22cd70 0x55db5e22f230 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f4a980393c0 msgr2=0x7f4a98039820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f4a980393c0 0x7f4a98039820 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55db5e236130 msgr2=0x55db5e2385f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55db5e236130 0x55db5e2385f0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f496c03bb50 msgr2=0x7f496c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f496c03bb50 0x7f496c03e010 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800bf70 tx=0x7f4a98011000 comp rx=0 tx=0).stop 2024-05-08T18:06:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e21a230 msgr2=0x55db5e221630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e21a230 0x55db5e221630 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4a940082a0 tx=0x7f4a94053260 comp rx=0 tx=0).stop 2024-05-08T18:06:09.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa3d12640 1 -- 172.21.15.50:0/2296335280 reap_dead start 2024-05-08T18:06:09.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 shutdown_connections 2024-05-08T18:06:09.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/2296335280 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e21a230 0x55db5e221630 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 >> 172.21.15.50:0/2296335280 conn(0x55db5e216540 msgr2=0x55db5e213ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 shutdown_connections 2024-05-08T18:06:09.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2296335280 wait complete. 2024-05-08T18:06:09.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2074652684 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f497c03bb50 msgr2=0x7f497c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/2074652684 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f497c03bb50 0x7f497c03e010 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55db5e1f2f40 tx=0x7f4a98006000 comp rx=0 tx=0).stop 2024-05-08T18:06:09.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2074652684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e20d0a0 msgr2=0x55db5e20d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.321+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/2074652684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e20d0a0 0x55db5e20d4c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f4a940923d0 tx=0x7f4a9406e6f0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2074652684 shutdown_connections 2024-05-08T18:06:09.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/2074652684 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f497c03bb50 0x7f497c03e010 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/2074652684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e20d0a0 0x55db5e20d4c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2074652684 >> 172.21.15.50:0/2074652684 conn(0x55db5e201140 msgr2=0x55db5e201540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2074652684 shutdown_connections 2024-05-08T18:06:09.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 -- 172.21.15.50:0/2074652684 wait complete. 2024-05-08T18:06:09.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 -- 172.21.15.50:0/102434307 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55db5e1f81d0 msgr2=0x55db5e1fa660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55db5e1f81d0 0x55db5e1fa660 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 -- 172.21.15.50:0/102434307 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55db5e0f6af0 msgr2=0x55db5e1f5a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55db5e0f6af0 0x55db5e1f5a50 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 -- 172.21.15.50:0/102434307 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55db5e1fc250 msgr2=0x55db5e1fe640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55db5e1fc250 0x55db5e1fe640 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 -- 172.21.15.50:0/102434307 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f498803bb50 msgr2=0x7f498803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f498803bb50 0x7f498803e010 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4a98001e50 tx=0x7f4a980013d0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 -- 172.21.15.50:0/102434307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 msgr2=0x55db5e1ee810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 --2- 172.21.15.50:0/102434307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55db5e0f41c0 0x55db5e1ee810 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4a94052540 tx=0x7f4a94003020 comp rx=0 tx=0).stop 2024-05-08T18:06:09.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa3d12640 1 -- 172.21.15.50:0/102434307 reap_dead start 2024-05-08T18:06:09.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 -- 172.21.15.50:0/102434307 shutdown_connections 2024-05-08T18:06:09.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 -- 172.21.15.50:0/102434307 >> 172.21.15.50:0/102434307 conn(0x55db5e0edaf0 msgr2=0x55db5e0f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 -- 172.21.15.50:0/102434307 shutdown_connections 2024-05-08T18:06:09.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.322+0000 7f4aa5a19800 1 -- 172.21.15.50:0/102434307 wait complete. 2024-05-08T18:06:09.331 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-05-08T18:06:09.378 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:06:09.379 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2024-05-08T18:06:09.383 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2024-05-08T18:06:09.383 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=8 failed=False 2024-05-08T18:06:09.383 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud', '--purge-data'] 2024-05-08T18:06:09.383 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud', '--purge-data'] 2024-05-08T18:06:09.383 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud --purge-data 2024-05-08T18:06:09.430 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:09.430 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:09.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.433+0000 7f19fd936800 1 Processor -- start 2024-05-08T18:06:09.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.433+0000 7f19fd936800 1 -- start start 2024-05-08T18:06:09.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.433+0000 7f19fd936800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x558f08ed4a00 0x558f08ed6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.433+0000 7f19fd936800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x558f08ed73c0 con 0x558f08ed0570 2024-05-08T18:06:09.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.433+0000 7f19fd936800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x558f08d71e20 con 0x558f08ed4a00 2024-05-08T18:06:09.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.433+0000 7f19f0e2e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x558f08ed4a00 0x558f08ed6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.433+0000 7f19f962f640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x558f08ed0570 0x558f08ed0990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:55710/0 (socket says 172.21.15.50:55710) 2024-05-08T18:06:09.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.433+0000 7f19f0e2e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x558f08ed4a00 0x558f08ed6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50644/0 (socket says 172.21.15.50:50644) 2024-05-08T18:06:09.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.433+0000 7f19f962f640 1 -- 172.21.15.50:0/4188379658 learned_addr learned my addr 172.21.15.50:0/4188379658 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:09.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.434+0000 7f19f0e2e640 1 -- 172.21.15.50:0/4188379658 >> v1:172.21.15.46:6789/0 conn(0x558f08ed0570 legacy=0x558f08ed0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.434+0000 7f19f0e2e640 1 -- 172.21.15.50:0/4188379658 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f08ed85f0 con 0x558f08ed4a00 2024-05-08T18:06:09.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.436+0000 7f19f0e2e640 1 --2- 172.21.15.50:0/4188379658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed4a00 0x558f08ed6df0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f19e400a470 tx=0x7f19e401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d73740a97c3daf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.436+0000 7f19f8e2e640 1 -- 172.21.15.50:0/4188379658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19e4057070 con 0x558f08ed4a00 2024-05-08T18:06:09.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.436+0000 7f19f8e2e640 1 -- 172.21.15.50:0/4188379658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e4052570 con 0x558f08ed4a00 2024-05-08T18:06:09.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.436+0000 7f19f8e2e640 1 -- 172.21.15.50:0/4188379658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19e4052870 con 0x558f08ed4a00 2024-05-08T18:06:09.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.436+0000 7f19fd936800 1 -- 172.21.15.50:0/4188379658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed4a00 msgr2=0x558f08ed6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.436+0000 7f19fd936800 1 --2- 172.21.15.50:0/4188379658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed4a00 0x558f08ed6df0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f19e400a470 tx=0x7f19e401ce90 comp rx=0 tx=0).stop 2024-05-08T18:06:09.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.436+0000 7f19fd936800 1 -- 172.21.15.50:0/4188379658 shutdown_connections 2024-05-08T18:06:09.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.436+0000 7f19fd936800 1 --2- 172.21.15.50:0/4188379658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed4a00 0x558f08ed6df0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.436+0000 7f19fd936800 1 -- 172.21.15.50:0/4188379658 >> 172.21.15.50:0/4188379658 conn(0x558f08ecbb70 msgr2=0x558f08ecdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.436+0000 7f19fd936800 1 -- 172.21.15.50:0/4188379658 shutdown_connections 2024-05-08T18:06:09.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.436+0000 7f19fd936800 1 -- 172.21.15.50:0/4188379658 wait complete. 2024-05-08T18:06:09.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.452+0000 7f19fd936800 1 Processor -- start 2024-05-08T18:06:09.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.452+0000 7f19fd936800 1 -- start start 2024-05-08T18:06:09.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.452+0000 7f19fd936800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed0570 0x558f08e44550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.452+0000 7f19fd936800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558f08e1ca30 con 0x558f08ed0570 2024-05-08T18:06:09.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.452+0000 7f19f962f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed0570 0x558f08e44550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.452+0000 7f19f962f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed0570 0x558f08e44550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50648/0 (socket says 172.21.15.50:50648) 2024-05-08T18:06:09.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.452+0000 7f19f962f640 1 -- 172.21.15.50:0/1913298876 learned_addr learned my addr 172.21.15.50:0/1913298876 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.453+0000 7f19f962f640 1 -- 172.21.15.50:0/1913298876 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f08fc5830 con 0x558f08ed0570 2024-05-08T18:06:09.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.453+0000 7f19f962f640 1 --2- 172.21.15.50:0/1913298876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed0570 0x558f08e44550 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f19ec001e40 tx=0x7f19ec001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275b6b8428724bd3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.453+0000 7f18e7606640 1 -- 172.21.15.50:0/1913298876 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19ec012020 con 0x558f08ed0570 2024-05-08T18:06:09.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.453+0000 7f18e7606640 1 -- 172.21.15.50:0/1913298876 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ec00e5a0 con 0x558f08ed0570 2024-05-08T18:06:09.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.453+0000 7f18e7606640 1 -- 172.21.15.50:0/1913298876 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19ec00e8a0 con 0x558f08ed0570 2024-05-08T18:06:09.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.454+0000 7f19fd936800 1 -- 172.21.15.50:0/1913298876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed0570 msgr2=0x558f08e44550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.454+0000 7f19fd936800 1 --2- 172.21.15.50:0/1913298876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed0570 0x558f08e44550 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f19ec001e40 tx=0x7f19ec001e70 comp rx=0 tx=0).stop 2024-05-08T18:06:09.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.454+0000 7f19fd936800 1 -- 172.21.15.50:0/1913298876 shutdown_connections 2024-05-08T18:06:09.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.454+0000 7f19fd936800 1 --2- 172.21.15.50:0/1913298876 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed0570 0x558f08e44550 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.454+0000 7f19fd936800 1 -- 172.21.15.50:0/1913298876 >> 172.21.15.50:0/1913298876 conn(0x558f08ecbb70 msgr2=0x558f08fbd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.454+0000 7f19fd936800 1 -- 172.21.15.50:0/1913298876 shutdown_connections 2024-05-08T18:06:09.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.454+0000 7f19fd936800 1 -- 172.21.15.50:0/1913298876 wait complete. 2024-05-08T18:06:09.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.454+0000 7f19fd936800 1 Processor -- start 2024-05-08T18:06:09.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.454+0000 7f19fd936800 1 -- start start 2024-05-08T18:06:09.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.454+0000 7f19fd936800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed0570 0x558f08fca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.454+0000 7f19fd936800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558f08ed7890 con 0x558f08ed0570 2024-05-08T18:06:09.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.455+0000 7f19f962f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed0570 0x558f08fca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.455+0000 7f19f962f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed0570 0x558f08fca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50654/0 (socket says 172.21.15.50:50654) 2024-05-08T18:06:09.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.455+0000 7f19f962f640 1 -- 172.21.15.50:0/1114157385 learned_addr learned my addr 172.21.15.50:0/1114157385 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.455+0000 7f19f962f640 1 -- 172.21.15.50:0/1114157385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f08fcac20 con 0x558f08ed0570 2024-05-08T18:06:09.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.455+0000 7f19f962f640 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed0570 0x558f08fca6e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f19ec000c00 tx=0x7f19ec000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.455+0000 7f18e5e03640 1 -- 172.21.15.50:0/1114157385 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19ec012030 con 0x558f08ed0570 2024-05-08T18:06:09.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.455+0000 7f18e5e03640 1 -- 172.21.15.50:0/1114157385 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ec0014a0 con 0x558f08ed0570 2024-05-08T18:06:09.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.455+0000 7f18e5e03640 1 -- 172.21.15.50:0/1114157385 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19ec01a7a0 con 0x558f08ed0570 2024-05-08T18:06:09.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.455+0000 7f19fd936800 1 -- 172.21.15.50:0/1114157385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558f08fcaeb0 con 0x558f08ed0570 2024-05-08T18:06:09.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.455+0000 7f19fd936800 1 -- 172.21.15.50:0/1114157385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f08fcdb60 con 0x558f08ed0570 2024-05-08T18:06:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.456+0000 7f18e5e03640 1 -- 172.21.15.50:0/1114157385 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f19ec015020 con 0x558f08ed0570 2024-05-08T18:06:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.456+0000 7f18e5e03640 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f18e003bba0 0x7f18e003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.456+0000 7f18e5e03640 1 -- 172.21.15.50:0/1114157385 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f19ec020030 con 0x558f08ed0570 2024-05-08T18:06:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.457+0000 7f19f0e2e640 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f18e003bba0 0x7f18e003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.457+0000 7f19fd936800 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558f08ed4a00 0x558f08fd1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.457+0000 7f19fd936800 1 -- 172.21.15.50:0/1114157385 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f08fd1e60 con 0x558f08ed4a00 2024-05-08T18:06:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.457+0000 7f19f9e30640 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558f08ed4a00 0x558f08fd1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.457+0000 7f19f0e2e640 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f18e003bba0 0x7f18e003e060 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f19e4054000 tx=0x7f19e404e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.457+0000 7f19f9e30640 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558f08ed4a00 0x558f08fd1920 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.457+0000 7f19f9e30640 1 -- 172.21.15.50:0/1114157385 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f19f400a4a0 con 0x558f08ed4a00 2024-05-08T18:06:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.457+0000 7f19fd936800 1 -- 172.21.15.50:0/1114157385 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f08fceed0 con 0x558f08ed4a00 2024-05-08T18:06:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.458+0000 7f19f9e30640 1 -- 172.21.15.50:0/1114157385 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f19f400a4a0 con 0x558f08ed4a00 2024-05-08T18:06:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.458+0000 7f19fd936800 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558f08fd4180 0x558f08fd6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.458+0000 7f19fd936800 1 -- 172.21.15.50:0/1114157385 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f08fd6c60 con 0x558f08fd4180 2024-05-08T18:06:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.458+0000 7f19f962f640 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558f08fd4180 0x558f08fd6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.458+0000 7f19f962f640 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558f08fd4180 0x558f08fd6570 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.459+0000 7f19f962f640 1 -- 172.21.15.50:0/1114157385 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f19ec0430c0 con 0x558f08fd4180 2024-05-08T18:06:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.459+0000 7f19fd936800 1 -- 172.21.15.50:0/1114157385 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f08fd3020 con 0x558f08ed4a00 2024-05-08T18:06:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.459+0000 7f19f9e30640 1 -- 172.21.15.50:0/1114157385 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f19f400a4a0 con 0x558f08ed4a00 2024-05-08T18:06:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.459+0000 7f19fd936800 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558f08fd80c0 0x558f08fda4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.459+0000 7f19fd936800 1 -- 172.21.15.50:0/1114157385 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f08fdabb0 con 0x558f08fd80c0 2024-05-08T18:06:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.459+0000 7f19f0e2e640 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558f08fd80c0 0x558f08fda4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.460+0000 7f19f0e2e640 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558f08fd80c0 0x558f08fda4b0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.460+0000 7f19f0e2e640 1 -- 172.21.15.50:0/1114157385 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f19e4052a40 con 0x558f08fd80c0 2024-05-08T18:06:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.460+0000 7f19fd936800 1 Processor -- start 2024-05-08T18:06:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.460+0000 7f19fd936800 1 -- start start 2024-05-08T18:06:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.460+0000 7f19fd936800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08fc33c0 0x558f08fc1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.460+0000 7f19fd936800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558f08dc3330 con 0x558f08fc33c0 2024-05-08T18:06:09.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.461+0000 7f19f962f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08fc33c0 0x558f08fc1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.461+0000 7f19f962f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08fc33c0 0x558f08fc1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50658/0 (socket says 172.21.15.50:50658) 2024-05-08T18:06:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.461+0000 7f19f962f640 1 -- 172.21.15.50:0/3344152026 learned_addr learned my addr 172.21.15.50:0/3344152026 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.461+0000 7f19f962f640 1 -- 172.21.15.50:0/3344152026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f08fc1d70 con 0x558f08fc33c0 2024-05-08T18:06:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.461+0000 7f19f962f640 1 --2- 172.21.15.50:0/3344152026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08fc33c0 0x558f08fc1830 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f19ec050f30 tx=0x7f19ec00db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.461+0000 7f18df7fe640 1 -- 172.21.15.50:0/3344152026 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19ec049da0 con 0x558f08fc33c0 2024-05-08T18:06:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.461+0000 7f18df7fe640 1 -- 172.21.15.50:0/3344152026 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ec049f40 con 0x558f08fc33c0 2024-05-08T18:06:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.461+0000 7f18df7fe640 1 -- 172.21.15.50:0/3344152026 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19ec04a260 con 0x558f08fc33c0 2024-05-08T18:06:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.461+0000 7f19fd936800 1 -- 172.21.15.50:0/3344152026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558f08fc2000 con 0x558f08fc33c0 2024-05-08T18:06:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.461+0000 7f19fd936800 1 -- 172.21.15.50:0/3344152026 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f08fc2500 con 0x558f08fc33c0 2024-05-08T18:06:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.462+0000 7f18df7fe640 1 -- 172.21.15.50:0/3344152026 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f19ec03d250 con 0x558f08fc33c0 2024-05-08T18:06:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.462+0000 7f18df7fe640 1 --2- 172.21.15.50:0/3344152026 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f18d003bb50 0x7f18d003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.462+0000 7f18df7fe640 1 -- 172.21.15.50:0/3344152026 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f19ec025020 con 0x558f08fc33c0 2024-05-08T18:06:09.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.462+0000 7f19f0e2e640 1 --2- 172.21.15.50:0/3344152026 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f18d003bb50 0x7f18d003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.463+0000 7f19fd936800 1 Processor -- start 2024-05-08T18:06:09.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.463+0000 7f19fd936800 1 -- start start 2024-05-08T18:06:09.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.463+0000 7f19f0e2e640 1 --2- 172.21.15.50:0/3344152026 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f18d003bb50 0x7f18d003e010 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x558f08fd2730 tx=0x7f19e4005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.463+0000 7f19fd936800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ff8290 0x558f08ff86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.463+0000 7f19fd936800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558f08fc5b80 con 0x558f08ff8290 2024-05-08T18:06:09.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.463+0000 7f19f962f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ff8290 0x558f08ff86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.463+0000 7f19f962f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ff8290 0x558f08ff86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50668/0 (socket says 172.21.15.50:50668) 2024-05-08T18:06:09.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.463+0000 7f19f962f640 1 -- 172.21.15.50:0/112703910 learned_addr learned my addr 172.21.15.50:0/112703910 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.463+0000 7f19f962f640 1 -- 172.21.15.50:0/112703910 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f08ff8c80 con 0x558f08ff8290 2024-05-08T18:06:09.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.463+0000 7f19f962f640 1 --2- 172.21.15.50:0/112703910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ff8290 0x558f08ff86b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0165e0 tx=0x7f19ec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbce588d80f962f2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.464+0000 7f18dd7fa640 1 -- 172.21.15.50:0/112703910 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19ec063070 con 0x558f08ff8290 2024-05-08T18:06:09.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.464+0000 7f18dd7fa640 1 -- 172.21.15.50:0/112703910 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ec05a040 con 0x558f08ff8290 2024-05-08T18:06:09.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.464+0000 7f18dd7fa640 1 -- 172.21.15.50:0/112703910 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19ec059050 con 0x558f08ff8290 2024-05-08T18:06:09.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.464+0000 7f19fd936800 1 -- 172.21.15.50:0/112703910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ff8290 msgr2=0x558f08ff86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.464+0000 7f19fd936800 1 --2- 172.21.15.50:0/112703910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ff8290 0x558f08ff86b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f19ec0165e0 tx=0x7f19ec00cbf0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.464+0000 7f19fd936800 1 -- 172.21.15.50:0/112703910 shutdown_connections 2024-05-08T18:06:09.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.464+0000 7f19fd936800 1 --2- 172.21.15.50:0/112703910 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ff8290 0x558f08ff86b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.464+0000 7f19fd936800 1 -- 172.21.15.50:0/112703910 >> 172.21.15.50:0/112703910 conn(0x558f08ff4560 msgr2=0x558f08ff25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.464+0000 7f19fd936800 1 -- 172.21.15.50:0/112703910 shutdown_connections 2024-05-08T18:06:09.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.464+0000 7f19fd936800 1 -- 172.21.15.50:0/112703910 wait complete. 2024-05-08T18:06:09.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.464+0000 7f19fd936800 1 Processor -- start 2024-05-08T18:06:09.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.465+0000 7f19fd936800 1 -- start start 2024-05-08T18:06:09.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.465+0000 7f19fd936800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ff8290 0x558f08fff640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.465+0000 7f19fd936800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558f08ff9350 con 0x558f08ff8290 2024-05-08T18:06:09.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.465+0000 7f19f962f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ff8290 0x558f08fff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.465+0000 7f19f962f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ff8290 0x558f08fff640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50676/0 (socket says 172.21.15.50:50676) 2024-05-08T18:06:09.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.465+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 learned_addr learned my addr 172.21.15.50:0/3392264669 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.465+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f08fffb80 con 0x558f08ff8290 2024-05-08T18:06:09.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.465+0000 7f19f962f640 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ff8290 0x558f08fff640 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f19ec016590 tx=0x7f19ec012f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.465+0000 7f18d6ffd640 1 -- 172.21.15.50:0/3392264669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19ec06c020 con 0x558f08ff8290 2024-05-08T18:06:09.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.466+0000 7f18d6ffd640 1 -- 172.21.15.50:0/3392264669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19ec02d9b0 con 0x558f08ff8290 2024-05-08T18:06:09.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.466+0000 7f18d6ffd640 1 -- 172.21.15.50:0/3392264669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19ec063040 con 0x558f08ff8290 2024-05-08T18:06:09.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.466+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558f08fffb80 con 0x558f08ff8290 2024-05-08T18:06:09.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.466+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f08fffb80 con 0x558f08ff8290 2024-05-08T18:06:09.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.466+0000 7f18d6ffd640 1 -- 172.21.15.50:0/3392264669 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f19ec06f020 con 0x558f08ff8290 2024-05-08T18:06:09.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.467+0000 7f18d6ffd640 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f18c003bba0 0x7f18c003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.467+0000 7f18d6ffd640 1 -- 172.21.15.50:0/3392264669 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f19ec072d20 con 0x558f08ff8290 2024-05-08T18:06:09.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.467+0000 7f19f0e2e640 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f18c003bba0 0x7f18c003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.467+0000 7f19f0e2e640 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f18c003bba0 0x7f18c003e060 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f19e4052fd0 tx=0x7f19e404f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.467+0000 7f19fd936800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.467+0000 7f19fd936800 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558f0900ade0 0x558f0900d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.467+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f0900d9f0 con 0x558f0900ade0 2024-05-08T18:06:09.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.467+0000 7f19f9e30640 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558f0900ade0 0x558f0900d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.468+0000 7f19f9e30640 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558f0900ade0 0x558f0900d2a0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.468+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f19f400a4a0 con 0x558f0900ade0 2024-05-08T18:06:09.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.468+0000 7f19fd936800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.468+0000 7f19fd936800 20 realm 2024-05-08T18:06:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.468+0000 7f19fd936800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.468+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f0900de00 con 0x558f0900ade0 2024-05-08T18:06:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.468+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f19f400a4a0 con 0x558f0900ade0 2024-05-08T18:06:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.468+0000 7f19fd936800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.468+0000 7f19fd936800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.468+0000 7f19fd936800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.468+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f0900e6d0 con 0x558f0900ade0 2024-05-08T18:06:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.468+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f19f400a4a0 con 0x558f0900ade0 2024-05-08T18:06:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.468+0000 7f19fd936800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.468+0000 7f19fd936800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.468+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f0900ea70 con 0x558f0900ade0 2024-05-08T18:06:09.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.469+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f19f400a4a0 con 0x558f0900ade0 2024-05-08T18:06:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.469+0000 7f19fd936800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.469+0000 7f19fd936800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.469+0000 7f19fd936800 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558f0900f7d0 0x558f09011bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.469+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f090121d0 con 0x558f0900f7d0 2024-05-08T18:06:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.469+0000 7f19f962f640 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558f0900f7d0 0x558f09011bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.469+0000 7f19f962f640 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558f0900f7d0 0x558f09011bc0 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.470+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.470+0000 7f19fd936800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.470+0000 7f19fd936800 20 searching for the correct realm 2024-05-08T18:06:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.470+0000 7f19fd936800 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558f09014180 0x558f09016640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.470+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x558f09016d10 con 0x558f09014180 2024-05-08T18:06:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.470+0000 7f19f0e2e640 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558f09014180 0x558f09016640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.488 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.470+0000 7f19f0e2e640 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558f09014180 0x558f09016640 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.470+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.470+0000 7f19f0e2e640 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f19e4076040 0x7f19e4083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.470+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x558f09016d10 con 0x7f19e4076040 2024-05-08T18:06:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.470+0000 7f19f9e30640 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f19e4076040 0x7f19e4083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.471+0000 7f19f9e30640 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f19e4076040 0x7f19e4083410 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.471+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19f400a4a0 con 0x7f19e4076040 2024-05-08T18:06:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.471+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x558f0900ea70 con 0x558f0900f7d0 2024-05-08T18:06:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.471+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.471+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x558f0900ea70 con 0x558f0900ade0 2024-05-08T18:06:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.472+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19f400a4a0 con 0x558f0900ade0 2024-05-08T18:06:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.472+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x558f0900ea70 con 0x558f0900ade0 2024-05-08T18:06:09.489 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.472+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19f400a4a0 con 0x558f0900ade0 2024-05-08T18:06:09.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.472+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x558f0900ea70 con 0x558f0900ade0 2024-05-08T18:06:09.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.472+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19f400a4a0 con 0x558f0900ade0 2024-05-08T18:06:09.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.472+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x558f0900ea70 con 0x558f0900f7d0 2024-05-08T18:06:09.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.472+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.472+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x558f0900ea70 con 0x7f19e4076040 2024-05-08T18:06:09.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.473+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19f400a4a0 con 0x7f19e4076040 2024-05-08T18:06:09.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.473+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x558f0900ea70 con 0x558f09014180 2024-05-08T18:06:09.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.473+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.473+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x558f0900ea70 con 0x558f0900ade0 2024-05-08T18:06:09.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.473+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19f400a4a0 con 0x558f0900ade0 2024-05-08T18:06:09.490 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.473+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x558f0900ea70 con 0x558f0900f7d0 2024-05-08T18:06:09.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.473+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.473+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x558f0900ea70 con 0x558f0900ade0 2024-05-08T18:06:09.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.474+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19f400a4a0 con 0x558f0900ade0 2024-05-08T18:06:09.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.474+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x558f0900ea70 con 0x7f19e4076040 2024-05-08T18:06:09.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.474+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.474+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x558f0900ea70 con 0x558f09014180 2024-05-08T18:06:09.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.474+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.474+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x558f09014180 2024-05-08T18:06:09.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.474+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.474+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x558f0900f7d0 2024-05-08T18:06:09.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.475+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.475+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x558f09014180 2024-05-08T18:06:09.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.475+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.475+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x558f09014180 2024-05-08T18:06:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.475+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.475+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x558f0900ade0 2024-05-08T18:06:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.475+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.475+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x558f0900ade0 2024-05-08T18:06:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.475+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.475+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x558f0900f7d0 2024-05-08T18:06:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.476+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.476+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x558f09014180 2024-05-08T18:06:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.476+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.476+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x7f19e4076040 2024-05-08T18:06:09.492 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.476+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.476+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x558f0900ade0 2024-05-08T18:06:09.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.476+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.476+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x558f0900f7d0 2024-05-08T18:06:09.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.477+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.477+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x558f09014180 2024-05-08T18:06:09.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.477+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.477+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x558f0900f7d0 2024-05-08T18:06:09.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.477+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.477+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x7f19e4076040 2024-05-08T18:06:09.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.478+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.493 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.478+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x558f09014180 2024-05-08T18:06:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.478+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.478+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x558f0900f7d0 2024-05-08T18:06:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.478+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.478+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x558f0900ade0 2024-05-08T18:06:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f19e4090040 con 0x7f19e4076040 2024-05-08T18:06:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19fd936800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19fd936800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:09.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19fd936800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:09.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19fd936800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:09.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19fd936800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:09.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19fd936800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:09.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19fd936800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09012a90 con 0x558f0900ade0 2024-05-08T18:06:09.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19fd936800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19fd936800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09012a90 con 0x558f0900ade0 2024-05-08T18:06:09.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19fd936800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:09.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19fd936800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.479+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09012a90 con 0x558f09014180 2024-05-08T18:06:09.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19fd936800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:09.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19fd936800 20 zone default found 2024-05-08T18:06:09.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19fd936800 4 Realm: () 2024-05-08T18:06:09.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19fd936800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:09.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19fd936800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:09.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19fd936800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:09.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19fd936800 20 zonegroup default 2024-05-08T18:06:09.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19fd936800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09013420 con 0x558f0900f7d0 2024-05-08T18:06:09.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19fd936800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19fd936800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09012fd0 con 0x558f09014180 2024-05-08T18:06:09.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19fd936800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19fd936800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f090190c0 con 0x558f09014180 2024-05-08T18:06:09.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.480+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.481+0000 7f19fd936800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.481+0000 7f19fd936800 20 started sync module instance, tier type = 2024-05-08T18:06:09.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.481+0000 7f19fd936800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:09.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.481+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f090184b0 con 0x7f19e4076040 2024-05-08T18:06:09.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.482+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.482+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94072819780096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f090184b0 con 0x7f19e4076040 2024-05-08T18:06:09.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.482+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f09019b40 con 0x558f0900ade0 2024-05-08T18:06:09.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.483+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94072819780096] v29'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.483+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.483+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94072819783792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f090184b0 con 0x558f0900ade0 2024-05-08T18:06:09.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.483+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901a590 con 0x558f09014180 2024-05-08T18:06:09.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.484+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94072819783792] v29'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.504 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.484+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.484+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94072819777536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f09018420 con 0x558f09014180 2024-05-08T18:06:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.484+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901b480 con 0x7f19e4076040 2024-05-08T18:06:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.485+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94072819777536] v29'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.485+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.485+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94072819789776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901b8a0 con 0x7f19e4076040 2024-05-08T18:06:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.485+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901c260 con 0x558f0900ade0 2024-05-08T18:06:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.486+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94072819789776] v29'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.486+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.486+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94072819791312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901c260 con 0x558f0900ade0 2024-05-08T18:06:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.486+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901cc20 con 0x558f0900f7d0 2024-05-08T18:06:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.488+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.488+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94072819791312] v29'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.505 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.488+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94072819793328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901cfc0 con 0x558f0900f7d0 2024-05-08T18:06:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.488+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901d980 con 0x558f0900f7d0 2024-05-08T18:06:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.489+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94072819793328] v29'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.489+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.489+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94072819795824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901e1d0 con 0x558f0900f7d0 2024-05-08T18:06:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.489+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901eb90 con 0x558f0900f7d0 2024-05-08T18:06:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.490+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(58 notify.7 [create] v29'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.490+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94072819803872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901efb0 con 0x558f0900f7d0 2024-05-08T18:06:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.490+0000 7f19fd936800 20 add_watcher() i=0 2024-05-08T18:06:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.490+0000 7f19fd936800 20 add_watcher() i=1 2024-05-08T18:06:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.490+0000 7f19fd936800 20 add_watcher() i=2 2024-05-08T18:06:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.490+0000 7f19fd936800 20 add_watcher() i=3 2024-05-08T18:06:09.506 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.490+0000 7f19fd936800 20 add_watcher() i=4 2024-05-08T18:06:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.490+0000 7f19fd936800 20 add_watcher() i=5 2024-05-08T18:06:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.490+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94072819795824] v29'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.490+0000 7f19fd936800 20 add_watcher() i=6 2024-05-08T18:06:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.491+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94072819803872] v29'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.491+0000 7f19fd936800 20 add_watcher() i=7 2024-05-08T18:06:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.491+0000 7f19fd936800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.491+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f0901dd20 con 0x558f0900ade0 2024-05-08T18:06:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.491+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.491+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94072819804928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901dd20 con 0x558f0900ade0 2024-05-08T18:06:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.492+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94072819804928] v29'218 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.492+0000 7f18ab7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.493+0000 7f19fd936800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:09.507 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.493+0000 7f19fd936800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.493+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902a9d0 con 0x558f0900ade0 2024-05-08T18:06:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.494+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v29'32 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.494+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902b0a0 con 0x558f0900f7d0 2024-05-08T18:06:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.495+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(63 gc.1 [create,call,call,call] v29'27 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.495+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902b0a0 con 0x558f09014180 2024-05-08T18:06:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.496+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v29'26 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.496+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902b690 con 0x558f0900ade0 2024-05-08T18:06:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.497+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v29'219 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.497+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902bb40 con 0x7f19e4076040 2024-05-08T18:06:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.498+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v29'68 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.498+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c0b0 con 0x558f09014180 2024-05-08T18:06:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.499+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v29'45 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.508 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.499+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c560 con 0x558f0900ade0 2024-05-08T18:06:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.500+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v29'29 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.500+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c390 con 0x558f0900f7d0 2024-05-08T18:06:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.501+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(69 gc.7 [create,call,call,call] v29'45 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.501+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c390 con 0x7f19e4076040 2024-05-08T18:06:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.502+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v29'26 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.502+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c420 con 0x558f09014180 2024-05-08T18:06:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.504+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v29'32 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.504+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c8d0 con 0x7f19e4076040 2024-05-08T18:06:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.505+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v29'69 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.505+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c8d0 con 0x558f0900f7d0 2024-05-08T18:06:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.506+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(73 gc.11 [create,call,call,call] v29'30 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.509 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.506+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c8d0 con 0x7f19e4076040 2024-05-08T18:06:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.507+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v29'27 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.507+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c840 con 0x558f0900f7d0 2024-05-08T18:06:09.511 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.508+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(75 gc.13 [create,call,call,call] v29'51 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.508+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c840 con 0x558f0900ade0 2024-05-08T18:06:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.509+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v29'220 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.509+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c840 con 0x558f09014180 2024-05-08T18:06:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.510+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v29'63 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.510+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c900 con 0x7f19e4076040 2024-05-08T18:06:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.511+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v29'70 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.511+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c900 con 0x558f09014180 2024-05-08T18:06:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.512+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v29'26 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.512+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c8d0 con 0x558f0900ade0 2024-05-08T18:06:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.513+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v29'34 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.512 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.513+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c8d0 con 0x558f0900ade0 2024-05-08T18:06:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.514+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v29'221 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.514+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c8d0 con 0x558f09014180 2024-05-08T18:06:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.515+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v29'64 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.515+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c8d0 con 0x7f19e4076040 2024-05-08T18:06:09.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.516+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v29'36 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.516+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c8d0 con 0x558f09014180 2024-05-08T18:06:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.517+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v29'28 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.517+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c8d0 con 0x558f09014180 2024-05-08T18:06:09.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.518+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v29'32 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.518+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c840 con 0x558f09014180 2024-05-08T18:06:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.519+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v29'24 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.519+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c840 con 0x558f09014180 2024-05-08T18:06:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.520+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v29'46 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.520+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c840 con 0x558f0900f7d0 2024-05-08T18:06:09.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.521+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(88 gc.26 [create,call,call,call] v29'28 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.521+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c840 con 0x558f09014180 2024-05-08T18:06:09.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.522+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v29'26 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.522+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c840 con 0x558f0900f7d0 2024-05-08T18:06:09.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.523+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(90 gc.28 [create,call,call,call] v29'46 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.523+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c840 con 0x558f09014180 2024-05-08T18:06:09.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.525+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v29'65 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.525+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c840 con 0x558f0900f7d0 2024-05-08T18:06:09.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.526+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(92 gc.30 [create,call,call,call] v29'52 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.526+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0902c840 con 0x7f19e4076040 2024-05-08T18:06:09.524 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.527+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v29'32 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.527+0000 7f18aa7fc640 20 reqs_thread_entry: start 2024-05-08T18:06:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.527+0000 7f19fd936800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.527+0000 7f19fd936800 20 Filter name: none 2024-05-08T18:06:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.527+0000 7f181bfff640 20 reqs_thread_entry: start 2024-05-08T18:06:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.527+0000 7f19fd936800 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-05-08T18:06:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.527+0000 7f19fd936800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.527+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:94 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09057970 con 0x7f19e4076040 2024-05-08T18:06:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(94 bud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19fd936800 20 rados_obj.operate() r=0 bl.length=380 2024-05-08T18:06:09.527 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19fd936800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2024-05-08T18:06:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19fd936800 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-05-08T18:06:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19fd936800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:95 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f0905a400 con 0x558f0900ade0 2024-05-08T18:06:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(95 bud.buckets [call out=184b] v0'0 uv6 ondisk = 0) ==== 155+0+184 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19fd936800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 : miss 2024-05-08T18:06:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19fd936800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:96 8.19 8:98c2a5ec:root::.bucket.meta.myfoo6%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f0905a210 con 0x558f0900ade0 2024-05-08T18:06:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(96 .bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 [call out=48b,stat out=16b,read 0~380 out=380b,getxattrs out=212b] v0'0 uv7 ondisk = 0) ==== 333+0+656 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19fd936800 20 rados_obj.operate() r=0 bl.length=380 2024-05-08T18:06:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19fd936800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 info.flags=0x17 2024-05-08T18:06:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19fd936800 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 to cache LRU end 2024-05-08T18:06:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19fd936800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:09.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19fd936800 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-05-08T18:06:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19fd936800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 2024-05-08T18:06:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.528+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:97 4.1b 4:d9ac9af1:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f090d7700 con 0x7f19e4076040 2024-05-08T18:06:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.529+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(97 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.529+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:98 4.13 4:c87f6b5c:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f090d7780 con 0x7f19e4076040 2024-05-08T18:06:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.529+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 20 ==== osd_op_reply(98 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.1 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.529+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:99 4.3a 4:5f00aa60:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f090d7780 con 0x558f09014180 2024-05-08T18:06:09.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.529+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(99 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.2 [call out=85b] v0'0 uv4 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.529+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:100 4.12 4:48cbad65:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f090d7780 con 0x558f0900ade0 2024-05-08T18:06:09.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.530+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(100 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.3 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.530+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:101 4.2f 4:f4b659df:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f090d7780 con 0x558f0900ade0 2024-05-08T18:06:09.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.530+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(101 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.4 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.530+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:102 4.e 4:70417d9c:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f090d7700 con 0x7f19e4076040 2024-05-08T18:06:09.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.530+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 21 ==== osd_op_reply(102 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.5 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.530+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:103 4.25 4:a4bd2273:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f090d7700 con 0x558f09014180 2024-05-08T18:06:09.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.531+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 28 ==== osd_op_reply(103 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.6 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.531+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:104 4.26 4:652575e1:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f090d7680 con 0x558f0900f7d0 2024-05-08T18:06:09.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.531+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(104 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.7 [call out=85b] v0'0 uv14 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.531+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:105 4.28 4:162ce8db:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f090d7680 con 0x558f0900ade0 2024-05-08T18:06:09.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.532+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 32 ==== osd_op_reply(105 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.532+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:106 4.4 4:215f588e:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f090d7680 con 0x558f0900f7d0 2024-05-08T18:06:09.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.532+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(106 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9 [call out=289b] v0'0 uv5 ondisk = 0) ==== 194+0+289 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.532+0000 7f19fd936800 10 cls_bucket_list_unordered: got thirteen 2024-05-08T18:06:09.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.532+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:107 4.39 4:9c486660:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f090d7680 con 0x558f09014180 2024-05-08T18:06:09.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.532+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 29 ==== osd_op_reply(107 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.532+0000 7f19fd936800 20 get_obj_state: octx=0x7f19ec037eb0 obj=myfoo6:thirteen state=0x558f090d57a8 s->prefetch_data=0 2024-05-08T18:06:09.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.532+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:108 3.7s0 3:e145b1ed:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9_thirteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f090d65d0 con 0x558f09014180 2024-05-08T18:06:09.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.533+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 30 ==== osd_op_reply(108 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9_thirteen [getxattrs out=820b,stat out=16b] v0'0 uv2 ondisk = 0) ==== 238+0+836 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.533+0000 7f19fd936800 10 manifest: total_size = 8 2024-05-08T18:06:09.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.533+0000 7f19fd936800 20 get_obj_state: setting s->obj_tag to _5U4NTIKTrY3OinIySBkKJWwMlt3n61k 2024-05-08T18:06:09.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.533+0000 7f19fd936800 20 get_obj_state: octx=0x7f19ec037eb0 obj=myfoo6:thirteen state=0x558f090d57a8 s->prefetch_data=0 2024-05-08T18:06:09.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.533+0000 7f19fd936800 20 prepare_atomic_modification: state is not atomic. state=0x558f090d57a8 2024-05-08T18:06:09.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.533+0000 7f19fd936800 20 bucket index object: :.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9 2024-05-08T18:06:09.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.533+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:109 4.4 4:215f588e:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f090d6e90 con 0x558f0900f7d0 2024-05-08T18:06:09.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.534+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(109 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9 [stat,call,call] v29'6 uv6 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.534 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.534+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:110 3.7s0 3:e145b1ed:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9_thirteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f090d6e90 con 0x558f09014180 2024-05-08T18:06:09.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 31 ==== osd_op_reply(110 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9_thirteen [call] v29'3 uv3 ondisk = 0) ==== 196+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:111 4.4 4:215f588e:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=264b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0905e8f0 con 0x558f0900f7d0 2024-05-08T18:06:09.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19fd936800 20 list_objects_ordered: starting attempt 1 2024-05-08T18:06:09.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19fd936800 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-08T18:06:09.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:112 4.1b 4:d9ac9af1:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f0905e8f0 con 0x7f19e4076040 2024-05-08T18:06:09.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:113 4.13 4:c87f6b5c:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09062bc0 con 0x7f19e4076040 2024-05-08T18:06:09.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:114 4.3a 4:5f00aa60:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09065500 con 0x558f09014180 2024-05-08T18:06:09.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:115 4.12 4:48cbad65:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f090680f0 con 0x558f0900ade0 2024-05-08T18:06:09.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:116 4.2f 4:f4b659df:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09062bc0 con 0x558f0900ade0 2024-05-08T18:06:09.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:117 4.e 4:70417d9c:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f0906d0f0 con 0x7f19e4076040 2024-05-08T18:06:09.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:118 4.25 4:a4bd2273:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f0906ff70 con 0x558f09014180 2024-05-08T18:06:09.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:119 4.26 4:652575e1:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09072df0 con 0x558f0900f7d0 2024-05-08T18:06:09.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:120 4.28 4:162ce8db:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09075c70 con 0x558f0900ade0 2024-05-08T18:06:09.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:121 4.4 4:215f588e:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09078980 con 0x558f0900f7d0 2024-05-08T18:06:09.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:122 4.39 4:9c486660:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f0907b4d0 con 0x558f09014180 2024-05-08T18:06:09.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(119 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.7 [call out=85b] v0'0 uv14 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.536+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 33 ==== osd_op_reply(115 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.3 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.537+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 22 ==== osd_op_reply(112 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.537+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 23 ==== osd_op_reply(113 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.1 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.537+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 32 ==== osd_op_reply(114 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.2 [call out=85b] v0'0 uv4 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.537+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 34 ==== osd_op_reply(116 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.4 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.537+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 35 ==== osd_op_reply(120 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.537+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 33 ==== osd_op_reply(118 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.6 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.537+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 34 ==== osd_op_reply(122 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.537+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 24 ==== osd_op_reply(117 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.5 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.537+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(111 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9 [stat,call,call] v29'7 uv7 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f18dd7fa640 20 handle_completion(): completion ok for obj=thirteen 2024-05-08T18:06:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(121 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9 [call out=124b] v0'0 uv7 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19fd936800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-08T18:06:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19fd936800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-08T18:06:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19fd936800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-08T18:06:09.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19fd936800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-08T18:06:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19fd936800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo6[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9])"; uploads.size()=0; is_truncated=0 2024-05-08T18:06:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:123 4.1b 4:d9ac9af1:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f0907b4d0 con 0x7f19e4076040 2024-05-08T18:06:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:124 4.13 4:c87f6b5c:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f090727a0 con 0x7f19e4076040 2024-05-08T18:06:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:125 4.3a 4:5f00aa60:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09068050 con 0x558f09014180 2024-05-08T18:06:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:126 4.12 4:48cbad65:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09078940 con 0x558f0900ade0 2024-05-08T18:06:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:127 4.2f 4:f4b659df:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09075ad0 con 0x558f0900ade0 2024-05-08T18:06:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:128 4.e 4:70417d9c:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09068050 con 0x7f19e4076040 2024-05-08T18:06:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:129 4.25 4:a4bd2273:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f0907b4d0 con 0x558f09014180 2024-05-08T18:06:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:130 4.26 4:652575e1:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09068050 con 0x558f0900f7d0 2024-05-08T18:06:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:131 4.28 4:162ce8db:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09078940 con 0x558f0900ade0 2024-05-08T18:06:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:132 4.4 4:215f588e:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f0906f630 con 0x558f0900f7d0 2024-05-08T18:06:09.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:133 4.39 4:9c486660:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09068050 con 0x558f09014180 2024-05-08T18:06:09.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 25 ==== osd_op_reply(123 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 26 ==== osd_op_reply(124 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.1 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 35 ==== osd_op_reply(125 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.2 [call out=85b] v0'0 uv4 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 36 ==== osd_op_reply(126 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.3 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== osd_op_reply(130 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.7 [call out=85b] v0'0 uv14 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 37 ==== osd_op_reply(127 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.4 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 27 ==== osd_op_reply(128 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.5 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 31 ==== osd_op_reply(132 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9 [call out=124b] v0'0 uv7 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.538+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 36 ==== osd_op_reply(129 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.6 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.539+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 37 ==== osd_op_reply(133 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.539+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 38 ==== osd_op_reply(131 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.539+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:134 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f09078940 con 0x558f0900ade0 2024-05-08T18:06:09.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.540+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 39 ==== osd_op_reply(134 bud.buckets [call] v29'8 uv8 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.540+0000 7f19fd936800 10 cache get: name=default.rgw.meta+root+myfoo6 : miss 2024-05-08T18:06:09.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.540+0000 7f19fd936800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.540+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:135 8.18 8:1944a257:root::myfoo6:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09078490 con 0x558f0900ade0 2024-05-08T18:06:09.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.540+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 40 ==== osd_op_reply(135 myfoo6 [call out=48b,read 0~166 out=166b] v0'0 uv17 ondisk = 0) ==== 192+0+214 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.540+0000 7f19fd936800 20 rados_obj.operate() r=0 bl.length=166 2024-05-08T18:06:09.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.540+0000 7f19fd936800 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x11 2024-05-08T18:06:09.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.540+0000 7f19fd936800 10 adding default.rgw.meta+root+myfoo6 to cache LRU end 2024-05-08T18:06:09.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.540+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:136 8.18 8:1944a257:root::myfoo6:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f09078490 con 0x558f0900ade0 2024-05-08T18:06:09.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.541+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 41 ==== osd_op_reply(136 myfoo6 [call,call,delete] v29'18 uv18 ondisk = 0) ==== 234+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.541+0000 7f19fd936800 10 removing default.rgw.meta+root+myfoo6 from cache 2024-05-08T18:06:09.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.541+0000 7f19fd936800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-05-08T18:06:09.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.541+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:137 7.e 7:7759931f:::notify.3:head [notify cookie 94072820559280 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09078490 con 0x7f19e4076040 2024-05-08T18:06:09.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.541+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 28 ==== watch-notify(notify (1) cookie 94072819789776 notify 124554051601 ret 0) ==== 192+0+0 (crc 0 0 0) 0x558f09059720 con 0x7f19e4076040 2024-05-08T18:06:09.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.541+0000 7f18dcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051601 cookie 94072819789776 notifier 4746 bl.length()=150 2024-05-08T18:06:09.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.541+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 29 ==== osd_op_reply(137 notify.3 [notify cookie 94072820559280 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.541+0000 7f18dcff9640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:138 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f18240032c0 con 0x7f19e4076040 2024-05-08T18:06:09.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.542+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 30 ==== osd_op_reply(138 notify.3 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x7f19e4076040 2024-05-08T18:06:09.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.542+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 31 ==== watch-notify(notify_complete (2) cookie 94072820559280 notify 124554051601 ret 0) ==== 42+0+48 (crc 0 0 0) 0x558f0902a340 con 0x7f19e4076040 2024-05-08T18:06:09.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.542+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:139 8.19 8:98c2a5ec:root::.bucket.meta.myfoo6%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f09072720 con 0x558f0900ade0 2024-05-08T18:06:09.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.543+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 42 ==== osd_op_reply(139 .bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 [call,call,delete] v29'9 uv9 ondisk = 0) ==== 291+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.543+0000 7f19fd936800 10 removing default.rgw.meta+root+.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 from cache 2024-05-08T18:06:09.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.543+0000 7f19fd936800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo6:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9, ofs0, ns] 2024-05-08T18:06:09.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.543+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:140 7.0 7:05bf5b68:::notify.1:head [notify cookie 94072820559280 in=219b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09067a70 con 0x558f0900ade0 2024-05-08T18:06:09.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.543+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 43 ==== watch-notify(notify (1) cookie 94072819783792 notify 124554051612 ret 0) ==== 249+0+0 (crc 0 0 0) 0x558f0902a340 con 0x558f0900ade0 2024-05-08T18:06:09.549 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.543+0000 7f18dcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051612 cookie 94072819783792 notifier 4746 bl.length()=207 2024-05-08T18:06:09.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.543+0000 7f18dcff9640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:141 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f18240032a0 con 0x558f0900ade0 2024-05-08T18:06:09.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.543+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 44 ==== osd_op_reply(140 notify.1 [notify cookie 94072820559280 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.543+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 45 ==== osd_op_reply(141 notify.1 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19f4032040 con 0x558f0900ade0 2024-05-08T18:06:09.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.543+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 46 ==== watch-notify(notify_complete (2) cookie 94072820559280 notify 124554051612 ret 0) ==== 42+0+48 (crc 0 0 0) 0x558f0902b270 con 0x558f0900ade0 2024-05-08T18:06:09.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.543+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:142 4.1b 4:d9ac9af1:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0905a130 con 0x7f19e4076040 2024-05-08T18:06:09.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.544+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:143 4.13 4:c87f6b5c:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f09067f20 con 0x7f19e4076040 2024-05-08T18:06:09.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.544+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:144 4.3a 4:5f00aa60:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f090759c0 con 0x558f09014180 2024-05-08T18:06:09.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.544+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:145 4.12 4:48cbad65:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f09076160 con 0x558f0900ade0 2024-05-08T18:06:09.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.544+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:146 4.2f 4:f4b659df:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f09076500 con 0x558f0900ade0 2024-05-08T18:06:09.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.544+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:147 4.e 4:70417d9c:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f090759c0 con 0x7f19e4076040 2024-05-08T18:06:09.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.544+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:148 4.25 4:a4bd2273:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f09076d50 con 0x558f09014180 2024-05-08T18:06:09.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.544+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:149 4.26 4:652575e1:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f09058ef0 con 0x558f0900f7d0 2024-05-08T18:06:09.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.544+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:150 4.28 4:162ce8db:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f09076d50 con 0x558f0900ade0 2024-05-08T18:06:09.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.544+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:151 4.4 4:215f588e:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f09077c00 con 0x558f0900f7d0 2024-05-08T18:06:09.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.544+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:152 4.39 4:9c486660:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f09071a60 con 0x558f09014180 2024-05-08T18:06:09.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.545+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 38 ==== osd_op_reply(148 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.6 [delete] v29'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.545+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 32 ==== osd_op_reply(142 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.0 [delete] v29'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x7f19e4076040 2024-05-08T18:06:09.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 33 ==== osd_op_reply(143 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.1 [delete] v29'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x7f19e4076040 2024-05-08T18:06:09.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 34 ==== osd_op_reply(147 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.5 [delete] v29'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x7f19e4076040 2024-05-08T18:06:09.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 39 ==== osd_op_reply(144 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.2 [delete] v29'5 uv5 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 32 ==== osd_op_reply(151 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.9 [delete] v29'8 uv8 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 40 ==== osd_op_reply(152 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.10 [delete] v29'2 uv2 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.557 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 33 ==== osd_op_reply(149 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.7 [delete] v29'18 uv18 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 47 ==== osd_op_reply(150 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.8 [delete] v29'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x558f0900ade0 2024-05-08T18:06:09.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 48 ==== osd_op_reply(146 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.4 [delete] v29'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x558f0900ade0 2024-05-08T18:06:09.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 49 ==== osd_op_reply(145 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9.3 [delete] v29'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x558f0900ade0 2024-05-08T18:06:09.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19fd936800 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo6/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 : miss 2024-05-08T18:06:09.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19fd936800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:153 6.14 6:2a908f2b:::pubsub..bucket.myfoo6%2f1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f0905a130 con 0x7f19e4076040 2024-05-08T18:06:09.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 35 ==== osd_op_reply(153 pubsub..bucket.myfoo6/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 209+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x7f19e4076040 2024-05-08T18:06:09.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19fd936800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19fd936800 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo6/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 info.flags=0x0 2024-05-08T18:06:09.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19fd936800 10 adding default.rgw.log++pubsub..bucket.myfoo6/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 to cache LRU end 2024-05-08T18:06:09.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.546+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:154 6.14 6:2a908f2b:::pubsub..bucket.myfoo6%2f1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0905a130 con 0x7f19e4076040 2024-05-08T18:06:09.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.547+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 36 ==== osd_op_reply(154 pubsub..bucket.myfoo6/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.9 [delete] v29'11 uv10 ondisk = -2 ((2) No such file or directory)) ==== 209+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x7f19e4076040 2024-05-08T18:06:09.558 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.547+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:155 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.remove_bucket in=145b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0905a130 con 0x558f0900ade0 2024-05-08T18:06:09.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.548+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 50 ==== osd_op_reply(155 bud.buckets [call] v29'10 uv10 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x558f0900ade0 2024-05-08T18:06:09.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.548+0000 7f19fd936800 10 removing key index: NX5QOQKC6BH2IDN8HC7A 2024-05-08T18:06:09.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.548+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:156 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0905a0d0 con 0x558f0900ade0 2024-05-08T18:06:09.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.549+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 51 ==== osd_op_reply(156 NX5QOQKC6BH2IDN8HC7A [delete] v29'3 uv3 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x558f0900ade0 2024-05-08T18:06:09.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.549+0000 7f19fd936800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-05-08T18:06:09.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.549+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:157 7.3 7:c609908c:::notify.5:head [notify cookie 94072820559280 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09075440 con 0x558f0900f7d0 2024-05-08T18:06:09.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.550+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 34 ==== watch-notify(notify (1) cookie 94072819793328 notify 124554051622 ret 0) ==== 212+0+0 (crc 0 0 0) 0x558f0902bee0 con 0x558f0900f7d0 2024-05-08T18:06:09.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.550+0000 7f18dd7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051622 cookie 94072819793328 notifier 4746 bl.length()=170 2024-05-08T18:06:09.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.550+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 35 ==== osd_op_reply(157 notify.5 [notify cookie 94072820559280 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.550+0000 7f18dd7fa640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:158 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f18c4003d60 con 0x558f0900f7d0 2024-05-08T18:06:09.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.550+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 36 ==== osd_op_reply(158 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.559 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.550+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 37 ==== watch-notify(notify_complete (2) cookie 94072820559280 notify 124554051622 ret 0) ==== 42+0+48 (crc 0 0 0) 0x558f08fc1d70 con 0x558f0900f7d0 2024-05-08T18:06:09.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.550+0000 7f19fd936800 10 removing email index: 2024-05-08T18:06:09.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.550+0000 7f19fd936800 10 removing user buckets index 2024-05-08T18:06:09.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.550+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:159 8.19 8:99adfd5d:users.uid::bud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0905a0d0 con 0x558f0900ade0 2024-05-08T18:06:09.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.551+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 52 ==== osd_op_reply(159 bud.buckets [delete] v29'11 uv11 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x558f0900ade0 2024-05-08T18:06:09.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.551+0000 7f19fd936800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:bud.buckets, ofs0, ns] 2024-05-08T18:06:09.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.551+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:160 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94072820559280 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09075440 con 0x558f0900f7d0 2024-05-08T18:06:09.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.551+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 38 ==== watch-notify(notify (1) cookie 94072819795824 notify 124554051623 ret 0) ==== 202+0+0 (crc 0 0 0) 0x558f08fc1d70 con 0x558f0900f7d0 2024-05-08T18:06:09.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.551+0000 7f18dcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051623 cookie 94072819795824 notifier 4746 bl.length()=160 2024-05-08T18:06:09.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.551+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 39 ==== osd_op_reply(160 notify.6 [notify cookie 94072820559280 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.551+0000 7f18dcff9640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:161 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f18240032a0 con 0x558f0900f7d0 2024-05-08T18:06:09.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.552+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 40 ==== osd_op_reply(161 notify.6 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.552+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 41 ==== watch-notify(notify_complete (2) cookie 94072820559280 notify 124554051623 ret 0) ==== 42+0+48 (crc 0 0 0) 0x558f08fcdb60 con 0x558f0900f7d0 2024-05-08T18:06:09.560 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.552+0000 7f19fd936800 10 removing user index: bud 2024-05-08T18:06:09.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.552+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:162 8.4 8:267425a8:users.uid::bud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f09058f40 con 0x7f19e4076040 2024-05-08T18:06:09.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.553+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 37 ==== osd_op_reply(162 bud [call,call,delete] v29'4 uv4 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x7f19e4076040 2024-05-08T18:06:09.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.553+0000 7f19fd936800 10 removing default.rgw.meta+users.uid+bud from cache 2024-05-08T18:06:09.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.553+0000 7f19fd936800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-05-08T18:06:09.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.553+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:163 7.3 7:c609908c:::notify.5:head [notify cookie 94072820559280 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558f09075440 con 0x558f0900f7d0 2024-05-08T18:06:09.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.553+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 42 ==== watch-notify(notify (1) cookie 94072819793328 notify 124554051624 ret 0) ==== 194+0+0 (crc 0 0 0) 0x558f08fcdb60 con 0x558f0900f7d0 2024-05-08T18:06:09.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.553+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 43 ==== osd_op_reply(163 notify.5 [notify cookie 94072820559280 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.553+0000 7f18dcff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051624 cookie 94072819793328 notifier 4746 bl.length()=152 2024-05-08T18:06:09.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.553+0000 7f18dcff9640 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:164 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f18240032a0 con 0x558f0900f7d0 2024-05-08T18:06:09.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.553+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 44 ==== watch-notify(notify_complete (2) cookie 94072820559280 notify 124554051624 ret 0) ==== 42+0+48 (crc 0 0 0) 0x558f08fcaeb0 con 0x558f0900f7d0 2024-05-08T18:06:09.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.553+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 45 ==== osd_op_reply(164 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.554+0000 7f18ab7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:09.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.554+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:165 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94072819804928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f090284f0 con 0x558f0900ade0 2024-05-08T18:06:09.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.555+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 53 ==== osd_op_reply(165 data_loggenerations_metadata [watch unwatch cookie 94072819804928] v29'222 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x558f0900ade0 2024-05-08T18:06:09.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.555+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:166 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94072819780096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901eb00 con 0x7f19e4076040 2024-05-08T18:06:09.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.556+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 38 ==== osd_op_reply(166 notify.0 [watch unwatch cookie 94072819780096] v29'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x7f19e4076040 2024-05-08T18:06:09.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.557+0000 7f19fd936800 20 remove_watcher() i=0 2024-05-08T18:06:09.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.557+0000 7f19fd936800 2 removed watcher, disabling cache 2024-05-08T18:06:09.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.557+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:167 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94072819783792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901eb00 con 0x558f0900ade0 2024-05-08T18:06:09.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.557+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 54 ==== osd_op_reply(167 notify.1 [watch unwatch cookie 94072819783792] v29'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x558f0900ade0 2024-05-08T18:06:09.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.558+0000 7f19fd936800 20 remove_watcher() i=1 2024-05-08T18:06:09.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.558+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:168 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94072819777536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901eb00 con 0x558f09014180 2024-05-08T18:06:09.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.559+0000 7f19f0e2e640 1 -- 172.21.15.50:0/3392264669 <== osd.3 v2:172.21.15.50:6800/3130492744 41 ==== osd_op_reply(168 notify.2 [watch unwatch cookie 94072819777536] v29'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x558f09014180 2024-05-08T18:06:09.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.559+0000 7f19fd936800 20 remove_watcher() i=2 2024-05-08T18:06:09.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.559+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:169 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94072819789776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901eb00 con 0x7f19e4076040 2024-05-08T18:06:09.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.560+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.2 v2:172.21.15.50:6811/3039646617 39 ==== osd_op_reply(169 notify.3 [watch unwatch cookie 94072819789776] v29'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x7f19e4076040 2024-05-08T18:06:09.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.560+0000 7f19fd936800 20 remove_watcher() i=3 2024-05-08T18:06:09.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.560+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:170 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94072819791312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901afd0 con 0x558f0900ade0 2024-05-08T18:06:09.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.561+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 <== osd.1 v2:172.21.15.50:6808/2141923372 55 ==== osd_op_reply(170 notify.4 [watch unwatch cookie 94072819791312] v29'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19f4065040 con 0x558f0900ade0 2024-05-08T18:06:09.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.561+0000 7f19fd936800 20 remove_watcher() i=4 2024-05-08T18:06:09.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.561+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:171 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94072819793328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901eb00 con 0x558f0900f7d0 2024-05-08T18:06:09.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.562+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 46 ==== osd_op_reply(171 notify.5 [watch unwatch cookie 94072819793328] v29'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.562+0000 7f19fd936800 20 remove_watcher() i=5 2024-05-08T18:06:09.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.562+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:172 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94072819795824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901afd0 con 0x558f0900f7d0 2024-05-08T18:06:09.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.563+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 47 ==== osd_op_reply(172 notify.6 [watch unwatch cookie 94072819795824] v29'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.563+0000 7f19fd936800 20 remove_watcher() i=6 2024-05-08T18:06:09.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.563+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:173 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94072819803872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x558f0901bdb0 con 0x558f0900f7d0 2024-05-08T18:06:09.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.564+0000 7f19f962f640 1 -- 172.21.15.50:0/3392264669 <== osd.0 v2:172.21.15.46:6800/337143975 48 ==== osd_op_reply(173 notify.7 [watch unwatch cookie 94072819803872] v29'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19ec066040 con 0x558f0900f7d0 2024-05-08T18:06:09.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.564+0000 7f19fd936800 20 remove_watcher() i=7 2024-05-08T18:06:09.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.564+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558f0900f7d0 msgr2=0x558f09011bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.564+0000 7f19fd936800 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558f0900f7d0 0x558f09011bc0 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.564+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558f0900ade0 msgr2=0x558f0900d2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.564+0000 7f19fd936800 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558f0900ade0 0x558f0900d2a0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.564+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f19e4076040 msgr2=0x7f19e4083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.564+0000 7f19fd936800 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f19e4076040 0x7f19e4083410 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.564+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558f09014180 msgr2=0x558f09016640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.564+0000 7f19fd936800 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558f09014180 0x558f09016640 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.564+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f18c003bba0 msgr2=0x7f18c003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.564+0000 7f19fd936800 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f18c003bba0 0x7f18c003e060 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f19e4052fd0 tx=0x7f19e404f000 comp rx=0 tx=0).stop 2024-05-08T18:06:09.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.564+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ff8290 msgr2=0x558f08fff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19f9e30640 1 -- 172.21.15.50:0/3392264669 reap_dead start 2024-05-08T18:06:09.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ff8290 0x558f08fff640 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f19ec016590 tx=0x7f19ec012f20 comp rx=0 tx=0).stop 2024-05-08T18:06:09.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 shutdown_connections 2024-05-08T18:06:09.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 --2- 172.21.15.50:0/3392264669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ff8290 0x558f08fff640 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 >> 172.21.15.50:0/3392264669 conn(0x558f08ff4560 msgr2=0x558f08ff1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 shutdown_connections 2024-05-08T18:06:09.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 -- 172.21.15.50:0/3392264669 wait complete. 2024-05-08T18:06:09.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 -- 172.21.15.50:0/3344152026 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f18d003bb50 msgr2=0x7f18d003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 --2- 172.21.15.50:0/3344152026 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f18d003bb50 0x7f18d003e010 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x558f08fd2730 tx=0x7f19e4005a00 comp rx=0 tx=0).stop 2024-05-08T18:06:09.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 -- 172.21.15.50:0/3344152026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08fc33c0 msgr2=0x558f08fc1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 --2- 172.21.15.50:0/3344152026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08fc33c0 0x558f08fc1830 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f19ec050f30 tx=0x7f19ec00db20 comp rx=0 tx=0).stop 2024-05-08T18:06:09.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 -- 172.21.15.50:0/3344152026 shutdown_connections 2024-05-08T18:06:09.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 --2- 172.21.15.50:0/3344152026 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f18d003bb50 0x7f18d003e010 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 --2- 172.21.15.50:0/3344152026 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08fc33c0 0x558f08fc1830 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 -- 172.21.15.50:0/3344152026 >> 172.21.15.50:0/3344152026 conn(0x558f08fdcf00 msgr2=0x558f08fdd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 -- 172.21.15.50:0/3344152026 shutdown_connections 2024-05-08T18:06:09.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 -- 172.21.15.50:0/3344152026 wait complete. 2024-05-08T18:06:09.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 -- 172.21.15.50:0/1114157385 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558f08fd4180 msgr2=0x558f08fd6570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.565+0000 7f19fd936800 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x558f08fd4180 0x558f08fd6570 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.566+0000 7f19fd936800 1 -- 172.21.15.50:0/1114157385 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558f08ed4a00 msgr2=0x558f08fd1920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.566+0000 7f19fd936800 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x558f08ed4a00 0x558f08fd1920 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.566+0000 7f19fd936800 1 -- 172.21.15.50:0/1114157385 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558f08fd80c0 msgr2=0x558f08fda4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.566+0000 7f19fd936800 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x558f08fd80c0 0x558f08fda4b0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.566+0000 7f19fd936800 1 -- 172.21.15.50:0/1114157385 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f18e003bba0 msgr2=0x7f18e003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.566+0000 7f19fd936800 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f18e003bba0 0x7f18e003e060 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f19e4054000 tx=0x7f19e404e000 comp rx=0 tx=0).stop 2024-05-08T18:06:09.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.566+0000 7f19fd936800 1 -- 172.21.15.50:0/1114157385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed0570 msgr2=0x558f08fca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.566+0000 7f19fd936800 1 --2- 172.21.15.50:0/1114157385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f08ed0570 0x558f08fca6e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f19ec000c00 tx=0x7f19ec000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.566+0000 7f19f9e30640 1 -- 172.21.15.50:0/1114157385 reap_dead start 2024-05-08T18:06:09.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.566+0000 7f19fd936800 1 -- 172.21.15.50:0/1114157385 shutdown_connections 2024-05-08T18:06:09.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.566+0000 7f19fd936800 1 -- 172.21.15.50:0/1114157385 >> 172.21.15.50:0/1114157385 conn(0x558f08ecbb70 msgr2=0x558f08fbbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.566+0000 7f19fd936800 1 -- 172.21.15.50:0/1114157385 shutdown_connections 2024-05-08T18:06:09.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.566+0000 7f19fd936800 1 -- 172.21.15.50:0/1114157385 wait complete. 2024-05-08T18:06:09.573 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud', '--purge-data'] 2024-05-08T18:06:09.573 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud', '--purge-data'] 2024-05-08T18:06:09.573 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud --purge-data 2024-05-08T18:06:09.620 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:09.620 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:09.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.623+0000 7f770ca87800 1 Processor -- start 2024-05-08T18:06:09.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.623+0000 7f770ca87800 1 -- start start 2024-05-08T18:06:09.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.623+0000 7f770ca87800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55f11f4ddb00 0x55f11f4dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.623+0000 7f770ca87800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55f11f4e04c0 con 0x55f11f4d9710 2024-05-08T18:06:09.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.623+0000 7f770ca87800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55f11f379e20 con 0x55f11f4ddb00 2024-05-08T18:06:09.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.624+0000 7f7709d7e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55f11f4ddb00 0x55f11f4dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.624+0000 7f7709d7e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55f11f4ddb00 0x55f11f4dfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50682/0 (socket says 172.21.15.50:50682) 2024-05-08T18:06:09.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.624+0000 7f7709d7e640 1 -- 172.21.15.50:0/4061552702 learned_addr learned my addr 172.21.15.50:0/4061552702 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.624+0000 7f770a57f640 1 --1- 172.21.15.50:0/4061552702 >> v1:172.21.15.46:6789/0 conn(0x55f11f4d9710 0x55f11f4d9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:55726/0 (socket says 172.21.15.50:55726) 2024-05-08T18:06:09.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.624+0000 7f7709d7e640 1 -- 172.21.15.50:0/4061552702 >> v1:172.21.15.46:6789/0 conn(0x55f11f4d9710 legacy=0x55f11f4d9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.624+0000 7f7709d7e640 1 -- 172.21.15.50:0/4061552702 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11f4e16f0 con 0x55f11f4ddb00 2024-05-08T18:06:09.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.626+0000 7f7709d7e640 1 --2- 172.21.15.50:0/4061552702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4ddb00 0x55f11f4dfef0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f770000a520 tx=0x7f770001d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ee8ecc800a409 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.626+0000 7f770957d640 1 -- 172.21.15.50:0/4061552702 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7700057070 con 0x55f11f4ddb00 2024-05-08T18:06:09.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.626+0000 7f770957d640 1 -- 172.21.15.50:0/4061552702 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7700052570 con 0x55f11f4ddb00 2024-05-08T18:06:09.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.626+0000 7f770957d640 1 -- 172.21.15.50:0/4061552702 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f770004ae60 con 0x55f11f4ddb00 2024-05-08T18:06:09.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.626+0000 7f770ca87800 1 -- 172.21.15.50:0/4061552702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4ddb00 msgr2=0x55f11f4dfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.626+0000 7f770ca87800 1 --2- 172.21.15.50:0/4061552702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4ddb00 0x55f11f4dfef0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f770000a520 tx=0x7f770001d010 comp rx=0 tx=0).stop 2024-05-08T18:06:09.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.626+0000 7f770ca87800 1 -- 172.21.15.50:0/4061552702 shutdown_connections 2024-05-08T18:06:09.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.626+0000 7f770ca87800 1 --2- 172.21.15.50:0/4061552702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4ddb00 0x55f11f4dfef0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.626+0000 7f770ca87800 1 -- 172.21.15.50:0/4061552702 >> 172.21.15.50:0/4061552702 conn(0x55f11f4d4c90 msgr2=0x55f11f4d70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.627+0000 7f770ca87800 1 -- 172.21.15.50:0/4061552702 shutdown_connections 2024-05-08T18:06:09.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.627+0000 7f770ca87800 1 -- 172.21.15.50:0/4061552702 wait complete. 2024-05-08T18:06:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.642+0000 7f770ca87800 1 Processor -- start 2024-05-08T18:06:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.642+0000 7f770ca87800 1 -- start start 2024-05-08T18:06:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.642+0000 7f770ca87800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4d9710 0x55f11f5cab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.642+0000 7f770ca87800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f11f424a30 con 0x55f11f4d9710 2024-05-08T18:06:09.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.642+0000 7f770a57f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4d9710 0x55f11f5cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.642+0000 7f770a57f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4d9710 0x55f11f5cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50692/0 (socket says 172.21.15.50:50692) 2024-05-08T18:06:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.642+0000 7f770a57f640 1 -- 172.21.15.50:0/1540992445 learned_addr learned my addr 172.21.15.50:0/1540992445 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.643+0000 7f770a57f640 1 -- 172.21.15.50:0/1540992445 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11f5cb100 con 0x55f11f4d9710 2024-05-08T18:06:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.643+0000 7f770a57f640 1 --2- 172.21.15.50:0/1540992445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4d9710 0x55f11f5cab30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f76fc00b260 tx=0x7f76fc00b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c05572ad52f084 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.643+0000 7f75f8608640 1 -- 172.21.15.50:0/1540992445 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc00be60 con 0x55f11f4d9710 2024-05-08T18:06:09.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.643+0000 7f75f8608640 1 -- 172.21.15.50:0/1540992445 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76fc006070 con 0x55f11f4d9710 2024-05-08T18:06:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.643+0000 7f75f8608640 1 -- 172.21.15.50:0/1540992445 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc00e710 con 0x55f11f4d9710 2024-05-08T18:06:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.644+0000 7f770ca87800 1 -- 172.21.15.50:0/1540992445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4d9710 msgr2=0x55f11f5cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.644+0000 7f770ca87800 1 --2- 172.21.15.50:0/1540992445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4d9710 0x55f11f5cab30 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f76fc00b260 tx=0x7f76fc00b660 comp rx=0 tx=0).stop 2024-05-08T18:06:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.644+0000 7f770ca87800 1 -- 172.21.15.50:0/1540992445 shutdown_connections 2024-05-08T18:06:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.644+0000 7f770ca87800 1 --2- 172.21.15.50:0/1540992445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4d9710 0x55f11f5cab30 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.644+0000 7f770ca87800 1 -- 172.21.15.50:0/1540992445 >> 172.21.15.50:0/1540992445 conn(0x55f11f4d4c90 msgr2=0x55f11f5c6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.644+0000 7f770ca87800 1 -- 172.21.15.50:0/1540992445 shutdown_connections 2024-05-08T18:06:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.644+0000 7f770ca87800 1 -- 172.21.15.50:0/1540992445 wait complete. 2024-05-08T18:06:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.644+0000 7f770ca87800 1 Processor -- start 2024-05-08T18:06:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.644+0000 7f770ca87800 1 -- start start 2024-05-08T18:06:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.644+0000 7f770ca87800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4d9710 0x55f11f44eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.644+0000 7f770ca87800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f11f4e0990 con 0x55f11f4d9710 2024-05-08T18:06:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.645+0000 7f770a57f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4d9710 0x55f11f44eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.645+0000 7f770a57f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4d9710 0x55f11f44eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50696/0 (socket says 172.21.15.50:50696) 2024-05-08T18:06:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.645+0000 7f770a57f640 1 -- 172.21.15.50:0/699784423 learned_addr learned my addr 172.21.15.50:0/699784423 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.645+0000 7f770a57f640 1 -- 172.21.15.50:0/699784423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11f44d5d0 con 0x55f11f4d9710 2024-05-08T18:06:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.645+0000 7f770a57f640 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4d9710 0x55f11f44eee0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x55f11f5cb8c0 tx=0x7f76fc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.645+0000 7f75f6e05640 1 -- 172.21.15.50:0/699784423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc01a7c0 con 0x55f11f4d9710 2024-05-08T18:06:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.645+0000 7f75f6e05640 1 -- 172.21.15.50:0/699784423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76fc01a980 con 0x55f11f4d9710 2024-05-08T18:06:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.645+0000 7f770ca87800 1 -- 172.21.15.50:0/699784423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f11f44d860 con 0x55f11f4d9710 2024-05-08T18:06:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.645+0000 7f75f6e05640 1 -- 172.21.15.50:0/699784423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc01aca0 con 0x55f11f4d9710 2024-05-08T18:06:09.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.645+0000 7f770ca87800 1 -- 172.21.15.50:0/699784423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11f44dd80 con 0x55f11f4d9710 2024-05-08T18:06:09.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.646+0000 7f75f6e05640 1 -- 172.21.15.50:0/699784423 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f76fc017070 con 0x55f11f4d9710 2024-05-08T18:06:09.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.646+0000 7f75f6e05640 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f75f003bba0 0x7f75f003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.646+0000 7f75f6e05640 1 -- 172.21.15.50:0/699784423 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f76fc015020 con 0x55f11f4d9710 2024-05-08T18:06:09.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.647+0000 7f7709d7e640 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f75f003bba0 0x7f75f003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.647+0000 7f770ca87800 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f11f4ddb00 0x55f11f5dbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.647+0000 7f770ca87800 1 -- 172.21.15.50:0/699784423 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f5dc070 con 0x55f11f4ddb00 2024-05-08T18:06:09.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.647+0000 7f770ad80640 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f11f4ddb00 0x55f11f5dbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.647+0000 7f7709d7e640 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f75f003bba0 0x7f75f003e060 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7700054000 tx=0x7f770004e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.647+0000 7f770ad80640 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f11f4ddb00 0x55f11f5dbaa0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.647+0000 7f770ad80640 1 -- 172.21.15.50:0/699784423 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f770400a4a0 con 0x55f11f4ddb00 2024-05-08T18:06:09.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.647+0000 7f770ca87800 1 -- 172.21.15.50:0/699784423 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f44e110 con 0x55f11f4ddb00 2024-05-08T18:06:09.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.648+0000 7f770ad80640 1 -- 172.21.15.50:0/699784423 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f770400a4a0 con 0x55f11f4ddb00 2024-05-08T18:06:09.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.648+0000 7f770ca87800 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f11f5de3c0 0x55f11f5e07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.648+0000 7f770ca87800 1 -- 172.21.15.50:0/699784423 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f5e0e50 con 0x55f11f5de3c0 2024-05-08T18:06:09.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.648+0000 7f770a57f640 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f11f5de3c0 0x55f11f5e07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.648+0000 7f770a57f640 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f11f5de3c0 0x55f11f5e07b0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.649+0000 7f770a57f640 1 -- 172.21.15.50:0/699784423 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f76fc02e070 con 0x55f11f5de3c0 2024-05-08T18:06:09.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.649+0000 7f770ca87800 1 -- 172.21.15.50:0/699784423 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f5dd260 con 0x55f11f4ddb00 2024-05-08T18:06:09.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.649+0000 7f770ad80640 1 -- 172.21.15.50:0/699784423 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f770400a4a0 con 0x55f11f4ddb00 2024-05-08T18:06:09.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.649+0000 7f770ca87800 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f11f5e22b0 0x55f11f5e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.649+0000 7f770ca87800 1 -- 172.21.15.50:0/699784423 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f5e4da0 con 0x55f11f5e22b0 2024-05-08T18:06:09.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.649+0000 7f7709d7e640 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f11f5e22b0 0x55f11f5e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.650+0000 7f7709d7e640 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f11f5e22b0 0x55f11f5e46a0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.650+0000 7f7709d7e640 1 -- 172.21.15.50:0/699784423 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f770005a040 con 0x55f11f5e22b0 2024-05-08T18:06:09.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.650+0000 7f770ca87800 1 Processor -- start 2024-05-08T18:06:09.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.650+0000 7f770ca87800 1 -- start start 2024-05-08T18:06:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.650+0000 7f770ca87800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f5f3110 0x55f11f5f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.650+0000 7f770ca87800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f11f3cb330 con 0x55f11f5f3110 2024-05-08T18:06:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.651+0000 7f770a57f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f5f3110 0x55f11f5f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.651+0000 7f770a57f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f5f3110 0x55f11f5f3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50702/0 (socket says 172.21.15.50:50702) 2024-05-08T18:06:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.651+0000 7f770a57f640 1 -- 172.21.15.50:0/788207095 learned_addr learned my addr 172.21.15.50:0/788207095 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.651+0000 7f770a57f640 1 -- 172.21.15.50:0/788207095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11f5f3a70 con 0x55f11f5f3110 2024-05-08T18:06:09.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.651+0000 7f770a57f640 1 --2- 172.21.15.50:0/788207095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f5f3110 0x55f11f5f3530 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f76fc001840 tx=0x7f76fc01d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.651+0000 7f75effff640 1 -- 172.21.15.50:0/788207095 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc04a3d0 con 0x55f11f5f3110 2024-05-08T18:06:09.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.651+0000 7f75effff640 1 -- 172.21.15.50:0/788207095 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76fc04ae50 con 0x55f11f5f3110 2024-05-08T18:06:09.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.651+0000 7f770ca87800 1 -- 172.21.15.50:0/788207095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f11f5f44e0 con 0x55f11f5f3110 2024-05-08T18:06:09.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.651+0000 7f75effff640 1 -- 172.21.15.50:0/788207095 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc0531e0 con 0x55f11f5f3110 2024-05-08T18:06:09.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.651+0000 7f770ca87800 1 -- 172.21.15.50:0/788207095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11f5f49e0 con 0x55f11f5f3110 2024-05-08T18:06:09.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.652+0000 7f75effff640 1 -- 172.21.15.50:0/788207095 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f76fc01b590 con 0x55f11f5f3110 2024-05-08T18:06:09.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.652+0000 7f75effff640 1 --2- 172.21.15.50:0/788207095 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f75e403bb50 0x7f75e403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.652+0000 7f75effff640 1 -- 172.21.15.50:0/788207095 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f76fc03d9d0 con 0x55f11f5f3110 2024-05-08T18:06:09.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.652+0000 7f7709d7e640 1 --2- 172.21.15.50:0/788207095 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f75e403bb50 0x7f75e403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.653+0000 7f770ca87800 1 Processor -- start 2024-05-08T18:06:09.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.653+0000 7f770ca87800 1 -- start start 2024-05-08T18:06:09.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.653+0000 7f7709d7e640 1 --2- 172.21.15.50:0/788207095 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f75e403bb50 0x7f75e403e010 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55f11f44e030 tx=0x7f7700005ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.653+0000 7f770ca87800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f6001a0 0x55f11f6005c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.653+0000 7f770ca87800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f11f5cb480 con 0x55f11f6001a0 2024-05-08T18:06:09.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.653+0000 7f770a57f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f6001a0 0x55f11f6005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.653+0000 7f770a57f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f6001a0 0x55f11f6005c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50704/0 (socket says 172.21.15.50:50704) 2024-05-08T18:06:09.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.653+0000 7f770a57f640 1 -- 172.21.15.50:0/1470272345 learned_addr learned my addr 172.21.15.50:0/1470272345 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.653+0000 7f770a57f640 1 -- 172.21.15.50:0/1470272345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11f600b90 con 0x55f11f6001a0 2024-05-08T18:06:09.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.654+0000 7f770a57f640 1 --2- 172.21.15.50:0/1470272345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f6001a0 0x55f11f6005c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f76fc00ef10 tx=0x7f76fc00fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a891ccda55fb7f1d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.654+0000 7f75edffb640 1 -- 172.21.15.50:0/1470272345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc04a6f0 con 0x55f11f6001a0 2024-05-08T18:06:09.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.654+0000 7f75edffb640 1 -- 172.21.15.50:0/1470272345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76fc03a3f0 con 0x55f11f6001a0 2024-05-08T18:06:09.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.654+0000 7f75edffb640 1 -- 172.21.15.50:0/1470272345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc04a890 con 0x55f11f6001a0 2024-05-08T18:06:09.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.654+0000 7f770ca87800 1 -- 172.21.15.50:0/1470272345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f6001a0 msgr2=0x55f11f6005c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.654+0000 7f770ca87800 1 --2- 172.21.15.50:0/1470272345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f6001a0 0x55f11f6005c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f76fc00ef10 tx=0x7f76fc00fee0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.654+0000 7f770ca87800 1 -- 172.21.15.50:0/1470272345 shutdown_connections 2024-05-08T18:06:09.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.654+0000 7f770ca87800 1 --2- 172.21.15.50:0/1470272345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f6001a0 0x55f11f6005c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.654+0000 7f770ca87800 1 -- 172.21.15.50:0/1470272345 >> 172.21.15.50:0/1470272345 conn(0x55f11f5fc500 msgr2=0x55f11f5fa560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.654+0000 7f770ca87800 1 -- 172.21.15.50:0/1470272345 shutdown_connections 2024-05-08T18:06:09.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.654+0000 7f770ca87800 1 -- 172.21.15.50:0/1470272345 wait complete. 2024-05-08T18:06:09.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.655+0000 7f770ca87800 1 Processor -- start 2024-05-08T18:06:09.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.655+0000 7f770ca87800 1 -- start start 2024-05-08T18:06:09.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.655+0000 7f770ca87800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f6001a0 0x55f11f607640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.655+0000 7f770ca87800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f11f601240 con 0x55f11f6001a0 2024-05-08T18:06:09.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.655+0000 7f770a57f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f6001a0 0x55f11f607640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.655+0000 7f770a57f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f6001a0 0x55f11f607640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:50706/0 (socket says 172.21.15.50:50706) 2024-05-08T18:06:09.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.655+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 learned_addr learned my addr 172.21.15.50:0/1388377193 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:09.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.655+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f11f607b80 con 0x55f11f6001a0 2024-05-08T18:06:09.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.655+0000 7f770a57f640 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f6001a0 0x55f11f607640 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f76fc027f90 tx=0x7f76fc017460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.656+0000 7f75dbfff640 1 -- 172.21.15.50:0/1388377193 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc068070 con 0x55f11f6001a0 2024-05-08T18:06:09.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.656+0000 7f75dbfff640 1 -- 172.21.15.50:0/1388377193 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76fc0193e0 con 0x55f11f6001a0 2024-05-08T18:06:09.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.656+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f11f607b80 con 0x55f11f6001a0 2024-05-08T18:06:09.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.656+0000 7f75dbfff640 1 -- 172.21.15.50:0/1388377193 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f76fc019720 con 0x55f11f6001a0 2024-05-08T18:06:09.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.656+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f11f608020 con 0x55f11f6001a0 2024-05-08T18:06:09.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.656+0000 7f75dbfff640 1 -- 172.21.15.50:0/1388377193 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f76fc06b070 con 0x55f11f6001a0 2024-05-08T18:06:09.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.657+0000 7f75dbfff640 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f75d403bb50 0x7f75d403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.657+0000 7f75dbfff640 1 -- 172.21.15.50:0/1388377193 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f76fc05a070 con 0x55f11f6001a0 2024-05-08T18:06:09.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.657+0000 7f7709d7e640 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f75d403bb50 0x7f75d403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.657+0000 7f7709d7e640 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f75d403bb50 0x7f75d403e010 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7700000c00 tx=0x7f770004f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.657+0000 7f770ca87800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.657+0000 7f770ca87800 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f11f612da0 0x55f11f615260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.657+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6159b0 con 0x55f11f612da0 2024-05-08T18:06:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.658+0000 7f770ad80640 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f11f612da0 0x55f11f615260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.658+0000 7f770ad80640 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f11f612da0 0x55f11f615260 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.658+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f770400a4a0 con 0x55f11f612da0 2024-05-08T18:06:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.658+0000 7f770ca87800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.658+0000 7f770ca87800 20 realm 2024-05-08T18:06:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.658+0000 7f770ca87800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.658+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f615dc0 con 0x55f11f612da0 2024-05-08T18:06:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.658+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f770400a4a0 con 0x55f11f612da0 2024-05-08T18:06:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.658+0000 7f770ca87800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.658+0000 7f770ca87800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:09.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.658+0000 7f770ca87800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.658+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f616690 con 0x55f11f612da0 2024-05-08T18:06:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.659+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f770400a4a0 con 0x55f11f612da0 2024-05-08T18:06:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.659+0000 7f770ca87800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.659+0000 7f770ca87800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.659+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f616a30 con 0x55f11f612da0 2024-05-08T18:06:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.659+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f770400a4a0 con 0x55f11f612da0 2024-05-08T18:06:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.659+0000 7f770ca87800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.659+0000 7f770ca87800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.659+0000 7f770ca87800 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f11f617790 0x55f11f619b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.659+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f61a190 con 0x55f11f617790 2024-05-08T18:06:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.659+0000 7f770a57f640 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f11f617790 0x55f11f619b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.659+0000 7f770a57f640 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f11f617790 0x55f11f619b80 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.660+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.660+0000 7f770ca87800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:09.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.660+0000 7f770ca87800 20 searching for the correct realm 2024-05-08T18:06:09.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.660+0000 7f770ca87800 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f11f61c140 0x55f11f61e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.660+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55f11f61ecd0 con 0x55f11f61c140 2024-05-08T18:06:09.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.660+0000 7f7709d7e640 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f11f61c140 0x55f11f61e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.660+0000 7f7709d7e640 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f11f61c140 0x55f11f61e600 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.660+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.660+0000 7f7709d7e640 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7700074040 0x7f7700081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:09.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.660+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55f11f61ecd0 con 0x7f7700074040 2024-05-08T18:06:09.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.661+0000 7f770ad80640 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7700074040 0x7f7700081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:09.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.661+0000 7f770ad80640 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7700074040 0x7f7700081410 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:09.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.661+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f770400a4a0 con 0x7f7700074040 2024-05-08T18:06:09.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.661+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55f11f616a30 con 0x55f11f617790 2024-05-08T18:06:09.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.662+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.662+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55f11f616a30 con 0x55f11f612da0 2024-05-08T18:06:09.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.662+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f770400a4a0 con 0x55f11f612da0 2024-05-08T18:06:09.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.662+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55f11f616a30 con 0x55f11f612da0 2024-05-08T18:06:09.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.662+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f770400a4a0 con 0x55f11f612da0 2024-05-08T18:06:09.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.662+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55f11f616a30 con 0x55f11f612da0 2024-05-08T18:06:09.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.662+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f770400a4a0 con 0x55f11f612da0 2024-05-08T18:06:09.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.662+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55f11f616a30 con 0x55f11f617790 2024-05-08T18:06:09.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.663+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.663+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55f11f616a30 con 0x7f7700074040 2024-05-08T18:06:09.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.663+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f770400a4a0 con 0x7f7700074040 2024-05-08T18:06:09.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.663+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55f11f616a30 con 0x55f11f61c140 2024-05-08T18:06:09.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.663+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.663+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55f11f616a30 con 0x55f11f612da0 2024-05-08T18:06:09.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.663+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f770400a4a0 con 0x55f11f612da0 2024-05-08T18:06:09.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.663+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55f11f616a30 con 0x55f11f617790 2024-05-08T18:06:09.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.664+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.664+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55f11f616a30 con 0x55f11f612da0 2024-05-08T18:06:09.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.664+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f770400a4a0 con 0x55f11f612da0 2024-05-08T18:06:09.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.664+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55f11f616a30 con 0x7f7700074040 2024-05-08T18:06:09.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.664+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.664+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55f11f616a30 con 0x55f11f61c140 2024-05-08T18:06:09.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.665+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.665+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x55f11f61c140 2024-05-08T18:06:09.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.665+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.665+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x55f11f617790 2024-05-08T18:06:09.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.665+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.665+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x55f11f61c140 2024-05-08T18:06:09.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.665+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.666+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x55f11f61c140 2024-05-08T18:06:09.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.666+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.666+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x55f11f612da0 2024-05-08T18:06:09.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.666+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.666+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x55f11f612da0 2024-05-08T18:06:09.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.666+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.666+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x55f11f617790 2024-05-08T18:06:09.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.667+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.667+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x55f11f61c140 2024-05-08T18:06:09.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.667+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.667+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x7f7700074040 2024-05-08T18:06:09.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.667+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.667+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x55f11f612da0 2024-05-08T18:06:09.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.667+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.667+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x55f11f617790 2024-05-08T18:06:09.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.668+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.668+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x55f11f61c140 2024-05-08T18:06:09.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.668+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.668+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x55f11f617790 2024-05-08T18:06:09.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.668+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.668+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x7f7700074040 2024-05-08T18:06:09.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.669+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.669+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x55f11f61c140 2024-05-08T18:06:09.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.669+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.669+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x55f11f617790 2024-05-08T18:06:09.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.669+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.669+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x55f11f612da0 2024-05-08T18:06:09.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f770008e040 con 0x7f7700074040 2024-05-08T18:06:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ca87800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ca87800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ca87800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ca87800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ca87800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ca87800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ca87800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f61aa50 con 0x55f11f612da0 2024-05-08T18:06:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ca87800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ca87800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f61aa50 con 0x55f11f612da0 2024-05-08T18:06:09.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ca87800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ca87800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.670+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f61aa50 con 0x55f11f61c140 2024-05-08T18:06:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f770ca87800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f770ca87800 20 zone default found 2024-05-08T18:06:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f770ca87800 4 Realm: () 2024-05-08T18:06:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f770ca87800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f770ca87800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f770ca87800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f770ca87800 20 zonegroup default 2024-05-08T18:06:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f770ca87800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f61b3b0 con 0x55f11f617790 2024-05-08T18:06:09.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f770ca87800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f770ca87800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f620b90 con 0x55f11f61c140 2024-05-08T18:06:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f770ca87800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f770ca87800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.671+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f621020 con 0x55f11f61c140 2024-05-08T18:06:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.672+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.672+0000 7f770ca87800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.672+0000 7f770ca87800 20 started sync module instance, tier type = 2024-05-08T18:06:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.672+0000 7f770ca87800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.672+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f61ffd0 con 0x7f7700074040 2024-05-08T18:06:09.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.673+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.673+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94494101996752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f61ffd0 con 0x7f7700074040 2024-05-08T18:06:09.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.673+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f621ae0 con 0x55f11f612da0 2024-05-08T18:06:09.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.674+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94494101996752] v29'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.674+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.674+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94494102001136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f61ffd0 con 0x55f11f612da0 2024-05-08T18:06:09.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.674+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f622530 con 0x55f11f61c140 2024-05-08T18:06:09.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.675+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94494102001136] v29'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.675+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.675+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94494101994272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f61ff40 con 0x55f11f61c140 2024-05-08T18:06:09.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.675+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f623420 con 0x7f7700074040 2024-05-08T18:06:09.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.676+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.676+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94494101994272] v29'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.677+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94494102007664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f623840 con 0x7f7700074040 2024-05-08T18:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.677+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f623e60 con 0x55f11f612da0 2024-05-08T18:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.678+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94494102007664] v29'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.678+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.678+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94494102010288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f624280 con 0x55f11f612da0 2024-05-08T18:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.678+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f624b30 con 0x55f11f617790 2024-05-08T18:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.679+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94494102010288] v29'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.679+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.679+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94494102011824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f624f20 con 0x55f11f617790 2024-05-08T18:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.679+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f625940 con 0x55f11f617790 2024-05-08T18:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.680+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(56 notify.6 [create] v29'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.680+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94494102019296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6265b0 con 0x55f11f617790 2024-05-08T18:06:09.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.680+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f627000 con 0x55f11f617790 2024-05-08T18:06:09.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.680+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94494102011824] v29'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.681+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94494102019296] v29'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.681+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.681+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94494102022896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f626100 con 0x55f11f617790 2024-05-08T18:06:09.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.681+0000 7f770ca87800 20 add_watcher() i=0 2024-05-08T18:06:09.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.681+0000 7f770ca87800 20 add_watcher() i=1 2024-05-08T18:06:09.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.681+0000 7f770ca87800 20 add_watcher() i=2 2024-05-08T18:06:09.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.681+0000 7f770ca87800 20 add_watcher() i=3 2024-05-08T18:06:09.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.681+0000 7f770ca87800 20 add_watcher() i=4 2024-05-08T18:06:09.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.681+0000 7f770ca87800 20 add_watcher() i=5 2024-05-08T18:06:09.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.681+0000 7f770ca87800 20 add_watcher() i=6 2024-05-08T18:06:09.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.682+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94494102022896] v29'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.682+0000 7f770ca87800 20 add_watcher() i=7 2024-05-08T18:06:09.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.682+0000 7f770ca87800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:09.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.682+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f62a100 con 0x55f11f612da0 2024-05-08T18:06:09.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.682+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.682+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94494102012944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f627580 con 0x55f11f612da0 2024-05-08T18:06:09.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.683+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94494102012944] v29'223 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.683+0000 7f75c0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:09.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.683+0000 7f770ca87800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:09.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.683+0000 7f770ca87800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.684+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f632850 con 0x55f11f612da0 2024-05-08T18:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.684+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v29'33 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.685+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f632ea0 con 0x55f11f617790 2024-05-08T18:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.686+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(63 gc.1 [create,call,call,call] v29'28 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.686+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f632ea0 con 0x55f11f61c140 2024-05-08T18:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.687+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v29'27 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.687+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6334b0 con 0x55f11f612da0 2024-05-08T18:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.688+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v29'224 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.688+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f633960 con 0x7f7700074040 2024-05-08T18:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.689+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v29'71 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.689+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f633ed0 con 0x55f11f61c140 2024-05-08T18:06:09.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.690+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v29'47 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.690+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634380 con 0x55f11f612da0 2024-05-08T18:06:09.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.691+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v29'30 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.691+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6341b0 con 0x55f11f617790 2024-05-08T18:06:09.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.692+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(69 gc.7 [create,call,call,call] v29'47 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.692+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634240 con 0x7f7700074040 2024-05-08T18:06:09.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.693+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v29'27 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.693+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634350 con 0x55f11f61c140 2024-05-08T18:06:09.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.694+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v29'33 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.694+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6347f0 con 0x7f7700074040 2024-05-08T18:06:09.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.695+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v29'72 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.695+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6347f0 con 0x55f11f617790 2024-05-08T18:06:09.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.696+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(73 gc.11 [create,call,call,call] v29'31 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.696+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6347f0 con 0x7f7700074040 2024-05-08T18:06:09.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.697+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v29'28 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.697+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6347f0 con 0x55f11f617790 2024-05-08T18:06:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.698+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(75 gc.13 [create,call,call,call] v29'53 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.698+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6347f0 con 0x55f11f612da0 2024-05-08T18:06:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.699+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v29'225 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.699+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6347f0 con 0x55f11f61c140 2024-05-08T18:06:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.700+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v29'66 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.700+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634e00 con 0x7f7700074040 2024-05-08T18:06:09.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.701+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v29'73 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.701+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634de0 con 0x55f11f61c140 2024-05-08T18:06:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.702+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v29'27 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.702+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634de0 con 0x55f11f612da0 2024-05-08T18:06:09.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.703+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v29'35 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.703+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634c10 con 0x55f11f612da0 2024-05-08T18:06:09.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.704+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v29'226 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.704+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634c10 con 0x55f11f61c140 2024-05-08T18:06:09.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.705+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v29'67 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.705+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634c10 con 0x7f7700074040 2024-05-08T18:06:09.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.706+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v29'37 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.706+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634c10 con 0x55f11f61c140 2024-05-08T18:06:09.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.707+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v29'29 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.707+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634c10 con 0x55f11f61c140 2024-05-08T18:06:09.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.708+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v29'33 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.708+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634c10 con 0x55f11f61c140 2024-05-08T18:06:09.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.709+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v29'25 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.707 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.709+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634c10 con 0x55f11f61c140 2024-05-08T18:06:09.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.710+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v29'48 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.710+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634c10 con 0x55f11f617790 2024-05-08T18:06:09.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.712+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(88 gc.26 [create,call,call,call] v29'29 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.712+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634c10 con 0x55f11f61c140 2024-05-08T18:06:09.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.713+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v29'27 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.713+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634c10 con 0x55f11f617790 2024-05-08T18:06:09.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.714+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(90 gc.28 [create,call,call,call] v29'48 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.714+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634c10 con 0x55f11f61c140 2024-05-08T18:06:09.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.715+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v29'68 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.715+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634c10 con 0x55f11f617790 2024-05-08T18:06:09.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.716+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(92 gc.30 [create,call,call,call] v29'54 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.714 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.716+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f634c10 con 0x7f7700074040 2024-05-08T18:06:09.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.718+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v29'33 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.718+0000 7f753b7fe640 20 reqs_thread_entry: start 2024-05-08T18:06:09.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.718+0000 7f770ca87800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:09.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.718+0000 7f770ca87800 20 Filter name: none 2024-05-08T18:06:09.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.718+0000 7f75397fa640 20 reqs_thread_entry: start 2024-05-08T18:06:09.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.718+0000 7f770ca87800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-05-08T18:06:09.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.718+0000 7f770ca87800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.718+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f65fa10 con 0x55f11f617790 2024-05-08T18:06:09.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.719+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(94 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.719+0000 7f770ca87800 20 rados_obj.operate() r=0 bl.length=380 2024-05-08T18:06:09.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.719+0000 7f770ca87800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-05-08T18:06:09.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.719+0000 7f770ca87800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-05-08T18:06:09.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.719+0000 7f770ca87800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:09.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.719+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:95 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6624a0 con 0x55f11f617790 2024-05-08T18:06:09.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.720+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(95 fud.buckets [call out=186b] v0'0 uv5 ondisk = 0) ==== 155+0+186 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.720+0000 7f770ca87800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo7:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10 : miss 2024-05-08T18:06:09.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.720+0000 7f770ca87800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.720+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:96 8.4 8:25389097:root::.bucket.meta.myfoo7%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f662380 con 0x7f7700074040 2024-05-08T18:06:09.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.720+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(96 .bucket.meta.myfoo7:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10 [call out=48b,stat out=16b,read 0~382 out=382b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 334+0+664 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.720+0000 7f770ca87800 20 rados_obj.operate() r=0 bl.length=382 2024-05-08T18:06:09.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.720+0000 7f770ca87800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10 info.flags=0x17 2024-05-08T18:06:09.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.720+0000 7f770ca87800 10 adding default.rgw.meta+root+.bucket.meta.myfoo7:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10 to cache LRU end 2024-05-08T18:06:09.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.720+0000 7f770ca87800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:09.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.720+0000 7f770ca87800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:06:09.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.720+0000 7f770ca87800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo7:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10 2024-05-08T18:06:09.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.720+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:97 4.19 4:996f7102:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df870 con 0x55f11f61c140 2024-05-08T18:06:09.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.720+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(97 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.720+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:98 4.b 4:d037c012:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df8b0 con 0x7f7700074040 2024-05-08T18:06:09.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.721+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(98 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.1 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7704032040 con 0x7f7700074040 2024-05-08T18:06:09.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.721+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:99 4.7 4:e23db9fa:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df8f0 con 0x55f11f617790 2024-05-08T18:06:09.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.721+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(99 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.2 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.721+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:100 4.2b 4:d6fede13:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df8f0 con 0x55f11f612da0 2024-05-08T18:06:09.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.722+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(100 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.3 [call out=85b] v0'0 uv2 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.722+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:101 4.5 4:a287cb1c:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df8f0 con 0x55f11f61c140 2024-05-08T18:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.722+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 28 ==== osd_op_reply(101 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.722+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:102 4.26 4:648658f7:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df8f0 con 0x55f11f617790 2024-05-08T18:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.722+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(102 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.5 [call out=378b] v0'0 uv17 ondisk = 0) ==== 195+0+378 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.723+0000 7f770ca87800 10 cls_bucket_list_unordered: got fourteen 2024-05-08T18:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.723+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:103 4.3c 4:3c1744ff:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df8f0 con 0x55f11f612da0 2024-05-08T18:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.723+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(103 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.6 [call out=85b] v0'0 uv5 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.723+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:104 4.33 4:ce14587e:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df8f0 con 0x55f11f617790 2024-05-08T18:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.723+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(104 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.7 [call out=85b] v0'0 uv7 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.723+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:105 4.3c 4:3c841d73:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df8f0 con 0x55f11f612da0 2024-05-08T18:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.724+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(105 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.8 [call out=85b] v0'0 uv6 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.724+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:106 4.37 4:ede12abf:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df8f0 con 0x55f11f617790 2024-05-08T18:06:09.724 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.724+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(106 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.9 [call out=85b] v0'0 uv6 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.724+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:107 4.32 4:4dfd62fe:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df8f0 con 0x55f11f612da0 2024-05-08T18:06:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.725+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(107 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.10 [call out=85b] v0'0 uv7 ondisk = 0) ==== 196+0+85 (crc 0 0 0) 0x7f7704032040 con 0x55f11f612da0 2024-05-08T18:06:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.725+0000 7f770ca87800 20 get_obj_state: octx=0x7f76fc06d450 obj=myfoo7:fourteen state=0x55f11f6dd8f8 s->prefetch_data=0 2024-05-08T18:06:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.725+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:108 3.9s0 3:92e49b35:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10_fourteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6de720 con 0x55f11f61c140 2024-05-08T18:06:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.725+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 29 ==== osd_op_reply(108 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10_fourteen [getxattrs out=858b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 239+0+874 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.725+0000 7f770ca87800 10 manifest: total_size = 8 2024-05-08T18:06:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.725+0000 7f770ca87800 20 get_obj_state: setting s->obj_tag to 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11128921894693864694 2024-05-08T18:06:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.725+0000 7f770ca87800 20 get_obj_state: octx=0x7f76fc06d450 obj=myfoo7:fourteen state=0x55f11f6dd8f8 s->prefetch_data=0 2024-05-08T18:06:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.725+0000 7f770ca87800 20 prepare_atomic_modification: state is not atomic. state=0x55f11f6dd8f8 2024-05-08T18:06:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.725+0000 7f770ca87800 20 bucket index object: :.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.5 2024-05-08T18:06:09.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.725+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:109 4.26 4:648658f7:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=188b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6defe0 con 0x55f11f617790 2024-05-08T18:06:09.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.726+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== osd_op_reply(109 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.5 [stat,call,call] v29'19 uv19 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.726+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:110 3.9s0 3:92e49b35:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10_fourteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6defe0 con 0x55f11f61c140 2024-05-08T18:06:09.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.728+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 30 ==== osd_op_reply(110 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10_fourteen [call] v29'2 uv2 ondisk = 0) ==== 197+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.728+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:111 4.26 4:648658f7:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=265b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df0b0 con 0x55f11f617790 2024-05-08T18:06:09.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.728+0000 7f770ca87800 20 list_objects_ordered: starting attempt 1 2024-05-08T18:06:09.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.728+0000 7f770ca87800 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-08T18:06:09.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.728+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:112 4.19 4:996f7102:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df0b0 con 0x55f11f61c140 2024-05-08T18:06:09.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.728+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:113 4.b 4:d037c012:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f66adf0 con 0x7f7700074040 2024-05-08T18:06:09.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.728+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:114 4.7 4:e23db9fa:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f66d750 con 0x55f11f617790 2024-05-08T18:06:09.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.728+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:115 4.2b 4:d6fede13:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6701e0 con 0x55f11f612da0 2024-05-08T18:06:09.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.728+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:116 4.5 4:a287cb1c:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df0b0 con 0x55f11f61c140 2024-05-08T18:06:09.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 20 ==== osd_op_reply(113 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.1 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f770404e060 con 0x7f7700074040 2024-05-08T18:06:09.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:117 4.26 4:648658f7:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f66adf0 con 0x55f11f617790 2024-05-08T18:06:09.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 32 ==== osd_op_reply(115 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.3 [call out=85b] v0'0 uv2 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:118 4.3c 4:3c1744ff:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f675630 con 0x55f11f612da0 2024-05-08T18:06:09.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 31 ==== osd_op_reply(114 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.2 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:119 4.33 4:ce14587e:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f66e1c0 con 0x55f11f617790 2024-05-08T18:06:09.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 31 ==== osd_op_reply(112 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 32 ==== osd_op_reply(116 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:120 4.3c 4:3c841d73:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f66adf0 con 0x55f11f612da0 2024-05-08T18:06:09.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:121 4.37 4:ede12abf:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f675630 con 0x55f11f617790 2024-05-08T18:06:09.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:122 4.32 4:4dfd62fe:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f66e1c0 con 0x55f11f612da0 2024-05-08T18:06:09.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 32 ==== osd_op_reply(111 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.5 [stat,call,call] v29'20 uv20 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f75edffb640 20 handle_completion(): completion ok for obj=fourteen 2024-05-08T18:06:09.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 33 ==== osd_op_reply(118 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.6 [call out=85b] v0'0 uv5 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 33 ==== osd_op_reply(119 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.7 [call out=85b] v0'0 uv7 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 34 ==== osd_op_reply(117 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.5 [call out=124b] v0'0 uv20 ondisk = 0) ==== 195+0+124 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 34 ==== osd_op_reply(122 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.10 [call out=85b] v0'0 uv7 ondisk = 0) ==== 196+0+85 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 35 ==== osd_op_reply(121 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.9 [call out=85b] v0'0 uv6 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 35 ==== osd_op_reply(120 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.8 [call out=85b] v0'0 uv6 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-08T18:06:09.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-08T18:06:09.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-08T18:06:09.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-08T18:06:09.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo7[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10])"; uploads.size()=0; is_truncated=0 2024-05-08T18:06:09.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:123 4.19 4:996f7102:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df060 con 0x55f11f61c140 2024-05-08T18:06:09.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:124 4.b 4:d037c012:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f66e0b0 con 0x7f7700074040 2024-05-08T18:06:09.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:125 4.7 4:e23db9fa:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f66f6b0 con 0x55f11f617790 2024-05-08T18:06:09.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:126 4.2b 4:d6fede13:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df060 con 0x55f11f612da0 2024-05-08T18:06:09.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:127 4.5 4:a287cb1c:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6751e0 con 0x55f11f61c140 2024-05-08T18:06:09.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:128 4.26 4:648658f7:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f66f6b0 con 0x55f11f617790 2024-05-08T18:06:09.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:129 4.3c 4:3c1744ff:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f66e0b0 con 0x55f11f612da0 2024-05-08T18:06:09.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.729+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:130 4.33 4:ce14587e:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df060 con 0x55f11f617790 2024-05-08T18:06:09.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.730+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:131 4.3c 4:3c841d73:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f66f6b0 con 0x55f11f612da0 2024-05-08T18:06:09.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.730+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:132 4.37 4:ede12abf:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df060 con 0x55f11f617790 2024-05-08T18:06:09.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.730+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:133 4.32 4:4dfd62fe:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f675620 con 0x55f11f612da0 2024-05-08T18:06:09.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.730+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 36 ==== osd_op_reply(125 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.2 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.730+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 21 ==== osd_op_reply(124 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.1 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f770404e060 con 0x7f7700074040 2024-05-08T18:06:09.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.730+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 37 ==== osd_op_reply(128 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.5 [call out=124b] v0'0 uv20 ondisk = 0) ==== 195+0+124 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.730+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 33 ==== osd_op_reply(123 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.730+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 38 ==== osd_op_reply(130 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.7 [call out=85b] v0'0 uv7 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.730+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 39 ==== osd_op_reply(132 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.9 [call out=85b] v0'0 uv6 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.730+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 34 ==== osd_op_reply(127 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.730+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 36 ==== osd_op_reply(126 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.3 [call out=85b] v0'0 uv2 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.730+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 37 ==== osd_op_reply(129 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.6 [call out=85b] v0'0 uv5 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.730+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 38 ==== osd_op_reply(133 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.10 [call out=85b] v0'0 uv7 ondisk = 0) ==== 196+0+85 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.730+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 39 ==== osd_op_reply(131 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.8 [call out=85b] v0'0 uv6 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.730+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:134 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df060 con 0x55f11f617790 2024-05-08T18:06:09.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.731+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 40 ==== osd_op_reply(134 fud.buckets [call] v29'6 uv6 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.731+0000 7f770ca87800 10 cache get: name=default.rgw.meta+root+myfoo7 : miss 2024-05-08T18:06:09.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.731+0000 7f770ca87800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.731+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:135 8.9 8:938c738d:root::myfoo7:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df060 con 0x7f7700074040 2024-05-08T18:06:09.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.732+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 22 ==== osd_op_reply(135 myfoo7 [call out=48b,read 0~168 out=168b] v0'0 uv46 ondisk = 0) ==== 192+0+216 (crc 0 0 0) 0x7f770404e060 con 0x7f7700074040 2024-05-08T18:06:09.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.732+0000 7f770ca87800 20 rados_obj.operate() r=0 bl.length=168 2024-05-08T18:06:09.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.732+0000 7f770ca87800 10 cache put: name=default.rgw.meta+root+myfoo7 info.flags=0x11 2024-05-08T18:06:09.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.732+0000 7f770ca87800 10 adding default.rgw.meta+root+myfoo7 to cache LRU end 2024-05-08T18:06:09.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.732+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:136 8.9 8:938c738d:root::myfoo7:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df060 con 0x7f7700074040 2024-05-08T18:06:09.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.733+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 23 ==== osd_op_reply(136 myfoo7 [call,call,delete] v29'47 uv47 ondisk = 0) ==== 234+0+0 (crc 0 0 0) 0x7f770404e060 con 0x7f7700074040 2024-05-08T18:06:09.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.733+0000 7f770ca87800 10 removing default.rgw.meta+root+myfoo7 from cache 2024-05-08T18:06:09.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.733+0000 7f770ca87800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo7, ofs0, ns] 2024-05-08T18:06:09.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.733+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:137 7.e 7:7759931f:::notify.3:head [notify cookie 94494102313984 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df060 con 0x7f7700074040 2024-05-08T18:06:09.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.733+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 24 ==== watch-notify(notify (1) cookie 94494102007664 notify 124554051602 ret 0) ==== 192+0+0 (crc 0 0 0) 0x55f11f661880 con 0x7f7700074040 2024-05-08T18:06:09.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.733+0000 7f75ed7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051602 cookie 94494102007664 notifier 4753 bl.length()=150 2024-05-08T18:06:09.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.733+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 25 ==== osd_op_reply(137 notify.3 [notify cookie 94494102313984 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f770404e060 con 0x7f7700074040 2024-05-08T18:06:09.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.733+0000 7f75ed7fa640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:138 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f75400032c0 con 0x7f7700074040 2024-05-08T18:06:09.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.733+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 26 ==== osd_op_reply(138 notify.3 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f770404e060 con 0x7f7700074040 2024-05-08T18:06:09.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.733+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 27 ==== watch-notify(notify_complete (2) cookie 94494102313984 notify 124554051602 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55f11f632220 con 0x7f7700074040 2024-05-08T18:06:09.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.733+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:139 8.4 8:25389097:root::.bucket.meta.myfoo7%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f672630 con 0x7f7700074040 2024-05-08T18:06:09.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.734+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 28 ==== osd_op_reply(139 .bucket.meta.myfoo7:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10 [call,call,delete] v29'5 uv5 ondisk = 0) ==== 292+0+0 (crc 0 0 0) 0x7f770404e060 con 0x7f7700074040 2024-05-08T18:06:09.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.734+0000 7f770ca87800 10 removing default.rgw.meta+root+.bucket.meta.myfoo7:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10 from cache 2024-05-08T18:06:09.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.734+0000 7f770ca87800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo7:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10, ofs0, ns] 2024-05-08T18:06:09.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.734+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:140 7.d 7:b4812045:::notify.4:head [notify cookie 94494102313984 in=220b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6622f0 con 0x55f11f612da0 2024-05-08T18:06:09.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 40 ==== watch-notify(notify (1) cookie 94494102010288 notify 124554051613 ret 0) ==== 250+0+0 (crc 0 0 0) 0x55f11f632220 con 0x55f11f612da0 2024-05-08T18:06:09.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f75ed7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051613 cookie 94494102010288 notifier 4753 bl.length()=208 2024-05-08T18:06:09.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f75ed7fa640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:141 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f75400032a0 con 0x55f11f612da0 2024-05-08T18:06:09.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 41 ==== osd_op_reply(140 notify.4 [notify cookie 94494102313984 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 42 ==== watch-notify(notify_complete (2) cookie 94494102313984 notify 124554051613 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55f11f633070 con 0x55f11f612da0 2024-05-08T18:06:09.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 43 ==== osd_op_reply(141 notify.4 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:142 4.19 4:996f7102:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df060 con 0x55f11f61c140 2024-05-08T18:06:09.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:143 4.b 4:d037c012:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f66fa30 con 0x7f7700074040 2024-05-08T18:06:09.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:144 4.7 4:e23db9fa:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f660ae0 con 0x55f11f617790 2024-05-08T18:06:09.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:145 4.2b 4:d6fede13:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df060 con 0x55f11f612da0 2024-05-08T18:06:09.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:146 4.5 4:a287cb1c:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f66e6a0 con 0x55f11f61c140 2024-05-08T18:06:09.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:147 4.26 4:648658f7:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f660ae0 con 0x55f11f617790 2024-05-08T18:06:09.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:148 4.3c 4:3c1744ff:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f66e6a0 con 0x55f11f612da0 2024-05-08T18:06:09.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:149 4.33 4:ce14587e:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df060 con 0x55f11f617790 2024-05-08T18:06:09.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:150 4.3c 4:3c841d73:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f670ac0 con 0x55f11f612da0 2024-05-08T18:06:09.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:151 4.37 4:ede12abf:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f671470 con 0x55f11f617790 2024-05-08T18:06:09.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.735+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:152 4.32 4:4dfd62fe:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f671e40 con 0x55f11f612da0 2024-05-08T18:06:09.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.737+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 35 ==== osd_op_reply(146 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.4 [delete] v29'2 uv2 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.737+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 44 ==== osd_op_reply(145 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.3 [delete] v29'3 uv3 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.737+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 36 ==== osd_op_reply(142 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.0 [delete] v29'2 uv2 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.737+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 29 ==== osd_op_reply(143 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.1 [delete] v29'4 uv4 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f770404e060 con 0x7f7700074040 2024-05-08T18:06:09.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.737+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 45 ==== osd_op_reply(148 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.6 [delete] v29'7 uv7 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.737+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 41 ==== osd_op_reply(149 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.7 [delete] v29'8 uv8 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.737+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 42 ==== osd_op_reply(147 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.5 [delete] v29'21 uv21 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.737+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 46 ==== osd_op_reply(150 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.8 [delete] v29'8 uv8 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.738+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 47 ==== osd_op_reply(152 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.10 [delete] v29'8 uv8 ondisk = 0) ==== 196+0+0 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.738+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 43 ==== osd_op_reply(151 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.9 [delete] v29'7 uv7 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.738+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 44 ==== osd_op_reply(144 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10.2 [delete] v29'4 uv4 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.738+0000 7f770ca87800 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo7/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10 : miss 2024-05-08T18:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.738+0000 7f770ca87800 20 rados->read ofs=0 len=0 2024-05-08T18:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.738+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:153 6.19 6:9ba8dda5:::pubsub..bucket.myfoo7%2f1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df060 con 0x55f11f612da0 2024-05-08T18:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.738+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 48 ==== osd_op_reply(153 pubsub..bucket.myfoo7/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 210+0+0 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.738+0000 7f770ca87800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.738+0000 7f770ca87800 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo7/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10 info.flags=0x0 2024-05-08T18:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.738+0000 7f770ca87800 10 adding default.rgw.log++pubsub..bucket.myfoo7/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10 to cache LRU end 2024-05-08T18:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.738+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:154 6.19 6:9ba8dda5:::pubsub..bucket.myfoo7%2f1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df060 con 0x55f11f612da0 2024-05-08T18:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.739+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 49 ==== osd_op_reply(154 pubsub..bucket.myfoo7/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.10 [delete] v29'34 uv27 ondisk = -2 ((2) No such file or directory)) ==== 210+0+0 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.739+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:155 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=147b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df060 con 0x55f11f617790 2024-05-08T18:06:09.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.740+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 45 ==== osd_op_reply(155 fud.buckets [call] v29'7 uv7 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.740+0000 7f770ca87800 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-05-08T18:06:09.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.740+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:156 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6622f0 con 0x55f11f617790 2024-05-08T18:06:09.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.741+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 46 ==== osd_op_reply(156 p5YnriCv1nAtykxBrupQ [delete] v29'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.741+0000 7f770ca87800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-05-08T18:06:09.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.741+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:157 7.15 7:a93a5511:::notify.2:head [notify cookie 94494102313984 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df060 con 0x55f11f61c140 2024-05-08T18:06:09.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.741+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 37 ==== watch-notify(notify (1) cookie 94494101994272 notify 124554051591 ret 0) ==== 212+0+0 (crc 0 0 0) 0x55f11f634920 con 0x55f11f61c140 2024-05-08T18:06:09.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.741+0000 7f75ed7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051591 cookie 94494101994272 notifier 4753 bl.length()=170 2024-05-08T18:06:09.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.741+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 38 ==== osd_op_reply(157 notify.2 [notify cookie 94494102313984 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.741+0000 7f75ed7fa640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:158 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f75400032a0 con 0x55f11f61c140 2024-05-08T18:06:09.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.741+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 39 ==== watch-notify(notify_complete (2) cookie 94494102313984 notify 124554051591 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55f11f633b30 con 0x55f11f61c140 2024-05-08T18:06:09.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.741+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 40 ==== osd_op_reply(158 notify.2 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.741+0000 7f770ca87800 10 removing email index: 2024-05-08T18:06:09.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.741+0000 7f770ca87800 10 removing user buckets index 2024-05-08T18:06:09.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.741+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:159 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6622f0 con 0x55f11f617790 2024-05-08T18:06:09.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.743+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 47 ==== osd_op_reply(159 fud.buckets [delete] v29'8 uv8 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.743+0000 7f770ca87800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2024-05-08T18:06:09.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.743+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:160 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94494102313984 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6622f0 con 0x55f11f617790 2024-05-08T18:06:09.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.743+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 48 ==== watch-notify(notify (1) cookie 94494102019296 notify 124554051625 ret 0) ==== 202+0+0 (crc 0 0 0) 0x55f11f634350 con 0x55f11f617790 2024-05-08T18:06:09.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.743+0000 7f75edffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051625 cookie 94494102019296 notifier 4753 bl.length()=160 2024-05-08T18:06:09.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.743+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 49 ==== osd_op_reply(160 notify.6 [notify cookie 94494102313984 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.743+0000 7f75edffb640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:161 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f75d0003d60 con 0x55f11f617790 2024-05-08T18:06:09.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.743+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 50 ==== osd_op_reply(161 notify.6 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.743+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 51 ==== watch-notify(notify_complete (2) cookie 94494102313984 notify 124554051625 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55f11f5f3a70 con 0x55f11f617790 2024-05-08T18:06:09.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.743+0000 7f770ca87800 10 removing user index: fud 2024-05-08T18:06:09.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.743+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:162 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6df060 con 0x55f11f617790 2024-05-08T18:06:09.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.744+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 52 ==== osd_op_reply(162 fud [call,call,delete] v29'7 uv7 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.744+0000 7f770ca87800 10 removing default.rgw.meta+users.uid+fud from cache 2024-05-08T18:06:09.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.744+0000 7f770ca87800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-05-08T18:06:09.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.744+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:163 7.3 7:c609908c:::notify.5:head [notify cookie 94494102313984 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6622f0 con 0x55f11f617790 2024-05-08T18:06:09.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.745+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 53 ==== watch-notify(notify (1) cookie 94494102011824 notify 124554051626 ret 0) ==== 194+0+0 (crc 0 0 0) 0x55f11f5f3a70 con 0x55f11f617790 2024-05-08T18:06:09.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.745+0000 7f75edffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051626 cookie 94494102011824 notifier 4753 bl.length()=152 2024-05-08T18:06:09.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.745+0000 7f75edffb640 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:164 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f75d0003d60 con 0x55f11f617790 2024-05-08T18:06:09.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.745+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 54 ==== osd_op_reply(163 notify.5 [notify cookie 94494102313984 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.745+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 55 ==== osd_op_reply(164 notify.5 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.745+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 56 ==== watch-notify(notify_complete (2) cookie 94494102313984 notify 124554051626 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55f11f44dd80 con 0x55f11f617790 2024-05-08T18:06:09.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.746+0000 7f75c0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:09.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.746+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:165 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94494102012944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f630390 con 0x55f11f612da0 2024-05-08T18:06:09.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.747+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 50 ==== osd_op_reply(165 data_loggenerations_metadata [watch unwatch cookie 94494102012944] v29'227 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.747+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:166 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94494101996752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f624410 con 0x7f7700074040 2024-05-08T18:06:09.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.748+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 30 ==== osd_op_reply(166 notify.0 [watch unwatch cookie 94494101996752] v29'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f770404e060 con 0x7f7700074040 2024-05-08T18:06:09.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.748+0000 7f770ca87800 20 remove_watcher() i=0 2024-05-08T18:06:09.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.748+0000 7f770ca87800 2 removed watcher, disabling cache 2024-05-08T18:06:09.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.748+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:167 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94494102001136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f624410 con 0x55f11f612da0 2024-05-08T18:06:09.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.749+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 51 ==== osd_op_reply(167 notify.1 [watch unwatch cookie 94494102001136] v29'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.749+0000 7f770ca87800 20 remove_watcher() i=1 2024-05-08T18:06:09.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.749+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:168 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94494101994272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6215f0 con 0x55f11f61c140 2024-05-08T18:06:09.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.750+0000 7f7709d7e640 1 -- 172.21.15.50:0/1388377193 <== osd.3 v2:172.21.15.50:6800/3130492744 41 ==== osd_op_reply(168 notify.2 [watch unwatch cookie 94494101994272] v29'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f770007f070 con 0x55f11f61c140 2024-05-08T18:06:09.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.750+0000 7f770ca87800 20 remove_watcher() i=2 2024-05-08T18:06:09.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.750+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:169 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94494102007664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f624410 con 0x7f7700074040 2024-05-08T18:06:09.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.751+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.2 v2:172.21.15.50:6811/3039646617 31 ==== osd_op_reply(169 notify.3 [watch unwatch cookie 94494102007664] v29'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f770404e060 con 0x7f7700074040 2024-05-08T18:06:09.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.751+0000 7f770ca87800 20 remove_watcher() i=3 2024-05-08T18:06:09.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.751+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:170 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94494102010288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f61fb20 con 0x55f11f612da0 2024-05-08T18:06:09.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.752+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 <== osd.1 v2:172.21.15.50:6808/2141923372 52 ==== osd_op_reply(170 notify.4 [watch unwatch cookie 94494102010288] v29'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f770404e060 con 0x55f11f612da0 2024-05-08T18:06:09.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.752+0000 7f770ca87800 20 remove_watcher() i=4 2024-05-08T18:06:09.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.752+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:171 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94494102011824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6215f0 con 0x55f11f617790 2024-05-08T18:06:09.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.753+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 57 ==== osd_op_reply(171 notify.5 [watch unwatch cookie 94494102011824] v29'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.753+0000 7f770ca87800 20 remove_watcher() i=5 2024-05-08T18:06:09.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.753+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:172 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94494102019296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f622f70 con 0x55f11f617790 2024-05-08T18:06:09.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.754+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 58 ==== osd_op_reply(172 notify.6 [watch unwatch cookie 94494102019296] v29'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.754+0000 7f770ca87800 20 remove_watcher() i=6 2024-05-08T18:06:09.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.754+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:173 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94494102022896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55f11f6239b0 con 0x55f11f617790 2024-05-08T18:06:09.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.755+0000 7f770a57f640 1 -- 172.21.15.50:0/1388377193 <== osd.0 v2:172.21.15.46:6800/337143975 59 ==== osd_op_reply(173 notify.7 [watch unwatch cookie 94494102022896] v29'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f76fc06a030 con 0x55f11f617790 2024-05-08T18:06:09.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.755+0000 7f770ca87800 20 remove_watcher() i=7 2024-05-08T18:06:09.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.755+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f11f617790 msgr2=0x55f11f619b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.755+0000 7f770ca87800 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f11f617790 0x55f11f619b80 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.755+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f11f612da0 msgr2=0x55f11f615260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.755+0000 7f770ca87800 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f11f612da0 0x55f11f615260 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.755+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7700074040 msgr2=0x7f7700081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.755+0000 7f770ca87800 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7700074040 0x7f7700081410 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.755+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f11f61c140 msgr2=0x55f11f61e600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.755+0000 7f770ca87800 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f11f61c140 0x55f11f61e600 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.755+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f75d403bb50 msgr2=0x7f75d403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.755+0000 7f770ca87800 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f75d403bb50 0x7f75d403e010 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7700000c00 tx=0x7f770004f000 comp rx=0 tx=0).stop 2024-05-08T18:06:09.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.755+0000 7f770ad80640 1 -- 172.21.15.50:0/1388377193 reap_dead start 2024-05-08T18:06:09.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.755+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f6001a0 msgr2=0x55f11f607640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.755+0000 7f770ca87800 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f6001a0 0x55f11f607640 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f76fc027f90 tx=0x7f76fc017460 comp rx=0 tx=0).stop 2024-05-08T18:06:09.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 shutdown_connections 2024-05-08T18:06:09.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 --2- 172.21.15.50:0/1388377193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f6001a0 0x55f11f607640 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 >> 172.21.15.50:0/1388377193 conn(0x55f11f5fc500 msgr2=0x55f11f5f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 shutdown_connections 2024-05-08T18:06:09.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 -- 172.21.15.50:0/1388377193 wait complete. 2024-05-08T18:06:09.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 -- 172.21.15.50:0/788207095 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f75e403bb50 msgr2=0x7f75e403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 --2- 172.21.15.50:0/788207095 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f75e403bb50 0x7f75e403e010 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55f11f44e030 tx=0x7f7700005ac0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 -- 172.21.15.50:0/788207095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f5f3110 msgr2=0x55f11f5f3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 --2- 172.21.15.50:0/788207095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f5f3110 0x55f11f5f3530 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f76fc001840 tx=0x7f76fc01d7b0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 -- 172.21.15.50:0/788207095 shutdown_connections 2024-05-08T18:06:09.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 --2- 172.21.15.50:0/788207095 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f75e403bb50 0x7f75e403e010 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 --2- 172.21.15.50:0/788207095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f5f3110 0x55f11f5f3530 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 -- 172.21.15.50:0/788207095 >> 172.21.15.50:0/788207095 conn(0x55f11f5e71b0 msgr2=0x55f11f5e75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 -- 172.21.15.50:0/788207095 shutdown_connections 2024-05-08T18:06:09.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 -- 172.21.15.50:0/788207095 wait complete. 2024-05-08T18:06:09.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 -- 172.21.15.50:0/699784423 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f11f5de3c0 msgr2=0x55f11f5e07b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55f11f5de3c0 0x55f11f5e07b0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 -- 172.21.15.50:0/699784423 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f11f4ddb00 msgr2=0x55f11f5dbaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55f11f4ddb00 0x55f11f5dbaa0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 -- 172.21.15.50:0/699784423 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f11f5e22b0 msgr2=0x55f11f5e46a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.756+0000 7f770ca87800 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55f11f5e22b0 0x55f11f5e46a0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.757+0000 7f770ca87800 1 -- 172.21.15.50:0/699784423 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f75f003bba0 msgr2=0x7f75f003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.757+0000 7f770ca87800 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f75f003bba0 0x7f75f003e060 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7700054000 tx=0x7f770004e000 comp rx=0 tx=0).stop 2024-05-08T18:06:09.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.757+0000 7f770ca87800 1 -- 172.21.15.50:0/699784423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4d9710 msgr2=0x55f11f44eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:09.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.757+0000 7f770ca87800 1 --2- 172.21.15.50:0/699784423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f11f4d9710 0x55f11f44eee0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x55f11f5cb8c0 tx=0x7f76fc000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:09.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.757+0000 7f770ad80640 1 -- 172.21.15.50:0/699784423 reap_dead start 2024-05-08T18:06:09.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.757+0000 7f770ca87800 1 -- 172.21.15.50:0/699784423 shutdown_connections 2024-05-08T18:06:09.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.757+0000 7f770ca87800 1 -- 172.21.15.50:0/699784423 >> 172.21.15.50:0/699784423 conn(0x55f11f4d4c90 msgr2=0x55f11f5c4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:09.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.757+0000 7f770ca87800 1 -- 172.21.15.50:0/699784423 shutdown_connections 2024-05-08T18:06:09.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:09.757+0000 7f770ca87800 1 -- 172.21.15.50:0/699784423 wait complete. 2024-05-08T18:06:49.765 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'trim', '--uid', 'foo'] 2024-05-08T18:06:49.766 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'trim', '--uid', 'foo'] 2024-05-08T18:06:49.766 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage trim --uid foo 2024-05-08T18:06:49.820 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:49.820 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:49.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.823+0000 7f7381738800 1 Processor -- start 2024-05-08T18:06:49.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.823+0000 7f7381738800 1 -- start start 2024-05-08T18:06:49.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.823+0000 7f7381738800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x556519778a60 0x55651977ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.823+0000 7f7381738800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55651977b420 con 0x556519775b60 2024-05-08T18:06:49.821 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.823+0000 7f7381738800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x556519615e20 con 0x556519778a60 2024-05-08T18:06:49.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.823+0000 7f737cc32640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x556519778a60 0x55651977ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.823+0000 7f737cc32640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x556519778a60 0x55651977ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45632/0 (socket says 172.21.15.50:45632) 2024-05-08T18:06:49.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.823+0000 7f737cc32640 1 -- 172.21.15.50:0/1307491132 learned_addr learned my addr 172.21.15.50:0/1307491132 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:49.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.824+0000 7f737cc32640 1 -- 172.21.15.50:0/1307491132 >> v1:172.21.15.46:6789/0 conn(0x556519775b60 legacy=0x556519775f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.822 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.824+0000 7f737cc32640 1 -- 172.21.15.50:0/1307491132 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55651977c600 con 0x556519778a60 2024-05-08T18:06:49.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.826+0000 7f737cc32640 1 --2- 172.21.15.50:0/1307491132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519778a60 0x55651977ae50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f737400a6b0 tx=0x7f737401eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=594a68a3630f4bf3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.826+0000 7f736ffff640 1 -- 172.21.15.50:0/1307491132 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7374059070 con 0x556519778a60 2024-05-08T18:06:49.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.826+0000 7f736ffff640 1 -- 172.21.15.50:0/1307491132 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7374050050 con 0x556519778a60 2024-05-08T18:06:49.824 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.826+0000 7f736ffff640 1 -- 172.21.15.50:0/1307491132 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f737400a350 con 0x556519778a60 2024-05-08T18:06:49.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.826+0000 7f7381738800 1 -- 172.21.15.50:0/1307491132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519778a60 msgr2=0x55651977ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.826+0000 7f7381738800 1 --2- 172.21.15.50:0/1307491132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519778a60 0x55651977ae50 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f737400a6b0 tx=0x7f737401eed0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.827+0000 7f7381738800 1 -- 172.21.15.50:0/1307491132 shutdown_connections 2024-05-08T18:06:49.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.827+0000 7f7381738800 1 --2- 172.21.15.50:0/1307491132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519778a60 0x55651977ae50 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.827+0000 7f7381738800 1 -- 172.21.15.50:0/1307491132 >> 172.21.15.50:0/1307491132 conn(0x55651976f2d0 msgr2=0x5565197716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:49.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.827+0000 7f7381738800 1 -- 172.21.15.50:0/1307491132 shutdown_connections 2024-05-08T18:06:49.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.827+0000 7f7381738800 1 -- 172.21.15.50:0/1307491132 wait complete. 2024-05-08T18:06:49.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.846+0000 7f7381738800 1 Processor -- start 2024-05-08T18:06:49.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.846+0000 7f7381738800 1 -- start start 2024-05-08T18:06:49.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.846+0000 7f7381738800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519775b60 0x5565198648f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.846+0000 7f7381738800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5565196c0a10 con 0x556519775b60 2024-05-08T18:06:49.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.847+0000 7f737d433640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519775b60 0x5565198648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.847+0000 7f737d433640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519775b60 0x5565198648f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45636/0 (socket says 172.21.15.50:45636) 2024-05-08T18:06:49.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.847+0000 7f737d433640 1 -- 172.21.15.50:0/2757330582 learned_addr learned my addr 172.21.15.50:0/2757330582 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:49.845 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.847+0000 7f737d433640 1 -- 172.21.15.50:0/2757330582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556519864ec0 con 0x556519775b60 2024-05-08T18:06:49.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.847+0000 7f737d433640 1 --2- 172.21.15.50:0/2757330582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519775b60 0x5565198648f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7370001680 tx=0x7f7370001b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7db0325781f83 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.848+0000 7f726b606640 1 -- 172.21.15.50:0/2757330582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7370016070 con 0x556519775b60 2024-05-08T18:06:49.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.848+0000 7f726b606640 1 -- 172.21.15.50:0/2757330582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f737000f640 con 0x556519775b60 2024-05-08T18:06:49.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.848+0000 7f726b606640 1 -- 172.21.15.50:0/2757330582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f737000f960 con 0x556519775b60 2024-05-08T18:06:49.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.848+0000 7f7381738800 1 -- 172.21.15.50:0/2757330582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519775b60 msgr2=0x5565198648f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.848+0000 7f7381738800 1 --2- 172.21.15.50:0/2757330582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519775b60 0x5565198648f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7370001680 tx=0x7f7370001b40 comp rx=0 tx=0).stop 2024-05-08T18:06:49.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.848+0000 7f7381738800 1 -- 172.21.15.50:0/2757330582 shutdown_connections 2024-05-08T18:06:49.847 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.848+0000 7f7381738800 1 --2- 172.21.15.50:0/2757330582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519775b60 0x5565198648f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.848+0000 7f7381738800 1 -- 172.21.15.50:0/2757330582 >> 172.21.15.50:0/2757330582 conn(0x55651976f2d0 msgr2=0x556519860210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:49.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.848+0000 7f7381738800 1 -- 172.21.15.50:0/2757330582 shutdown_connections 2024-05-08T18:06:49.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.848+0000 7f7381738800 1 -- 172.21.15.50:0/2757330582 wait complete. 2024-05-08T18:06:49.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.849+0000 7f7381738800 1 Processor -- start 2024-05-08T18:06:49.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.849+0000 7f7381738800 1 -- start start 2024-05-08T18:06:49.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.849+0000 7f7381738800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519775b60 0x55651986c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.849+0000 7f7381738800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55651977b8c0 con 0x556519775b60 2024-05-08T18:06:49.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.849+0000 7f737d433640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519775b60 0x55651986c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.849+0000 7f737d433640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519775b60 0x55651986c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45652/0 (socket says 172.21.15.50:45652) 2024-05-08T18:06:49.848 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.849+0000 7f737d433640 1 -- 172.21.15.50:0/2789834326 learned_addr learned my addr 172.21.15.50:0/2789834326 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:49.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.850+0000 7f737d433640 1 -- 172.21.15.50:0/2789834326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55651986cad0 con 0x556519775b60 2024-05-08T18:06:49.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.850+0000 7f737d433640 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519775b60 0x55651986c590 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x556519865680 tx=0x7f7370001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.850+0000 7f7269e03640 1 -- 172.21.15.50:0/2789834326 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7370016040 con 0x556519775b60 2024-05-08T18:06:49.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.850+0000 7f7269e03640 1 -- 172.21.15.50:0/2789834326 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73700117c0 con 0x556519775b60 2024-05-08T18:06:49.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.850+0000 7f7269e03640 1 -- 172.21.15.50:0/2789834326 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7370011b00 con 0x556519775b60 2024-05-08T18:06:49.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.850+0000 7f7381738800 1 -- 172.21.15.50:0/2789834326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55651986cd60 con 0x556519775b60 2024-05-08T18:06:49.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.851+0000 7f7381738800 1 -- 172.21.15.50:0/2789834326 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55651986f9c0 con 0x556519775b60 2024-05-08T18:06:49.850 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.851+0000 7f7269e03640 1 -- 172.21.15.50:0/2789834326 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f737001c3f0 con 0x556519775b60 2024-05-08T18:06:49.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.852+0000 7f7269e03640 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f726403bb50 0x7f726403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.852+0000 7f7269e03640 1 -- 172.21.15.50:0/2789834326 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f7370052060 con 0x556519775b60 2024-05-08T18:06:49.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.852+0000 7f737cc32640 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f726403bb50 0x7f726403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.852+0000 7f7381738800 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556519778a60 0x556519873780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.852+0000 7f7381738800 1 -- 172.21.15.50:0/2789834326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556519873d50 con 0x556519778a60 2024-05-08T18:06:49.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.852+0000 7f737dc34640 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556519778a60 0x556519873780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.852+0000 7f737cc32640 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f726403bb50 0x7f726403e010 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7374056000 tx=0x7f737400a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.852+0000 7f737dc34640 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556519778a60 0x556519873780 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.853+0000 7f737dc34640 1 -- 172.21.15.50:0/2789834326 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f73780083e0 con 0x556519778a60 2024-05-08T18:06:49.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.853+0000 7f7381738800 1 -- 172.21.15.50:0/2789834326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556519870d30 con 0x556519778a60 2024-05-08T18:06:49.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.853+0000 7f737dc34640 1 -- 172.21.15.50:0/2789834326 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f73780083e0 con 0x556519778a60 2024-05-08T18:06:49.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.853+0000 7f7381738800 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556519876040 0x556519878430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.853+0000 7f7381738800 1 -- 172.21.15.50:0/2789834326 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556519878b20 con 0x556519876040 2024-05-08T18:06:49.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.853+0000 7f737d433640 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556519876040 0x556519878430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.854+0000 7f737d433640 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556519876040 0x556519878430 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.854+0000 7f737d433640 1 -- 172.21.15.50:0/2789834326 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7370019050 con 0x556519876040 2024-05-08T18:06:49.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.854+0000 7f7381738800 1 -- 172.21.15.50:0/2789834326 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556519874eb0 con 0x556519778a60 2024-05-08T18:06:49.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.854+0000 7f737dc34640 1 -- 172.21.15.50:0/2789834326 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f73780083e0 con 0x556519778a60 2024-05-08T18:06:49.853 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.855+0000 7f7381738800 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556519879f80 0x55651987c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.855+0000 7f7381738800 1 -- 172.21.15.50:0/2789834326 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55651987ca10 con 0x556519879f80 2024-05-08T18:06:49.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.855+0000 7f737cc32640 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556519879f80 0x55651987c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.855+0000 7f737cc32640 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556519879f80 0x55651987c370 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.855+0000 7f737cc32640 1 -- 172.21.15.50:0/2789834326 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f737405c040 con 0x556519879f80 2024-05-08T18:06:49.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.856+0000 7f7381738800 1 Processor -- start 2024-05-08T18:06:49.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.856+0000 7f7381738800 1 -- start start 2024-05-08T18:06:49.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.856+0000 7f7381738800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651988f0d0 0x55651988f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.856+0000 7f7381738800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556519667330 con 0x55651988f0d0 2024-05-08T18:06:49.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.856+0000 7f737d433640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651988f0d0 0x55651988f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.856+0000 7f737d433640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651988f0d0 0x55651988f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45666/0 (socket says 172.21.15.50:45666) 2024-05-08T18:06:49.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.856+0000 7f737d433640 1 -- 172.21.15.50:0/503287350 learned_addr learned my addr 172.21.15.50:0/503287350 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:49.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.856+0000 7f737d433640 1 -- 172.21.15.50:0/503287350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55651988fa30 con 0x55651988f0d0 2024-05-08T18:06:49.857 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.857+0000 7f737d433640 1 --2- 172.21.15.50:0/503287350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651988f0d0 0x55651988f4f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7370023ff0 tx=0x7f737003ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.857+0000 7f7262ffd640 1 -- 172.21.15.50:0/503287350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73700233a0 con 0x55651988f0d0 2024-05-08T18:06:49.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.857+0000 7f7262ffd640 1 -- 172.21.15.50:0/503287350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7370046070 con 0x55651988f0d0 2024-05-08T18:06:49.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.857+0000 7f7381738800 1 -- 172.21.15.50:0/503287350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5565198904a0 con 0x55651988f0d0 2024-05-08T18:06:49.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.857+0000 7f7262ffd640 1 -- 172.21.15.50:0/503287350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73700236b0 con 0x55651988f0d0 2024-05-08T18:06:49.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.857+0000 7f7381738800 1 -- 172.21.15.50:0/503287350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565198909a0 con 0x55651988f0d0 2024-05-08T18:06:49.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.858+0000 7f7262ffd640 1 -- 172.21.15.50:0/503287350 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f73700421a0 con 0x55651988f0d0 2024-05-08T18:06:49.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.858+0000 7f7262ffd640 1 --2- 172.21.15.50:0/503287350 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f725803bb00 0x7f725803dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.858+0000 7f7262ffd640 1 -- 172.21.15.50:0/503287350 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f737003a1b0 con 0x55651988f0d0 2024-05-08T18:06:49.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.858+0000 7f737cc32640 1 --2- 172.21.15.50:0/503287350 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f725803bb00 0x7f725803dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.858+0000 7f7381738800 1 Processor -- start 2024-05-08T18:06:49.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.859+0000 7f7381738800 1 -- start start 2024-05-08T18:06:49.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.859+0000 7f737cc32640 1 --2- 172.21.15.50:0/503287350 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f725803bb00 0x7f725803dfc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5565198745f0 tx=0x7f7374005ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.859+0000 7f7381738800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651989c230 0x55651989c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.859+0000 7f7381738800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556519865240 con 0x55651989c230 2024-05-08T18:06:49.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.859+0000 7f737d433640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651989c230 0x55651989c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.859+0000 7f737d433640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651989c230 0x55651989c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45672/0 (socket says 172.21.15.50:45672) 2024-05-08T18:06:49.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.859+0000 7f737d433640 1 -- 172.21.15.50:0/1497614806 learned_addr learned my addr 172.21.15.50:0/1497614806 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:49.859 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.859+0000 7f737d433640 1 -- 172.21.15.50:0/1497614806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55651989cc20 con 0x55651989c230 2024-05-08T18:06:49.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.859+0000 7f737d433640 1 --2- 172.21.15.50:0/1497614806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651989c230 0x55651989c650 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7370038500 tx=0x7f7370051ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509cd9239075480c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.860+0000 7f7260ff9640 1 -- 172.21.15.50:0/1497614806 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7370006fc0 con 0x55651989c230 2024-05-08T18:06:49.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.860+0000 7f7260ff9640 1 -- 172.21.15.50:0/1497614806 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7370057070 con 0x55651989c230 2024-05-08T18:06:49.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.860+0000 7f7260ff9640 1 -- 172.21.15.50:0/1497614806 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73700192e0 con 0x55651989c230 2024-05-08T18:06:49.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.860+0000 7f7381738800 1 -- 172.21.15.50:0/1497614806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651989c230 msgr2=0x55651989c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.864 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.860+0000 7f7381738800 1 --2- 172.21.15.50:0/1497614806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651989c230 0x55651989c650 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7370038500 tx=0x7f7370051ae0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.860+0000 7f7381738800 1 -- 172.21.15.50:0/1497614806 shutdown_connections 2024-05-08T18:06:49.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.860+0000 7f7381738800 1 --2- 172.21.15.50:0/1497614806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651989c230 0x55651989c650 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.860+0000 7f7381738800 1 -- 172.21.15.50:0/1497614806 >> 172.21.15.50:0/1497614806 conn(0x556519898500 msgr2=0x556519896560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:49.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.860+0000 7f7381738800 1 -- 172.21.15.50:0/1497614806 shutdown_connections 2024-05-08T18:06:49.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.860+0000 7f7381738800 1 -- 172.21.15.50:0/1497614806 wait complete. 2024-05-08T18:06:49.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.861+0000 7f7381738800 1 Processor -- start 2024-05-08T18:06:49.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.861+0000 7f7381738800 1 -- start start 2024-05-08T18:06:49.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.861+0000 7f7381738800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651989c230 0x5565198a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.861+0000 7f7381738800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55651989d2f0 con 0x55651989c230 2024-05-08T18:06:49.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.861+0000 7f737d433640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651989c230 0x5565198a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.861+0000 7f737d433640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651989c230 0x5565198a3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45684/0 (socket says 172.21.15.50:45684) 2024-05-08T18:06:49.865 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.861+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 learned_addr learned my addr 172.21.15.50:0/1229454527 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:49.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.861+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565198a3ab0 con 0x55651989c230 2024-05-08T18:06:49.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.862+0000 7f737d433640 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651989c230 0x5565198a3570 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7370037420 tx=0x7f7370038490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.862+0000 7f724effd640 1 -- 172.21.15.50:0/1229454527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f737006b070 con 0x55651989c230 2024-05-08T18:06:49.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.862+0000 7f724effd640 1 -- 172.21.15.50:0/1229454527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73700192e0 con 0x55651989c230 2024-05-08T18:06:49.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.862+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5565198a3ab0 con 0x55651989c230 2024-05-08T18:06:49.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.862+0000 7f724effd640 1 -- 172.21.15.50:0/1229454527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f73700108a0 con 0x55651989c230 2024-05-08T18:06:49.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.862+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565198a4010 con 0x55651989c230 2024-05-08T18:06:49.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.863+0000 7f724effd640 1 -- 172.21.15.50:0/1229454527 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f737006e070 con 0x55651989c230 2024-05-08T18:06:49.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.863+0000 7f724effd640 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f724803bb50 0x7f724803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.863+0000 7f724effd640 1 -- 172.21.15.50:0/1229454527 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f7370061030 con 0x55651989c230 2024-05-08T18:06:49.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.863+0000 7f737cc32640 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f724803bb50 0x7f724803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.866 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.864+0000 7f737cc32640 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f724803bb50 0x7f724803e010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55651987ce10 tx=0x7f7374062040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.864+0000 7f7381738800 20 rados->read ofs=0 len=0 2024-05-08T18:06:49.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.864+0000 7f7381738800 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5565198aed70 0x5565198b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.864+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5565198b1980 con 0x5565198aed70 2024-05-08T18:06:49.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.864+0000 7f737dc34640 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5565198aed70 0x5565198b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.865+0000 7f737dc34640 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5565198aed70 0x5565198b1230 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.865+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f73780083e0 con 0x5565198aed70 2024-05-08T18:06:49.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.865+0000 7f7381738800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:49.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.865+0000 7f7381738800 20 realm 2024-05-08T18:06:49.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.865+0000 7f7381738800 20 rados->read ofs=0 len=0 2024-05-08T18:06:49.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.865+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5565198b1db0 con 0x5565198aed70 2024-05-08T18:06:49.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.865+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f73780083e0 con 0x5565198aed70 2024-05-08T18:06:49.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.865+0000 7f7381738800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:49.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.865+0000 7f7381738800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:49.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.865+0000 7f7381738800 20 rados->read ofs=0 len=0 2024-05-08T18:06:49.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.865+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198aed70 2024-05-08T18:06:49.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.866+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f73780083e0 con 0x5565198aed70 2024-05-08T18:06:49.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.866+0000 7f7381738800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:49.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.866+0000 7f7381738800 20 rados->read ofs=0 len=0 2024-05-08T18:06:49.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.866+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2ab0 con 0x5565198aed70 2024-05-08T18:06:49.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.866+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f73780083e0 con 0x5565198aed70 2024-05-08T18:06:49.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.866+0000 7f7381738800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:49.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.866+0000 7f7381738800 20 rados->read ofs=0 len=0 2024-05-08T18:06:49.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.866+0000 7f7381738800 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5565198b3050 0x5565198b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.866+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5565198b5bb0 con 0x5565198b3050 2024-05-08T18:06:49.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.866+0000 7f737d433640 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5565198b3050 0x5565198b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.867+0000 7f737d433640 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5565198b3050 0x5565198b5440 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.867+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.867+0000 7f7381738800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:49.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.867+0000 7f7381738800 20 searching for the correct realm 2024-05-08T18:06:49.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.867+0000 7f7381738800 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5565198b7b60 0x5565198ba020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.867+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198ba6f0 con 0x5565198b7b60 2024-05-08T18:06:49.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.867+0000 7f737cc32640 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5565198b7b60 0x5565198ba020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.879 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.867+0000 7f737cc32640 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5565198b7b60 0x5565198ba020 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.868+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f737405c040 con 0x5565198b7b60 2024-05-08T18:06:49.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.868+0000 7f737cc32640 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7374085040 0x7f7374054950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.868+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198ba6f0 con 0x7f7374085040 2024-05-08T18:06:49.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.868+0000 7f737dc34640 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7374085040 0x7f7374054950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.868+0000 7f737dc34640 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7374085040 0x7f7374054950 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.869+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73780083e0 con 0x7f7374085040 2024-05-08T18:06:49.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.869+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198b3050 2024-05-08T18:06:49.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.869+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.869+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198aed70 2024-05-08T18:06:49.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.869+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f73780083e0 con 0x5565198aed70 2024-05-08T18:06:49.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.869+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198aed70 2024-05-08T18:06:49.880 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.869+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.869+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198aed70 2024-05-08T18:06:49.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.870+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.870+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198b3050 2024-05-08T18:06:49.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.870+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.870+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x7f7374085040 2024-05-08T18:06:49.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.870+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7378026040 con 0x7f7374085040 2024-05-08T18:06:49.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.870+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198b7b60 2024-05-08T18:06:49.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.871+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f737405c040 con 0x5565198b7b60 2024-05-08T18:06:49.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.871+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198aed70 2024-05-08T18:06:49.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.871+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.871+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198b3050 2024-05-08T18:06:49.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.871+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.871+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198aed70 2024-05-08T18:06:49.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.872+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.872+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x7f7374085040 2024-05-08T18:06:49.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.872+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7378026040 con 0x7f7374085040 2024-05-08T18:06:49.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.872+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198b7b60 2024-05-08T18:06:49.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.872+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f737405c040 con 0x5565198b7b60 2024-05-08T18:06:49.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.872+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198b7b60 2024-05-08T18:06:49.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.872+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f737405c040 con 0x5565198b7b60 2024-05-08T18:06:49.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.872+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198b3050 2024-05-08T18:06:49.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.873+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.873+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198b7b60 2024-05-08T18:06:49.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.873+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f737405c040 con 0x5565198b7b60 2024-05-08T18:06:49.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.873+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198b7b60 2024-05-08T18:06:49.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.873+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f737405c040 con 0x5565198b7b60 2024-05-08T18:06:49.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.873+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198aed70 2024-05-08T18:06:49.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.873+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.874+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198aed70 2024-05-08T18:06:49.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.874+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.874+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198b3050 2024-05-08T18:06:49.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.874+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.874+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198b7b60 2024-05-08T18:06:49.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.874+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f737405c040 con 0x5565198b7b60 2024-05-08T18:06:49.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.874+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x7f7374085040 2024-05-08T18:06:49.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.875+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7378026040 con 0x7f7374085040 2024-05-08T18:06:49.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.875+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198aed70 2024-05-08T18:06:49.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.875+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.875+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198b3050 2024-05-08T18:06:49.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.875+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.876+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198b7b60 2024-05-08T18:06:49.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.876+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f737405c040 con 0x5565198b7b60 2024-05-08T18:06:49.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.876+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198b3050 2024-05-08T18:06:49.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.876+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.876+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x7f7374085040 2024-05-08T18:06:49.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.876+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7378026040 con 0x7f7374085040 2024-05-08T18:06:49.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.876+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198b7b60 2024-05-08T18:06:49.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.877+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f737405c040 con 0x5565198b7b60 2024-05-08T18:06:49.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.877+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198b3050 2024-05-08T18:06:49.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.877+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.877+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x5565198aed70 2024-05-08T18:06:49.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5565198b2210 con 0x7f7374085040 2024-05-08T18:06:49.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7378026040 con 0x7f7374085040 2024-05-08T18:06:49.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f7381738800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:49.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f7381738800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:49.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f7381738800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:49.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f7381738800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:49.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f7381738800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:49.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f7381738800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:49.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f7381738800 20 rados->read ofs=0 len=0 2024-05-08T18:06:49.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5565198b6470 con 0x5565198aed70 2024-05-08T18:06:49.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f7381738800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:49.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f7381738800 20 rados->read ofs=0 len=0 2024-05-08T18:06:49.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5565198b6470 con 0x5565198aed70 2024-05-08T18:06:49.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.878+0000 7f7381738800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:49.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.879+0000 7f7381738800 20 rados->read ofs=0 len=0 2024-05-08T18:06:49.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.879+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5565198b6470 con 0x5565198b7b60 2024-05-08T18:06:49.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.879+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f737405c040 con 0x5565198b7b60 2024-05-08T18:06:49.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.879+0000 7f7381738800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:49.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.879+0000 7f7381738800 20 zone default found 2024-05-08T18:06:49.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.879+0000 7f7381738800 4 Realm: () 2024-05-08T18:06:49.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.879+0000 7f7381738800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:49.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.879+0000 7f7381738800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:49.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.879+0000 7f7381738800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:49.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.879+0000 7f7381738800 20 zonegroup default 2024-05-08T18:06:49.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.879+0000 7f7381738800 20 rados->read ofs=0 len=0 2024-05-08T18:06:49.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.879+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5565198b6db0 con 0x5565198b3050 2024-05-08T18:06:49.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.879+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.879+0000 7f7381738800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:49.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.879+0000 7f7381738800 20 rados->read ofs=0 len=0 2024-05-08T18:06:49.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.879+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5565198bc5b0 con 0x5565198b7b60 2024-05-08T18:06:49.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.880+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f737405c040 con 0x5565198b7b60 2024-05-08T18:06:49.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.880+0000 7f7381738800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:49.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.880+0000 7f7381738800 20 rados->read ofs=0 len=0 2024-05-08T18:06:49.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.880+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5565198bca40 con 0x5565198b7b60 2024-05-08T18:06:49.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.880+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f737405c040 con 0x5565198b7b60 2024-05-08T18:06:49.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.880+0000 7f7381738800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:49.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.880+0000 7f7381738800 20 started sync module instance, tier type = 2024-05-08T18:06:49.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.880+0000 7f7381738800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:49.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.880+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198bb9f0 con 0x7f7374085040 2024-05-08T18:06:49.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.881+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7378026040 con 0x7f7374085040 2024-05-08T18:06:49.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.881+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93892708646640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198bb9f0 con 0x7f7374085040 2024-05-08T18:06:49.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.881+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198bd4c0 con 0x5565198aed70 2024-05-08T18:06:49.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.882+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93892708646640] v29'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7378026040 con 0x7f7374085040 2024-05-08T18:06:49.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.883+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.883+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93892708651504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198bb9f0 con 0x5565198aed70 2024-05-08T18:06:49.891 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.883+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198bdf60 con 0x5565198b7b60 2024-05-08T18:06:49.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.884+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93892708651504] v29'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.884+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f737405c040 con 0x5565198b7b60 2024-05-08T18:06:49.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.884+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93892708644160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198bb960 con 0x5565198b7b60 2024-05-08T18:06:49.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.884+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198bee20 con 0x7f7374085040 2024-05-08T18:06:49.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.885+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7378026040 con 0x7f7374085040 2024-05-08T18:06:49.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.885+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93892708644160] v29'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f737405c040 con 0x5565198b7b60 2024-05-08T18:06:49.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.885+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93892708658000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198bf6c0 con 0x7f7374085040 2024-05-08T18:06:49.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.885+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198be390 con 0x5565198aed70 2024-05-08T18:06:49.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.886+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93892708658000] v29'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7378026040 con 0x7f7374085040 2024-05-08T18:06:49.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.887+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.887+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93892708659600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198bfc00 con 0x5565198aed70 2024-05-08T18:06:49.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.887+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198c0600 con 0x5565198b3050 2024-05-08T18:06:49.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.888+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93892708659600] v29'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.892 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.888+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.888+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93892708660656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198c09f0 con 0x5565198b3050 2024-05-08T18:06:49.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.888+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198c13f0 con 0x5565198b3050 2024-05-08T18:06:49.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.889+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93892708660656] v29'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.889+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.889+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93892708663568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198c1c40 con 0x5565198b3050 2024-05-08T18:06:49.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.889+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198c2600 con 0x5565198b3050 2024-05-08T18:06:49.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.890+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93892708663568] v29'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.890+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.890+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93892708669808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198c2600 con 0x5565198b3050 2024-05-08T18:06:49.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.890+0000 7f7381738800 20 add_watcher() i=0 2024-05-08T18:06:49.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.890+0000 7f7381738800 20 add_watcher() i=1 2024-05-08T18:06:49.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.890+0000 7f7381738800 20 add_watcher() i=2 2024-05-08T18:06:49.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.890+0000 7f7381738800 20 add_watcher() i=3 2024-05-08T18:06:49.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.890+0000 7f7381738800 20 add_watcher() i=4 2024-05-08T18:06:49.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.890+0000 7f7381738800 20 add_watcher() i=5 2024-05-08T18:06:49.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.890+0000 7f7381738800 20 add_watcher() i=6 2024-05-08T18:06:49.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.891+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93892708669808] v29'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.891+0000 7f7381738800 20 add_watcher() i=7 2024-05-08T18:06:49.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.891+0000 7f7381738800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:49.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.892+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5565198c5930 con 0x5565198aed70 2024-05-08T18:06:49.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.892+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.892+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93892708672192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198c2b90 con 0x5565198aed70 2024-05-08T18:06:49.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.893+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93892708672192] v29'228 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.893+0000 7f7233fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:49.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.893+0000 7f7381738800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:49.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.893+0000 7f7381738800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:49.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.893+0000 7f7381738800 5 note: GC not initialized 2024-05-08T18:06:49.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.893+0000 7f7232fe5640 20 reqs_thread_entry: start 2024-05-08T18:06:49.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.894+0000 7f7381738800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:49.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.894+0000 7f7381738800 20 Filter name: none 2024-05-08T18:06:49.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.894+0000 7f7230fe1640 20 reqs_thread_entry: start 2024-05-08T18:06:49.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.894+0000 7f7381738800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:06:49.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.894+0000 7f7381738800 20 rados->read ofs=0 len=0 2024-05-08T18:06:49.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.894+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5565198f8ec0 con 0x5565198aed70 2024-05-08T18:06:49.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.894+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.894+0000 7f7381738800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:06:49.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.894+0000 7f7381738800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:06:49.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.894+0000 7f7381738800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:06:49.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.894+0000 7f7381738800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:49.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.894+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198fb950 con 0x5565198b3050 2024-05-08T18:06:49.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.896+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(63 usage.10 [call] v29'32 uv32 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.896+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:64 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198fb950 con 0x5565198b3050 2024-05-08T18:06:49.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.897+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(64 usage.10 [call] v29'33 uv32 ondisk = -61 ((61) No data available)) ==== 152+0+0 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.898+0000 7f7233fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:49.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.898+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93892708672192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198c4f70 con 0x5565198aed70 2024-05-08T18:06:49.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.899+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 93892708672192] v29'229 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.899+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93892708646640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198c22c0 con 0x7f7374085040 2024-05-08T18:06:49.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.900+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 93892708646640] v29'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7378026040 con 0x7f7374085040 2024-05-08T18:06:49.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.900+0000 7f7381738800 20 remove_watcher() i=0 2024-05-08T18:06:49.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.900+0000 7f7381738800 2 removed watcher, disabling cache 2024-05-08T18:06:49.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.900+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93892708651504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198f8a60 con 0x5565198aed70 2024-05-08T18:06:49.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.901+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 93892708651504] v29'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.902+0000 7f7381738800 20 remove_watcher() i=1 2024-05-08T18:06:49.900 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.902+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93892708644160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198f8a60 con 0x5565198b7b60 2024-05-08T18:06:49.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.903+0000 7f737cc32640 1 -- 172.21.15.50:0/1229454527 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 93892708644160] v29'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f737405c040 con 0x5565198b7b60 2024-05-08T18:06:49.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.903+0000 7f7381738800 20 remove_watcher() i=2 2024-05-08T18:06:49.901 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.903+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93892708658000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198f8a60 con 0x7f7374085040 2024-05-08T18:06:49.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.904+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 93892708658000] v29'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7378026040 con 0x7f7374085040 2024-05-08T18:06:49.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.904+0000 7f7381738800 20 remove_watcher() i=3 2024-05-08T18:06:49.902 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.904+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93892708659600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198f8a60 con 0x5565198aed70 2024-05-08T18:06:49.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.905+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 93892708659600] v29'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7378026040 con 0x5565198aed70 2024-05-08T18:06:49.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.905+0000 7f7381738800 20 remove_watcher() i=4 2024-05-08T18:06:49.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.905+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93892708660656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198f8a60 con 0x5565198b3050 2024-05-08T18:06:49.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.906+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 93892708660656] v29'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.906+0000 7f7381738800 20 remove_watcher() i=5 2024-05-08T18:06:49.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.906+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93892708663568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198f8a60 con 0x5565198b3050 2024-05-08T18:06:49.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.908+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 93892708663568] v29'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.908+0000 7f7381738800 20 remove_watcher() i=6 2024-05-08T18:06:49.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.908+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93892708669808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5565198f8a60 con 0x5565198b3050 2024-05-08T18:06:49.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f737d433640 1 -- 172.21.15.50:0/1229454527 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 93892708669808] v29'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f737006d030 con 0x5565198b3050 2024-05-08T18:06:49.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f7381738800 20 remove_watcher() i=7 2024-05-08T18:06:49.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5565198b3050 msgr2=0x5565198b5440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f7381738800 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5565198b3050 0x5565198b5440 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5565198aed70 msgr2=0x5565198b1230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f7381738800 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5565198aed70 0x5565198b1230 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7374085040 msgr2=0x7f7374054950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f7381738800 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7374085040 0x7f7374054950 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5565198b7b60 msgr2=0x5565198ba020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f7381738800 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5565198b7b60 0x5565198ba020 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f724803bb50 msgr2=0x7f724803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f7381738800 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f724803bb50 0x7f724803e010 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x55651987ce10 tx=0x7f7374062040 comp rx=0 tx=0).stop 2024-05-08T18:06:49.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f737dc34640 1 -- 172.21.15.50:0/1229454527 reap_dead start 2024-05-08T18:06:49.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651989c230 msgr2=0x5565198a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f7381738800 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651989c230 0x5565198a3570 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f7370037420 tx=0x7f7370038490 comp rx=0 tx=0).stop 2024-05-08T18:06:49.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 shutdown_connections 2024-05-08T18:06:49.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f7381738800 1 --2- 172.21.15.50:0/1229454527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651989c230 0x5565198a3570 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.909+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 >> 172.21.15.50:0/1229454527 conn(0x556519898500 msgr2=0x556519895b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:49.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 shutdown_connections 2024-05-08T18:06:49.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 -- 172.21.15.50:0/1229454527 wait complete. 2024-05-08T18:06:49.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 -- 172.21.15.50:0/503287350 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f725803bb00 msgr2=0x7f725803dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 --2- 172.21.15.50:0/503287350 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f725803bb00 0x7f725803dfc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5565198745f0 tx=0x7f7374005ac0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 -- 172.21.15.50:0/503287350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651988f0d0 msgr2=0x55651988f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 --2- 172.21.15.50:0/503287350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651988f0d0 0x55651988f4f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7370023ff0 tx=0x7f737003ce30 comp rx=0 tx=0).stop 2024-05-08T18:06:49.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 -- 172.21.15.50:0/503287350 shutdown_connections 2024-05-08T18:06:49.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 --2- 172.21.15.50:0/503287350 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f725803bb00 0x7f725803dfc0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 --2- 172.21.15.50:0/503287350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55651988f0d0 0x55651988f4f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 -- 172.21.15.50:0/503287350 >> 172.21.15.50:0/503287350 conn(0x55651987ed60 msgr2=0x55651987f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:49.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 -- 172.21.15.50:0/503287350 shutdown_connections 2024-05-08T18:06:49.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 -- 172.21.15.50:0/503287350 wait complete. 2024-05-08T18:06:49.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 -- 172.21.15.50:0/2789834326 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556519876040 msgr2=0x556519878430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556519876040 0x556519878430 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 -- 172.21.15.50:0/2789834326 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556519778a60 msgr2=0x556519873780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556519778a60 0x556519873780 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 -- 172.21.15.50:0/2789834326 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556519879f80 msgr2=0x55651987c370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556519879f80 0x55651987c370 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 -- 172.21.15.50:0/2789834326 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f726403bb50 msgr2=0x7f726403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f726403bb50 0x7f726403e010 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7374056000 tx=0x7f737400a000 comp rx=0 tx=0).stop 2024-05-08T18:06:49.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 -- 172.21.15.50:0/2789834326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519775b60 msgr2=0x55651986c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f7381738800 1 --2- 172.21.15.50:0/2789834326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556519775b60 0x55651986c590 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x556519865680 tx=0x7f7370001250 comp rx=0 tx=0).stop 2024-05-08T18:06:49.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.910+0000 7f737dc34640 1 -- 172.21.15.50:0/2789834326 reap_dead start 2024-05-08T18:06:49.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.911+0000 7f7381738800 1 -- 172.21.15.50:0/2789834326 shutdown_connections 2024-05-08T18:06:49.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.911+0000 7f7381738800 1 -- 172.21.15.50:0/2789834326 >> 172.21.15.50:0/2789834326 conn(0x55651976f2d0 msgr2=0x55651985ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:49.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.911+0000 7f7381738800 1 -- 172.21.15.50:0/2789834326 shutdown_connections 2024-05-08T18:06:49.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.911+0000 7f7381738800 1 -- 172.21.15.50:0/2789834326 wait complete. 2024-05-08T18:06:49.918 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2024-05-08T18:06:49.918 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2024-05-08T18:06:49.919 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2024-05-08T18:06:49.964 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:49.965 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:49.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.967+0000 7feec5e11800 1 Processor -- start 2024-05-08T18:06:49.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.968+0000 7feec5e11800 1 -- start start 2024-05-08T18:06:49.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.968+0000 7feec5e11800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55fbab14dc90 0x55fbab150080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.968+0000 7feec5e11800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55fbab1505c0 con 0x55fbab14a180 2024-05-08T18:06:49.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.968+0000 7feec5e11800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55fbaafe9e20 con 0x55fbab14dc90 2024-05-08T18:06:49.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.968+0000 7feec0d3f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55fbab14dc90 0x55fbab150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.968+0000 7feec1540640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55fbab14a180 0x55fbab14c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50708/0 (socket says 172.21.15.50:50708) 2024-05-08T18:06:49.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.968+0000 7feec1540640 1 -- 172.21.15.50:0/1891252237 learned_addr learned my addr 172.21.15.50:0/1891252237 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:49.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.968+0000 7feec0d3f640 1 -- 172.21.15.50:0/1891252237 >> v1:172.21.15.46:6789/0 conn(0x55fbab14a180 legacy=0x55fbab14c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.968+0000 7feec0d3f640 1 -- 172.21.15.50:0/1891252237 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbab1517a0 con 0x55fbab14dc90 2024-05-08T18:06:49.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.970+0000 7feec0d3f640 1 --2- 172.21.15.50:0/1891252237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14dc90 0x55fbab150080 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7feeb800a470 tx=0x7feeb801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0cd18088e991518 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.971+0000 7feeb3fff640 1 -- 172.21.15.50:0/1891252237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeb8057070 con 0x55fbab14dc90 2024-05-08T18:06:49.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.971+0000 7feeb3fff640 1 -- 172.21.15.50:0/1891252237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb8052570 con 0x55fbab14dc90 2024-05-08T18:06:49.969 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.971+0000 7feeb3fff640 1 -- 172.21.15.50:0/1891252237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeb8052870 con 0x55fbab14dc90 2024-05-08T18:06:49.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.971+0000 7feec5e11800 1 -- 172.21.15.50:0/1891252237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14dc90 msgr2=0x55fbab150080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.971+0000 7feec5e11800 1 --2- 172.21.15.50:0/1891252237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14dc90 0x55fbab150080 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7feeb800a470 tx=0x7feeb801ce90 comp rx=0 tx=0).stop 2024-05-08T18:06:49.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.971+0000 7feec5e11800 1 -- 172.21.15.50:0/1891252237 shutdown_connections 2024-05-08T18:06:49.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.971+0000 7feec5e11800 1 --2- 172.21.15.50:0/1891252237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14dc90 0x55fbab150080 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.971+0000 7feec5e11800 1 -- 172.21.15.50:0/1891252237 >> 172.21.15.50:0/1891252237 conn(0x55fbab143ad0 msgr2=0x55fbab145f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:49.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.971+0000 7feec5e11800 1 -- 172.21.15.50:0/1891252237 shutdown_connections 2024-05-08T18:06:49.970 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.971+0000 7feec5e11800 1 -- 172.21.15.50:0/1891252237 wait complete. 2024-05-08T18:06:49.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.986+0000 7feec5e11800 1 Processor -- start 2024-05-08T18:06:49.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.987+0000 7feec5e11800 1 -- start start 2024-05-08T18:06:49.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.987+0000 7feec5e11800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14a180 0x55fbab23ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.987+0000 7feec5e11800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fbab094a10 con 0x55fbab14a180 2024-05-08T18:06:49.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.987+0000 7feec1540640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14a180 0x55fbab23ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.987+0000 7feec1540640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14a180 0x55fbab23ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45696/0 (socket says 172.21.15.50:45696) 2024-05-08T18:06:49.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.987+0000 7feec1540640 1 -- 172.21.15.50:0/766287475 learned_addr learned my addr 172.21.15.50:0/766287475 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:49.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.987+0000 7feec1540640 1 -- 172.21.15.50:0/766287475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbab23b130 con 0x55fbab14a180 2024-05-08T18:06:49.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.988+0000 7feec1540640 1 --2- 172.21.15.50:0/766287475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14a180 0x55fbab23ab60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7feeb4001e10 tx=0x7feeb400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e75dae77d181b17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.988+0000 7fedaf606640 1 -- 172.21.15.50:0/766287475 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeb4012020 con 0x55fbab14a180 2024-05-08T18:06:49.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.988+0000 7fedaf606640 1 -- 172.21.15.50:0/766287475 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb400e5a0 con 0x55fbab14a180 2024-05-08T18:06:49.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.988+0000 7fedaf606640 1 -- 172.21.15.50:0/766287475 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeb400e8a0 con 0x55fbab14a180 2024-05-08T18:06:49.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.988+0000 7feec5e11800 1 -- 172.21.15.50:0/766287475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14a180 msgr2=0x55fbab23ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:49.988 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.988+0000 7feec5e11800 1 --2- 172.21.15.50:0/766287475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14a180 0x55fbab23ab60 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7feeb4001e10 tx=0x7feeb400b780 comp rx=0 tx=0).stop 2024-05-08T18:06:49.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.989+0000 7feec5e11800 1 -- 172.21.15.50:0/766287475 shutdown_connections 2024-05-08T18:06:49.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.989+0000 7feec5e11800 1 --2- 172.21.15.50:0/766287475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14a180 0x55fbab23ab60 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:49.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.989+0000 7feec5e11800 1 -- 172.21.15.50:0/766287475 >> 172.21.15.50:0/766287475 conn(0x55fbab143ad0 msgr2=0x55fbab236480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:49.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.989+0000 7feec5e11800 1 -- 172.21.15.50:0/766287475 shutdown_connections 2024-05-08T18:06:49.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.989+0000 7feec5e11800 1 -- 172.21.15.50:0/766287475 wait complete. 2024-05-08T18:06:49.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.989+0000 7feec5e11800 1 Processor -- start 2024-05-08T18:06:49.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.989+0000 7feec5e11800 1 -- start start 2024-05-08T18:06:49.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.989+0000 7feec5e11800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14dc90 0x55fbab149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.989+0000 7feec5e11800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fbab150a90 con 0x55fbab14dc90 2024-05-08T18:06:49.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.989+0000 7feec1540640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14dc90 0x55fbab149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.989+0000 7feec1540640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14dc90 0x55fbab149a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45698/0 (socket says 172.21.15.50:45698) 2024-05-08T18:06:49.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.989+0000 7feec1540640 1 -- 172.21.15.50:0/3523903791 learned_addr learned my addr 172.21.15.50:0/3523903791 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:49.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.990+0000 7feec1540640 1 -- 172.21.15.50:0/3523903791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbab149f60 con 0x55fbab14dc90 2024-05-08T18:06:49.989 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.990+0000 7feec1540640 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14dc90 0x55fbab149a20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x55fbab23b8f0 tx=0x7feeb4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.990+0000 7fedade03640 1 -- 172.21.15.50:0/3523903791 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeb4012030 con 0x55fbab14dc90 2024-05-08T18:06:49.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.990+0000 7fedade03640 1 -- 172.21.15.50:0/3523903791 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb4001470 con 0x55fbab14dc90 2024-05-08T18:06:49.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.990+0000 7feec5e11800 1 -- 172.21.15.50:0/3523903791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fbab1481b0 con 0x55fbab14dc90 2024-05-08T18:06:49.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.990+0000 7fedade03640 1 -- 172.21.15.50:0/3523903791 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeb401a700 con 0x55fbab14dc90 2024-05-08T18:06:49.990 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.990+0000 7feec5e11800 1 -- 172.21.15.50:0/3523903791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbab1486d0 con 0x55fbab14dc90 2024-05-08T18:06:49.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.991+0000 7fedade03640 1 -- 172.21.15.50:0/3523903791 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7feeb4015020 con 0x55fbab14dc90 2024-05-08T18:06:49.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.991+0000 7fedade03640 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7feda803bba0 0x7feda803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.991+0000 7fedade03640 1 -- 172.21.15.50:0/3523903791 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7feeb4020030 con 0x55fbab14dc90 2024-05-08T18:06:49.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.991+0000 7feec0d3f640 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7feda803bba0 0x7feda803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.992+0000 7feec5e11800 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fbab249db0 0x55fbab24c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.994 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.992+0000 7feec5e11800 1 -- 172.21.15.50:0/3523903791 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab24c6e0 con 0x55fbab249db0 2024-05-08T18:06:49.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.992+0000 7feec1d41640 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fbab249db0 0x55fbab24c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.992+0000 7feec0d3f640 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7feda803bba0 0x7feda803e060 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feeb8054000 tx=0x7feeb804e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.992+0000 7feec1d41640 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fbab249db0 0x55fbab24c1a0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.992+0000 7feec1d41640 1 -- 172.21.15.50:0/3523903791 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7feebc00a4a0 con 0x55fbab249db0 2024-05-08T18:06:49.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.992+0000 7feec5e11800 1 -- 172.21.15.50:0/3523903791 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab148b30 con 0x55fbab249db0 2024-05-08T18:06:49.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.993+0000 7feec1d41640 1 -- 172.21.15.50:0/3523903791 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7feebc00a4a0 con 0x55fbab249db0 2024-05-08T18:06:49.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.993+0000 7feec5e11800 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fbab24e790 0x55fbab250c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.993+0000 7feec5e11800 1 -- 172.21.15.50:0/3523903791 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab2513e0 con 0x55fbab24e790 2024-05-08T18:06:49.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.993+0000 7feec1540640 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fbab24e790 0x55fbab250c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.993+0000 7feec1540640 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fbab24e790 0x55fbab250c50 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.994+0000 7feec1540640 1 -- 172.21.15.50:0/3523903791 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7feeb4017cd0 con 0x55fbab24e790 2024-05-08T18:06:49.995 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.994+0000 7feec5e11800 1 -- 172.21.15.50:0/3523903791 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab24cf60 con 0x55fbab249db0 2024-05-08T18:06:49.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.994+0000 7feec1d41640 1 -- 172.21.15.50:0/3523903791 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7feebc00a4a0 con 0x55fbab249db0 2024-05-08T18:06:49.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.994+0000 7feec5e11800 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fbab252840 0x55fbab254c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.994+0000 7feec5e11800 1 -- 172.21.15.50:0/3523903791 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab255330 con 0x55fbab252840 2024-05-08T18:06:49.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.994+0000 7feec0d3f640 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fbab252840 0x55fbab254c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.994+0000 7feec0d3f640 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fbab252840 0x55fbab254c30 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.995+0000 7feec0d3f640 1 -- 172.21.15.50:0/3523903791 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7feeb8052a40 con 0x55fbab252840 2024-05-08T18:06:49.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.995+0000 7feec5e11800 1 Processor -- start 2024-05-08T18:06:49.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.995+0000 7feec5e11800 1 -- start start 2024-05-08T18:06:49.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.995+0000 7feec5e11800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab263660 0x55fbab263a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:49.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.995+0000 7feec5e11800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fbab03b330 con 0x55fbab263660 2024-05-08T18:06:49.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.995+0000 7feec1540640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab263660 0x55fbab263a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:49.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.995+0000 7feec1540640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab263660 0x55fbab263a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45708/0 (socket says 172.21.15.50:45708) 2024-05-08T18:06:49.996 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.995+0000 7feec1540640 1 -- 172.21.15.50:0/104741794 learned_addr learned my addr 172.21.15.50:0/104741794 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:49.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.996+0000 7feec1540640 1 -- 172.21.15.50:0/104741794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbab263fc0 con 0x55fbab263660 2024-05-08T18:06:49.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.996+0000 7feec1540640 1 --2- 172.21.15.50:0/104741794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab263660 0x55fbab263a80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7feeb400c720 tx=0x7feeb401ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:49.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.996+0000 7feda6ffd640 1 -- 172.21.15.50:0/104741794 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeb4050e60 con 0x55fbab263660 2024-05-08T18:06:49.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.996+0000 7feda6ffd640 1 -- 172.21.15.50:0/104741794 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb4049670 con 0x55fbab263660 2024-05-08T18:06:49.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.996+0000 7feec5e11800 1 -- 172.21.15.50:0/104741794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fbab264a30 con 0x55fbab263660 2024-05-08T18:06:49.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.996+0000 7feda6ffd640 1 -- 172.21.15.50:0/104741794 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeb4049970 con 0x55fbab263660 2024-05-08T18:06:49.997 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.996+0000 7feec5e11800 1 -- 172.21.15.50:0/104741794 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbab264a30 con 0x55fbab263660 2024-05-08T18:06:50.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.997+0000 7feda6ffd640 1 -- 172.21.15.50:0/104741794 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7feeb40163b0 con 0x55fbab263660 2024-05-08T18:06:50.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.997+0000 7feda6ffd640 1 --2- 172.21.15.50:0/104741794 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fed9803bb00 0x7fed9803dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.997+0000 7feda6ffd640 1 -- 172.21.15.50:0/104741794 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7feeb40425b0 con 0x55fbab263660 2024-05-08T18:06:50.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.997+0000 7feec0d3f640 1 --2- 172.21.15.50:0/104741794 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fed9803bb00 0x7fed9803dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.997+0000 7feec5e11800 1 Processor -- start 2024-05-08T18:06:50.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.998+0000 7feec5e11800 1 -- start start 2024-05-08T18:06:50.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.998+0000 7feec0d3f640 1 --2- 172.21.15.50:0/104741794 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fed9803bb00 0x7fed9803dfc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55fbab148ef0 tx=0x7feeb8005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.998+0000 7feec5e11800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab270640 0x55fbab270a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.998+0000 7feec5e11800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fbab23b4b0 con 0x55fbab270640 2024-05-08T18:06:50.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.998+0000 7feec1540640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab270640 0x55fbab270a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.998+0000 7feec1540640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab270640 0x55fbab270a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45724/0 (socket says 172.21.15.50:45724) 2024-05-08T18:06:50.009 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.998+0000 7feec1540640 1 -- 172.21.15.50:0/1055370444 learned_addr learned my addr 172.21.15.50:0/1055370444 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.998+0000 7feec1540640 1 -- 172.21.15.50:0/1055370444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbab271030 con 0x55fbab270640 2024-05-08T18:06:50.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.998+0000 7feec1540640 1 --2- 172.21.15.50:0/1055370444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab270640 0x55fbab270a60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7feeb4040880 tx=0x7feeb4053f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5d4b056e5e4129 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.999+0000 7feda4ff9640 1 -- 172.21.15.50:0/1055370444 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeb404ee80 con 0x55fbab270640 2024-05-08T18:06:50.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.999+0000 7feda4ff9640 1 -- 172.21.15.50:0/1055370444 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb404c400 con 0x55fbab270640 2024-05-08T18:06:50.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.999+0000 7feda4ff9640 1 -- 172.21.15.50:0/1055370444 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeb400d5f0 con 0x55fbab270640 2024-05-08T18:06:50.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.999+0000 7feec5e11800 1 -- 172.21.15.50:0/1055370444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab270640 msgr2=0x55fbab270a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.999+0000 7feec5e11800 1 --2- 172.21.15.50:0/1055370444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab270640 0x55fbab270a60 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7feeb4040880 tx=0x7feeb4053f10 comp rx=0 tx=0).stop 2024-05-08T18:06:50.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.999+0000 7feec5e11800 1 -- 172.21.15.50:0/1055370444 shutdown_connections 2024-05-08T18:06:50.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.999+0000 7feec5e11800 1 --2- 172.21.15.50:0/1055370444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab270640 0x55fbab270a60 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.999+0000 7feec5e11800 1 -- 172.21.15.50:0/1055370444 >> 172.21.15.50:0/1055370444 conn(0x55fbab26c910 msgr2=0x55fbab26a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.999+0000 7feec5e11800 1 -- 172.21.15.50:0/1055370444 shutdown_connections 2024-05-08T18:06:50.010 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.999+0000 7feec5e11800 1 -- 172.21.15.50:0/1055370444 wait complete. 2024-05-08T18:06:50.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.999+0000 7feec5e11800 1 Processor -- start 2024-05-08T18:06:50.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:49.999+0000 7feec5e11800 1 -- start start 2024-05-08T18:06:50.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.000+0000 7feec5e11800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab270640 0x55fbab2779a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.000+0000 7feec5e11800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fbab271700 con 0x55fbab270640 2024-05-08T18:06:50.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.000+0000 7feec1540640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab270640 0x55fbab2779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.000+0000 7feec1540640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab270640 0x55fbab2779a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45726/0 (socket says 172.21.15.50:45726) 2024-05-08T18:06:50.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.000+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 learned_addr learned my addr 172.21.15.50:0/3330459562 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.000+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbab277ee0 con 0x55fbab270640 2024-05-08T18:06:50.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.000+0000 7feec1540640 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab270640 0x55fbab2779a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7feeb403d040 tx=0x7feeb4040810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.000+0000 7fed96ffd640 1 -- 172.21.15.50:0/3330459562 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeb4024b30 con 0x55fbab270640 2024-05-08T18:06:50.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.000+0000 7fed96ffd640 1 -- 172.21.15.50:0/3330459562 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb4024cd0 con 0x55fbab270640 2024-05-08T18:06:50.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.001+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55fbab277ee0 con 0x55fbab270640 2024-05-08T18:06:50.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.001+0000 7fed96ffd640 1 -- 172.21.15.50:0/3330459562 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feeb403f1b0 con 0x55fbab270640 2024-05-08T18:06:50.011 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.001+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbab2783e0 con 0x55fbab270640 2024-05-08T18:06:50.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.001+0000 7fed96ffd640 1 -- 172.21.15.50:0/3330459562 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7feeb406e030 con 0x55fbab270640 2024-05-08T18:06:50.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.002+0000 7fed96ffd640 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fed8c03bb50 0x7fed8c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.002+0000 7fed96ffd640 1 -- 172.21.15.50:0/3330459562 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7feeb406c070 con 0x55fbab270640 2024-05-08T18:06:50.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.002+0000 7feec0d3f640 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fed8c03bb50 0x7fed8c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.002+0000 7feec5e11800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.002+0000 7feec5e11800 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fbab283140 0x55fbab285600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.002+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab285d50 con 0x55fbab283140 2024-05-08T18:06:50.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.002+0000 7feec1d41640 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fbab283140 0x55fbab285600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.002+0000 7feec0d3f640 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fed8c03bb50 0x7fed8c03e010 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feeb8052fd0 tx=0x7feeb804f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.012 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.002+0000 7feec1d41640 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fbab283140 0x55fbab285600 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.014 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7feebc00a4a0 con 0x55fbab283140 2024-05-08T18:06:50.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec5e11800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec5e11800 20 realm 2024-05-08T18:06:50.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec5e11800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab286160 con 0x55fbab283140 2024-05-08T18:06:50.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7feebc00a4a0 con 0x55fbab283140 2024-05-08T18:06:50.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec5e11800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec5e11800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:50.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec5e11800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.015 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab286a30 con 0x55fbab283140 2024-05-08T18:06:50.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7feebc00a4a0 con 0x55fbab283140 2024-05-08T18:06:50.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec5e11800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec5e11800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab286dd0 con 0x55fbab283140 2024-05-08T18:06:50.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7feebc00a4a0 con 0x55fbab283140 2024-05-08T18:06:50.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec5e11800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:50.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec5e11800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.016 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec5e11800 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fbab2878c0 0x55fbab289cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.003+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab28a410 con 0x55fbab2878c0 2024-05-08T18:06:50.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.004+0000 7feec1540640 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fbab2878c0 0x55fbab289cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.004+0000 7feec1540640 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fbab2878c0 0x55fbab289cb0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.004+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.004+0000 7feec5e11800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:50.017 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.004+0000 7feec5e11800 20 searching for the correct realm 2024-05-08T18:06:50.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.004+0000 7feec5e11800 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fbab28c4c0 0x55fbab28e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.004+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55fbab28f050 con 0x55fbab28c4c0 2024-05-08T18:06:50.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.005+0000 7feec0d3f640 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fbab28c4c0 0x55fbab28e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.005+0000 7feec0d3f640 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fbab28c4c0 0x55fbab28e980 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.005+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feeb8081070 con 0x55fbab28c4c0 2024-05-08T18:06:50.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.005+0000 7feec0d3f640 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7feeb8076040 0x7feeb8083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.018 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.005+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55fbab28f050 con 0x7feeb8076040 2024-05-08T18:06:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.005+0000 7feec1d41640 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7feeb8076040 0x7feeb8083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.006+0000 7feec1d41640 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7feeb8076040 0x7feeb8083410 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.006+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feebc00a4a0 con 0x7feeb8076040 2024-05-08T18:06:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.006+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55fbab286dd0 con 0x55fbab2878c0 2024-05-08T18:06:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.006+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.006+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55fbab286dd0 con 0x55fbab283140 2024-05-08T18:06:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.007+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feebc00a4a0 con 0x55fbab283140 2024-05-08T18:06:50.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.007+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55fbab286dd0 con 0x55fbab283140 2024-05-08T18:06:50.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.007+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feebc00a4a0 con 0x55fbab283140 2024-05-08T18:06:50.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.007+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55fbab286dd0 con 0x55fbab283140 2024-05-08T18:06:50.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.007+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feebc00a4a0 con 0x55fbab283140 2024-05-08T18:06:50.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.007+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55fbab286dd0 con 0x55fbab2878c0 2024-05-08T18:06:50.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.007+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.007+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55fbab286dd0 con 0x7feeb8076040 2024-05-08T18:06:50.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.008+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feebc00a4a0 con 0x7feeb8076040 2024-05-08T18:06:50.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.008+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55fbab286dd0 con 0x55fbab28c4c0 2024-05-08T18:06:50.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.008+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feeb8081070 con 0x55fbab28c4c0 2024-05-08T18:06:50.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.008+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55fbab286dd0 con 0x55fbab283140 2024-05-08T18:06:50.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.008+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feebc00a4a0 con 0x55fbab283140 2024-05-08T18:06:50.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.008+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55fbab286dd0 con 0x55fbab2878c0 2024-05-08T18:06:50.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.009+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.009+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55fbab286dd0 con 0x55fbab283140 2024-05-08T18:06:50.021 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.009+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feebc00a4a0 con 0x55fbab283140 2024-05-08T18:06:50.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.009+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55fbab286dd0 con 0x7feeb8076040 2024-05-08T18:06:50.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.009+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7feebc032040 con 0x7feeb8076040 2024-05-08T18:06:50.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.009+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55fbab286dd0 con 0x55fbab28c4c0 2024-05-08T18:06:50.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.009+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feeb8081070 con 0x55fbab28c4c0 2024-05-08T18:06:50.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.009+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x55fbab28c4c0 2024-05-08T18:06:50.022 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.010+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feeb8081070 con 0x55fbab28c4c0 2024-05-08T18:06:50.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.010+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x55fbab2878c0 2024-05-08T18:06:50.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.010+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.010+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x55fbab28c4c0 2024-05-08T18:06:50.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.010+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feeb8081070 con 0x55fbab28c4c0 2024-05-08T18:06:50.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.010+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x55fbab28c4c0 2024-05-08T18:06:50.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.010+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7feeb8081070 con 0x55fbab28c4c0 2024-05-08T18:06:50.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.010+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x55fbab283140 2024-05-08T18:06:50.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.011+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feebc032040 con 0x55fbab283140 2024-05-08T18:06:50.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.011+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x55fbab283140 2024-05-08T18:06:50.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.011+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7feebc032040 con 0x55fbab283140 2024-05-08T18:06:50.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.011+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x55fbab2878c0 2024-05-08T18:06:50.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.011+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.011+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x55fbab28c4c0 2024-05-08T18:06:50.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.012+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feeb8081070 con 0x55fbab28c4c0 2024-05-08T18:06:50.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.012+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x7feeb8076040 2024-05-08T18:06:50.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.012+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feebc032040 con 0x7feeb8076040 2024-05-08T18:06:50.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.012+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x55fbab283140 2024-05-08T18:06:50.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.012+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feebc032040 con 0x55fbab283140 2024-05-08T18:06:50.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.012+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x55fbab2878c0 2024-05-08T18:06:50.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.012+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.012+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x55fbab28c4c0 2024-05-08T18:06:50.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.013+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feeb8081070 con 0x55fbab28c4c0 2024-05-08T18:06:50.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.013+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x55fbab2878c0 2024-05-08T18:06:50.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.013+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.013+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x7feeb8076040 2024-05-08T18:06:50.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.013+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feebc032040 con 0x7feeb8076040 2024-05-08T18:06:50.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.013+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x55fbab28c4c0 2024-05-08T18:06:50.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.013+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feeb8081070 con 0x55fbab28c4c0 2024-05-08T18:06:50.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.013+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x55fbab2878c0 2024-05-08T18:06:50.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.014+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.014+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x55fbab283140 2024-05-08T18:06:50.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.014+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7feebc032040 con 0x55fbab283140 2024-05-08T18:06:50.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.014+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7feeb8090040 con 0x7feeb8076040 2024-05-08T18:06:50.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.014+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7feebc032040 con 0x7feeb8076040 2024-05-08T18:06:50.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.014+0000 7feec5e11800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:50.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.014+0000 7feec5e11800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:50.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.014+0000 7feec5e11800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:50.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.014+0000 7feec5e11800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:50.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.014+0000 7feec5e11800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:50.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.014+0000 7feec5e11800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:50.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.014+0000 7feec5e11800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.038 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.014+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab28adc0 con 0x55fbab283140 2024-05-08T18:06:50.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7feebc032040 con 0x55fbab283140 2024-05-08T18:06:50.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec5e11800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec5e11800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab28adc0 con 0x55fbab283140 2024-05-08T18:06:50.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7feebc032040 con 0x55fbab283140 2024-05-08T18:06:50.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec5e11800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:50.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec5e11800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab28ad90 con 0x55fbab28c4c0 2024-05-08T18:06:50.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7feeb8081070 con 0x55fbab28c4c0 2024-05-08T18:06:50.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec5e11800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:50.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec5e11800 20 zone default found 2024-05-08T18:06:50.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec5e11800 4 Realm: () 2024-05-08T18:06:50.039 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec5e11800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:50.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec5e11800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:50.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec5e11800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:50.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec5e11800 20 zonegroup default 2024-05-08T18:06:50.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec5e11800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab28b660 con 0x55fbab2878c0 2024-05-08T18:06:50.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.015+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.016+0000 7feec5e11800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.016+0000 7feec5e11800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.016+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab290eb0 con 0x55fbab28c4c0 2024-05-08T18:06:50.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.016+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7feeb8081070 con 0x55fbab28c4c0 2024-05-08T18:06:50.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.016+0000 7feec5e11800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.016+0000 7feec5e11800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.016+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab291250 con 0x55fbab28c4c0 2024-05-08T18:06:50.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.016+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7feeb8081070 con 0x55fbab28c4c0 2024-05-08T18:06:50.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.016+0000 7feec5e11800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.016+0000 7feec5e11800 20 started sync module instance, tier type = 2024-05-08T18:06:50.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.016+0000 7feec5e11800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:50.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.016+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab290350 con 0x7feeb8076040 2024-05-08T18:06:50.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.017+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feebc032040 con 0x7feeb8076040 2024-05-08T18:06:50.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.017+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94539396745376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab290350 con 0x7feeb8076040 2024-05-08T18:06:50.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.017+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab291c50 con 0x55fbab283140 2024-05-08T18:06:50.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.018+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94539396745376] v29'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feebc032040 con 0x7feeb8076040 2024-05-08T18:06:50.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.018+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feebc032040 con 0x55fbab283140 2024-05-08T18:06:50.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.018+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94539396749152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab290350 con 0x55fbab283140 2024-05-08T18:06:50.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.019+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab2926a0 con 0x55fbab28c4c0 2024-05-08T18:06:50.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.020+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feeb8081070 con 0x55fbab28c4c0 2024-05-08T18:06:50.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.020+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94539396751792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab292f70 con 0x55fbab28c4c0 2024-05-08T18:06:50.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.020+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab293a20 con 0x7feeb8076040 2024-05-08T18:06:50.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.020+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94539396749152] v29'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feebc032040 con 0x55fbab283140 2024-05-08T18:06:50.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.021+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feebc032040 con 0x7feeb8076040 2024-05-08T18:06:50.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.021+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94539396751792] v29'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feeb8081070 con 0x55fbab28c4c0 2024-05-08T18:06:50.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.021+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94539396742816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab293e10 con 0x7feeb8076040 2024-05-08T18:06:50.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.021+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab292ac0 con 0x55fbab283140 2024-05-08T18:06:50.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.022+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94539396742816] v29'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feebc032040 con 0x7feeb8076040 2024-05-08T18:06:50.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.022+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feebc032040 con 0x55fbab283140 2024-05-08T18:06:50.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.022+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94539396756736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab292ac0 con 0x55fbab283140 2024-05-08T18:06:50.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.022+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab294c80 con 0x55fbab2878c0 2024-05-08T18:06:50.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.023+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94539396756736] v29'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feebc032040 con 0x55fbab283140 2024-05-08T18:06:50.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.023+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.023+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94539396760352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab295070 con 0x55fbab2878c0 2024-05-08T18:06:50.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.023+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab295a70 con 0x55fbab2878c0 2024-05-08T18:06:50.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.024+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94539396760352] v29'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.024+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.024+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94539396761408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab296310 con 0x55fbab2878c0 2024-05-08T18:06:50.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.024+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab296be0 con 0x55fbab2878c0 2024-05-08T18:06:50.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.025+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94539396761408] v29'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.025+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.025+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94539396767840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab296ba0 con 0x55fbab2878c0 2024-05-08T18:06:50.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.026+0000 7feec5e11800 20 add_watcher() i=0 2024-05-08T18:06:50.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.026+0000 7feec5e11800 20 add_watcher() i=1 2024-05-08T18:06:50.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.026+0000 7feec5e11800 20 add_watcher() i=2 2024-05-08T18:06:50.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.026+0000 7feec5e11800 20 add_watcher() i=3 2024-05-08T18:06:50.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.026+0000 7feec5e11800 20 add_watcher() i=4 2024-05-08T18:06:50.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.026+0000 7feec5e11800 20 add_watcher() i=5 2024-05-08T18:06:50.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.026+0000 7feec5e11800 20 add_watcher() i=6 2024-05-08T18:06:50.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.026+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94539396767840] v29'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.027+0000 7feec5e11800 20 add_watcher() i=7 2024-05-08T18:06:50.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.027+0000 7feec5e11800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:50.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.027+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab29a060 con 0x55fbab283140 2024-05-08T18:06:50.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.027+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7feebc032040 con 0x55fbab283140 2024-05-08T18:06:50.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.027+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94539396769152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab297580 con 0x55fbab283140 2024-05-08T18:06:50.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.028+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94539396769152] v29'230 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7feebc032040 con 0x55fbab283140 2024-05-08T18:06:50.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.028+0000 7fed777e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:50.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.028+0000 7feec5e11800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:50.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.028+0000 7feec5e11800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:50.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.028+0000 7feec5e11800 5 note: GC not initialized 2024-05-08T18:06:50.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.028+0000 7fed767e4640 20 reqs_thread_entry: start 2024-05-08T18:06:50.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.028+0000 7feec5e11800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:50.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.029+0000 7feec5e11800 20 Filter name: none 2024-05-08T18:06:50.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.029+0000 7fece7fff640 20 reqs_thread_entry: start 2024-05-08T18:06:50.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.029+0000 7feec5e11800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.029+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab2cd0a0 con 0x55fbab283140 2024-05-08T18:06:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.029+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7feebc032040 con 0x55fbab283140 2024-05-08T18:06:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.029+0000 7feec5e11800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:06:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.029+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55fbab2ce5f0 con 0x55fbab2878c0 2024-05-08T18:06:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.029+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(63 usage.10 [call out=15b] v0'0 uv32 ondisk = 0) ==== 152+0+15 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.030+0000 7fed777e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.030+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94539396769152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab299550 con 0x55fbab283140 2024-05-08T18:06:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.031+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94539396769152] v29'231 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7feebc032040 con 0x55fbab283140 2024-05-08T18:06:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.031+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94539396745376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab2ccac0 con 0x7feeb8076040 2024-05-08T18:06:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.032+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94539396745376] v29'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feebc032040 con 0x7feeb8076040 2024-05-08T18:06:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.032+0000 7feec5e11800 20 remove_watcher() i=0 2024-05-08T18:06:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.032+0000 7feec5e11800 2 removed watcher, disabling cache 2024-05-08T18:06:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.032+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94539396749152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab2908a0 con 0x55fbab283140 2024-05-08T18:06:50.045 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.034+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94539396749152] v29'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feebc032040 con 0x55fbab283140 2024-05-08T18:06:50.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.034+0000 7feec5e11800 20 remove_watcher() i=1 2024-05-08T18:06:50.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.034+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94539396751792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab2908a0 con 0x55fbab28c4c0 2024-05-08T18:06:50.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.035+0000 7feec0d3f640 1 -- 172.21.15.50:0/3330459562 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94539396751792] v29'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feeb8081070 con 0x55fbab28c4c0 2024-05-08T18:06:50.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.035+0000 7feec5e11800 20 remove_watcher() i=2 2024-05-08T18:06:50.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.035+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94539396742816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab291760 con 0x7feeb8076040 2024-05-08T18:06:50.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.036+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94539396742816] v29'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feebc032040 con 0x7feeb8076040 2024-05-08T18:06:50.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.036+0000 7feec5e11800 20 remove_watcher() i=3 2024-05-08T18:06:50.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.036+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94539396756736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab291760 con 0x55fbab283140 2024-05-08T18:06:50.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.037+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94539396756736] v29'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feebc032040 con 0x55fbab283140 2024-05-08T18:06:50.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.037+0000 7feec5e11800 20 remove_watcher() i=4 2024-05-08T18:06:50.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.037+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94539396760352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab293500 con 0x55fbab2878c0 2024-05-08T18:06:50.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.038+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94539396760352] v29'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.046 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.038+0000 7feec5e11800 20 remove_watcher() i=5 2024-05-08T18:06:50.047 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.038+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94539396761408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab294320 con 0x55fbab2878c0 2024-05-08T18:06:50.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.039+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94539396761408] v29'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.039+0000 7feec5e11800 20 remove_watcher() i=6 2024-05-08T18:06:50.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.039+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94539396767840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55fbab293500 con 0x55fbab2878c0 2024-05-08T18:06:50.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.040+0000 7feec1540640 1 -- 172.21.15.50:0/3330459562 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94539396767840] v29'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7feeb4045040 con 0x55fbab2878c0 2024-05-08T18:06:50.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.040+0000 7feec5e11800 20 remove_watcher() i=7 2024-05-08T18:06:50.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.040+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fbab2878c0 msgr2=0x55fbab289cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fbab2878c0 0x55fbab289cb0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fbab283140 msgr2=0x55fbab285600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fbab283140 0x55fbab285600 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7feeb8076040 msgr2=0x7feeb8083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7feeb8076040 0x7feeb8083410 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fbab28c4c0 msgr2=0x55fbab28e980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fbab28c4c0 0x55fbab28e980 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fed8c03bb50 msgr2=0x7fed8c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fed8c03bb50 0x7fed8c03e010 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feeb8052fd0 tx=0x7feeb804f000 comp rx=0 tx=0).stop 2024-05-08T18:06:50.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab270640 msgr2=0x55fbab2779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 --2- 172.21.15.50:0/3330459562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab270640 0x55fbab2779a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7feeb403d040 tx=0x7feeb4040810 comp rx=0 tx=0).stop 2024-05-08T18:06:50.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 reap_dead start 2024-05-08T18:06:50.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec1d41640 1 -- 172.21.15.50:0/3330459562 reap_dead start 2024-05-08T18:06:50.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 shutdown_connections 2024-05-08T18:06:50.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 >> 172.21.15.50:0/3330459562 conn(0x55fbab26c910 msgr2=0x55fbab269f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 shutdown_connections 2024-05-08T18:06:50.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 -- 172.21.15.50:0/3330459562 wait complete. 2024-05-08T18:06:50.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 -- 172.21.15.50:0/104741794 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fed9803bb00 msgr2=0x7fed9803dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 --2- 172.21.15.50:0/104741794 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fed9803bb00 0x7fed9803dfc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55fbab148ef0 tx=0x7feeb8005a00 comp rx=0 tx=0).stop 2024-05-08T18:06:50.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 -- 172.21.15.50:0/104741794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab263660 msgr2=0x55fbab263a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 --2- 172.21.15.50:0/104741794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab263660 0x55fbab263a80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7feeb400c720 tx=0x7feeb401ae80 comp rx=0 tx=0).stop 2024-05-08T18:06:50.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 -- 172.21.15.50:0/104741794 shutdown_connections 2024-05-08T18:06:50.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 --2- 172.21.15.50:0/104741794 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fed9803bb00 0x7fed9803dfc0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 --2- 172.21.15.50:0/104741794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab263660 0x55fbab263a80 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 -- 172.21.15.50:0/104741794 >> 172.21.15.50:0/104741794 conn(0x55fbab257700 msgr2=0x55fbab257b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 -- 172.21.15.50:0/104741794 shutdown_connections 2024-05-08T18:06:50.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.041+0000 7feec5e11800 1 -- 172.21.15.50:0/104741794 wait complete. 2024-05-08T18:06:50.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.042+0000 7feec5e11800 1 -- 172.21.15.50:0/3523903791 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fbab24e790 msgr2=0x55fbab250c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.042+0000 7feec5e11800 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55fbab24e790 0x55fbab250c50 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.042+0000 7feec5e11800 1 -- 172.21.15.50:0/3523903791 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fbab249db0 msgr2=0x55fbab24c1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.042+0000 7feec5e11800 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55fbab249db0 0x55fbab24c1a0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.042+0000 7feec5e11800 1 -- 172.21.15.50:0/3523903791 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fbab252840 msgr2=0x55fbab254c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.042+0000 7feec5e11800 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55fbab252840 0x55fbab254c30 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.042+0000 7feec5e11800 1 -- 172.21.15.50:0/3523903791 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7feda803bba0 msgr2=0x7feda803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.042+0000 7feec5e11800 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7feda803bba0 0x7feda803e060 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7feeb8054000 tx=0x7feeb804e000 comp rx=0 tx=0).stop 2024-05-08T18:06:50.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.042+0000 7feec5e11800 1 -- 172.21.15.50:0/3523903791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14dc90 msgr2=0x55fbab149a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.042+0000 7feec5e11800 1 --2- 172.21.15.50:0/3523903791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fbab14dc90 0x55fbab149a20 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x55fbab23b8f0 tx=0x7feeb4000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.042+0000 7feec1d41640 1 -- 172.21.15.50:0/3523903791 reap_dead start 2024-05-08T18:06:50.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.042+0000 7feec5e11800 1 -- 172.21.15.50:0/3523903791 shutdown_connections 2024-05-08T18:06:50.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.042+0000 7feec5e11800 1 -- 172.21.15.50:0/3523903791 >> 172.21.15.50:0/3523903791 conn(0x55fbab143ad0 msgr2=0x55fbab234c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.042+0000 7feec5e11800 1 -- 172.21.15.50:0/3523903791 shutdown_connections 2024-05-08T18:06:50.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.042+0000 7feec5e11800 1 -- 172.21.15.50:0/3523903791 wait complete. 2024-05-08T18:06:50.054 INFO:teuthology.orchestra.run.smithi050.stdout:{"entries":[],"summary":[]} 2024-05-08T18:06:50.054 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2024-05-08T18:06:50.055 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo', '--purge-data'] 2024-05-08T18:06:50.055 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo', '--purge-data'] 2024-05-08T18:06:50.055 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo --purge-data 2024-05-08T18:06:50.101 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:50.101 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.104+0000 7f43ddfa2800 1 Processor -- start 2024-05-08T18:06:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.104+0000 7f43ddfa2800 1 -- start start 2024-05-08T18:06:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.104+0000 7f43ddfa2800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x564ba249a710 0x564ba249ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.104+0000 7f43ddfa2800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x564ba249b1b0 con 0x564ba249b8d0 2024-05-08T18:06:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.104+0000 7f43ddfa2800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x564ba233ae20 con 0x564ba249a710 2024-05-08T18:06:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.104+0000 7f43dbaa1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x564ba249a710 0x564ba249ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.104+0000 7f43dbaa1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x564ba249a710 0x564ba249ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45732/0 (socket says 172.21.15.50:45732) 2024-05-08T18:06:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.104+0000 7f43dbaa1640 1 -- 172.21.15.50:0/3349958638 learned_addr learned my addr 172.21.15.50:0/3349958638 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.104+0000 7f43db2a0640 1 --1- 172.21.15.50:0/3349958638 >> v1:172.21.15.46:6789/0 conn(0x564ba249b8d0 0x564ba249bd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50724/0 (socket says 172.21.15.50:50724) 2024-05-08T18:06:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.105+0000 7f43dbaa1640 1 -- 172.21.15.50:0/3349958638 >> v1:172.21.15.46:6789/0 conn(0x564ba249b8d0 legacy=0x564ba249bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.105+0000 7f43dbaa1640 1 -- 172.21.15.50:0/3349958638 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba249c490 con 0x564ba249a710 2024-05-08T18:06:50.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.107+0000 7f43dbaa1640 1 --2- 172.21.15.50:0/3349958638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 0x564ba249ab30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00a520 tx=0x7f43cc01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ae376f8ababeae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.107+0000 7f43daa9f640 1 -- 172.21.15.50:0/3349958638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43cc057070 con 0x564ba249a710 2024-05-08T18:06:50.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.107+0000 7f43daa9f640 1 -- 172.21.15.50:0/3349958638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43cc052570 con 0x564ba249a710 2024-05-08T18:06:50.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.107+0000 7f43daa9f640 1 -- 172.21.15.50:0/3349958638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43cc052870 con 0x564ba249a710 2024-05-08T18:06:50.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.107+0000 7f43ddfa2800 1 -- 172.21.15.50:0/3349958638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 msgr2=0x564ba249ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.107+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/3349958638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 0x564ba249ab30 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00a520 tx=0x7f43cc01d010 comp rx=0 tx=0).stop 2024-05-08T18:06:50.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.107+0000 7f43ddfa2800 1 -- 172.21.15.50:0/3349958638 shutdown_connections 2024-05-08T18:06:50.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.107+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/3349958638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 0x564ba249ab30 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.107+0000 7f43ddfa2800 1 -- 172.21.15.50:0/3349958638 >> 172.21.15.50:0/3349958638 conn(0x564ba2495c90 msgr2=0x564ba24980d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.107+0000 7f43ddfa2800 1 -- 172.21.15.50:0/3349958638 shutdown_connections 2024-05-08T18:06:50.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.107+0000 7f43ddfa2800 1 -- 172.21.15.50:0/3349958638 wait complete. 2024-05-08T18:06:50.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.123+0000 7f43ddfa2800 1 Processor -- start 2024-05-08T18:06:50.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.123+0000 7f43ddfa2800 1 -- start start 2024-05-08T18:06:50.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.123+0000 7f43ddfa2800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 0x564ba258bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.123+0000 7f43ddfa2800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ba23e5a30 con 0x564ba249a710 2024-05-08T18:06:50.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.123+0000 7f43dbaa1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 0x564ba258bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.123+0000 7f43dbaa1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 0x564ba258bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45736/0 (socket says 172.21.15.50:45736) 2024-05-08T18:06:50.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.123+0000 7f43dbaa1640 1 -- 172.21.15.50:0/3758429571 learned_addr learned my addr 172.21.15.50:0/3758429571 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.123+0000 7f43dbaa1640 1 -- 172.21.15.50:0/3758429571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba258c240 con 0x564ba249a710 2024-05-08T18:06:50.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.124+0000 7f43dbaa1640 1 --2- 172.21.15.50:0/3758429571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 0x564ba258bc70 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f43cc055040 tx=0x7f43cc04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff04eff73e3cdbc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.124+0000 7f42c9e0b640 1 -- 172.21.15.50:0/3758429571 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43cc057040 con 0x564ba249a710 2024-05-08T18:06:50.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.124+0000 7f42c9e0b640 1 -- 172.21.15.50:0/3758429571 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43cc052870 con 0x564ba249a710 2024-05-08T18:06:50.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.124+0000 7f42c9e0b640 1 -- 172.21.15.50:0/3758429571 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43cc052b90 con 0x564ba249a710 2024-05-08T18:06:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.125+0000 7f43ddfa2800 1 -- 172.21.15.50:0/3758429571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 msgr2=0x564ba258bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.125+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/3758429571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 0x564ba258bc70 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f43cc055040 tx=0x7f43cc04c000 comp rx=0 tx=0).stop 2024-05-08T18:06:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.125+0000 7f43ddfa2800 1 -- 172.21.15.50:0/3758429571 shutdown_connections 2024-05-08T18:06:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.125+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/3758429571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 0x564ba258bc70 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.125+0000 7f43ddfa2800 1 -- 172.21.15.50:0/3758429571 >> 172.21.15.50:0/3758429571 conn(0x564ba2495c90 msgr2=0x564ba2587590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.125+0000 7f43ddfa2800 1 -- 172.21.15.50:0/3758429571 shutdown_connections 2024-05-08T18:06:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.125+0000 7f43ddfa2800 1 -- 172.21.15.50:0/3758429571 wait complete. 2024-05-08T18:06:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.125+0000 7f43ddfa2800 1 Processor -- start 2024-05-08T18:06:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.125+0000 7f43ddfa2800 1 -- start start 2024-05-08T18:06:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.125+0000 7f43ddfa2800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 0x564ba25936c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.125+0000 7f43ddfa2800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ba24a3da0 con 0x564ba249a710 2024-05-08T18:06:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.125+0000 7f43dbaa1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 0x564ba25936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.125+0000 7f43dbaa1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 0x564ba25936c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45748/0 (socket says 172.21.15.50:45748) 2024-05-08T18:06:50.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.125+0000 7f43dbaa1640 1 -- 172.21.15.50:0/1880189890 learned_addr learned my addr 172.21.15.50:0/1880189890 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.126+0000 7f43dbaa1640 1 -- 172.21.15.50:0/1880189890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba2593c00 con 0x564ba249a710 2024-05-08T18:06:50.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.126+0000 7f43dbaa1640 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 0x564ba25936c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0524f0 tx=0x7f43cc003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.126+0000 7f42c3fff640 1 -- 172.21.15.50:0/1880189890 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43cc073070 con 0x564ba249a710 2024-05-08T18:06:50.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.126+0000 7f42c3fff640 1 -- 172.21.15.50:0/1880189890 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43cc05d570 con 0x564ba249a710 2024-05-08T18:06:50.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.126+0000 7f42c3fff640 1 -- 172.21.15.50:0/1880189890 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43cc057070 con 0x564ba249a710 2024-05-08T18:06:50.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.126+0000 7f43ddfa2800 1 -- 172.21.15.50:0/1880189890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564ba2593e90 con 0x564ba249a710 2024-05-08T18:06:50.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.126+0000 7f43ddfa2800 1 -- 172.21.15.50:0/1880189890 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba2596b40 con 0x564ba249a710 2024-05-08T18:06:50.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.127+0000 7f42c3fff640 1 -- 172.21.15.50:0/1880189890 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f43cc076070 con 0x564ba249a710 2024-05-08T18:06:50.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.127+0000 7f42c3fff640 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f42c403bb50 0x7f42c403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.127+0000 7f42c3fff640 1 -- 172.21.15.50:0/1880189890 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f43cc094350 con 0x564ba249a710 2024-05-08T18:06:50.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.127+0000 7f43db2a0640 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f42c403bb50 0x7f42c403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.127+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ba249b8d0 0x564ba259a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.128+0000 7f43ddfa2800 1 -- 172.21.15.50:0/1880189890 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba259ae80 con 0x564ba249b8d0 2024-05-08T18:06:50.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.128+0000 7f43dc2a2640 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ba249b8d0 0x564ba259a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.128+0000 7f43db2a0640 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f42c403bb50 0x7f42c403e010 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f43d0001e30 tx=0x7f43d0001390 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.128+0000 7f43dc2a2640 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ba249b8d0 0x564ba259a900 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.128+0000 7f43dc2a2640 1 -- 172.21.15.50:0/1880189890 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f43d400a4a0 con 0x564ba249b8d0 2024-05-08T18:06:50.131 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.128+0000 7f43ddfa2800 1 -- 172.21.15.50:0/1880189890 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba259b740 con 0x564ba249b8d0 2024-05-08T18:06:50.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.128+0000 7f43dc2a2640 1 -- 172.21.15.50:0/1880189890 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f43d400a4a0 con 0x564ba249b8d0 2024-05-08T18:06:50.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.129+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ba259d050 0x564ba259f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.129+0000 7f43ddfa2800 1 -- 172.21.15.50:0/1880189890 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba259fc80 con 0x564ba259d050 2024-05-08T18:06:50.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.129+0000 7f43dbaa1640 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ba259d050 0x564ba259f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.129+0000 7f43dbaa1640 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ba259d050 0x564ba259f4f0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.129+0000 7f43dbaa1640 1 -- 172.21.15.50:0/1880189890 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f43cc075030 con 0x564ba259d050 2024-05-08T18:06:50.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.130+0000 7f43ddfa2800 1 -- 172.21.15.50:0/1880189890 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba259bf90 con 0x564ba249b8d0 2024-05-08T18:06:50.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.130+0000 7f43dc2a2640 1 -- 172.21.15.50:0/1880189890 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f43d400a4a0 con 0x564ba249b8d0 2024-05-08T18:06:50.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.130+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ba25a10e0 0x564ba25a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.130+0000 7f43ddfa2800 1 -- 172.21.15.50:0/1880189890 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba25a3bc0 con 0x564ba25a10e0 2024-05-08T18:06:50.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.130+0000 7f43db2a0640 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ba25a10e0 0x564ba25a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.130+0000 7f43db2a0640 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ba25a10e0 0x564ba25a34d0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.131+0000 7f43db2a0640 1 -- 172.21.15.50:0/1880189890 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f43d000bd50 con 0x564ba25a10e0 2024-05-08T18:06:50.132 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.131+0000 7f43ddfa2800 1 Processor -- start 2024-05-08T18:06:50.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.131+0000 7f43ddfa2800 1 -- start start 2024-05-08T18:06:50.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.131+0000 7f43ddfa2800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba2410180 0x564ba240e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.131+0000 7f43ddfa2800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ba238c330 con 0x564ba2410180 2024-05-08T18:06:50.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.131+0000 7f43dbaa1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba2410180 0x564ba240e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.131+0000 7f43dbaa1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba2410180 0x564ba240e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45750/0 (socket says 172.21.15.50:45750) 2024-05-08T18:06:50.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.131+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2383886032 learned_addr learned my addr 172.21.15.50:0/2383886032 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.132+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2383886032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba240eb30 con 0x564ba2410180 2024-05-08T18:06:50.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.132+0000 7f43dbaa1640 1 --2- 172.21.15.50:0/2383886032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba2410180 0x564ba240e5f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f43cc006d00 tx=0x7f43cc0064c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.132+0000 7f42c17fa640 1 -- 172.21.15.50:0/2383886032 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43cc05d890 con 0x564ba2410180 2024-05-08T18:06:50.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.132+0000 7f42c17fa640 1 -- 172.21.15.50:0/2383886032 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43cc05da30 con 0x564ba2410180 2024-05-08T18:06:50.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.132+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2383886032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564ba240edc0 con 0x564ba2410180 2024-05-08T18:06:50.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.132+0000 7f42c17fa640 1 -- 172.21.15.50:0/2383886032 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43cc05dd10 con 0x564ba2410180 2024-05-08T18:06:50.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.132+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2383886032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba240f2c0 con 0x564ba2410180 2024-05-08T18:06:50.133 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.133+0000 7f42c17fa640 1 -- 172.21.15.50:0/2383886032 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f43cc092260 con 0x564ba2410180 2024-05-08T18:06:50.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.133+0000 7f42c17fa640 1 --2- 172.21.15.50:0/2383886032 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f42b403bba0 0x7f42b403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.133+0000 7f42c17fa640 1 -- 172.21.15.50:0/2383886032 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f43cc089070 con 0x564ba2410180 2024-05-08T18:06:50.134 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.133+0000 7f43db2a0640 1 --2- 172.21.15.50:0/2383886032 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f42b403bba0 0x7f42b403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.133+0000 7f43ddfa2800 1 Processor -- start 2024-05-08T18:06:50.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.133+0000 7f43ddfa2800 1 -- start start 2024-05-08T18:06:50.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.134+0000 7f43db2a0640 1 --2- 172.21.15.50:0/2383886032 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f42b403bba0 0x7f42b403e060 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x564ba2598030 tx=0x7f43d000f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.134+0000 7f43ddfa2800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba25c1310 0x564ba25c1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.134+0000 7f43ddfa2800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ba258c500 con 0x564ba25c1310 2024-05-08T18:06:50.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.134+0000 7f43dbaa1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba25c1310 0x564ba25c1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.134+0000 7f43dbaa1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba25c1310 0x564ba25c1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45762/0 (socket says 172.21.15.50:45762) 2024-05-08T18:06:50.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.134+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2187820538 learned_addr learned my addr 172.21.15.50:0/2187820538 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.134+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2187820538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba25c1d00 con 0x564ba25c1310 2024-05-08T18:06:50.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.134+0000 7f43dbaa1640 1 --2- 172.21.15.50:0/2187820538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba25c1310 0x564ba25c1730 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f43cc05efa0 tx=0x7f43cc076580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5736561683c3c60f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.134+0000 7f42baffd640 1 -- 172.21.15.50:0/2187820538 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43cc08c3c0 con 0x564ba25c1310 2024-05-08T18:06:50.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.134+0000 7f42baffd640 1 -- 172.21.15.50:0/2187820538 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43cc08c580 con 0x564ba25c1310 2024-05-08T18:06:50.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.134+0000 7f42baffd640 1 -- 172.21.15.50:0/2187820538 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43cc06f040 con 0x564ba25c1310 2024-05-08T18:06:50.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.135+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2187820538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba25c1310 msgr2=0x564ba25c1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.135+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/2187820538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba25c1310 0x564ba25c1730 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f43cc05efa0 tx=0x7f43cc076580 comp rx=0 tx=0).stop 2024-05-08T18:06:50.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.135+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2187820538 shutdown_connections 2024-05-08T18:06:50.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.135+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/2187820538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba25c1310 0x564ba25c1730 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.135+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2187820538 >> 172.21.15.50:0/2187820538 conn(0x564ba25bd590 msgr2=0x564ba25bb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.135+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2187820538 shutdown_connections 2024-05-08T18:06:50.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.135+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2187820538 wait complete. 2024-05-08T18:06:50.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.135+0000 7f43ddfa2800 1 Processor -- start 2024-05-08T18:06:50.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.135+0000 7f43ddfa2800 1 -- start start 2024-05-08T18:06:50.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.135+0000 7f43ddfa2800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba25c1310 0x564ba25c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.135+0000 7f43ddfa2800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ba25c2370 con 0x564ba25c1310 2024-05-08T18:06:50.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.136+0000 7f43dbaa1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba25c1310 0x564ba25c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.136+0000 7f43dbaa1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba25c1310 0x564ba25c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45774/0 (socket says 172.21.15.50:45774) 2024-05-08T18:06:50.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.136+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 learned_addr learned my addr 172.21.15.50:0/2595754535 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.136+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ba25c8b70 con 0x564ba25c1310 2024-05-08T18:06:50.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.136+0000 7f43dbaa1640 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba25c1310 0x564ba25c8630 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f43cc05bfd0 tx=0x7f43cc06ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.136+0000 7f42b97fa640 1 -- 172.21.15.50:0/2595754535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43cc06ec90 con 0x564ba25c1310 2024-05-08T18:06:50.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.136+0000 7f42b97fa640 1 -- 172.21.15.50:0/2595754535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43cc08b370 con 0x564ba25c1310 2024-05-08T18:06:50.146 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.136+0000 7f42b97fa640 1 -- 172.21.15.50:0/2595754535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43cc08cd50 con 0x564ba25c1310 2024-05-08T18:06:50.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.136+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564ba25c8b70 con 0x564ba25c1310 2024-05-08T18:06:50.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.136+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ba25c8b70 con 0x564ba25c1310 2024-05-08T18:06:50.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.137+0000 7f42b97fa640 1 -- 172.21.15.50:0/2595754535 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f43cc0b7030 con 0x564ba25c1310 2024-05-08T18:06:50.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.138+0000 7f42b97fa640 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f42a403bb50 0x7f42a403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.138+0000 7f42b97fa640 1 -- 172.21.15.50:0/2595754535 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f43cc0b5070 con 0x564ba25c1310 2024-05-08T18:06:50.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.138+0000 7f43db2a0640 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f42a403bb50 0x7f42a403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.138+0000 7f43db2a0640 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f42a403bb50 0x7f42a403e010 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x564ba25a3fc0 tx=0x7f43d0010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.138+0000 7f43ddfa2800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.138+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ba25d3dc0 0x564ba25d6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.138+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d69d0 con 0x564ba25d3dc0 2024-05-08T18:06:50.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.138+0000 7f43dc2a2640 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ba25d3dc0 0x564ba25d6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.139+0000 7f43dc2a2640 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ba25d3dc0 0x564ba25d6280 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.139+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f43d400a4a0 con 0x564ba25d3dc0 2024-05-08T18:06:50.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.139+0000 7f43ddfa2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.139+0000 7f43ddfa2800 20 realm 2024-05-08T18:06:50.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.139+0000 7f43ddfa2800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.139+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d6de0 con 0x564ba25d3dc0 2024-05-08T18:06:50.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.139+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f43d400a4a0 con 0x564ba25d3dc0 2024-05-08T18:06:50.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.139+0000 7f43ddfa2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.139+0000 7f43ddfa2800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:50.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.139+0000 7f43ddfa2800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.139+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d76b0 con 0x564ba25d3dc0 2024-05-08T18:06:50.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.140+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f43d400a4a0 con 0x564ba25d3dc0 2024-05-08T18:06:50.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.140+0000 7f43ddfa2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.140+0000 7f43ddfa2800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.140+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d3dc0 2024-05-08T18:06:50.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.140+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f43d400a4a0 con 0x564ba25d3dc0 2024-05-08T18:06:50.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.140+0000 7f43ddfa2800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:50.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.140+0000 7f43ddfa2800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.140+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ba25d87b0 0x564ba25daba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.140+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba25db1f0 con 0x564ba25d87b0 2024-05-08T18:06:50.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.140+0000 7f43dbaa1640 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ba25d87b0 0x564ba25daba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.141+0000 7f43dbaa1640 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ba25d87b0 0x564ba25daba0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.141+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.141+0000 7f43ddfa2800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:50.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.141+0000 7f43ddfa2800 20 searching for the correct realm 2024-05-08T18:06:50.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.141+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ba25dd1c0 0x564ba25df680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.141+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25dfd00 con 0x564ba25dd1c0 2024-05-08T18:06:50.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.141+0000 7f43db2a0640 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ba25dd1c0 0x564ba25df680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.141+0000 7f43db2a0640 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ba25dd1c0 0x564ba25df680 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.142+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.142+0000 7f43db2a0640 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f43d000f070 0x7f43d002e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.149 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.142+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25dfd00 con 0x7f43d000f070 2024-05-08T18:06:50.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.142+0000 7f43dc2a2640 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f43d000f070 0x7f43d002e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.142+0000 7f43dc2a2640 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f43d000f070 0x7f43d002e910 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.142+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d400a4a0 con 0x7f43d000f070 2024-05-08T18:06:50.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.142+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d87b0 2024-05-08T18:06:50.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.143+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.143+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d3dc0 2024-05-08T18:06:50.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.143+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d400a4a0 con 0x564ba25d3dc0 2024-05-08T18:06:50.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.143+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d3dc0 2024-05-08T18:06:50.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.143+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d400a4a0 con 0x564ba25d3dc0 2024-05-08T18:06:50.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.143+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d3dc0 2024-05-08T18:06:50.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.143+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d400a4a0 con 0x564ba25d3dc0 2024-05-08T18:06:50.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.143+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d87b0 2024-05-08T18:06:50.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.144+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.150 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.144+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x7f43d000f070 2024-05-08T18:06:50.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.144+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d400a4a0 con 0x7f43d000f070 2024-05-08T18:06:50.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.144+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25dd1c0 2024-05-08T18:06:50.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.144+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.144+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d3dc0 2024-05-08T18:06:50.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.144+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d400a4a0 con 0x564ba25d3dc0 2024-05-08T18:06:50.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.144+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d87b0 2024-05-08T18:06:50.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.145+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.145+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d3dc0 2024-05-08T18:06:50.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.145+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d400a4a0 con 0x564ba25d3dc0 2024-05-08T18:06:50.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.145+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x7f43d000f070 2024-05-08T18:06:50.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.145+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.145+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25dd1c0 2024-05-08T18:06:50.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.145+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.145+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25dd1c0 2024-05-08T18:06:50.151 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.146+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.146+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d87b0 2024-05-08T18:06:50.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.146+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.146+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25dd1c0 2024-05-08T18:06:50.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.146+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.146+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25dd1c0 2024-05-08T18:06:50.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.146+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.152 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.146+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d3dc0 2024-05-08T18:06:50.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.147+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.160 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.147+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d3dc0 2024-05-08T18:06:50.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.147+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.147+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d87b0 2024-05-08T18:06:50.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.147+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.147+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25dd1c0 2024-05-08T18:06:50.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.148+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.148+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x7f43d000f070 2024-05-08T18:06:50.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.148+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.148+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d3dc0 2024-05-08T18:06:50.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.148+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.148+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d87b0 2024-05-08T18:06:50.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.149+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.149+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25dd1c0 2024-05-08T18:06:50.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.149+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.161 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.149+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d87b0 2024-05-08T18:06:50.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.149+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.149+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x7f43d000f070 2024-05-08T18:06:50.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.149+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.149+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25dd1c0 2024-05-08T18:06:50.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.150+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.150+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d87b0 2024-05-08T18:06:50.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.150+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.150+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x564ba25d3dc0 2024-05-08T18:06:50.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.150+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.150+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ba25d7a50 con 0x7f43d000f070 2024-05-08T18:06:50.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.150+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.150+0000 7f43ddfa2800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:50.162 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.150+0000 7f43ddfa2800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:50.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.150+0000 7f43ddfa2800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:50.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.150+0000 7f43ddfa2800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:50.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.150+0000 7f43ddfa2800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:50.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.150+0000 7f43ddfa2800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:50.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba25dbab0 con 0x564ba25d3dc0 2024-05-08T18:06:50.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba25dbab0 con 0x564ba25d3dc0 2024-05-08T18:06:50.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:50.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba25dbab0 con 0x564ba25dd1c0 2024-05-08T18:06:50.163 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:50.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 20 zone default found 2024-05-08T18:06:50.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 4 Realm: () 2024-05-08T18:06:50.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:50.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:50.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:50.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 20 zonegroup default 2024-05-08T18:06:50.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.151+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba25dc440 con 0x564ba25d87b0 2024-05-08T18:06:50.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.152+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.152+0000 7f43ddfa2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.152+0000 7f43ddfa2800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.152+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba25dc010 con 0x564ba25dd1c0 2024-05-08T18:06:50.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.152+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.164 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.152+0000 7f43ddfa2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.152+0000 7f43ddfa2800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.152+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e20f0 con 0x564ba25dd1c0 2024-05-08T18:06:50.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.152+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.152+0000 7f43ddfa2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.152+0000 7f43ddfa2800 20 started sync module instance, tier type = 2024-05-08T18:06:50.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.152+0000 7f43ddfa2800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:50.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.152+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e14a0 con 0x7f43d000f070 2024-05-08T18:06:50.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.153+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.154+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94882846611904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e14a0 con 0x7f43d000f070 2024-05-08T18:06:50.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.154+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e2b70 con 0x564ba25d3dc0 2024-05-08T18:06:50.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.155+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.155+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94882846611904] v29'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.155+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94882846615232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e2fd0 con 0x564ba25d3dc0 2024-05-08T18:06:50.165 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.155+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e1010 con 0x564ba25dd1c0 2024-05-08T18:06:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.156+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94882846615232] v29'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.156+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.156+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94882846616288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e3a70 con 0x564ba25dd1c0 2024-05-08T18:06:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.156+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e4430 con 0x7f43d000f070 2024-05-08T18:06:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.158+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94882846616288] v29'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.158+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.158+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94882846621568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e4850 con 0x7f43d000f070 2024-05-08T18:06:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.158+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e5210 con 0x564ba25d3dc0 2024-05-08T18:06:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.159+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.159+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94882846625120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e5180 con 0x564ba25d3dc0 2024-05-08T18:06:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.159+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e61b0 con 0x564ba25d87b0 2024-05-08T18:06:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.159+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94882846621568] v29'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.160+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94882846625120] v29'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.166 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.160+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.160+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94882846628992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e5600 con 0x564ba25d87b0 2024-05-08T18:06:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.160+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e6a50 con 0x564ba25d87b0 2024-05-08T18:06:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.161+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94882846628992] v29'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.161+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.161+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94882846622624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e6a50 con 0x564ba25d87b0 2024-05-08T18:06:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.161+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e74a0 con 0x564ba25d87b0 2024-05-08T18:06:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.162+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94882846622624] v29'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.162+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.162+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94882846633872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e7680 con 0x564ba25d87b0 2024-05-08T18:06:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.162+0000 7f43ddfa2800 20 add_watcher() i=0 2024-05-08T18:06:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.162+0000 7f43ddfa2800 20 add_watcher() i=1 2024-05-08T18:06:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.162+0000 7f43ddfa2800 20 add_watcher() i=2 2024-05-08T18:06:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.162+0000 7f43ddfa2800 20 add_watcher() i=3 2024-05-08T18:06:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.162+0000 7f43ddfa2800 20 add_watcher() i=4 2024-05-08T18:06:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.162+0000 7f43ddfa2800 20 add_watcher() i=5 2024-05-08T18:06:50.167 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.162+0000 7f43ddfa2800 20 add_watcher() i=6 2024-05-08T18:06:50.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.163+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94882846633872] v29'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.163+0000 7f43ddfa2800 20 add_watcher() i=7 2024-05-08T18:06:50.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.163+0000 7f43ddfa2800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:50.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.163+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba25eaa50 con 0x564ba25d3dc0 2024-05-08T18:06:50.170 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.164+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.164+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94882846635376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25e7e40 con 0x564ba25d3dc0 2024-05-08T18:06:50.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.164+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94882846635376] v29'232 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.165+0000 7f4291fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:50.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.165+0000 7f43ddfa2800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:50.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.165+0000 7f43ddfa2800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:50.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.165+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f3210 con 0x564ba25d3dc0 2024-05-08T18:06:50.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.166+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v29'35 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.166+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f38e0 con 0x564ba25d87b0 2024-05-08T18:06:50.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.167+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(63 gc.1 [create,call,call,call] v29'29 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.167+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f38e0 con 0x564ba25dd1c0 2024-05-08T18:06:50.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.168+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v29'28 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.168+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f4380 con 0x564ba25d3dc0 2024-05-08T18:06:50.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.169+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v29'233 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.171 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.169+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f1050 con 0x7f43d000f070 2024-05-08T18:06:50.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.170+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v29'74 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.170+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f1050 con 0x564ba25dd1c0 2024-05-08T18:06:50.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.171+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v29'49 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.171+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f50b0 con 0x564ba25d3dc0 2024-05-08T18:06:50.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.172+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v29'31 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.172 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.172+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f50b0 con 0x564ba25d87b0 2024-05-08T18:06:50.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.173+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(69 gc.7 [create,call,call,call] v29'49 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.173+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f50b0 con 0x7f43d000f070 2024-05-08T18:06:50.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.174+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v29'28 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.174+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f5140 con 0x564ba25dd1c0 2024-05-08T18:06:50.173 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.176+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v29'34 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.176+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f51d0 con 0x7f43d000f070 2024-05-08T18:06:50.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.176+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v29'75 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.174 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.176+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f51d0 con 0x564ba25d87b0 2024-05-08T18:06:50.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.178+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(73 gc.11 [create,call,call,call] v29'34 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.178+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f53a0 con 0x7f43d000f070 2024-05-08T18:06:50.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.179+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v29'29 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.179+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f53a0 con 0x564ba25d87b0 2024-05-08T18:06:50.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.180+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(75 gc.13 [create,call,call,call] v29'55 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.180+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f53a0 con 0x564ba25d3dc0 2024-05-08T18:06:50.179 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.181+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v29'234 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.181+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f53a0 con 0x564ba25dd1c0 2024-05-08T18:06:50.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.182+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v29'69 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.182+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f53a0 con 0x7f43d000f070 2024-05-08T18:06:50.180 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.183+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v29'76 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.183+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f53a0 con 0x564ba25dd1c0 2024-05-08T18:06:50.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.184+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v29'28 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.181 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.184+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f57c0 con 0x564ba25d3dc0 2024-05-08T18:06:50.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.185+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v29'36 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.183 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.185+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f57c0 con 0x564ba25d3dc0 2024-05-08T18:06:50.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.186+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v29'235 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.184 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.186+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f57c0 con 0x564ba25dd1c0 2024-05-08T18:06:50.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.187+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v29'70 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.187+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f57c0 con 0x7f43d000f070 2024-05-08T18:06:50.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.188+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v29'38 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.188+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f57c0 con 0x564ba25dd1c0 2024-05-08T18:06:50.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.189+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v29'30 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.189+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f57c0 con 0x564ba25dd1c0 2024-05-08T18:06:50.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.190+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v29'34 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.190+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f57c0 con 0x564ba25dd1c0 2024-05-08T18:06:50.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.191+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v29'26 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.191+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f57c0 con 0x564ba25dd1c0 2024-05-08T18:06:50.189 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.192+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v29'50 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.190 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.192+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f57c0 con 0x564ba25d87b0 2024-05-08T18:06:50.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.193+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(88 gc.26 [create,call,call,call] v29'30 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.191 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.193+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f57c0 con 0x564ba25dd1c0 2024-05-08T18:06:50.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.194+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v29'29 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.192 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.194+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f57c0 con 0x564ba25d87b0 2024-05-08T18:06:50.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.195+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(90 gc.28 [create,call,call,call] v29'50 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.193 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.195+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f57c0 con 0x564ba25dd1c0 2024-05-08T18:06:50.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.196+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v29'71 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.194 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.196+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f57c0 con 0x564ba25d87b0 2024-05-08T18:06:50.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.197+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(92 gc.30 [create,call,call,call] v29'56 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.195 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.197+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f57c0 con 0x7f43d000f070 2024-05-08T18:06:50.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.198+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v29'34 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.199+0000 7f4290fe1640 20 reqs_thread_entry: start 2024-05-08T18:06:50.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.199+0000 7f43ddfa2800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:50.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.199+0000 7f43ddfa2800 20 Filter name: none 2024-05-08T18:06:50.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.199+0000 7f420a7fc640 20 reqs_thread_entry: start 2024-05-08T18:06:50.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.199+0000 7f43ddfa2800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-05-08T18:06:50.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.199+0000 7f43ddfa2800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.199+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba26205e0 con 0x564ba25d3dc0 2024-05-08T18:06:50.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.199+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(94 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.199+0000 7f43ddfa2800 20 rados_obj.operate() r=0 bl.length=391 2024-05-08T18:06:50.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.199+0000 7f43ddfa2800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-05-08T18:06:50.198 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.199+0000 7f43ddfa2800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-05-08T18:06:50.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.199+0000 7f43ddfa2800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:50.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.199+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:95 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba2620550 con 0x7f43d000f070 2024-05-08T18:06:50.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.200+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(95 foo.buckets [call out=183b] v0'0 uv45 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.200+0000 7f43ddfa2800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7 : miss 2024-05-08T18:06:50.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.200+0000 7f43ddfa2800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.199 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.200+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:96 8.11 8:8dcfa685:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba2622c90 con 0x564ba25dd1c0 2024-05-08T18:06:50.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.200+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(96 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.200+0000 7f43ddfa2800 20 rados_obj.operate() r=0 bl.length=379 2024-05-08T18:06:50.201 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.200+0000 7f43ddfa2800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7 info.flags=0x17 2024-05-08T18:06:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.200+0000 7f43ddfa2800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7 to cache LRU end 2024-05-08T18:06:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.200+0000 7f43ddfa2800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.200+0000 7f43ddfa2800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-05-08T18:06:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.200+0000 7f43ddfa2800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7 2024-05-08T18:06:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.200+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:97 4.26 4:656c7161:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba26a0370 con 0x564ba25d87b0 2024-05-08T18:06:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.201+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(97 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.0 [call out=85b] v0'0 uv11 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.201+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:98 4.13 4:caae27a5:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba26a0370 con 0x7f43d000f070 2024-05-08T18:06:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.201+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(98 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.1 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.201+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:99 4.21 4:87933c90:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba26a0370 con 0x564ba25d87b0 2024-05-08T18:06:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.202+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(99 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.2 [call out=85b] v0'0 uv11 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.202+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:100 4.2e 4:7408f608:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba26a0370 con 0x564ba25dd1c0 2024-05-08T18:06:50.202 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.202+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 28 ==== osd_op_reply(100 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.3 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.202+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:101 4.20 4:044ed8a5:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba26a03b0 con 0x564ba25d3dc0 2024-05-08T18:06:50.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.202+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(101 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.4 [call out=425b] v0'0 uv3 ondisk = 0) ==== 194+0+425 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.202+0000 7f43ddfa2800 10 cls_bucket_list_unordered: got 15f6f8dc036519d7fe15b39338f6e5db 2024-05-08T18:06:50.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.202+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:102 4.2b 4:d6c558e4:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba26a03b0 con 0x564ba25d3dc0 2024-05-08T18:06:50.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.203+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(102 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.5 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.203+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:103 4.37 4:ee400b20:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba26a0a30 con 0x564ba25d87b0 2024-05-08T18:06:50.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.203+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(103 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.6 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.203+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:104 4.1c 4:3a9e36b8:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba26a0a30 con 0x564ba25d87b0 2024-05-08T18:06:50.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.204+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(104 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.7 [call out=85b] v0'0 uv9 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43cc075030 con 0x564ba25d87b0 2024-05-08T18:06:50.203 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.204+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:105 4.35 4:ace44e4b:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba26a0a30 con 0x564ba25d3dc0 2024-05-08T18:06:50.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.204+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(105 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.8 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.204+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:106 4.13 4:ca75e525:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba26a0a30 con 0x7f43d000f070 2024-05-08T18:06:50.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.204+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 20 ==== osd_op_reply(106 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.9 [call out=85b] v0'0 uv4 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.204+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:107 4.3a 4:5f16430d:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba26a0a30 con 0x564ba25dd1c0 2024-05-08T18:06:50.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.205+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 29 ==== osd_op_reply(107 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.10 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.205+0000 7f43ddfa2800 20 get_obj_state: octx=0x7f43cc06cb20 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x564ba269e3f8 s->prefetch_data=0 2024-05-08T18:06:50.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.205+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:108 3.3es0 3:7c3d61fc:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7_15f6f8dc036519d7fe15b39338f6e5db:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba269f480 con 0x564ba25d87b0 2024-05-08T18:06:50.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.205+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(108 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7_15f6f8dc036519d7fe15b39338f6e5db [getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 262+0+893 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.205 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.205+0000 7f43ddfa2800 10 manifest: total_size = 6 2024-05-08T18:06:50.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.205+0000 7f43ddfa2800 20 get_obj_state: setting s->obj_tag to 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7198157717793758438 2024-05-08T18:06:50.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.205+0000 7f43ddfa2800 20 get_obj_state: octx=0x7f43cc06cb20 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x564ba269e3f8 s->prefetch_data=0 2024-05-08T18:06:50.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.205+0000 7f43ddfa2800 20 prepare_atomic_modification: state is not atomic. state=0x564ba269e3f8 2024-05-08T18:06:50.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.205+0000 7f43ddfa2800 20 bucket index object: :.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.4 2024-05-08T18:06:50.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.205+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:109 4.20 4:044ed8a5:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba269fde0 con 0x564ba25d3dc0 2024-05-08T18:06:50.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.207+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 32 ==== osd_op_reply(109 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.4 [stat,call,call] v29'4 uv4 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.206 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.207+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:110 3.3es0 3:7c3d61fc:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7_15f6f8dc036519d7fe15b39338f6e5db:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba269fde0 con 0x564ba25d87b0 2024-05-08T18:06:50.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.208+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(110 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7_15f6f8dc036519d7fe15b39338f6e5db [call] v29'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.208+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:111 4.20 4:044ed8a5:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba2627a40 con 0x564ba25d3dc0 2024-05-08T18:06:50.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.208+0000 7f43ddfa2800 20 list_objects_ordered: starting attempt 1 2024-05-08T18:06:50.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.208+0000 7f43ddfa2800 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-05-08T18:06:50.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.208+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:112 4.26 4:656c7161:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba2627a40 con 0x564ba25d87b0 2024-05-08T18:06:50.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.208+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:113 4.13 4:caae27a5:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba262bce0 con 0x7f43d000f070 2024-05-08T18:06:50.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.209+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:114 4.21 4:87933c90:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba2627a40 con 0x564ba25d87b0 2024-05-08T18:06:50.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.209+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== osd_op_reply(112 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.0 [call out=85b] v0'0 uv11 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.209+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:115 4.2e 4:7408f608:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba2627a40 con 0x564ba25dd1c0 2024-05-08T18:06:50.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.209+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:116 4.20 4:044ed8a5:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba262bce0 con 0x564ba25d3dc0 2024-05-08T18:06:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.209+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:117 4.2b 4:d6c558e4:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba26335f0 con 0x564ba25d3dc0 2024-05-08T18:06:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.209+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:118 4.37 4:ee400b20:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba2636410 con 0x564ba25d87b0 2024-05-08T18:06:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.209+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:119 4.1c 4:3a9e36b8:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba2636410 con 0x564ba25d87b0 2024-05-08T18:06:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.209+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 31 ==== osd_op_reply(114 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.2 [call out=85b] v0'0 uv11 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.209+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:120 4.35 4:ace44e4b:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba262d5b0 con 0x564ba25d3dc0 2024-05-08T18:06:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.209+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:121 4.13 4:ca75e525:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba263be20 con 0x7f43d000f070 2024-05-08T18:06:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.209+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:122 4.3a 4:5f16430d:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba263e990 con 0x564ba25dd1c0 2024-05-08T18:06:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.209+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 30 ==== osd_op_reply(115 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.3 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.209+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 32 ==== osd_op_reply(118 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.6 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.209+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 21 ==== osd_op_reply(113 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.1 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 33 ==== osd_op_reply(119 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.7 [call out=85b] v0'0 uv9 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 31 ==== osd_op_reply(122 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.10 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 33 ==== osd_op_reply(117 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.5 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 34 ==== osd_op_reply(120 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.8 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 22 ==== osd_op_reply(121 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.9 [call out=85b] v0'0 uv4 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 35 ==== osd_op_reply(111 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.4 [stat,call,call] v29'5 uv5 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f42ba7fc640 20 handle_completion(): completion ok for obj=15f6f8dc036519d7fe15b39338f6e5db 2024-05-08T18:06:50.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 36 ==== osd_op_reply(116 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.4 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43ddfa2800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-05-08T18:06:50.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43ddfa2800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-05-08T18:06:50.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43ddfa2800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-05-08T18:06:50.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43ddfa2800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-05-08T18:06:50.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43ddfa2800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7])"; uploads.size()=0; is_truncated=0 2024-05-08T18:06:50.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:123 4.26 4:656c7161:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba2635dc0 con 0x564ba25d87b0 2024-05-08T18:06:50.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:124 4.13 4:caae27a5:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba262daa0 con 0x7f43d000f070 2024-05-08T18:06:50.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:125 4.21 4:87933c90:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba262d100 con 0x564ba25d87b0 2024-05-08T18:06:50.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:126 4.2e 4:7408f608:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba263e310 con 0x564ba25dd1c0 2024-05-08T18:06:50.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:127 4.20 4:044ed8a5:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba262d100 con 0x564ba25d3dc0 2024-05-08T18:06:50.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:128 4.2b 4:d6c558e4:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba262daa0 con 0x564ba25d3dc0 2024-05-08T18:06:50.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:129 4.37 4:ee400b20:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba2633150 con 0x564ba25d87b0 2024-05-08T18:06:50.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:130 4.1c 4:3a9e36b8:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba263be20 con 0x564ba25d87b0 2024-05-08T18:06:50.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:131 4.35 4:ace44e4b:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba263e310 con 0x564ba25d3dc0 2024-05-08T18:06:50.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:132 4.13 4:ca75e525:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba2633150 con 0x7f43d000f070 2024-05-08T18:06:50.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.210+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:133 4.3a 4:5f16430d:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba263be20 con 0x564ba25dd1c0 2024-05-08T18:06:50.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.211+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 34 ==== osd_op_reply(125 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.2 [call out=85b] v0'0 uv11 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.211+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 35 ==== osd_op_reply(123 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.0 [call out=85b] v0'0 uv11 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.211+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 36 ==== osd_op_reply(129 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.6 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.211+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 23 ==== osd_op_reply(124 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.1 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.211+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 37 ==== osd_op_reply(130 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.7 [call out=85b] v0'0 uv9 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.211+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 32 ==== osd_op_reply(126 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.3 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.211+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 37 ==== osd_op_reply(127 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.4 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.211+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 33 ==== osd_op_reply(133 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.10 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.211+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 38 ==== osd_op_reply(128 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.5 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.211+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 39 ==== osd_op_reply(131 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.8 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.211+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 24 ==== osd_op_reply(132 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.9 [call out=85b] v0'0 uv4 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.211+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:134 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba263e8d0 con 0x7f43d000f070 2024-05-08T18:06:50.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.212+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 25 ==== osd_op_reply(134 foo.buckets [call] v29'48 uv48 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.212+0000 7f43ddfa2800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-05-08T18:06:50.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.212+0000 7f43ddfa2800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.212+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:135 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba263e8d0 con 0x7f43d000f070 2024-05-08T18:06:50.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.213+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 26 ==== osd_op_reply(135 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv42 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.213+0000 7f43ddfa2800 20 rados_obj.operate() r=0 bl.length=165 2024-05-08T18:06:50.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.213+0000 7f43ddfa2800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-05-08T18:06:50.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.213+0000 7f43ddfa2800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-05-08T18:06:50.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.213+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:136 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba263e8d0 con 0x7f43d000f070 2024-05-08T18:06:50.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.214+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 27 ==== osd_op_reply(136 myfoo [call,call,delete] v29'49 uv49 ondisk = 0) ==== 233+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.214+0000 7f43ddfa2800 10 removing default.rgw.meta+root+myfoo from cache 2024-05-08T18:06:50.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.214+0000 7f43ddfa2800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-05-08T18:06:50.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.214+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:137 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94882846946688 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba263e8d0 con 0x564ba25d87b0 2024-05-08T18:06:50.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.214+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 38 ==== watch-notify(notify (1) cookie 94882846622624 notify 124554051627 ret 0) ==== 191+0+0 (crc 0 0 0) 0x564ba25f0e00 con 0x564ba25d87b0 2024-05-08T18:06:50.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.214+0000 7f42baffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051627 cookie 94882846622624 notifier 4774 bl.length()=149 2024-05-08T18:06:50.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.214+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 39 ==== osd_op_reply(137 notify.6 [notify cookie 94882846946688 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.214+0000 7f42baffd640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:138 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f42a0003d60 con 0x564ba25d87b0 2024-05-08T18:06:50.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.214+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 40 ==== osd_op_reply(138 notify.6 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.214+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 41 ==== watch-notify(notify_complete (2) cookie 94882846946688 notify 124554051627 ret 0) ==== 42+0+48 (crc 0 0 0) 0x564ba240eb30 con 0x564ba25d87b0 2024-05-08T18:06:50.218 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.214+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:139 8.11 8:8dcfa685:root::.bucket.meta.myfoo%3a1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba263e8d0 con 0x564ba25dd1c0 2024-05-08T18:06:50.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.215+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 34 ==== osd_op_reply(139 .bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7 [call,call,delete] v29'2 uv2 ondisk = 0) ==== 290+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.215+0000 7f43ddfa2800 10 removing default.rgw.meta+root+.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7 from cache 2024-05-08T18:06:50.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.215+0000 7f43ddfa2800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7, ofs0, ns] 2024-05-08T18:06:50.225 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:140 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94882846946688 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba263e8d0 con 0x564ba25d87b0 2024-05-08T18:06:50.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 42 ==== watch-notify(notify (1) cookie 94882846622624 notify 124554051628 ret 0) ==== 248+0+0 (crc 0 0 0) 0x564ba240eb30 con 0x564ba25d87b0 2024-05-08T18:06:50.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f42baffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051628 cookie 94882846622624 notifier 4774 bl.length()=206 2024-05-08T18:06:50.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 43 ==== osd_op_reply(140 notify.6 [notify cookie 94882846946688 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f42baffd640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:141 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f42a0003d60 con 0x564ba25d87b0 2024-05-08T18:06:50.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 44 ==== osd_op_reply(141 notify.6 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 45 ==== watch-notify(notify_complete (2) cookie 94882846946688 notify 124554051628 ret 0) ==== 42+0+48 (crc 0 0 0) 0x564ba2596b40 con 0x564ba25d87b0 2024-05-08T18:06:50.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:142 4.26 4:656c7161:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba263e8d0 con 0x564ba25d87b0 2024-05-08T18:06:50.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:143 4.13 4:caae27a5:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba2635cb0 con 0x7f43d000f070 2024-05-08T18:06:50.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:144 4.21 4:87933c90:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba26279f0 con 0x564ba25d87b0 2024-05-08T18:06:50.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:145 4.2e 4:7408f608:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba263e8d0 con 0x564ba25dd1c0 2024-05-08T18:06:50.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:146 4.20 4:044ed8a5:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba26279f0 con 0x564ba25d3dc0 2024-05-08T18:06:50.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:147 4.2b 4:d6c558e4:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba2634440 con 0x564ba25d3dc0 2024-05-08T18:06:50.226 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:148 4.37 4:ee400b20:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba2621640 con 0x564ba25d87b0 2024-05-08T18:06:50.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:149 4.1c 4:3a9e36b8:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba263e8d0 con 0x564ba25d87b0 2024-05-08T18:06:50.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:150 4.35 4:ace44e4b:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba26279f0 con 0x564ba25d3dc0 2024-05-08T18:06:50.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.216+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:151 4.13 4:ca75e525:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba2634440 con 0x7f43d000f070 2024-05-08T18:06:50.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.217+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:152 4.3a 4:5f16430d:::.dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba2621640 con 0x564ba25dd1c0 2024-05-08T18:06:50.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.217+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 46 ==== osd_op_reply(142 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.0 [delete] v29'22 uv22 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.218+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 47 ==== osd_op_reply(149 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.7 [delete] v29'10 uv10 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.218+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 35 ==== osd_op_reply(145 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.3 [delete] v29'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.218+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 40 ==== osd_op_reply(146 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.4 [delete] v29'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.218+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 41 ==== osd_op_reply(150 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.8 [delete] v29'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.218+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 42 ==== osd_op_reply(147 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.5 [delete] v29'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.219+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 28 ==== osd_op_reply(143 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.1 [delete] v29'7 uv7 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.219+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 48 ==== osd_op_reply(148 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.6 [delete] v29'8 uv8 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.227 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.219+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 49 ==== osd_op_reply(144 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.2 [delete] v29'14 uv14 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.219+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 36 ==== osd_op_reply(152 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.10 [delete] v29'6 uv6 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f43d0037040 con 0x564ba25dd1c0 2024-05-08T18:06:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.219+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 29 ==== osd_op_reply(151 .dir.1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7.9 [delete] v29'8 uv8 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.219+0000 7f43ddfa2800 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7 : miss 2024-05-08T18:06:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.219+0000 7f43ddfa2800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.219+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:153 6.5 6:a11ceaee:::pubsub..bucket.myfoo%2f1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba2634440 con 0x7f43d000f070 2024-05-08T18:06:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.219+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 30 ==== osd_op_reply(153 pubsub..bucket.myfoo/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 208+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.219+0000 7f43ddfa2800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.219+0000 7f43ddfa2800 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7 info.flags=0x0 2024-05-08T18:06:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.219+0000 7f43ddfa2800 10 adding default.rgw.log++pubsub..bucket.myfoo/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7 to cache LRU end 2024-05-08T18:06:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.219+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:154 6.5 6:a11ceaee:::pubsub..bucket.myfoo%2f1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba26279f0 con 0x7f43d000f070 2024-05-08T18:06:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.220+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 31 ==== osd_op_reply(154 pubsub..bucket.myfoo/1c041ffd-e40a-486a-88e5-3547b85128ea.4221.7 [delete] v29'16 uv15 ondisk = -2 ((2) No such file or directory)) ==== 208+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.220+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:155 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba26279f0 con 0x7f43d000f070 2024-05-08T18:06:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.221+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 32 ==== osd_op_reply(155 foo.buckets [call] v29'50 uv50 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.221+0000 7f43ddfa2800 10 removing key index: 9te6NH5mcdcq0Tc5i8i1 2024-05-08T18:06:50.228 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.221+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:156 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba26274f0 con 0x564ba25d87b0 2024-05-08T18:06:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.223+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 50 ==== osd_op_reply(156 9te6NH5mcdcq0Tc5i8i1 [delete] v29'4 uv4 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.223+0000 7f43ddfa2800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2024-05-08T18:06:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.223+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:157 7.e 7:7759931f:::notify.3:head [notify cookie 94882846898672 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba2622560 con 0x7f43d000f070 2024-05-08T18:06:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.223+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 33 ==== watch-notify(notify (1) cookie 94882846621568 notify 124554051603 ret 0) ==== 212+0+0 (crc 0 0 0) 0x564ba25f31d0 con 0x7f43d000f070 2024-05-08T18:06:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.223+0000 7f42baffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051603 cookie 94882846621568 notifier 4774 bl.length()=170 2024-05-08T18:06:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.223+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 34 ==== osd_op_reply(157 notify.3 [notify cookie 94882846898672 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.223+0000 7f42baffd640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:158 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f42a0003d60 con 0x7f43d000f070 2024-05-08T18:06:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.223+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 35 ==== osd_op_reply(158 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.223+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 36 ==== watch-notify(notify_complete (2) cookie 94882846898672 notify 124554051603 ret 0) ==== 42+0+48 (crc 0 0 0) 0x564ba25f09d0 con 0x7f43d000f070 2024-05-08T18:06:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.223+0000 7f43ddfa2800 10 removing email index: foo@foo.com 2024-05-08T18:06:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.223+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:159 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba26274f0 con 0x564ba25d3dc0 2024-05-08T18:06:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.224+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 43 ==== osd_op_reply(159 foo@foo.com [delete] v29'4 uv4 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.224+0000 7f43ddfa2800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2024-05-08T18:06:50.229 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.224+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:160 7.e 7:7759931f:::notify.3:head [notify cookie 94882846877024 in=174b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba26274f0 con 0x7f43d000f070 2024-05-08T18:06:50.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.225+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 37 ==== watch-notify(notify (1) cookie 94882846621568 notify 124554051604 ret 0) ==== 204+0+0 (crc 0 0 0) 0x564ba25f09d0 con 0x7f43d000f070 2024-05-08T18:06:50.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.225+0000 7f42ba7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051604 cookie 94882846621568 notifier 4774 bl.length()=162 2024-05-08T18:06:50.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.225+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 38 ==== osd_op_reply(160 notify.3 [notify cookie 94882846877024 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.225+0000 7f42ba7fc640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:161 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f42100032c0 con 0x7f43d000f070 2024-05-08T18:06:50.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.225+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 39 ==== watch-notify(notify_complete (2) cookie 94882846877024 notify 124554051604 ret 0) ==== 42+0+48 (crc 0 0 0) 0x564ba25f2d90 con 0x7f43d000f070 2024-05-08T18:06:50.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.225+0000 7f43ddfa2800 10 removing user buckets index 2024-05-08T18:06:50.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.225+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:162 8.9 8:96d98480:users.uid::foo.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba2633580 con 0x7f43d000f070 2024-05-08T18:06:50.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.225+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 40 ==== osd_op_reply(161 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.226+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 41 ==== osd_op_reply(162 foo.buckets [delete] v29'51 uv51 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.226+0000 7f43ddfa2800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:foo.buckets, ofs0, ns] 2024-05-08T18:06:50.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.226+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:163 7.3 7:c609908c:::notify.5:head [notify cookie 94882846877024 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba2634c00 con 0x564ba25d87b0 2024-05-08T18:06:50.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.226+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 51 ==== watch-notify(notify (1) cookie 94882846628992 notify 124554051629 ret 0) ==== 202+0+0 (crc 0 0 0) 0x564ba2596b40 con 0x564ba25d87b0 2024-05-08T18:06:50.230 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.226+0000 7f42baffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051629 cookie 94882846628992 notifier 4774 bl.length()=160 2024-05-08T18:06:50.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.226+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 52 ==== osd_op_reply(163 notify.5 [notify cookie 94882846877024 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.226+0000 7f42baffd640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:164 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f42a0003d60 con 0x564ba25d87b0 2024-05-08T18:06:50.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.227+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 53 ==== osd_op_reply(164 notify.5 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.227+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 54 ==== watch-notify(notify_complete (2) cookie 94882846877024 notify 124554051629 ret 0) ==== 42+0+48 (crc 0 0 0) 0x564ba2593e90 con 0x564ba25d87b0 2024-05-08T18:06:50.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.227+0000 7f43ddfa2800 10 removing user index: foo 2024-05-08T18:06:50.231 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.227+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:165 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba2621b40 con 0x564ba25d3dc0 2024-05-08T18:06:50.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.228+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 44 ==== osd_op_reply(165 foo [call,call,delete] v29'19 uv19 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.228+0000 7f43ddfa2800 10 removing default.rgw.meta+users.uid+foo from cache 2024-05-08T18:06:50.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.228+0000 7f43ddfa2800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-05-08T18:06:50.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.228+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:166 7.d 7:b4812045:::notify.4:head [notify cookie 94882846877024 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ba2634c00 con 0x564ba25d3dc0 2024-05-08T18:06:50.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.228+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 45 ==== watch-notify(notify (1) cookie 94882846625120 notify 124554051614 ret 0) ==== 194+0+0 (crc 0 0 0) 0x564ba25f2d90 con 0x564ba25d3dc0 2024-05-08T18:06:50.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.228+0000 7f42baffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051614 cookie 94882846625120 notifier 4774 bl.length()=152 2024-05-08T18:06:50.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.228+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 46 ==== osd_op_reply(166 notify.4 [notify cookie 94882846877024 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.228+0000 7f42baffd640 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:167 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f42a0003d60 con 0x564ba25d3dc0 2024-05-08T18:06:50.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.229+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 47 ==== watch-notify(notify_complete (2) cookie 94882846877024 notify 124554051614 ret 0) ==== 42+0+48 (crc 0 0 0) 0x564ba25f2bc0 con 0x564ba25d3dc0 2024-05-08T18:06:50.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.229+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 48 ==== osd_op_reply(167 notify.4 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.229+0000 7f4291fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:50.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.230+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:168 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94882846635376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f4170 con 0x564ba25d3dc0 2024-05-08T18:06:50.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.231+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 49 ==== osd_op_reply(168 data_loggenerations_metadata [watch unwatch cookie 94882846635376] v29'236 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.231+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:169 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94882846611904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f4170 con 0x7f43d000f070 2024-05-08T18:06:50.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.231+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 42 ==== osd_op_reply(169 notify.0 [watch unwatch cookie 94882846611904] v29'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.232+0000 7f43ddfa2800 20 remove_watcher() i=0 2024-05-08T18:06:50.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.232+0000 7f43ddfa2800 2 removed watcher, disabling cache 2024-05-08T18:06:50.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.232+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:170 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94882846615232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f4170 con 0x564ba25d3dc0 2024-05-08T18:06:50.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.233+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 50 ==== osd_op_reply(170 notify.1 [watch unwatch cookie 94882846615232] v29'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.233+0000 7f43ddfa2800 20 remove_watcher() i=1 2024-05-08T18:06:50.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.233+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:171 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94882846616288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f4170 con 0x564ba25dd1c0 2024-05-08T18:06:50.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.234+0000 7f43db2a0640 1 -- 172.21.15.50:0/2595754535 <== osd.3 v2:172.21.15.50:6800/3130492744 37 ==== osd_op_reply(171 notify.2 [watch unwatch cookie 94882846616288] v29'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d0057040 con 0x564ba25dd1c0 2024-05-08T18:06:50.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.234+0000 7f43ddfa2800 20 remove_watcher() i=2 2024-05-08T18:06:50.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.234+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:172 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94882846621568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f4170 con 0x7f43d000f070 2024-05-08T18:06:50.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.235+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.2 v2:172.21.15.50:6811/3039646617 43 ==== osd_op_reply(172 notify.3 [watch unwatch cookie 94882846621568] v29'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x7f43d000f070 2024-05-08T18:06:50.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.235+0000 7f43ddfa2800 20 remove_watcher() i=3 2024-05-08T18:06:50.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.235+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:173 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94882846625120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f4170 con 0x564ba25d3dc0 2024-05-08T18:06:50.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.236+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 <== osd.1 v2:172.21.15.50:6808/2141923372 51 ==== osd_op_reply(173 notify.4 [watch unwatch cookie 94882846625120] v29'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43d4032040 con 0x564ba25d3dc0 2024-05-08T18:06:50.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.236+0000 7f43ddfa2800 20 remove_watcher() i=4 2024-05-08T18:06:50.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.236+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:174 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94882846628992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f4170 con 0x564ba25d87b0 2024-05-08T18:06:50.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.237+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 55 ==== osd_op_reply(174 notify.5 [watch unwatch cookie 94882846628992] v29'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.237+0000 7f43ddfa2800 20 remove_watcher() i=5 2024-05-08T18:06:50.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.237+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:175 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94882846622624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f4170 con 0x564ba25d87b0 2024-05-08T18:06:50.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.238+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 56 ==== osd_op_reply(175 notify.6 [watch unwatch cookie 94882846622624] v29'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.238+0000 7f43ddfa2800 20 remove_watcher() i=6 2024-05-08T18:06:50.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.238+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:176 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94882846633872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ba25f4170 con 0x564ba25d87b0 2024-05-08T18:06:50.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.239+0000 7f43dbaa1640 1 -- 172.21.15.50:0/2595754535 <== osd.0 v2:172.21.15.46:6800/337143975 57 ==== osd_op_reply(176 notify.7 [watch unwatch cookie 94882846633872] v29'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f43cc0e0390 con 0x564ba25d87b0 2024-05-08T18:06:50.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.239+0000 7f43ddfa2800 20 remove_watcher() i=7 2024-05-08T18:06:50.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.239+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ba25d87b0 msgr2=0x564ba25daba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.239+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ba25d87b0 0x564ba25daba0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.239+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ba25d3dc0 msgr2=0x564ba25d6280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.239+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ba25d3dc0 0x564ba25d6280 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.239+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f43d000f070 msgr2=0x7f43d002e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.239+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f43d000f070 0x7f43d002e910 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.239+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ba25dd1c0 msgr2=0x564ba25df680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.239+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ba25dd1c0 0x564ba25df680 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.239+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f42a403bb50 msgr2=0x7f42a403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.239+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f42a403bb50 0x7f42a403e010 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x564ba25a3fc0 tx=0x7f43d0010000 comp rx=0 tx=0).stop 2024-05-08T18:06:50.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba25c1310 msgr2=0x564ba25c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/2595754535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba25c1310 0x564ba25c8630 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f43cc05bfd0 tx=0x7f43cc06ee70 comp rx=0 tx=0).stop 2024-05-08T18:06:50.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 reap_dead start 2024-05-08T18:06:50.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43dc2a2640 1 -- 172.21.15.50:0/2595754535 reap_dead start 2024-05-08T18:06:50.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 shutdown_connections 2024-05-08T18:06:50.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 >> 172.21.15.50:0/2595754535 conn(0x564ba25bd590 msgr2=0x564ba25babf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 shutdown_connections 2024-05-08T18:06:50.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2595754535 wait complete. 2024-05-08T18:06:50.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2383886032 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f42b403bba0 msgr2=0x7f42b403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/2383886032 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f42b403bba0 0x7f42b403e060 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x564ba2598030 tx=0x7f43d000f000 comp rx=0 tx=0).stop 2024-05-08T18:06:50.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2383886032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba2410180 msgr2=0x564ba240e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/2383886032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba2410180 0x564ba240e5f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f43cc006d00 tx=0x7f43cc0064c0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2383886032 shutdown_connections 2024-05-08T18:06:50.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/2383886032 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f42b403bba0 0x7f42b403e060 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/2383886032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba2410180 0x564ba240e5f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2383886032 >> 172.21.15.50:0/2383886032 conn(0x564ba25a5fd0 msgr2=0x564ba25a63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2383886032 shutdown_connections 2024-05-08T18:06:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 -- 172.21.15.50:0/2383886032 wait complete. 2024-05-08T18:06:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 -- 172.21.15.50:0/1880189890 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ba259d050 msgr2=0x564ba259f4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ba259d050 0x564ba259f4f0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 -- 172.21.15.50:0/1880189890 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ba249b8d0 msgr2=0x564ba259a900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ba249b8d0 0x564ba259a900 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 -- 172.21.15.50:0/1880189890 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ba25a10e0 msgr2=0x564ba25a34d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.240+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ba25a10e0 0x564ba25a34d0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.241+0000 7f43ddfa2800 1 -- 172.21.15.50:0/1880189890 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f42c403bb50 msgr2=0x7f42c403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.241+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f42c403bb50 0x7f42c403e010 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f43d0001e30 tx=0x7f43d0001390 comp rx=0 tx=0).stop 2024-05-08T18:06:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.241+0000 7f43ddfa2800 1 -- 172.21.15.50:0/1880189890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 msgr2=0x564ba25936c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.241+0000 7f43ddfa2800 1 --2- 172.21.15.50:0/1880189890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ba249a710 0x564ba25936c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0524f0 tx=0x7f43cc003020 comp rx=0 tx=0).stop 2024-05-08T18:06:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.241+0000 7f43dc2a2640 1 -- 172.21.15.50:0/1880189890 reap_dead start 2024-05-08T18:06:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.241+0000 7f43ddfa2800 1 -- 172.21.15.50:0/1880189890 shutdown_connections 2024-05-08T18:06:50.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.241+0000 7f43ddfa2800 1 -- 172.21.15.50:0/1880189890 >> 172.21.15.50:0/1880189890 conn(0x564ba2495c90 msgr2=0x564ba2585d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.241+0000 7f43ddfa2800 1 -- 172.21.15.50:0/1880189890 shutdown_connections 2024-05-08T18:06:50.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.241+0000 7f43ddfa2800 1 -- 172.21.15.50:0/1880189890 wait complete. 2024-05-08T18:06:50.248 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-05-08T18:06:50.248 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-05-08T18:06:50.248 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-05-08T18:06:50.294 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:50.294 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:50.295 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.297+0000 7f235ed27800 1 Processor -- start 2024-05-08T18:06:50.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.297+0000 7f235ed27800 1 -- start start 2024-05-08T18:06:50.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.297+0000 7f235ed27800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5573ba844ad0 0x5573ba84d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.297+0000 7f235ed27800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5573ba7b8010 con 0x5573ba842f90 2024-05-08T18:06:50.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.297+0000 7f235ed27800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5573ba6e4e20 con 0x5573ba844ad0 2024-05-08T18:06:50.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.297+0000 7f23522e3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5573ba844ad0 0x5573ba84d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.297+0000 7f23522e3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5573ba844ad0 0x5573ba84d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45788/0 (socket says 172.21.15.50:45788) 2024-05-08T18:06:50.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.297+0000 7f23522e3640 1 -- 172.21.15.50:0/3224695266 learned_addr learned my addr 172.21.15.50:0/3224695266 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.297+0000 7f235aae4640 1 --1- 172.21.15.50:0/3224695266 >> v1:172.21.15.46:6789/0 conn(0x5573ba842f90 0x5573ba8433b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50734/0 (socket says 172.21.15.50:50734) 2024-05-08T18:06:50.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.297+0000 7f23522e3640 1 -- 172.21.15.50:0/3224695266 >> v1:172.21.15.46:6789/0 conn(0x5573ba842f90 legacy=0x5573ba8433b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.296 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.298+0000 7f23522e3640 1 -- 172.21.15.50:0/3224695266 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573ba84eb20 con 0x5573ba844ad0 2024-05-08T18:06:50.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.299+0000 7f23522e3640 1 --2- 172.21.15.50:0/3224695266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba844ad0 0x5573ba84d5d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f234400a520 tx=0x7f234401ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5642bf65173d6146 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.297 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.300+0000 7f235a2e3640 1 -- 172.21.15.50:0/3224695266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2344057070 con 0x5573ba844ad0 2024-05-08T18:06:50.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.300+0000 7f235a2e3640 1 -- 172.21.15.50:0/3224695266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234404acd0 con 0x5573ba844ad0 2024-05-08T18:06:50.298 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.300+0000 7f235a2e3640 1 -- 172.21.15.50:0/3224695266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2344052440 con 0x5573ba844ad0 2024-05-08T18:06:50.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.300+0000 7f235ed27800 1 -- 172.21.15.50:0/3224695266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba844ad0 msgr2=0x5573ba84d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.300+0000 7f235ed27800 1 --2- 172.21.15.50:0/3224695266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba844ad0 0x5573ba84d5d0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f234400a520 tx=0x7f234401ce40 comp rx=0 tx=0).stop 2024-05-08T18:06:50.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.300+0000 7f235ed27800 1 -- 172.21.15.50:0/3224695266 shutdown_connections 2024-05-08T18:06:50.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.300+0000 7f235ed27800 1 --2- 172.21.15.50:0/3224695266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba844ad0 0x5573ba84d5d0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.300+0000 7f235ed27800 1 -- 172.21.15.50:0/3224695266 >> 172.21.15.50:0/3224695266 conn(0x5573ba83ead0 msgr2=0x5573ba840f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.300+0000 7f235ed27800 1 -- 172.21.15.50:0/3224695266 shutdown_connections 2024-05-08T18:06:50.299 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.300+0000 7f235ed27800 1 -- 172.21.15.50:0/3224695266 wait complete. 2024-05-08T18:06:50.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.315+0000 7f235ed27800 1 Processor -- start 2024-05-08T18:06:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.316+0000 7f235ed27800 1 -- start start 2024-05-08T18:06:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.316+0000 7f235ed27800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba842f90 0x5573ba937eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.316+0000 7f235ed27800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5573ba78fa10 con 0x5573ba842f90 2024-05-08T18:06:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.316+0000 7f235aae4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba842f90 0x5573ba937eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.316+0000 7f235aae4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba842f90 0x5573ba937eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45796/0 (socket says 172.21.15.50:45796) 2024-05-08T18:06:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.316+0000 7f235aae4640 1 -- 172.21.15.50:0/2533842611 learned_addr learned my addr 172.21.15.50:0/2533842611 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.316+0000 7f235aae4640 1 -- 172.21.15.50:0/2533842611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573ba938480 con 0x5573ba842f90 2024-05-08T18:06:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.316+0000 7f235aae4640 1 --2- 172.21.15.50:0/2533842611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba842f90 0x5573ba937eb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f234c0065f0 tx=0x7f234c006ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf0eadc31d53420 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.317+0000 7f2248e09640 1 -- 172.21.15.50:0/2533842611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f234c003450 con 0x5573ba842f90 2024-05-08T18:06:50.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.317+0000 7f2248e09640 1 -- 172.21.15.50:0/2533842611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234c0035f0 con 0x5573ba842f90 2024-05-08T18:06:50.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.317+0000 7f2248e09640 1 -- 172.21.15.50:0/2533842611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f234c003910 con 0x5573ba842f90 2024-05-08T18:06:50.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.317+0000 7f235ed27800 1 -- 172.21.15.50:0/2533842611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba842f90 msgr2=0x5573ba937eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.317+0000 7f235ed27800 1 --2- 172.21.15.50:0/2533842611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba842f90 0x5573ba937eb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f234c0065f0 tx=0x7f234c006ab0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.317+0000 7f235ed27800 1 -- 172.21.15.50:0/2533842611 shutdown_connections 2024-05-08T18:06:50.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.317+0000 7f235ed27800 1 --2- 172.21.15.50:0/2533842611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba842f90 0x5573ba937eb0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.317+0000 7f235ed27800 1 -- 172.21.15.50:0/2533842611 >> 172.21.15.50:0/2533842611 conn(0x5573ba83ead0 msgr2=0x5573ba9337d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.317+0000 7f235ed27800 1 -- 172.21.15.50:0/2533842611 shutdown_connections 2024-05-08T18:06:50.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.317+0000 7f235ed27800 1 -- 172.21.15.50:0/2533842611 wait complete. 2024-05-08T18:06:50.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.318+0000 7f235ed27800 1 Processor -- start 2024-05-08T18:06:50.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.318+0000 7f235ed27800 1 -- start start 2024-05-08T18:06:50.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.318+0000 7f235ed27800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba842f90 0x5573ba93fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.318+0000 7f235ed27800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5573ba84dde0 con 0x5573ba842f90 2024-05-08T18:06:50.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.318+0000 7f235aae4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba842f90 0x5573ba93fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.318+0000 7f235aae4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba842f90 0x5573ba93fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45808/0 (socket says 172.21.15.50:45808) 2024-05-08T18:06:50.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.318+0000 7f235aae4640 1 -- 172.21.15.50:0/4171545640 learned_addr learned my addr 172.21.15.50:0/4171545640 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.318+0000 7f235aae4640 1 -- 172.21.15.50:0/4171545640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573ba93ffd0 con 0x5573ba842f90 2024-05-08T18:06:50.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.318+0000 7f235aae4640 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba842f90 0x5573ba93fa90 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f234c001330 tx=0x7f234c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.319+0000 7f2247606640 1 -- 172.21.15.50:0/4171545640 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f234c022020 con 0x5573ba842f90 2024-05-08T18:06:50.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.319+0000 7f2247606640 1 -- 172.21.15.50:0/4171545640 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234c001a80 con 0x5573ba842f90 2024-05-08T18:06:50.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.319+0000 7f235ed27800 1 -- 172.21.15.50:0/4171545640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5573ba940260 con 0x5573ba842f90 2024-05-08T18:06:50.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.319+0000 7f2247606640 1 -- 172.21.15.50:0/4171545640 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f234c014b40 con 0x5573ba842f90 2024-05-08T18:06:50.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.319+0000 7f235ed27800 1 -- 172.21.15.50:0/4171545640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573ba942ec0 con 0x5573ba842f90 2024-05-08T18:06:50.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.319+0000 7f2247606640 1 -- 172.21.15.50:0/4171545640 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f234c025020 con 0x5573ba842f90 2024-05-08T18:06:50.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.320+0000 7f2247606640 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f224003bba0 0x7f224003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.320+0000 7f2247606640 1 -- 172.21.15.50:0/4171545640 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f234c018030 con 0x5573ba842f90 2024-05-08T18:06:50.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.320+0000 7f23522e3640 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f224003bba0 0x7f224003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.320+0000 7f235ed27800 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5573ba844ad0 0x5573ba946c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.320+0000 7f235ed27800 1 -- 172.21.15.50:0/4171545640 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba947210 con 0x5573ba844ad0 2024-05-08T18:06:50.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.320+0000 7f235b2e5640 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5573ba844ad0 0x5573ba946c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.320+0000 7f23522e3640 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f224003bba0 0x7f224003e060 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2344054000 tx=0x7f234404e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.320+0000 7f235b2e5640 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5573ba844ad0 0x5573ba946c80 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.321+0000 7f235b2e5640 1 -- 172.21.15.50:0/4171545640 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f23540083e0 con 0x5573ba844ad0 2024-05-08T18:06:50.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.321+0000 7f235ed27800 1 -- 172.21.15.50:0/4171545640 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba944230 con 0x5573ba844ad0 2024-05-08T18:06:50.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.321+0000 7f235b2e5640 1 -- 172.21.15.50:0/4171545640 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f23540083e0 con 0x5573ba844ad0 2024-05-08T18:06:50.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.321+0000 7f235ed27800 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5573ba949540 0x5573ba94b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.321+0000 7f235ed27800 1 -- 172.21.15.50:0/4171545640 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba94c020 con 0x5573ba949540 2024-05-08T18:06:50.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.321+0000 7f235aae4640 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5573ba949540 0x5573ba94b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.322+0000 7f235aae4640 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5573ba949540 0x5573ba94b930 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.322+0000 7f235aae4640 1 -- 172.21.15.50:0/4171545640 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f234c01b070 con 0x5573ba949540 2024-05-08T18:06:50.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.322+0000 7f235ed27800 1 -- 172.21.15.50:0/4171545640 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba9483e0 con 0x5573ba844ad0 2024-05-08T18:06:50.338 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.323+0000 7f235b2e5640 1 -- 172.21.15.50:0/4171545640 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f23540083e0 con 0x5573ba844ad0 2024-05-08T18:06:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.323+0000 7f235ed27800 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5573ba94d480 0x5573ba94f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.323+0000 7f235ed27800 1 -- 172.21.15.50:0/4171545640 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba94ffb0 con 0x5573ba94d480 2024-05-08T18:06:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.323+0000 7f23522e3640 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5573ba94d480 0x5573ba94f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.323+0000 7f23522e3640 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5573ba94d480 0x5573ba94f870 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.323+0000 7f23522e3640 1 -- 172.21.15.50:0/4171545640 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f234405a040 con 0x5573ba94d480 2024-05-08T18:06:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.324+0000 7f235ed27800 1 Processor -- start 2024-05-08T18:06:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.324+0000 7f235ed27800 1 -- start start 2024-05-08T18:06:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.324+0000 7f235ed27800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba95e2e0 0x5573ba95e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.324+0000 7f235ed27800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5573ba736330 con 0x5573ba95e2e0 2024-05-08T18:06:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.324+0000 7f235aae4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba95e2e0 0x5573ba95e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.324+0000 7f235aae4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba95e2e0 0x5573ba95e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45824/0 (socket says 172.21.15.50:45824) 2024-05-08T18:06:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.324+0000 7f235aae4640 1 -- 172.21.15.50:0/3505141570 learned_addr learned my addr 172.21.15.50:0/3505141570 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.324+0000 7f235aae4640 1 -- 172.21.15.50:0/3505141570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573ba95ec40 con 0x5573ba95e2e0 2024-05-08T18:06:50.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.324+0000 7f235aae4640 1 --2- 172.21.15.50:0/3505141570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba95e2e0 0x5573ba95e700 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f234c006af0 tx=0x7f234c043490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.324+0000 7f2244e01640 1 -- 172.21.15.50:0/3505141570 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f234c010700 con 0x5573ba95e2e0 2024-05-08T18:06:50.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.325+0000 7f2244e01640 1 -- 172.21.15.50:0/3505141570 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234c026780 con 0x5573ba95e2e0 2024-05-08T18:06:50.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.325+0000 7f2244e01640 1 -- 172.21.15.50:0/3505141570 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f234c01e3f0 con 0x5573ba95e2e0 2024-05-08T18:06:50.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.325+0000 7f235ed27800 1 -- 172.21.15.50:0/3505141570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5573ba95f6b0 con 0x5573ba95e2e0 2024-05-08T18:06:50.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.325+0000 7f235ed27800 1 -- 172.21.15.50:0/3505141570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573ba95fbb0 con 0x5573ba95e2e0 2024-05-08T18:06:50.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.325+0000 7f2244e01640 1 -- 172.21.15.50:0/3505141570 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f234c010070 con 0x5573ba95e2e0 2024-05-08T18:06:50.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.325+0000 7f2244e01640 1 --2- 172.21.15.50:0/3505141570 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f223403bba0 0x7f223403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.326+0000 7f2244e01640 1 -- 172.21.15.50:0/3505141570 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f234c03b980 con 0x5573ba95e2e0 2024-05-08T18:06:50.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.326+0000 7f23522e3640 1 --2- 172.21.15.50:0/3505141570 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f223403bba0 0x7f223403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.326+0000 7f235ed27800 1 Processor -- start 2024-05-08T18:06:50.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.326+0000 7f235ed27800 1 -- start start 2024-05-08T18:06:50.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.326+0000 7f23522e3640 1 --2- 172.21.15.50:0/3505141570 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f223403bba0 0x7f223403e060 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x5573ba9476e0 tx=0x7f2344005ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.326+0000 7f235ed27800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba96b670 0x5573ba96ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.326+0000 7f235ed27800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5573ba938800 con 0x5573ba96b670 2024-05-08T18:06:50.340 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.326+0000 7f235aae4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba96b670 0x5573ba96ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.326+0000 7f235aae4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba96b670 0x5573ba96ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45840/0 (socket says 172.21.15.50:45840) 2024-05-08T18:06:50.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.326+0000 7f235aae4640 1 -- 172.21.15.50:0/2254465609 learned_addr learned my addr 172.21.15.50:0/2254465609 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.327+0000 7f235aae4640 1 -- 172.21.15.50:0/2254465609 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573ba96c060 con 0x5573ba96b670 2024-05-08T18:06:50.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.327+0000 7f235aae4640 1 --2- 172.21.15.50:0/2254465609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba96b670 0x5573ba96ba90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f234c00b740 tx=0x7f234c051bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96549f6d420d83e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.327+0000 7f223e7fc640 1 -- 172.21.15.50:0/2254465609 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f234c04a4e0 con 0x5573ba96b670 2024-05-08T18:06:50.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.327+0000 7f223e7fc640 1 -- 172.21.15.50:0/2254465609 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234c00b7f0 con 0x5573ba96b670 2024-05-08T18:06:50.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.327+0000 7f223e7fc640 1 -- 172.21.15.50:0/2254465609 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f234c049fc0 con 0x5573ba96b670 2024-05-08T18:06:50.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.327+0000 7f235ed27800 1 -- 172.21.15.50:0/2254465609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba96b670 msgr2=0x5573ba96ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.327+0000 7f235ed27800 1 --2- 172.21.15.50:0/2254465609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba96b670 0x5573ba96ba90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f234c00b740 tx=0x7f234c051bf0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.327+0000 7f235ed27800 1 -- 172.21.15.50:0/2254465609 shutdown_connections 2024-05-08T18:06:50.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.327+0000 7f235ed27800 1 --2- 172.21.15.50:0/2254465609 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba96b670 0x5573ba96ba90 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.327+0000 7f235ed27800 1 -- 172.21.15.50:0/2254465609 >> 172.21.15.50:0/2254465609 conn(0x5573ba967940 msgr2=0x5573ba9659a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.327+0000 7f235ed27800 1 -- 172.21.15.50:0/2254465609 shutdown_connections 2024-05-08T18:06:50.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.327+0000 7f235ed27800 1 -- 172.21.15.50:0/2254465609 wait complete. 2024-05-08T18:06:50.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.328+0000 7f235ed27800 1 Processor -- start 2024-05-08T18:06:50.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.328+0000 7f235ed27800 1 -- start start 2024-05-08T18:06:50.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.328+0000 7f235ed27800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba96b670 0x5573ba972990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.328+0000 7f235ed27800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5573ba96c6c0 con 0x5573ba96b670 2024-05-08T18:06:50.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.328+0000 7f235aae4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba96b670 0x5573ba972990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.328+0000 7f235aae4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba96b670 0x5573ba972990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45852/0 (socket says 172.21.15.50:45852) 2024-05-08T18:06:50.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.328+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 learned_addr learned my addr 172.21.15.50:0/3434365025 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.328+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573ba972ed0 con 0x5573ba96b670 2024-05-08T18:06:50.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.329+0000 7f235aae4640 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba96b670 0x5573ba972990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f234c00b6f0 tx=0x7f234c018ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.329+0000 7f223cff9640 1 -- 172.21.15.50:0/3434365025 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f234c06b020 con 0x5573ba96b670 2024-05-08T18:06:50.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.329+0000 7f223cff9640 1 -- 172.21.15.50:0/3434365025 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f234c061040 con 0x5573ba96b670 2024-05-08T18:06:50.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.329+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5573ba972ed0 con 0x5573ba96b670 2024-05-08T18:06:50.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.329+0000 7f223cff9640 1 -- 172.21.15.50:0/3434365025 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f234c051db0 con 0x5573ba96b670 2024-05-08T18:06:50.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.329+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573ba972ed0 con 0x5573ba96b670 2024-05-08T18:06:50.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.330+0000 7f223cff9640 1 -- 172.21.15.50:0/3434365025 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f234c012b50 con 0x5573ba96b670 2024-05-08T18:06:50.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.330+0000 7f223cff9640 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f222403bab0 0x7f222403df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.330+0000 7f223cff9640 1 -- 172.21.15.50:0/3434365025 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f234c046150 con 0x5573ba96b670 2024-05-08T18:06:50.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.330+0000 7f23522e3640 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f222403bab0 0x7f222403df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.330+0000 7f235ed27800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.330+0000 7f235ed27800 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5573ba97df60 0x5573ba980420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.330+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba980b70 con 0x5573ba97df60 2024-05-08T18:06:50.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.330+0000 7f23522e3640 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f222403bab0 0x7f222403df70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x5573ba950390 tx=0x7f2344060040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.330+0000 7f235b2e5640 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5573ba97df60 0x5573ba980420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.331+0000 7f235b2e5640 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5573ba97df60 0x5573ba980420 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.331+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f23540083e0 con 0x5573ba97df60 2024-05-08T18:06:50.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.331+0000 7f235ed27800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.343 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.331+0000 7f235ed27800 20 realm 2024-05-08T18:06:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.331+0000 7f235ed27800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.331+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba980f80 con 0x5573ba97df60 2024-05-08T18:06:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.331+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f23540083e0 con 0x5573ba97df60 2024-05-08T18:06:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.331+0000 7f235ed27800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.331+0000 7f235ed27800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.331+0000 7f235ed27800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.331+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba97df60 2024-05-08T18:06:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.331+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f23540083e0 con 0x5573ba97df60 2024-05-08T18:06:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.331+0000 7f235ed27800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.331+0000 7f235ed27800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.331+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981bf0 con 0x5573ba97df60 2024-05-08T18:06:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.332+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f23540083e0 con 0x5573ba97df60 2024-05-08T18:06:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.332+0000 7f235ed27800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.332+0000 7f235ed27800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.332+0000 7f235ed27800 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5573ba9825b0 0x5573ba9849a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.332+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba984ee0 con 0x5573ba9825b0 2024-05-08T18:06:50.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.332+0000 7f235aae4640 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5573ba9825b0 0x5573ba9849a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.332+0000 7f235aae4640 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5573ba9825b0 0x5573ba9849a0 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.333+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.333+0000 7f235ed27800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:50.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.333+0000 7f235ed27800 20 searching for the correct realm 2024-05-08T18:06:50.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.333+0000 7f235ed27800 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5573ba986fc0 0x5573ba989480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.333+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba989b50 con 0x5573ba986fc0 2024-05-08T18:06:50.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.333+0000 7f23522e3640 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5573ba986fc0 0x5573ba989480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.333+0000 7f23522e3640 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5573ba986fc0 0x5573ba989480 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.333+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2344070070 con 0x5573ba986fc0 2024-05-08T18:06:50.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.333+0000 7f23522e3640 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2344081040 0x7f2344082940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.333+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba989b50 con 0x7f2344081040 2024-05-08T18:06:50.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.333+0000 7f235b2e5640 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2344081040 0x7f2344082940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.334+0000 7f235b2e5640 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2344081040 0x7f2344082940 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.334+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f23540083e0 con 0x7f2344081040 2024-05-08T18:06:50.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.334+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba9825b0 2024-05-08T18:06:50.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.334+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.334+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba97df60 2024-05-08T18:06:50.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.335+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f23540083e0 con 0x5573ba97df60 2024-05-08T18:06:50.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.335+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba97df60 2024-05-08T18:06:50.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.335+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2354026040 con 0x5573ba97df60 2024-05-08T18:06:50.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.335+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba97df60 2024-05-08T18:06:50.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.335+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2354026040 con 0x5573ba97df60 2024-05-08T18:06:50.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.335+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba9825b0 2024-05-08T18:06:50.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.335+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.335+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x7f2344081040 2024-05-08T18:06:50.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.336+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2354026040 con 0x7f2344081040 2024-05-08T18:06:50.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.336+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba986fc0 2024-05-08T18:06:50.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.336+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2344070070 con 0x5573ba986fc0 2024-05-08T18:06:50.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.336+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba97df60 2024-05-08T18:06:50.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.336+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2354026040 con 0x5573ba97df60 2024-05-08T18:06:50.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.336+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba9825b0 2024-05-08T18:06:50.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.336+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.336+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba97df60 2024-05-08T18:06:50.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.337+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2354026040 con 0x5573ba97df60 2024-05-08T18:06:50.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.337+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x7f2344081040 2024-05-08T18:06:50.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.337+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2354026040 con 0x7f2344081040 2024-05-08T18:06:50.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.337+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba986fc0 2024-05-08T18:06:50.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.337+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2344070070 con 0x5573ba986fc0 2024-05-08T18:06:50.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.337+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba986fc0 2024-05-08T18:06:50.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.337+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2344070070 con 0x5573ba986fc0 2024-05-08T18:06:50.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.337+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba9825b0 2024-05-08T18:06:50.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.338+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.338+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba986fc0 2024-05-08T18:06:50.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.338+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2344070070 con 0x5573ba986fc0 2024-05-08T18:06:50.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.338+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba986fc0 2024-05-08T18:06:50.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.338+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f2344070070 con 0x5573ba986fc0 2024-05-08T18:06:50.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.338+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba97df60 2024-05-08T18:06:50.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.339+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2354026040 con 0x5573ba97df60 2024-05-08T18:06:50.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.339+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba97df60 2024-05-08T18:06:50.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.339+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f2354026040 con 0x5573ba97df60 2024-05-08T18:06:50.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.339+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba9825b0 2024-05-08T18:06:50.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.340+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.340+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba986fc0 2024-05-08T18:06:50.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.340+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2344070070 con 0x5573ba986fc0 2024-05-08T18:06:50.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.340+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x7f2344081040 2024-05-08T18:06:50.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.340+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2354026040 con 0x7f2344081040 2024-05-08T18:06:50.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.340+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba97df60 2024-05-08T18:06:50.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.340+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2354026040 con 0x5573ba97df60 2024-05-08T18:06:50.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.340+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba9825b0 2024-05-08T18:06:50.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.341+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.341+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba986fc0 2024-05-08T18:06:50.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.341+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2344070070 con 0x5573ba986fc0 2024-05-08T18:06:50.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.341+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba9825b0 2024-05-08T18:06:50.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.341+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.341+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x7f2344081040 2024-05-08T18:06:50.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.341+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2354026040 con 0x7f2344081040 2024-05-08T18:06:50.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.342+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba986fc0 2024-05-08T18:06:50.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.342+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2344070070 con 0x5573ba986fc0 2024-05-08T18:06:50.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.342+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba9825b0 2024-05-08T18:06:50.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.342+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.342+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x5573ba97df60 2024-05-08T18:06:50.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.342+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f2354026040 con 0x5573ba97df60 2024-05-08T18:06:50.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.342+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5573ba981850 con 0x7f2344081040 2024-05-08T18:06:50.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2354026040 con 0x7f2344081040 2024-05-08T18:06:50.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:50.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:50.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:50.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:50.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:50.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:50.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba985880 con 0x5573ba97df60 2024-05-08T18:06:50.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2354026040 con 0x5573ba97df60 2024-05-08T18:06:50.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba985880 con 0x5573ba97df60 2024-05-08T18:06:50.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2354035040 con 0x5573ba97df60 2024-05-08T18:06:50.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:50.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba985830 con 0x5573ba986fc0 2024-05-08T18:06:50.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f2344070070 con 0x5573ba986fc0 2024-05-08T18:06:50.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:50.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 20 zone default found 2024-05-08T18:06:50.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 4 Realm: () 2024-05-08T18:06:50.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:50.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:50.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:50.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 20 zonegroup default 2024-05-08T18:06:50.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.343+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba986120 con 0x5573ba9825b0 2024-05-08T18:06:50.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.344+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.344+0000 7f235ed27800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.344+0000 7f235ed27800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.344+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba98b9b0 con 0x5573ba986fc0 2024-05-08T18:06:50.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.344+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2344070070 con 0x5573ba986fc0 2024-05-08T18:06:50.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.344+0000 7f235ed27800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.362 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.344+0000 7f235ed27800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.344+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba98bd50 con 0x5573ba986fc0 2024-05-08T18:06:50.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.344+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2344070070 con 0x5573ba986fc0 2024-05-08T18:06:50.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.344+0000 7f235ed27800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.344+0000 7f235ed27800 20 started sync module instance, tier type = 2024-05-08T18:06:50.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.344+0000 7f235ed27800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:50.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.344+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba98ae50 con 0x7f2344081040 2024-05-08T18:06:50.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.345+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2354035040 con 0x7f2344081040 2024-05-08T18:06:50.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.345+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93955540169632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba98ae50 con 0x7f2344081040 2024-05-08T18:06:50.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.345+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba98c770 con 0x5573ba97df60 2024-05-08T18:06:50.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.346+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93955540169632] v29'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2354035040 con 0x7f2344081040 2024-05-08T18:06:50.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.346+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2354035040 con 0x5573ba97df60 2024-05-08T18:06:50.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.347+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93955540173408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba98ae50 con 0x5573ba97df60 2024-05-08T18:06:50.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.347+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba98d210 con 0x5573ba986fc0 2024-05-08T18:06:50.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.348+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93955540173408] v29'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2354035040 con 0x5573ba97df60 2024-05-08T18:06:50.363 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.348+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2344070070 con 0x5573ba986fc0 2024-05-08T18:06:50.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.348+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93955540167072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba98adc0 con 0x5573ba986fc0 2024-05-08T18:06:50.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.348+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba98dfc0 con 0x7f2344081040 2024-05-08T18:06:50.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.349+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93955540167072] v29'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2344070070 con 0x5573ba986fc0 2024-05-08T18:06:50.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.349+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2354035040 con 0x7f2344081040 2024-05-08T18:06:50.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.349+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93955540178432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba98e3b0 con 0x7f2344081040 2024-05-08T18:06:50.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.349+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba98edb0 con 0x5573ba97df60 2024-05-08T18:06:50.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.350+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93955540178432] v29'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2354035040 con 0x7f2344081040 2024-05-08T18:06:50.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.350+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2354035040 con 0x5573ba97df60 2024-05-08T18:06:50.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.350+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93955540179488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba98ed70 con 0x5573ba97df60 2024-05-08T18:06:50.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.350+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba98f7c0 con 0x5573ba9825b0 2024-05-08T18:06:50.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.351+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.351+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93955540185872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba990010 con 0x5573ba9825b0 2024-05-08T18:06:50.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.351+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba990ad0 con 0x5573ba9825b0 2024-05-08T18:06:50.364 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.351+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93955540179488] v29'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2354035040 con 0x5573ba97df60 2024-05-08T18:06:50.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.352+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93955540185872] v29'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.352+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.352+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93955540190624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba990e70 con 0x5573ba9825b0 2024-05-08T18:06:50.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.352+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba991700 con 0x5573ba9825b0 2024-05-08T18:06:50.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.353+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93955540190624] v29'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.353+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.353+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93955540183232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba9918e0 con 0x5573ba9825b0 2024-05-08T18:06:50.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.353+0000 7f235ed27800 20 add_watcher() i=0 2024-05-08T18:06:50.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.353+0000 7f235ed27800 20 add_watcher() i=1 2024-05-08T18:06:50.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.353+0000 7f235ed27800 20 add_watcher() i=2 2024-05-08T18:06:50.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.353+0000 7f235ed27800 20 add_watcher() i=3 2024-05-08T18:06:50.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.353+0000 7f235ed27800 20 add_watcher() i=4 2024-05-08T18:06:50.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.353+0000 7f235ed27800 20 add_watcher() i=5 2024-05-08T18:06:50.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.353+0000 7f235ed27800 20 add_watcher() i=6 2024-05-08T18:06:50.365 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.354+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93955540183232] v29'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.354+0000 7f235ed27800 20 add_watcher() i=7 2024-05-08T18:06:50.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.354+0000 7f235ed27800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:50.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.354+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba994cb0 con 0x5573ba97df60 2024-05-08T18:06:50.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.355+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f2354035040 con 0x5573ba97df60 2024-05-08T18:06:50.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.355+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93955540195280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba9920d0 con 0x5573ba97df60 2024-05-08T18:06:50.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.356+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93955540195280] v29'237 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2354035040 con 0x5573ba97df60 2024-05-08T18:06:50.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.356+0000 7f22117e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:50.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.356+0000 7f235ed27800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:50.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.356+0000 7f235ed27800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:50.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.356+0000 7f235ed27800 5 note: GC not initialized 2024-05-08T18:06:50.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.356+0000 7f218bfff640 20 reqs_thread_entry: start 2024-05-08T18:06:50.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.356+0000 7f235ed27800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:50.366 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.356+0000 7f235ed27800 20 Filter name: none 2024-05-08T18:06:50.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.357+0000 7f2189ffb640 20 reqs_thread_entry: start 2024-05-08T18:06:50.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.357+0000 7f235ed27800 20 rados->read ofs=0 len=0 2024-05-08T18:06:50.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.357+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5573ba9c7c20 con 0x5573ba97df60 2024-05-08T18:06:50.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.357+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f2354035040 con 0x5573ba97df60 2024-05-08T18:06:50.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.357+0000 7f235ed27800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:50.367 INFO:teuthology.orchestra.run.smithi050.stderr:could not fetch user info: no user info saved 2024-05-08T18:06:50.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.358+0000 7f22117e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:50.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.358+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93955540195280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba9c7be0 con 0x5573ba97df60 2024-05-08T18:06:50.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.359+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 93955540195280] v29'238 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2354035040 con 0x5573ba97df60 2024-05-08T18:06:50.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.359+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93955540169632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba9c7be0 con 0x7f2344081040 2024-05-08T18:06:50.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.360+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 93955540169632] v29'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2354035040 con 0x7f2344081040 2024-05-08T18:06:50.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.360+0000 7f235ed27800 20 remove_watcher() i=0 2024-05-08T18:06:50.367 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.360+0000 7f235ed27800 2 removed watcher, disabling cache 2024-05-08T18:06:50.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.360+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93955540173408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba9c7be0 con 0x5573ba97df60 2024-05-08T18:06:50.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.361+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 93955540173408] v29'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2354035040 con 0x5573ba97df60 2024-05-08T18:06:50.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.361+0000 7f235ed27800 20 remove_watcher() i=1 2024-05-08T18:06:50.368 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.361+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93955540167072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba9c7be0 con 0x5573ba986fc0 2024-05-08T18:06:50.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.362+0000 7f23522e3640 1 -- 172.21.15.50:0/3434365025 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 93955540167072] v29'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2344070070 con 0x5573ba986fc0 2024-05-08T18:06:50.372 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.362+0000 7f235ed27800 20 remove_watcher() i=2 2024-05-08T18:06:50.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.362+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93955540178432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba9c7be0 con 0x7f2344081040 2024-05-08T18:06:50.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.363+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 93955540178432] v29'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2354035040 con 0x7f2344081040 2024-05-08T18:06:50.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.363+0000 7f235ed27800 20 remove_watcher() i=3 2024-05-08T18:06:50.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.363+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93955540179488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba9c7be0 con 0x5573ba97df60 2024-05-08T18:06:50.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.364+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 93955540179488] v29'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2354035040 con 0x5573ba97df60 2024-05-08T18:06:50.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.364+0000 7f235ed27800 20 remove_watcher() i=4 2024-05-08T18:06:50.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.364+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93955540185872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba9c7be0 con 0x5573ba9825b0 2024-05-08T18:06:50.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.365+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 93955540185872] v29'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.365+0000 7f235ed27800 20 remove_watcher() i=5 2024-05-08T18:06:50.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.365+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93955540190624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba9c7be0 con 0x5573ba9825b0 2024-05-08T18:06:50.373 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.366+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 93955540190624] v29'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.366+0000 7f235ed27800 20 remove_watcher() i=6 2024-05-08T18:06:50.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.366+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93955540183232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5573ba9c7be0 con 0x5573ba9825b0 2024-05-08T18:06:50.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.367+0000 7f235aae4640 1 -- 172.21.15.50:0/3434365025 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 93955540183232] v29'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f234c06d020 con 0x5573ba9825b0 2024-05-08T18:06:50.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.367+0000 7f235ed27800 20 remove_watcher() i=7 2024-05-08T18:06:50.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.367+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5573ba9825b0 msgr2=0x5573ba9849a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.367+0000 7f235ed27800 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5573ba9825b0 0x5573ba9849a0 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.367+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5573ba97df60 msgr2=0x5573ba980420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.367+0000 7f235ed27800 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5573ba97df60 0x5573ba980420 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.367+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2344081040 msgr2=0x7f2344082940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.367+0000 7f235ed27800 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2344081040 0x7f2344082940 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.367+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5573ba986fc0 msgr2=0x5573ba989480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.367+0000 7f235ed27800 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5573ba986fc0 0x5573ba989480 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.374 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.367+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f222403bab0 msgr2=0x7f222403df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.367+0000 7f235ed27800 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f222403bab0 0x7f222403df70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x5573ba950390 tx=0x7f2344060040 comp rx=0 tx=0).stop 2024-05-08T18:06:50.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba96b670 msgr2=0x5573ba972990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235b2e5640 1 -- 172.21.15.50:0/3434365025 reap_dead start 2024-05-08T18:06:50.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba96b670 0x5573ba972990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f234c00b6f0 tx=0x7f234c018ef0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 shutdown_connections 2024-05-08T18:06:50.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 --2- 172.21.15.50:0/3434365025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba96b670 0x5573ba972990 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 >> 172.21.15.50:0/3434365025 conn(0x5573ba967940 msgr2=0x5573ba964fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 shutdown_connections 2024-05-08T18:06:50.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 -- 172.21.15.50:0/3434365025 wait complete. 2024-05-08T18:06:50.375 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 -- 172.21.15.50:0/3505141570 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f223403bba0 msgr2=0x7f223403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 --2- 172.21.15.50:0/3505141570 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f223403bba0 0x7f223403e060 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x5573ba9476e0 tx=0x7f2344005ac0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 -- 172.21.15.50:0/3505141570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba95e2e0 msgr2=0x5573ba95e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 --2- 172.21.15.50:0/3505141570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba95e2e0 0x5573ba95e700 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f234c006af0 tx=0x7f234c043490 comp rx=0 tx=0).stop 2024-05-08T18:06:50.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 -- 172.21.15.50:0/3505141570 shutdown_connections 2024-05-08T18:06:50.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 --2- 172.21.15.50:0/3505141570 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f223403bba0 0x7f223403e060 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 --2- 172.21.15.50:0/3505141570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba95e2e0 0x5573ba95e700 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 -- 172.21.15.50:0/3505141570 >> 172.21.15.50:0/3505141570 conn(0x5573ba952380 msgr2=0x5573ba952780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 -- 172.21.15.50:0/3505141570 shutdown_connections 2024-05-08T18:06:50.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 -- 172.21.15.50:0/3505141570 wait complete. 2024-05-08T18:06:50.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 -- 172.21.15.50:0/4171545640 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5573ba949540 msgr2=0x5573ba94b930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5573ba949540 0x5573ba94b930 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 -- 172.21.15.50:0/4171545640 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5573ba844ad0 msgr2=0x5573ba946c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.376 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5573ba844ad0 0x5573ba946c80 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 -- 172.21.15.50:0/4171545640 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5573ba94d480 msgr2=0x5573ba94f870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.368+0000 7f235ed27800 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5573ba94d480 0x5573ba94f870 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.369+0000 7f235ed27800 1 -- 172.21.15.50:0/4171545640 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f224003bba0 msgr2=0x7f224003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.369+0000 7f235ed27800 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f224003bba0 0x7f224003e060 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2344054000 tx=0x7f234404e000 comp rx=0 tx=0).stop 2024-05-08T18:06:50.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.369+0000 7f235ed27800 1 -- 172.21.15.50:0/4171545640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba842f90 msgr2=0x5573ba93fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.369+0000 7f235ed27800 1 --2- 172.21.15.50:0/4171545640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573ba842f90 0x5573ba93fa90 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f234c001330 tx=0x7f234c001250 comp rx=0 tx=0).stop 2024-05-08T18:06:50.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.369+0000 7f235b2e5640 1 -- 172.21.15.50:0/4171545640 reap_dead start 2024-05-08T18:06:50.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.369+0000 7f235ed27800 1 -- 172.21.15.50:0/4171545640 shutdown_connections 2024-05-08T18:06:50.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.369+0000 7f235ed27800 1 -- 172.21.15.50:0/4171545640 >> 172.21.15.50:0/4171545640 conn(0x5573ba83ead0 msgr2=0x5573ba931fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.369+0000 7f235ed27800 1 -- 172.21.15.50:0/4171545640 shutdown_connections 2024-05-08T18:06:50.377 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.369+0000 7f235ed27800 1 -- 172.21.15.50:0/4171545640 wait complete. 2024-05-08T18:06:50.378 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T18:06:50.381 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get', '--rgw-zone', 'default'] 2024-05-08T18:06:50.381 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get', '--rgw-zone', 'default'] 2024-05-08T18:06:50.382 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get --rgw-zone default 2024-05-08T18:06:50.426 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:50.426 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:50.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.429+0000 7f197e136800 1 Processor -- start 2024-05-08T18:06:50.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.429+0000 7f197e136800 1 -- start start 2024-05-08T18:06:50.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.429+0000 7f197e136800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x556ba3cbeb50 0x556ba3cc7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.429+0000 7f197e136800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x556ba3c31870 con 0x556ba3cbd010 2024-05-08T18:06:50.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.429+0000 7f197e136800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x556ba3b5ee20 con 0x556ba3cbeb50 2024-05-08T18:06:50.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.429+0000 7f1973fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x556ba3cbeb50 0x556ba3cc7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.429+0000 7f1979ec5640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x556ba3cbd010 0x556ba3cbd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50744/0 (socket says 172.21.15.50:50744) 2024-05-08T18:06:50.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.429+0000 7f1979ec5640 1 -- 172.21.15.50:0/1114690742 learned_addr learned my addr 172.21.15.50:0/1114690742 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:50.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.429+0000 7f1973fff640 1 -- 172.21.15.50:0/1114690742 >> v1:172.21.15.46:6789/0 conn(0x556ba3cbd010 legacy=0x556ba3cbd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.429+0000 7f1973fff640 1 -- 172.21.15.50:0/1114690742 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba3cc8ba0 con 0x556ba3cbeb50 2024-05-08T18:06:50.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.431+0000 7f1973fff640 1 --2- 172.21.15.50:0/1114690742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbeb50 0x556ba3cc7650 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f196400a470 tx=0x7f196401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6d4e1cb291d88e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.432+0000 7f19796c4640 1 -- 172.21.15.50:0/1114690742 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1964057070 con 0x556ba3cbeb50 2024-05-08T18:06:50.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.432+0000 7f19796c4640 1 -- 172.21.15.50:0/1114690742 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196404adc0 con 0x556ba3cbeb50 2024-05-08T18:06:50.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.432+0000 7f19796c4640 1 -- 172.21.15.50:0/1114690742 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1964052590 con 0x556ba3cbeb50 2024-05-08T18:06:50.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.432+0000 7f197e136800 1 -- 172.21.15.50:0/1114690742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbeb50 msgr2=0x556ba3cc7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.432+0000 7f197e136800 1 --2- 172.21.15.50:0/1114690742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbeb50 0x556ba3cc7650 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f196400a470 tx=0x7f196401ce90 comp rx=0 tx=0).stop 2024-05-08T18:06:50.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.432+0000 7f197e136800 1 -- 172.21.15.50:0/1114690742 shutdown_connections 2024-05-08T18:06:50.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.432+0000 7f197e136800 1 --2- 172.21.15.50:0/1114690742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbeb50 0x556ba3cc7650 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.432+0000 7f197e136800 1 -- 172.21.15.50:0/1114690742 >> 172.21.15.50:0/1114690742 conn(0x556ba3cb8c10 msgr2=0x556ba3cbb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.432+0000 7f197e136800 1 -- 172.21.15.50:0/1114690742 shutdown_connections 2024-05-08T18:06:50.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.432+0000 7f197e136800 1 -- 172.21.15.50:0/1114690742 wait complete. 2024-05-08T18:06:50.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.447+0000 7f197e136800 1 Processor -- start 2024-05-08T18:06:50.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.448+0000 7f197e136800 1 -- start start 2024-05-08T18:06:50.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.448+0000 7f197e136800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbd010 0x556ba3db1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.448+0000 7f197e136800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556ba3c09a10 con 0x556ba3cbd010 2024-05-08T18:06:50.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.448+0000 7f1979ec5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbd010 0x556ba3db1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.448+0000 7f1979ec5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbd010 0x556ba3db1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45876/0 (socket says 172.21.15.50:45876) 2024-05-08T18:06:50.446 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.448+0000 7f1979ec5640 1 -- 172.21.15.50:0/1470563762 learned_addr learned my addr 172.21.15.50:0/1470563762 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.448+0000 7f1979ec5640 1 -- 172.21.15.50:0/1470563762 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba3db2510 con 0x556ba3cbd010 2024-05-08T18:06:50.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.449+0000 7f1979ec5640 1 --2- 172.21.15.50:0/1470563762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbd010 0x556ba3db1f40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f196c001e10 tx=0x7f196c00b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a9ff3a96c27319 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.449+0000 7f1867e07640 1 -- 172.21.15.50:0/1470563762 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f196c012020 con 0x556ba3cbd010 2024-05-08T18:06:50.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.449+0000 7f1867e07640 1 -- 172.21.15.50:0/1470563762 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196c00e5a0 con 0x556ba3cbd010 2024-05-08T18:06:50.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.449+0000 7f1867e07640 1 -- 172.21.15.50:0/1470563762 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f196c00e8a0 con 0x556ba3cbd010 2024-05-08T18:06:50.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.449+0000 7f197e136800 1 -- 172.21.15.50:0/1470563762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbd010 msgr2=0x556ba3db1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.450+0000 7f197e136800 1 --2- 172.21.15.50:0/1470563762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbd010 0x556ba3db1f40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f196c001e10 tx=0x7f196c00b780 comp rx=0 tx=0).stop 2024-05-08T18:06:50.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.450+0000 7f197e136800 1 -- 172.21.15.50:0/1470563762 shutdown_connections 2024-05-08T18:06:50.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.450+0000 7f197e136800 1 --2- 172.21.15.50:0/1470563762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbd010 0x556ba3db1f40 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.450+0000 7f197e136800 1 -- 172.21.15.50:0/1470563762 >> 172.21.15.50:0/1470563762 conn(0x556ba3cb8c10 msgr2=0x556ba3dad860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.450+0000 7f197e136800 1 -- 172.21.15.50:0/1470563762 shutdown_connections 2024-05-08T18:06:50.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.450+0000 7f197e136800 1 -- 172.21.15.50:0/1470563762 wait complete. 2024-05-08T18:06:50.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.450+0000 7f197e136800 1 Processor -- start 2024-05-08T18:06:50.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.450+0000 7f197e136800 1 -- start start 2024-05-08T18:06:50.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.450+0000 7f197e136800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbd010 0x556ba3db9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.450+0000 7f197e136800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556ba3cc7e60 con 0x556ba3cbd010 2024-05-08T18:06:50.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.450+0000 7f1979ec5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbd010 0x556ba3db9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.450+0000 7f1979ec5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbd010 0x556ba3db9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45892/0 (socket says 172.21.15.50:45892) 2024-05-08T18:06:50.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.450+0000 7f1979ec5640 1 -- 172.21.15.50:0/2945232961 learned_addr learned my addr 172.21.15.50:0/2945232961 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.451+0000 7f1979ec5640 1 -- 172.21.15.50:0/2945232961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba3db9ff0 con 0x556ba3cbd010 2024-05-08T18:06:50.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.451+0000 7f1979ec5640 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbd010 0x556ba3db9ab0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f196c000f60 tx=0x7f196c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.451+0000 7f1866604640 1 -- 172.21.15.50:0/2945232961 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f196c012030 con 0x556ba3cbd010 2024-05-08T18:06:50.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.451+0000 7f197e136800 1 -- 172.21.15.50:0/2945232961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556ba3dba280 con 0x556ba3cbd010 2024-05-08T18:06:50.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.451+0000 7f1866604640 1 -- 172.21.15.50:0/2945232961 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196c00ee50 con 0x556ba3cbd010 2024-05-08T18:06:50.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.451+0000 7f1866604640 1 -- 172.21.15.50:0/2945232961 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f196c01a880 con 0x556ba3cbd010 2024-05-08T18:06:50.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.451+0000 7f197e136800 1 -- 172.21.15.50:0/2945232961 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba3dbcf30 con 0x556ba3cbd010 2024-05-08T18:06:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.452+0000 7f197e136800 1 Processor -- start 2024-05-08T18:06:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.452+0000 7f197e136800 1 -- start start 2024-05-08T18:06:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.452+0000 7f197e136800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3dc7140 0x556ba3dc7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.452+0000 7f197e136800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556ba3bb0330 con 0x556ba3dc7140 2024-05-08T18:06:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.452+0000 7f1866604640 1 -- 172.21.15.50:0/2945232961 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f196c015020 con 0x556ba3cbd010 2024-05-08T18:06:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.452+0000 7f197a6c6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3dc7140 0x556ba3dc7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.452+0000 7f197a6c6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3dc7140 0x556ba3dc7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45906/0 (socket says 172.21.15.50:45906) 2024-05-08T18:06:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.452+0000 7f197a6c6640 1 -- 172.21.15.50:0/4133103063 learned_addr learned my addr 172.21.15.50:0/4133103063 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.452+0000 7f1866604640 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f186003bb50 0x7f186003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.452+0000 7f197a6c6640 1 -- 172.21.15.50:0/4133103063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba3dc7aa0 con 0x556ba3dc7140 2024-05-08T18:06:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.452+0000 7f1866604640 1 -- 172.21.15.50:0/2945232961 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f196c020030 con 0x556ba3cbd010 2024-05-08T18:06:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.453+0000 7f1979ec5640 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f186003bb50 0x7f186003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.453+0000 7f197a6c6640 1 --2- 172.21.15.50:0/4133103063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3dc7140 0x556ba3dc7560 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f197400a430 tx=0x7f197400a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90acc3b9ba1ae010 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.453+0000 7f185ffff640 1 -- 172.21.15.50:0/4133103063 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1974015070 con 0x556ba3dc7140 2024-05-08T18:06:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.453+0000 7f1979ec5640 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f186003bb50 0x7f186003e010 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f196c024fd0 tx=0x7f196c024f20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.453+0000 7f185ffff640 1 -- 172.21.15.50:0/4133103063 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1974001670 con 0x556ba3dc7140 2024-05-08T18:06:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.453+0000 7f185ffff640 1 -- 172.21.15.50:0/4133103063 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1974001990 con 0x556ba3dc7140 2024-05-08T18:06:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.453+0000 7f197e136800 1 -- 172.21.15.50:0/4133103063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3dc7140 msgr2=0x556ba3dc7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.453+0000 7f197e136800 1 --2- 172.21.15.50:0/4133103063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3dc7140 0x556ba3dc7560 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f197400a430 tx=0x7f197400a8f0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.453+0000 7f197e136800 1 -- 172.21.15.50:0/4133103063 shutdown_connections 2024-05-08T18:06:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.453+0000 7f197e136800 1 --2- 172.21.15.50:0/4133103063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3dc7140 0x556ba3dc7560 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.453+0000 7f197e136800 1 -- 172.21.15.50:0/4133103063 >> 172.21.15.50:0/4133103063 conn(0x556ba3cbeb50 msgr2=0x556ba3dc3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.453+0000 7f197e136800 1 -- 172.21.15.50:0/4133103063 shutdown_connections 2024-05-08T18:06:50.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.453+0000 7f197e136800 1 -- 172.21.15.50:0/4133103063 wait complete. 2024-05-08T18:06:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.454+0000 7f197e136800 1 Processor -- start 2024-05-08T18:06:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.454+0000 7f197e136800 1 -- start start 2024-05-08T18:06:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.454+0000 7f197e136800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3dc7140 0x556ba3dce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.454+0000 7f197e136800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556ba3dc8430 con 0x556ba3dc7140 2024-05-08T18:06:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.454+0000 7f197a6c6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3dc7140 0x556ba3dce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.454+0000 7f197a6c6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3dc7140 0x556ba3dce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45912/0 (socket says 172.21.15.50:45912) 2024-05-08T18:06:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.454+0000 7f197a6c6640 1 -- 172.21.15.50:0/2682881138 learned_addr learned my addr 172.21.15.50:0/2682881138 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.454+0000 7f197a6c6640 1 -- 172.21.15.50:0/2682881138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba3dceb00 con 0x556ba3dc7140 2024-05-08T18:06:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.455+0000 7f197a6c6640 1 --2- 172.21.15.50:0/2682881138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3dc7140 0x556ba3dce5c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f197400a930 tx=0x7f1974003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.455+0000 7f185e7fc640 1 -- 172.21.15.50:0/2682881138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1974020020 con 0x556ba3dc7140 2024-05-08T18:06:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.455+0000 7f185e7fc640 1 -- 172.21.15.50:0/2682881138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19740108b0 con 0x556ba3dc7140 2024-05-08T18:06:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.455+0000 7f197e136800 1 -- 172.21.15.50:0/2682881138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556ba3dced90 con 0x556ba3dc7140 2024-05-08T18:06:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.455+0000 7f185e7fc640 1 -- 172.21.15.50:0/2682881138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1974015040 con 0x556ba3dc7140 2024-05-08T18:06:50.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.455+0000 7f197e136800 1 -- 172.21.15.50:0/2682881138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba3dd1a40 con 0x556ba3dc7140 2024-05-08T18:06:50.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.456+0000 7f185e7fc640 1 -- 172.21.15.50:0/2682881138 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f1974023020 con 0x556ba3dc7140 2024-05-08T18:06:50.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.456+0000 7f185e7fc640 1 --2- 172.21.15.50:0/2682881138 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f185403bba0 0x7f185403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.456+0000 7f185e7fc640 1 -- 172.21.15.50:0/2682881138 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f1974052390 con 0x556ba3dc7140 2024-05-08T18:06:50.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.456+0000 7f1973fff640 1 --2- 172.21.15.50:0/2682881138 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f185403bba0 0x7f185403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.456+0000 7f197e136800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:50.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.456+0000 7f197e136800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:50.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.456+0000 7f1973fff640 1 --2- 172.21.15.50:0/2682881138 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f185403bba0 0x7f185403e060 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1964054000 tx=0x7f196404e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.457+0000 7f197e136800 1 Processor -- start 2024-05-08T18:06:50.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.457+0000 7f197e136800 1 -- start start 2024-05-08T18:06:50.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.457+0000 7f197e136800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3de8c40 0x556ba3de9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.457+0000 7f197e136800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556ba3db2890 con 0x556ba3de8c40 2024-05-08T18:06:50.462 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.457+0000 7f1979ec5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3de8c40 0x556ba3de9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.457+0000 7f1979ec5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3de8c40 0x556ba3de9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45922/0 (socket says 172.21.15.50:45922) 2024-05-08T18:06:50.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.457+0000 7f1979ec5640 1 -- 172.21.15.50:0/3006971347 learned_addr learned my addr 172.21.15.50:0/3006971347 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.458+0000 7f1979ec5640 1 -- 172.21.15.50:0/3006971347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba3de95a0 con 0x556ba3de8c40 2024-05-08T18:06:50.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.458+0000 7f1979ec5640 1 --2- 172.21.15.50:0/3006971347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3de8c40 0x556ba3de9060 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f196c02df10 tx=0x7f196c018d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.458+0000 7f183efe5640 1 -- 172.21.15.50:0/3006971347 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f196c024ce0 con 0x556ba3de8c40 2024-05-08T18:06:50.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.458+0000 7f183efe5640 1 -- 172.21.15.50:0/3006971347 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f196c050920 con 0x556ba3de8c40 2024-05-08T18:06:50.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.458+0000 7f197e136800 1 -- 172.21.15.50:0/3006971347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556ba3dea010 con 0x556ba3de8c40 2024-05-08T18:06:50.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.458+0000 7f183efe5640 1 -- 172.21.15.50:0/3006971347 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f196c050c00 con 0x556ba3de8c40 2024-05-08T18:06:50.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.458+0000 7f197e136800 1 -- 172.21.15.50:0/3006971347 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba3dea510 con 0x556ba3de8c40 2024-05-08T18:06:50.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.459+0000 7f183efe5640 1 -- 172.21.15.50:0/3006971347 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f196c036070 con 0x556ba3de8c40 2024-05-08T18:06:50.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.459+0000 7f183efe5640 1 --2- 172.21.15.50:0/3006971347 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f17b803bb50 0x7f17b803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.459+0000 7f183efe5640 1 -- 172.21.15.50:0/3006971347 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f196c043070 con 0x556ba3de8c40 2024-05-08T18:06:50.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.459+0000 7f1973fff640 1 --2- 172.21.15.50:0/3006971347 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f17b803bb50 0x7f17b803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.459+0000 7f183cfe1640 20 reqs_thread_entry: start 2024-05-08T18:06:50.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.459+0000 7f197e136800 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556ba3df6640 0x556ba3df8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.459+0000 7f197e136800 1 -- 172.21.15.50:0/2945232961 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556ba3df91d0 con 0x556ba3df6640 2024-05-08T18:06:50.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.459+0000 7f1973fff640 1 --2- 172.21.15.50:0/3006971347 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f17b803bb50 0x7f17b803e010 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x556ba3c30c70 tx=0x7f1964005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.459+0000 7f197a6c6640 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556ba3df6640 0x556ba3df8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.460+0000 7f197a6c6640 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556ba3df6640 0x556ba3df8b00 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.460+0000 7f197a6c6640 1 -- 172.21.15.50:0/2945232961 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f1974010d70 con 0x556ba3df6640 2024-05-08T18:06:50.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.460+0000 7f197e136800 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556ba3dfafa0 0x556ba3dfd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.460+0000 7f197e136800 1 -- 172.21.15.50:0/2945232961 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:2 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556ba3dfda90 con 0x556ba3dfafa0 2024-05-08T18:06:50.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.460+0000 7f1979ec5640 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556ba3dfafa0 0x556ba3dfd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.461+0000 7f1979ec5640 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556ba3dfafa0 0x556ba3dfd390 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.461+0000 7f1979ec5640 1 -- 172.21.15.50:0/2945232961 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(2 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f196c05b040 con 0x556ba3dfafa0 2024-05-08T18:06:50.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 -- 172.21.15.50:0/2682881138 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f185403bba0 msgr2=0x7f185403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 --2- 172.21.15.50:0/2682881138 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f185403bba0 0x7f185403e060 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1964054000 tx=0x7f196404e000 comp rx=0 tx=0).stop 2024-05-08T18:06:50.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 -- 172.21.15.50:0/2682881138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3dc7140 msgr2=0x556ba3dce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 --2- 172.21.15.50:0/2682881138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3dc7140 0x556ba3dce5c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f197400a930 tx=0x7f1974003370 comp rx=0 tx=0).stop 2024-05-08T18:06:50.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 -- 172.21.15.50:0/2682881138 shutdown_connections 2024-05-08T18:06:50.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 --2- 172.21.15.50:0/2682881138 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f185403bba0 0x7f185403e060 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 --2- 172.21.15.50:0/2682881138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3dc7140 0x556ba3dce5c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 -- 172.21.15.50:0/2682881138 >> 172.21.15.50:0/2682881138 conn(0x556ba3cbeb50 msgr2=0x556ba3dc1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 -- 172.21.15.50:0/2682881138 shutdown_connections 2024-05-08T18:06:50.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 -- 172.21.15.50:0/2682881138 wait complete. 2024-05-08T18:06:50.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 -- 172.21.15.50:0/3006971347 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f17b803bb50 msgr2=0x7f17b803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 --2- 172.21.15.50:0/3006971347 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f17b803bb50 0x7f17b803e010 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x556ba3c30c70 tx=0x7f1964005a00 comp rx=0 tx=0).stop 2024-05-08T18:06:50.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 -- 172.21.15.50:0/3006971347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3de8c40 msgr2=0x556ba3de9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 --2- 172.21.15.50:0/3006971347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3de8c40 0x556ba3de9060 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f196c02df10 tx=0x7f196c018d20 comp rx=0 tx=0).stop 2024-05-08T18:06:50.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 -- 172.21.15.50:0/3006971347 shutdown_connections 2024-05-08T18:06:50.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 --2- 172.21.15.50:0/3006971347 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f17b803bb50 0x7f17b803e010 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 --2- 172.21.15.50:0/3006971347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3de8c40 0x556ba3de9060 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 -- 172.21.15.50:0/3006971347 >> 172.21.15.50:0/3006971347 conn(0x556ba3ddd6d0 msgr2=0x556ba3dde0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 -- 172.21.15.50:0/3006971347 shutdown_connections 2024-05-08T18:06:50.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.462+0000 7f197e136800 1 -- 172.21.15.50:0/3006971347 wait complete. 2024-05-08T18:06:50.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.463+0000 7f197e136800 1 -- 172.21.15.50:0/2945232961 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556ba3dfafa0 msgr2=0x556ba3dfd390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.463+0000 7f197e136800 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556ba3dfafa0 0x556ba3dfd390 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.463+0000 7f197e136800 1 -- 172.21.15.50:0/2945232961 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556ba3df6640 msgr2=0x556ba3df8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.463+0000 7f197e136800 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556ba3df6640 0x556ba3df8b00 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.463+0000 7f197e136800 1 -- 172.21.15.50:0/2945232961 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f186003bb50 msgr2=0x7f186003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.463+0000 7f197e136800 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f186003bb50 0x7f186003e010 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f196c024fd0 tx=0x7f196c024f20 comp rx=0 tx=0).stop 2024-05-08T18:06:50.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.463+0000 7f197e136800 1 -- 172.21.15.50:0/2945232961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbd010 msgr2=0x556ba3db9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.463+0000 7f197e136800 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbd010 0x556ba3db9ab0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f196c000f60 tx=0x7f196c000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.463+0000 7f197e136800 1 -- 172.21.15.50:0/2945232961 shutdown_connections 2024-05-08T18:06:50.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.463+0000 7f197e136800 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556ba3dfafa0 0x556ba3dfd390 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.463+0000 7f197e136800 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556ba3df6640 0x556ba3df8b00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.463+0000 7f197e136800 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f186003bb50 0x7f186003e010 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.463+0000 7f197e136800 1 --2- 172.21.15.50:0/2945232961 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556ba3cbd010 0x556ba3db9ab0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.463+0000 7f197e136800 1 -- 172.21.15.50:0/2945232961 >> 172.21.15.50:0/2945232961 conn(0x556ba3cb8c10 msgr2=0x556ba3dac060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.463+0000 7f197e136800 1 -- 172.21.15.50:0/2945232961 shutdown_connections 2024-05-08T18:06:50.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.463+0000 7f197e136800 1 -- 172.21.15.50:0/2945232961 wait complete. 2024-05-08T18:06:50.471 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"1c041ffd-e40a-486a-88e5-3547b85128ea","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-05-08T18:06:50.471 DEBUG:tasks.util.rgw: json result: {'id': '1c041ffd-e40a-486a-88e5-3547b85128ea', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-05-08T18:06:50.471 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'set'] 2024-05-08T18:06:50.471 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'set'] 2024-05-08T18:06:50.471 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone set 2024-05-08T18:06:50.515 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:50.515 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:50.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.518+0000 7f85dc010800 1 Processor -- start 2024-05-08T18:06:50.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.518+0000 7f85dc010800 1 -- start start 2024-05-08T18:06:50.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.518+0000 7f85dc010800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x557fdf5a89f0 0x557fdf5aade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.518+0000 7f85dc010800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x557fdf5ab3b0 con 0x557fdf5a4a40 2024-05-08T18:06:50.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.518+0000 7f85dc010800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x557fdf445e20 con 0x557fdf5a89f0 2024-05-08T18:06:50.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.519+0000 7f85d6e75640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x557fdf5a89f0 0x557fdf5aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.519+0000 7f85d6e75640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x557fdf5a89f0 0x557fdf5aade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45930/0 (socket says 172.21.15.50:45930) 2024-05-08T18:06:50.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.519+0000 7f85d6e75640 1 -- 172.21.15.50:0/2609991149 learned_addr learned my addr 172.21.15.50:0/2609991149 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.519+0000 7f85d6e75640 1 -- 172.21.15.50:0/2609991149 >> v1:172.21.15.46:6789/0 conn(0x557fdf5a4a40 legacy=0x557fdf5a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.519+0000 7f85d6e75640 1 -- 172.21.15.50:0/2609991149 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fdf5ac590 con 0x557fdf5a89f0 2024-05-08T18:06:50.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.521+0000 7f85d6e75640 1 --2- 172.21.15.50:0/2609991149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a89f0 0x557fdf5aade0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00a520 tx=0x7f85cc01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99fd14458489d616 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.521+0000 7f85d6674640 1 -- 172.21.15.50:0/2609991149 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85cc057070 con 0x557fdf5a89f0 2024-05-08T18:06:50.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.521+0000 7f85d6674640 1 -- 172.21.15.50:0/2609991149 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85cc052570 con 0x557fdf5a89f0 2024-05-08T18:06:50.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.521+0000 7f85d6674640 1 -- 172.21.15.50:0/2609991149 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85cc04ae60 con 0x557fdf5a89f0 2024-05-08T18:06:50.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.521+0000 7f85dc010800 1 -- 172.21.15.50:0/2609991149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a89f0 msgr2=0x557fdf5aade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.521+0000 7f85dc010800 1 --2- 172.21.15.50:0/2609991149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a89f0 0x557fdf5aade0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00a520 tx=0x7f85cc01d010 comp rx=0 tx=0).stop 2024-05-08T18:06:50.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.521+0000 7f85dc010800 1 -- 172.21.15.50:0/2609991149 shutdown_connections 2024-05-08T18:06:50.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.521+0000 7f85dc010800 1 --2- 172.21.15.50:0/2609991149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a89f0 0x557fdf5aade0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.521+0000 7f85dc010800 1 -- 172.21.15.50:0/2609991149 >> 172.21.15.50:0/2609991149 conn(0x557fdf59f2c0 msgr2=0x557fdf5a16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.521+0000 7f85dc010800 1 -- 172.21.15.50:0/2609991149 shutdown_connections 2024-05-08T18:06:50.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.522+0000 7f85dc010800 1 -- 172.21.15.50:0/2609991149 wait complete. 2024-05-08T18:06:50.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.537+0000 7f85dc010800 1 Processor -- start 2024-05-08T18:06:50.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.537+0000 7f85dc010800 1 -- start start 2024-05-08T18:06:50.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.537+0000 7f85dc010800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a4a40 0x557fdf6959f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.537+0000 7f85dc010800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557fdf4f0a00 con 0x557fdf5a4a40 2024-05-08T18:06:50.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.537+0000 7f85d7676640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a4a40 0x557fdf6959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.537+0000 7f85d7676640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a4a40 0x557fdf6959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45934/0 (socket says 172.21.15.50:45934) 2024-05-08T18:06:50.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.537+0000 7f85d7676640 1 -- 172.21.15.50:0/3768128219 learned_addr learned my addr 172.21.15.50:0/3768128219 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.538+0000 7f85d7676640 1 -- 172.21.15.50:0/3768128219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fdf695fc0 con 0x557fdf5a4a40 2024-05-08T18:06:50.536 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.538+0000 7f85d7676640 1 --2- 172.21.15.50:0/3768128219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a4a40 0x557fdf6959f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f85c8001d70 tx=0x7f85c800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324eeb4ced392229 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.538+0000 7f84c560a640 1 -- 172.21.15.50:0/3768128219 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85c800be60 con 0x557fdf5a4a40 2024-05-08T18:06:50.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.538+0000 7f84c560a640 1 -- 172.21.15.50:0/3768128219 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c800f400 con 0x557fdf5a4a40 2024-05-08T18:06:50.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.538+0000 7f84c560a640 1 -- 172.21.15.50:0/3768128219 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85c800f720 con 0x557fdf5a4a40 2024-05-08T18:06:50.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.539+0000 7f85dc010800 1 -- 172.21.15.50:0/3768128219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a4a40 msgr2=0x557fdf6959f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.539+0000 7f85dc010800 1 --2- 172.21.15.50:0/3768128219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a4a40 0x557fdf6959f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f85c8001d70 tx=0x7f85c800b5b0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.539+0000 7f85dc010800 1 -- 172.21.15.50:0/3768128219 shutdown_connections 2024-05-08T18:06:50.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.539+0000 7f85dc010800 1 --2- 172.21.15.50:0/3768128219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a4a40 0x557fdf6959f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.539+0000 7f85dc010800 1 -- 172.21.15.50:0/3768128219 >> 172.21.15.50:0/3768128219 conn(0x557fdf59f2c0 msgr2=0x557fdf691310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.539+0000 7f85dc010800 1 -- 172.21.15.50:0/3768128219 shutdown_connections 2024-05-08T18:06:50.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.539+0000 7f85dc010800 1 -- 172.21.15.50:0/3768128219 wait complete. 2024-05-08T18:06:50.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.539+0000 7f85dc010800 1 Processor -- start 2024-05-08T18:06:50.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.539+0000 7f85dc010800 1 -- start start 2024-05-08T18:06:50.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.539+0000 7f85dc010800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a4a40 0x557fdf6a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.539+0000 7f85dc010800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557fdf5ab880 con 0x557fdf5a4a40 2024-05-08T18:06:50.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.540+0000 7f85d7676640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a4a40 0x557fdf6a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.540+0000 7f85d7676640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a4a40 0x557fdf6a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45942/0 (socket says 172.21.15.50:45942) 2024-05-08T18:06:50.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.540+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 learned_addr learned my addr 172.21.15.50:0/4080233049 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.540+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fdf6a0dc0 con 0x557fdf5a4a40 2024-05-08T18:06:50.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.540+0000 7f85d7676640 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a4a40 0x557fdf6a0880 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f85c8000c00 tx=0x7f85c8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.540+0000 7f84c3e07640 1 -- 172.21.15.50:0/4080233049 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85c801a5b0 con 0x557fdf5a4a40 2024-05-08T18:06:50.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.540+0000 7f84c3e07640 1 -- 172.21.15.50:0/4080233049 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c801a750 con 0x557fdf5a4a40 2024-05-08T18:06:50.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.540+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557fdf6a1050 con 0x557fdf5a4a40 2024-05-08T18:06:50.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.540+0000 7f84c3e07640 1 -- 172.21.15.50:0/4080233049 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85c801aa50 con 0x557fdf5a4a40 2024-05-08T18:06:50.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.540+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fdf6a3d00 con 0x557fdf5a4a40 2024-05-08T18:06:50.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.541+0000 7f85dc010800 1 Processor -- start 2024-05-08T18:06:50.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.541+0000 7f85dc010800 1 -- start start 2024-05-08T18:06:50.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.541+0000 7f85dc010800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6adf10 0x557fdf6ae330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.541+0000 7f84c3e07640 1 -- 172.21.15.50:0/4080233049 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f85c8017070 con 0x557fdf5a4a40 2024-05-08T18:06:50.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.541+0000 7f85dc010800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557fdf497330 con 0x557fdf6adf10 2024-05-08T18:06:50.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.541+0000 7f85d7e77640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6adf10 0x557fdf6ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.541+0000 7f85d7e77640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6adf10 0x557fdf6ae330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45952/0 (socket says 172.21.15.50:45952) 2024-05-08T18:06:50.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.541+0000 7f85d7e77640 1 -- 172.21.15.50:0/663162832 learned_addr learned my addr 172.21.15.50:0/663162832 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.541+0000 7f84c3e07640 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f84bc03bb50 0x7f84bc03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.541+0000 7f84c3e07640 1 -- 172.21.15.50:0/4080233049 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f85c8015020 con 0x557fdf5a4a40 2024-05-08T18:06:50.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.542+0000 7f85d7676640 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f84bc03bb50 0x7f84bc03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.542+0000 7f85d7e77640 1 -- 172.21.15.50:0/663162832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fdf6ae870 con 0x557fdf6adf10 2024-05-08T18:06:50.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.542+0000 7f85d7e77640 1 --2- 172.21.15.50:0/663162832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6adf10 0x557fdf6ae330 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f85d000a430 tx=0x7f85d000a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec126fc42c895ef1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.542+0000 7f85d7676640 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f84bc03bb50 0x7f84bc03e010 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f85c801bf90 tx=0x7f85c8022e90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.542+0000 7f84c1e03640 1 -- 172.21.15.50:0/663162832 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85d0015070 con 0x557fdf6adf10 2024-05-08T18:06:50.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.542+0000 7f84c1e03640 1 -- 172.21.15.50:0/663162832 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d0001670 con 0x557fdf6adf10 2024-05-08T18:06:50.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.542+0000 7f84c1e03640 1 -- 172.21.15.50:0/663162832 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85d0001990 con 0x557fdf6adf10 2024-05-08T18:06:50.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.542+0000 7f85dc010800 1 -- 172.21.15.50:0/663162832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6adf10 msgr2=0x557fdf6ae330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.542+0000 7f85dc010800 1 --2- 172.21.15.50:0/663162832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6adf10 0x557fdf6ae330 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f85d000a430 tx=0x7f85d000a8f0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.542+0000 7f85dc010800 1 -- 172.21.15.50:0/663162832 shutdown_connections 2024-05-08T18:06:50.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.542+0000 7f85dc010800 1 --2- 172.21.15.50:0/663162832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6adf10 0x557fdf6ae330 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.542+0000 7f85dc010800 1 -- 172.21.15.50:0/663162832 >> 172.21.15.50:0/663162832 conn(0x557fdf5a89f0 msgr2=0x557fdf6aa9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.542+0000 7f85dc010800 1 -- 172.21.15.50:0/663162832 shutdown_connections 2024-05-08T18:06:50.546 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.542+0000 7f85dc010800 1 -- 172.21.15.50:0/663162832 wait complete. 2024-05-08T18:06:50.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.543+0000 7f85dc010800 1 Processor -- start 2024-05-08T18:06:50.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.543+0000 7f85dc010800 1 -- start start 2024-05-08T18:06:50.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.543+0000 7f85dc010800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6adf10 0x557fdf6b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.543+0000 7f85dc010800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557fdf6af200 con 0x557fdf6adf10 2024-05-08T18:06:50.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.543+0000 7f85d7e77640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6adf10 0x557fdf6b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.543+0000 7f85d7e77640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6adf10 0x557fdf6b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45954/0 (socket says 172.21.15.50:45954) 2024-05-08T18:06:50.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.543+0000 7f85d7e77640 1 -- 172.21.15.50:0/63290964 learned_addr learned my addr 172.21.15.50:0/63290964 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.543+0000 7f85d7e77640 1 -- 172.21.15.50:0/63290964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fdf6b58e0 con 0x557fdf6adf10 2024-05-08T18:06:50.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.544+0000 7f85d7e77640 1 --2- 172.21.15.50:0/63290964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6adf10 0x557fdf6b53a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f85d000a930 tx=0x7f85d0003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.544+0000 7f84b3fff640 1 -- 172.21.15.50:0/63290964 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85d0020020 con 0x557fdf6adf10 2024-05-08T18:06:50.547 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.544+0000 7f84b3fff640 1 -- 172.21.15.50:0/63290964 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85d00108b0 con 0x557fdf6adf10 2024-05-08T18:06:50.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.544+0000 7f85dc010800 1 -- 172.21.15.50:0/63290964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557fdf6b5b70 con 0x557fdf6adf10 2024-05-08T18:06:50.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.544+0000 7f84b3fff640 1 -- 172.21.15.50:0/63290964 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85d0015040 con 0x557fdf6adf10 2024-05-08T18:06:50.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.544+0000 7f85dc010800 1 -- 172.21.15.50:0/63290964 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fdf6b8820 con 0x557fdf6adf10 2024-05-08T18:06:50.548 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.545+0000 7f84b3fff640 1 -- 172.21.15.50:0/63290964 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f85d0023020 con 0x557fdf6adf10 2024-05-08T18:06:50.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.545+0000 7f84b3fff640 1 --2- 172.21.15.50:0/63290964 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f84b803bb50 0x7f84b803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.545+0000 7f84b3fff640 1 -- 172.21.15.50:0/63290964 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f85d0052400 con 0x557fdf6adf10 2024-05-08T18:06:50.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.545+0000 7f85d6e75640 1 --2- 172.21.15.50:0/63290964 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f84b803bb50 0x7f84b803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.561 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.545+0000 7f85dc010800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:50.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.545+0000 7f85dc010800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:50.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.546+0000 7f85dc010800 1 Processor -- start 2024-05-08T18:06:50.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.546+0000 7f85d6e75640 1 --2- 172.21.15.50:0/63290964 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f84b803bb50 0x7f84b803e010 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f85cc054000 tx=0x7f85cc04e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.546+0000 7f85dc010800 1 -- start start 2024-05-08T18:06:50.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.546+0000 7f85dc010800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6cfa80 0x557fdf6cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.546+0000 7f85dc010800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557fdf696340 con 0x557fdf6cfa80 2024-05-08T18:06:50.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.546+0000 7f85d7676640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6cfa80 0x557fdf6cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.546+0000 7f85d7676640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6cfa80 0x557fdf6cfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45970/0 (socket says 172.21.15.50:45970) 2024-05-08T18:06:50.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.546+0000 7f85d7676640 1 -- 172.21.15.50:0/1537084373 learned_addr learned my addr 172.21.15.50:0/1537084373 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.547+0000 7f85d7676640 1 -- 172.21.15.50:0/1537084373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557fdf6d03e0 con 0x557fdf6cfa80 2024-05-08T18:06:50.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.547+0000 7f85d7676640 1 --2- 172.21.15.50:0/1537084373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6cfa80 0x557fdf6cfea0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f85c800b5f0 tx=0x7f85c8023d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.547+0000 7f8467fff640 1 -- 172.21.15.50:0/1537084373 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85c801b3d0 con 0x557fdf6cfa80 2024-05-08T18:06:50.562 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.547+0000 7f8467fff640 1 -- 172.21.15.50:0/1537084373 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c80236e0 con 0x557fdf6cfa80 2024-05-08T18:06:50.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.547+0000 7f8467fff640 1 -- 172.21.15.50:0/1537084373 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85c8022470 con 0x557fdf6cfa80 2024-05-08T18:06:50.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.547+0000 7f85dc010800 1 -- 172.21.15.50:0/1537084373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557fdf6d0e50 con 0x557fdf6cfa80 2024-05-08T18:06:50.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.547+0000 7f85dc010800 1 -- 172.21.15.50:0/1537084373 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557fdf6d0e50 con 0x557fdf6cfa80 2024-05-08T18:06:50.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.548+0000 7f8467fff640 1 -- 172.21.15.50:0/1537084373 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f85c800e070 con 0x557fdf6cfa80 2024-05-08T18:06:50.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.548+0000 7f8467fff640 1 --2- 172.21.15.50:0/1537084373 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f841803bba0 0x7f841803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.548+0000 7f8467fff640 1 -- 172.21.15.50:0/1537084373 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f85c8044030 con 0x557fdf6cfa80 2024-05-08T18:06:50.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.548+0000 7f85d6e75640 1 --2- 172.21.15.50:0/1537084373 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f841803bba0 0x7f841803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.548+0000 7f8465ffb640 20 reqs_thread_entry: start 2024-05-08T18:06:50.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.548+0000 7f85d6e75640 1 --2- 172.21.15.50:0/1537084373 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f841803bba0 0x7f841803e060 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f85cc04b010 tx=0x7f85cc005ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.563 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.548+0000 7f85dc010800 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557fdf6dd310 0x557fdf6df700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.548+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6dfd20 con 0x557fdf6dd310 2024-05-08T18:06:50.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.549+0000 7f85d7e77640 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557fdf6dd310 0x557fdf6df700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.549+0000 7f85d7e77640 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557fdf6dd310 0x557fdf6df700 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.549+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f85d000e070 con 0x557fdf6dd310 2024-05-08T18:06:50.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.549+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6e0600 con 0x557fdf6dd310 2024-05-08T18:06:50.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.549+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f85d000e070 con 0x557fdf6dd310 2024-05-08T18:06:50.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.549+0000 7f85dc010800 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557fdf6e1f10 0x557fdf6e4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.549+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6e4b10 con 0x557fdf6e1f10 2024-05-08T18:06:50.564 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.550+0000 7f85d7676640 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557fdf6e1f10 0x557fdf6e4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.550+0000 7f85d7676640 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557fdf6e1f10 0x557fdf6e4380 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.550+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f85c802e070 con 0x557fdf6e1f10 2024-05-08T18:06:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.550+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6f1c30 con 0x557fdf6dd310 2024-05-08T18:06:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.551+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f85d000e070 con 0x557fdf6dd310 2024-05-08T18:06:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:NOTICE: set zone's realm_id= 2024-05-08T18:06:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.551+0000 7f85dc010800 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557fdf6e8ff0 0x557fdf6ebf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.551+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ebaa0 con 0x557fdf6e8ff0 2024-05-08T18:06:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.551+0000 7f85d6e75640 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557fdf6e8ff0 0x557fdf6ebf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.551+0000 7f85d6e75640 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557fdf6e8ff0 0x557fdf6ebf20 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.551+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85cc066030 con 0x557fdf6e8ff0 2024-05-08T18:06:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.551+0000 7f85d6e75640 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f85cc06a040 0x7f85cc06b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.551+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:6 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ebaa0 con 0x7f85cc06a040 2024-05-08T18:06:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.551+0000 7f85d7e77640 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f85cc06a040 0x7f85cc06b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.552+0000 7f85d7e77640 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f85cc06a040 0x7f85cc06b940 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.565 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.552+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85d000e070 con 0x7f85cc06a040 2024-05-08T18:06:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.552+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:7 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ebaa0 con 0x557fdf6e1f10 2024-05-08T18:06:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.552+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85c802e070 con 0x557fdf6e1f10 2024-05-08T18:06:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.552+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:8 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ebaa0 con 0x557fdf6dd310 2024-05-08T18:06:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.553+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85d000e070 con 0x557fdf6dd310 2024-05-08T18:06:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.553+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ebaa0 con 0x557fdf6dd310 2024-05-08T18:06:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.553+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85d000e070 con 0x557fdf6dd310 2024-05-08T18:06:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.553+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ebaa0 con 0x557fdf6dd310 2024-05-08T18:06:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.553+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85d000e070 con 0x557fdf6dd310 2024-05-08T18:06:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.553+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:11 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ebaa0 con 0x557fdf6e1f10 2024-05-08T18:06:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.553+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(11 [pgnls start_epoch 29 out=107b] v27'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f85c802e070 con 0x557fdf6e1f10 2024-05-08T18:06:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.553+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:12 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ebaa0 con 0x7f85cc06a040 2024-05-08T18:06:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.553+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(12 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85d000e070 con 0x7f85cc06a040 2024-05-08T18:06:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.553+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:13 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ebaa0 con 0x557fdf6e8ff0 2024-05-08T18:06:50.566 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.554+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85cc066030 con 0x557fdf6e8ff0 2024-05-08T18:06:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.554+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:14 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ebaa0 con 0x557fdf6dd310 2024-05-08T18:06:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.554+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85d000e070 con 0x557fdf6dd310 2024-05-08T18:06:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.554+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:15 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ebaa0 con 0x557fdf6e1f10 2024-05-08T18:06:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.554+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85c802e070 con 0x557fdf6e1f10 2024-05-08T18:06:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.554+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:16 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ebaa0 con 0x557fdf6dd310 2024-05-08T18:06:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.555+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85d000e070 con 0x557fdf6dd310 2024-05-08T18:06:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.555+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:17 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ebaa0 con 0x7f85cc06a040 2024-05-08T18:06:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.555+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(17 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f85d000e070 con 0x7f85cc06a040 2024-05-08T18:06:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.555+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:18 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ebaa0 con 0x557fdf6e8ff0 2024-05-08T18:06:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.555+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85cc066030 con 0x557fdf6e8ff0 2024-05-08T18:06:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.555+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ebaa0 con 0x557fdf6e8ff0 2024-05-08T18:06:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.555+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85cc066030 con 0x557fdf6e8ff0 2024-05-08T18:06:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.555+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:20 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x557fdf6e1f10 2024-05-08T18:06:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.556+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85c802e070 con 0x557fdf6e1f10 2024-05-08T18:06:50.567 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.556+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:21 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x557fdf6e8ff0 2024-05-08T18:06:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.556+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85cc066030 con 0x557fdf6e8ff0 2024-05-08T18:06:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.556+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x557fdf6e8ff0 2024-05-08T18:06:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.556+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(22 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f85cc066030 con 0x557fdf6e8ff0 2024-05-08T18:06:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.556+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:23 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x557fdf6dd310 2024-05-08T18:06:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.557+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(23 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85d000e070 con 0x557fdf6dd310 2024-05-08T18:06:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.557+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x557fdf6dd310 2024-05-08T18:06:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.557+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=74b] v20'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f85d000e070 con 0x557fdf6dd310 2024-05-08T18:06:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.557+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:25 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x557fdf6e1f10 2024-05-08T18:06:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.557+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(25 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85c802e070 con 0x557fdf6e1f10 2024-05-08T18:06:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.557+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:26 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x557fdf6e8ff0 2024-05-08T18:06:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.557+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85cc066030 con 0x557fdf6e8ff0 2024-05-08T18:06:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.557+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:27 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x7f85cc06a040 2024-05-08T18:06:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.558+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85d000e070 con 0x7f85cc06a040 2024-05-08T18:06:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.558+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:28 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x557fdf6dd310 2024-05-08T18:06:50.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.558+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85d000e070 con 0x557fdf6dd310 2024-05-08T18:06:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.558+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:29 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x557fdf6e1f10 2024-05-08T18:06:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.558+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85c802e070 con 0x557fdf6e1f10 2024-05-08T18:06:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.558+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:30 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x557fdf6e8ff0 2024-05-08T18:06:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.558+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85cc066030 con 0x557fdf6e8ff0 2024-05-08T18:06:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.558+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:31 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x557fdf6e1f10 2024-05-08T18:06:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.559+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85c802e070 con 0x557fdf6e1f10 2024-05-08T18:06:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.559+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:32 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x7f85cc06a040 2024-05-08T18:06:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.559+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85d000e070 con 0x7f85cc06a040 2024-05-08T18:06:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.559+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:33 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x557fdf6e8ff0 2024-05-08T18:06:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.559+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85cc066030 con 0x557fdf6e8ff0 2024-05-08T18:06:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.559+0000 7f85d6e75640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:34 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x557fdf6e1f10 2024-05-08T18:06:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.559+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85c802e070 con 0x557fdf6e1f10 2024-05-08T18:06:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.559+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:35 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x557fdf6dd310 2024-05-08T18:06:50.569 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.560+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(35 [pgnls start_epoch 29 out=115b] v20'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f85d000e070 con 0x557fdf6dd310 2024-05-08T18:06:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.560+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:36 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f85cc078040 con 0x7f85cc06a040 2024-05-08T18:06:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.560+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(36 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f85d000e070 con 0x7f85cc06a040 2024-05-08T18:06:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.560+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:37 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6efb00 con 0x557fdf6dd310 2024-05-08T18:06:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.560+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(37 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f85d000e070 con 0x557fdf6dd310 2024-05-08T18:06:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.560+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:38 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ee480 con 0x557fdf6e1f10 2024-05-08T18:06:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.561+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(38 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f85c802e070 con 0x557fdf6e1f10 2024-05-08T18:06:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.561+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:39 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [create,call version.set in=58b,writefull 0~1155 in=1155b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6eb330 con 0x557fdf6e1f10 2024-05-08T18:06:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.562+0000 7f85d7676640 1 -- 172.21.15.50:0/4080233049 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(39 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [create,call,writefull 0~1155] v29'3 uv3 ondisk = 0) ==== 274+0+0 (crc 0 0 0) 0x7f85c802e070 con 0x557fdf6e1f10 2024-05-08T18:06:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.562+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:40 5.f 5:f43b6ece:::zone_names.default:head [create,call version.set in=58b,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6ed0d0 con 0x557fdf6dd310 2024-05-08T18:06:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.563+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(40 zone_names.default [create,call,writefull 0~46] v29'3 uv3 ondisk = 0) ==== 246+0+0 (crc 0 0 0) 0x7f85d000e070 con 0x557fdf6dd310 2024-05-08T18:06:50.570 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.563+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:41 5.19 5:9f02474b:::default.zone.:head [create,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557fdf6f14d0 con 0x557fdf6dd310 2024-05-08T18:06:50.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.564+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(41 default.zone. [create,writefull 0~46] v29'2 uv1 ondisk = -17 ((17) File exists)) ==== 199+0+0 (crc 0 0 0) 0x7f85d000e070 con 0x557fdf6dd310 2024-05-08T18:06:50.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 -- 172.21.15.50:0/63290964 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f84b803bb50 msgr2=0x7f84b803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 --2- 172.21.15.50:0/63290964 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f84b803bb50 0x7f84b803e010 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f85cc054000 tx=0x7f85cc04e000 comp rx=0 tx=0).stop 2024-05-08T18:06:50.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 -- 172.21.15.50:0/63290964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6adf10 msgr2=0x557fdf6b53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 --2- 172.21.15.50:0/63290964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6adf10 0x557fdf6b53a0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f85d000a930 tx=0x7f85d0003370 comp rx=0 tx=0).stop 2024-05-08T18:06:50.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 -- 172.21.15.50:0/63290964 shutdown_connections 2024-05-08T18:06:50.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 --2- 172.21.15.50:0/63290964 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f84b803bb50 0x7f84b803e010 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 --2- 172.21.15.50:0/63290964 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6adf10 0x557fdf6b53a0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 -- 172.21.15.50:0/63290964 >> 172.21.15.50:0/63290964 conn(0x557fdf5a89f0 msgr2=0x557fdf6a7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 -- 172.21.15.50:0/63290964 shutdown_connections 2024-05-08T18:06:50.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 -- 172.21.15.50:0/63290964 wait complete. 2024-05-08T18:06:50.574 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 -- 172.21.15.50:0/1537084373 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f841803bba0 msgr2=0x7f841803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 --2- 172.21.15.50:0/1537084373 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f841803bba0 0x7f841803e060 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f85cc04b010 tx=0x7f85cc005ac0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 -- 172.21.15.50:0/1537084373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6cfa80 msgr2=0x557fdf6cfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 --2- 172.21.15.50:0/1537084373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6cfa80 0x557fdf6cfea0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f85c800b5f0 tx=0x7f85c8023d80 comp rx=0 tx=0).stop 2024-05-08T18:06:50.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 -- 172.21.15.50:0/1537084373 shutdown_connections 2024-05-08T18:06:50.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 --2- 172.21.15.50:0/1537084373 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f841803bba0 0x7f841803e060 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.565+0000 7f85dc010800 1 --2- 172.21.15.50:0/1537084373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf6cfa80 0x557fdf6cfea0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 -- 172.21.15.50:0/1537084373 >> 172.21.15.50:0/1537084373 conn(0x557fdf6c4530 msgr2=0x557fdf6c4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 -- 172.21.15.50:0/1537084373 shutdown_connections 2024-05-08T18:06:50.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 -- 172.21.15.50:0/1537084373 wait complete. 2024-05-08T18:06:50.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557fdf6e1f10 msgr2=0x557fdf6e4380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557fdf6e1f10 0x557fdf6e4380 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557fdf6dd310 msgr2=0x557fdf6df700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557fdf6dd310 0x557fdf6df700 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f85cc06a040 msgr2=0x7f85cc06b940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f85cc06a040 0x7f85cc06b940 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557fdf6e8ff0 msgr2=0x557fdf6ebf20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x557fdf6e8ff0 0x557fdf6ebf20 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f84bc03bb50 msgr2=0x7f84bc03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f84bc03bb50 0x7f84bc03e010 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f85c801bf90 tx=0x7f85c8022e90 comp rx=0 tx=0).stop 2024-05-08T18:06:50.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85d7e77640 1 -- 172.21.15.50:0/4080233049 reap_dead start 2024-05-08T18:06:50.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a4a40 msgr2=0x557fdf6a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a4a40 0x557fdf6a0880 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f85c8000c00 tx=0x7f85c8000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 shutdown_connections 2024-05-08T18:06:50.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 --2- 172.21.15.50:0/4080233049 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557fdf5a4a40 0x557fdf6a0880 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 >> 172.21.15.50:0/4080233049 conn(0x557fdf59f2c0 msgr2=0x557fdf5a37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 shutdown_connections 2024-05-08T18:06:50.577 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.566+0000 7f85dc010800 1 -- 172.21.15.50:0/4080233049 wait complete. 2024-05-08T18:06:50.578 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"1c041ffd-e40a-486a-88e5-3547b85128ea","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}},{"key":"new-placement","val":{"index_pool":".rgw.buckets.index.2","storage_classes":{"STANDARD":{"data_pool":".rgw.buckets.2"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-05-08T18:06:50.578 DEBUG:tasks.util.rgw: json result: {'id': '1c041ffd-e40a-486a-88e5-3547b85128ea', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-05-08T18:06:50.578 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get'] 2024-05-08T18:06:50.578 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get'] 2024-05-08T18:06:50.578 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get 2024-05-08T18:06:50.661 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:50.662 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:50.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.664+0000 7f6989c13800 1 Processor -- start 2024-05-08T18:06:50.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.664+0000 7f6989c13800 1 -- start start 2024-05-08T18:06:50.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.664+0000 7f6989c13800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x556bfd54cad0 0x556bfd54eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.664+0000 7f6989c13800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x556bfd54f490 con 0x556bfd5486e0 2024-05-08T18:06:50.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.664+0000 7f6989c13800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x556bfd3e8e20 con 0x556bfd54cad0 2024-05-08T18:06:50.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.665+0000 7f6984a19640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x556bfd54cad0 0x556bfd54eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.665+0000 7f6984a19640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x556bfd54cad0 0x556bfd54eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45978/0 (socket says 172.21.15.50:45978) 2024-05-08T18:06:50.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.665+0000 7f6984a19640 1 -- 172.21.15.50:0/4078493765 learned_addr learned my addr 172.21.15.50:0/4078493765 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.665+0000 7f698521a640 1 --1- 172.21.15.50:0/4078493765 >> v1:172.21.15.46:6789/0 conn(0x556bfd5486e0 0x556bfd548b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50754/0 (socket says 172.21.15.50:50754) 2024-05-08T18:06:50.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.665+0000 7f6984a19640 1 -- 172.21.15.50:0/4078493765 >> v1:172.21.15.46:6789/0 conn(0x556bfd5486e0 legacy=0x556bfd548b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.665+0000 7f6984a19640 1 -- 172.21.15.50:0/4078493765 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bfd5506c0 con 0x556bfd54cad0 2024-05-08T18:06:50.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.667+0000 7f6984a19640 1 --2- 172.21.15.50:0/4078493765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd54cad0 0x556bfd54eec0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f697c00a520 tx=0x7f697c01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14eaf1ed45b6ce43 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.667+0000 7f6977fff640 1 -- 172.21.15.50:0/4078493765 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f697c057070 con 0x556bfd54cad0 2024-05-08T18:06:50.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.667+0000 7f6977fff640 1 -- 172.21.15.50:0/4078493765 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697c052570 con 0x556bfd54cad0 2024-05-08T18:06:50.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.667+0000 7f6977fff640 1 -- 172.21.15.50:0/4078493765 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f697c04ae60 con 0x556bfd54cad0 2024-05-08T18:06:50.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.668+0000 7f6989c13800 1 -- 172.21.15.50:0/4078493765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd54cad0 msgr2=0x556bfd54eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.668+0000 7f6989c13800 1 --2- 172.21.15.50:0/4078493765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd54cad0 0x556bfd54eec0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f697c00a520 tx=0x7f697c01d010 comp rx=0 tx=0).stop 2024-05-08T18:06:50.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.668+0000 7f6989c13800 1 -- 172.21.15.50:0/4078493765 shutdown_connections 2024-05-08T18:06:50.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.668+0000 7f6989c13800 1 --2- 172.21.15.50:0/4078493765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd54cad0 0x556bfd54eec0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.668+0000 7f6989c13800 1 -- 172.21.15.50:0/4078493765 >> 172.21.15.50:0/4078493765 conn(0x556bfd543c60 msgr2=0x556bfd5460a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.668+0000 7f6989c13800 1 -- 172.21.15.50:0/4078493765 shutdown_connections 2024-05-08T18:06:50.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.668+0000 7f6989c13800 1 -- 172.21.15.50:0/4078493765 wait complete. 2024-05-08T18:06:50.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.683+0000 7f6989c13800 1 Processor -- start 2024-05-08T18:06:50.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.683+0000 7f6989c13800 1 -- start start 2024-05-08T18:06:50.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.683+0000 7f6989c13800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd5486e0 0x556bfd4bded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.683+0000 7f6989c13800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556bfd493a00 con 0x556bfd5486e0 2024-05-08T18:06:50.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.683+0000 7f698521a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd5486e0 0x556bfd4bded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.683+0000 7f698521a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd5486e0 0x556bfd4bded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45982/0 (socket says 172.21.15.50:45982) 2024-05-08T18:06:50.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.683+0000 7f698521a640 1 -- 172.21.15.50:0/2279769903 learned_addr learned my addr 172.21.15.50:0/2279769903 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.684+0000 7f698521a640 1 -- 172.21.15.50:0/2279769903 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bfd4bc490 con 0x556bfd5486e0 2024-05-08T18:06:50.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.684+0000 7f698521a640 1 --2- 172.21.15.50:0/2279769903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd5486e0 0x556bfd4bded0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f697800b260 tx=0x7f697800b660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e492a88321f595 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.684+0000 7f6873606640 1 -- 172.21.15.50:0/2279769903 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f697800be60 con 0x556bfd5486e0 2024-05-08T18:06:50.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.684+0000 7f6873606640 1 -- 172.21.15.50:0/2279769903 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6978006070 con 0x556bfd5486e0 2024-05-08T18:06:50.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.684+0000 7f6873606640 1 -- 172.21.15.50:0/2279769903 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f697800e710 con 0x556bfd5486e0 2024-05-08T18:06:50.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.685+0000 7f6989c13800 1 -- 172.21.15.50:0/2279769903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd5486e0 msgr2=0x556bfd4bded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.685+0000 7f6989c13800 1 --2- 172.21.15.50:0/2279769903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd5486e0 0x556bfd4bded0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f697800b260 tx=0x7f697800b660 comp rx=0 tx=0).stop 2024-05-08T18:06:50.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.685+0000 7f6989c13800 1 -- 172.21.15.50:0/2279769903 shutdown_connections 2024-05-08T18:06:50.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.685+0000 7f6989c13800 1 --2- 172.21.15.50:0/2279769903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd5486e0 0x556bfd4bded0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.685+0000 7f6989c13800 1 -- 172.21.15.50:0/2279769903 >> 172.21.15.50:0/2279769903 conn(0x556bfd543c60 msgr2=0x556bfd635420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.685+0000 7f6989c13800 1 -- 172.21.15.50:0/2279769903 shutdown_connections 2024-05-08T18:06:50.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.685+0000 7f6989c13800 1 -- 172.21.15.50:0/2279769903 wait complete. 2024-05-08T18:06:50.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.685+0000 7f6989c13800 1 Processor -- start 2024-05-08T18:06:50.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.685+0000 7f6989c13800 1 -- start start 2024-05-08T18:06:50.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.685+0000 7f6989c13800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd5486e0 0x556bfd6438f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.685+0000 7f6989c13800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556bfd54f960 con 0x556bfd5486e0 2024-05-08T18:06:50.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.686+0000 7f698521a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd5486e0 0x556bfd6438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.686+0000 7f698521a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd5486e0 0x556bfd6438f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:45990/0 (socket says 172.21.15.50:45990) 2024-05-08T18:06:50.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.686+0000 7f698521a640 1 -- 172.21.15.50:0/3908503312 learned_addr learned my addr 172.21.15.50:0/3908503312 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.686+0000 7f698521a640 1 -- 172.21.15.50:0/3908503312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bfd643e30 con 0x556bfd5486e0 2024-05-08T18:06:50.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.686+0000 7f698521a640 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd5486e0 0x556bfd6438f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x556bfd4bd500 tx=0x7f6978000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.686+0000 7f6871e03640 1 -- 172.21.15.50:0/3908503312 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f697801a7c0 con 0x556bfd5486e0 2024-05-08T18:06:50.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.686+0000 7f6871e03640 1 -- 172.21.15.50:0/3908503312 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f697801a980 con 0x556bfd5486e0 2024-05-08T18:06:50.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.686+0000 7f6989c13800 1 -- 172.21.15.50:0/3908503312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556bfd6440c0 con 0x556bfd5486e0 2024-05-08T18:06:50.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.686+0000 7f6871e03640 1 -- 172.21.15.50:0/3908503312 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f697801aca0 con 0x556bfd5486e0 2024-05-08T18:06:50.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.686+0000 7f6989c13800 1 -- 172.21.15.50:0/3908503312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bfd646d20 con 0x556bfd5486e0 2024-05-08T18:06:50.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.687+0000 7f6989c13800 1 Processor -- start 2024-05-08T18:06:50.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.687+0000 7f6989c13800 1 -- start start 2024-05-08T18:06:50.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.687+0000 7f6989c13800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd650f30 0x556bfd651350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.687+0000 7f6989c13800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556bfd43a330 con 0x556bfd650f30 2024-05-08T18:06:50.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.687+0000 7f6871e03640 1 -- 172.21.15.50:0/3908503312 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6978017070 con 0x556bfd5486e0 2024-05-08T18:06:50.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.687+0000 7f6985a1b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd650f30 0x556bfd651350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.687+0000 7f6985a1b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd650f30 0x556bfd651350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46004/0 (socket says 172.21.15.50:46004) 2024-05-08T18:06:50.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.687+0000 7f6985a1b640 1 -- 172.21.15.50:0/2833114836 learned_addr learned my addr 172.21.15.50:0/2833114836 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.688+0000 7f6871e03640 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f686c03bb50 0x7f686c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.688+0000 7f6871e03640 1 -- 172.21.15.50:0/3908503312 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f6978015020 con 0x556bfd5486e0 2024-05-08T18:06:50.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.688+0000 7f6985a1b640 1 -- 172.21.15.50:0/2833114836 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bfd651890 con 0x556bfd650f30 2024-05-08T18:06:50.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.688+0000 7f698521a640 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f686c03bb50 0x7f686c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.688+0000 7f6985a1b640 1 --2- 172.21.15.50:0/2833114836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd650f30 0x556bfd651350 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f698000a430 tx=0x7f698000a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e44a2dcdf35782 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.688+0000 7f686b7fe640 1 -- 172.21.15.50:0/2833114836 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6980015070 con 0x556bfd650f30 2024-05-08T18:06:50.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.688+0000 7f698521a640 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f686c03bb50 0x7f686c03e010 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f697800a010 tx=0x7f6978035040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.688+0000 7f686b7fe640 1 -- 172.21.15.50:0/2833114836 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6980001670 con 0x556bfd650f30 2024-05-08T18:06:50.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.688+0000 7f686b7fe640 1 -- 172.21.15.50:0/2833114836 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6980001990 con 0x556bfd650f30 2024-05-08T18:06:50.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.688+0000 7f6989c13800 1 -- 172.21.15.50:0/2833114836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd650f30 msgr2=0x556bfd651350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.688+0000 7f6989c13800 1 --2- 172.21.15.50:0/2833114836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd650f30 0x556bfd651350 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f698000a430 tx=0x7f698000a8f0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.689+0000 7f6989c13800 1 -- 172.21.15.50:0/2833114836 shutdown_connections 2024-05-08T18:06:50.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.689+0000 7f6989c13800 1 --2- 172.21.15.50:0/2833114836 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd650f30 0x556bfd651350 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.689+0000 7f6989c13800 1 -- 172.21.15.50:0/2833114836 >> 172.21.15.50:0/2833114836 conn(0x556bfd54cad0 msgr2=0x556bfd64d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.689+0000 7f6989c13800 1 -- 172.21.15.50:0/2833114836 shutdown_connections 2024-05-08T18:06:50.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.689+0000 7f6989c13800 1 -- 172.21.15.50:0/2833114836 wait complete. 2024-05-08T18:06:50.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.689+0000 7f6989c13800 1 Processor -- start 2024-05-08T18:06:50.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.689+0000 7f6989c13800 1 -- start start 2024-05-08T18:06:50.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.689+0000 7f6989c13800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd650f30 0x556bfd658420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.689+0000 7f6989c13800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556bfd652220 con 0x556bfd650f30 2024-05-08T18:06:50.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.689+0000 7f6985a1b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd650f30 0x556bfd658420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.689+0000 7f6985a1b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd650f30 0x556bfd658420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46008/0 (socket says 172.21.15.50:46008) 2024-05-08T18:06:50.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.689+0000 7f6985a1b640 1 -- 172.21.15.50:0/3970373325 learned_addr learned my addr 172.21.15.50:0/3970373325 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.690+0000 7f6985a1b640 1 -- 172.21.15.50:0/3970373325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bfd658960 con 0x556bfd650f30 2024-05-08T18:06:50.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.690+0000 7f6985a1b640 1 --2- 172.21.15.50:0/3970373325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd650f30 0x556bfd658420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f698000a930 tx=0x7f6980003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.690+0000 7f6869ffb640 1 -- 172.21.15.50:0/3970373325 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6980020020 con 0x556bfd650f30 2024-05-08T18:06:50.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.690+0000 7f6869ffb640 1 -- 172.21.15.50:0/3970373325 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69800108b0 con 0x556bfd650f30 2024-05-08T18:06:50.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.690+0000 7f6989c13800 1 -- 172.21.15.50:0/3970373325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556bfd658bf0 con 0x556bfd650f30 2024-05-08T18:06:50.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.690+0000 7f6869ffb640 1 -- 172.21.15.50:0/3970373325 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6980015040 con 0x556bfd650f30 2024-05-08T18:06:50.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.690+0000 7f6989c13800 1 -- 172.21.15.50:0/3970373325 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bfd65b8a0 con 0x556bfd650f30 2024-05-08T18:06:50.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.691+0000 7f6869ffb640 1 -- 172.21.15.50:0/3970373325 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6980023020 con 0x556bfd650f30 2024-05-08T18:06:50.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.691+0000 7f6869ffb640 1 --2- 172.21.15.50:0/3970373325 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f685c03bba0 0x7f685c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.691+0000 7f6869ffb640 1 -- 172.21.15.50:0/3970373325 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f6980052390 con 0x556bfd650f30 2024-05-08T18:06:50.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.691+0000 7f6984a19640 1 --2- 172.21.15.50:0/3970373325 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f685c03bba0 0x7f685c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.692+0000 7f6989c13800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:50.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.692+0000 7f6989c13800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:50.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.692+0000 7f6984a19640 1 --2- 172.21.15.50:0/3970373325 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f685c03bba0 0x7f685c03e060 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f697c054000 tx=0x7f697c04e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.692+0000 7f6989c13800 1 Processor -- start 2024-05-08T18:06:50.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.692+0000 7f6989c13800 1 -- start start 2024-05-08T18:06:50.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.692+0000 7f6989c13800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd672aa0 0x556bfd672ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.692+0000 7f6989c13800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556bfd4bc810 con 0x556bfd672aa0 2024-05-08T18:06:50.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.692+0000 7f698521a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd672aa0 0x556bfd672ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.692+0000 7f698521a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd672aa0 0x556bfd672ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46022/0 (socket says 172.21.15.50:46022) 2024-05-08T18:06:50.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.692+0000 7f698521a640 1 -- 172.21.15.50:0/4209024427 learned_addr learned my addr 172.21.15.50:0/4209024427 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.693+0000 7f698521a640 1 -- 172.21.15.50:0/4209024427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556bfd673400 con 0x556bfd672aa0 2024-05-08T18:06:50.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.693+0000 7f698521a640 1 --2- 172.21.15.50:0/4209024427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd672aa0 0x556bfd672ec0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6978051860 tx=0x7f6978009410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.693+0000 7f684a7e4640 1 -- 172.21.15.50:0/4209024427 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f697804a070 con 0x556bfd672aa0 2024-05-08T18:06:50.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.693+0000 7f684a7e4640 1 -- 172.21.15.50:0/4209024427 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6978009d50 con 0x556bfd672aa0 2024-05-08T18:06:50.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.693+0000 7f6989c13800 1 -- 172.21.15.50:0/4209024427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556bfd673e70 con 0x556bfd672aa0 2024-05-08T18:06:50.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.693+0000 7f684a7e4640 1 -- 172.21.15.50:0/4209024427 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f697801d8d0 con 0x556bfd672aa0 2024-05-08T18:06:50.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.693+0000 7f6989c13800 1 -- 172.21.15.50:0/4209024427 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556bfd674370 con 0x556bfd672aa0 2024-05-08T18:06:50.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.694+0000 7f684a7e4640 1 -- 172.21.15.50:0/4209024427 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f697804d070 con 0x556bfd672aa0 2024-05-08T18:06:50.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.694+0000 7f684a7e4640 1 --2- 172.21.15.50:0/4209024427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f67bc03bb50 0x7f67bc03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.694+0000 7f684a7e4640 1 -- 172.21.15.50:0/4209024427 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f6978040040 con 0x556bfd672aa0 2024-05-08T18:06:50.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.694+0000 7f6984a19640 1 --2- 172.21.15.50:0/4209024427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f67bc03bb50 0x7f67bc03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.694+0000 7f68437fe640 20 reqs_thread_entry: start 2024-05-08T18:06:50.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.695+0000 7f6989c13800 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556bfd6804f0 0x556bfd682940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.695+0000 7f6989c13800 1 -- 172.21.15.50:0/3908503312 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556bfd682fc0 con 0x556bfd6804f0 2024-05-08T18:06:50.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.695+0000 7f6984a19640 1 --2- 172.21.15.50:0/4209024427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f67bc03bb50 0x7f67bc03e010 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f697c005930 tx=0x7f697c005ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.695+0000 7f6985a1b640 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556bfd6804f0 0x556bfd682940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.695+0000 7f6985a1b640 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556bfd6804f0 0x556bfd682940 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.695+0000 7f6985a1b640 1 -- 172.21.15.50:0/3908503312 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6980010d70 con 0x556bfd6804f0 2024-05-08T18:06:50.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.695+0000 7f6989c13800 1 -- 172.21.15.50:0/3908503312 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556bfd6838a0 con 0x556bfd6804f0 2024-05-08T18:06:50.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.696+0000 7f6985a1b640 1 -- 172.21.15.50:0/3908503312 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6980010d70 con 0x556bfd6804f0 2024-05-08T18:06:50.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.696+0000 7f6989c13800 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556bfd6851b0 0x556bfd687600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.696+0000 7f6989c13800 1 -- 172.21.15.50:0/3908503312 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556bfd687d90 con 0x556bfd6851b0 2024-05-08T18:06:50.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.696+0000 7f698521a640 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556bfd6851b0 0x556bfd687600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.696+0000 7f698521a640 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556bfd6851b0 0x556bfd687600 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.697+0000 7f698521a640 1 -- 172.21.15.50:0/3908503312 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~1155 out=1155b] v0'0 uv3 ondisk = 0) ==== 232+0+1203 (crc 0 0 0) 0x7f697804c030 con 0x556bfd6851b0 2024-05-08T18:06:50.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 -- 172.21.15.50:0/3970373325 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f685c03bba0 msgr2=0x7f685c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 --2- 172.21.15.50:0/3970373325 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f685c03bba0 0x7f685c03e060 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f697c054000 tx=0x7f697c04e000 comp rx=0 tx=0).stop 2024-05-08T18:06:50.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 -- 172.21.15.50:0/3970373325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd650f30 msgr2=0x556bfd658420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 --2- 172.21.15.50:0/3970373325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd650f30 0x556bfd658420 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f698000a930 tx=0x7f6980003370 comp rx=0 tx=0).stop 2024-05-08T18:06:50.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 -- 172.21.15.50:0/3970373325 shutdown_connections 2024-05-08T18:06:50.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 --2- 172.21.15.50:0/3970373325 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f685c03bba0 0x7f685c03e060 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 --2- 172.21.15.50:0/3970373325 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd650f30 0x556bfd658420 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 -- 172.21.15.50:0/3970373325 >> 172.21.15.50:0/3970373325 conn(0x556bfd54cad0 msgr2=0x556bfd64b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 -- 172.21.15.50:0/3970373325 shutdown_connections 2024-05-08T18:06:50.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 -- 172.21.15.50:0/3970373325 wait complete. 2024-05-08T18:06:50.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 -- 172.21.15.50:0/4209024427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f67bc03bb50 msgr2=0x7f67bc03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 --2- 172.21.15.50:0/4209024427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f67bc03bb50 0x7f67bc03e010 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f697c005930 tx=0x7f697c005ac0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 -- 172.21.15.50:0/4209024427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd672aa0 msgr2=0x556bfd672ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 --2- 172.21.15.50:0/4209024427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd672aa0 0x556bfd672ec0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6978051860 tx=0x7f6978009410 comp rx=0 tx=0).stop 2024-05-08T18:06:50.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 -- 172.21.15.50:0/4209024427 shutdown_connections 2024-05-08T18:06:50.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 --2- 172.21.15.50:0/4209024427 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f67bc03bb50 0x7f67bc03e010 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 --2- 172.21.15.50:0/4209024427 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd672aa0 0x556bfd672ec0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 -- 172.21.15.50:0/4209024427 >> 172.21.15.50:0/4209024427 conn(0x556bfd667550 msgr2=0x556bfd667f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 -- 172.21.15.50:0/4209024427 shutdown_connections 2024-05-08T18:06:50.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.698+0000 7f6989c13800 1 -- 172.21.15.50:0/4209024427 wait complete. 2024-05-08T18:06:50.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.699+0000 7f6989c13800 1 -- 172.21.15.50:0/3908503312 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556bfd6851b0 msgr2=0x556bfd687600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.699+0000 7f6989c13800 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556bfd6851b0 0x556bfd687600 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.699+0000 7f6989c13800 1 -- 172.21.15.50:0/3908503312 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556bfd6804f0 msgr2=0x556bfd682940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.699+0000 7f6989c13800 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556bfd6804f0 0x556bfd682940 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.699+0000 7f6989c13800 1 -- 172.21.15.50:0/3908503312 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f686c03bb50 msgr2=0x7f686c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.699+0000 7f6989c13800 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f686c03bb50 0x7f686c03e010 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f697800a010 tx=0x7f6978035040 comp rx=0 tx=0).stop 2024-05-08T18:06:50.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.699+0000 7f6989c13800 1 -- 172.21.15.50:0/3908503312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd5486e0 msgr2=0x556bfd6438f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.699+0000 7f6989c13800 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd5486e0 0x556bfd6438f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x556bfd4bd500 tx=0x7f6978000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.699+0000 7f6989c13800 1 -- 172.21.15.50:0/3908503312 shutdown_connections 2024-05-08T18:06:50.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.699+0000 7f6989c13800 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556bfd6851b0 0x556bfd687600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.699+0000 7f6989c13800 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556bfd6804f0 0x556bfd682940 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.699+0000 7f6989c13800 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f686c03bb50 0x7f686c03e010 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.699+0000 7f6989c13800 1 --2- 172.21.15.50:0/3908503312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556bfd5486e0 0x556bfd6438f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.699+0000 7f6989c13800 1 -- 172.21.15.50:0/3908503312 >> 172.21.15.50:0/3908503312 conn(0x556bfd543c60 msgr2=0x556bfd4bcd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.699+0000 7f6989c13800 1 -- 172.21.15.50:0/3908503312 shutdown_connections 2024-05-08T18:06:50.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.699+0000 7f6989c13800 1 -- 172.21.15.50:0/3908503312 wait complete. 2024-05-08T18:06:50.706 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"1c041ffd-e40a-486a-88e5-3547b85128ea","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}},{"key":"new-placement","val":{"index_pool":".rgw.buckets.index.2","storage_classes":{"STANDARD":{"data_pool":".rgw.buckets.2"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-05-08T18:06:50.707 DEBUG:tasks.util.rgw: json result: {'id': '1c041ffd-e40a-486a-88e5-3547b85128ea', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-05-08T18:06:50.707 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-05-08T18:06:50.707 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-05-08T18:06:50.707 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2024-05-08T18:06:50.795 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:50.795 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:50.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.798+0000 7fbf9612f800 1 Processor -- start 2024-05-08T18:06:50.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.798+0000 7fbf9612f800 1 -- start start 2024-05-08T18:06:50.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.798+0000 7fbf9612f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x564959ef97a0 0x564959ef9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.798+0000 7fbf9612f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x564959efa240 con 0x564959efa960 2024-05-08T18:06:50.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.798+0000 7fbf9612f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x564959d99e20 con 0x564959ef97a0 2024-05-08T18:06:50.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.798+0000 7fbf91da7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x564959ef97a0 0x564959ef9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.798+0000 7fbf915a6640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x564959efa960 0x564959efade0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50770/0 (socket says 172.21.15.50:50770) 2024-05-08T18:06:50.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.798+0000 7fbf91da7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x564959ef97a0 0x564959ef9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46036/0 (socket says 172.21.15.50:46036) 2024-05-08T18:06:50.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.798+0000 7fbf915a6640 1 -- 172.21.15.50:0/589910484 learned_addr learned my addr 172.21.15.50:0/589910484 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:50.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.798+0000 7fbf91da7640 1 -- 172.21.15.50:0/589910484 >> v1:172.21.15.46:6789/0 conn(0x564959efa960 legacy=0x564959efade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.798+0000 7fbf91da7640 1 -- 172.21.15.50:0/589910484 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x564959efb520 con 0x564959ef97a0 2024-05-08T18:06:50.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.800+0000 7fbf91da7640 1 --2- 172.21.15.50:0/589910484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 0x564959ef9bc0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbf84005fd0 tx=0x7fbf8401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be65e30d2ca9b06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.800+0000 7fbf90da5640 1 -- 172.21.15.50:0/589910484 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf84052560 con 0x564959ef97a0 2024-05-08T18:06:50.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.800+0000 7fbf90da5640 1 -- 172.21.15.50:0/589910484 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf84052700 con 0x564959ef97a0 2024-05-08T18:06:50.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.800+0000 7fbf90da5640 1 -- 172.21.15.50:0/589910484 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf84052a60 con 0x564959ef97a0 2024-05-08T18:06:50.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.801+0000 7fbf9612f800 1 -- 172.21.15.50:0/589910484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 msgr2=0x564959ef9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.801+0000 7fbf9612f800 1 --2- 172.21.15.50:0/589910484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 0x564959ef9bc0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbf84005fd0 tx=0x7fbf8401ccc0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.801+0000 7fbf9612f800 1 -- 172.21.15.50:0/589910484 shutdown_connections 2024-05-08T18:06:50.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.801+0000 7fbf9612f800 1 --2- 172.21.15.50:0/589910484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 0x564959ef9bc0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.801+0000 7fbf9612f800 1 -- 172.21.15.50:0/589910484 >> 172.21.15.50:0/589910484 conn(0x564959ef4dc0 msgr2=0x564959ef71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.801+0000 7fbf9612f800 1 -- 172.21.15.50:0/589910484 shutdown_connections 2024-05-08T18:06:50.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.801+0000 7fbf9612f800 1 -- 172.21.15.50:0/589910484 wait complete. 2024-05-08T18:06:50.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.816+0000 7fbf9612f800 1 Processor -- start 2024-05-08T18:06:50.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.817+0000 7fbf9612f800 1 -- start start 2024-05-08T18:06:50.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.817+0000 7fbf9612f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 0x564959feac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.817+0000 7fbf9612f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564959e44a30 con 0x564959ef97a0 2024-05-08T18:06:50.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.817+0000 7fbf91da7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 0x564959feac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.817+0000 7fbf91da7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 0x564959feac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46050/0 (socket says 172.21.15.50:46050) 2024-05-08T18:06:50.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.817+0000 7fbf91da7640 1 -- 172.21.15.50:0/44487365 learned_addr learned my addr 172.21.15.50:0/44487365 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.817+0000 7fbf91da7640 1 -- 172.21.15.50:0/44487365 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564959feb260 con 0x564959ef97a0 2024-05-08T18:06:50.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.817+0000 7fbf91da7640 1 --2- 172.21.15.50:0/44487365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 0x564959feac90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbf84054040 tx=0x7fbf8404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533cc4f6285c2d16 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.818+0000 7fbe7fe07640 1 -- 172.21.15.50:0/44487365 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf840529e0 con 0x564959ef97a0 2024-05-08T18:06:50.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.818+0000 7fbe7fe07640 1 -- 172.21.15.50:0/44487365 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf84052b80 con 0x564959ef97a0 2024-05-08T18:06:50.816 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.818+0000 7fbe7fe07640 1 -- 172.21.15.50:0/44487365 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf8405c430 con 0x564959ef97a0 2024-05-08T18:06:50.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.818+0000 7fbf9612f800 1 -- 172.21.15.50:0/44487365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 msgr2=0x564959feac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.818+0000 7fbf9612f800 1 --2- 172.21.15.50:0/44487365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 0x564959feac90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbf84054040 tx=0x7fbf8404c000 comp rx=0 tx=0).stop 2024-05-08T18:06:50.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.818+0000 7fbf9612f800 1 -- 172.21.15.50:0/44487365 shutdown_connections 2024-05-08T18:06:50.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.818+0000 7fbf9612f800 1 --2- 172.21.15.50:0/44487365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 0x564959feac90 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.818+0000 7fbf9612f800 1 -- 172.21.15.50:0/44487365 >> 172.21.15.50:0/44487365 conn(0x564959ef4dc0 msgr2=0x564959fe65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.818+0000 7fbf9612f800 1 -- 172.21.15.50:0/44487365 shutdown_connections 2024-05-08T18:06:50.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.818+0000 7fbf9612f800 1 -- 172.21.15.50:0/44487365 wait complete. 2024-05-08T18:06:50.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.819+0000 7fbf9612f800 1 Processor -- start 2024-05-08T18:06:50.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.819+0000 7fbf9612f800 1 -- start start 2024-05-08T18:06:50.825 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.819+0000 7fbf9612f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 0x564959ff27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.819+0000 7fbf9612f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564959f02e30 con 0x564959ef97a0 2024-05-08T18:06:50.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.819+0000 7fbf91da7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 0x564959ff27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.819+0000 7fbf91da7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 0x564959ff27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46064/0 (socket says 172.21.15.50:46064) 2024-05-08T18:06:50.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.819+0000 7fbf91da7640 1 -- 172.21.15.50:0/495641622 learned_addr learned my addr 172.21.15.50:0/495641622 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.819+0000 7fbf91da7640 1 -- 172.21.15.50:0/495641622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564959ff2ce0 con 0x564959ef97a0 2024-05-08T18:06:50.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.820+0000 7fbf91da7640 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 0x564959ff27a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbf84044810 tx=0x7fbf84003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.820+0000 7fbe7e604640 1 -- 172.21.15.50:0/495641622 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf84066410 con 0x564959ef97a0 2024-05-08T18:06:50.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.820+0000 7fbe7e604640 1 -- 172.21.15.50:0/495641622 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf840665b0 con 0x564959ef97a0 2024-05-08T18:06:50.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.820+0000 7fbf9612f800 1 -- 172.21.15.50:0/495641622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564959ff2f70 con 0x564959ef97a0 2024-05-08T18:06:50.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.820+0000 7fbe7e604640 1 -- 172.21.15.50:0/495641622 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf84066890 con 0x564959ef97a0 2024-05-08T18:06:50.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.820+0000 7fbf9612f800 1 -- 172.21.15.50:0/495641622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564959ff5bd0 con 0x564959ef97a0 2024-05-08T18:06:50.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.821+0000 7fbf9612f800 1 Processor -- start 2024-05-08T18:06:50.826 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.821+0000 7fbf9612f800 1 -- start start 2024-05-08T18:06:50.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.821+0000 7fbe7e604640 1 -- 172.21.15.50:0/495641622 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fbf84078020 con 0x564959ef97a0 2024-05-08T18:06:50.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.821+0000 7fbf9612f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959fffe50 0x56495a000270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.821+0000 7fbf9612f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564959deb330 con 0x564959fffe50 2024-05-08T18:06:50.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.821+0000 7fbe7e604640 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbe7803bba0 0x7fbe7803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.821+0000 7fbe7e604640 1 -- 172.21.15.50:0/495641622 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fbf84094ee0 con 0x564959ef97a0 2024-05-08T18:06:50.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.821+0000 7fbf91da7640 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbe7803bba0 0x7fbe7803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.821+0000 7fbf925a8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959fffe50 0x56495a000270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.821+0000 7fbf925a8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959fffe50 0x56495a000270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46076/0 (socket says 172.21.15.50:46076) 2024-05-08T18:06:50.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.821+0000 7fbf925a8640 1 -- 172.21.15.50:0/1993993646 learned_addr learned my addr 172.21.15.50:0/1993993646 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.821+0000 7fbf91da7640 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbe7803bba0 0x7fbe7803e060 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf84079790 tx=0x7fbf84071f90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.821+0000 7fbf925a8640 1 -- 172.21.15.50:0/1993993646 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56495a0007b0 con 0x564959fffe50 2024-05-08T18:06:50.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.822+0000 7fbf925a8640 1 --2- 172.21.15.50:0/1993993646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959fffe50 0x56495a000270 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00a400 tx=0x7fbf8c00a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e4f66b7c4af81 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.822+0000 7fbe77fff640 1 -- 172.21.15.50:0/1993993646 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf8c0087e0 con 0x564959fffe50 2024-05-08T18:06:50.827 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.822+0000 7fbe77fff640 1 -- 172.21.15.50:0/1993993646 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8c0089a0 con 0x564959fffe50 2024-05-08T18:06:50.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.822+0000 7fbe77fff640 1 -- 172.21.15.50:0/1993993646 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf8c008d00 con 0x564959fffe50 2024-05-08T18:06:50.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.822+0000 7fbf9612f800 1 -- 172.21.15.50:0/1993993646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959fffe50 msgr2=0x56495a000270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.822+0000 7fbf9612f800 1 --2- 172.21.15.50:0/1993993646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959fffe50 0x56495a000270 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c00a400 tx=0x7fbf8c00a8c0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.822+0000 7fbf9612f800 1 -- 172.21.15.50:0/1993993646 shutdown_connections 2024-05-08T18:06:50.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.822+0000 7fbf9612f800 1 --2- 172.21.15.50:0/1993993646 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959fffe50 0x56495a000270 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.822+0000 7fbf9612f800 1 -- 172.21.15.50:0/1993993646 >> 172.21.15.50:0/1993993646 conn(0x564959efa960 msgr2=0x564959ffc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.822+0000 7fbf9612f800 1 -- 172.21.15.50:0/1993993646 shutdown_connections 2024-05-08T18:06:50.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.822+0000 7fbf9612f800 1 -- 172.21.15.50:0/1993993646 wait complete. 2024-05-08T18:06:50.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.823+0000 7fbf9612f800 1 Processor -- start 2024-05-08T18:06:50.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.823+0000 7fbf9612f800 1 -- start start 2024-05-08T18:06:50.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.823+0000 7fbf9612f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959fffe50 0x56495a007380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.823+0000 7fbf9612f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56495a001140 con 0x564959fffe50 2024-05-08T18:06:50.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.823+0000 7fbf925a8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959fffe50 0x56495a007380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.823+0000 7fbf925a8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959fffe50 0x56495a007380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46086/0 (socket says 172.21.15.50:46086) 2024-05-08T18:06:50.828 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.823+0000 7fbf925a8640 1 -- 172.21.15.50:0/1285348852 learned_addr learned my addr 172.21.15.50:0/1285348852 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.823+0000 7fbf925a8640 1 -- 172.21.15.50:0/1285348852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56495a0078c0 con 0x564959fffe50 2024-05-08T18:06:50.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.823+0000 7fbf925a8640 1 --2- 172.21.15.50:0/1285348852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959fffe50 0x56495a007380 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x56495a000c00 tx=0x7fbf8c002ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.824+0000 7fbe767fc640 1 -- 172.21.15.50:0/1285348852 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf8c022020 con 0x564959fffe50 2024-05-08T18:06:50.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.824+0000 7fbe767fc640 1 -- 172.21.15.50:0/1285348852 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8c019040 con 0x564959fffe50 2024-05-08T18:06:50.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.824+0000 7fbf9612f800 1 -- 172.21.15.50:0/1285348852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56495a007b50 con 0x564959fffe50 2024-05-08T18:06:50.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.824+0000 7fbe767fc640 1 -- 172.21.15.50:0/1285348852 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf8c013950 con 0x564959fffe50 2024-05-08T18:06:50.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.824+0000 7fbf9612f800 1 -- 172.21.15.50:0/1285348852 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56495a00a800 con 0x564959fffe50 2024-05-08T18:06:50.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.824+0000 7fbe767fc640 1 -- 172.21.15.50:0/1285348852 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fbf8c025020 con 0x564959fffe50 2024-05-08T18:06:50.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.825+0000 7fbe767fc640 1 --2- 172.21.15.50:0/1285348852 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbe6c03bba0 0x7fbe6c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.825+0000 7fbe767fc640 1 -- 172.21.15.50:0/1285348852 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fbf8c0527d0 con 0x564959fffe50 2024-05-08T18:06:50.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.825+0000 7fbf915a6640 1 --2- 172.21.15.50:0/1285348852 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbe6c03bba0 0x7fbe6c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.825+0000 7fbf9612f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:50.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.825+0000 7fbf9612f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:50.829 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.825+0000 7fbf915a6640 1 --2- 172.21.15.50:0/1285348852 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbe6c03bba0 0x7fbe6c03e060 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x564959ffbd70 tx=0x7fbf88001250 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.826+0000 7fbf9612f800 1 Processor -- start 2024-05-08T18:06:50.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.826+0000 7fbf9612f800 1 -- start start 2024-05-08T18:06:50.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.826+0000 7fbf9612f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495a021a90 0x56495a021eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.826+0000 7fbf9612f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564959feb520 con 0x56495a021a90 2024-05-08T18:06:50.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.826+0000 7fbf91da7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495a021a90 0x56495a021eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.826+0000 7fbf91da7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495a021a90 0x56495a021eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46088/0 (socket says 172.21.15.50:46088) 2024-05-08T18:06:50.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.826+0000 7fbf91da7640 1 -- 172.21.15.50:0/4238325625 learned_addr learned my addr 172.21.15.50:0/4238325625 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.826+0000 7fbf91da7640 1 -- 172.21.15.50:0/4238325625 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56495a0223f0 con 0x56495a021a90 2024-05-08T18:06:50.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.827+0000 7fbf91da7640 1 --2- 172.21.15.50:0/4238325625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495a021a90 0x56495a021eb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbf8405bf70 tx=0x7fbf8401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.827+0000 7fbe56fe5640 1 -- 172.21.15.50:0/4238325625 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf8406f900 con 0x56495a021a90 2024-05-08T18:06:50.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.827+0000 7fbe56fe5640 1 -- 172.21.15.50:0/4238325625 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8406fb40 con 0x56495a021a90 2024-05-08T18:06:50.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.827+0000 7fbf9612f800 1 -- 172.21.15.50:0/4238325625 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56495a022e60 con 0x56495a021a90 2024-05-08T18:06:50.830 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.827+0000 7fbe56fe5640 1 -- 172.21.15.50:0/4238325625 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf84070970 con 0x56495a021a90 2024-05-08T18:06:50.831 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.827+0000 7fbf9612f800 1 -- 172.21.15.50:0/4238325625 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56495a023360 con 0x56495a021a90 2024-05-08T18:06:50.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.827+0000 7fbe56fe5640 1 -- 172.21.15.50:0/4238325625 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fbf84099020 con 0x56495a021a90 2024-05-08T18:06:50.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.828+0000 7fbe56fe5640 1 --2- 172.21.15.50:0/4238325625 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbdd003bb00 0x7fbdd003dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.828+0000 7fbe56fe5640 1 -- 172.21.15.50:0/4238325625 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fbf84097020 con 0x56495a021a90 2024-05-08T18:06:50.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.828+0000 7fbf915a6640 1 --2- 172.21.15.50:0/4238325625 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbdd003bb00 0x7fbdd003dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.828+0000 7fbe54fe1640 20 reqs_thread_entry: start 2024-05-08T18:06:50.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.828+0000 7fbf9612f800 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56495a02f490 0x56495a031900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.828+0000 7fbf915a6640 1 --2- 172.21.15.50:0/4238325625 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbdd003bb00 0x7fbdd003dfc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbf88001ed0 tx=0x7fbf8800f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.828+0000 7fbf9612f800 1 -- 172.21.15.50:0/495641622 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56495a031fd0 con 0x56495a02f490 2024-05-08T18:06:50.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.828+0000 7fbf925a8640 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56495a02f490 0x56495a031900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.829+0000 7fbf925a8640 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56495a02f490 0x56495a031900 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.829+0000 7fbf925a8640 1 -- 172.21.15.50:0/495641622 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbf8c01e420 con 0x56495a02f490 2024-05-08T18:06:50.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.829+0000 7fbf9612f800 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56495a033da0 0x56495a036190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.829+0000 7fbf9612f800 1 -- 172.21.15.50:0/495641622 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:2 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56495a036880 con 0x56495a033da0 2024-05-08T18:06:50.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.829+0000 7fbf91da7640 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56495a033da0 0x56495a036190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.829+0000 7fbf91da7640 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56495a033da0 0x56495a036190 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.830+0000 7fbf91da7640 1 -- 172.21.15.50:0/495641622 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(2 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~1155 out=1155b] v0'0 uv3 ondisk = 0) ==== 232+0+1203 (crc 0 0 0) 0x7fbf840648c0 con 0x56495a033da0 2024-05-08T18:06:50.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.830+0000 7fbf9612f800 1 -- 172.21.15.50:0/495641622 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [stat,call version.check_conds in=122b,call version.inc in=38b,writefull 0~993 in=993b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56495a033470 con 0x56495a033da0 2024-05-08T18:06:50.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.831+0000 7fbf91da7640 1 -- 172.21.15.50:0/495641622 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [stat,call,call,writefull 0~993] v29'4 uv4 ondisk = 0) ==== 316+0+0 (crc 0 0 0) 0x7fbf840648c0 con 0x56495a033da0 2024-05-08T18:06:50.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.832+0000 7fbf9612f800 1 -- 172.21.15.50:0/1285348852 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbe6c03bba0 msgr2=0x7fbe6c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.832+0000 7fbf9612f800 1 --2- 172.21.15.50:0/1285348852 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbe6c03bba0 0x7fbe6c03e060 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x564959ffbd70 tx=0x7fbf88001250 comp rx=0 tx=0).stop 2024-05-08T18:06:50.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.832+0000 7fbf9612f800 1 -- 172.21.15.50:0/1285348852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959fffe50 msgr2=0x56495a007380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.832+0000 7fbf9612f800 1 --2- 172.21.15.50:0/1285348852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959fffe50 0x56495a007380 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x56495a000c00 tx=0x7fbf8c002ff0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.832+0000 7fbf9612f800 1 -- 172.21.15.50:0/1285348852 shutdown_connections 2024-05-08T18:06:50.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.832+0000 7fbf9612f800 1 --2- 172.21.15.50:0/1285348852 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbe6c03bba0 0x7fbe6c03e060 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.832+0000 7fbf9612f800 1 --2- 172.21.15.50:0/1285348852 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959fffe50 0x56495a007380 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.832+0000 7fbf9612f800 1 -- 172.21.15.50:0/1285348852 >> 172.21.15.50:0/1285348852 conn(0x564959efa960 msgr2=0x564959ff9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.832+0000 7fbf9612f800 1 -- 172.21.15.50:0/1285348852 shutdown_connections 2024-05-08T18:06:50.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.832+0000 7fbf9612f800 1 -- 172.21.15.50:0/1285348852 wait complete. 2024-05-08T18:06:50.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.832+0000 7fbf9612f800 1 -- 172.21.15.50:0/4238325625 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbdd003bb00 msgr2=0x7fbdd003dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.832+0000 7fbf9612f800 1 --2- 172.21.15.50:0/4238325625 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbdd003bb00 0x7fbdd003dfc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbf88001ed0 tx=0x7fbf8800f000 comp rx=0 tx=0).stop 2024-05-08T18:06:50.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.832+0000 7fbf9612f800 1 -- 172.21.15.50:0/4238325625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495a021a90 msgr2=0x56495a021eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.832+0000 7fbf9612f800 1 --2- 172.21.15.50:0/4238325625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495a021a90 0x56495a021eb0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbf8405bf70 tx=0x7fbf8401ccc0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 -- 172.21.15.50:0/4238325625 shutdown_connections 2024-05-08T18:06:50.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 --2- 172.21.15.50:0/4238325625 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbdd003bb00 0x7fbdd003dfc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 --2- 172.21.15.50:0/4238325625 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495a021a90 0x56495a021eb0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 -- 172.21.15.50:0/4238325625 >> 172.21.15.50:0/4238325625 conn(0x56495a016520 msgr2=0x56495a016f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.837 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 -- 172.21.15.50:0/4238325625 shutdown_connections 2024-05-08T18:06:50.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 -- 172.21.15.50:0/4238325625 wait complete. 2024-05-08T18:06:50.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 -- 172.21.15.50:0/495641622 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56495a033da0 msgr2=0x56495a036190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56495a033da0 0x56495a036190 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 -- 172.21.15.50:0/495641622 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56495a02f490 msgr2=0x56495a031900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56495a02f490 0x56495a031900 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 -- 172.21.15.50:0/495641622 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbe7803bba0 msgr2=0x7fbe7803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbe7803bba0 0x7fbe7803e060 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf84079790 tx=0x7fbf84071f90 comp rx=0 tx=0).stop 2024-05-08T18:06:50.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 -- 172.21.15.50:0/495641622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 msgr2=0x564959ff27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 0x564959ff27a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbf84044810 tx=0x7fbf84003330 comp rx=0 tx=0).stop 2024-05-08T18:06:50.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 -- 172.21.15.50:0/495641622 shutdown_connections 2024-05-08T18:06:50.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56495a033da0 0x56495a036190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56495a02f490 0x56495a031900 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.838 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fbe7803bba0 0x7fbe7803e060 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 --2- 172.21.15.50:0/495641622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564959ef97a0 0x564959ff27a0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 -- 172.21.15.50:0/495641622 >> 172.21.15.50:0/495641622 conn(0x564959ef4dc0 msgr2=0x564959fe4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 -- 172.21.15.50:0/495641622 shutdown_connections 2024-05-08T18:06:50.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.833+0000 7fbf9612f800 1 -- 172.21.15.50:0/495641622 wait complete. 2024-05-08T18:06:50.840 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"1c041ffd-e40a-486a-88e5-3547b85128ea","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-05-08T18:06:50.841 DEBUG:tasks.util.rgw: json result: {'id': '1c041ffd-e40a-486a-88e5-3547b85128ea', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-05-08T18:06:50.841 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'get'] 2024-05-08T18:06:50.841 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'get'] 2024-05-08T18:06:50.841 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup get 2024-05-08T18:06:50.884 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:50.884 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:50.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.887+0000 7f6b2e31f800 1 Processor -- start 2024-05-08T18:06:50.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.887+0000 7f6b2e31f800 1 -- start start 2024-05-08T18:06:50.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.887+0000 7f6b2e31f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55801350d540 0x55801350d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.887+0000 7f6b2e31f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55801350dfe0 con 0x55801350e7a0 2024-05-08T18:06:50.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.887+0000 7f6b2e31f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5580133aee20 con 0x55801350d540 2024-05-08T18:06:50.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.888+0000 7f6b29fbc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55801350d540 0x55801350d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.888+0000 7f6b29fbc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55801350d540 0x55801350d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46102/0 (socket says 172.21.15.50:46102) 2024-05-08T18:06:50.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.888+0000 7f6b29fbc640 1 -- 172.21.15.50:0/2703068092 learned_addr learned my addr 172.21.15.50:0/2703068092 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.888+0000 7f6b297bb640 1 --1- 172.21.15.50:0/2703068092 >> v1:172.21.15.46:6789/0 conn(0x55801350e7a0 0x55801350ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50780/0 (socket says 172.21.15.50:50780) 2024-05-08T18:06:50.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.888+0000 7f6b29fbc640 1 -- 172.21.15.50:0/2703068092 >> v1:172.21.15.46:6789/0 conn(0x55801350e7a0 legacy=0x55801350ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.887 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.888+0000 7f6b29fbc640 1 -- 172.21.15.50:0/2703068092 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55801350f360 con 0x55801350d540 2024-05-08T18:06:50.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.890+0000 7f6b29fbc640 1 --2- 172.21.15.50:0/2703068092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 0x55801350d960 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00a520 tx=0x7f6b1c01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4535e73b16e955a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.890+0000 7f6b28fba640 1 -- 172.21.15.50:0/2703068092 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b1c057070 con 0x55801350d540 2024-05-08T18:06:50.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.890+0000 7f6b28fba640 1 -- 172.21.15.50:0/2703068092 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1c052570 con 0x55801350d540 2024-05-08T18:06:50.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.890+0000 7f6b28fba640 1 -- 172.21.15.50:0/2703068092 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b1c052870 con 0x55801350d540 2024-05-08T18:06:50.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.890+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2703068092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 msgr2=0x55801350d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.890+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/2703068092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 0x55801350d960 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00a520 tx=0x7f6b1c01d010 comp rx=0 tx=0).stop 2024-05-08T18:06:50.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.890+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2703068092 shutdown_connections 2024-05-08T18:06:50.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.890+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/2703068092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 0x55801350d960 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.890+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2703068092 >> 172.21.15.50:0/2703068092 conn(0x558013508b40 msgr2=0x55801350af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.891+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2703068092 shutdown_connections 2024-05-08T18:06:50.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.891+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2703068092 wait complete. 2024-05-08T18:06:50.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.906+0000 7f6b2e31f800 1 Processor -- start 2024-05-08T18:06:50.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.906+0000 7f6b2e31f800 1 -- start start 2024-05-08T18:06:50.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.906+0000 7f6b2e31f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 0x5580135fb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.906+0000 7f6b2e31f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558013459a00 con 0x55801350d540 2024-05-08T18:06:50.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.906+0000 7f6b29fbc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 0x5580135fb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.906+0000 7f6b29fbc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 0x5580135fb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46106/0 (socket says 172.21.15.50:46106) 2024-05-08T18:06:50.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.906+0000 7f6b29fbc640 1 -- 172.21.15.50:0/2130946321 learned_addr learned my addr 172.21.15.50:0/2130946321 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.906+0000 7f6b29fbc640 1 -- 172.21.15.50:0/2130946321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580135f9f40 con 0x55801350d540 2024-05-08T18:06:50.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.907+0000 7f6b29fbc640 1 --2- 172.21.15.50:0/2130946321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 0x5580135fb980 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c055040 tx=0x7f6b1c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9c190f03f28f0f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.907+0000 7f6a17e07640 1 -- 172.21.15.50:0/2130946321 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b1c057040 con 0x55801350d540 2024-05-08T18:06:50.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.907+0000 7f6a17e07640 1 -- 172.21.15.50:0/2130946321 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1c052870 con 0x55801350d540 2024-05-08T18:06:50.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.907+0000 7f6a17e07640 1 -- 172.21.15.50:0/2130946321 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b1c052b90 con 0x55801350d540 2024-05-08T18:06:50.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.908+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2130946321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 msgr2=0x5580135fb980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.908+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/2130946321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 0x5580135fb980 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c055040 tx=0x7f6b1c04c000 comp rx=0 tx=0).stop 2024-05-08T18:06:50.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.908+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2130946321 shutdown_connections 2024-05-08T18:06:50.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.908+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/2130946321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 0x5580135fb980 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.908+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2130946321 >> 172.21.15.50:0/2130946321 conn(0x558013508b40 msgr2=0x5580135fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.908+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2130946321 shutdown_connections 2024-05-08T18:06:50.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.908+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2130946321 wait complete. 2024-05-08T18:06:50.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.908+0000 7f6b2e31f800 1 Processor -- start 2024-05-08T18:06:50.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.908+0000 7f6b2e31f800 1 -- start start 2024-05-08T18:06:50.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.908+0000 7f6b2e31f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 0x5580136098f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.908+0000 7f6b2e31f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558013516c70 con 0x55801350d540 2024-05-08T18:06:50.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.908+0000 7f6b29fbc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 0x5580136098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.908+0000 7f6b29fbc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 0x5580136098f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46116/0 (socket says 172.21.15.50:46116) 2024-05-08T18:06:50.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.908+0000 7f6b29fbc640 1 -- 172.21.15.50:0/960485884 learned_addr learned my addr 172.21.15.50:0/960485884 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.909+0000 7f6b29fbc640 1 -- 172.21.15.50:0/960485884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558013609e30 con 0x55801350d540 2024-05-08T18:06:50.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.909+0000 7f6b29fbc640 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 0x5580136098f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c052540 tx=0x7f6b1c003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.909+0000 7f6a16604640 1 -- 172.21.15.50:0/960485884 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b1c073070 con 0x55801350d540 2024-05-08T18:06:50.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.909+0000 7f6a16604640 1 -- 172.21.15.50:0/960485884 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1c05d570 con 0x55801350d540 2024-05-08T18:06:50.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.909+0000 7f6b2e31f800 1 -- 172.21.15.50:0/960485884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55801360a0c0 con 0x55801350d540 2024-05-08T18:06:50.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.909+0000 7f6a16604640 1 -- 172.21.15.50:0/960485884 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b1c057070 con 0x55801350d540 2024-05-08T18:06:50.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.909+0000 7f6b2e31f800 1 -- 172.21.15.50:0/960485884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55801360cd20 con 0x55801350d540 2024-05-08T18:06:50.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.910+0000 7f6b2e31f800 1 Processor -- start 2024-05-08T18:06:50.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.910+0000 7f6b2e31f800 1 -- start start 2024-05-08T18:06:50.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.910+0000 7f6a16604640 1 -- 172.21.15.50:0/960485884 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6b1c076070 con 0x55801350d540 2024-05-08T18:06:50.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.910+0000 7f6b2e31f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013616f30 0x558013617350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.910+0000 7f6b2e31f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558013400330 con 0x558013616f30 2024-05-08T18:06:50.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.910+0000 7f6a16604640 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a1003bb50 0x7f6a1003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.910+0000 7f6b2a7bd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013616f30 0x558013617350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.910+0000 7f6b2a7bd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013616f30 0x558013617350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46122/0 (socket says 172.21.15.50:46122) 2024-05-08T18:06:50.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.910+0000 7f6b2a7bd640 1 -- 172.21.15.50:0/3241672698 learned_addr learned my addr 172.21.15.50:0/3241672698 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.910+0000 7f6a16604640 1 -- 172.21.15.50:0/960485884 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f6b1c094250 con 0x55801350d540 2024-05-08T18:06:50.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.910+0000 7f6b29fbc640 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a1003bb50 0x7f6a1003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.910+0000 7f6b2a7bd640 1 -- 172.21.15.50:0/3241672698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558013617890 con 0x558013616f30 2024-05-08T18:06:50.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.911+0000 7f6b2a7bd640 1 --2- 172.21.15.50:0/3241672698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013616f30 0x558013617350 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400a430 tx=0x7f6b2400a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16cef50d7f391773 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.911+0000 7f6b29fbc640 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a1003bb50 0x7f6a1003e010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c066fd0 tx=0x7f6b1c05cee0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.911+0000 7f6a0ffff640 1 -- 172.21.15.50:0/3241672698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b24015070 con 0x558013616f30 2024-05-08T18:06:50.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.911+0000 7f6a0ffff640 1 -- 172.21.15.50:0/3241672698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b24001670 con 0x558013616f30 2024-05-08T18:06:50.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.911+0000 7f6a0ffff640 1 -- 172.21.15.50:0/3241672698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b24001990 con 0x558013616f30 2024-05-08T18:06:50.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.911+0000 7f6b2e31f800 1 -- 172.21.15.50:0/3241672698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013616f30 msgr2=0x558013617350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.911+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/3241672698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013616f30 0x558013617350 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400a430 tx=0x7f6b2400a8f0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.911+0000 7f6b2e31f800 1 -- 172.21.15.50:0/3241672698 shutdown_connections 2024-05-08T18:06:50.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.911+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/3241672698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013616f30 0x558013617350 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.911+0000 7f6b2e31f800 1 -- 172.21.15.50:0/3241672698 >> 172.21.15.50:0/3241672698 conn(0x55801350e7a0 msgr2=0x5580136139d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.911+0000 7f6b2e31f800 1 -- 172.21.15.50:0/3241672698 shutdown_connections 2024-05-08T18:06:50.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.911+0000 7f6b2e31f800 1 -- 172.21.15.50:0/3241672698 wait complete. 2024-05-08T18:06:50.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.912+0000 7f6b2e31f800 1 Processor -- start 2024-05-08T18:06:50.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.912+0000 7f6b2e31f800 1 -- start start 2024-05-08T18:06:50.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.912+0000 7f6b2e31f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013616f30 0x55801361e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.912+0000 7f6b2e31f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558013618220 con 0x558013616f30 2024-05-08T18:06:50.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.912+0000 7f6b2a7bd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013616f30 0x55801361e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.912+0000 7f6b2a7bd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013616f30 0x55801361e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46134/0 (socket says 172.21.15.50:46134) 2024-05-08T18:06:50.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.912+0000 7f6b2a7bd640 1 -- 172.21.15.50:0/2911706041 learned_addr learned my addr 172.21.15.50:0/2911706041 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.912+0000 7f6b2a7bd640 1 -- 172.21.15.50:0/2911706041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55801361e980 con 0x558013616f30 2024-05-08T18:06:50.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.912+0000 7f6b2a7bd640 1 --2- 172.21.15.50:0/2911706041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013616f30 0x55801361e440 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400a930 tx=0x7f6b24003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.913+0000 7f6a0e7fc640 1 -- 172.21.15.50:0/2911706041 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b24020020 con 0x558013616f30 2024-05-08T18:06:50.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.913+0000 7f6a0e7fc640 1 -- 172.21.15.50:0/2911706041 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b240108b0 con 0x558013616f30 2024-05-08T18:06:50.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.913+0000 7f6a0e7fc640 1 -- 172.21.15.50:0/2911706041 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b24015040 con 0x558013616f30 2024-05-08T18:06:50.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.913+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2911706041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55801361ec10 con 0x558013616f30 2024-05-08T18:06:50.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.913+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2911706041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580136218c0 con 0x558013616f30 2024-05-08T18:06:50.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.913+0000 7f6a0e7fc640 1 -- 172.21.15.50:0/2911706041 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6b24023020 con 0x558013616f30 2024-05-08T18:06:50.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.914+0000 7f6a0e7fc640 1 --2- 172.21.15.50:0/2911706041 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a0403bba0 0x7f6a0403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.914+0000 7f6a0e7fc640 1 -- 172.21.15.50:0/2911706041 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f6b24052450 con 0x558013616f30 2024-05-08T18:06:50.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.914+0000 7f6b297bb640 1 --2- 172.21.15.50:0/2911706041 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a0403bba0 0x7f6a0403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.914+0000 7f6b2e31f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:50.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.914+0000 7f6b2e31f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:50.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.914+0000 7f6b297bb640 1 --2- 172.21.15.50:0/2911706041 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a0403bba0 0x7f6a0403e060 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x558013612e50 tx=0x7f6b20001360 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.915+0000 7f6b2e31f800 1 Processor -- start 2024-05-08T18:06:50.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.915+0000 7f6b2e31f800 1 -- start start 2024-05-08T18:06:50.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.915+0000 7f6b2e31f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013638ac0 0x558013638ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.915+0000 7f6b2e31f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5580135fa200 con 0x558013638ac0 2024-05-08T18:06:50.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.915+0000 7f6b29fbc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013638ac0 0x558013638ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.915+0000 7f6b29fbc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013638ac0 0x558013638ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46140/0 (socket says 172.21.15.50:46140) 2024-05-08T18:06:50.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.915+0000 7f6b29fbc640 1 -- 172.21.15.50:0/2493557088 learned_addr learned my addr 172.21.15.50:0/2493557088 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:50.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.915+0000 7f6b29fbc640 1 -- 172.21.15.50:0/2493557088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558013639420 con 0x558013638ac0 2024-05-08T18:06:50.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.915+0000 7f6b29fbc640 1 --2- 172.21.15.50:0/2493557088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013638ac0 0x558013638ee0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c006e20 tx=0x7f6b1c077a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.916+0000 7f69eefe5640 1 -- 172.21.15.50:0/2493557088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b1c05c8f0 con 0x558013638ac0 2024-05-08T18:06:50.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.916+0000 7f69eefe5640 1 -- 172.21.15.50:0/2493557088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1c08c070 con 0x558013638ac0 2024-05-08T18:06:50.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.916+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2493557088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558013639e90 con 0x558013638ac0 2024-05-08T18:06:50.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.916+0000 7f69eefe5640 1 -- 172.21.15.50:0/2493557088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6b1c0596a0 con 0x558013638ac0 2024-05-08T18:06:50.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.916+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2493557088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55801363a390 con 0x558013638ac0 2024-05-08T18:06:50.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.916+0000 7f69eefe5640 1 -- 172.21.15.50:0/2493557088 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f6b1c076a70 con 0x558013638ac0 2024-05-08T18:06:50.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.917+0000 7f69eefe5640 1 --2- 172.21.15.50:0/2493557088 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f696003bba0 0x7f696003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.917+0000 7f69eefe5640 1 -- 172.21.15.50:0/2493557088 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f6b1c05e030 con 0x558013638ac0 2024-05-08T18:06:50.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.917+0000 7f6b297bb640 1 --2- 172.21.15.50:0/2493557088 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f696003bba0 0x7f696003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.917+0000 7f69ecfe1640 20 reqs_thread_entry: start 2024-05-08T18:06:50.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.917+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5580136464c0 0x5580136488d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.917+0000 7f6b297bb640 1 --2- 172.21.15.50:0/2493557088 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f696003bba0 0x7f696003e060 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6b20000f60 tx=0x7f6b2000f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.917+0000 7f6b2e31f800 1 -- 172.21.15.50:0/960485884 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558013648f80 con 0x5580136464c0 2024-05-08T18:06:50.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.917+0000 7f6b2a7bd640 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5580136464c0 0x5580136488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.918+0000 7f6b2a7bd640 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5580136464c0 0x5580136488d0 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.918+0000 7f6b2a7bd640 1 -- 172.21.15.50:0/960485884 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6b24010d70 con 0x5580136464c0 2024-05-08T18:06:50.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.918+0000 7f6b2e31f800 1 -- 172.21.15.50:0/960485884 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x558013649500 con 0x5580136464c0 2024-05-08T18:06:50.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.918+0000 7f6b2a7bd640 1 -- 172.21.15.50:0/960485884 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6b24010d70 con 0x5580136464c0 2024-05-08T18:06:50.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.918+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55801364b120 0x55801364d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:50.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.918+0000 7f6b2e31f800 1 -- 172.21.15.50:0/960485884 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:3 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55801364dd40 con 0x55801364b120 2024-05-08T18:06:50.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.918+0000 7f6b29fbc640 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55801364b120 0x55801364d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:50.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.919+0000 7f6b29fbc640 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55801364b120 0x55801364d5b0 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:50.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.919+0000 7f6b29fbc640 1 -- 172.21.15.50:0/960485884 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(3 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f6b1c09b070 con 0x55801364b120 2024-05-08T18:06:50.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2911706041 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a0403bba0 msgr2=0x7f6a0403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/2911706041 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a0403bba0 0x7f6a0403e060 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x558013612e50 tx=0x7f6b20001360 comp rx=0 tx=0).stop 2024-05-08T18:06:50.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2911706041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013616f30 msgr2=0x55801361e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/2911706041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013616f30 0x55801361e440 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400a930 tx=0x7f6b24003370 comp rx=0 tx=0).stop 2024-05-08T18:06:50.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2911706041 shutdown_connections 2024-05-08T18:06:50.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/2911706041 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a0403bba0 0x7f6a0403e060 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/2911706041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013616f30 0x55801361e440 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2911706041 >> 172.21.15.50:0/2911706041 conn(0x55801350e7a0 msgr2=0x558013611010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2911706041 shutdown_connections 2024-05-08T18:06:50.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2911706041 wait complete. 2024-05-08T18:06:50.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2493557088 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f696003bba0 msgr2=0x7f696003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/2493557088 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f696003bba0 0x7f696003e060 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6b20000f60 tx=0x7f6b2000f000 comp rx=0 tx=0).stop 2024-05-08T18:06:50.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2493557088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013638ac0 msgr2=0x558013638ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/2493557088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013638ac0 0x558013638ee0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c006e20 tx=0x7f6b1c077a60 comp rx=0 tx=0).stop 2024-05-08T18:06:50.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2493557088 shutdown_connections 2024-05-08T18:06:50.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/2493557088 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f696003bba0 0x7f696003e060 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/2493557088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558013638ac0 0x558013638ee0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2493557088 >> 172.21.15.50:0/2493557088 conn(0x55801362d550 msgr2=0x55801362df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2493557088 shutdown_connections 2024-05-08T18:06:50.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.920+0000 7f6b2e31f800 1 -- 172.21.15.50:0/2493557088 wait complete. 2024-05-08T18:06:50.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.921+0000 7f6b2e31f800 1 -- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5580136464c0 msgr2=0x5580136488d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.921+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5580136464c0 0x5580136488d0 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.921+0000 7f6b2e31f800 1 -- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55801364b120 msgr2=0x55801364d5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.921+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55801364b120 0x55801364d5b0 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.921+0000 7f6b2e31f800 1 -- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a1003bb50 msgr2=0x7f6a1003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.921+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a1003bb50 0x7f6a1003e010 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c066fd0 tx=0x7f6b1c05cee0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.921+0000 7f6b2e31f800 1 -- 172.21.15.50:0/960485884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 msgr2=0x5580136098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:50.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.921+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 0x5580136098f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c052540 tx=0x7f6b1c003020 comp rx=0 tx=0).stop 2024-05-08T18:06:50.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.921+0000 7f6b2e31f800 1 -- 172.21.15.50:0/960485884 shutdown_connections 2024-05-08T18:06:50.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.921+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55801364b120 0x55801364d5b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.921+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5580136464c0 0x5580136488d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.921+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f6a1003bb50 0x7f6a1003e010 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.921+0000 7f6b2e31f800 1 --2- 172.21.15.50:0/960485884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55801350d540 0x5580136098f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:50.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.921+0000 7f6b2e31f800 1 -- 172.21.15.50:0/960485884 >> 172.21.15.50:0/960485884 conn(0x558013508b40 msgr2=0x5580135f8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:50.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.921+0000 7f6b2e31f800 1 -- 172.21.15.50:0/960485884 shutdown_connections 2024-05-08T18:06:50.930 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:50.921+0000 7f6b2e31f800 1 -- 172.21.15.50:0/960485884 wait complete. 2024-05-08T18:06:50.931 INFO:teuthology.orchestra.run.smithi050.stdout:{"id":"36133b23-abaa-446a-bf67-ceb1d1559ceb","name":"default","api_name":"default","is_master":true,"endpoints":[],"hostnames":[],"hostnames_s3website":[],"master_zone":"1c041ffd-e40a-486a-88e5-3547b85128ea","zones":[{"id":"1c041ffd-e40a-486a-88e5-3547b85128ea","name":"default","endpoints":[],"log_meta":false,"log_data":false,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["notification_v2","resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]} 2024-05-08T18:06:50.931 DEBUG:tasks.util.rgw: json result: {'id': '36133b23-abaa-446a-bf67-ceb1d1559ceb', 'name': 'default', 'api_name': 'default', 'is_master': True, 'endpoints': [], 'hostnames': [], 'hostnames_s3website': [], 'master_zone': '1c041ffd-e40a-486a-88e5-3547b85128ea', 'zones': [{'id': '1c041ffd-e40a-486a-88e5-3547b85128ea', 'name': 'default', 'endpoints': [], 'log_meta': False, 'log_data': False, 'bucket_index_max_shards': 11, 'read_only': False, 'tier_type': '', 'sync_from_all': True, 'sync_from': [], 'redirect_zone': '', 'supported_features': ['notification_v2', 'resharding']}], 'placement_targets': [{'name': 'default-placement', 'tags': [], 'storage_classes': ['STANDARD']}], 'default_placement': 'default-placement', 'realm_id': '', 'sync_policy': {'groups': []}, 'enabled_features': ['notification_v2', 'resharding']} 2024-05-08T18:06:50.934 INFO:teuthology.run_tasks:Running task radosgw-admin-rest... 2024-05-08T18:06:50.955 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'ada', '--display-name', 'Ms. Admin User', '--access-key', 'MH1WC2XQ1S8UISFDZC8W', '--secret', 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', '--max-buckets', '0', '--caps', 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write'] 2024-05-08T18:06:50.955 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'ada', '--display-name', 'Ms. Admin User', '--access-key', 'MH1WC2XQ1S8UISFDZC8W', '--secret', 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', '--max-buckets', '0', '--caps', 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write'] 2024-05-08T18:06:50.955 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid ada --display-name 'Ms. Admin User' --access-key MH1WC2XQ1S8UISFDZC8W --secret dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG --max-buckets 0 --caps 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write' 2024-05-08T18:06:51.027 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:51.027 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:51.028 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.030+0000 7f872cd1f800 1 Processor -- start 2024-05-08T18:06:51.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.030+0000 7f872cd1f800 1 -- start start 2024-05-08T18:06:51.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.030+0000 7f872cd1f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55d404426560 0x55d404426980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.030+0000 7f872cd1f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d404427000 con 0x55d4044277c0 2024-05-08T18:06:51.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.030+0000 7f872cd1f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55d4042c7e20 con 0x55d404426560 2024-05-08T18:06:51.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.030+0000 7f87289bc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55d404426560 0x55d404426980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.030+0000 7f87289bc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55d404426560 0x55d404426980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46148/0 (socket says 172.21.15.50:46148) 2024-05-08T18:06:51.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.030+0000 7f87281bb640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55d4044277c0 0x55d404427c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50794/0 (socket says 172.21.15.50:50794) 2024-05-08T18:06:51.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.030+0000 7f87289bc640 1 -- 172.21.15.50:0/262139483 learned_addr learned my addr 172.21.15.50:0/262139483 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.030+0000 7f87289bc640 1 -- 172.21.15.50:0/262139483 >> v1:172.21.15.46:6789/0 conn(0x55d4044277c0 legacy=0x55d404427c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.030+0000 7f87279ba640 1 -- 172.21.15.50:0/262139483 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1642481151 0 0) 0x55d404427000 con 0x55d4044277c0 2024-05-08T18:06:51.029 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.030+0000 7f87289bc640 1 -- 172.21.15.50:0/262139483 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d404428380 con 0x55d404426560 2024-05-08T18:06:51.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.032+0000 7f87289bc640 1 --2- 172.21.15.50:0/262139483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 0x55d404426980 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f871800a6b0 tx=0x7f871801eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8fadae876b4e8c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.033+0000 7f87279ba640 1 -- 172.21.15.50:0/262139483 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f871804ce60 con 0x55d404426560 2024-05-08T18:06:51.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.033+0000 7f87279ba640 1 -- 172.21.15.50:0/262139483 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871800a200 con 0x55d404426560 2024-05-08T18:06:51.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.033+0000 7f87279ba640 1 -- 172.21.15.50:0/262139483 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8718008590 con 0x55d404426560 2024-05-08T18:06:51.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.033+0000 7f872cd1f800 1 -- 172.21.15.50:0/262139483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 msgr2=0x55d404426980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.033+0000 7f872cd1f800 1 --2- 172.21.15.50:0/262139483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 0x55d404426980 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f871800a6b0 tx=0x7f871801eed0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.031 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.033+0000 7f872cd1f800 1 -- 172.21.15.50:0/262139483 shutdown_connections 2024-05-08T18:06:51.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.033+0000 7f872cd1f800 1 --2- 172.21.15.50:0/262139483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 0x55d404426980 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.033+0000 7f872cd1f800 1 -- 172.21.15.50:0/262139483 >> 172.21.15.50:0/262139483 conn(0x55d404421b00 msgr2=0x55d404423f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.033+0000 7f872cd1f800 1 -- 172.21.15.50:0/262139483 shutdown_connections 2024-05-08T18:06:51.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.033+0000 7f872cd1f800 1 -- 172.21.15.50:0/262139483 wait complete. 2024-05-08T18:06:51.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.048+0000 7f872cd1f800 1 Processor -- start 2024-05-08T18:06:51.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.049+0000 7f872cd1f800 1 -- start start 2024-05-08T18:06:51.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.049+0000 7f872cd1f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 0x55d40451aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.049+0000 7f872cd1f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d404372b20 con 0x55d404426560 2024-05-08T18:06:51.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.049+0000 7f87289bc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 0x55d40451aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.049+0000 7f87289bc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 0x55d40451aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46164/0 (socket says 172.21.15.50:46164) 2024-05-08T18:06:51.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.049+0000 7f87289bc640 1 -- 172.21.15.50:0/3684701792 learned_addr learned my addr 172.21.15.50:0/3684701792 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.049+0000 7f87289bc640 1 -- 172.21.15.50:0/3684701792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40451b470 con 0x55d404426560 2024-05-08T18:06:51.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.049+0000 7f87289bc640 1 --2- 172.21.15.50:0/3684701792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 0x55d40451aea0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8718056040 tx=0x7f871804e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba23b79a1550dad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.050+0000 7f8616e0d640 1 -- 172.21.15.50:0/3684701792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f871800a3b0 con 0x55d404426560 2024-05-08T18:06:51.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.050+0000 7f8616e0d640 1 -- 172.21.15.50:0/3684701792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8718063070 con 0x55d404426560 2024-05-08T18:06:51.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.050+0000 7f8616e0d640 1 -- 172.21.15.50:0/3684701792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8718007560 con 0x55d404426560 2024-05-08T18:06:51.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.050+0000 7f872cd1f800 1 -- 172.21.15.50:0/3684701792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 msgr2=0x55d40451aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.050+0000 7f872cd1f800 1 --2- 172.21.15.50:0/3684701792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 0x55d40451aea0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8718056040 tx=0x7f871804e000 comp rx=0 tx=0).stop 2024-05-08T18:06:51.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.050+0000 7f872cd1f800 1 -- 172.21.15.50:0/3684701792 shutdown_connections 2024-05-08T18:06:51.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.050+0000 7f872cd1f800 1 --2- 172.21.15.50:0/3684701792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 0x55d40451aea0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.050+0000 7f872cd1f800 1 -- 172.21.15.50:0/3684701792 >> 172.21.15.50:0/3684701792 conn(0x55d404421b00 msgr2=0x55d4045123b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.050+0000 7f872cd1f800 1 -- 172.21.15.50:0/3684701792 shutdown_connections 2024-05-08T18:06:51.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.050+0000 7f872cd1f800 1 -- 172.21.15.50:0/3684701792 wait complete. 2024-05-08T18:06:51.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.051+0000 7f872cd1f800 1 Processor -- start 2024-05-08T18:06:51.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.051+0000 7f872cd1f800 1 -- start start 2024-05-08T18:06:51.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.051+0000 7f872cd1f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 0x55d404522950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.051+0000 7f872cd1f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d40439a5d0 con 0x55d404426560 2024-05-08T18:06:51.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.051+0000 7f87289bc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 0x55d404522950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.051+0000 7f87289bc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 0x55d404522950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46174/0 (socket says 172.21.15.50:46174) 2024-05-08T18:06:51.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.051+0000 7f87289bc640 1 -- 172.21.15.50:0/864784095 learned_addr learned my addr 172.21.15.50:0/864784095 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.052+0000 7f87289bc640 1 -- 172.21.15.50:0/864784095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d404522e90 con 0x55d404426560 2024-05-08T18:06:51.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.052+0000 7f87289bc640 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 0x55d404522950 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f871801eea0 tx=0x7f8718002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.052+0000 7f861560a640 1 -- 172.21.15.50:0/864784095 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87180077c0 con 0x55d404426560 2024-05-08T18:06:51.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.052+0000 7f861560a640 1 -- 172.21.15.50:0/864784095 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8718007960 con 0x55d404426560 2024-05-08T18:06:51.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.052+0000 7f872cd1f800 1 -- 172.21.15.50:0/864784095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d404523120 con 0x55d404426560 2024-05-08T18:06:51.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.052+0000 7f861560a640 1 -- 172.21.15.50:0/864784095 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8718007c40 con 0x55d404426560 2024-05-08T18:06:51.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.052+0000 7f872cd1f800 1 -- 172.21.15.50:0/864784095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d404525d80 con 0x55d404426560 2024-05-08T18:06:51.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.053+0000 7f861560a640 1 -- 172.21.15.50:0/864784095 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f871807a430 con 0x55d404426560 2024-05-08T18:06:51.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.053+0000 7f861560a640 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f861003bb50 0x7f861003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.053+0000 7f861560a640 1 -- 172.21.15.50:0/864784095 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f8718094cd0 con 0x55d404426560 2024-05-08T18:06:51.083 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.053+0000 7f87281bb640 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f861003bb50 0x7f861003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.053+0000 7f872cd1f800 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d4044277c0 0x55d404529b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.053+0000 7f872cd1f800 1 -- 172.21.15.50:0/864784095 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d40452a110 con 0x55d4044277c0 2024-05-08T18:06:51.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.053+0000 7f87291bd640 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d4044277c0 0x55d404529b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.054+0000 7f87281bb640 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f861003bb50 0x7f861003e010 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f871c000c00 tx=0x7f871c0013b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.054+0000 7f87291bd640 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d4044277c0 0x55d404529b40 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.054+0000 7f87291bd640 1 -- 172.21.15.50:0/864784095 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f872000a4a0 con 0x55d4044277c0 2024-05-08T18:06:51.084 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.054+0000 7f872cd1f800 1 -- 172.21.15.50:0/864784095 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d4045270f0 con 0x55d4044277c0 2024-05-08T18:06:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.054+0000 7f87291bd640 1 -- 172.21.15.50:0/864784095 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f872000a4a0 con 0x55d4044277c0 2024-05-08T18:06:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.054+0000 7f872cd1f800 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d40452c420 0x55d40452e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.054+0000 7f872cd1f800 1 -- 172.21.15.50:0/864784095 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d40452eee0 con 0x55d40452c420 2024-05-08T18:06:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.055+0000 7f87289bc640 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d40452c420 0x55d40452e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.055+0000 7f87289bc640 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d40452c420 0x55d40452e810 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.055+0000 7f87289bc640 1 -- 172.21.15.50:0/864784095 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f871807ac90 con 0x55d40452c420 2024-05-08T18:06:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.055+0000 7f872cd1f800 1 -- 172.21.15.50:0/864784095 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d40452b2c0 con 0x55d4044277c0 2024-05-08T18:06:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.056+0000 7f87291bd640 1 -- 172.21.15.50:0/864784095 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f872000a4a0 con 0x55d4044277c0 2024-05-08T18:06:51.085 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.056+0000 7f872cd1f800 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d404530320 0x55d404532710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.056+0000 7f872cd1f800 1 -- 172.21.15.50:0/864784095 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d404532e50 con 0x55d404530320 2024-05-08T18:06:51.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.056+0000 7f87281bb640 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d404530320 0x55d404532710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.056+0000 7f87281bb640 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d404530320 0x55d404532710 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.056+0000 7f87281bb640 1 -- 172.21.15.50:0/864784095 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f871c00f670 con 0x55d404530320 2024-05-08T18:06:51.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.057+0000 7f872cd1f800 1 Processor -- start 2024-05-08T18:06:51.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.057+0000 7f872cd1f800 1 -- start start 2024-05-08T18:06:51.086 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.057+0000 7f872cd1f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4045411c0 0x55d4045415e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.057+0000 7f872cd1f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d404319330 con 0x55d4045411c0 2024-05-08T18:06:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.057+0000 7f87289bc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4045411c0 0x55d4045415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.057+0000 7f87289bc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4045411c0 0x55d4045415e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46190/0 (socket says 172.21.15.50:46190) 2024-05-08T18:06:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.057+0000 7f87289bc640 1 -- 172.21.15.50:0/3408693744 learned_addr learned my addr 172.21.15.50:0/3408693744 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.057+0000 7f87289bc640 1 -- 172.21.15.50:0/3408693744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d404541b20 con 0x55d4045411c0 2024-05-08T18:06:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.057+0000 7f87289bc640 1 --2- 172.21.15.50:0/3408693744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4045411c0 0x55d4045415e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8718060040 tx=0x7f87180686f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.058+0000 7f860e7fc640 1 -- 172.21.15.50:0/3408693744 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8718081040 con 0x55d4045411c0 2024-05-08T18:06:51.087 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.058+0000 7f860e7fc640 1 -- 172.21.15.50:0/3408693744 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8718079af0 con 0x55d4045411c0 2024-05-08T18:06:51.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.058+0000 7f872cd1f800 1 -- 172.21.15.50:0/3408693744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d404542590 con 0x55d4045411c0 2024-05-08T18:06:51.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.058+0000 7f860e7fc640 1 -- 172.21.15.50:0/3408693744 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8718079df0 con 0x55d4045411c0 2024-05-08T18:06:51.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.058+0000 7f872cd1f800 1 -- 172.21.15.50:0/3408693744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d404542a90 con 0x55d4045411c0 2024-05-08T18:06:51.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.058+0000 7f860e7fc640 1 -- 172.21.15.50:0/3408693744 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f8718070830 con 0x55d4045411c0 2024-05-08T18:06:51.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.059+0000 7f860e7fc640 1 --2- 172.21.15.50:0/3408693744 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f860003bba0 0x7f860003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.059+0000 7f860e7fc640 1 -- 172.21.15.50:0/3408693744 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f8718086d80 con 0x55d4045411c0 2024-05-08T18:06:51.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.059+0000 7f87281bb640 1 --2- 172.21.15.50:0/3408693744 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f860003bba0 0x7f860003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.088 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.059+0000 7f872cd1f800 1 Processor -- start 2024-05-08T18:06:51.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.059+0000 7f872cd1f800 1 -- start start 2024-05-08T18:06:51.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.059+0000 7f87281bb640 1 --2- 172.21.15.50:0/3408693744 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f860003bba0 0x7f860003e060 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55d40452a5c0 tx=0x7f871c006000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.059+0000 7f872cd1f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40454e370 0x55d40454e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.059+0000 7f872cd1f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d40451b730 con 0x55d40454e370 2024-05-08T18:06:51.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.059+0000 7f87289bc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40454e370 0x55d40454e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.059+0000 7f87289bc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40454e370 0x55d40454e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46200/0 (socket says 172.21.15.50:46200) 2024-05-08T18:06:51.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.059+0000 7f87289bc640 1 -- 172.21.15.50:0/880633348 learned_addr learned my addr 172.21.15.50:0/880633348 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.089 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.060+0000 7f87289bc640 1 -- 172.21.15.50:0/880633348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40454ed60 con 0x55d40454e370 2024-05-08T18:06:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.060+0000 7f87289bc640 1 --2- 172.21.15.50:0/880633348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40454e370 0x55d40454e790 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8718007e40 tx=0x7f8718071ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7b303f855045f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.060+0000 7f85fffff640 1 -- 172.21.15.50:0/880633348 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87180a4020 con 0x55d40454e370 2024-05-08T18:06:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.060+0000 7f85fffff640 1 -- 172.21.15.50:0/880633348 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8718070b20 con 0x55d40454e370 2024-05-08T18:06:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.060+0000 7f85fffff640 1 -- 172.21.15.50:0/880633348 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8718054d90 con 0x55d40454e370 2024-05-08T18:06:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.060+0000 7f872cd1f800 1 -- 172.21.15.50:0/880633348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40454e370 msgr2=0x55d40454e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.060+0000 7f872cd1f800 1 --2- 172.21.15.50:0/880633348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40454e370 0x55d40454e790 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8718007e40 tx=0x7f8718071ce0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.060+0000 7f872cd1f800 1 -- 172.21.15.50:0/880633348 shutdown_connections 2024-05-08T18:06:51.090 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.060+0000 7f872cd1f800 1 --2- 172.21.15.50:0/880633348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40454e370 0x55d40454e790 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.060+0000 7f872cd1f800 1 -- 172.21.15.50:0/880633348 >> 172.21.15.50:0/880633348 conn(0x55d40454a680 msgr2=0x55d4045486e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.061+0000 7f872cd1f800 1 -- 172.21.15.50:0/880633348 shutdown_connections 2024-05-08T18:06:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.061+0000 7f872cd1f800 1 -- 172.21.15.50:0/880633348 wait complete. 2024-05-08T18:06:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.061+0000 7f872cd1f800 1 Processor -- start 2024-05-08T18:06:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.061+0000 7f872cd1f800 1 -- start start 2024-05-08T18:06:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.061+0000 7f872cd1f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40454e370 0x55d4045556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.061+0000 7f872cd1f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d40454f3f0 con 0x55d40454e370 2024-05-08T18:06:51.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.061+0000 7f87289bc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40454e370 0x55d4045556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.061+0000 7f87289bc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40454e370 0x55d4045556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46202/0 (socket says 172.21.15.50:46202) 2024-05-08T18:06:51.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.061+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 learned_addr learned my addr 172.21.15.50:0/2351875985 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.062+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d404555c20 con 0x55d40454e370 2024-05-08T18:06:51.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.062+0000 7f87289bc640 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40454e370 0x55d4045556e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8718071e30 tx=0x7f8718077f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.062+0000 7f85fe7fc640 1 -- 172.21.15.50:0/2351875985 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87180b9070 con 0x55d40454e370 2024-05-08T18:06:51.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.062+0000 7f85fe7fc640 1 -- 172.21.15.50:0/2351875985 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871808b3f0 con 0x55d40454e370 2024-05-08T18:06:51.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.062+0000 7f85fe7fc640 1 -- 172.21.15.50:0/2351875985 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8718079510 con 0x55d40454e370 2024-05-08T18:06:51.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.062+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d404555c20 con 0x55d40454e370 2024-05-08T18:06:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.062+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4045560c0 con 0x55d40454e370 2024-05-08T18:06:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.063+0000 7f85fe7fc640 1 -- 172.21.15.50:0/2351875985 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f87180bc070 con 0x55d40454e370 2024-05-08T18:06:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.063+0000 7f85fe7fc640 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f85f403bba0 0x7f85f403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.063+0000 7f85fe7fc640 1 -- 172.21.15.50:0/2351875985 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f87180af030 con 0x55d40454e370 2024-05-08T18:06:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.063+0000 7f87281bb640 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f85f403bba0 0x7f85f403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.064+0000 7f87281bb640 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f85f403bba0 0x7f85f403e060 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f871c00ffd0 tx=0x7f871c023040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.064+0000 7f872cd1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.064+0000 7f872cd1f800 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d404560d30 0x55d4045631f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.064+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d404563940 con 0x55d404560d30 2024-05-08T18:06:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.064+0000 7f87291bd640 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d404560d30 0x55d4045631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.064+0000 7f87291bd640 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d404560d30 0x55d4045631f0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f872000a4a0 con 0x55d404560d30 2024-05-08T18:06:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f872cd1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f872cd1f800 20 realm 2024-05-08T18:06:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f872cd1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d404563d50 con 0x55d404560d30 2024-05-08T18:06:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f872000a4a0 con 0x55d404560d30 2024-05-08T18:06:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f872cd1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f872cd1f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f872cd1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d404564620 con 0x55d404560d30 2024-05-08T18:06:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f872000a4a0 con 0x55d404560d30 2024-05-08T18:06:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f872cd1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f872cd1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404560d30 2024-05-08T18:06:51.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f872000a4a0 con 0x55d404560d30 2024-05-08T18:06:51.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f872cd1f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:51.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f872cd1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f872cd1f800 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d404565720 0x55d404567b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.065+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d4045680d0 con 0x55d404565720 2024-05-08T18:06:51.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.066+0000 7f87289bc640 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d404565720 0x55d404567b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.066+0000 7f87289bc640 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d404565720 0x55d404567b10 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.066+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.066+0000 7f872cd1f800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:51.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.066+0000 7f872cd1f800 20 searching for the correct realm 2024-05-08T18:06:51.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.066+0000 7f872cd1f800 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d40456a080 0x55d40456c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.066+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d40456cc10 con 0x55d40456a080 2024-05-08T18:06:51.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.066+0000 7f87281bb640 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d40456a080 0x55d40456c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.067+0000 7f87281bb640 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d40456a080 0x55d40456c540 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.067+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.067+0000 7f87281bb640 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f871c02e910 0x7f871c03a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.067+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d40456cc10 con 0x7f871c02e910 2024-05-08T18:06:51.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.067+0000 7f87291bd640 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f871c02e910 0x7f871c03a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.068+0000 7f87291bd640 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f871c02e910 0x7f871c03a3c0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.068+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f872000a4a0 con 0x7f871c02e910 2024-05-08T18:06:51.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.068+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404565720 2024-05-08T18:06:51.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.068+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.068+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404560d30 2024-05-08T18:06:51.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.068+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f872000a4a0 con 0x55d404560d30 2024-05-08T18:06:51.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.068+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404560d30 2024-05-08T18:06:51.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.069+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f872000a4a0 con 0x55d404560d30 2024-05-08T18:06:51.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.069+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404560d30 2024-05-08T18:06:51.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.069+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f872000a4a0 con 0x55d404560d30 2024-05-08T18:06:51.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.069+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404565720 2024-05-08T18:06:51.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.069+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v29'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.069+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x7f871c02e910 2024-05-08T18:06:51.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.069+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f872000a4a0 con 0x7f871c02e910 2024-05-08T18:06:51.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.069+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d40456a080 2024-05-08T18:06:51.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.070+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.070+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404560d30 2024-05-08T18:06:51.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.070+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f872000a4a0 con 0x55d404560d30 2024-05-08T18:06:51.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.070+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404565720 2024-05-08T18:06:51.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.070+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.070+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404560d30 2024-05-08T18:06:51.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.071+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f872000a4a0 con 0x55d404560d30 2024-05-08T18:06:51.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.071+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x7f871c02e910 2024-05-08T18:06:51.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.071+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f8720032040 con 0x7f871c02e910 2024-05-08T18:06:51.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.071+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d40456a080 2024-05-08T18:06:51.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.071+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.071+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d40456a080 2024-05-08T18:06:51.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.071+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.071+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404565720 2024-05-08T18:06:51.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.072+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.072+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d40456a080 2024-05-08T18:06:51.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.072+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.072+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d40456a080 2024-05-08T18:06:51.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.072+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.072+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404560d30 2024-05-08T18:06:51.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.072+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.072+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404560d30 2024-05-08T18:06:51.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.072+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v29'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.072+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404565720 2024-05-08T18:06:51.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.073+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.073+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d40456a080 2024-05-08T18:06:51.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.073+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.073+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x7f871c02e910 2024-05-08T18:06:51.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.073+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8720032040 con 0x7f871c02e910 2024-05-08T18:06:51.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.073+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404560d30 2024-05-08T18:06:51.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.074+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.074+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404565720 2024-05-08T18:06:51.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.074+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.074+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d40456a080 2024-05-08T18:06:51.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.074+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.074+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404565720 2024-05-08T18:06:51.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.075+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.075+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x7f871c02e910 2024-05-08T18:06:51.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.075+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8720032040 con 0x7f871c02e910 2024-05-08T18:06:51.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.075+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d40456a080 2024-05-08T18:06:51.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.075+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.075+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404565720 2024-05-08T18:06:51.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.075+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.075+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x55d404560d30 2024-05-08T18:06:51.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v29'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d4045649c0 con 0x7f871c02e910 2024-05-08T18:06:51.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8720032040 con 0x7f871c02e910 2024-05-08T18:06:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f872cd1f800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f872cd1f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f872cd1f800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f872cd1f800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f872cd1f800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f872cd1f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f872cd1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d404568990 con 0x55d404560d30 2024-05-08T18:06:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f872cd1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f872cd1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d404568990 con 0x55d404560d30 2024-05-08T18:06:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f872cd1f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f872cd1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.076+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d404568990 con 0x55d40456a080 2024-05-08T18:06:51.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f872cd1f800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:51.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f872cd1f800 20 zone default found 2024-05-08T18:06:51.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f872cd1f800 4 Realm: () 2024-05-08T18:06:51.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f872cd1f800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:51.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f872cd1f800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:51.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f872cd1f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:51.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f872cd1f800 20 zonegroup default 2024-05-08T18:06:51.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f872cd1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d4045692d0 con 0x55d404565720 2024-05-08T18:06:51.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f872cd1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f872cd1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d40456ead0 con 0x55d40456a080 2024-05-08T18:06:51.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f872cd1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f872cd1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.077+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d40456efc0 con 0x55d40456a080 2024-05-08T18:06:51.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.078+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.078+0000 7f872cd1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.078+0000 7f872cd1f800 20 started sync module instance, tier type = 2024-05-08T18:06:51.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.078+0000 7f872cd1f800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:51.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.078+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d40456df10 con 0x7f871c02e910 2024-05-08T18:06:51.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.079+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8720032040 con 0x7f871c02e910 2024-05-08T18:06:51.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.079+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94369094231088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d40456df10 con 0x7f871c02e910 2024-05-08T18:06:51.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.079+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d40456fa80 con 0x55d404560d30 2024-05-08T18:06:51.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.080+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94369094231088] v29'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8720032040 con 0x7f871c02e910 2024-05-08T18:06:51.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.080+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.080+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94369094235536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d40456df10 con 0x55d404560d30 2024-05-08T18:06:51.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.080+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d404570520 con 0x55d40456a080 2024-05-08T18:06:51.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.081+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94369094235536] v29'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.081+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.082+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94369094228576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d40456de80 con 0x55d40456a080 2024-05-08T18:06:51.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.082+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d404571410 con 0x7f871c02e910 2024-05-08T18:06:51.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.082+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94369094228576] v29'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.083+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8720032040 con 0x7f871c02e910 2024-05-08T18:06:51.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.083+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94369094242144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d404571380 con 0x7f871c02e910 2024-05-08T18:06:51.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.083+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d4045722d0 con 0x55d404560d30 2024-05-08T18:06:51.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.084+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94369094242144] v29'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8720032040 con 0x7f871c02e910 2024-05-08T18:06:51.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.084+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.084+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94369094244272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d4045722d0 con 0x55d404560d30 2024-05-08T18:06:51.110 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.084+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d404572c90 con 0x55d404565720 2024-05-08T18:06:51.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.085+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.085+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94369094248416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d4045734e0 con 0x55d404565720 2024-05-08T18:06:51.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.085+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d404573f80 con 0x55d404565720 2024-05-08T18:06:51.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.085+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94369094244272] v29'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.086+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94369094248416] v29'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.086+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.086+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94369094253136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d404574320 con 0x55d404565720 2024-05-08T18:06:51.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.086+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d404574bb0 con 0x55d404565720 2024-05-08T18:06:51.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.087+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94369094253136] v29'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.087+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.087+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94369094245920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d404574e50 con 0x55d404565720 2024-05-08T18:06:51.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.087+0000 7f872cd1f800 20 add_watcher() i=0 2024-05-08T18:06:51.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.087+0000 7f872cd1f800 20 add_watcher() i=1 2024-05-08T18:06:51.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.087+0000 7f872cd1f800 20 add_watcher() i=2 2024-05-08T18:06:51.111 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.087+0000 7f872cd1f800 20 add_watcher() i=3 2024-05-08T18:06:51.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.087+0000 7f872cd1f800 20 add_watcher() i=4 2024-05-08T18:06:51.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.087+0000 7f872cd1f800 20 add_watcher() i=5 2024-05-08T18:06:51.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.087+0000 7f872cd1f800 20 add_watcher() i=6 2024-05-08T18:06:51.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.088+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94369094245920] v29'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.088+0000 7f872cd1f800 20 add_watcher() i=7 2024-05-08T18:06:51.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.088+0000 7f872cd1f800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:51.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.088+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d404578390 con 0x55d404560d30 2024-05-08T18:06:51.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.088+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.088+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94369094256384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d404575850 con 0x55d404560d30 2024-05-08T18:06:51.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.089+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94369094256384] v29'239 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.089+0000 7f85defe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:51.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.089+0000 7f872cd1f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:51.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.089+0000 7f872cd1f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:51.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.090+0000 7f872cd1f800 5 note: GC not initialized 2024-05-08T18:06:51.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.090+0000 7f85ddfe3640 20 reqs_thread_entry: start 2024-05-08T18:06:51.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.090+0000 7f872cd1f800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:51.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.090+0000 7f872cd1f800 20 Filter name: none 2024-05-08T18:06:51.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.090+0000 7f854f7fe640 20 reqs_thread_entry: start 2024-05-08T18:06:51.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.090+0000 7f872cd1f800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-05-08T18:06:51.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.090+0000 7f872cd1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.090+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d4045ab9e0 con 0x55d404565720 2024-05-08T18:06:51.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.090+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(62 ada [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.091+0000 7f872cd1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.091+0000 7f872cd1f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x0 2024-05-08T18:06:51.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.091+0000 7f872cd1f800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:51.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.091+0000 7f872cd1f800 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : miss 2024-05-08T18:06:51.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.091+0000 7f872cd1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.091+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:63 8.18 8:1f190392:users.keys::MH1WC2XQ1S8UISFDZC8W:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d4045aa180 con 0x55d404560d30 2024-05-08T18:06:51.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.091+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(63 MH1WC2XQ1S8UISFDZC8W [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.091+0000 7f872cd1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.091+0000 7f872cd1f800 10 cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x0 2024-05-08T18:06:51.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.091+0000 7f872cd1f800 10 adding default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-05-08T18:06:51.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.091+0000 7f872cd1f800 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : hit (negative entry) 2024-05-08T18:06:51.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.091+0000 7f872cd1f800 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : hit (negative entry) 2024-05-08T18:06:51.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.091+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:64 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.set in=58b,writefull 0~473 in=473b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d4045aaba0 con 0x55d404565720 2024-05-08T18:06:51.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.092+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(64 ada [delete,create,call,writefull 0~473] v29'5 uv5 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.092+0000 7f872cd1f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:51.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.092+0000 7f872cd1f800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:51.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.092+0000 7f872cd1f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-05-08T18:06:51.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.092+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94369094477248 in=661b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d4045af480 con 0x55d404560d30 2024-05-08T18:06:51.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.092+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== watch-notify(notify (1) cookie 94369094244272 notify 124554051615 ret 0) ==== 691+0+0 (crc 0 0 0) 0x7f872003d040 con 0x55d404560d30 2024-05-08T18:06:51.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.092+0000 7f85fffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051615 cookie 94369094244272 notifier 4823 bl.length()=649 2024-05-08T18:06:51.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.092+0000 7f85fffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:51.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.092+0000 7f85fffff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:51.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.092+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(65 notify.4 [notify cookie 94369094477248 out=8b] v0'0 uv240 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.092+0000 7f85fffff640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f85ec003ea0 con 0x55d404560d30 2024-05-08T18:06:51.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.093+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.093+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== watch-notify(notify_complete (2) cookie 94369094477248 notify 124554051615 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f8720037440 con 0x55d404560d30 2024-05-08T18:06:51.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.093+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:67 8.18 8:1f190392:users.keys::MH1WC2XQ1S8UISFDZC8W:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d4045aa5c0 con 0x55d404560d30 2024-05-08T18:06:51.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.094+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(67 MH1WC2XQ1S8UISFDZC8W [delete,create,writefull 0~7] v29'20 uv20 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.094+0000 7f872cd1f800 10 cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x7 2024-05-08T18:06:51.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.094+0000 7f872cd1f800 10 moving default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-05-08T18:06:51.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.094+0000 7f872cd1f800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:MH1WC2XQ1S8UISFDZC8W, ofs0, ns] 2024-05-08T18:06:51.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.094+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94369094477248 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d4045af480 con 0x55d40456a080 2024-05-08T18:06:51.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.094+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== watch-notify(notify (1) cookie 94369094228576 notify 124554051592 ret 0) ==== 219+0+0 (crc 0 0 0) 0x7f871c018040 con 0x55d40456a080 2024-05-08T18:06:51.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.094+0000 7f85fffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051592 cookie 94369094228576 notifier 4823 bl.length()=177 2024-05-08T18:06:51.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.094+0000 7f85fffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x7 2024-05-08T18:06:51.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.094+0000 7f85fffff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-05-08T18:06:51.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.094+0000 7f85fffff640 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f85ec003ea0 con 0x55d40456a080 2024-05-08T18:06:51.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.094+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(68 notify.2 [notify cookie 94369094477248 out=8b] v0'0 uv240 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.095+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== watch-notify(notify_complete (2) cookie 94369094477248 notify 124554051592 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f871c01fc10 con 0x55d40456a080 2024-05-08T18:06:51.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.095+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.116 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-05-08T18:06:51.092020Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"users","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:06:51.092006Z","tags":[],"group_ids":[]} 2024-05-08T18:06:51.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.096+0000 7f85defe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:51.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.096+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94369094256384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d404577a80 con 0x55d404560d30 2024-05-08T18:06:51.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.097+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94369094256384] v29'240 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.097+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94369094231088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d4045746c0 con 0x7f871c02e910 2024-05-08T18:06:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.098+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94369094231088] v29'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8720032040 con 0x7f871c02e910 2024-05-08T18:06:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.098+0000 7f872cd1f800 20 remove_watcher() i=0 2024-05-08T18:06:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.098+0000 7f872cd1f800 2 removed watcher, disabling cache 2024-05-08T18:06:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.098+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94369094235536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d40456e410 con 0x55d404560d30 2024-05-08T18:06:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.099+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94369094235536] v29'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.099+0000 7f872cd1f800 20 remove_watcher() i=1 2024-05-08T18:06:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.099+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94369094228576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d40456e410 con 0x55d40456a080 2024-05-08T18:06:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.100+0000 7f87281bb640 1 -- 172.21.15.50:0/2351875985 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94369094228576] v29'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f871c037040 con 0x55d40456a080 2024-05-08T18:06:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.100+0000 7f872cd1f800 20 remove_watcher() i=2 2024-05-08T18:06:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.100+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94369094242144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d40456da60 con 0x7f871c02e910 2024-05-08T18:06:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.101+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94369094242144] v29'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8720032040 con 0x7f871c02e910 2024-05-08T18:06:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.101+0000 7f872cd1f800 20 remove_watcher() i=3 2024-05-08T18:06:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.101+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94369094244272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d40456da60 con 0x55d404560d30 2024-05-08T18:06:51.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.102+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94369094244272] v29'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8720032040 con 0x55d404560d30 2024-05-08T18:06:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.102+0000 7f872cd1f800 20 remove_watcher() i=4 2024-05-08T18:06:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.102+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94369094248416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d4045717b0 con 0x55d404565720 2024-05-08T18:06:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.103+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94369094248416] v29'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.103+0000 7f872cd1f800 20 remove_watcher() i=5 2024-05-08T18:06:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.103+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94369094253136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d40456da60 con 0x55d404565720 2024-05-08T18:06:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.104+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94369094253136] v29'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.104+0000 7f872cd1f800 20 remove_watcher() i=6 2024-05-08T18:06:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.104+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94369094245920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d4045717b0 con 0x55d404565720 2024-05-08T18:06:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f87289bc640 1 -- 172.21.15.50:0/2351875985 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94369094245920] v29'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87180bb030 con 0x55d404565720 2024-05-08T18:06:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f872cd1f800 20 remove_watcher() i=7 2024-05-08T18:06:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d404565720 msgr2=0x55d404567b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f872cd1f800 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d404565720 0x55d404567b10 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d404560d30 msgr2=0x55d4045631f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f872cd1f800 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d404560d30 0x55d4045631f0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f871c02e910 msgr2=0x7f871c03a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f872cd1f800 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f871c02e910 0x7f871c03a3c0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d40456a080 msgr2=0x55d40456c540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f872cd1f800 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d40456a080 0x55d40456c540 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f85f403bba0 msgr2=0x7f85f403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f872cd1f800 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f85f403bba0 0x7f85f403e060 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f871c00ffd0 tx=0x7f871c023040 comp rx=0 tx=0).stop 2024-05-08T18:06:51.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40454e370 msgr2=0x55d4045556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f872cd1f800 1 --2- 172.21.15.50:0/2351875985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d40454e370 0x55d4045556e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8718071e30 tx=0x7f8718077f00 comp rx=0 tx=0).stop 2024-05-08T18:06:51.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 reap_dead start 2024-05-08T18:06:51.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f87291bd640 1 -- 172.21.15.50:0/2351875985 reap_dead start 2024-05-08T18:06:51.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 shutdown_connections 2024-05-08T18:06:51.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 >> 172.21.15.50:0/2351875985 conn(0x55d40454a680 msgr2=0x55d404547ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.105+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 shutdown_connections 2024-05-08T18:06:51.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 -- 172.21.15.50:0/2351875985 wait complete. 2024-05-08T18:06:51.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 -- 172.21.15.50:0/3408693744 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f860003bba0 msgr2=0x7f860003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 --2- 172.21.15.50:0/3408693744 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f860003bba0 0x7f860003e060 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55d40452a5c0 tx=0x7f871c006000 comp rx=0 tx=0).stop 2024-05-08T18:06:51.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 -- 172.21.15.50:0/3408693744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4045411c0 msgr2=0x55d4045415e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 --2- 172.21.15.50:0/3408693744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4045411c0 0x55d4045415e0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8718060040 tx=0x7f87180686f0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 -- 172.21.15.50:0/3408693744 shutdown_connections 2024-05-08T18:06:51.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 --2- 172.21.15.50:0/3408693744 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f860003bba0 0x7f860003e060 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 --2- 172.21.15.50:0/3408693744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4045411c0 0x55d4045415e0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 -- 172.21.15.50:0/3408693744 >> 172.21.15.50:0/3408693744 conn(0x55d404535260 msgr2=0x55d404535660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 -- 172.21.15.50:0/3408693744 shutdown_connections 2024-05-08T18:06:51.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 -- 172.21.15.50:0/3408693744 wait complete. 2024-05-08T18:06:51.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 -- 172.21.15.50:0/864784095 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d40452c420 msgr2=0x55d40452e810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d40452c420 0x55d40452e810 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 -- 172.21.15.50:0/864784095 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d4044277c0 msgr2=0x55d404529b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d4044277c0 0x55d404529b40 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 -- 172.21.15.50:0/864784095 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d404530320 msgr2=0x55d404532710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d404530320 0x55d404532710 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 -- 172.21.15.50:0/864784095 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f861003bb50 msgr2=0x7f861003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f861003bb50 0x7f861003e010 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f871c000c00 tx=0x7f871c0013b0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 -- 172.21.15.50:0/864784095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 msgr2=0x55d404522950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 --2- 172.21.15.50:0/864784095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d404426560 0x55d404522950 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f871801eea0 tx=0x7f8718002fb0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f87291bd640 1 -- 172.21.15.50:0/864784095 reap_dead start 2024-05-08T18:06:51.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.106+0000 7f872cd1f800 1 -- 172.21.15.50:0/864784095 shutdown_connections 2024-05-08T18:06:51.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.107+0000 7f872cd1f800 1 -- 172.21.15.50:0/864784095 >> 172.21.15.50:0/864784095 conn(0x55d404421b00 msgr2=0x55d404510bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.107+0000 7f872cd1f800 1 -- 172.21.15.50:0/864784095 shutdown_connections 2024-05-08T18:06:51.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.107+0000 7f872cd1f800 1 -- 172.21.15.50:0/864784095 wait complete. 2024-05-08T18:06:51.128 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:51.128 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:51.128 ERROR:root:0 2024-05-08T18:06:51.129 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:51.135 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchUser', 'Message': '', 'RequestId': 'tx0000079cc67014dcd501c-00663bbf3b-4221-default', 'HostId': '4221-default-default'} 2024-05-08T18:06:51.136 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'foo', 'display-name': 'Foo', 'email': 'foo@foo.com', 'access-key': '9te6NH5mcdcq0Tc5i8i1', 'secret-key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'max-buckets': '4'} 2024-05-08T18:06:51.146 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:51.146 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 0} 2024-05-08T18:06:51.159 INFO:tasks.radosgw_admin_rest: json result: {'keys': [], 'truncated': True, 'count': 0, 'marker': '8:1e6ca5ed:users.uid::foo:head'} 2024-05-08T18:06:51.159 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 1} 2024-05-08T18:06:51.174 INFO:tasks.radosgw_admin_rest: json result: {'keys': ['foo'], 'truncated': True, 'count': 1, 'marker': '8:a00000ee:users.uid::bar.buckets:head'} 2024-05-08T18:06:51.175 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 1, 'marker': '8:a00000ee:users.uid::bar.buckets:head'} 2024-05-08T18:06:51.183 INFO:tasks.radosgw_admin_rest: json result: {'keys': ['ada'], 'truncated': False, 'count': 1} 2024-05-08T18:06:51.184 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:51.187 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:51.187 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'access-key': '9te6NH5mcdcq0Tc5i8i1', 'uid': 'uid_not_exist'} 2024-05-08T18:06:51.191 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:51.192 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'ada', '--caps', 'users=read, write'] 2024-05-08T18:06:51.192 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'ada', '--caps', 'users=read, write'] 2024-05-08T18:06:51.192 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid ada --caps 'users=read, write' 2024-05-08T18:06:51.239 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:51.239 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:51.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.242+0000 7fa442923800 1 Processor -- start 2024-05-08T18:06:51.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.242+0000 7fa442923800 1 -- start start 2024-05-08T18:06:51.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.242+0000 7fa442923800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55e899e7ccb0 0x55e899e7f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.242+0000 7fa442923800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55e899e7f5e0 con 0x55e899e791a0 2024-05-08T18:06:51.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.242+0000 7fa442923800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55e899d18e20 con 0x55e899e7ccb0 2024-05-08T18:06:51.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.243+0000 7fa437fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55e899e7ccb0 0x55e899e7f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.243+0000 7fa437fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55e899e7ccb0 0x55e899e7f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46214/0 (socket says 172.21.15.50:46214) 2024-05-08T18:06:51.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.243+0000 7fa437fff640 1 -- 172.21.15.50:0/1211964343 learned_addr learned my addr 172.21.15.50:0/1211964343 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.243+0000 7fa43e652640 1 --1- 172.21.15.50:0/1211964343 >> v1:172.21.15.46:6789/0 conn(0x55e899e791a0 0x55e899e7b590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50810/0 (socket says 172.21.15.50:50810) 2024-05-08T18:06:51.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.243+0000 7fa437fff640 1 -- 172.21.15.50:0/1211964343 >> v1:172.21.15.46:6789/0 conn(0x55e899e791a0 legacy=0x55e899e7b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.243+0000 7fa437fff640 1 -- 172.21.15.50:0/1211964343 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e899e807c0 con 0x55e899e7ccb0 2024-05-08T18:06:51.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.245+0000 7fa437fff640 1 --2- 172.21.15.50:0/1211964343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e7ccb0 0x55e899e7f0a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa42800a520 tx=0x7fa42801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec9b86bdfb05d84 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.245+0000 7fa43de51640 1 -- 172.21.15.50:0/1211964343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa428057070 con 0x55e899e7ccb0 2024-05-08T18:06:51.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.245+0000 7fa43de51640 1 -- 172.21.15.50:0/1211964343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa428052570 con 0x55e899e7ccb0 2024-05-08T18:06:51.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.245+0000 7fa43de51640 1 -- 172.21.15.50:0/1211964343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa42804ae60 con 0x55e899e7ccb0 2024-05-08T18:06:51.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.245+0000 7fa442923800 1 -- 172.21.15.50:0/1211964343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e7ccb0 msgr2=0x55e899e7f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.245+0000 7fa442923800 1 --2- 172.21.15.50:0/1211964343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e7ccb0 0x55e899e7f0a0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa42800a520 tx=0x7fa42801d010 comp rx=0 tx=0).stop 2024-05-08T18:06:51.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.245+0000 7fa442923800 1 -- 172.21.15.50:0/1211964343 shutdown_connections 2024-05-08T18:06:51.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.245+0000 7fa442923800 1 --2- 172.21.15.50:0/1211964343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e7ccb0 0x55e899e7f0a0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.245+0000 7fa442923800 1 -- 172.21.15.50:0/1211964343 >> 172.21.15.50:0/1211964343 conn(0x55e899e72af0 msgr2=0x55e899e74f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.245+0000 7fa442923800 1 -- 172.21.15.50:0/1211964343 shutdown_connections 2024-05-08T18:06:51.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.245+0000 7fa442923800 1 -- 172.21.15.50:0/1211964343 wait complete. 2024-05-08T18:06:51.259 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.260+0000 7fa442923800 1 Processor -- start 2024-05-08T18:06:51.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.261+0000 7fa442923800 1 -- start start 2024-05-08T18:06:51.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.261+0000 7fa442923800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e791a0 0x55e899f69be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.261+0000 7fa442923800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e899dc3a30 con 0x55e899e791a0 2024-05-08T18:06:51.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.261+0000 7fa43e652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e791a0 0x55e899f69be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.261+0000 7fa43e652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e791a0 0x55e899f69be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46218/0 (socket says 172.21.15.50:46218) 2024-05-08T18:06:51.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.261+0000 7fa43e652640 1 -- 172.21.15.50:0/2321341488 learned_addr learned my addr 172.21.15.50:0/2321341488 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.261+0000 7fa43e652640 1 -- 172.21.15.50:0/2321341488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e899f6a1b0 con 0x55e899e791a0 2024-05-08T18:06:51.260 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.261+0000 7fa43e652640 1 --2- 172.21.15.50:0/2321341488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e791a0 0x55e899f69be0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa43000b260 tx=0x7fa43000b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58a047eda35b53d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.262+0000 7fa32c608640 1 -- 172.21.15.50:0/2321341488 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa43000e5c0 con 0x55e899e791a0 2024-05-08T18:06:51.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.262+0000 7fa32c608640 1 -- 172.21.15.50:0/2321341488 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa430006070 con 0x55e899e791a0 2024-05-08T18:06:51.261 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.262+0000 7fa32c608640 1 -- 172.21.15.50:0/2321341488 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa43000e890 con 0x55e899e791a0 2024-05-08T18:06:51.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.262+0000 7fa442923800 1 -- 172.21.15.50:0/2321341488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e791a0 msgr2=0x55e899f69be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.262+0000 7fa442923800 1 --2- 172.21.15.50:0/2321341488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e791a0 0x55e899f69be0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa43000b260 tx=0x7fa43000b720 comp rx=0 tx=0).stop 2024-05-08T18:06:51.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.262+0000 7fa442923800 1 -- 172.21.15.50:0/2321341488 shutdown_connections 2024-05-08T18:06:51.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.262+0000 7fa442923800 1 --2- 172.21.15.50:0/2321341488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e791a0 0x55e899f69be0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.277 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.262+0000 7fa442923800 1 -- 172.21.15.50:0/2321341488 >> 172.21.15.50:0/2321341488 conn(0x55e899e72af0 msgr2=0x55e899f65500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.262+0000 7fa442923800 1 -- 172.21.15.50:0/2321341488 shutdown_connections 2024-05-08T18:06:51.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.262+0000 7fa442923800 1 -- 172.21.15.50:0/2321341488 wait complete. 2024-05-08T18:06:51.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.263+0000 7fa442923800 1 Processor -- start 2024-05-08T18:06:51.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.263+0000 7fa442923800 1 -- start start 2024-05-08T18:06:51.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.263+0000 7fa442923800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e791a0 0x55e899f71780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.263+0000 7fa442923800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e899e7fab0 con 0x55e899e791a0 2024-05-08T18:06:51.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.263+0000 7fa43e652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e791a0 0x55e899f71780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.278 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.263+0000 7fa43e652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e791a0 0x55e899f71780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46224/0 (socket says 172.21.15.50:46224) 2024-05-08T18:06:51.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.263+0000 7fa43e652640 1 -- 172.21.15.50:0/926222426 learned_addr learned my addr 172.21.15.50:0/926222426 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.263+0000 7fa43e652640 1 -- 172.21.15.50:0/926222426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e899f71cc0 con 0x55e899e791a0 2024-05-08T18:06:51.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.264+0000 7fa43e652640 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e791a0 0x55e899f71780 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa430000f60 tx=0x7fa430000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.264+0000 7fa32ae05640 1 -- 172.21.15.50:0/926222426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa43001a7c0 con 0x55e899e791a0 2024-05-08T18:06:51.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.264+0000 7fa32ae05640 1 -- 172.21.15.50:0/926222426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43001a960 con 0x55e899e791a0 2024-05-08T18:06:51.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.264+0000 7fa442923800 1 -- 172.21.15.50:0/926222426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e899f71f50 con 0x55e899e791a0 2024-05-08T18:06:51.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.264+0000 7fa32ae05640 1 -- 172.21.15.50:0/926222426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa43001ac40 con 0x55e899e791a0 2024-05-08T18:06:51.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.264+0000 7fa442923800 1 -- 172.21.15.50:0/926222426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e899f74c00 con 0x55e899e791a0 2024-05-08T18:06:51.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.265+0000 7fa32ae05640 1 -- 172.21.15.50:0/926222426 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa430017070 con 0x55e899e791a0 2024-05-08T18:06:51.279 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.265+0000 7fa32ae05640 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa32403bba0 0x7fa32403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.265+0000 7fa32ae05640 1 -- 172.21.15.50:0/926222426 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fa430015020 con 0x55e899e791a0 2024-05-08T18:06:51.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.265+0000 7fa437fff640 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa32403bba0 0x7fa32403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.265+0000 7fa442923800 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e899e7ccb0 0x55e899f789c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.265+0000 7fa442923800 1 -- 172.21.15.50:0/926222426 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899f78f00 con 0x55e899e7ccb0 2024-05-08T18:06:51.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.265+0000 7fa43ee53640 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e899e7ccb0 0x55e899f789c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.265+0000 7fa437fff640 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa32403bba0 0x7fa32403e060 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa428054000 tx=0x7fa42804e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.266+0000 7fa43ee53640 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e899e7ccb0 0x55e899f789c0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.266+0000 7fa43ee53640 1 -- 172.21.15.50:0/926222426 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa43800a4a0 con 0x55e899e7ccb0 2024-05-08T18:06:51.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.266+0000 7fa442923800 1 -- 172.21.15.50:0/926222426 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899f75f90 con 0x55e899e7ccb0 2024-05-08T18:06:51.280 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.266+0000 7fa43ee53640 1 -- 172.21.15.50:0/926222426 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa43800a4a0 con 0x55e899e7ccb0 2024-05-08T18:06:51.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.266+0000 7fa442923800 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e899f7b250 0x55e899f7d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.266+0000 7fa442923800 1 -- 172.21.15.50:0/926222426 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899f7dd30 con 0x55e899f7b250 2024-05-08T18:06:51.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.267+0000 7fa43e652640 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e899f7b250 0x55e899f7d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.267+0000 7fa43e652640 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e899f7b250 0x55e899f7d640 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.267+0000 7fa43e652640 1 -- 172.21.15.50:0/926222426 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fa43002e070 con 0x55e899f7b250 2024-05-08T18:06:51.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.267+0000 7fa442923800 1 -- 172.21.15.50:0/926222426 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899f7a0f0 con 0x55e899e7ccb0 2024-05-08T18:06:51.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.267+0000 7fa43ee53640 1 -- 172.21.15.50:0/926222426 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa43800a4a0 con 0x55e899e7ccb0 2024-05-08T18:06:51.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.268+0000 7fa442923800 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e899f7f190 0x55e899f81580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.268+0000 7fa442923800 1 -- 172.21.15.50:0/926222426 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899f81c80 con 0x55e899f7f190 2024-05-08T18:06:51.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.268+0000 7fa437fff640 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e899f7f190 0x55e899f81580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.268+0000 7fa437fff640 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e899f7f190 0x55e899f81580 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.281 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.268+0000 7fa437fff640 1 -- 172.21.15.50:0/926222426 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fa42805a040 con 0x55e899f7f190 2024-05-08T18:06:51.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.268+0000 7fa442923800 1 Processor -- start 2024-05-08T18:06:51.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.269+0000 7fa442923800 1 -- start start 2024-05-08T18:06:51.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.269+0000 7fa442923800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899f8fff0 0x55e899f90410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.269+0000 7fa442923800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e899d6a330 con 0x55e899f8fff0 2024-05-08T18:06:51.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.269+0000 7fa43e652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899f8fff0 0x55e899f90410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.269+0000 7fa43e652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899f8fff0 0x55e899f90410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46226/0 (socket says 172.21.15.50:46226) 2024-05-08T18:06:51.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.269+0000 7fa43e652640 1 -- 172.21.15.50:0/4150256784 learned_addr learned my addr 172.21.15.50:0/4150256784 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.269+0000 7fa43e652640 1 -- 172.21.15.50:0/4150256784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e899f90950 con 0x55e899f8fff0 2024-05-08T18:06:51.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.269+0000 7fa43e652640 1 --2- 172.21.15.50:0/4150256784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899f8fff0 0x55e899f90410 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa430051800 tx=0x7fa430050510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.282 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.269+0000 7fa323fff640 1 -- 172.21.15.50:0/4150256784 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa430017b70 con 0x55e899f8fff0 2024-05-08T18:06:51.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.269+0000 7fa323fff640 1 -- 172.21.15.50:0/4150256784 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa430017d10 con 0x55e899f8fff0 2024-05-08T18:06:51.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.269+0000 7fa442923800 1 -- 172.21.15.50:0/4150256784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e899f913c0 con 0x55e899f8fff0 2024-05-08T18:06:51.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.270+0000 7fa323fff640 1 -- 172.21.15.50:0/4150256784 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa43001b590 con 0x55e899f8fff0 2024-05-08T18:06:51.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.270+0000 7fa442923800 1 -- 172.21.15.50:0/4150256784 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e899f918c0 con 0x55e899f8fff0 2024-05-08T18:06:51.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.270+0000 7fa323fff640 1 -- 172.21.15.50:0/4150256784 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa43001b750 con 0x55e899f8fff0 2024-05-08T18:06:51.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.270+0000 7fa323fff640 1 --2- 172.21.15.50:0/4150256784 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa31803bba0 0x7fa31803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.270+0000 7fa323fff640 1 -- 172.21.15.50:0/4150256784 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fa43003bc00 con 0x55e899f8fff0 2024-05-08T18:06:51.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.270+0000 7fa437fff640 1 --2- 172.21.15.50:0/4150256784 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa31803bba0 0x7fa31803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.271+0000 7fa442923800 1 Processor -- start 2024-05-08T18:06:51.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.271+0000 7fa442923800 1 -- start start 2024-05-08T18:06:51.283 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.271+0000 7fa437fff640 1 --2- 172.21.15.50:0/4150256784 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa31803bba0 0x7fa31803e060 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55e899f79800 tx=0x7fa428005ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.271+0000 7fa442923800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e78790 0x55e899e78bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.271+0000 7fa442923800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e899f6a530 con 0x55e899e78790 2024-05-08T18:06:51.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.271+0000 7fa43e652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e78790 0x55e899e78bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.271+0000 7fa43e652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e78790 0x55e899e78bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46236/0 (socket says 172.21.15.50:46236) 2024-05-08T18:06:51.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.271+0000 7fa43e652640 1 -- 172.21.15.50:0/4102529578 learned_addr learned my addr 172.21.15.50:0/4102529578 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.271+0000 7fa43e652640 1 -- 172.21.15.50:0/4102529578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e899e76f90 con 0x55e899e78790 2024-05-08T18:06:51.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.272+0000 7fa43e652640 1 --2- 172.21.15.50:0/4102529578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e78790 0x55e899e78bb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa43003b8d0 tx=0x7fa430043100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b10363b5e70df95 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.272+0000 7fa321ffb640 1 -- 172.21.15.50:0/4102529578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa430027430 con 0x55e899e78790 2024-05-08T18:06:51.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.272+0000 7fa321ffb640 1 -- 172.21.15.50:0/4102529578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa430019860 con 0x55e899e78790 2024-05-08T18:06:51.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.272+0000 7fa321ffb640 1 -- 172.21.15.50:0/4102529578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa43004a670 con 0x55e899e78790 2024-05-08T18:06:51.284 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.272+0000 7fa442923800 1 -- 172.21.15.50:0/4102529578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e78790 msgr2=0x55e899e78bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.272+0000 7fa442923800 1 --2- 172.21.15.50:0/4102529578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e78790 0x55e899e78bb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa43003b8d0 tx=0x7fa430043100 comp rx=0 tx=0).stop 2024-05-08T18:06:51.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.272+0000 7fa442923800 1 -- 172.21.15.50:0/4102529578 shutdown_connections 2024-05-08T18:06:51.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.272+0000 7fa442923800 1 --2- 172.21.15.50:0/4102529578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e78790 0x55e899e78bb0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.272+0000 7fa442923800 1 -- 172.21.15.50:0/4102529578 >> 172.21.15.50:0/4102529578 conn(0x55e899f99410 msgr2=0x55e899f97470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.272+0000 7fa442923800 1 -- 172.21.15.50:0/4102529578 shutdown_connections 2024-05-08T18:06:51.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.272+0000 7fa442923800 1 -- 172.21.15.50:0/4102529578 wait complete. 2024-05-08T18:06:51.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.273+0000 7fa442923800 1 Processor -- start 2024-05-08T18:06:51.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.273+0000 7fa442923800 1 -- start start 2024-05-08T18:06:51.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.273+0000 7fa442923800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e78790 0x55e899fa66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.285 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.273+0000 7fa442923800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e899e773a0 con 0x55e899e78790 2024-05-08T18:06:51.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.273+0000 7fa43e652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e78790 0x55e899fa66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.273+0000 7fa43e652640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e78790 0x55e899fa66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46246/0 (socket says 172.21.15.50:46246) 2024-05-08T18:06:51.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.273+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 learned_addr learned my addr 172.21.15.50:0/617799172 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.273+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e899fa6be0 con 0x55e899e78790 2024-05-08T18:06:51.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.273+0000 7fa43e652640 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e78790 0x55e899fa66a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa43001be20 tx=0x7fa43001ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.274+0000 7fa30ffff640 1 -- 172.21.15.50:0/617799172 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa430027430 con 0x55e899e78790 2024-05-08T18:06:51.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.274+0000 7fa30ffff640 1 -- 172.21.15.50:0/617799172 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43003c0c0 con 0x55e899e78790 2024-05-08T18:06:51.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.274+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e899fa6be0 con 0x55e899e78790 2024-05-08T18:06:51.286 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.274+0000 7fa30ffff640 1 -- 172.21.15.50:0/617799172 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa43003c3a0 con 0x55e899e78790 2024-05-08T18:06:51.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.274+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e899fa6be0 con 0x55e899e78790 2024-05-08T18:06:51.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.274+0000 7fa30ffff640 1 -- 172.21.15.50:0/617799172 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fa4300434a0 con 0x55e899e78790 2024-05-08T18:06:51.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.275+0000 7fa30ffff640 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa30803bba0 0x7fa30803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.275+0000 7fa30ffff640 1 -- 172.21.15.50:0/617799172 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fa43005b030 con 0x55e899e78790 2024-05-08T18:06:51.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.275+0000 7fa437fff640 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa30803bba0 0x7fa30803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.275+0000 7fa437fff640 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa30803bba0 0x7fa30803e060 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55e899f82080 tx=0x7fa42804f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.275+0000 7fa442923800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.276+0000 7fa442923800 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e899fb1bc0 0x55e899fb4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.276+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb47d0 con 0x55e899fb1bc0 2024-05-08T18:06:51.287 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.276+0000 7fa43ee53640 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e899fb1bc0 0x55e899fb4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.276+0000 7fa43ee53640 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e899fb1bc0 0x55e899fb4080 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.276+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa43800a4a0 con 0x55e899fb1bc0 2024-05-08T18:06:51.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.276+0000 7fa442923800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.276+0000 7fa442923800 20 realm 2024-05-08T18:06:51.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.276+0000 7fa442923800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.276+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb4be0 con 0x55e899fb1bc0 2024-05-08T18:06:51.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.276+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa43800a4a0 con 0x55e899fb1bc0 2024-05-08T18:06:51.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.276+0000 7fa442923800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.288 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.276+0000 7fa442923800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:51.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.276+0000 7fa442923800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.276+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb54b0 con 0x55e899fb1bc0 2024-05-08T18:06:51.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.277+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa43800a4a0 con 0x55e899fb1bc0 2024-05-08T18:06:51.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.277+0000 7fa442923800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.277+0000 7fa442923800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.277+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb5850 con 0x55e899fb1bc0 2024-05-08T18:06:51.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.277+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa43800a4a0 con 0x55e899fb1bc0 2024-05-08T18:06:51.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.277+0000 7fa442923800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:51.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.277+0000 7fa442923800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.277+0000 7fa442923800 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e899fb65b0 0x55e899fb89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.277+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb8fb0 con 0x55e899fb65b0 2024-05-08T18:06:51.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.277+0000 7fa43e652640 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e899fb65b0 0x55e899fb89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.278+0000 7fa43e652640 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e899fb65b0 0x55e899fb89a0 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.278+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.289 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.278+0000 7fa442923800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:51.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.278+0000 7fa442923800 20 searching for the correct realm 2024-05-08T18:06:51.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.278+0000 7fa442923800 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e899fbaf60 0x55e899fbd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.278+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55e899fbdaf0 con 0x55e899fbaf60 2024-05-08T18:06:51.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.278+0000 7fa437fff640 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e899fbaf60 0x55e899fbd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.278+0000 7fa437fff640 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e899fbaf60 0x55e899fbd420 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.278+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa42807f070 con 0x55e899fbaf60 2024-05-08T18:06:51.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.279+0000 7fa437fff640 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa428074040 0x7fa428081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.279+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55e899fbdaf0 con 0x7fa428074040 2024-05-08T18:06:51.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.279+0000 7fa43ee53640 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa428074040 0x7fa428081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.290 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.279+0000 7fa43ee53640 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa428074040 0x7fa428081410 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.279+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa43800a4a0 con 0x7fa428074040 2024-05-08T18:06:51.291 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.279+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb5850 con 0x55e899fb65b0 2024-05-08T18:06:51.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.280+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.306 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.280+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb5850 con 0x55e899fb1bc0 2024-05-08T18:06:51.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.280+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa43800a4a0 con 0x55e899fb1bc0 2024-05-08T18:06:51.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.280+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb5850 con 0x55e899fb1bc0 2024-05-08T18:06:51.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.280+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa43800a4a0 con 0x55e899fb1bc0 2024-05-08T18:06:51.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.280+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb5850 con 0x55e899fb1bc0 2024-05-08T18:06:51.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.280+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa43800a4a0 con 0x55e899fb1bc0 2024-05-08T18:06:51.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.280+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb5850 con 0x55e899fb65b0 2024-05-08T18:06:51.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.281+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v29'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.281+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb5850 con 0x7fa428074040 2024-05-08T18:06:51.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.281+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa43800a4a0 con 0x7fa428074040 2024-05-08T18:06:51.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.281+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb5850 con 0x55e899fbaf60 2024-05-08T18:06:51.307 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.281+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa42807f070 con 0x55e899fbaf60 2024-05-08T18:06:51.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.281+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb5850 con 0x55e899fb1bc0 2024-05-08T18:06:51.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.281+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa43800a4a0 con 0x55e899fb1bc0 2024-05-08T18:06:51.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.281+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb5850 con 0x55e899fb65b0 2024-05-08T18:06:51.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.282+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.282+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb5850 con 0x55e899fb1bc0 2024-05-08T18:06:51.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.282+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa43800a4a0 con 0x55e899fb1bc0 2024-05-08T18:06:51.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.282+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb5850 con 0x7fa428074040 2024-05-08T18:06:51.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.283+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fa438032040 con 0x7fa428074040 2024-05-08T18:06:51.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.283+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb5850 con 0x55e899fbaf60 2024-05-08T18:06:51.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.283+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa42807f070 con 0x55e899fbaf60 2024-05-08T18:06:51.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.283+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x55e899fbaf60 2024-05-08T18:06:51.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.283+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa42807f070 con 0x55e899fbaf60 2024-05-08T18:06:51.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.283+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x55e899fb65b0 2024-05-08T18:06:51.308 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.284+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.284+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x55e899fbaf60 2024-05-08T18:06:51.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.284+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa42807f070 con 0x55e899fbaf60 2024-05-08T18:06:51.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.284+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x55e899fbaf60 2024-05-08T18:06:51.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.284+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fa42807f070 con 0x55e899fbaf60 2024-05-08T18:06:51.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.284+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x55e899fb1bc0 2024-05-08T18:06:51.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.284+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.284+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x55e899fb1bc0 2024-05-08T18:06:51.309 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.285+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v29'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.285+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x55e899fb65b0 2024-05-08T18:06:51.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.285+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.285+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x55e899fbaf60 2024-05-08T18:06:51.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.285+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa42807f070 con 0x55e899fbaf60 2024-05-08T18:06:51.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.285+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x7fa428074040 2024-05-08T18:06:51.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.285+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa438032040 con 0x7fa428074040 2024-05-08T18:06:51.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.285+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x55e899fb1bc0 2024-05-08T18:06:51.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.286+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.286+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x55e899fb65b0 2024-05-08T18:06:51.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.286+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.286+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x55e899fbaf60 2024-05-08T18:06:51.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.286+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa42807f070 con 0x55e899fbaf60 2024-05-08T18:06:51.310 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.286+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x55e899fb65b0 2024-05-08T18:06:51.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.286+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.286+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x7fa428074040 2024-05-08T18:06:51.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.287+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa438032040 con 0x7fa428074040 2024-05-08T18:06:51.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.287+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x55e899fbaf60 2024-05-08T18:06:51.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.287+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa42807f070 con 0x55e899fbaf60 2024-05-08T18:06:51.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.287+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x55e899fb65b0 2024-05-08T18:06:51.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.287+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.287+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x55e899fb1bc0 2024-05-08T18:06:51.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v29'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fa42808e040 con 0x7fa428074040 2024-05-08T18:06:51.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa438032040 con 0x7fa428074040 2024-05-08T18:06:51.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa442923800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:51.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa442923800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:51.311 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa442923800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:51.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa442923800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:51.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa442923800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:51.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa442923800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:51.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa442923800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb9870 con 0x55e899fb1bc0 2024-05-08T18:06:51.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa442923800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa442923800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb9870 con 0x55e899fb1bc0 2024-05-08T18:06:51.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa442923800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:51.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa442923800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.288+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899fb9870 con 0x55e899fbaf60 2024-05-08T18:06:51.312 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fa42807f070 con 0x55e899fbaf60 2024-05-08T18:06:51.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa442923800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:51.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa442923800 20 zone default found 2024-05-08T18:06:51.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa442923800 4 Realm: () 2024-05-08T18:06:51.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa442923800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:51.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa442923800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:51.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa442923800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:51.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa442923800 20 zonegroup default 2024-05-08T18:06:51.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa442923800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899fba1e0 con 0x55e899fb65b0 2024-05-08T18:06:51.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa442923800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa442923800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899fbf9e0 con 0x55e899fbaf60 2024-05-08T18:06:51.313 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa42807f070 con 0x55e899fbaf60 2024-05-08T18:06:51.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa442923800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa442923800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899fbfe40 con 0x55e899fbaf60 2024-05-08T18:06:51.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.289+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa42807f070 con 0x55e899fbaf60 2024-05-08T18:06:51.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.290+0000 7fa442923800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.290+0000 7fa442923800 20 started sync module instance, tier type = 2024-05-08T18:06:51.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.290+0000 7fa442923800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:51.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.290+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fbedf0 con 0x7fa428074040 2024-05-08T18:06:51.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.291+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa438032040 con 0x7fa428074040 2024-05-08T18:06:51.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.291+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94457504199488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fbedf0 con 0x7fa428074040 2024-05-08T18:06:51.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.291+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc0920 con 0x55e899fb1bc0 2024-05-08T18:06:51.314 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.292+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94457504199488] v29'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa438032040 con 0x7fa428074040 2024-05-08T18:06:51.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.292+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.292+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94457504203792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fbedf0 con 0x55e899fb1bc0 2024-05-08T18:06:51.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.292+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc13c0 con 0x55e899fbaf60 2024-05-08T18:06:51.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.293+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94457504203792] v29'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.293+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa42807f070 con 0x55e899fbaf60 2024-05-08T18:06:51.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.293+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94457504196928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fbed60 con 0x55e899fbaf60 2024-05-08T18:06:51.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.293+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc2170 con 0x7fa428074040 2024-05-08T18:06:51.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.294+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa438032040 con 0x7fa428074040 2024-05-08T18:06:51.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.294+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94457504210016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc2a10 con 0x7fa428074040 2024-05-08T18:06:51.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.294+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94457504196928] v29'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa42807f070 con 0x55e899fbaf60 2024-05-08T18:06:51.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.294+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc3000 con 0x55e899fb1bc0 2024-05-08T18:06:51.315 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.295+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94457504210016] v29'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa438032040 con 0x7fa428074040 2024-05-08T18:06:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.295+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.295+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94457504208816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc2940 con 0x55e899fb1bc0 2024-05-08T18:06:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.295+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc3a10 con 0x55e899fb65b0 2024-05-08T18:06:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.296+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.296+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94457504208816] v29'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.296+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94457504212112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc28b0 con 0x55e899fb65b0 2024-05-08T18:06:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.296+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc4710 con 0x55e899fb65b0 2024-05-08T18:06:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.297+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94457504212112] v29'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.297+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.297+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94457504219024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc4f60 con 0x55e899fb65b0 2024-05-08T18:06:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.298+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc5920 con 0x55e899fb65b0 2024-05-08T18:06:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.299+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94457504219024] v29'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.316 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.299+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.299+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94457504224368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc5d40 con 0x55e899fb65b0 2024-05-08T18:06:51.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.299+0000 7fa442923800 20 add_watcher() i=0 2024-05-08T18:06:51.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.299+0000 7fa442923800 20 add_watcher() i=1 2024-05-08T18:06:51.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.299+0000 7fa442923800 20 add_watcher() i=2 2024-05-08T18:06:51.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.299+0000 7fa442923800 20 add_watcher() i=3 2024-05-08T18:06:51.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.299+0000 7fa442923800 20 add_watcher() i=4 2024-05-08T18:06:51.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.299+0000 7fa442923800 20 add_watcher() i=5 2024-05-08T18:06:51.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.299+0000 7fa442923800 20 add_watcher() i=6 2024-05-08T18:06:51.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.299+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94457504224368] v29'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.300+0000 7fa442923800 20 add_watcher() i=7 2024-05-08T18:06:51.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.300+0000 7fa442923800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:51.317 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.300+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc9020 con 0x55e899fb1bc0 2024-05-08T18:06:51.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.300+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.300+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94457504225792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc62d0 con 0x55e899fb1bc0 2024-05-08T18:06:51.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.301+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94457504225792] v29'241 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.301+0000 7fa2f4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:51.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.301+0000 7fa442923800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:51.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.301+0000 7fa442923800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:51.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.301+0000 7fa442923800 5 note: GC not initialized 2024-05-08T18:06:51.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.301+0000 7fa26f7fe640 20 reqs_thread_entry: start 2024-05-08T18:06:51.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.301+0000 7fa442923800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:51.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.302+0000 7fa442923800 20 Filter name: none 2024-05-08T18:06:51.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.302+0000 7fa26d7fa640 20 reqs_thread_entry: start 2024-05-08T18:06:51.318 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.302+0000 7fa442923800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-05-08T18:06:51.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.302+0000 7fa442923800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.302+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899ffc5f0 con 0x55e899fb65b0 2024-05-08T18:06:51.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.302+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(62 ada [call out=48b,read 0~473 out=473b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 231+0+596 (crc 0 0 0) 0x7fa430037020 con 0x55e899fb65b0 2024-05-08T18:06:51.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.302+0000 7fa442923800 20 rados_obj.operate() r=0 bl.length=473 2024-05-08T18:06:51.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.302+0000 7fa442923800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-05-08T18:06:51.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.302+0000 7fa442923800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:51.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.302+0000 7fa442923800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:51.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.302+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~460 in=460b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e89a002fa0 con 0x55e899fb65b0 2024-05-08T18:06:51.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.304+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~460] v29'7 uv7 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fa430094040 con 0x55e899fb65b0 2024-05-08T18:06:51.319 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.304+0000 7fa442923800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:51.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.304+0000 7fa442923800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:51.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.304+0000 7fa442923800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-05-08T18:06:51.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.304+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94457504475488 in=648b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55e899ffe600 con 0x55e899fb1bc0 2024-05-08T18:06:51.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.304+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== watch-notify(notify (1) cookie 94457504208816 notify 124554051617 ret 0) ==== 678+0+0 (crc 0 0 0) 0x7fa43803d040 con 0x55e899fb1bc0 2024-05-08T18:06:51.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.304+0000 7fa321ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051617 cookie 94457504208816 notifier 4830 bl.length()=636 2024-05-08T18:06:51.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.304+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 notify.4 [notify cookie 94457504475488 out=8b] v0'0 uv243 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.304+0000 7fa321ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:51.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.304+0000 7fa321ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:51.320 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.304+0000 7fa321ffb640 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7fa304003ea0 con 0x55e899fb1bc0 2024-05-08T18:06:51.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.304+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.321 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.304+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== watch-notify(notify_complete (2) cookie 94457504475488 notify 124554051617 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fa438018b80 con 0x55e899fb1bc0 2024-05-08T18:06:51.321 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-05-08T18:06:51.092020Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:06:51.092006Z","tags":[],"group_ids":[]} 2024-05-08T18:06:51.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.305+0000 7fa2f4fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:51.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.305+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94457504225792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e89a001e50 con 0x55e899fb1bc0 2024-05-08T18:06:51.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.306+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94457504225792] v29'242 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.306+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94457504199488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc5a00 con 0x7fa428074040 2024-05-08T18:06:51.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.307+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94457504199488] v29'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa438032040 con 0x7fa428074040 2024-05-08T18:06:51.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.307+0000 7fa442923800 20 remove_watcher() i=0 2024-05-08T18:06:51.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.307+0000 7fa442923800 2 removed watcher, disabling cache 2024-05-08T18:06:51.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.307+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94457504203792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc5a00 con 0x55e899fb1bc0 2024-05-08T18:06:51.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.308+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94457504203792] v29'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.328 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.308+0000 7fa442923800 20 remove_watcher() i=1 2024-05-08T18:06:51.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.308+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94457504196928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc5a00 con 0x55e899fbaf60 2024-05-08T18:06:51.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.309+0000 7fa437fff640 1 -- 172.21.15.50:0/617799172 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94457504196928] v29'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa42807f070 con 0x55e899fbaf60 2024-05-08T18:06:51.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.309+0000 7fa442923800 20 remove_watcher() i=2 2024-05-08T18:06:51.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.309+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94457504210016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fbe940 con 0x7fa428074040 2024-05-08T18:06:51.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.310+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94457504210016] v29'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa438032040 con 0x7fa428074040 2024-05-08T18:06:51.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.310+0000 7fa442923800 20 remove_watcher() i=3 2024-05-08T18:06:51.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.310+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94457504208816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc5a00 con 0x55e899fb1bc0 2024-05-08T18:06:51.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.311+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94457504208816] v29'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa438032040 con 0x55e899fb1bc0 2024-05-08T18:06:51.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.311+0000 7fa442923800 20 remove_watcher() i=4 2024-05-08T18:06:51.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.311+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94457504212112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc1c60 con 0x55e899fb65b0 2024-05-08T18:06:51.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.312+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94457504212112] v29'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa430094040 con 0x55e899fb65b0 2024-05-08T18:06:51.329 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.312+0000 7fa442923800 20 remove_watcher() i=5 2024-05-08T18:06:51.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.312+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94457504219024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fbe940 con 0x55e899fb65b0 2024-05-08T18:06:51.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.313+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94457504219024] v29'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa430094040 con 0x55e899fb65b0 2024-05-08T18:06:51.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.313+0000 7fa442923800 20 remove_watcher() i=6 2024-05-08T18:06:51.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.313+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94457504224368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55e899fc5a00 con 0x55e899fb65b0 2024-05-08T18:06:51.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.314+0000 7fa43e652640 1 -- 172.21.15.50:0/617799172 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94457504224368] v29'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa430094040 con 0x55e899fb65b0 2024-05-08T18:06:51.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.314+0000 7fa442923800 20 remove_watcher() i=7 2024-05-08T18:06:51.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e899fb65b0 msgr2=0x55e899fb89a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e899fb65b0 0x55e899fb89a0 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.330 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e899fb1bc0 msgr2=0x55e899fb4080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e899fb1bc0 0x55e899fb4080 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa428074040 msgr2=0x7fa428081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fa428074040 0x7fa428081410 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e899fbaf60 msgr2=0x55e899fbd420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e899fbaf60 0x55e899fbd420 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa30803bba0 msgr2=0x7fa30803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa30803bba0 0x7fa30803e060 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55e899f82080 tx=0x7fa42804f000 comp rx=0 tx=0).stop 2024-05-08T18:06:51.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e78790 msgr2=0x55e899fa66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e78790 0x55e899fa66a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa43001be20 tx=0x7fa43001ba40 comp rx=0 tx=0).stop 2024-05-08T18:06:51.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa43ee53640 1 -- 172.21.15.50:0/617799172 reap_dead start 2024-05-08T18:06:51.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 shutdown_connections 2024-05-08T18:06:51.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 --2- 172.21.15.50:0/617799172 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e78790 0x55e899fa66a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 >> 172.21.15.50:0/617799172 conn(0x55e899f99410 msgr2=0x55e899f96a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.331 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 shutdown_connections 2024-05-08T18:06:51.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 -- 172.21.15.50:0/617799172 wait complete. 2024-05-08T18:06:51.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 -- 172.21.15.50:0/4150256784 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa31803bba0 msgr2=0x7fa31803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 --2- 172.21.15.50:0/4150256784 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa31803bba0 0x7fa31803e060 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55e899f79800 tx=0x7fa428005ac0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 -- 172.21.15.50:0/4150256784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899f8fff0 msgr2=0x55e899f90410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 --2- 172.21.15.50:0/4150256784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899f8fff0 0x55e899f90410 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa430051800 tx=0x7fa430050510 comp rx=0 tx=0).stop 2024-05-08T18:06:51.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 -- 172.21.15.50:0/4150256784 shutdown_connections 2024-05-08T18:06:51.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 --2- 172.21.15.50:0/4150256784 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa31803bba0 0x7fa31803e060 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 --2- 172.21.15.50:0/4150256784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899f8fff0 0x55e899f90410 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 -- 172.21.15.50:0/4150256784 >> 172.21.15.50:0/4150256784 conn(0x55e899f84090 msgr2=0x55e899f84490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 -- 172.21.15.50:0/4150256784 shutdown_connections 2024-05-08T18:06:51.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.315+0000 7fa442923800 1 -- 172.21.15.50:0/4150256784 wait complete. 2024-05-08T18:06:51.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.316+0000 7fa442923800 1 -- 172.21.15.50:0/926222426 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e899f7b250 msgr2=0x55e899f7d640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.316+0000 7fa442923800 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55e899f7b250 0x55e899f7d640 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.316+0000 7fa442923800 1 -- 172.21.15.50:0/926222426 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e899e7ccb0 msgr2=0x55e899f789c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.332 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.316+0000 7fa442923800 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55e899e7ccb0 0x55e899f789c0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.316+0000 7fa442923800 1 -- 172.21.15.50:0/926222426 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e899f7f190 msgr2=0x55e899f81580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.316+0000 7fa442923800 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55e899f7f190 0x55e899f81580 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.316+0000 7fa442923800 1 -- 172.21.15.50:0/926222426 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa32403bba0 msgr2=0x7fa32403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.316+0000 7fa442923800 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fa32403bba0 0x7fa32403e060 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa428054000 tx=0x7fa42804e000 comp rx=0 tx=0).stop 2024-05-08T18:06:51.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.316+0000 7fa442923800 1 -- 172.21.15.50:0/926222426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e791a0 msgr2=0x55e899f71780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.316+0000 7fa442923800 1 --2- 172.21.15.50:0/926222426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e899e791a0 0x55e899f71780 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa430000f60 tx=0x7fa430000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.316+0000 7fa43ee53640 1 -- 172.21.15.50:0/926222426 reap_dead start 2024-05-08T18:06:51.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.316+0000 7fa442923800 1 -- 172.21.15.50:0/926222426 shutdown_connections 2024-05-08T18:06:51.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.316+0000 7fa442923800 1 -- 172.21.15.50:0/926222426 >> 172.21.15.50:0/926222426 conn(0x55e899e72af0 msgr2=0x55e899f63d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.316+0000 7fa442923800 1 -- 172.21.15.50:0/926222426 shutdown_connections 2024-05-08T18:06:51.333 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.316+0000 7fa442923800 1 -- 172.21.15.50:0/926222426 wait complete. 2024-05-08T18:06:51.333 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:51.334 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:51.334 ERROR:root:0 2024-05-08T18:06:51.334 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'system_user', '--display-name', 'System User', '--access-key', 'system_akey', '--secret', 'system_skey', '--caps', 'user-info-without-keys=read', '--system'] 2024-05-08T18:06:51.334 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'system_user', '--display-name', 'System User', '--access-key', 'system_akey', '--secret', 'system_skey', '--caps', 'user-info-without-keys=read', '--system'] 2024-05-08T18:06:51.334 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid system_user --display-name 'System User' --access-key system_akey --secret system_skey --caps user-info-without-keys=read --system 2024-05-08T18:06:51.389 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:51.389 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:51.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.392+0000 7f800bb25800 1 Processor -- start 2024-05-08T18:06:51.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.392+0000 7f800bb25800 1 -- start start 2024-05-08T18:06:51.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.392+0000 7f800bb25800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55cfacdf6dd0 0x55cface8b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.392+0000 7f800bb25800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55cfacdf7280 con 0x55cfacdf8110 2024-05-08T18:06:51.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.392+0000 7f800bb25800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55cfacd22e20 con 0x55cfacdf6dd0 2024-05-08T18:06:51.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.392+0000 7f7ffffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55cfacdf6dd0 0x55cface8b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.392+0000 7f80078b0640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55cfacdf8110 0x55cfacdf66c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50826/0 (socket says 172.21.15.50:50826) 2024-05-08T18:06:51.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.392+0000 7f80078b0640 1 -- 172.21.15.50:0/4261948557 learned_addr learned my addr 172.21.15.50:0/4261948557 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:51.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.392+0000 7f80070af640 1 -- 172.21.15.50:0/4261948557 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 394066684 0 0) 0x55cfacdf7280 con 0x55cfacdf8110 2024-05-08T18:06:51.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.392+0000 7f80070af640 1 -- 172.21.15.50:0/4261948557 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ff4001e10 con 0x55cfacdf8110 2024-05-08T18:06:51.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.393+0000 7f7ffffff640 1 -- 172.21.15.50:0/4261948557 >> v1:172.21.15.46:6789/0 conn(0x55cfacdf8110 legacy=0x55cfacdf66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.393+0000 7f7ffffff640 1 -- 172.21.15.50:0/4261948557 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfacdf7480 con 0x55cfacdf6dd0 2024-05-08T18:06:51.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.394+0000 7f7ffffff640 1 --2- 172.21.15.50:0/4261948557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 0x55cface8b5c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000a470 tx=0x7f7ff001cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76901e100541b7c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.395+0000 7f80070af640 1 -- 172.21.15.50:0/4261948557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff0057070 con 0x55cfacdf6dd0 2024-05-08T18:06:51.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.395+0000 7f80070af640 1 -- 172.21.15.50:0/4261948557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff0052570 con 0x55cfacdf6dd0 2024-05-08T18:06:51.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.395+0000 7f80070af640 1 -- 172.21.15.50:0/4261948557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff0052850 con 0x55cfacdf6dd0 2024-05-08T18:06:51.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.395+0000 7f800bb25800 1 -- 172.21.15.50:0/4261948557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 msgr2=0x55cface8b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.395+0000 7f800bb25800 1 --2- 172.21.15.50:0/4261948557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 0x55cface8b5c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000a470 tx=0x7f7ff001cdb0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.395+0000 7f800bb25800 1 -- 172.21.15.50:0/4261948557 shutdown_connections 2024-05-08T18:06:51.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.395+0000 7f800bb25800 1 --2- 172.21.15.50:0/4261948557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 0x55cface8b5c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.395+0000 7f800bb25800 1 -- 172.21.15.50:0/4261948557 >> 172.21.15.50:0/4261948557 conn(0x55cface7dce0 msgr2=0x55cface80100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.395+0000 7f800bb25800 1 -- 172.21.15.50:0/4261948557 shutdown_connections 2024-05-08T18:06:51.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.395+0000 7f800bb25800 1 -- 172.21.15.50:0/4261948557 wait complete. 2024-05-08T18:06:51.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.410+0000 7f800bb25800 1 Processor -- start 2024-05-08T18:06:51.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.411+0000 7f800bb25800 1 -- start start 2024-05-08T18:06:51.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.411+0000 7f800bb25800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 0x55cfacf75b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.411+0000 7f800bb25800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cfacdcdb20 con 0x55cfacdf6dd0 2024-05-08T18:06:51.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.411+0000 7f80078b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 0x55cfacf75b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.409 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.411+0000 7f80078b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 0x55cfacf75b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46262/0 (socket says 172.21.15.50:46262) 2024-05-08T18:06:51.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.411+0000 7f80078b0640 1 -- 172.21.15.50:0/1815120650 learned_addr learned my addr 172.21.15.50:0/1815120650 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.411+0000 7f80078b0640 1 -- 172.21.15.50:0/1815120650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfacf76140 con 0x55cfacdf6dd0 2024-05-08T18:06:51.410 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.412+0000 7f80078b0640 1 --2- 172.21.15.50:0/1815120650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 0x55cfacf75b70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8001220 tx=0x7f7ff800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc9bd406429665c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.412+0000 7f7ef5e0b640 1 -- 172.21.15.50:0/1815120650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff8012020 con 0x55cfacdf6dd0 2024-05-08T18:06:51.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.412+0000 7f7ef5e0b640 1 -- 172.21.15.50:0/1815120650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff800e470 con 0x55cfacdf6dd0 2024-05-08T18:06:51.413 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.412+0000 7f7ef5e0b640 1 -- 172.21.15.50:0/1815120650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff800e790 con 0x55cfacdf6dd0 2024-05-08T18:06:51.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.412+0000 7f800bb25800 1 -- 172.21.15.50:0/1815120650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 msgr2=0x55cfacf75b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.412+0000 7f800bb25800 1 --2- 172.21.15.50:0/1815120650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 0x55cfacf75b70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8001220 tx=0x7f7ff800b6c0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.412+0000 7f800bb25800 1 -- 172.21.15.50:0/1815120650 shutdown_connections 2024-05-08T18:06:51.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.412+0000 7f800bb25800 1 --2- 172.21.15.50:0/1815120650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 0x55cfacf75b70 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.412+0000 7f800bb25800 1 -- 172.21.15.50:0/1815120650 >> 172.21.15.50:0/1815120650 conn(0x55cface7dce0 msgr2=0x55cfacf71490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.412+0000 7f800bb25800 1 -- 172.21.15.50:0/1815120650 shutdown_connections 2024-05-08T18:06:51.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.412+0000 7f800bb25800 1 -- 172.21.15.50:0/1815120650 wait complete. 2024-05-08T18:06:51.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.413+0000 7f800bb25800 1 Processor -- start 2024-05-08T18:06:51.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.413+0000 7f800bb25800 1 -- start start 2024-05-08T18:06:51.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.413+0000 7f800bb25800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 0x55cfacf7d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.413+0000 7f800bb25800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cface8be00 con 0x55cfacdf6dd0 2024-05-08T18:06:51.414 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.413+0000 7f80078b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 0x55cfacf7d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.413+0000 7f80078b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 0x55cfacf7d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46266/0 (socket says 172.21.15.50:46266) 2024-05-08T18:06:51.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.413+0000 7f80078b0640 1 -- 172.21.15.50:0/2120927077 learned_addr learned my addr 172.21.15.50:0/2120927077 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.414+0000 7f80078b0640 1 -- 172.21.15.50:0/2120927077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfacf7dc90 con 0x55cfacdf6dd0 2024-05-08T18:06:51.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.414+0000 7f80078b0640 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 0x55cfacf7d750 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x55cfacf76900 tx=0x7f7ff8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.414+0000 7f7eeffff640 1 -- 172.21.15.50:0/2120927077 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff8012030 con 0x55cfacdf6dd0 2024-05-08T18:06:51.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.414+0000 7f7eeffff640 1 -- 172.21.15.50:0/2120927077 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff800be70 con 0x55cfacdf6dd0 2024-05-08T18:06:51.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.414+0000 7f7eeffff640 1 -- 172.21.15.50:0/2120927077 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff801a7a0 con 0x55cfacdf6dd0 2024-05-08T18:06:51.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.414+0000 7f800bb25800 1 -- 172.21.15.50:0/2120927077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cfacf7df20 con 0x55cfacdf6dd0 2024-05-08T18:06:51.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.414+0000 7f800bb25800 1 -- 172.21.15.50:0/2120927077 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfacf80b30 con 0x55cfacdf6dd0 2024-05-08T18:06:51.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.415+0000 7f7eeffff640 1 -- 172.21.15.50:0/2120927077 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7ff8015020 con 0x55cfacdf6dd0 2024-05-08T18:06:51.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.415+0000 7f7eeffff640 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ef003bb50 0x7f7ef003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.415+0000 7f7eeffff640 1 -- 172.21.15.50:0/2120927077 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f7ff8020030 con 0x55cfacdf6dd0 2024-05-08T18:06:51.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.415+0000 7f7ffffff640 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ef003bb50 0x7f7ef003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.415 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.415+0000 7f800bb25800 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfacdf8110 0x55cfacf848f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.415+0000 7f800bb25800 1 -- 172.21.15.50:0/2120927077 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacf84ec0 con 0x55cfacdf8110 2024-05-08T18:06:51.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.415+0000 7f80080b1640 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfacdf8110 0x55cfacf848f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.416+0000 7f7ffffff640 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ef003bb50 0x7f7ef003e010 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0054000 tx=0x7f7ff004e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.416 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.416+0000 7f80080b1640 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfacdf8110 0x55cfacf848f0 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.416+0000 7f80080b1640 1 -- 172.21.15.50:0/2120927077 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f800000a4a0 con 0x55cfacdf8110 2024-05-08T18:06:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.416+0000 7f800bb25800 1 -- 172.21.15.50:0/2120927077 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacf81ec0 con 0x55cfacdf8110 2024-05-08T18:06:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.416+0000 7f80080b1640 1 -- 172.21.15.50:0/2120927077 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f800000a4a0 con 0x55cfacdf8110 2024-05-08T18:06:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.416+0000 7f800bb25800 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfacf871f0 0x55cfacf895e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.416+0000 7f800bb25800 1 -- 172.21.15.50:0/2120927077 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacf89cd0 con 0x55cfacf871f0 2024-05-08T18:06:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.417+0000 7f80078b0640 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfacf871f0 0x55cfacf895e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.417+0000 7f80078b0640 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfacf871f0 0x55cfacf895e0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.424 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.417+0000 7f80078b0640 1 -- 172.21.15.50:0/2120927077 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7ff8043130 con 0x55cfacf871f0 2024-05-08T18:06:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.417+0000 7f800bb25800 1 -- 172.21.15.50:0/2120927077 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacf86090 con 0x55cfacdf8110 2024-05-08T18:06:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.418+0000 7f80080b1640 1 -- 172.21.15.50:0/2120927077 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f800000a4a0 con 0x55cfacdf8110 2024-05-08T18:06:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.418+0000 7f800bb25800 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfacf8b130 0x55cfacf8d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.418+0000 7f800bb25800 1 -- 172.21.15.50:0/2120927077 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacf8dc20 con 0x55cfacf8b130 2024-05-08T18:06:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.418+0000 7f7ffffff640 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfacf8b130 0x55cfacf8d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.418+0000 7f7ffffff640 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfacf8b130 0x55cfacf8d520 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.418+0000 7f7ffffff640 1 -- 172.21.15.50:0/2120927077 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7ff0052960 con 0x55cfacf8b130 2024-05-08T18:06:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.419+0000 7f800bb25800 1 Processor -- start 2024-05-08T18:06:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.419+0000 7f800bb25800 1 -- start start 2024-05-08T18:06:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.419+0000 7f800bb25800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacf9bf90 0x55cfacf9c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.425 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.419+0000 7f800bb25800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cfacd74330 con 0x55cfacf9bf90 2024-05-08T18:06:51.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.419+0000 7f80078b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacf9bf90 0x55cfacf9c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.419+0000 7f80078b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacf9bf90 0x55cfacf9c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46274/0 (socket says 172.21.15.50:46274) 2024-05-08T18:06:51.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.419+0000 7f80078b0640 1 -- 172.21.15.50:0/1172280954 learned_addr learned my addr 172.21.15.50:0/1172280954 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.419+0000 7f80078b0640 1 -- 172.21.15.50:0/1172280954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfacf9c8f0 con 0x55cfacf9bf90 2024-05-08T18:06:51.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.419+0000 7f80078b0640 1 --2- 172.21.15.50:0/1172280954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacf9bf90 0x55cfacf9c3b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8050fa0 tx=0x7f7ff8022680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.420+0000 7f7eed7fa640 1 -- 172.21.15.50:0/1172280954 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff801ab20 con 0x55cfacf9bf90 2024-05-08T18:06:51.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.420+0000 7f7eed7fa640 1 -- 172.21.15.50:0/1172280954 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff801acc0 con 0x55cfacf9bf90 2024-05-08T18:06:51.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.420+0000 7f800bb25800 1 -- 172.21.15.50:0/1172280954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cfacf9d360 con 0x55cfacf9bf90 2024-05-08T18:06:51.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.420+0000 7f7eed7fa640 1 -- 172.21.15.50:0/1172280954 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff8016b00 con 0x55cfacf9bf90 2024-05-08T18:06:51.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.420+0000 7f800bb25800 1 -- 172.21.15.50:0/1172280954 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfacf9d860 con 0x55cfacf9bf90 2024-05-08T18:06:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.420+0000 7f7eed7fa640 1 -- 172.21.15.50:0/1172280954 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7ff8016ca0 con 0x55cfacf9bf90 2024-05-08T18:06:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.421+0000 7f7eed7fa640 1 --2- 172.21.15.50:0/1172280954 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ee003bba0 0x7f7ee003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.421+0000 7f7eed7fa640 1 -- 172.21.15.50:0/1172280954 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f7ff8037d80 con 0x55cfacf9bf90 2024-05-08T18:06:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.421+0000 7f7ffffff640 1 --2- 172.21.15.50:0/1172280954 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ee003bba0 0x7f7ee003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.421+0000 7f800bb25800 1 Processor -- start 2024-05-08T18:06:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.421+0000 7f800bb25800 1 -- start start 2024-05-08T18:06:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.421+0000 7f7ffffff640 1 --2- 172.21.15.50:0/1172280954 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ee003bba0 0x7f7ee003e060 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55cfacf857a0 tx=0x7f7ff0005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.421+0000 7f800bb25800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacfa9150 0x55cfacfa9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.421+0000 7f800bb25800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cfacf764c0 con 0x55cfacfa9150 2024-05-08T18:06:51.427 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.421+0000 7f80078b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacfa9150 0x55cfacfa9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.421+0000 7f80078b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacfa9150 0x55cfacfa9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46290/0 (socket says 172.21.15.50:46290) 2024-05-08T18:06:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.421+0000 7f80078b0640 1 -- 172.21.15.50:0/418041205 learned_addr learned my addr 172.21.15.50:0/418041205 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.422+0000 7f80078b0640 1 -- 172.21.15.50:0/418041205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfacfa9b40 con 0x55cfacfa9150 2024-05-08T18:06:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.422+0000 7f80078b0640 1 --2- 172.21.15.50:0/418041205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacfa9150 0x55cfacfa9570 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800d7e0 tx=0x7f7ff802ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c25a7a867b01beb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.422+0000 7f7ee6ffd640 1 -- 172.21.15.50:0/418041205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff8049290 con 0x55cfacfa9150 2024-05-08T18:06:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.422+0000 7f7ee6ffd640 1 -- 172.21.15.50:0/418041205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff800a330 con 0x55cfacfa9150 2024-05-08T18:06:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.422+0000 7f7ee6ffd640 1 -- 172.21.15.50:0/418041205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff804a190 con 0x55cfacfa9150 2024-05-08T18:06:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.422+0000 7f800bb25800 1 -- 172.21.15.50:0/418041205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacfa9150 msgr2=0x55cfacfa9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.422+0000 7f800bb25800 1 --2- 172.21.15.50:0/418041205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacfa9150 0x55cfacfa9570 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800d7e0 tx=0x7f7ff802ff50 comp rx=0 tx=0).stop 2024-05-08T18:06:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.422+0000 7f800bb25800 1 -- 172.21.15.50:0/418041205 shutdown_connections 2024-05-08T18:06:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.422+0000 7f800bb25800 1 --2- 172.21.15.50:0/418041205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacfa9150 0x55cfacfa9570 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.428 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.422+0000 7f800bb25800 1 -- 172.21.15.50:0/418041205 >> 172.21.15.50:0/418041205 conn(0x55cfacfa5460 msgr2=0x55cfacfa34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.422+0000 7f800bb25800 1 -- 172.21.15.50:0/418041205 shutdown_connections 2024-05-08T18:06:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.423+0000 7f800bb25800 1 -- 172.21.15.50:0/418041205 wait complete. 2024-05-08T18:06:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.423+0000 7f800bb25800 1 Processor -- start 2024-05-08T18:06:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.423+0000 7f800bb25800 1 -- start start 2024-05-08T18:06:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.423+0000 7f800bb25800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacfa9150 0x55cfacfb0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.423+0000 7f800bb25800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cfacfaa1d0 con 0x55cfacfa9150 2024-05-08T18:06:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.423+0000 7f80078b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacfa9150 0x55cfacfb0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.423+0000 7f80078b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacfa9150 0x55cfacfb0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46300/0 (socket says 172.21.15.50:46300) 2024-05-08T18:06:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.423+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 learned_addr learned my addr 172.21.15.50:0/3472047966 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.424+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfacfb0aa0 con 0x55cfacfa9150 2024-05-08T18:06:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.424+0000 7f80078b0640 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacfa9150 0x55cfacfb0560 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800d7e0 tx=0x7f7ff803eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.424+0000 7f7ee57fa640 1 -- 172.21.15.50:0/3472047966 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff8016570 con 0x55cfacfa9150 2024-05-08T18:06:51.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.424+0000 7f7ee57fa640 1 -- 172.21.15.50:0/3472047966 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ff8016710 con 0x55cfacfa9150 2024-05-08T18:06:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.424+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cfacfb0aa0 con 0x55cfacfa9150 2024-05-08T18:06:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.424+0000 7f7ee57fa640 1 -- 172.21.15.50:0/3472047966 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ff803d0e0 con 0x55cfacfa9150 2024-05-08T18:06:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.424+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfacfb0aa0 con 0x55cfacfa9150 2024-05-08T18:06:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.425+0000 7f7ee57fa640 1 -- 172.21.15.50:0/3472047966 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7ff8069070 con 0x55cfacfa9150 2024-05-08T18:06:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.425+0000 7f7ee57fa640 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ed003bba0 0x7f7ed003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.425+0000 7f7ee57fa640 1 -- 172.21.15.50:0/3472047966 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f7ff805a070 con 0x55cfacfa9150 2024-05-08T18:06:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.425+0000 7f7ffffff640 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ed003bba0 0x7f7ed003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.426+0000 7f7ffffff640 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ed003bba0 0x7f7ed003e060 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0052fd0 tx=0x7f7ff004f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.426+0000 7f800bb25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.426+0000 7f800bb25800 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfacfbbc60 0x55cfacfbe120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.426+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfbe870 con 0x55cfacfbbc60 2024-05-08T18:06:51.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.426+0000 7f80080b1640 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfacfbbc60 0x55cfacfbe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.426+0000 7f80080b1640 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfacfbbc60 0x55cfacfbe120 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f800000a4a0 con 0x55cfacfbbc60 2024-05-08T18:06:51.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f800bb25800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f800bb25800 20 realm 2024-05-08T18:06:51.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f800bb25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfbec80 con 0x55cfacfbbc60 2024-05-08T18:06:51.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f800000a4a0 con 0x55cfacfbbc60 2024-05-08T18:06:51.447 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f800bb25800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f800bb25800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:51.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f800bb25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfbf550 con 0x55cfacfbbc60 2024-05-08T18:06:51.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f800000a4a0 con 0x55cfacfbbc60 2024-05-08T18:06:51.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f800bb25800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f800bb25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfbf8f0 con 0x55cfacfbbc60 2024-05-08T18:06:51.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f800000a4a0 con 0x55cfacfbbc60 2024-05-08T18:06:51.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f800bb25800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:51.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f800bb25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f800bb25800 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfacfc0650 0x55cfacfc2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.427+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfc3050 con 0x55cfacfc0650 2024-05-08T18:06:51.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.428+0000 7f80078b0640 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfacfc0650 0x55cfacfc2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.428+0000 7f80078b0640 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfacfc0650 0x55cfacfc2a40 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.428+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.428+0000 7f800bb25800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:51.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.428+0000 7f800bb25800 20 searching for the correct realm 2024-05-08T18:06:51.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.428+0000 7f800bb25800 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfacfc5000 0x55cfacfc74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.428+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfc7b90 con 0x55cfacfc5000 2024-05-08T18:06:51.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.428+0000 7f7ffffff640 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfacfc5000 0x55cfacfc74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.429+0000 7f7ffffff640 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfacfc5000 0x55cfacfc74c0 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.429+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.429+0000 7f7ffffff640 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7ff0076040 0x7f7ff005e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.429+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfc7b90 con 0x7f7ff0076040 2024-05-08T18:06:51.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.429+0000 7f80080b1640 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7ff0076040 0x7f7ff005e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.449 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.429+0000 7f80080b1640 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7ff0076040 0x7f7ff005e480 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.430+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f800000a4a0 con 0x7f7ff0076040 2024-05-08T18:06:51.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.430+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfbf8f0 con 0x55cfacfc0650 2024-05-08T18:06:51.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.430+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.430+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfbf8f0 con 0x55cfacfbbc60 2024-05-08T18:06:51.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.430+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f800000a4a0 con 0x55cfacfbbc60 2024-05-08T18:06:51.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.430+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfbf8f0 con 0x55cfacfbbc60 2024-05-08T18:06:51.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.430+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f800000a4a0 con 0x55cfacfbbc60 2024-05-08T18:06:51.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.430+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfbf8f0 con 0x55cfacfbbc60 2024-05-08T18:06:51.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.431+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f800000a4a0 con 0x55cfacfbbc60 2024-05-08T18:06:51.450 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.431+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfbf8f0 con 0x55cfacfc0650 2024-05-08T18:06:51.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.431+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v29'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.431+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfbf8f0 con 0x7f7ff0076040 2024-05-08T18:06:51.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.431+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f800000a4a0 con 0x7f7ff0076040 2024-05-08T18:06:51.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.431+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfbf8f0 con 0x55cfacfc5000 2024-05-08T18:06:51.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.431+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.431+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfbf8f0 con 0x55cfacfbbc60 2024-05-08T18:06:51.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.432+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f800000a4a0 con 0x55cfacfbbc60 2024-05-08T18:06:51.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.432+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfbf8f0 con 0x55cfacfc0650 2024-05-08T18:06:51.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.432+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.432+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfbf8f0 con 0x55cfacfbbc60 2024-05-08T18:06:51.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.432+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f800000a4a0 con 0x55cfacfbbc60 2024-05-08T18:06:51.451 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.432+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfbf8f0 con 0x7f7ff0076040 2024-05-08T18:06:51.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.432+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f8000032040 con 0x7f7ff0076040 2024-05-08T18:06:51.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.432+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfbf8f0 con 0x55cfacfc5000 2024-05-08T18:06:51.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.433+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.433+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x55cfacfc5000 2024-05-08T18:06:51.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.433+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.433+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x55cfacfc0650 2024-05-08T18:06:51.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.433+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.433+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x55cfacfc5000 2024-05-08T18:06:51.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.433+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.433+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x55cfacfc5000 2024-05-08T18:06:51.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.434+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.452 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.434+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x55cfacfbbc60 2024-05-08T18:06:51.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.434+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.434+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x55cfacfbbc60 2024-05-08T18:06:51.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.434+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v29'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.434+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x55cfacfc0650 2024-05-08T18:06:51.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.435+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.435+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x55cfacfc5000 2024-05-08T18:06:51.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.435+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.435+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x7f7ff0076040 2024-05-08T18:06:51.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.435+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8000032040 con 0x7f7ff0076040 2024-05-08T18:06:51.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.435+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x55cfacfbbc60 2024-05-08T18:06:51.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.435+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.435+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x55cfacfc0650 2024-05-08T18:06:51.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.436+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.453 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.436+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x55cfacfc5000 2024-05-08T18:06:51.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.436+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.436+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x55cfacfc0650 2024-05-08T18:06:51.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.436+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.436+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x7f7ff0076040 2024-05-08T18:06:51.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.437+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8000032040 con 0x7f7ff0076040 2024-05-08T18:06:51.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.437+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x55cfacfc5000 2024-05-08T18:06:51.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.437+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.454 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.437+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x55cfacfc0650 2024-05-08T18:06:51.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.437+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.437+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x55cfacfbbc60 2024-05-08T18:06:51.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.437+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v29'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.437+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7ff0090040 con 0x7f7ff0076040 2024-05-08T18:06:51.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8000032040 con 0x7f7ff0076040 2024-05-08T18:06:51.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f800bb25800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:51.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f800bb25800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:51.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f800bb25800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:51.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f800bb25800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:51.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f800bb25800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:51.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f800bb25800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:51.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f800bb25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.455 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfc3910 con 0x55cfacfbbc60 2024-05-08T18:06:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f800bb25800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f800bb25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfc3910 con 0x55cfacfbbc60 2024-05-08T18:06:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f800bb25800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f800bb25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.438+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfc3910 con 0x55cfacfc5000 2024-05-08T18:06:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f800bb25800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f800bb25800 20 zone default found 2024-05-08T18:06:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f800bb25800 4 Realm: () 2024-05-08T18:06:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f800bb25800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:51.456 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f800bb25800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:51.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f800bb25800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:51.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f800bb25800 20 zonegroup default 2024-05-08T18:06:51.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f800bb25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfc42a0 con 0x55cfacfc0650 2024-05-08T18:06:51.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f800bb25800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f800bb25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfc3e50 con 0x55cfacfc5000 2024-05-08T18:06:51.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f800bb25800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f800bb25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfc9f80 con 0x55cfacfc5000 2024-05-08T18:06:51.457 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.439+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.440+0000 7f800bb25800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.440+0000 7f800bb25800 20 started sync module instance, tier type = 2024-05-08T18:06:51.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.440+0000 7f800bb25800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:51.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.440+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfc9330 con 0x7f7ff0076040 2024-05-08T18:06:51.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.441+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8000032040 con 0x7f7ff0076040 2024-05-08T18:06:51.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.441+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94350448826448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfc9330 con 0x7f7ff0076040 2024-05-08T18:06:51.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.441+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfcaa00 con 0x55cfacfbbc60 2024-05-08T18:06:51.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.442+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94350448826448] v29'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8000032040 con 0x7f7ff0076040 2024-05-08T18:06:51.458 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.442+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.442+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94350448824416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfcaa00 con 0x55cfacfbbc60 2024-05-08T18:06:51.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.442+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfcb420 con 0x55cfacfc5000 2024-05-08T18:06:51.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.443+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.443+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94350448832272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfcbcc0 con 0x55cfacfc5000 2024-05-08T18:06:51.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.443+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfcc7a0 con 0x7f7ff0076040 2024-05-08T18:06:51.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.443+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94350448824416] v29'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.444+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94350448832272] v29'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.444+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8000032040 con 0x7f7ff0076040 2024-05-08T18:06:51.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.444+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94350448834576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfca550 con 0x7f7ff0076040 2024-05-08T18:06:51.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.444+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfcd040 con 0x55cfacfbbc60 2024-05-08T18:06:51.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.445+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94350448834576] v29'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8000032040 con 0x7f7ff0076040 2024-05-08T18:06:51.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.445+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.459 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.445+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94350448837296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfccff0 con 0x55cfacfbbc60 2024-05-08T18:06:51.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.445+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfcda50 con 0x55cfacfc0650 2024-05-08T18:06:51.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.446+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94350448837296] v29'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.446+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.446+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94350448839488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfccf60 con 0x55cfacfc0650 2024-05-08T18:06:51.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.446+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfce790 con 0x55cfacfc0650 2024-05-08T18:06:51.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.447+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94350448839488] v29'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.447+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.447+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94350448844752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfcefe0 con 0x55cfacfc0650 2024-05-08T18:06:51.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.447+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfcf9a0 con 0x55cfacfc0650 2024-05-08T18:06:51.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.448+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94350448844752] v29'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.448+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.449+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94350448848144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfcf9a0 con 0x55cfacfc0650 2024-05-08T18:06:51.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.449+0000 7f800bb25800 20 add_watcher() i=0 2024-05-08T18:06:51.460 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.449+0000 7f800bb25800 20 add_watcher() i=1 2024-05-08T18:06:51.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.449+0000 7f800bb25800 20 add_watcher() i=2 2024-05-08T18:06:51.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.449+0000 7f800bb25800 20 add_watcher() i=3 2024-05-08T18:06:51.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.449+0000 7f800bb25800 20 add_watcher() i=4 2024-05-08T18:06:51.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.449+0000 7f800bb25800 20 add_watcher() i=5 2024-05-08T18:06:51.461 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.449+0000 7f800bb25800 20 add_watcher() i=6 2024-05-08T18:06:51.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.449+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94350448848144] v29'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.449+0000 7f800bb25800 20 add_watcher() i=7 2024-05-08T18:06:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.449+0000 7f800bb25800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.450+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfd2d00 con 0x55cfacfbbc60 2024-05-08T18:06:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.450+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.450+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94350448862976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfcfb10 con 0x55cfacfbbc60 2024-05-08T18:06:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.451+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94350448862976] v29'243 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.451+0000 7f7ebdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.451+0000 7f800bb25800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.451+0000 7f800bb25800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.451+0000 7f800bb25800 5 note: GC not initialized 2024-05-08T18:06:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.451+0000 7f7ebcfe1640 20 reqs_thread_entry: start 2024-05-08T18:06:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.451+0000 7f800bb25800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.451+0000 7f800bb25800 20 Filter name: none 2024-05-08T18:06:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.452+0000 7f7e367fc640 20 reqs_thread_entry: start 2024-05-08T18:06:51.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.452+0000 7f800bb25800 10 cache get: name=default.rgw.meta+users.uid+system_user : miss 2024-05-08T18:06:51.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.452+0000 7f800bb25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.452+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.1e 8:78ce4db0:users.uid::system_user:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfad006300 con 0x55cfacfc0650 2024-05-08T18:06:51.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.452+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(62 system_user [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 239+0+0 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.452+0000 7f800bb25800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.452+0000 7f800bb25800 10 cache put: name=default.rgw.meta+users.uid+system_user info.flags=0x0 2024-05-08T18:06:51.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.452+0000 7f800bb25800 10 adding default.rgw.meta+users.uid+system_user to cache LRU end 2024-05-08T18:06:51.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.452+0000 7f800bb25800 10 cache get: name=default.rgw.meta+users.keys+system_akey : miss 2024-05-08T18:06:51.468 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.452+0000 7f800bb25800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.452+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:63 8.0 8:01223cbf:users.keys::system_akey:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfad004aa0 con 0x55cfacfc5000 2024-05-08T18:06:51.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.452+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(63 system_akey [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 197+0+0 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.453+0000 7f800bb25800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.453+0000 7f800bb25800 10 cache put: name=default.rgw.meta+users.keys+system_akey info.flags=0x0 2024-05-08T18:06:51.469 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.453+0000 7f800bb25800 10 adding default.rgw.meta+users.keys+system_akey to cache LRU end 2024-05-08T18:06:51.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.453+0000 7f800bb25800 10 cache get: name=default.rgw.meta+users.keys+system_akey : hit (negative entry) 2024-05-08T18:06:51.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.453+0000 7f800bb25800 10 cache get: name=default.rgw.meta+users.keys+system_akey : hit (negative entry) 2024-05-08T18:06:51.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.453+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:64 8.1e 8:78ce4db0:users.uid::system_user:head [delete,create,call version.set in=58b,writefull 0~349 in=349b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfad005340 con 0x55cfacfc0650 2024-05-08T18:06:51.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.454+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(64 system_user [delete,create,call,writefull 0~349] v29'9 uv9 ondisk = 0) ==== 281+0+0 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.454+0000 7f800bb25800 10 cache put: name=default.rgw.meta+users.uid+system_user info.flags=0x17 2024-05-08T18:06:51.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.454+0000 7f800bb25800 10 moving default.rgw.meta+users.uid+system_user to cache LRU end 2024-05-08T18:06:51.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.454+0000 7f800bb25800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:system_user, ofs0, ns] 2024-05-08T18:06:51.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.454+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94350449069712 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfad0098f0 con 0x7f7ff0076040 2024-05-08T18:06:51.470 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.454+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== watch-notify(notify (1) cookie 94350448826448 notify 124554051607 ret 0) ==== 575+0+0 (crc 0 0 0) 0x7f800003d040 con 0x7f7ff0076040 2024-05-08T18:06:51.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.454+0000 7f7ee6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051607 cookie 94350448826448 notifier 4837 bl.length()=533 2024-05-08T18:06:51.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.454+0000 7f7ee6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+system_user info.flags=0x17 2024-05-08T18:06:51.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.454+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(65 notify.0 [notify cookie 94350449069712 out=8b] v0'0 uv246 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f8000032040 con 0x7f7ff0076040 2024-05-08T18:06:51.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.454+0000 7f7ee6ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+system_user to cache LRU end 2024-05-08T18:06:51.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.454+0000 7f7ee6ffd640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f7ecc003ea0 con 0x7f7ff0076040 2024-05-08T18:06:51.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.454+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(66 notify.0 [notify-ack] v0'0 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8000032040 con 0x7f7ff0076040 2024-05-08T18:06:51.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.455+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== watch-notify(notify_complete (2) cookie 94350449069712 notify 124554051607 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f8000018b10 con 0x7f7ff0076040 2024-05-08T18:06:51.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.455+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 8.0 8:01223cbf:users.keys::system_akey:head [delete,create,writefull 0~15 in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfad005340 con 0x55cfacfc5000 2024-05-08T18:06:51.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.456+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 system_akey [delete,create,writefull 0~15] v29'3 uv3 ondisk = 0) ==== 239+0+0 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.456+0000 7f800bb25800 10 cache put: name=default.rgw.meta+users.keys+system_akey info.flags=0x7 2024-05-08T18:06:51.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.456+0000 7f800bb25800 10 moving default.rgw.meta+users.keys+system_akey to cache LRU end 2024-05-08T18:06:51.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.456+0000 7f800bb25800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.keys:system_akey, ofs0, ns] 2024-05-08T18:06:51.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.456+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [notify cookie 94350449069712 in=188b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55cfad009ce0 con 0x55cfacfbbc60 2024-05-08T18:06:51.471 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.456+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== watch-notify(notify (1) cookie 94350448824416 notify 124554051618 ret 0) ==== 218+0+0 (crc 0 0 0) 0x7f8000018b10 con 0x55cfacfbbc60 2024-05-08T18:06:51.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.456+0000 7f7ee6ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051618 cookie 94350448824416 notifier 4837 bl.length()=176 2024-05-08T18:06:51.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.456+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(68 notify.1 [notify cookie 94350449069712 out=8b] v0'0 uv246 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.456+0000 7f7ee6ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+system_akey info.flags=0x7 2024-05-08T18:06:51.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.456+0000 7f7ee6ffd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+system_akey to cache LRU end 2024-05-08T18:06:51.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.456+0000 7f7ee6ffd640 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f7ecc003ea0 con 0x55cfacfbbc60 2024-05-08T18:06:51.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.456+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== watch-notify(notify_complete (2) cookie 94350449069712 notify 124554051618 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f8000037440 con 0x55cfacfbbc60 2024-05-08T18:06:51.472 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.456+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(69 notify.1 [notify-ack] v0'0 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.472 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"system_user","display_name":"System User","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"system_user","access_key":"system_akey","secret_key":"system_skey","active":true,"create_date":"2024-05-08T18:06:51.453741Z"}],"swift_keys":[],"caps":[{"type":"user-info-without-keys","perm":"read"}],"op_mask":"read, write, delete","system":true,"default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:06:51.453727Z","tags":[],"group_ids":[]} 2024-05-08T18:06:51.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.457+0000 7f7ebdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:51.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.457+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94350448862976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfd1a30 con 0x55cfacfbbc60 2024-05-08T18:06:51.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.458+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94350448862976] v29'244 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.458+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94350448826448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfd2700 con 0x7f7ff0076040 2024-05-08T18:06:51.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.459+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94350448826448] v29'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8000032040 con 0x7f7ff0076040 2024-05-08T18:06:51.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.459+0000 7f800bb25800 20 remove_watcher() i=0 2024-05-08T18:06:51.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.459+0000 7f800bb25800 2 removed watcher, disabling cache 2024-05-08T18:06:51.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.459+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94350448824416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfc9830 con 0x55cfacfbbc60 2024-05-08T18:06:51.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.461+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94350448824416] v29'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.461+0000 7f800bb25800 20 remove_watcher() i=1 2024-05-08T18:06:51.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.461+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94350448832272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfc9060 con 0x55cfacfc5000 2024-05-08T18:06:51.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.462+0000 7f7ffffff640 1 -- 172.21.15.50:0/3472047966 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94350448832272] v29'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ff0081070 con 0x55cfacfc5000 2024-05-08T18:06:51.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.462+0000 7f800bb25800 20 remove_watcher() i=2 2024-05-08T18:06:51.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.462+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94350448834576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfc9830 con 0x7f7ff0076040 2024-05-08T18:06:51.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.463+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94350448834576] v29'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8000032040 con 0x7f7ff0076040 2024-05-08T18:06:51.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.463+0000 7f800bb25800 20 remove_watcher() i=3 2024-05-08T18:06:51.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.463+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94350448837296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfcb810 con 0x55cfacfbbc60 2024-05-08T18:06:51.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.464+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94350448837296] v29'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8000032040 con 0x55cfacfbbc60 2024-05-08T18:06:51.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.464+0000 7f800bb25800 20 remove_watcher() i=4 2024-05-08T18:06:51.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.464+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94350448839488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfcc2b0 con 0x55cfacfc0650 2024-05-08T18:06:51.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.465+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94350448839488] v29'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.465+0000 7f800bb25800 20 remove_watcher() i=5 2024-05-08T18:06:51.480 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.465+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94350448844752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfccb40 con 0x55cfacfc0650 2024-05-08T18:06:51.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.466+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94350448844752] v29'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.466+0000 7f800bb25800 20 remove_watcher() i=6 2024-05-08T18:06:51.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.466+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94350448848144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55cfacfccb40 con 0x55cfacfc0650 2024-05-08T18:06:51.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f80078b0640 1 -- 172.21.15.50:0/3472047966 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94350448848144] v29'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7ff8072020 con 0x55cfacfc0650 2024-05-08T18:06:51.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 20 remove_watcher() i=7 2024-05-08T18:06:51.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfacfc0650 msgr2=0x55cfacfc2a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfacfc0650 0x55cfacfc2a40 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfacfbbc60 msgr2=0x55cfacfbe120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfacfbbc60 0x55cfacfbe120 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7ff0076040 msgr2=0x7f7ff005e480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7ff0076040 0x7f7ff005e480 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfacfc5000 msgr2=0x55cfacfc74c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfacfc5000 0x55cfacfc74c0 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ed003bba0 msgr2=0x7f7ed003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ed003bba0 0x7f7ed003e060 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0052fd0 tx=0x7f7ff004f000 comp rx=0 tx=0).stop 2024-05-08T18:06:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacfa9150 msgr2=0x55cfacfb0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacfa9150 0x55cfacfb0560 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800d7e0 tx=0x7f7ff803eed0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f80080b1640 1 -- 172.21.15.50:0/3472047966 reap_dead start 2024-05-08T18:06:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 shutdown_connections 2024-05-08T18:06:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 --2- 172.21.15.50:0/3472047966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacfa9150 0x55cfacfb0560 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 >> 172.21.15.50:0/3472047966 conn(0x55cfacfa5460 msgr2=0x55cfacfa2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 shutdown_connections 2024-05-08T18:06:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.467+0000 7f800bb25800 1 -- 172.21.15.50:0/3472047966 wait complete. 2024-05-08T18:06:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 -- 172.21.15.50:0/1172280954 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ee003bba0 msgr2=0x7f7ee003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 --2- 172.21.15.50:0/1172280954 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ee003bba0 0x7f7ee003e060 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55cfacf857a0 tx=0x7f7ff0005a00 comp rx=0 tx=0).stop 2024-05-08T18:06:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 -- 172.21.15.50:0/1172280954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacf9bf90 msgr2=0x55cfacf9c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 --2- 172.21.15.50:0/1172280954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacf9bf90 0x55cfacf9c3b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8050fa0 tx=0x7f7ff8022680 comp rx=0 tx=0).stop 2024-05-08T18:06:51.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 -- 172.21.15.50:0/1172280954 shutdown_connections 2024-05-08T18:06:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 --2- 172.21.15.50:0/1172280954 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ee003bba0 0x7f7ee003e060 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 --2- 172.21.15.50:0/1172280954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacf9bf90 0x55cfacf9c3b0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 -- 172.21.15.50:0/1172280954 >> 172.21.15.50:0/1172280954 conn(0x55cfacf90030 msgr2=0x55cfacf90430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 -- 172.21.15.50:0/1172280954 shutdown_connections 2024-05-08T18:06:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 -- 172.21.15.50:0/1172280954 wait complete. 2024-05-08T18:06:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 -- 172.21.15.50:0/2120927077 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfacf871f0 msgr2=0x55cfacf895e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55cfacf871f0 0x55cfacf895e0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 -- 172.21.15.50:0/2120927077 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfacdf8110 msgr2=0x55cfacf848f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55cfacdf8110 0x55cfacf848f0 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 -- 172.21.15.50:0/2120927077 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfacf8b130 msgr2=0x55cfacf8d520 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55cfacf8b130 0x55cfacf8d520 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 -- 172.21.15.50:0/2120927077 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ef003bb50 msgr2=0x7f7ef003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7ef003bb50 0x7f7ef003e010 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0054000 tx=0x7f7ff004e000 comp rx=0 tx=0).stop 2024-05-08T18:06:51.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 -- 172.21.15.50:0/2120927077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 msgr2=0x55cfacf7d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 --2- 172.21.15.50:0/2120927077 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfacdf6dd0 0x55cfacf7d750 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x55cfacf76900 tx=0x7f7ff8000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f80080b1640 1 -- 172.21.15.50:0/2120927077 reap_dead start 2024-05-08T18:06:51.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 -- 172.21.15.50:0/2120927077 shutdown_connections 2024-05-08T18:06:51.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 -- 172.21.15.50:0/2120927077 >> 172.21.15.50:0/2120927077 conn(0x55cface7dce0 msgr2=0x55cfacf6fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 -- 172.21.15.50:0/2120927077 shutdown_connections 2024-05-08T18:06:51.484 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.468+0000 7f800bb25800 1 -- 172.21.15.50:0/2120927077 wait complete. 2024-05-08T18:06:51.484 DEBUG:tasks.util.rgw: json result: {'user_id': 'system_user', 'display_name': 'System User', 'email': '', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'system_user', 'access_key': 'system_akey', 'secret_key': 'system_skey', 'active': True, 'create_date': '2024-05-08T18:06:51.453741Z'}], 'swift_keys': [], 'caps': [{'type': 'user-info-without-keys', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'system': True, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.453727Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:51.485 ERROR:root:{'user_id': 'system_user', 'display_name': 'System User', 'email': '', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'system_user', 'access_key': 'system_akey', 'secret_key': 'system_skey', 'active': True, 'create_date': '2024-05-08T18:06:51.453741Z'}], 'swift_keys': [], 'caps': [{'type': 'user-info-without-keys', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'system': True, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.453727Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:51.485 ERROR:root:0 2024-05-08T18:06:51.485 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:51.490 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:51.491 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'system_user'] 2024-05-08T18:06:51.491 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'system_user'] 2024-05-08T18:06:51.491 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid system_user 2024-05-08T18:06:51.537 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:51.537 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:51.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.539+0000 7f390f12d800 1 Processor -- start 2024-05-08T18:06:51.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.539+0000 7f390f12d800 1 -- start start 2024-05-08T18:06:51.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.539+0000 7f390f12d800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55d721cbbfb0 0x55d721cbc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.539+0000 7f390f12d800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d721c31130 con 0x55d721cbc910 2024-05-08T18:06:51.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.539+0000 7f390f12d800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55d721b5de20 con 0x55d721cbbfb0 2024-05-08T18:06:51.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.540+0000 7f390ad71640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55d721cbbfb0 0x55d721cbc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.540+0000 7f390a570640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55d721cbc910 0x55d721cbcdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50838/0 (socket says 172.21.15.50:50838) 2024-05-08T18:06:51.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.540+0000 7f390a570640 1 -- 172.21.15.50:0/1192364629 learned_addr learned my addr 172.21.15.50:0/1192364629 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:51.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.540+0000 7f390ad71640 1 -- 172.21.15.50:0/1192364629 >> v1:172.21.15.46:6789/0 conn(0x55d721cbc910 legacy=0x55d721cbcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.540+0000 7f390ad71640 1 -- 172.21.15.50:0/1192364629 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d721cbd4d0 con 0x55d721cbbfb0 2024-05-08T18:06:51.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.542+0000 7f390ad71640 1 --2- 172.21.15.50:0/1192364629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 0x55d721cbc3d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f38fc005fd0 tx=0x7f38fc01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd3e1595f91bbbc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.542+0000 7f3909d6f640 1 -- 172.21.15.50:0/1192364629 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc052560 con 0x55d721cbbfb0 2024-05-08T18:06:51.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.542+0000 7f3909d6f640 1 -- 172.21.15.50:0/1192364629 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38fc052700 con 0x55d721cbbfb0 2024-05-08T18:06:51.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.542+0000 7f3909d6f640 1 -- 172.21.15.50:0/1192364629 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc052a60 con 0x55d721cbbfb0 2024-05-08T18:06:51.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.543+0000 7f390f12d800 1 -- 172.21.15.50:0/1192364629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 msgr2=0x55d721cbc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.543+0000 7f390f12d800 1 --2- 172.21.15.50:0/1192364629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 0x55d721cbc3d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f38fc005fd0 tx=0x7f38fc01ccc0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.543+0000 7f390f12d800 1 -- 172.21.15.50:0/1192364629 shutdown_connections 2024-05-08T18:06:51.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.543+0000 7f390f12d800 1 --2- 172.21.15.50:0/1192364629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 0x55d721cbc3d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.543+0000 7f390f12d800 1 -- 172.21.15.50:0/1192364629 >> 172.21.15.50:0/1192364629 conn(0x55d721cb7ad0 msgr2=0x55d721cb9f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.543+0000 7f390f12d800 1 -- 172.21.15.50:0/1192364629 shutdown_connections 2024-05-08T18:06:51.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.543+0000 7f390f12d800 1 -- 172.21.15.50:0/1192364629 wait complete. 2024-05-08T18:06:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.558+0000 7f390f12d800 1 Processor -- start 2024-05-08T18:06:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.559+0000 7f390f12d800 1 -- start start 2024-05-08T18:06:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.559+0000 7f390f12d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 0x55d721db0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.559+0000 7f390f12d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d721c08a10 con 0x55d721cbbfb0 2024-05-08T18:06:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.559+0000 7f390ad71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 0x55d721db0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.559+0000 7f390ad71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 0x55d721db0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46306/0 (socket says 172.21.15.50:46306) 2024-05-08T18:06:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.559+0000 7f390ad71640 1 -- 172.21.15.50:0/1099307087 learned_addr learned my addr 172.21.15.50:0/1099307087 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.559+0000 7f390ad71640 1 -- 172.21.15.50:0/1099307087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d721db14f0 con 0x55d721cbbfb0 2024-05-08T18:06:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.560+0000 7f390ad71640 1 --2- 172.21.15.50:0/1099307087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 0x55d721db0f20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f38fc054040 tx=0x7f38fc04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1caafff3ca683797 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.560+0000 7f37f8e09640 1 -- 172.21.15.50:0/1099307087 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc0529e0 con 0x55d721cbbfb0 2024-05-08T18:06:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.560+0000 7f37f8e09640 1 -- 172.21.15.50:0/1099307087 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38fc052b80 con 0x55d721cbbfb0 2024-05-08T18:06:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.560+0000 7f37f8e09640 1 -- 172.21.15.50:0/1099307087 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc05c430 con 0x55d721cbbfb0 2024-05-08T18:06:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.561+0000 7f390f12d800 1 -- 172.21.15.50:0/1099307087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 msgr2=0x55d721db0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.561+0000 7f390f12d800 1 --2- 172.21.15.50:0/1099307087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 0x55d721db0f20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f38fc054040 tx=0x7f38fc04c000 comp rx=0 tx=0).stop 2024-05-08T18:06:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.561+0000 7f390f12d800 1 -- 172.21.15.50:0/1099307087 shutdown_connections 2024-05-08T18:06:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.561+0000 7f390f12d800 1 --2- 172.21.15.50:0/1099307087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 0x55d721db0f20 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.561+0000 7f390f12d800 1 -- 172.21.15.50:0/1099307087 >> 172.21.15.50:0/1099307087 conn(0x55d721cb7ad0 msgr2=0x55d721dac840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.561+0000 7f390f12d800 1 -- 172.21.15.50:0/1099307087 shutdown_connections 2024-05-08T18:06:51.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.561+0000 7f390f12d800 1 -- 172.21.15.50:0/1099307087 wait complete. 2024-05-08T18:06:51.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.561+0000 7f390f12d800 1 Processor -- start 2024-05-08T18:06:51.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.561+0000 7f390f12d800 1 -- start start 2024-05-08T18:06:51.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.561+0000 7f390f12d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 0x55d721db8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.561+0000 7f390f12d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d721cbda00 con 0x55d721cbbfb0 2024-05-08T18:06:51.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.561+0000 7f390ad71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 0x55d721db8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.561+0000 7f390ad71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 0x55d721db8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46320/0 (socket says 172.21.15.50:46320) 2024-05-08T18:06:51.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.561+0000 7f390ad71640 1 -- 172.21.15.50:0/3000254693 learned_addr learned my addr 172.21.15.50:0/3000254693 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.562+0000 7f390ad71640 1 -- 172.21.15.50:0/3000254693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d721db8eb0 con 0x55d721cbbfb0 2024-05-08T18:06:51.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.562+0000 7f390ad71640 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 0x55d721db8970 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f38fc044920 tx=0x7f38fc003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.562+0000 7f37f7606640 1 -- 172.21.15.50:0/3000254693 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc066540 con 0x55d721cbbfb0 2024-05-08T18:06:51.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.562+0000 7f37f7606640 1 -- 172.21.15.50:0/3000254693 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38fc0666e0 con 0x55d721cbbfb0 2024-05-08T18:06:51.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.562+0000 7f390f12d800 1 -- 172.21.15.50:0/3000254693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d721db9140 con 0x55d721cbbfb0 2024-05-08T18:06:51.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.562+0000 7f37f7606640 1 -- 172.21.15.50:0/3000254693 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc066a00 con 0x55d721cbbfb0 2024-05-08T18:06:51.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.562+0000 7f390f12d800 1 -- 172.21.15.50:0/3000254693 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d721dbbdf0 con 0x55d721cbbfb0 2024-05-08T18:06:51.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.563+0000 7f37f7606640 1 -- 172.21.15.50:0/3000254693 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f38fc078020 con 0x55d721cbbfb0 2024-05-08T18:06:51.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.563+0000 7f37f7606640 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f37f003bb50 0x7f37f003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.563+0000 7f37f7606640 1 -- 172.21.15.50:0/3000254693 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f38fc094eb0 con 0x55d721cbbfb0 2024-05-08T18:06:51.590 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.563+0000 7f390a570640 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f37f003bb50 0x7f37f003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.563+0000 7f390f12d800 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d721cbc910 0x55d721dbfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.563+0000 7f390f12d800 1 -- 172.21.15.50:0/3000254693 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721dc0180 con 0x55d721cbc910 2024-05-08T18:06:51.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.563+0000 7f390b572640 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d721cbc910 0x55d721dbfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.563+0000 7f390a570640 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f37f003bb50 0x7f37f003e010 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3900001d70 tx=0x7f39000012d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.564+0000 7f390b572640 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d721cbc910 0x55d721dbfbb0 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.564+0000 7f390b572640 1 -- 172.21.15.50:0/3000254693 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f39040083e0 con 0x55d721cbc910 2024-05-08T18:06:51.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.564+0000 7f390f12d800 1 -- 172.21.15.50:0/3000254693 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721dbd180 con 0x55d721cbc910 2024-05-08T18:06:51.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.564+0000 7f390b572640 1 -- 172.21.15.50:0/3000254693 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f39040083e0 con 0x55d721cbc910 2024-05-08T18:06:51.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.564+0000 7f390f12d800 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d721dc24b0 0x55d721dc48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.591 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.564+0000 7f390f12d800 1 -- 172.21.15.50:0/3000254693 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721dc4f70 con 0x55d721dc24b0 2024-05-08T18:06:51.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.565+0000 7f390ad71640 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d721dc24b0 0x55d721dc48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.565+0000 7f390ad71640 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d721dc24b0 0x55d721dc48a0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.565+0000 7f390ad71640 1 -- 172.21.15.50:0/3000254693 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dc24b0 2024-05-08T18:06:51.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.565+0000 7f390f12d800 1 -- 172.21.15.50:0/3000254693 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721dc1350 con 0x55d721cbc910 2024-05-08T18:06:51.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.565+0000 7f390b572640 1 -- 172.21.15.50:0/3000254693 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f39040083e0 con 0x55d721cbc910 2024-05-08T18:06:51.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.566+0000 7f390f12d800 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d721dc63d0 0x55d721dc87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.566+0000 7f390f12d800 1 -- 172.21.15.50:0/3000254693 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721dc8f30 con 0x55d721dc63d0 2024-05-08T18:06:51.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.566+0000 7f390a570640 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d721dc63d0 0x55d721dc87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.566+0000 7f390a570640 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d721dc63d0 0x55d721dc87c0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.566+0000 7f390a570640 1 -- 172.21.15.50:0/3000254693 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3900009050 con 0x55d721dc63d0 2024-05-08T18:06:51.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.567+0000 7f390f12d800 1 Processor -- start 2024-05-08T18:06:51.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.567+0000 7f390f12d800 1 -- start start 2024-05-08T18:06:51.592 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.567+0000 7f390f12d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721dd7260 0x55d721dd7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.567+0000 7f390f12d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d721baf330 con 0x55d721dd7260 2024-05-08T18:06:51.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.567+0000 7f390ad71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721dd7260 0x55d721dd7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.567+0000 7f390ad71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721dd7260 0x55d721dd7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46324/0 (socket says 172.21.15.50:46324) 2024-05-08T18:06:51.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.567+0000 7f390ad71640 1 -- 172.21.15.50:0/3516173055 learned_addr learned my addr 172.21.15.50:0/3516173055 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.567+0000 7f390ad71640 1 -- 172.21.15.50:0/3516173055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d721dd7bc0 con 0x55d721dd7260 2024-05-08T18:06:51.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.567+0000 7f390ad71640 1 --2- 172.21.15.50:0/3516173055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721dd7260 0x55d721dd7680 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f38fc093260 tx=0x7f38fc071c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.568+0000 7f37f4e01640 1 -- 172.21.15.50:0/3516173055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc08c2e0 con 0x55d721dd7260 2024-05-08T18:06:51.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.568+0000 7f37f4e01640 1 -- 172.21.15.50:0/3516173055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38fc08c480 con 0x55d721dd7260 2024-05-08T18:06:51.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.568+0000 7f37f4e01640 1 -- 172.21.15.50:0/3516173055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc08d840 con 0x55d721dd7260 2024-05-08T18:06:51.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.568+0000 7f390f12d800 1 -- 172.21.15.50:0/3516173055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d721dd8630 con 0x55d721dd7260 2024-05-08T18:06:51.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.568+0000 7f390f12d800 1 -- 172.21.15.50:0/3516173055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d721dd8b30 con 0x55d721dd7260 2024-05-08T18:06:51.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.568+0000 7f37f4e01640 1 -- 172.21.15.50:0/3516173055 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f38fc08d9e0 con 0x55d721dd7260 2024-05-08T18:06:51.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.569+0000 7f37f4e01640 1 --2- 172.21.15.50:0/3516173055 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f37e403bb50 0x7f37e403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.569+0000 7f37f4e01640 1 -- 172.21.15.50:0/3516173055 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f38fc080040 con 0x55d721dd7260 2024-05-08T18:06:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.569+0000 7f390a570640 1 --2- 172.21.15.50:0/3516173055 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f37e403bb50 0x7f37e403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.569+0000 7f390f12d800 1 Processor -- start 2024-05-08T18:06:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.569+0000 7f390a570640 1 --2- 172.21.15.50:0/3516173055 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f37e403bb50 0x7f37e403e010 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f390000bf40 tx=0x7f390000f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.569+0000 7f390f12d800 1 -- start start 2024-05-08T18:06:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.569+0000 7f390f12d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721de43d0 0x55d721de47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.569+0000 7f390f12d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d721db17b0 con 0x55d721de43d0 2024-05-08T18:06:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.569+0000 7f390ad71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721de43d0 0x55d721de47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.570+0000 7f390ad71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721de43d0 0x55d721de47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46328/0 (socket says 172.21.15.50:46328) 2024-05-08T18:06:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.570+0000 7f390ad71640 1 -- 172.21.15.50:0/3249828769 learned_addr learned my addr 172.21.15.50:0/3249828769 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.570+0000 7f390ad71640 1 -- 172.21.15.50:0/3249828769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d721de4dc0 con 0x55d721de43d0 2024-05-08T18:06:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.570+0000 7f390ad71640 1 --2- 172.21.15.50:0/3249828769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721de43d0 0x55d721de47f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f38fc05cfd0 tx=0x7f38fc0986d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260fd8de9c8f3628 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.570+0000 7f37ee7fc640 1 -- 172.21.15.50:0/3249828769 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc0ac070 con 0x55d721de43d0 2024-05-08T18:06:51.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.570+0000 7f37ee7fc640 1 -- 172.21.15.50:0/3249828769 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38fc079d00 con 0x55d721de43d0 2024-05-08T18:06:51.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.570+0000 7f37ee7fc640 1 -- 172.21.15.50:0/3249828769 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc083ad0 con 0x55d721de43d0 2024-05-08T18:06:51.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.571+0000 7f390f12d800 1 -- 172.21.15.50:0/3249828769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721de43d0 msgr2=0x55d721de47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.571+0000 7f390f12d800 1 --2- 172.21.15.50:0/3249828769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721de43d0 0x55d721de47f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f38fc05cfd0 tx=0x7f38fc0986d0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.571+0000 7f390f12d800 1 -- 172.21.15.50:0/3249828769 shutdown_connections 2024-05-08T18:06:51.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.571+0000 7f390f12d800 1 --2- 172.21.15.50:0/3249828769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721de43d0 0x55d721de47f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.571+0000 7f390f12d800 1 -- 172.21.15.50:0/3249828769 >> 172.21.15.50:0/3249828769 conn(0x55d721de06e0 msgr2=0x55d721dde740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.571+0000 7f390f12d800 1 -- 172.21.15.50:0/3249828769 shutdown_connections 2024-05-08T18:06:51.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.571+0000 7f390f12d800 1 -- 172.21.15.50:0/3249828769 wait complete. 2024-05-08T18:06:51.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.571+0000 7f390f12d800 1 Processor -- start 2024-05-08T18:06:51.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.571+0000 7f390f12d800 1 -- start start 2024-05-08T18:06:51.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.571+0000 7f390f12d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721de43d0 0x55d721deb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.571+0000 7f390f12d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d721de5450 con 0x55d721de43d0 2024-05-08T18:06:51.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.571+0000 7f390ad71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721de43d0 0x55d721deb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.571+0000 7f390ad71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721de43d0 0x55d721deb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46330/0 (socket says 172.21.15.50:46330) 2024-05-08T18:06:51.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.571+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 learned_addr learned my addr 172.21.15.50:0/2172543379 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.572+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d721debcf0 con 0x55d721de43d0 2024-05-08T18:06:51.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.572+0000 7f390ad71640 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721de43d0 0x55d721deb7b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f38fc06af40 tx=0x7f38fc06aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.572+0000 7f37ecff9640 1 -- 172.21.15.50:0/2172543379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc0b7020 con 0x55d721de43d0 2024-05-08T18:06:51.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.572+0000 7f37ecff9640 1 -- 172.21.15.50:0/2172543379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38fc0ad040 con 0x55d721de43d0 2024-05-08T18:06:51.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.572+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d721debcf0 con 0x55d721de43d0 2024-05-08T18:06:51.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.572+0000 7f37ecff9640 1 -- 172.21.15.50:0/2172543379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38fc0ac070 con 0x55d721de43d0 2024-05-08T18:06:51.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.572+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d721debcf0 con 0x55d721de43d0 2024-05-08T18:06:51.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.573+0000 7f37ecff9640 1 -- 172.21.15.50:0/2172543379 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f38fc0ba020 con 0x55d721de43d0 2024-05-08T18:06:51.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.573+0000 7f37ecff9640 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f37d403bb50 0x7f37d403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.573+0000 7f37ecff9640 1 -- 172.21.15.50:0/2172543379 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f38fc0a4070 con 0x55d721de43d0 2024-05-08T18:06:51.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.573+0000 7f390a570640 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f37d403bb50 0x7f37d403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.574+0000 7f390a570640 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f37d403bb50 0x7f37d403e010 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f390000c010 tx=0x7f3900010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.574+0000 7f390f12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.574+0000 7f390f12d800 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d721df6e00 0x55d721df92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.574+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721df9a10 con 0x55d721df6e00 2024-05-08T18:06:51.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.574+0000 7f390b572640 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d721df6e00 0x55d721df92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.574+0000 7f390b572640 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d721df6e00 0x55d721df92c0 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f39040083e0 con 0x55d721df6e00 2024-05-08T18:06:51.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390f12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390f12d800 20 realm 2024-05-08T18:06:51.600 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390f12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721df9e20 con 0x55d721df6e00 2024-05-08T18:06:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f39040083e0 con 0x55d721df6e00 2024-05-08T18:06:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390f12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390f12d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390f12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721df6e00 2024-05-08T18:06:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f39040083e0 con 0x55d721df6e00 2024-05-08T18:06:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390f12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390f12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfaa90 con 0x55d721df6e00 2024-05-08T18:06:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f39040083e0 con 0x55d721df6e00 2024-05-08T18:06:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390f12d800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:51.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390f12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390f12d800 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d721dfb450 0x55d721dfd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.575+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfde00 con 0x55d721dfb450 2024-05-08T18:06:51.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.576+0000 7f390ad71640 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d721dfb450 0x55d721dfd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.576+0000 7f390ad71640 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d721dfb450 0x55d721dfd840 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.576+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.576+0000 7f390f12d800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:51.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.576+0000 7f390f12d800 20 searching for the correct realm 2024-05-08T18:06:51.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.576+0000 7f390f12d800 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d721dffdb0 0x55d721e02270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.576+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721e02940 con 0x55d721dffdb0 2024-05-08T18:06:51.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.576+0000 7f390a570640 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d721dffdb0 0x55d721e02270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.602 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.577+0000 7f390a570640 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d721dffdb0 0x55d721e02270 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.577+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.577+0000 7f390a570640 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3900037040 0x7f3900036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.577+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721e02940 con 0x7f3900037040 2024-05-08T18:06:51.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.577+0000 7f390b572640 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3900037040 0x7f3900036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.577+0000 7f390b572640 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3900037040 0x7f3900036410 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.578+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f39040083e0 con 0x7f3900037040 2024-05-08T18:06:51.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.578+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721dfb450 2024-05-08T18:06:51.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.578+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.578+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721df6e00 2024-05-08T18:06:51.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.578+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f39040083e0 con 0x55d721df6e00 2024-05-08T18:06:51.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.578+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721df6e00 2024-05-08T18:06:51.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.578+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.578+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721df6e00 2024-05-08T18:06:51.603 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.579+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.579+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721dfb450 2024-05-08T18:06:51.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.579+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v29'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.579+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x7f3900037040 2024-05-08T18:06:51.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.579+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.579+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721dffdb0 2024-05-08T18:06:51.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.580+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.580+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721df6e00 2024-05-08T18:06:51.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.580+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.580+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721dfb450 2024-05-08T18:06:51.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.580+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.580+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721df6e00 2024-05-08T18:06:51.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.580+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.604 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.580+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x7f3900037040 2024-05-08T18:06:51.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.581+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.581+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721dffdb0 2024-05-08T18:06:51.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.581+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.581+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721dffdb0 2024-05-08T18:06:51.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.581+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.581+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721dfb450 2024-05-08T18:06:51.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.581+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.581+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721dffdb0 2024-05-08T18:06:51.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.582+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.582+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721dffdb0 2024-05-08T18:06:51.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.582+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.582+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721df6e00 2024-05-08T18:06:51.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.582+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.605 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.582+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721df6e00 2024-05-08T18:06:51.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.582+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v29'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.582+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721dfb450 2024-05-08T18:06:51.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.583+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.583+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721dffdb0 2024-05-08T18:06:51.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.583+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.583+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x7f3900037040 2024-05-08T18:06:51.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.583+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.583+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721df6e00 2024-05-08T18:06:51.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.583+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.583+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721dfb450 2024-05-08T18:06:51.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.584+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.584+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721dffdb0 2024-05-08T18:06:51.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.584+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.584+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721dfb450 2024-05-08T18:06:51.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.584+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.584+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x7f3900037040 2024-05-08T18:06:51.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.585+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.585+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721dffdb0 2024-05-08T18:06:51.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.585+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.585+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721dfb450 2024-05-08T18:06:51.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.585+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.585+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x55d721df6e00 2024-05-08T18:06:51.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.585+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v29'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.585+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfa6f0 con 0x7f3900037040 2024-05-08T18:06:51.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:51.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:51.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:51.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:51.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:51.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:51.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfe6c0 con 0x55d721df6e00 2024-05-08T18:06:51.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfe6c0 con 0x55d721df6e00 2024-05-08T18:06:51.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:51.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfe6c0 con 0x55d721dffdb0 2024-05-08T18:06:51.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:51.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 20 zone default found 2024-05-08T18:06:51.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 4 Realm: () 2024-05-08T18:06:51.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:51.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.586+0000 7f390f12d800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:51.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.587+0000 7f390f12d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:51.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.587+0000 7f390f12d800 20 zonegroup default 2024-05-08T18:06:51.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.587+0000 7f390f12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.587+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721dff090 con 0x55d721dfb450 2024-05-08T18:06:51.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.587+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.587+0000 7f390f12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.587+0000 7f390f12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.587+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721dfec30 con 0x55d721dffdb0 2024-05-08T18:06:51.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.587+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.587+0000 7f390f12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.587+0000 7f390f12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.587+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721e04cf0 con 0x55d721dffdb0 2024-05-08T18:06:51.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.587+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.587+0000 7f390f12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.587+0000 7f390f12d800 20 started sync module instance, tier type = 2024-05-08T18:06:51.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.587+0000 7f390f12d800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:51.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.588+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e040e0 con 0x7f3900037040 2024-05-08T18:06:51.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.589+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.589+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94382474675712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e040e0 con 0x7f3900037040 2024-05-08T18:06:51.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.589+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e05790 con 0x55d721df6e00 2024-05-08T18:06:51.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.590+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94382474675712] v29'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.590+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.590+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94382474673200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e05750 con 0x55d721df6e00 2024-05-08T18:06:51.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.590+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e06250 con 0x55d721dffdb0 2024-05-08T18:06:51.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.591+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94382474673200] v29'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.591+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.591+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94382474678944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e056c0 con 0x55d721dffdb0 2024-05-08T18:06:51.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.591+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e07110 con 0x7f3900037040 2024-05-08T18:06:51.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.592+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.592+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94382474686016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e07870 con 0x7f3900037040 2024-05-08T18:06:51.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.592+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e08390 con 0x55d721df6e00 2024-05-08T18:06:51.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.593+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94382474678944] v29'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.593+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.593+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94382474690144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e08350 con 0x55d721df6e00 2024-05-08T18:06:51.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.593+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e08da0 con 0x55d721dfb450 2024-05-08T18:06:51.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.593+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94382474686016] v29'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.594+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94382474690144] v29'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.594+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.594+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94382474683968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e072c0 con 0x55d721dfb450 2024-05-08T18:06:51.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.594+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e09640 con 0x55d721dfb450 2024-05-08T18:06:51.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.595+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(56 notify.6 [create] v29'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.595+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94382474697184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e0a2b0 con 0x55d721dfb450 2024-05-08T18:06:51.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.595+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e0ad00 con 0x55d721dfb450 2024-05-08T18:06:51.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.595+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94382474683968] v29'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.596+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94382474697184] v29'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.596+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.596+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94382474698240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e08930 con 0x55d721dfb450 2024-05-08T18:06:51.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.596+0000 7f390f12d800 20 add_watcher() i=0 2024-05-08T18:06:51.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.596+0000 7f390f12d800 20 add_watcher() i=1 2024-05-08T18:06:51.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.596+0000 7f390f12d800 20 add_watcher() i=2 2024-05-08T18:06:51.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.596+0000 7f390f12d800 20 add_watcher() i=3 2024-05-08T18:06:51.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.596+0000 7f390f12d800 20 add_watcher() i=4 2024-05-08T18:06:51.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.596+0000 7f390f12d800 20 add_watcher() i=5 2024-05-08T18:06:51.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.596+0000 7f390f12d800 20 add_watcher() i=6 2024-05-08T18:06:51.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.597+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94382474698240] v29'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.597+0000 7f390f12d800 20 add_watcher() i=7 2024-05-08T18:06:51.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.597+0000 7f390f12d800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:51.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.597+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721e0dfd0 con 0x55d721df6e00 2024-05-08T18:06:51.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.598+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.598+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94382474700784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e0b4f0 con 0x55d721df6e00 2024-05-08T18:06:51.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.599+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94382474700784] v29'245 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.599+0000 7f37c17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:51.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.599+0000 7f390f12d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:51.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.599+0000 7f390f12d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:51.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.599+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e16920 con 0x55d721df6e00 2024-05-08T18:06:51.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.600+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v29'36 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.600+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e16ff0 con 0x55d721dfb450 2024-05-08T18:06:51.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.601+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(63 gc.1 [create,call,call,call] v29'30 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.601+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e16ff0 con 0x55d721dffdb0 2024-05-08T18:06:51.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.602+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v29'29 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.602+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e17a70 con 0x55d721df6e00 2024-05-08T18:06:51.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.603+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v29'246 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.603+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e14700 con 0x7f3900037040 2024-05-08T18:06:51.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.604+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v29'77 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.604+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e14700 con 0x55d721dffdb0 2024-05-08T18:06:51.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.605+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v29'51 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.605+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e187a0 con 0x55d721df6e00 2024-05-08T18:06:51.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.606+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v29'32 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.606+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e187a0 con 0x55d721dfb450 2024-05-08T18:06:51.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.607+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(69 gc.7 [create,call,call,call] v29'51 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.607+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18830 con 0x7f3900037040 2024-05-08T18:06:51.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.608+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v29'29 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.608+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18830 con 0x55d721dffdb0 2024-05-08T18:06:51.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.609+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v29'35 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.609+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18ce0 con 0x7f3900037040 2024-05-08T18:06:51.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.610+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v29'78 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.610+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18ce0 con 0x55d721dfb450 2024-05-08T18:06:51.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.612+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(73 gc.11 [create,call,call,call] v29'35 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.612+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18ce0 con 0x7f3900037040 2024-05-08T18:06:51.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.612+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v29'30 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.613+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18ce0 con 0x55d721dfb450 2024-05-08T18:06:51.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.614+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(75 gc.13 [create,call,call,call] v29'57 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.614+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18ce0 con 0x55d721df6e00 2024-05-08T18:06:51.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.615+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v29'247 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.615+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18ce0 con 0x55d721dffdb0 2024-05-08T18:06:51.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.616+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v29'72 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.616+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18ce0 con 0x7f3900037040 2024-05-08T18:06:51.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.617+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v29'79 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.617+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18ce0 con 0x55d721dffdb0 2024-05-08T18:06:51.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.618+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v29'29 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.618+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18d70 con 0x55d721df6e00 2024-05-08T18:06:51.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.619+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v29'37 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.619+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18d70 con 0x55d721df6e00 2024-05-08T18:06:51.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.620+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v29'248 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.620+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18d70 con 0x55d721dffdb0 2024-05-08T18:06:51.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.622+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v29'73 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.622+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18e00 con 0x7f3900037040 2024-05-08T18:06:51.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.622+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v29'39 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.623+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18e00 con 0x55d721dffdb0 2024-05-08T18:06:51.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.623+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v29'31 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.623+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18e00 con 0x55d721dffdb0 2024-05-08T18:06:51.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.624+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v29'35 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.625+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18e60 con 0x55d721dffdb0 2024-05-08T18:06:51.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.625+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v29'27 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.625+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18e20 con 0x55d721dffdb0 2024-05-08T18:06:51.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.626+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v29'52 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.626+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18e20 con 0x55d721dfb450 2024-05-08T18:06:51.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.628+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(88 gc.26 [create,call,call,call] v29'31 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.628+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18e20 con 0x55d721dffdb0 2024-05-08T18:06:51.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.628+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v29'30 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.628+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18e20 con 0x55d721dfb450 2024-05-08T18:06:51.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.630+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(90 gc.28 [create,call,call,call] v29'52 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.630+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18e20 con 0x55d721dffdb0 2024-05-08T18:06:51.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.631+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v29'74 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.631+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18e20 con 0x55d721dfb450 2024-05-08T18:06:51.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.632+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(92 gc.30 [create,call,call,call] v29'58 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.632+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e18e20 con 0x7f3900037040 2024-05-08T18:06:51.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.633+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v29'35 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.633+0000 7f373bfff640 20 reqs_thread_entry: start 2024-05-08T18:06:51.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.633+0000 7f390f12d800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:51.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.633+0000 7f390f12d800 20 Filter name: none 2024-05-08T18:06:51.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.633+0000 7f3739ffb640 20 reqs_thread_entry: start 2024-05-08T18:06:51.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.633+0000 7f390f12d800 10 cache get: name=default.rgw.meta+users.uid+system_user : miss 2024-05-08T18:06:51.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.633+0000 7f390f12d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.633+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:94 8.1e 8:78ce4db0:users.uid::system_user:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721e43de0 con 0x55d721dfb450 2024-05-08T18:06:51.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.634+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(94 system_user [call out=48b,read 0~349 out=349b,getxattrs out=75b] v0'0 uv9 ondisk = 0) ==== 239+0+472 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.634+0000 7f390f12d800 20 rados_obj.operate() r=0 bl.length=349 2024-05-08T18:06:51.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.634+0000 7f390f12d800 10 cache put: name=default.rgw.meta+users.uid+system_user info.flags=0x13 2024-05-08T18:06:51.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.634+0000 7f390f12d800 10 adding default.rgw.meta+users.uid+system_user to cache LRU end 2024-05-08T18:06:51.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.634+0000 7f390f12d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:51.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.634+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:95 8.1e 8:79f34a7d:users.uid::system_user.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721e43d90 con 0x55d721dfb450 2024-05-08T18:06:51.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.634+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(95 system_user.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 163+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.634+0000 7f390f12d800 10 removing key index: system_akey 2024-05-08T18:06:51.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.634+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:96 8.0 8:01223cbf:users.keys::system_akey:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e43d90 con 0x55d721dffdb0 2024-05-08T18:06:51.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.635+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(96 system_akey [delete] v29'4 uv4 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.635+0000 7f390f12d800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:users.keys:system_akey, ofs0, ns] 2024-05-08T18:06:51.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.635+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [notify cookie 94382474941360 in=173b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721e46d70 con 0x55d721df6e00 2024-05-08T18:06:51.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.636+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== watch-notify(notify (1) cookie 94382474673200 notify 124554051619 ret 0) ==== 203+0+0 (crc 0 0 0) 0x55d721e14680 con 0x55d721df6e00 2024-05-08T18:06:51.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.636+0000 7f37ee7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051619 cookie 94382474673200 notifier 4844 bl.length()=161 2024-05-08T18:06:51.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.636+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(97 notify.1 [notify cookie 94382474941360 out=8b] v0'0 uv249 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.636+0000 7f37ee7fc640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f37d0003d60 con 0x55d721df6e00 2024-05-08T18:06:51.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.636+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(98 notify.1 [notify-ack] v0'0 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.636+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== watch-notify(notify_complete (2) cookie 94382474941360 notify 124554051619 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d721e14080 con 0x55d721df6e00 2024-05-08T18:06:51.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.636+0000 7f390f12d800 10 removing email index: 2024-05-08T18:06:51.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.636+0000 7f390f12d800 10 removing user buckets index 2024-05-08T18:06:51.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.636+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:99 8.1e 8:79f34a7d:users.uid::system_user.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e44850 con 0x55d721dfb450 2024-05-08T18:06:51.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.637+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(99 system_user.buckets [delete] v29'10 uv9 ondisk = -2 ((2) No such file or directory)) ==== 163+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.637+0000 7f390f12d800 10 removing user index: system_user 2024-05-08T18:06:51.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.637+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:100 8.1e 8:78ce4db0:users.uid::system_user:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e46370 con 0x55d721dfb450 2024-05-08T18:06:51.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.638+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(100 system_user [call,call,delete] v29'11 uv11 ondisk = 0) ==== 239+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.638+0000 7f390f12d800 10 removing default.rgw.meta+users.uid+system_user from cache 2024-05-08T18:06:51.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.638+0000 7f390f12d800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.uid:system_user, ofs0, ns] 2024-05-08T18:06:51.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.638+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:101 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94382474937168 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55d721e45c70 con 0x7f3900037040 2024-05-08T18:06:51.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.639+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== watch-notify(notify (1) cookie 94382474675712 notify 124554051608 ret 0) ==== 202+0+0 (crc 0 0 0) 0x55d721e14080 con 0x7f3900037040 2024-05-08T18:06:51.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.639+0000 7f37edffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051608 cookie 94382474675712 notifier 4844 bl.length()=160 2024-05-08T18:06:51.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.639+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(101 notify.0 [notify cookie 94382474937168 out=8b] v0'0 uv249 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.639+0000 7f37edffb640 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f37400032c0 con 0x7f3900037040 2024-05-08T18:06:51.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.639+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 20 ==== watch-notify(notify_complete (2) cookie 94382474937168 notify 124554051608 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d721e16440 con 0x7f3900037040 2024-05-08T18:06:51.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.639+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 21 ==== osd_op_reply(102 notify.0 [notify-ack] v0'0 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.640+0000 7f37c17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:51.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.640+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:103 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94382474700784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e13410 con 0x55d721df6e00 2024-05-08T18:06:51.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.641+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 32 ==== osd_op_reply(103 data_loggenerations_metadata [watch unwatch cookie 94382474700784] v29'249 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.641+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94382474675712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e0a7f0 con 0x7f3900037040 2024-05-08T18:06:51.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.642+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 22 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 94382474675712] v29'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.642+0000 7f390f12d800 20 remove_watcher() i=0 2024-05-08T18:06:51.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.642+0000 7f390f12d800 2 removed watcher, disabling cache 2024-05-08T18:06:51.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.642+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94382474673200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e04600 con 0x55d721df6e00 2024-05-08T18:06:51.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.643+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 33 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 94382474673200] v29'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.643+0000 7f390f12d800 20 remove_watcher() i=1 2024-05-08T18:06:51.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.643+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94382474678944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e04600 con 0x55d721dffdb0 2024-05-08T18:06:51.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.644+0000 7f390a570640 1 -- 172.21.15.50:0/2172543379 <== osd.3 v2:172.21.15.50:6800/3130492744 28 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 94382474678944] v29'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3900009050 con 0x55d721dffdb0 2024-05-08T18:06:51.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.644+0000 7f390f12d800 20 remove_watcher() i=2 2024-05-08T18:06:51.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.644+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94382474686016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e052a0 con 0x7f3900037040 2024-05-08T18:06:51.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.645+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.2 v2:172.21.15.50:6811/3039646617 23 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 94382474686016] v29'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3904026040 con 0x7f3900037040 2024-05-08T18:06:51.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.645+0000 7f390f12d800 20 remove_watcher() i=3 2024-05-08T18:06:51.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.645+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94382474690144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e04600 con 0x55d721df6e00 2024-05-08T18:06:51.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.646+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 <== osd.1 v2:172.21.15.50:6808/2141923372 34 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 94382474690144] v29'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3904026040 con 0x55d721df6e00 2024-05-08T18:06:51.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.646+0000 7f390f12d800 20 remove_watcher() i=4 2024-05-08T18:06:51.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.646+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94382474683968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e06e40 con 0x55d721dfb450 2024-05-08T18:06:51.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.647+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 94382474683968] v29'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.647+0000 7f390f12d800 20 remove_watcher() i=5 2024-05-08T18:06:51.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.647+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94382474697184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e052a0 con 0x55d721dfb450 2024-05-08T18:06:51.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.648+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 94382474697184] v29'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.648+0000 7f390f12d800 20 remove_watcher() i=6 2024-05-08T18:06:51.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.648+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94382474698240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55d721e04600 con 0x55d721dfb450 2024-05-08T18:06:51.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.649+0000 7f390ad71640 1 -- 172.21.15.50:0/2172543379 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 94382474698240] v29'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f38fc0713f0 con 0x55d721dfb450 2024-05-08T18:06:51.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.649+0000 7f390f12d800 20 remove_watcher() i=7 2024-05-08T18:06:51.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d721dfb450 msgr2=0x55d721dfd840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d721dfb450 0x55d721dfd840 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d721df6e00 msgr2=0x55d721df92c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d721df6e00 0x55d721df92c0 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3900037040 msgr2=0x7f3900036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f3900037040 0x7f3900036410 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d721dffdb0 msgr2=0x55d721e02270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d721dffdb0 0x55d721e02270 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f37d403bb50 msgr2=0x7f37d403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f37d403bb50 0x7f37d403e010 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f390000c010 tx=0x7f3900010000 comp rx=0 tx=0).stop 2024-05-08T18:06:51.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721de43d0 msgr2=0x55d721deb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390b572640 1 -- 172.21.15.50:0/2172543379 reap_dead start 2024-05-08T18:06:51.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721de43d0 0x55d721deb7b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f38fc06af40 tx=0x7f38fc06aed0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 shutdown_connections 2024-05-08T18:06:51.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 --2- 172.21.15.50:0/2172543379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721de43d0 0x55d721deb7b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 >> 172.21.15.50:0/2172543379 conn(0x55d721de06e0 msgr2=0x55d721dddd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 shutdown_connections 2024-05-08T18:06:51.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 -- 172.21.15.50:0/2172543379 wait complete. 2024-05-08T18:06:51.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 -- 172.21.15.50:0/3516173055 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f37e403bb50 msgr2=0x7f37e403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 --2- 172.21.15.50:0/3516173055 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f37e403bb50 0x7f37e403e010 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f390000bf40 tx=0x7f390000f000 comp rx=0 tx=0).stop 2024-05-08T18:06:51.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 -- 172.21.15.50:0/3516173055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721dd7260 msgr2=0x55d721dd7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 --2- 172.21.15.50:0/3516173055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721dd7260 0x55d721dd7680 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f38fc093260 tx=0x7f38fc071c00 comp rx=0 tx=0).stop 2024-05-08T18:06:51.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 -- 172.21.15.50:0/3516173055 shutdown_connections 2024-05-08T18:06:51.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 --2- 172.21.15.50:0/3516173055 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f37e403bb50 0x7f37e403e010 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 --2- 172.21.15.50:0/3516173055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721dd7260 0x55d721dd7680 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 -- 172.21.15.50:0/3516173055 >> 172.21.15.50:0/3516173055 conn(0x55d721dcb300 msgr2=0x55d721dcb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 -- 172.21.15.50:0/3516173055 shutdown_connections 2024-05-08T18:06:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.650+0000 7f390f12d800 1 -- 172.21.15.50:0/3516173055 wait complete. 2024-05-08T18:06:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.651+0000 7f390f12d800 1 -- 172.21.15.50:0/3000254693 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d721dc24b0 msgr2=0x55d721dc48a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.651+0000 7f390f12d800 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55d721dc24b0 0x55d721dc48a0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.651+0000 7f390f12d800 1 -- 172.21.15.50:0/3000254693 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d721cbc910 msgr2=0x55d721dbfbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.651+0000 7f390f12d800 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55d721cbc910 0x55d721dbfbb0 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.651+0000 7f390f12d800 1 -- 172.21.15.50:0/3000254693 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d721dc63d0 msgr2=0x55d721dc87c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.651+0000 7f390f12d800 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55d721dc63d0 0x55d721dc87c0 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.651+0000 7f390f12d800 1 -- 172.21.15.50:0/3000254693 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f37f003bb50 msgr2=0x7f37f003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.651+0000 7f390f12d800 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f37f003bb50 0x7f37f003e010 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3900001d70 tx=0x7f39000012d0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.651+0000 7f390f12d800 1 -- 172.21.15.50:0/3000254693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 msgr2=0x55d721db8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.651+0000 7f390f12d800 1 --2- 172.21.15.50:0/3000254693 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d721cbbfb0 0x55d721db8970 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f38fc044920 tx=0x7f38fc003330 comp rx=0 tx=0).stop 2024-05-08T18:06:51.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.651+0000 7f390b572640 1 -- 172.21.15.50:0/3000254693 reap_dead start 2024-05-08T18:06:51.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.651+0000 7f390f12d800 1 -- 172.21.15.50:0/3000254693 shutdown_connections 2024-05-08T18:06:51.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.651+0000 7f390f12d800 1 -- 172.21.15.50:0/3000254693 >> 172.21.15.50:0/3000254693 conn(0x55d721cb7ad0 msgr2=0x55d721dab040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.651+0000 7f390f12d800 1 -- 172.21.15.50:0/3000254693 shutdown_connections 2024-05-08T18:06:51.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.651+0000 7f390f12d800 1 -- 172.21.15.50:0/3000254693 wait complete. 2024-05-08T18:06:51.659 ERROR:root:None 2024-05-08T18:06:51.659 ERROR:root:0 2024-05-08T18:06:51.659 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'a_user', '--display-name', 'True Admin User', '--access-key', 'true_admin_akey', '--secret', 'true_admin_skey', '--caps', 'user-info-without-keys=read', '--admin'] 2024-05-08T18:06:51.659 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'a_user', '--display-name', 'True Admin User', '--access-key', 'true_admin_akey', '--secret', 'true_admin_skey', '--caps', 'user-info-without-keys=read', '--admin'] 2024-05-08T18:06:51.659 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid a_user --display-name 'True Admin User' --access-key true_admin_akey --secret true_admin_skey --caps user-info-without-keys=read --admin 2024-05-08T18:06:51.713 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:51.714 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:51.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.716+0000 7fe486533800 1 Processor -- start 2024-05-08T18:06:51.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.716+0000 7fe486533800 1 -- start start 2024-05-08T18:06:51.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.717+0000 7fe486533800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55bf2caf1ac0 0x55bf2caf3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.717+0000 7fe486533800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55bf2caf4480 con 0x55bf2caed720 2024-05-08T18:06:51.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.717+0000 7fe486533800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55bf2c98de20 con 0x55bf2caf1ac0 2024-05-08T18:06:51.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.717+0000 7fe4819ed640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55bf2caf1ac0 0x55bf2caf3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.717+0000 7fe4821ee640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55bf2caed720 0x55bf2caedb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50850/0 (socket says 172.21.15.50:50850) 2024-05-08T18:06:51.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.717+0000 7fe4821ee640 1 -- 172.21.15.50:0/3268025073 learned_addr learned my addr 172.21.15.50:0/3268025073 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:51.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.717+0000 7fe4819ed640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55bf2caf1ac0 0x55bf2caf3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46344/0 (socket says 172.21.15.50:46344) 2024-05-08T18:06:51.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.717+0000 7fe4819ed640 1 -- 172.21.15.50:0/3268025073 >> v1:172.21.15.46:6789/0 conn(0x55bf2caed720 legacy=0x55bf2caedb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.717+0000 7fe4819ed640 1 -- 172.21.15.50:0/3268025073 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf2caf56b0 con 0x55bf2caf1ac0 2024-05-08T18:06:51.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.719+0000 7fe4819ed640 1 --2- 172.21.15.50:0/3268025073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caf1ac0 0x55bf2caf3eb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe47800a470 tx=0x7fe47801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f814a7d84b6ae4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.719+0000 7fe4811ec640 1 -- 172.21.15.50:0/3268025073 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe478057070 con 0x55bf2caf1ac0 2024-05-08T18:06:51.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.719+0000 7fe4811ec640 1 -- 172.21.15.50:0/3268025073 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe478052570 con 0x55bf2caf1ac0 2024-05-08T18:06:51.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.719+0000 7fe4811ec640 1 -- 172.21.15.50:0/3268025073 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe478052870 con 0x55bf2caf1ac0 2024-05-08T18:06:51.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.720+0000 7fe486533800 1 -- 172.21.15.50:0/3268025073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caf1ac0 msgr2=0x55bf2caf3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.720+0000 7fe486533800 1 --2- 172.21.15.50:0/3268025073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caf1ac0 0x55bf2caf3eb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe47800a470 tx=0x7fe47801ce90 comp rx=0 tx=0).stop 2024-05-08T18:06:51.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.720+0000 7fe486533800 1 -- 172.21.15.50:0/3268025073 shutdown_connections 2024-05-08T18:06:51.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.720+0000 7fe486533800 1 --2- 172.21.15.50:0/3268025073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caf1ac0 0x55bf2caf3eb0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.720+0000 7fe486533800 1 -- 172.21.15.50:0/3268025073 >> 172.21.15.50:0/3268025073 conn(0x55bf2cae8ce0 msgr2=0x55bf2caeb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.720+0000 7fe486533800 1 -- 172.21.15.50:0/3268025073 shutdown_connections 2024-05-08T18:06:51.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.720+0000 7fe486533800 1 -- 172.21.15.50:0/3268025073 wait complete. 2024-05-08T18:06:51.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.735+0000 7fe486533800 1 Processor -- start 2024-05-08T18:06:51.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.735+0000 7fe486533800 1 -- start start 2024-05-08T18:06:51.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.735+0000 7fe486533800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caed720 0x55bf2ca62ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.735+0000 7fe486533800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bf2ca38b20 con 0x55bf2caed720 2024-05-08T18:06:51.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.735+0000 7fe4821ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caed720 0x55bf2ca62ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.735+0000 7fe4821ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caed720 0x55bf2ca62ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46348/0 (socket says 172.21.15.50:46348) 2024-05-08T18:06:51.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.735+0000 7fe4821ee640 1 -- 172.21.15.50:0/436348701 learned_addr learned my addr 172.21.15.50:0/436348701 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.735+0000 7fe4821ee640 1 -- 172.21.15.50:0/436348701 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf2ca615b0 con 0x55bf2caed720 2024-05-08T18:06:51.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.736+0000 7fe4821ee640 1 --2- 172.21.15.50:0/436348701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caed720 0x55bf2ca62ff0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe474001e40 tx=0x7fe474001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f657974e07c4c44 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.736+0000 7fe370608640 1 -- 172.21.15.50:0/436348701 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe474012020 con 0x55bf2caed720 2024-05-08T18:06:51.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.736+0000 7fe370608640 1 -- 172.21.15.50:0/436348701 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe47400e5a0 con 0x55bf2caed720 2024-05-08T18:06:51.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.736+0000 7fe370608640 1 -- 172.21.15.50:0/436348701 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe47400e8a0 con 0x55bf2caed720 2024-05-08T18:06:51.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.737+0000 7fe486533800 1 -- 172.21.15.50:0/436348701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caed720 msgr2=0x55bf2ca62ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.737+0000 7fe486533800 1 --2- 172.21.15.50:0/436348701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caed720 0x55bf2ca62ff0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe474001e40 tx=0x7fe474001e70 comp rx=0 tx=0).stop 2024-05-08T18:06:51.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.737+0000 7fe486533800 1 -- 172.21.15.50:0/436348701 shutdown_connections 2024-05-08T18:06:51.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.737+0000 7fe486533800 1 --2- 172.21.15.50:0/436348701 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caed720 0x55bf2ca62ff0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.737+0000 7fe486533800 1 -- 172.21.15.50:0/436348701 >> 172.21.15.50:0/436348701 conn(0x55bf2cae8ce0 msgr2=0x55bf2cbda4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.737+0000 7fe486533800 1 -- 172.21.15.50:0/436348701 shutdown_connections 2024-05-08T18:06:51.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.737+0000 7fe486533800 1 -- 172.21.15.50:0/436348701 wait complete. 2024-05-08T18:06:51.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.737+0000 7fe486533800 1 Processor -- start 2024-05-08T18:06:51.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.737+0000 7fe486533800 1 -- start start 2024-05-08T18:06:51.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.737+0000 7fe486533800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caed720 0x55bf2cbe8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.737+0000 7fe486533800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bf2caf4950 con 0x55bf2caed720 2024-05-08T18:06:51.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.737+0000 7fe4821ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caed720 0x55bf2cbe8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.737+0000 7fe4821ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caed720 0x55bf2cbe8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46356/0 (socket says 172.21.15.50:46356) 2024-05-08T18:06:51.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.737+0000 7fe4821ee640 1 -- 172.21.15.50:0/2545312528 learned_addr learned my addr 172.21.15.50:0/2545312528 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.738+0000 7fe4821ee640 1 -- 172.21.15.50:0/2545312528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf2cbe8f40 con 0x55bf2caed720 2024-05-08T18:06:51.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.738+0000 7fe4821ee640 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caed720 0x55bf2cbe8a00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe474000c00 tx=0x7fe474000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.738+0000 7fe36ee05640 1 -- 172.21.15.50:0/2545312528 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe474012030 con 0x55bf2caed720 2024-05-08T18:06:51.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.738+0000 7fe486533800 1 -- 172.21.15.50:0/2545312528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bf2cbe91d0 con 0x55bf2caed720 2024-05-08T18:06:51.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.738+0000 7fe36ee05640 1 -- 172.21.15.50:0/2545312528 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4740014a0 con 0x55bf2caed720 2024-05-08T18:06:51.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.738+0000 7fe36ee05640 1 -- 172.21.15.50:0/2545312528 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe47401a7a0 con 0x55bf2caed720 2024-05-08T18:06:51.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.738+0000 7fe486533800 1 -- 172.21.15.50:0/2545312528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf2cbebe30 con 0x55bf2caed720 2024-05-08T18:06:51.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.739+0000 7fe36ee05640 1 -- 172.21.15.50:0/2545312528 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fe474015020 con 0x55bf2caed720 2024-05-08T18:06:51.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.739+0000 7fe36ee05640 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe36803bb50 0x7fe36803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.739+0000 7fe36ee05640 1 -- 172.21.15.50:0/2545312528 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fe474020030 con 0x55bf2caed720 2024-05-08T18:06:51.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.739+0000 7fe4819ed640 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe36803bb50 0x7fe36803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.739+0000 7fe486533800 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bf2caf1ac0 0x55bf2cbefbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.739+0000 7fe486533800 1 -- 172.21.15.50:0/2545312528 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cbf0170 con 0x55bf2caf1ac0 2024-05-08T18:06:51.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.739+0000 7fe4829ef640 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bf2caf1ac0 0x55bf2cbefbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.740+0000 7fe4819ed640 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe36803bb50 0x7fe36803e010 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe478054000 tx=0x7fe47804e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.740+0000 7fe4829ef640 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bf2caf1ac0 0x55bf2cbefbf0 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.740+0000 7fe4829ef640 1 -- 172.21.15.50:0/2545312528 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe47c00a4a0 con 0x55bf2caf1ac0 2024-05-08T18:06:51.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.740+0000 7fe486533800 1 -- 172.21.15.50:0/2545312528 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cbed1c0 con 0x55bf2caf1ac0 2024-05-08T18:06:51.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.740+0000 7fe4829ef640 1 -- 172.21.15.50:0/2545312528 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe47c00a4a0 con 0x55bf2caf1ac0 2024-05-08T18:06:51.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.740+0000 7fe486533800 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bf2cbf24a0 0x55bf2cbf4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.740+0000 7fe486533800 1 -- 172.21.15.50:0/2545312528 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cbf4f30 con 0x55bf2cbf24a0 2024-05-08T18:06:51.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.741+0000 7fe4821ee640 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bf2cbf24a0 0x55bf2cbf4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.741+0000 7fe4821ee640 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bf2cbf24a0 0x55bf2cbf4890 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.741+0000 7fe4821ee640 1 -- 172.21.15.50:0/2545312528 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe47401ab20 con 0x55bf2cbf24a0 2024-05-08T18:06:51.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.741+0000 7fe486533800 1 -- 172.21.15.50:0/2545312528 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cbf1340 con 0x55bf2caf1ac0 2024-05-08T18:06:51.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.742+0000 7fe4829ef640 1 -- 172.21.15.50:0/2545312528 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe47c00a4a0 con 0x55bf2caf1ac0 2024-05-08T18:06:51.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.742+0000 7fe486533800 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bf2cbf6390 0x55bf2cbf8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.742+0000 7fe486533800 1 -- 172.21.15.50:0/2545312528 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cbf8e80 con 0x55bf2cbf6390 2024-05-08T18:06:51.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.742+0000 7fe4819ed640 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bf2cbf6390 0x55bf2cbf8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.742+0000 7fe4819ed640 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bf2cbf6390 0x55bf2cbf8780 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.742+0000 7fe4819ed640 1 -- 172.21.15.50:0/2545312528 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fe478052a40 con 0x55bf2cbf6390 2024-05-08T18:06:51.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.743+0000 7fe486533800 1 Processor -- start 2024-05-08T18:06:51.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.743+0000 7fe486533800 1 -- start start 2024-05-08T18:06:51.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.743+0000 7fe486533800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc071b0 0x55bf2cc075d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.743+0000 7fe486533800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bf2c9df330 con 0x55bf2cc071b0 2024-05-08T18:06:51.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.743+0000 7fe4821ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc071b0 0x55bf2cc075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.743+0000 7fe4821ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc071b0 0x55bf2cc075d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46370/0 (socket says 172.21.15.50:46370) 2024-05-08T18:06:51.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.743+0000 7fe4821ee640 1 -- 172.21.15.50:0/537818620 learned_addr learned my addr 172.21.15.50:0/537818620 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.743+0000 7fe4821ee640 1 -- 172.21.15.50:0/537818620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf2cc07b10 con 0x55bf2cc071b0 2024-05-08T18:06:51.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.744+0000 7fe4821ee640 1 --2- 172.21.15.50:0/537818620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc071b0 0x55bf2cc075d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe47402dfb0 tx=0x7fe474018b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.744+0000 7fe367fff640 1 -- 172.21.15.50:0/537818620 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe474047020 con 0x55bf2cc071b0 2024-05-08T18:06:51.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.744+0000 7fe367fff640 1 -- 172.21.15.50:0/537818620 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4740167e0 con 0x55bf2cc071b0 2024-05-08T18:06:51.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.744+0000 7fe486533800 1 -- 172.21.15.50:0/537818620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bf2cc08580 con 0x55bf2cc071b0 2024-05-08T18:06:51.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.744+0000 7fe367fff640 1 -- 172.21.15.50:0/537818620 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe474016ae0 con 0x55bf2cc071b0 2024-05-08T18:06:51.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.744+0000 7fe486533800 1 -- 172.21.15.50:0/537818620 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf2cc08580 con 0x55bf2cc071b0 2024-05-08T18:06:51.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.744+0000 7fe367fff640 1 -- 172.21.15.50:0/537818620 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fe474016c80 con 0x55bf2cc071b0 2024-05-08T18:06:51.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.745+0000 7fe367fff640 1 --2- 172.21.15.50:0/537818620 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe35c03bba0 0x7fe35c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.745+0000 7fe367fff640 1 -- 172.21.15.50:0/537818620 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fe474038d40 con 0x55bf2cc071b0 2024-05-08T18:06:51.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.745+0000 7fe4819ed640 1 --2- 172.21.15.50:0/537818620 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe35c03bba0 0x7fe35c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.745+0000 7fe486533800 1 Processor -- start 2024-05-08T18:06:51.751 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.745+0000 7fe486533800 1 -- start start 2024-05-08T18:06:51.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.745+0000 7fe4819ed640 1 --2- 172.21.15.50:0/537818620 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe35c03bba0 0x7fe35c03e060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55bf2cbed0e0 tx=0x7fe478005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.745+0000 7fe486533800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc141b0 0x55bf2cc145d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.745+0000 7fe486533800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bf2ca61930 con 0x55bf2cc141b0 2024-05-08T18:06:51.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.745+0000 7fe4821ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc141b0 0x55bf2cc145d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.746+0000 7fe4821ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc141b0 0x55bf2cc145d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46372/0 (socket says 172.21.15.50:46372) 2024-05-08T18:06:51.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.746+0000 7fe4821ee640 1 -- 172.21.15.50:0/4063002171 learned_addr learned my addr 172.21.15.50:0/4063002171 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.746+0000 7fe4821ee640 1 -- 172.21.15.50:0/4063002171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf2cc14ba0 con 0x55bf2cc141b0 2024-05-08T18:06:51.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.746+0000 7fe4821ee640 1 --2- 172.21.15.50:0/4063002171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc141b0 0x55bf2cc145d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe47400d6e0 tx=0x7fe474049690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1222310ebe4ad1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.746+0000 7fe365ffb640 1 -- 172.21.15.50:0/4063002171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe47403dc60 con 0x55bf2cc141b0 2024-05-08T18:06:51.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.746+0000 7fe365ffb640 1 -- 172.21.15.50:0/4063002171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe474024420 con 0x55bf2cc141b0 2024-05-08T18:06:51.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.746+0000 7fe365ffb640 1 -- 172.21.15.50:0/4063002171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe47403de00 con 0x55bf2cc141b0 2024-05-08T18:06:51.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.746+0000 7fe486533800 1 -- 172.21.15.50:0/4063002171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc141b0 msgr2=0x55bf2cc145d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.746+0000 7fe486533800 1 --2- 172.21.15.50:0/4063002171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc141b0 0x55bf2cc145d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fe47400d6e0 tx=0x7fe474049690 comp rx=0 tx=0).stop 2024-05-08T18:06:51.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.746+0000 7fe486533800 1 -- 172.21.15.50:0/4063002171 shutdown_connections 2024-05-08T18:06:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.746+0000 7fe486533800 1 --2- 172.21.15.50:0/4063002171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc141b0 0x55bf2cc145d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.746+0000 7fe486533800 1 -- 172.21.15.50:0/4063002171 >> 172.21.15.50:0/4063002171 conn(0x55bf2cc10480 msgr2=0x55bf2cc0e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.747+0000 7fe486533800 1 -- 172.21.15.50:0/4063002171 shutdown_connections 2024-05-08T18:06:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.747+0000 7fe486533800 1 -- 172.21.15.50:0/4063002171 wait complete. 2024-05-08T18:06:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.747+0000 7fe486533800 1 Processor -- start 2024-05-08T18:06:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.747+0000 7fe486533800 1 -- start start 2024-05-08T18:06:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.747+0000 7fe486533800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc141b0 0x55bf2cc1b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.747+0000 7fe486533800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55bf2cc15270 con 0x55bf2cc141b0 2024-05-08T18:06:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.747+0000 7fe4821ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc141b0 0x55bf2cc1b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.747+0000 7fe4821ee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc141b0 0x55bf2cc1b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46382/0 (socket says 172.21.15.50:46382) 2024-05-08T18:06:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.747+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 learned_addr learned my addr 172.21.15.50:0/447400812 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.747+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf2cc1bb70 con 0x55bf2cc141b0 2024-05-08T18:06:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.748+0000 7fe4821ee640 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc141b0 0x55bf2cc1b630 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe47402d5a0 tx=0x7fe474004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.748+0000 7fe353fff640 1 -- 172.21.15.50:0/447400812 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe47403d2b0 con 0x55bf2cc141b0 2024-05-08T18:06:51.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.748+0000 7fe353fff640 1 -- 172.21.15.50:0/447400812 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe47403d450 con 0x55bf2cc141b0 2024-05-08T18:06:51.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.748+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bf2cc1bb70 con 0x55bf2cc141b0 2024-05-08T18:06:51.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.748+0000 7fe353fff640 1 -- 172.21.15.50:0/447400812 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe47402d740 con 0x55bf2cc141b0 2024-05-08T18:06:51.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.748+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf2cc1bb70 con 0x55bf2cc141b0 2024-05-08T18:06:51.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.749+0000 7fe353fff640 1 -- 172.21.15.50:0/447400812 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fe4740247b0 con 0x55bf2cc141b0 2024-05-08T18:06:51.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.749+0000 7fe353fff640 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe34c03bba0 0x7fe34c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.749+0000 7fe353fff640 1 -- 172.21.15.50:0/447400812 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fe474058030 con 0x55bf2cc141b0 2024-05-08T18:06:51.754 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.749+0000 7fe4819ed640 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe34c03bba0 0x7fe34c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.749+0000 7fe4819ed640 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe34c03bba0 0x7fe34c03e060 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe478052fd0 tx=0x7fe47804f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.750+0000 7fe486533800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.750+0000 7fe486533800 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bf2cc26db0 0x55bf2cc29270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.750+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc299c0 con 0x55bf2cc26db0 2024-05-08T18:06:51.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.750+0000 7fe4829ef640 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bf2cc26db0 0x55bf2cc29270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.750+0000 7fe4829ef640 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bf2cc26db0 0x55bf2cc29270 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.750+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe47c00a4a0 con 0x55bf2cc26db0 2024-05-08T18:06:51.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.750+0000 7fe486533800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.750+0000 7fe486533800 20 realm 2024-05-08T18:06:51.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.750+0000 7fe486533800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.750+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc29dd0 con 0x55bf2cc26db0 2024-05-08T18:06:51.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.751+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe47c00a4a0 con 0x55bf2cc26db0 2024-05-08T18:06:51.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.751+0000 7fe486533800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.773 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.751+0000 7fe486533800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:51.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.751+0000 7fe486533800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.751+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2a6a0 con 0x55bf2cc26db0 2024-05-08T18:06:51.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.751+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe47c00a4a0 con 0x55bf2cc26db0 2024-05-08T18:06:51.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.751+0000 7fe486533800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.751+0000 7fe486533800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.751+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2aa40 con 0x55bf2cc26db0 2024-05-08T18:06:51.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.751+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe47c00a4a0 con 0x55bf2cc26db0 2024-05-08T18:06:51.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.751+0000 7fe486533800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:51.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.751+0000 7fe486533800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.751+0000 7fe486533800 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bf2cc2b7a0 0x55bf2cc2db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.751+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2e1a0 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.752+0000 7fe4821ee640 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bf2cc2b7a0 0x55bf2cc2db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.752+0000 7fe4821ee640 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bf2cc2b7a0 0x55bf2cc2db90 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.752+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.752+0000 7fe486533800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:51.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.752+0000 7fe486533800 20 searching for the correct realm 2024-05-08T18:06:51.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.752+0000 7fe486533800 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bf2cc30150 0x55bf2cc32610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.752+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc32ce0 con 0x55bf2cc30150 2024-05-08T18:06:51.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.752+0000 7fe4819ed640 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bf2cc30150 0x55bf2cc32610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.753+0000 7fe4819ed640 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bf2cc30150 0x55bf2cc32610 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.753+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.753+0000 7fe4819ed640 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fe478076040 0x7fe478083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.753+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc32ce0 con 0x7fe478076040 2024-05-08T18:06:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.753+0000 7fe4829ef640 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fe478076040 0x7fe478083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.753+0000 7fe4829ef640 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fe478076040 0x7fe478083410 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.754+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47c00a4a0 con 0x7fe478076040 2024-05-08T18:06:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.754+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2aa40 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.754+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.754+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2aa40 con 0x55bf2cc26db0 2024-05-08T18:06:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.754+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47c00a4a0 con 0x55bf2cc26db0 2024-05-08T18:06:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.754+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2aa40 con 0x55bf2cc26db0 2024-05-08T18:06:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.754+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47c00a4a0 con 0x55bf2cc26db0 2024-05-08T18:06:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.754+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2aa40 con 0x55bf2cc26db0 2024-05-08T18:06:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.755+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47c00a4a0 con 0x55bf2cc26db0 2024-05-08T18:06:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.755+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2aa40 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.755+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v29'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.755+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2aa40 con 0x7fe478076040 2024-05-08T18:06:51.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.755+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47c00a4a0 con 0x7fe478076040 2024-05-08T18:06:51.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.755+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2aa40 con 0x55bf2cc30150 2024-05-08T18:06:51.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.755+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.755+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2aa40 con 0x55bf2cc26db0 2024-05-08T18:06:51.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.756+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47c00a4a0 con 0x55bf2cc26db0 2024-05-08T18:06:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.756+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2aa40 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.756+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.756+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2aa40 con 0x55bf2cc26db0 2024-05-08T18:06:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.756+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47c00a4a0 con 0x55bf2cc26db0 2024-05-08T18:06:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.756+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2aa40 con 0x7fe478076040 2024-05-08T18:06:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.756+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe47c032040 con 0x7fe478076040 2024-05-08T18:06:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.756+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2aa40 con 0x55bf2cc30150 2024-05-08T18:06:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.757+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.757+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x55bf2cc30150 2024-05-08T18:06:51.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.757+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.757+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.757+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.757+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x55bf2cc30150 2024-05-08T18:06:51.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.757+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.758+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x55bf2cc30150 2024-05-08T18:06:51.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.758+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.758+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x55bf2cc26db0 2024-05-08T18:06:51.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.758+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47c032040 con 0x55bf2cc26db0 2024-05-08T18:06:51.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.758+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x55bf2cc26db0 2024-05-08T18:06:51.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.758+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v29'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe47c032040 con 0x55bf2cc26db0 2024-05-08T18:06:51.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.758+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.759+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.759+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x55bf2cc30150 2024-05-08T18:06:51.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.759+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.759+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x7fe478076040 2024-05-08T18:06:51.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.759+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47c032040 con 0x7fe478076040 2024-05-08T18:06:51.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.759+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x55bf2cc26db0 2024-05-08T18:06:51.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.759+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47c032040 con 0x55bf2cc26db0 2024-05-08T18:06:51.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.759+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.760+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.760+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x55bf2cc30150 2024-05-08T18:06:51.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.760+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.760+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.760+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.760+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x7fe478076040 2024-05-08T18:06:51.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.761+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47c032040 con 0x7fe478076040 2024-05-08T18:06:51.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.761+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x55bf2cc30150 2024-05-08T18:06:51.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.761+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.761+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.761+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.761+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x55bf2cc26db0 2024-05-08T18:06:51.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.761+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v29'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe47c032040 con 0x55bf2cc26db0 2024-05-08T18:06:51.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.761+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fe478090040 con 0x7fe478076040 2024-05-08T18:06:51.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47c032040 con 0x7fe478076040 2024-05-08T18:06:51.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe486533800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:51.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe486533800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:51.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe486533800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:51.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe486533800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:51.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe486533800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:51.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe486533800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:51.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe486533800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2ea60 con 0x55bf2cc26db0 2024-05-08T18:06:51.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe47c032040 con 0x55bf2cc26db0 2024-05-08T18:06:51.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe486533800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.784 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe486533800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2ea60 con 0x55bf2cc26db0 2024-05-08T18:06:51.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe47c032040 con 0x55bf2cc26db0 2024-05-08T18:06:51.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe486533800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:51.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe486533800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.762+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2ea60 con 0x55bf2cc30150 2024-05-08T18:06:51.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe486533800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:51.785 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe486533800 20 zone default found 2024-05-08T18:06:51.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe486533800 4 Realm: () 2024-05-08T18:06:51.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe486533800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:51.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe486533800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:51.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe486533800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:51.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe486533800 20 zonegroup default 2024-05-08T18:06:51.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe486533800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc2f400 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.786 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe486533800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe486533800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc34bd0 con 0x55bf2cc30150 2024-05-08T18:06:51.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe486533800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe486533800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc35010 con 0x55bf2cc30150 2024-05-08T18:06:51.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.763+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.787 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.764+0000 7fe486533800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.764+0000 7fe486533800 20 started sync module instance, tier type = 2024-05-08T18:06:51.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.764+0000 7fe486533800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:51.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.764+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc33fe0 con 0x7fe478076040 2024-05-08T18:06:51.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.765+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47c032040 con 0x7fe478076040 2024-05-08T18:06:51.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.765+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94279578109184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc33fe0 con 0x7fe478076040 2024-05-08T18:06:51.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.765+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc35ae0 con 0x55bf2cc26db0 2024-05-08T18:06:51.788 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.766+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47c032040 con 0x55bf2cc26db0 2024-05-08T18:06:51.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.766+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94279578109184] v29'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47c032040 con 0x7fe478076040 2024-05-08T18:06:51.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.766+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94279578113520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc35ab0 con 0x55bf2cc26db0 2024-05-08T18:06:51.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.766+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc33b70 con 0x55bf2cc30150 2024-05-08T18:06:51.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.767+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94279578113520] v29'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47c032040 con 0x55bf2cc26db0 2024-05-08T18:06:51.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.767+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.767+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94279578115792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc36a50 con 0x55bf2cc30150 2024-05-08T18:06:51.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.767+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc37410 con 0x7fe478076040 2024-05-08T18:06:51.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.768+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94279578115792] v29'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.789 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.768+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47c032040 con 0x7fe478076040 2024-05-08T18:06:51.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.768+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94279578120032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc37830 con 0x7fe478076040 2024-05-08T18:06:51.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.768+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc381f0 con 0x55bf2cc26db0 2024-05-08T18:06:51.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.769+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94279578120032] v29'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47c032040 con 0x7fe478076040 2024-05-08T18:06:51.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.769+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47c032040 con 0x55bf2cc26db0 2024-05-08T18:06:51.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.769+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94279578121568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc381f0 con 0x55bf2cc26db0 2024-05-08T18:06:51.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.769+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc38bb0 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.770+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.770+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94279578121568] v29'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47c032040 con 0x55bf2cc26db0 2024-05-08T18:06:51.790 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.770+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94279578126080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc38f50 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.770+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc39950 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.771+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94279578126080] v29'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.771+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.771+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94279578123584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc3a1a0 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.771+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc3ab60 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.772+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94279578123584] v29'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.772+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.791 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.772+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94279578134192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc3af80 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.772+0000 7fe486533800 20 add_watcher() i=0 2024-05-08T18:06:51.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.772+0000 7fe486533800 20 add_watcher() i=1 2024-05-08T18:06:51.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.772+0000 7fe486533800 20 add_watcher() i=2 2024-05-08T18:06:51.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.772+0000 7fe486533800 20 add_watcher() i=3 2024-05-08T18:06:51.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.772+0000 7fe486533800 20 add_watcher() i=4 2024-05-08T18:06:51.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.772+0000 7fe486533800 20 add_watcher() i=5 2024-05-08T18:06:51.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.772+0000 7fe486533800 20 add_watcher() i=6 2024-05-08T18:06:51.792 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.773+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94279578134192] v29'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.773+0000 7fe486533800 20 add_watcher() i=7 2024-05-08T18:06:51.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.773+0000 7fe486533800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:51.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.773+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc3e1f0 con 0x55bf2cc26db0 2024-05-08T18:06:51.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.774+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fe47c032040 con 0x55bf2cc26db0 2024-05-08T18:06:51.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.774+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94279578135616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc3b510 con 0x55bf2cc26db0 2024-05-08T18:06:51.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.775+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94279578135616] v29'250 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe47c032040 con 0x55bf2cc26db0 2024-05-08T18:06:51.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.775+0000 7fe338fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:51.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.775+0000 7fe486533800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:51.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.775+0000 7fe486533800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:51.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.775+0000 7fe486533800 5 note: GC not initialized 2024-05-08T18:06:51.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.775+0000 7fe2b37fe640 20 reqs_thread_entry: start 2024-05-08T18:06:51.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.775+0000 7fe486533800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:51.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.775+0000 7fe486533800 20 Filter name: none 2024-05-08T18:06:51.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.775+0000 7fe2b17fa640 20 reqs_thread_entry: start 2024-05-08T18:06:51.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe486533800 10 cache get: name=default.rgw.meta+users.uid+a_user : miss 2024-05-08T18:06:51.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe486533800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:62 8.14 8:2efb937b:users.uid::a_user:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc717c0 con 0x55bf2cc30150 2024-05-08T18:06:51.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(62 a_user [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 234+0+0 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe486533800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe486533800 10 cache put: name=default.rgw.meta+users.uid+a_user info.flags=0x0 2024-05-08T18:06:51.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe486533800 10 adding default.rgw.meta+users.uid+a_user to cache LRU end 2024-05-08T18:06:51.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe486533800 10 cache get: name=default.rgw.meta+users.keys+true_admin_akey : miss 2024-05-08T18:06:51.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe486533800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:63 8.15 8:afe09604:users.keys::true_admin_akey:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc6ff60 con 0x55bf2cc30150 2024-05-08T18:06:51.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(63 true_admin_akey [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 201+0+0 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe486533800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe486533800 10 cache put: name=default.rgw.meta+users.keys+true_admin_akey info.flags=0x0 2024-05-08T18:06:51.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe486533800 10 adding default.rgw.meta+users.keys+true_admin_akey to cache LRU end 2024-05-08T18:06:51.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe486533800 10 cache get: name=default.rgw.meta+users.keys+true_admin_akey : hit (negative entry) 2024-05-08T18:06:51.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe486533800 10 cache get: name=default.rgw.meta+users.keys+true_admin_akey : hit (negative entry) 2024-05-08T18:06:51.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.776+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 8.14 8:2efb937b:users.uid::a_user:head [delete,create,call version.set in=58b,writefull 0~363 in=363b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc70800 con 0x55bf2cc30150 2024-05-08T18:06:51.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.777+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(64 a_user [delete,create,call,writefull 0~363] v29'3 uv3 ondisk = 0) ==== 276+0+0 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.777+0000 7fe486533800 10 cache put: name=default.rgw.meta+users.uid+a_user info.flags=0x17 2024-05-08T18:06:51.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.777+0000 7fe486533800 10 moving default.rgw.meta+users.uid+a_user to cache LRU end 2024-05-08T18:06:51.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.777+0000 7fe486533800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:a_user, ofs0, ns] 2024-05-08T18:06:51.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.777+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [notify cookie 94279578354512 in=554b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc75260 con 0x55bf2cc30150 2024-05-08T18:06:51.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.778+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== watch-notify(notify (1) cookie 94279578115792 notify 124554051593 ret 0) ==== 584+0+0 (crc 0 0 0) 0x55bf2cc70ba0 con 0x55bf2cc30150 2024-05-08T18:06:51.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.778+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(65 notify.2 [notify cookie 94279578354512 out=8b] v0'0 uv252 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.778+0000 7fe365ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051593 cookie 94279578115792 notifier 4851 bl.length()=542 2024-05-08T18:06:51.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.778+0000 7fe365ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+a_user info.flags=0x17 2024-05-08T18:06:51.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.778+0000 7fe365ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+a_user to cache LRU end 2024-05-08T18:06:51.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.778+0000 7fe365ffb640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7fe348003ea0 con 0x55bf2cc30150 2024-05-08T18:06:51.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.778+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(66 notify.2 [notify-ack] v0'0 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.778+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== watch-notify(notify_complete (2) cookie 94279578354512 notify 124554051593 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55bf2cc71210 con 0x55bf2cc30150 2024-05-08T18:06:51.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.778+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 8.15 8:afe09604:users.keys::true_admin_akey:head [delete,create,writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc702e0 con 0x55bf2cc30150 2024-05-08T18:06:51.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.779+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(67 true_admin_akey [delete,create,writefull 0~10] v29'3 uv3 ondisk = 0) ==== 243+0+0 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.779+0000 7fe486533800 10 cache put: name=default.rgw.meta+users.keys+true_admin_akey info.flags=0x7 2024-05-08T18:06:51.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.779+0000 7fe486533800 10 moving default.rgw.meta+users.keys+true_admin_akey to cache LRU end 2024-05-08T18:06:51.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.779+0000 7fe486533800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:true_admin_akey, ofs0, ns] 2024-05-08T18:06:51.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.779+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify cookie 94279578354400 in=187b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc75260 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.780+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== watch-notify(notify (1) cookie 94279578126080 notify 124554051630 ret 0) ==== 217+0+0 (crc 0 0 0) 0x7fe474032020 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.780+0000 7fe365ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051630 cookie 94279578126080 notifier 4851 bl.length()=175 2024-05-08T18:06:51.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.780+0000 7fe365ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+true_admin_akey info.flags=0x7 2024-05-08T18:06:51.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.780+0000 7fe365ffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+true_admin_akey to cache LRU end 2024-05-08T18:06:51.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.780+0000 7fe365ffb640 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7fe348003ea0 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.780+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(68 notify.5 [notify cookie 94279578354400 out=8b] v0'0 uv252 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.780+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== watch-notify(notify_complete (2) cookie 94279578354400 notify 124554051630 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55bf2cc07b10 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.780+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(69 notify.5 [notify-ack] v0'0 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.803 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"a_user","display_name":"True Admin User","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"a_user","access_key":"true_admin_akey","secret_key":"true_admin_skey","active":true,"create_date":"2024-05-08T18:06:51.777336Z"}],"swift_keys":[],"caps":[{"type":"user-info-without-keys","perm":"read"}],"op_mask":"read, write, delete","admin":true,"default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:06:51.777323Z","tags":[],"group_ids":[]} 2024-05-08T18:06:51.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.781+0000 7fe338fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:51.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.781+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94279578135616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc3d4d0 con 0x55bf2cc26db0 2024-05-08T18:06:51.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.782+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94279578135616] v29'251 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe47c032040 con 0x55bf2cc26db0 2024-05-08T18:06:51.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.782+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94279578109184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc3ac40 con 0x7fe478076040 2024-05-08T18:06:51.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.783+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94279578109184] v29'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47c032040 con 0x7fe478076040 2024-05-08T18:06:51.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.783+0000 7fe486533800 20 remove_watcher() i=0 2024-05-08T18:06:51.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.783+0000 7fe486533800 2 removed watcher, disabling cache 2024-05-08T18:06:51.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.783+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94279578113520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc3ac40 con 0x55bf2cc26db0 2024-05-08T18:06:51.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.784+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94279578113520] v29'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47c032040 con 0x55bf2cc26db0 2024-05-08T18:06:51.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.784+0000 7fe486533800 20 remove_watcher() i=1 2024-05-08T18:06:51.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.784+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94279578115792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc3ac40 con 0x55bf2cc30150 2024-05-08T18:06:51.809 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.785+0000 7fe4819ed640 1 -- 172.21.15.50:0/447400812 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94279578115792] v29'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe478081070 con 0x55bf2cc30150 2024-05-08T18:06:51.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.785+0000 7fe486533800 20 remove_watcher() i=2 2024-05-08T18:06:51.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.785+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94279578120032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc3ac40 con 0x7fe478076040 2024-05-08T18:06:51.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.786+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94279578120032] v29'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47c032040 con 0x7fe478076040 2024-05-08T18:06:51.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.786+0000 7fe486533800 20 remove_watcher() i=3 2024-05-08T18:06:51.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.786+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94279578121568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc3ac40 con 0x55bf2cc26db0 2024-05-08T18:06:51.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.787+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94279578121568] v29'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47c032040 con 0x55bf2cc26db0 2024-05-08T18:06:51.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.787+0000 7fe486533800 20 remove_watcher() i=4 2024-05-08T18:06:51.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.787+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94279578126080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc37560 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.788+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94279578126080] v29'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.788+0000 7fe486533800 20 remove_watcher() i=5 2024-05-08T18:06:51.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.788+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94279578123584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc3ac40 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.789+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94279578123584] v29'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.789+0000 7fe486533800 20 remove_watcher() i=6 2024-05-08T18:06:51.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.789+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94279578134192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55bf2cc37560 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.790+0000 7fe4821ee640 1 -- 172.21.15.50:0/447400812 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94279578134192] v29'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe474072070 con 0x55bf2cc2b7a0 2024-05-08T18:06:51.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.790+0000 7fe486533800 20 remove_watcher() i=7 2024-05-08T18:06:51.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bf2cc2b7a0 msgr2=0x55bf2cc2db90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bf2cc2b7a0 0x55bf2cc2db90 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bf2cc26db0 msgr2=0x55bf2cc29270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bf2cc26db0 0x55bf2cc29270 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fe478076040 msgr2=0x7fe478083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fe478076040 0x7fe478083410 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bf2cc30150 msgr2=0x55bf2cc32610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bf2cc30150 0x55bf2cc32610 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe34c03bba0 msgr2=0x7fe34c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe34c03bba0 0x7fe34c03e060 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe478052fd0 tx=0x7fe47804f000 comp rx=0 tx=0).stop 2024-05-08T18:06:51.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc141b0 msgr2=0x55bf2cc1b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 --2- 172.21.15.50:0/447400812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc141b0 0x55bf2cc1b630 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe47402d5a0 tx=0x7fe474004040 comp rx=0 tx=0).stop 2024-05-08T18:06:51.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 reap_dead start 2024-05-08T18:06:51.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe4829ef640 1 -- 172.21.15.50:0/447400812 reap_dead start 2024-05-08T18:06:51.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 shutdown_connections 2024-05-08T18:06:51.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 >> 172.21.15.50:0/447400812 conn(0x55bf2cc10480 msgr2=0x55bf2cc0dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 shutdown_connections 2024-05-08T18:06:51.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 -- 172.21.15.50:0/447400812 wait complete. 2024-05-08T18:06:51.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 -- 172.21.15.50:0/537818620 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe35c03bba0 msgr2=0x7fe35c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 --2- 172.21.15.50:0/537818620 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe35c03bba0 0x7fe35c03e060 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x55bf2cbed0e0 tx=0x7fe478005a00 comp rx=0 tx=0).stop 2024-05-08T18:06:51.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 -- 172.21.15.50:0/537818620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc071b0 msgr2=0x55bf2cc075d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 --2- 172.21.15.50:0/537818620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc071b0 0x55bf2cc075d0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe47402dfb0 tx=0x7fe474018b80 comp rx=0 tx=0).stop 2024-05-08T18:06:51.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 -- 172.21.15.50:0/537818620 shutdown_connections 2024-05-08T18:06:51.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 --2- 172.21.15.50:0/537818620 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe35c03bba0 0x7fe35c03e060 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 --2- 172.21.15.50:0/537818620 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2cc071b0 0x55bf2cc075d0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 -- 172.21.15.50:0/537818620 >> 172.21.15.50:0/537818620 conn(0x55bf2cbfb250 msgr2=0x55bf2cbfb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 -- 172.21.15.50:0/537818620 shutdown_connections 2024-05-08T18:06:51.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.791+0000 7fe486533800 1 -- 172.21.15.50:0/537818620 wait complete. 2024-05-08T18:06:51.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.792+0000 7fe486533800 1 -- 172.21.15.50:0/2545312528 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bf2cbf24a0 msgr2=0x55bf2cbf4890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.792+0000 7fe486533800 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55bf2cbf24a0 0x55bf2cbf4890 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.792+0000 7fe486533800 1 -- 172.21.15.50:0/2545312528 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bf2caf1ac0 msgr2=0x55bf2cbefbf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.792+0000 7fe486533800 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55bf2caf1ac0 0x55bf2cbefbf0 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.792+0000 7fe486533800 1 -- 172.21.15.50:0/2545312528 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bf2cbf6390 msgr2=0x55bf2cbf8780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.792+0000 7fe486533800 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55bf2cbf6390 0x55bf2cbf8780 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.792+0000 7fe486533800 1 -- 172.21.15.50:0/2545312528 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe36803bb50 msgr2=0x7fe36803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.792+0000 7fe486533800 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe36803bb50 0x7fe36803e010 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe478054000 tx=0x7fe47804e000 comp rx=0 tx=0).stop 2024-05-08T18:06:51.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.792+0000 7fe486533800 1 -- 172.21.15.50:0/2545312528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caed720 msgr2=0x55bf2cbe8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.792+0000 7fe486533800 1 --2- 172.21.15.50:0/2545312528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55bf2caed720 0x55bf2cbe8a00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe474000c00 tx=0x7fe474000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.792+0000 7fe4829ef640 1 -- 172.21.15.50:0/2545312528 reap_dead start 2024-05-08T18:06:51.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.792+0000 7fe486533800 1 -- 172.21.15.50:0/2545312528 shutdown_connections 2024-05-08T18:06:51.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.792+0000 7fe486533800 1 -- 172.21.15.50:0/2545312528 >> 172.21.15.50:0/2545312528 conn(0x55bf2cae8ce0 msgr2=0x55bf2ca61e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.792+0000 7fe486533800 1 -- 172.21.15.50:0/2545312528 shutdown_connections 2024-05-08T18:06:51.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.792+0000 7fe486533800 1 -- 172.21.15.50:0/2545312528 wait complete. 2024-05-08T18:06:51.814 DEBUG:tasks.util.rgw: json result: {'user_id': 'a_user', 'display_name': 'True Admin User', 'email': '', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'a_user', 'access_key': 'true_admin_akey', 'secret_key': 'true_admin_skey', 'active': True, 'create_date': '2024-05-08T18:06:51.777336Z'}], 'swift_keys': [], 'caps': [{'type': 'user-info-without-keys', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'admin': True, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.777323Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:51.814 ERROR:root:{'user_id': 'a_user', 'display_name': 'True Admin User', 'email': '', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'a_user', 'access_key': 'true_admin_akey', 'secret_key': 'true_admin_skey', 'active': True, 'create_date': '2024-05-08T18:06:51.777336Z'}], 'swift_keys': [], 'caps': [{'type': 'user-info-without-keys', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'admin': True, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.777323Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:51.814 ERROR:root:0 2024-05-08T18:06:51.817 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:51.822 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:51.822 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'a_user'] 2024-05-08T18:06:51.822 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'a_user'] 2024-05-08T18:06:51.823 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid a_user 2024-05-08T18:06:51.872 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:51.872 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:51.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.875+0000 7f227e72f800 1 Processor -- start 2024-05-08T18:06:51.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.875+0000 7f227e72f800 1 -- start start 2024-05-08T18:06:51.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.875+0000 7f227e72f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x556d42695c90 0x556d42698080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.875+0000 7f227e72f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x556d426985c0 con 0x556d42692180 2024-05-08T18:06:51.874 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.875+0000 7f227e72f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x556d42531e20 con 0x556d42695c90 2024-05-08T18:06:51.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.876+0000 7f2279cbd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x556d42695c90 0x556d42698080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.876+0000 7f227a4be640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x556d42692180 0x556d42694570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50852/0 (socket says 172.21.15.50:50852) 2024-05-08T18:06:51.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.876+0000 7f227a4be640 1 -- 172.21.15.50:0/3097299582 learned_addr learned my addr 172.21.15.50:0/3097299582 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:51.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.876+0000 7f2279cbd640 1 -- 172.21.15.50:0/3097299582 >> v1:172.21.15.46:6789/0 conn(0x556d42692180 legacy=0x556d42694570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.876+0000 7f22794bc640 1 -- 172.21.15.50:0/3097299582 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2455866255 0 0) 0x556d426985c0 con 0x556d42692180 2024-05-08T18:06:51.875 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.876+0000 7f2279cbd640 1 -- 172.21.15.50:0/3097299582 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d426997a0 con 0x556d42695c90 2024-05-08T18:06:51.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.878+0000 7f2279cbd640 1 --2- 172.21.15.50:0/3097299582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42695c90 0x556d42698080 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f226400a470 tx=0x7f226401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00b0796c815dc8a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.878+0000 7f22794bc640 1 -- 172.21.15.50:0/3097299582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2264057070 con 0x556d42695c90 2024-05-08T18:06:51.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.878+0000 7f22794bc640 1 -- 172.21.15.50:0/3097299582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2264052570 con 0x556d42695c90 2024-05-08T18:06:51.876 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.878+0000 7f22794bc640 1 -- 172.21.15.50:0/3097299582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2264052870 con 0x556d42695c90 2024-05-08T18:06:51.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.878+0000 7f227e72f800 1 -- 172.21.15.50:0/3097299582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42695c90 msgr2=0x556d42698080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.878+0000 7f227e72f800 1 --2- 172.21.15.50:0/3097299582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42695c90 0x556d42698080 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f226400a470 tx=0x7f226401ce90 comp rx=0 tx=0).stop 2024-05-08T18:06:51.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.879+0000 7f227e72f800 1 -- 172.21.15.50:0/3097299582 shutdown_connections 2024-05-08T18:06:51.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.879+0000 7f227e72f800 1 --2- 172.21.15.50:0/3097299582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42695c90 0x556d42698080 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.879+0000 7f227e72f800 1 -- 172.21.15.50:0/3097299582 >> 172.21.15.50:0/3097299582 conn(0x556d4268bad0 msgr2=0x556d4268df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.879+0000 7f227e72f800 1 -- 172.21.15.50:0/3097299582 shutdown_connections 2024-05-08T18:06:51.877 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.879+0000 7f227e72f800 1 -- 172.21.15.50:0/3097299582 wait complete. 2024-05-08T18:06:51.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.894+0000 7f227e72f800 1 Processor -- start 2024-05-08T18:06:51.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.894+0000 7f227e72f800 1 -- start start 2024-05-08T18:06:51.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.894+0000 7f227e72f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42692180 0x556d42782910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.894+0000 7f227e72f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556d425dca10 con 0x556d42692180 2024-05-08T18:06:51.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.894+0000 7f227a4be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42692180 0x556d42782910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.894+0000 7f227a4be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42692180 0x556d42782910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46396/0 (socket says 172.21.15.50:46396) 2024-05-08T18:06:51.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.894+0000 7f227a4be640 1 -- 172.21.15.50:0/2451792348 learned_addr learned my addr 172.21.15.50:0/2451792348 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.895+0000 7f227a4be640 1 -- 172.21.15.50:0/2451792348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d42782ee0 con 0x556d42692180 2024-05-08T18:06:51.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.895+0000 7f227a4be640 1 --2- 172.21.15.50:0/2451792348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42692180 0x556d42782910 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f226c001e70 tx=0x7f226c00b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a08613e702fc5fb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.895+0000 7f2168608640 1 -- 172.21.15.50:0/2451792348 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f226c008070 con 0x556d42692180 2024-05-08T18:06:51.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.895+0000 7f2168608640 1 -- 172.21.15.50:0/2451792348 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226c00d500 con 0x556d42692180 2024-05-08T18:06:51.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.895+0000 7f2168608640 1 -- 172.21.15.50:0/2451792348 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f226c00d8a0 con 0x556d42692180 2024-05-08T18:06:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.896+0000 7f227e72f800 1 -- 172.21.15.50:0/2451792348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42692180 msgr2=0x556d42782910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.896+0000 7f227e72f800 1 --2- 172.21.15.50:0/2451792348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42692180 0x556d42782910 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f226c001e70 tx=0x7f226c00b6a0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.896+0000 7f227e72f800 1 -- 172.21.15.50:0/2451792348 shutdown_connections 2024-05-08T18:06:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.896+0000 7f227e72f800 1 --2- 172.21.15.50:0/2451792348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42692180 0x556d42782910 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.896+0000 7f227e72f800 1 -- 172.21.15.50:0/2451792348 >> 172.21.15.50:0/2451792348 conn(0x556d4268bad0 msgr2=0x556d4277e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.896+0000 7f227e72f800 1 -- 172.21.15.50:0/2451792348 shutdown_connections 2024-05-08T18:06:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.896+0000 7f227e72f800 1 -- 172.21.15.50:0/2451792348 wait complete. 2024-05-08T18:06:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.896+0000 7f227e72f800 1 Processor -- start 2024-05-08T18:06:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.896+0000 7f227e72f800 1 -- start start 2024-05-08T18:06:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.896+0000 7f227e72f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42692180 0x556d4278a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.896+0000 7f227e72f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556d42698a90 con 0x556d42692180 2024-05-08T18:06:51.897 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.896+0000 7f227a4be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42692180 0x556d4278a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.896+0000 7f227a4be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42692180 0x556d4278a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46402/0 (socket says 172.21.15.50:46402) 2024-05-08T18:06:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.896+0000 7f227a4be640 1 -- 172.21.15.50:0/3767112570 learned_addr learned my addr 172.21.15.50:0/3767112570 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.897+0000 7f227a4be640 1 -- 172.21.15.50:0/3767112570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d4278aa30 con 0x556d42692180 2024-05-08T18:06:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.897+0000 7f227a4be640 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42692180 0x556d4278a4f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x556d427836a0 tx=0x7f226c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.897+0000 7f2166e05640 1 -- 172.21.15.50:0/3767112570 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f226c00f980 con 0x556d42692180 2024-05-08T18:06:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.897+0000 7f2166e05640 1 -- 172.21.15.50:0/3767112570 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226c00fb20 con 0x556d42692180 2024-05-08T18:06:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.897+0000 7f2166e05640 1 -- 172.21.15.50:0/3767112570 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f226c00fe00 con 0x556d42692180 2024-05-08T18:06:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.897+0000 7f227e72f800 1 -- 172.21.15.50:0/3767112570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556d4278acc0 con 0x556d42692180 2024-05-08T18:06:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.897+0000 7f227e72f800 1 -- 172.21.15.50:0/3767112570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d4278d920 con 0x556d42692180 2024-05-08T18:06:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.898+0000 7f2166e05640 1 -- 172.21.15.50:0/3767112570 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f226c017040 con 0x556d42692180 2024-05-08T18:06:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.898+0000 7f2166e05640 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f216003bba0 0x7f216003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.898 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.898+0000 7f2166e05640 1 -- 172.21.15.50:0/3767112570 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f226c015020 con 0x556d42692180 2024-05-08T18:06:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.898+0000 7f2279cbd640 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f216003bba0 0x7f216003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.898+0000 7f227e72f800 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556d42695c90 0x556d427916e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.898+0000 7f227e72f800 1 -- 172.21.15.50:0/3767112570 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d42791c20 con 0x556d42695c90 2024-05-08T18:06:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.898+0000 7f227acbf640 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556d42695c90 0x556d427916e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.899+0000 7f2279cbd640 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f216003bba0 0x7f216003e060 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2264054000 tx=0x7f226404e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.899+0000 7f227acbf640 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556d42695c90 0x556d427916e0 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.899+0000 7f227acbf640 1 -- 172.21.15.50:0/3767112570 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f227400a4a0 con 0x556d42695c90 2024-05-08T18:06:51.899 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.899+0000 7f227e72f800 1 -- 172.21.15.50:0/3767112570 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d4278ecb0 con 0x556d42695c90 2024-05-08T18:06:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.899+0000 7f227acbf640 1 -- 172.21.15.50:0/3767112570 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f227400a4a0 con 0x556d42695c90 2024-05-08T18:06:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.900+0000 7f227e72f800 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556d42793f30 0x556d42796320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.900+0000 7f227e72f800 1 -- 172.21.15.50:0/3767112570 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d42796a10 con 0x556d42793f30 2024-05-08T18:06:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.900+0000 7f227a4be640 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556d42793f30 0x556d42796320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.900+0000 7f227a4be640 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556d42793f30 0x556d42796320 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.900+0000 7f227a4be640 1 -- 172.21.15.50:0/3767112570 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f226c02d070 con 0x556d42793f30 2024-05-08T18:06:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.900+0000 7f227e72f800 1 -- 172.21.15.50:0/3767112570 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d42792da0 con 0x556d42695c90 2024-05-08T18:06:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.901+0000 7f227acbf640 1 -- 172.21.15.50:0/3767112570 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f227400a4a0 con 0x556d42695c90 2024-05-08T18:06:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.901+0000 7f227e72f800 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556d42797e70 0x556d4279a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.901+0000 7f227e72f800 1 -- 172.21.15.50:0/3767112570 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d4279a950 con 0x556d42797e70 2024-05-08T18:06:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.901+0000 7f2279cbd640 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556d42797e70 0x556d4279a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.901+0000 7f2279cbd640 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556d42797e70 0x556d4279a260 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.901+0000 7f2279cbd640 1 -- 172.21.15.50:0/3767112570 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f22640528e0 con 0x556d42797e70 2024-05-08T18:06:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.902+0000 7f227e72f800 1 Processor -- start 2024-05-08T18:06:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.902+0000 7f227e72f800 1 -- start start 2024-05-08T18:06:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.902+0000 7f227e72f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427a8c50 0x556d427a9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.902+0000 7f227e72f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556d42583330 con 0x556d427a8c50 2024-05-08T18:06:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.902+0000 7f227a4be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427a8c50 0x556d427a9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.902+0000 7f227a4be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427a8c50 0x556d427a9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46404/0 (socket says 172.21.15.50:46404) 2024-05-08T18:06:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.902+0000 7f227a4be640 1 -- 172.21.15.50:0/332789285 learned_addr learned my addr 172.21.15.50:0/332789285 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.903+0000 7f227a4be640 1 -- 172.21.15.50:0/332789285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d427a95b0 con 0x556d427a8c50 2024-05-08T18:06:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.903+0000 7f227a4be640 1 --2- 172.21.15.50:0/332789285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427a8c50 0x556d427a9070 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f226c0517c0 tx=0x7f226c01ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.903+0000 7f215ffff640 1 -- 172.21.15.50:0/332789285 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f226c01a540 con 0x556d427a8c50 2024-05-08T18:06:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.903+0000 7f215ffff640 1 -- 172.21.15.50:0/332789285 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226c0501b0 con 0x556d427a8c50 2024-05-08T18:06:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.903+0000 7f215ffff640 1 -- 172.21.15.50:0/332789285 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f226c0183d0 con 0x556d427a8c50 2024-05-08T18:06:51.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.903+0000 7f227e72f800 1 -- 172.21.15.50:0/332789285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556d427aa020 con 0x556d427a8c50 2024-05-08T18:06:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.903+0000 7f227e72f800 1 -- 172.21.15.50:0/332789285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d427aa520 con 0x556d427a8c50 2024-05-08T18:06:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.904+0000 7f215ffff640 1 -- 172.21.15.50:0/332789285 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f226c04a300 con 0x556d427a8c50 2024-05-08T18:06:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.904+0000 7f215ffff640 1 --2- 172.21.15.50:0/332789285 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f215403bba0 0x7f215403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.904+0000 7f215ffff640 1 -- 172.21.15.50:0/332789285 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f226c03bfb0 con 0x556d427a8c50 2024-05-08T18:06:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.904+0000 7f2279cbd640 1 --2- 172.21.15.50:0/332789285 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f215403bba0 0x7f215403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.904+0000 7f227e72f800 1 Processor -- start 2024-05-08T18:06:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.904+0000 7f227e72f800 1 -- start start 2024-05-08T18:06:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.904+0000 7f2279cbd640 1 --2- 172.21.15.50:0/332789285 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f215403bba0 0x7f215403e060 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x556d427924e0 tx=0x7f2264005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.905+0000 7f227e72f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427b5dd0 0x556d427b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.905+0000 7f227e72f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556d42783260 con 0x556d427b5dd0 2024-05-08T18:06:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.905+0000 7f227a4be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427b5dd0 0x556d427b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.905+0000 7f227a4be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427b5dd0 0x556d427b61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46416/0 (socket says 172.21.15.50:46416) 2024-05-08T18:06:51.911 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.905+0000 7f227a4be640 1 -- 172.21.15.50:0/2967495578 learned_addr learned my addr 172.21.15.50:0/2967495578 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.905+0000 7f227a4be640 1 -- 172.21.15.50:0/2967495578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d427b67c0 con 0x556d427b5dd0 2024-05-08T18:06:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.905+0000 7f227a4be640 1 --2- 172.21.15.50:0/2967495578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427b5dd0 0x556d427b61f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f226c0427f0 tx=0x7f226c026d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2a4a26be2c2443 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.905+0000 7f215dffb640 1 -- 172.21.15.50:0/2967495578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f226c057030 con 0x556d427b5dd0 2024-05-08T18:06:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.905+0000 7f215dffb640 1 -- 172.21.15.50:0/2967495578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226c02f3e0 con 0x556d427b5dd0 2024-05-08T18:06:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.905+0000 7f215dffb640 1 -- 172.21.15.50:0/2967495578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f226c030e50 con 0x556d427b5dd0 2024-05-08T18:06:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.906+0000 7f227e72f800 1 -- 172.21.15.50:0/2967495578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427b5dd0 msgr2=0x556d427b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.906+0000 7f227e72f800 1 --2- 172.21.15.50:0/2967495578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427b5dd0 0x556d427b61f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f226c0427f0 tx=0x7f226c026d00 comp rx=0 tx=0).stop 2024-05-08T18:06:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.906+0000 7f227e72f800 1 -- 172.21.15.50:0/2967495578 shutdown_connections 2024-05-08T18:06:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.906+0000 7f227e72f800 1 --2- 172.21.15.50:0/2967495578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427b5dd0 0x556d427b61f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.906+0000 7f227e72f800 1 -- 172.21.15.50:0/2967495578 >> 172.21.15.50:0/2967495578 conn(0x556d427b20a0 msgr2=0x556d427b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.906+0000 7f227e72f800 1 -- 172.21.15.50:0/2967495578 shutdown_connections 2024-05-08T18:06:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.906+0000 7f227e72f800 1 -- 172.21.15.50:0/2967495578 wait complete. 2024-05-08T18:06:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.906+0000 7f227e72f800 1 Processor -- start 2024-05-08T18:06:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.906+0000 7f227e72f800 1 -- start start 2024-05-08T18:06:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.906+0000 7f227e72f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427b5dd0 0x556d427bd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.906+0000 7f227e72f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556d427b6e90 con 0x556d427b5dd0 2024-05-08T18:06:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.906+0000 7f227a4be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427b5dd0 0x556d427bd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.906+0000 7f227a4be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427b5dd0 0x556d427bd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46418/0 (socket says 172.21.15.50:46418) 2024-05-08T18:06:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.906+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 learned_addr learned my addr 172.21.15.50:0/3754986506 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.907+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d427bd770 con 0x556d427b5dd0 2024-05-08T18:06:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.907+0000 7f227a4be640 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427b5dd0 0x556d427bd230 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f226c042570 tx=0x7f226c03a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.907+0000 7f214bfff640 1 -- 172.21.15.50:0/3754986506 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f226c026430 con 0x556d427b5dd0 2024-05-08T18:06:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.907+0000 7f214bfff640 1 -- 172.21.15.50:0/3754986506 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f226c026610 con 0x556d427b5dd0 2024-05-08T18:06:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.907+0000 7f214bfff640 1 -- 172.21.15.50:0/3754986506 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f226c026930 con 0x556d427b5dd0 2024-05-08T18:06:51.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.907+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556d427bd770 con 0x556d427b5dd0 2024-05-08T18:06:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.907+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d427bd770 con 0x556d427b5dd0 2024-05-08T18:06:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.908+0000 7f214bfff640 1 -- 172.21.15.50:0/3754986506 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f226c06f030 con 0x556d427b5dd0 2024-05-08T18:06:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.908+0000 7f214bfff640 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f214403bd40 0x7f214403e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.908+0000 7f214bfff640 1 -- 172.21.15.50:0/3754986506 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f226c06d070 con 0x556d427b5dd0 2024-05-08T18:06:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.908+0000 7f2279cbd640 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f214403bd40 0x7f214403e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.909+0000 7f2279cbd640 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f214403bd40 0x7f214403e200 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2264052fd0 tx=0x7f226404f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.909+0000 7f227e72f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.909+0000 7f227e72f800 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556d427c8890 0x556d427cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.909+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d427cb4a0 con 0x556d427c8890 2024-05-08T18:06:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.909+0000 7f227acbf640 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556d427c8890 0x556d427cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.909+0000 7f227acbf640 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556d427c8890 0x556d427cad50 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f227400a4a0 con 0x556d427c8890 2024-05-08T18:06:51.914 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227e72f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227e72f800 20 realm 2024-05-08T18:06:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227e72f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d427cb8b0 con 0x556d427c8890 2024-05-08T18:06:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f227400a4a0 con 0x556d427c8890 2024-05-08T18:06:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227e72f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227e72f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227e72f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d427cc180 con 0x556d427c8890 2024-05-08T18:06:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f227400a4a0 con 0x556d427c8890 2024-05-08T18:06:51.915 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227e72f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227e72f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d427cc520 con 0x556d427c8890 2024-05-08T18:06:51.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f227400a4a0 con 0x556d427c8890 2024-05-08T18:06:51.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227e72f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:51.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227e72f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227e72f800 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556d427cd280 0x556d427cf670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.910+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d427cfc80 con 0x556d427cd280 2024-05-08T18:06:51.916 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.911+0000 7f227a4be640 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556d427cd280 0x556d427cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.917 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.911+0000 7f227a4be640 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556d427cd280 0x556d427cf670 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.911+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.911+0000 7f227e72f800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:51.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.911+0000 7f227e72f800 20 searching for the correct realm 2024-05-08T18:06:51.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.911+0000 7f227e72f800 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556d427d1c50 0x556d427d40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.911+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556d427d47b0 con 0x556d427d1c50 2024-05-08T18:06:51.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.911+0000 7f2279cbd640 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556d427d1c50 0x556d427d40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.912+0000 7f2279cbd640 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556d427d1c50 0x556d427d40e0 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.912+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.912+0000 7f2279cbd640 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2264076040 0x7f226405e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:51.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.912+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556d427d47b0 con 0x7f2264076040 2024-05-08T18:06:51.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.912+0000 7f227acbf640 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2264076040 0x7f226405e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:51.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.912+0000 7f227acbf640 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2264076040 0x7f226405e480 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:51.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.913+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f227400a4a0 con 0x7f2264076040 2024-05-08T18:06:51.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.913+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556d427cc520 con 0x556d427cd280 2024-05-08T18:06:51.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.913+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.913+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556d427cc520 con 0x556d427c8890 2024-05-08T18:06:51.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.913+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f227400a4a0 con 0x556d427c8890 2024-05-08T18:06:51.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.913+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556d427cc520 con 0x556d427c8890 2024-05-08T18:06:51.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.913+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f227400a4a0 con 0x556d427c8890 2024-05-08T18:06:51.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.913+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556d427cc520 con 0x556d427c8890 2024-05-08T18:06:51.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.914+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f227400a4a0 con 0x556d427c8890 2024-05-08T18:06:51.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.914+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556d427cc520 con 0x556d427cd280 2024-05-08T18:06:51.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.914+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v29'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.914+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556d427cc520 con 0x7f2264076040 2024-05-08T18:06:51.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.914+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f227400a4a0 con 0x7f2264076040 2024-05-08T18:06:51.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.914+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556d427cc520 con 0x556d427d1c50 2024-05-08T18:06:51.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.914+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.914+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556d427cc520 con 0x556d427c8890 2024-05-08T18:06:51.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.915+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f227400a4a0 con 0x556d427c8890 2024-05-08T18:06:51.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.915+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556d427cc520 con 0x556d427cd280 2024-05-08T18:06:51.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.915+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.915+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556d427cc520 con 0x556d427c8890 2024-05-08T18:06:51.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.915+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f227400a4a0 con 0x556d427c8890 2024-05-08T18:06:51.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.915+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556d427cc520 con 0x7f2264076040 2024-05-08T18:06:51.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.915+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.915+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556d427cc520 con 0x556d427d1c50 2024-05-08T18:06:51.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.916+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.916+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x556d427d1c50 2024-05-08T18:06:51.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.916+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.916+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x556d427cd280 2024-05-08T18:06:51.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.916+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.916+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x556d427d1c50 2024-05-08T18:06:51.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.917+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.917+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x556d427d1c50 2024-05-08T18:06:51.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.917+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.917+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x556d427c8890 2024-05-08T18:06:51.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.917+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.917+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x556d427c8890 2024-05-08T18:06:51.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.917+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v29'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.917+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x556d427cd280 2024-05-08T18:06:51.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.918+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.918+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x556d427d1c50 2024-05-08T18:06:51.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.918+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.918+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x7f2264076040 2024-05-08T18:06:51.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.918+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.918+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x556d427c8890 2024-05-08T18:06:51.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.918+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.919+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x556d427cd280 2024-05-08T18:06:51.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.919+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.919+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x556d427d1c50 2024-05-08T18:06:51.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.919+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.919+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x556d427cd280 2024-05-08T18:06:51.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.919+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.938 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.919+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x7f2264076040 2024-05-08T18:06:51.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.920+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.920+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x556d427d1c50 2024-05-08T18:06:51.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.920+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.920+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x556d427cd280 2024-05-08T18:06:51.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.920+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.920+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x556d427c8890 2024-05-08T18:06:51.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.920+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v29'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.920+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f2264090040 con 0x7f2264076040 2024-05-08T18:06:51.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:51.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:51.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:51.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:51.939 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:51.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:51.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d427d0540 con 0x556d427c8890 2024-05-08T18:06:51.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d427d0540 con 0x556d427c8890 2024-05-08T18:06:51.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:51.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.940 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d427d0540 con 0x556d427d1c50 2024-05-08T18:06:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 20 zone default found 2024-05-08T18:06:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 4 Realm: () 2024-05-08T18:06:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 20 zonegroup default 2024-05-08T18:06:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.921+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d427d0e90 con 0x556d427cd280 2024-05-08T18:06:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.922+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.922+0000 7f227e72f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.941 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.922+0000 7f227e72f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.922+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d427d66a0 con 0x556d427d1c50 2024-05-08T18:06:51.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.922+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.922+0000 7f227e72f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.922+0000 7f227e72f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.922+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d427d6b20 con 0x556d427d1c50 2024-05-08T18:06:51.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.922+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.922+0000 7f227e72f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:51.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.922+0000 7f227e72f800 20 started sync module instance, tier type = 2024-05-08T18:06:51.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.922+0000 7f227e72f800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:51.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.922+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427d5ab0 con 0x7f2264076040 2024-05-08T18:06:51.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.923+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.923+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93927755308976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427d5ab0 con 0x7f2264076040 2024-05-08T18:06:51.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.923+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427d75a0 con 0x556d427c8890 2024-05-08T18:06:51.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.924+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.925+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93927755313392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427d7570 con 0x556d427c8890 2024-05-08T18:06:51.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.925+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93927755308976] v29'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.925+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427d8590 con 0x556d427d1c50 2024-05-08T18:06:51.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.926+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93927755313392] v29'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.926+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.926+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93927755315648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427d5600 con 0x556d427d1c50 2024-05-08T18:06:51.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.926+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427d8ea0 con 0x7f2264076040 2024-05-08T18:06:51.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.927+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93927755315648] v29'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.927+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.927+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93927755317344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427d9290 con 0x7f2264076040 2024-05-08T18:06:51.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.927+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427d9c90 con 0x556d427c8890 2024-05-08T18:06:51.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.928+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93927755317344] v29'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.928+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.928+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93927755319728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427d9c50 con 0x556d427c8890 2024-05-08T18:06:51.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.928+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427da6a0 con 0x556d427cd280 2024-05-08T18:06:51.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.929+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.929+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93927755325936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427daef0 con 0x556d427cd280 2024-05-08T18:06:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.929+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427db9b0 con 0x556d427cd280 2024-05-08T18:06:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.929+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93927755319728] v29'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.930+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93927755325936] v29'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.930+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.930+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93927755330688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427dbd50 con 0x556d427cd280 2024-05-08T18:06:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.930+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427dc5a0 con 0x556d427cd280 2024-05-08T18:06:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.931+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93927755330688] v29'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.931+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.931+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93927755333872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427dc9c0 con 0x556d427cd280 2024-05-08T18:06:51.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.931+0000 7f227e72f800 20 add_watcher() i=0 2024-05-08T18:06:51.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.931+0000 7f227e72f800 20 add_watcher() i=1 2024-05-08T18:06:51.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.931+0000 7f227e72f800 20 add_watcher() i=2 2024-05-08T18:06:51.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.931+0000 7f227e72f800 20 add_watcher() i=3 2024-05-08T18:06:51.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.931+0000 7f227e72f800 20 add_watcher() i=4 2024-05-08T18:06:51.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.931+0000 7f227e72f800 20 add_watcher() i=5 2024-05-08T18:06:51.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.931+0000 7f227e72f800 20 add_watcher() i=6 2024-05-08T18:06:51.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.932+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93927755333872] v29'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.932+0000 7f227e72f800 20 add_watcher() i=7 2024-05-08T18:06:51.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.932+0000 7f227e72f800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:51.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.932+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d427dfdc0 con 0x556d427c8890 2024-05-08T18:06:51.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.933+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.933+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93927755335392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427dd2e0 con 0x556d427c8890 2024-05-08T18:06:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.934+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93927755335392] v29'252 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.934+0000 7f2130fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.934+0000 7f227e72f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.934+0000 7f227e72f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.934+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427e8730 con 0x556d427c8890 2024-05-08T18:06:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.935+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v29'37 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.935+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427e8de0 con 0x556d427cd280 2024-05-08T18:06:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.936+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(63 gc.1 [create,call,call,call] v29'31 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.936+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427e9220 con 0x556d427d1c50 2024-05-08T18:06:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.937+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v29'30 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.937+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427e9610 con 0x556d427c8890 2024-05-08T18:06:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.938+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v29'253 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.938+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427e6670 con 0x7f2264076040 2024-05-08T18:06:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.939+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v29'80 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.939+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427e6670 con 0x556d427d1c50 2024-05-08T18:06:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.940+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v29'53 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.940+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427ea320 con 0x556d427c8890 2024-05-08T18:06:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.941+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v29'33 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.951 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.942+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427ea320 con 0x556d427cd280 2024-05-08T18:06:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.943+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(69 gc.7 [create,call,call,call] v29'53 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.943+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427ea7d0 con 0x7f2264076040 2024-05-08T18:06:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.944+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v29'30 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.944+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427ea7d0 con 0x556d427d1c50 2024-05-08T18:06:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.945+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v29'36 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.945+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427ea7d0 con 0x7f2264076040 2024-05-08T18:06:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.945+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v29'81 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.945+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427ea860 con 0x556d427cd280 2024-05-08T18:06:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.947+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(73 gc.11 [create,call,call,call] v29'36 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.947+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427ea920 con 0x7f2264076040 2024-05-08T18:06:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.948+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v29'31 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.948+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427ea920 con 0x556d427cd280 2024-05-08T18:06:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.949+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(75 gc.13 [create,call,call,call] v29'59 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.949+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427ea8f0 con 0x556d427c8890 2024-05-08T18:06:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.950+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v29'254 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.952 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.950+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427ea8f0 con 0x556d427d1c50 2024-05-08T18:06:51.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.951+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v29'75 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.951+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427eae50 con 0x7f2264076040 2024-05-08T18:06:51.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.952+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v29'82 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.952+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427eae50 con 0x556d427d1c50 2024-05-08T18:06:51.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.953+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v29'30 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.953 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.953+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427eae50 con 0x556d427c8890 2024-05-08T18:06:51.954 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.954+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v29'38 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.954+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427eae50 con 0x556d427c8890 2024-05-08T18:06:51.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.955+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v29'255 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.955+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427eae50 con 0x556d427d1c50 2024-05-08T18:06:51.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.956+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v29'76 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.956+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427eae50 con 0x7f2264076040 2024-05-08T18:06:51.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.956+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v29'40 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.955 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.957+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427eae50 con 0x556d427d1c50 2024-05-08T18:06:51.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.957+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v29'32 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.958+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427eae50 con 0x556d427d1c50 2024-05-08T18:06:51.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.958+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v29'36 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.956 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.958+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427eae50 con 0x556d427d1c50 2024-05-08T18:06:51.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.959+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v29'28 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.957 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.959+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427eae50 con 0x556d427d1c50 2024-05-08T18:06:51.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.960+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v29'54 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.958 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.960+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427eae50 con 0x556d427cd280 2024-05-08T18:06:51.959 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.962+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(88 gc.26 [create,call,call,call] v29'32 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.962+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427eae50 con 0x556d427d1c50 2024-05-08T18:06:51.960 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.963+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v29'31 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.963+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427eae50 con 0x556d427cd280 2024-05-08T18:06:51.961 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.964+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(90 gc.28 [create,call,call,call] v29'54 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.964+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427eae50 con 0x556d427d1c50 2024-05-08T18:06:51.962 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.965+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v29'77 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.963 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.965+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427eae50 con 0x556d427cd280 2024-05-08T18:06:51.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.966+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(92 gc.30 [create,call,call,call] v29'60 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.966+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427eae50 con 0x7f2264076040 2024-05-08T18:06:51.964 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.967+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v29'36 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.967+0000 7f20ab7fe640 20 reqs_thread_entry: start 2024-05-08T18:06:51.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.967+0000 7f227e72f800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:51.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.967+0000 7f227e72f800 20 Filter name: none 2024-05-08T18:06:51.966 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.967+0000 7f20a97fa640 20 reqs_thread_entry: start 2024-05-08T18:06:51.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.967+0000 7f227e72f800 10 cache get: name=default.rgw.meta+users.uid+a_user : miss 2024-05-08T18:06:51.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.967+0000 7f227e72f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:51.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.967+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:94 8.14 8:2efb937b:users.uid::a_user:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d42815d20 con 0x556d427d1c50 2024-05-08T18:06:51.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.968+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(94 a_user [call out=48b,read 0~363 out=363b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 234+0+486 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.968+0000 7f227e72f800 20 rados_obj.operate() r=0 bl.length=363 2024-05-08T18:06:51.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.968+0000 7f227e72f800 10 cache put: name=default.rgw.meta+users.uid+a_user info.flags=0x13 2024-05-08T18:06:51.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.968+0000 7f227e72f800 10 adding default.rgw.meta+users.uid+a_user to cache LRU end 2024-05-08T18:06:51.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.968+0000 7f227e72f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:51.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.968+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:95 8.11 8:8d3ad08f:users.uid::a_user.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d428187b0 con 0x556d427d1c50 2024-05-08T18:06:51.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.968+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 28 ==== osd_op_reply(95 a_user.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 158+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.967 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.968+0000 7f227e72f800 10 removing key index: true_admin_akey 2024-05-08T18:06:51.968 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.968+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:96 8.15 8:afe09604:users.keys::true_admin_akey:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d42816790 con 0x556d427d1c50 2024-05-08T18:06:51.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.969+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 29 ==== osd_op_reply(96 true_admin_akey [delete] v29'4 uv4 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.969+0000 7f227e72f800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.keys:true_admin_akey, ofs0, ns] 2024-05-08T18:06:51.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.969+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:97 7.3 7:c609908c:::notify.5:head [notify cookie 93927755575968 in=177b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d42818cc0 con 0x556d427cd280 2024-05-08T18:06:51.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.969+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== watch-notify(notify (1) cookie 93927755325936 notify 124554051631 ret 0) ==== 207+0+0 (crc 0 0 0) 0x556d427e64b0 con 0x556d427cd280 2024-05-08T18:06:51.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.969+0000 7f215dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051631 cookie 93927755325936 notifier 4858 bl.length()=165 2024-05-08T18:06:51.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.969+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(97 notify.5 [notify cookie 93927755575968 out=8b] v0'0 uv255 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.969+0000 7f215dffb640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:98 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f2140003d60 con 0x556d427cd280 2024-05-08T18:06:51.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.970+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(98 notify.5 [notify-ack] v0'0 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.970+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== watch-notify(notify_complete (2) cookie 93927755575968 notify 124554051631 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f226c035040 con 0x556d427cd280 2024-05-08T18:06:51.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.970+0000 7f227e72f800 10 removing email index: 2024-05-08T18:06:51.971 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.970+0000 7f227e72f800 10 removing user buckets index 2024-05-08T18:06:51.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.970+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:99 8.11 8:8d3ad08f:users.uid::a_user.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d42818c20 con 0x556d427d1c50 2024-05-08T18:06:51.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.971+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 30 ==== osd_op_reply(99 a_user.buckets [delete] v29'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 158+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.971+0000 7f227e72f800 10 removing user index: a_user 2024-05-08T18:06:51.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.971+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:100 8.14 8:2efb937b:users.uid::a_user:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d428182b0 con 0x556d427d1c50 2024-05-08T18:06:51.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.972+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 31 ==== osd_op_reply(100 a_user [call,call,delete] v29'4 uv4 ondisk = 0) ==== 234+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.972+0000 7f227e72f800 10 removing default.rgw.meta+users.uid+a_user from cache 2024-05-08T18:06:51.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.972+0000 7f227e72f800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.uid:a_user, ofs0, ns] 2024-05-08T18:06:51.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.972+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:101 7.15 7:a93a5511:::notify.2:head [notify cookie 93927755581200 in=167b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556d42817750 con 0x556d427d1c50 2024-05-08T18:06:51.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.972+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 32 ==== watch-notify(notify (1) cookie 93927755315648 notify 124554051594 ret 0) ==== 197+0+0 (crc 0 0 0) 0x556d427e61c0 con 0x556d427d1c50 2024-05-08T18:06:51.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.972+0000 7f215d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051594 cookie 93927755315648 notifier 4858 bl.length()=155 2024-05-08T18:06:51.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.972+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 33 ==== osd_op_reply(101 notify.2 [notify cookie 93927755581200 out=8b] v0'0 uv255 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.972+0000 7f215d7fa640 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:102 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f20b00032c0 con 0x556d427d1c50 2024-05-08T18:06:51.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.972+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 34 ==== osd_op_reply(102 notify.2 [notify-ack] v0'0 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.972 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.972+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 35 ==== watch-notify(notify_complete (2) cookie 93927755581200 notify 124554051594 ret 0) ==== 42+0+48 (crc 0 0 0) 0x556d427ea150 con 0x556d427d1c50 2024-05-08T18:06:51.973 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.973+0000 7f2130fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:51.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.973+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:103 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93927755335392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d42818710 con 0x556d427c8890 2024-05-08T18:06:51.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.974+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(103 data_loggenerations_metadata [watch unwatch cookie 93927755335392] v29'256 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.974+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93927755308976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427dc6b0 con 0x7f2264076040 2024-05-08T18:06:51.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.975+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 93927755308976] v29'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.975+0000 7f227e72f800 20 remove_watcher() i=0 2024-05-08T18:06:51.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.975+0000 7f227e72f800 2 removed watcher, disabling cache 2024-05-08T18:06:51.974 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.975+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93927755313392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427d5fb0 con 0x556d427c8890 2024-05-08T18:06:51.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.976+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 93927755313392] v29'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.976+0000 7f227e72f800 20 remove_watcher() i=1 2024-05-08T18:06:51.975 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.976+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93927755315648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427d5fb0 con 0x556d427d1c50 2024-05-08T18:06:51.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.977+0000 7f2279cbd640 1 -- 172.21.15.50:0/3754986506 <== osd.3 v2:172.21.15.50:6800/3130492744 36 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 93927755315648] v29'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2264081070 con 0x556d427d1c50 2024-05-08T18:06:51.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.977+0000 7f227e72f800 20 remove_watcher() i=2 2024-05-08T18:06:51.976 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.977+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93927755317344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427d5fb0 con 0x7f2264076040 2024-05-08T18:06:51.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.978+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 93927755317344] v29'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2274032040 con 0x7f2264076040 2024-05-08T18:06:51.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.978+0000 7f227e72f800 20 remove_watcher() i=3 2024-05-08T18:06:51.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.978+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93927755319728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427d5fb0 con 0x556d427c8890 2024-05-08T18:06:51.977 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.979+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 93927755319728] v29'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2274032040 con 0x556d427c8890 2024-05-08T18:06:51.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.979+0000 7f227e72f800 20 remove_watcher() i=4 2024-05-08T18:06:51.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.979+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93927755325936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427d89b0 con 0x556d427cd280 2024-05-08T18:06:51.978 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.980+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 93927755325936] v29'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.980+0000 7f227e72f800 20 remove_watcher() i=5 2024-05-08T18:06:51.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.980+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93927755330688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427da1f0 con 0x556d427cd280 2024-05-08T18:06:51.979 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.981+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 93927755330688] v29'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.981+0000 7f227e72f800 20 remove_watcher() i=6 2024-05-08T18:06:51.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.981+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93927755333872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556d427d89b0 con 0x556d427cd280 2024-05-08T18:06:51.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.982+0000 7f227a4be640 1 -- 172.21.15.50:0/3754986506 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 93927755333872] v29'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f226c02d070 con 0x556d427cd280 2024-05-08T18:06:51.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.982+0000 7f227e72f800 20 remove_watcher() i=7 2024-05-08T18:06:51.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556d427cd280 msgr2=0x556d427cf670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556d427cd280 0x556d427cf670 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556d427c8890 msgr2=0x556d427cad50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556d427c8890 0x556d427cad50 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2264076040 msgr2=0x7f226405e480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f2264076040 0x7f226405e480 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556d427d1c50 msgr2=0x556d427d40e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556d427d1c50 0x556d427d40e0 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f214403bd40 msgr2=0x7f214403e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f214403bd40 0x7f214403e200 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2264052fd0 tx=0x7f226404f000 comp rx=0 tx=0).stop 2024-05-08T18:06:51.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227acbf640 1 -- 172.21.15.50:0/3754986506 reap_dead start 2024-05-08T18:06:51.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427b5dd0 msgr2=0x556d427bd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427b5dd0 0x556d427bd230 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f226c042570 tx=0x7f226c03a290 comp rx=0 tx=0).stop 2024-05-08T18:06:51.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 shutdown_connections 2024-05-08T18:06:51.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 --2- 172.21.15.50:0/3754986506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427b5dd0 0x556d427bd230 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 >> 172.21.15.50:0/3754986506 conn(0x556d427b20a0 msgr2=0x556d427af700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 shutdown_connections 2024-05-08T18:06:51.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 -- 172.21.15.50:0/3754986506 wait complete. 2024-05-08T18:06:51.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 -- 172.21.15.50:0/332789285 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f215403bba0 msgr2=0x7f215403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 --2- 172.21.15.50:0/332789285 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f215403bba0 0x7f215403e060 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x556d427924e0 tx=0x7f2264005a00 comp rx=0 tx=0).stop 2024-05-08T18:06:51.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 -- 172.21.15.50:0/332789285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427a8c50 msgr2=0x556d427a9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 --2- 172.21.15.50:0/332789285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427a8c50 0x556d427a9070 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f226c0517c0 tx=0x7f226c01ab20 comp rx=0 tx=0).stop 2024-05-08T18:06:51.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 -- 172.21.15.50:0/332789285 shutdown_connections 2024-05-08T18:06:51.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 --2- 172.21.15.50:0/332789285 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f215403bba0 0x7f215403e060 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 --2- 172.21.15.50:0/332789285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d427a8c50 0x556d427a9070 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 -- 172.21.15.50:0/332789285 >> 172.21.15.50:0/332789285 conn(0x556d4279ccf0 msgr2=0x556d4279d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 -- 172.21.15.50:0/332789285 shutdown_connections 2024-05-08T18:06:51.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.983+0000 7f227e72f800 1 -- 172.21.15.50:0/332789285 wait complete. 2024-05-08T18:06:51.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.984+0000 7f227e72f800 1 -- 172.21.15.50:0/3767112570 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556d42793f30 msgr2=0x556d42796320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.984+0000 7f227e72f800 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556d42793f30 0x556d42796320 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.986 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.984+0000 7f227e72f800 1 -- 172.21.15.50:0/3767112570 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556d42695c90 msgr2=0x556d427916e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.984+0000 7f227e72f800 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556d42695c90 0x556d427916e0 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.984+0000 7f227e72f800 1 -- 172.21.15.50:0/3767112570 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556d42797e70 msgr2=0x556d4279a260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.984+0000 7f227e72f800 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556d42797e70 0x556d4279a260 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.984+0000 7f227e72f800 1 -- 172.21.15.50:0/3767112570 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f216003bba0 msgr2=0x7f216003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.984+0000 7f227e72f800 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f216003bba0 0x7f216003e060 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2264054000 tx=0x7f226404e000 comp rx=0 tx=0).stop 2024-05-08T18:06:51.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.984+0000 7f227e72f800 1 -- 172.21.15.50:0/3767112570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42692180 msgr2=0x556d4278a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:51.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.984+0000 7f227e72f800 1 --2- 172.21.15.50:0/3767112570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556d42692180 0x556d4278a4f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x556d427836a0 tx=0x7f226c000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:51.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.984+0000 7f227acbf640 1 -- 172.21.15.50:0/3767112570 reap_dead start 2024-05-08T18:06:51.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.984+0000 7f227e72f800 1 -- 172.21.15.50:0/3767112570 shutdown_connections 2024-05-08T18:06:51.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.984+0000 7f227e72f800 1 -- 172.21.15.50:0/3767112570 >> 172.21.15.50:0/3767112570 conn(0x556d4268bad0 msgr2=0x556d4277ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:51.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.984+0000 7f227e72f800 1 -- 172.21.15.50:0/3767112570 shutdown_connections 2024-05-08T18:06:51.987 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:51.984+0000 7f227e72f800 1 -- 172.21.15.50:0/3767112570 wait complete. 2024-05-08T18:06:51.991 ERROR:root:None 2024-05-08T18:06:51.991 ERROR:root:0 2024-05-08T18:06:51.991 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=read'] 2024-05-08T18:06:51.992 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=read'] 2024-05-08T18:06:51.992 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps user-info-without-keys=read 2024-05-08T18:06:52.039 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:52.039 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:52.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.042+0000 7f231881d800 1 Processor -- start 2024-05-08T18:06:52.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.042+0000 7f231881d800 1 -- start start 2024-05-08T18:06:52.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.042+0000 7f231881d800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5613725f2cd0 0x5613725f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.042+0000 7f231881d800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5613725f5600 con 0x5613725ef1c0 2024-05-08T18:06:52.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.042+0000 7f231881d800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x56137248ee20 con 0x5613725f2cd0 2024-05-08T18:06:52.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.043+0000 7f2315b14640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5613725f2cd0 0x5613725f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.043+0000 7f2315b14640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5613725f2cd0 0x5613725f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46434/0 (socket says 172.21.15.50:46434) 2024-05-08T18:06:52.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.043+0000 7f2315b14640 1 -- 172.21.15.50:0/3394217268 learned_addr learned my addr 172.21.15.50:0/3394217268 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.043+0000 7f2316315640 1 --1- 172.21.15.50:0/3394217268 >> v1:172.21.15.46:6789/0 conn(0x5613725ef1c0 0x5613725f15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50856/0 (socket says 172.21.15.50:50856) 2024-05-08T18:06:52.041 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.043+0000 7f2315b14640 1 -- 172.21.15.50:0/3394217268 >> v1:172.21.15.46:6789/0 conn(0x5613725ef1c0 legacy=0x5613725f15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.042 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.043+0000 7f2315b14640 1 -- 172.21.15.50:0/3394217268 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613725f67e0 con 0x5613725f2cd0 2024-05-08T18:06:52.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.045+0000 7f2315b14640 1 --2- 172.21.15.50:0/3394217268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725f2cd0 0x5613725f50c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f230c00a520 tx=0x7f230c01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f9af142cc6bb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.045+0000 7f2315313640 1 -- 172.21.15.50:0/3394217268 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f230c057070 con 0x5613725f2cd0 2024-05-08T18:06:52.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.045+0000 7f2315313640 1 -- 172.21.15.50:0/3394217268 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f230c052570 con 0x5613725f2cd0 2024-05-08T18:06:52.043 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.045+0000 7f2315313640 1 -- 172.21.15.50:0/3394217268 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f230c04ae60 con 0x5613725f2cd0 2024-05-08T18:06:52.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.045+0000 7f231881d800 1 -- 172.21.15.50:0/3394217268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725f2cd0 msgr2=0x5613725f50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.045+0000 7f231881d800 1 --2- 172.21.15.50:0/3394217268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725f2cd0 0x5613725f50c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f230c00a520 tx=0x7f230c01d010 comp rx=0 tx=0).stop 2024-05-08T18:06:52.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.045+0000 7f231881d800 1 -- 172.21.15.50:0/3394217268 shutdown_connections 2024-05-08T18:06:52.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.045+0000 7f231881d800 1 --2- 172.21.15.50:0/3394217268 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725f2cd0 0x5613725f50c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.045+0000 7f231881d800 1 -- 172.21.15.50:0/3394217268 >> 172.21.15.50:0/3394217268 conn(0x5613725e8af0 msgr2=0x5613725eaf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.045+0000 7f231881d800 1 -- 172.21.15.50:0/3394217268 shutdown_connections 2024-05-08T18:06:52.044 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.046+0000 7f231881d800 1 -- 172.21.15.50:0/3394217268 wait complete. 2024-05-08T18:06:52.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.061+0000 7f231881d800 1 Processor -- start 2024-05-08T18:06:52.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.061+0000 7f231881d800 1 -- start start 2024-05-08T18:06:52.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.061+0000 7f231881d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ef1c0 0x5613726dfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.061+0000 7f231881d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561372539a30 con 0x5613725ef1c0 2024-05-08T18:06:52.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.061+0000 7f2316315640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ef1c0 0x5613726dfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.061+0000 7f2316315640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ef1c0 0x5613726dfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46444/0 (socket says 172.21.15.50:46444) 2024-05-08T18:06:52.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.061+0000 7f2316315640 1 -- 172.21.15.50:0/2834295265 learned_addr learned my addr 172.21.15.50:0/2834295265 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.061+0000 7f2316315640 1 -- 172.21.15.50:0/2834295265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613726e01f0 con 0x5613725ef1c0 2024-05-08T18:06:52.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.062+0000 7f2316315640 1 --2- 172.21.15.50:0/2834295265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ef1c0 0x5613726dfc20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f230800b260 tx=0x7f230800b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5293f2d9940d299 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.062+0000 7f2204608640 1 -- 172.21.15.50:0/2834295265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f230800e5c0 con 0x5613725ef1c0 2024-05-08T18:06:52.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.062+0000 7f2204608640 1 -- 172.21.15.50:0/2834295265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2308006070 con 0x5613725ef1c0 2024-05-08T18:06:52.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.062+0000 7f2204608640 1 -- 172.21.15.50:0/2834295265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f230800e890 con 0x5613725ef1c0 2024-05-08T18:06:52.064 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.062+0000 7f231881d800 1 -- 172.21.15.50:0/2834295265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ef1c0 msgr2=0x5613726dfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.063+0000 7f231881d800 1 --2- 172.21.15.50:0/2834295265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ef1c0 0x5613726dfc20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f230800b260 tx=0x7f230800b720 comp rx=0 tx=0).stop 2024-05-08T18:06:52.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.063+0000 7f231881d800 1 -- 172.21.15.50:0/2834295265 shutdown_connections 2024-05-08T18:06:52.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.063+0000 7f231881d800 1 --2- 172.21.15.50:0/2834295265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ef1c0 0x5613726dfc20 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.063+0000 7f231881d800 1 -- 172.21.15.50:0/2834295265 >> 172.21.15.50:0/2834295265 conn(0x5613725e8af0 msgr2=0x5613726db540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.063+0000 7f231881d800 1 -- 172.21.15.50:0/2834295265 shutdown_connections 2024-05-08T18:06:52.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.063+0000 7f231881d800 1 -- 172.21.15.50:0/2834295265 wait complete. 2024-05-08T18:06:52.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.063+0000 7f231881d800 1 Processor -- start 2024-05-08T18:06:52.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.063+0000 7f231881d800 1 -- start start 2024-05-08T18:06:52.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.063+0000 7f231881d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ef1c0 0x5613726e77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.063+0000 7f231881d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5613725f5ad0 con 0x5613725ef1c0 2024-05-08T18:06:52.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.063+0000 7f2316315640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ef1c0 0x5613726e77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.065 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.063+0000 7f2316315640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ef1c0 0x5613726e77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46458/0 (socket says 172.21.15.50:46458) 2024-05-08T18:06:52.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.063+0000 7f2316315640 1 -- 172.21.15.50:0/1039387382 learned_addr learned my addr 172.21.15.50:0/1039387382 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.064+0000 7f2316315640 1 -- 172.21.15.50:0/1039387382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613726e7d00 con 0x5613725ef1c0 2024-05-08T18:06:52.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.064+0000 7f2316315640 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ef1c0 0x5613726e77c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2308000f60 tx=0x7f2308000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.064+0000 7f2202e05640 1 -- 172.21.15.50:0/1039387382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f230801a7c0 con 0x5613725ef1c0 2024-05-08T18:06:52.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.064+0000 7f2202e05640 1 -- 172.21.15.50:0/1039387382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f230801a960 con 0x5613725ef1c0 2024-05-08T18:06:52.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.064+0000 7f231881d800 1 -- 172.21.15.50:0/1039387382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5613726e7f90 con 0x5613725ef1c0 2024-05-08T18:06:52.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.064+0000 7f2202e05640 1 -- 172.21.15.50:0/1039387382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f230801ac40 con 0x5613725ef1c0 2024-05-08T18:06:52.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.064+0000 7f231881d800 1 -- 172.21.15.50:0/1039387382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613726eac40 con 0x5613725ef1c0 2024-05-08T18:06:52.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.065+0000 7f2202e05640 1 -- 172.21.15.50:0/1039387382 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2308017070 con 0x5613725ef1c0 2024-05-08T18:06:52.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.065+0000 7f2202e05640 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f21fc03bba0 0x7f21fc03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.065+0000 7f2202e05640 1 -- 172.21.15.50:0/1039387382 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f2308015020 con 0x5613725ef1c0 2024-05-08T18:06:52.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.065+0000 7f2315b14640 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f21fc03bba0 0x7f21fc03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.065+0000 7f231881d800 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5613725f2cd0 0x5613726eea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.065+0000 7f231881d800 1 -- 172.21.15.50:0/1039387382 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5613726eef80 con 0x5613725f2cd0 2024-05-08T18:06:52.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.065+0000 7f2316b16640 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5613725f2cd0 0x5613726eea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.066+0000 7f2315b14640 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f21fc03bba0 0x7f21fc03e060 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f230c054000 tx=0x7f230c04e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.066+0000 7f2316b16640 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5613725f2cd0 0x5613726eea00 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.066+0000 7f2316b16640 1 -- 172.21.15.50:0/1039387382 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f231000a4a0 con 0x5613725f2cd0 2024-05-08T18:06:52.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.066+0000 7f231881d800 1 -- 172.21.15.50:0/1039387382 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5613726ebfd0 con 0x5613725f2cd0 2024-05-08T18:06:52.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.066+0000 7f2316b16640 1 -- 172.21.15.50:0/1039387382 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f231000a4a0 con 0x5613725f2cd0 2024-05-08T18:06:52.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.066+0000 7f231881d800 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5613726f1290 0x5613726f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.066+0000 7f231881d800 1 -- 172.21.15.50:0/1039387382 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5613726f3d20 con 0x5613726f1290 2024-05-08T18:06:52.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.067+0000 7f2316315640 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5613726f1290 0x5613726f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.067+0000 7f2316315640 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5613726f1290 0x5613726f3680 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.067+0000 7f2316315640 1 -- 172.21.15.50:0/1039387382 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f230802e070 con 0x5613726f1290 2024-05-08T18:06:52.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.067+0000 7f231881d800 1 -- 172.21.15.50:0/1039387382 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5613726f0130 con 0x5613725f2cd0 2024-05-08T18:06:52.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.068+0000 7f2316b16640 1 -- 172.21.15.50:0/1039387382 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f231000a4a0 con 0x5613725f2cd0 2024-05-08T18:06:52.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.068+0000 7f231881d800 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5613726f5180 0x5613726f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.068+0000 7f231881d800 1 -- 172.21.15.50:0/1039387382 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5613726f7c70 con 0x5613726f5180 2024-05-08T18:06:52.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.068+0000 7f2315b14640 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5613726f5180 0x5613726f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.068+0000 7f2315b14640 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5613726f5180 0x5613726f7570 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.068+0000 7f2315b14640 1 -- 172.21.15.50:0/1039387382 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f230c05a040 con 0x5613726f5180 2024-05-08T18:06:52.091 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.069+0000 7f231881d800 1 Processor -- start 2024-05-08T18:06:52.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.069+0000 7f231881d800 1 -- start start 2024-05-08T18:06:52.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.069+0000 7f231881d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561372705fe0 0x561372706400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.069+0000 7f231881d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5613724e0330 con 0x561372705fe0 2024-05-08T18:06:52.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.069+0000 7f2316315640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561372705fe0 0x561372706400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.069+0000 7f2316315640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561372705fe0 0x561372706400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46474/0 (socket says 172.21.15.50:46474) 2024-05-08T18:06:52.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.069+0000 7f2316315640 1 -- 172.21.15.50:0/2422211559 learned_addr learned my addr 172.21.15.50:0/2422211559 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.069+0000 7f2316315640 1 -- 172.21.15.50:0/2422211559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561372706940 con 0x561372705fe0 2024-05-08T18:06:52.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.070+0000 7f2316315640 1 --2- 172.21.15.50:0/2422211559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561372705fe0 0x561372706400 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f23080174b0 tx=0x7f2308050510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.070+0000 7f21fbfff640 1 -- 172.21.15.50:0/2422211559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f230804a410 con 0x561372705fe0 2024-05-08T18:06:52.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.070+0000 7f21fbfff640 1 -- 172.21.15.50:0/2422211559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f230801da10 con 0x561372705fe0 2024-05-08T18:06:52.092 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.070+0000 7f231881d800 1 -- 172.21.15.50:0/2422211559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5613727073b0 con 0x561372705fe0 2024-05-08T18:06:52.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.070+0000 7f21fbfff640 1 -- 172.21.15.50:0/2422211559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f230801dcf0 con 0x561372705fe0 2024-05-08T18:06:52.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.070+0000 7f231881d800 1 -- 172.21.15.50:0/2422211559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613727078b0 con 0x561372705fe0 2024-05-08T18:06:52.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.070+0000 7f21fbfff640 1 -- 172.21.15.50:0/2422211559 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f23080518e0 con 0x561372705fe0 2024-05-08T18:06:52.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.071+0000 7f21fbfff640 1 --2- 172.21.15.50:0/2422211559 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f21f003bba0 0x7f21f003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.071+0000 7f21fbfff640 1 -- 172.21.15.50:0/2422211559 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f230803e770 con 0x561372705fe0 2024-05-08T18:06:52.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.071+0000 7f2315b14640 1 --2- 172.21.15.50:0/2422211559 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f21f003bba0 0x7f21f003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.071+0000 7f231881d800 1 Processor -- start 2024-05-08T18:06:52.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.071+0000 7f231881d800 1 -- start start 2024-05-08T18:06:52.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.071+0000 7f2315b14640 1 --2- 172.21.15.50:0/2422211559 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f21f003bba0 0x7f21f003e060 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x5613726ebef0 tx=0x7f230c005ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.071+0000 7f231881d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ee830 0x5613725eec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.071+0000 7f231881d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5613726e0570 con 0x5613725ee830 2024-05-08T18:06:52.093 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.071+0000 7f2316315640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ee830 0x5613725eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.071+0000 7f2316315640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ee830 0x5613725eec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46480/0 (socket says 172.21.15.50:46480) 2024-05-08T18:06:52.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.071+0000 7f2316315640 1 -- 172.21.15.50:0/276271415 learned_addr learned my addr 172.21.15.50:0/276271415 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.072+0000 7f2316315640 1 -- 172.21.15.50:0/276271415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613725ed040 con 0x5613725ee830 2024-05-08T18:06:52.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.072+0000 7f2316315640 1 --2- 172.21.15.50:0/276271415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ee830 0x5613725eec50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2308041fd0 tx=0x7f2308043a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a77093d10233f6e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.072+0000 7f21f9ffb640 1 -- 172.21.15.50:0/276271415 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2308031cd0 con 0x5613725ee830 2024-05-08T18:06:52.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.072+0000 7f21f9ffb640 1 -- 172.21.15.50:0/276271415 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f230803ce50 con 0x5613725ee830 2024-05-08T18:06:52.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.072+0000 7f21f9ffb640 1 -- 172.21.15.50:0/276271415 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2308019860 con 0x5613725ee830 2024-05-08T18:06:52.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.072+0000 7f231881d800 1 -- 172.21.15.50:0/276271415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ee830 msgr2=0x5613725eec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.072+0000 7f231881d800 1 --2- 172.21.15.50:0/276271415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ee830 0x5613725eec50 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f2308041fd0 tx=0x7f2308043a70 comp rx=0 tx=0).stop 2024-05-08T18:06:52.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.072+0000 7f231881d800 1 -- 172.21.15.50:0/276271415 shutdown_connections 2024-05-08T18:06:52.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.072+0000 7f231881d800 1 --2- 172.21.15.50:0/276271415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ee830 0x5613725eec50 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.094 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.072+0000 7f231881d800 1 -- 172.21.15.50:0/276271415 >> 172.21.15.50:0/276271415 conn(0x56137270f3d0 msgr2=0x56137270d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.073+0000 7f231881d800 1 -- 172.21.15.50:0/276271415 shutdown_connections 2024-05-08T18:06:52.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.073+0000 7f231881d800 1 -- 172.21.15.50:0/276271415 wait complete. 2024-05-08T18:06:52.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.073+0000 7f231881d800 1 Processor -- start 2024-05-08T18:06:52.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.073+0000 7f231881d800 1 -- start start 2024-05-08T18:06:52.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.073+0000 7f231881d800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ee830 0x56137271c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.073+0000 7f231881d800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5613725ed450 con 0x5613725ee830 2024-05-08T18:06:52.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.073+0000 7f2316315640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ee830 0x56137271c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.073+0000 7f2316315640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ee830 0x56137271c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46482/0 (socket says 172.21.15.50:46482) 2024-05-08T18:06:52.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.073+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 learned_addr learned my addr 172.21.15.50:0/3841305823 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.073+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56137271cb90 con 0x5613725ee830 2024-05-08T18:06:52.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.074+0000 7f2316315640 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ee830 0x56137271c650 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f230803ba20 tx=0x7f230804dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.074+0000 7f21e7fff640 1 -- 172.21.15.50:0/3841305823 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2308041430 con 0x5613725ee830 2024-05-08T18:06:52.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.074+0000 7f21e7fff640 1 -- 172.21.15.50:0/3841305823 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f230803e0d0 con 0x5613725ee830 2024-05-08T18:06:52.095 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.074+0000 7f21e7fff640 1 -- 172.21.15.50:0/3841305823 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f230803e3f0 con 0x5613725ee830 2024-05-08T18:06:52.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.074+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56137271cb90 con 0x5613725ee830 2024-05-08T18:06:52.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.074+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56137271cb90 con 0x5613725ee830 2024-05-08T18:06:52.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.075+0000 7f21e7fff640 1 -- 172.21.15.50:0/3841305823 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f230806d060 con 0x5613725ee830 2024-05-08T18:06:52.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.075+0000 7f21e7fff640 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f21e003bba0 0x7f21e003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.075+0000 7f21e7fff640 1 -- 172.21.15.50:0/3841305823 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f2308067d10 con 0x5613725ee830 2024-05-08T18:06:52.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.075+0000 7f2315b14640 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f21e003bba0 0x7f21e003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.076+0000 7f2315b14640 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f21e003bba0 0x7f21e003e060 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f230c01d050 tx=0x7f230c04f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.076+0000 7f231881d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.076+0000 7f231881d800 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561372727c20 0x56137272a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.076+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56137272a830 con 0x561372727c20 2024-05-08T18:06:52.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.076+0000 7f2316b16640 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561372727c20 0x56137272a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.076+0000 7f2316b16640 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561372727c20 0x56137272a0e0 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.076+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f231000a4a0 con 0x561372727c20 2024-05-08T18:06:52.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.076+0000 7f231881d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.096 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.076+0000 7f231881d800 20 realm 2024-05-08T18:06:52.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.076+0000 7f231881d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.076+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56137272ac40 con 0x561372727c20 2024-05-08T18:06:52.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.077+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f231000a4a0 con 0x561372727c20 2024-05-08T18:06:52.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.077+0000 7f231881d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.077+0000 7f231881d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:52.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.077+0000 7f231881d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.077+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56137272b510 con 0x561372727c20 2024-05-08T18:06:52.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.077+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f231000a4a0 con 0x561372727c20 2024-05-08T18:06:52.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.077+0000 7f231881d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.077+0000 7f231881d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.077+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56137272b8b0 con 0x561372727c20 2024-05-08T18:06:52.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.077+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f231000a4a0 con 0x561372727c20 2024-05-08T18:06:52.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.077+0000 7f231881d800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:52.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.077+0000 7f231881d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.097 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.077+0000 7f231881d800 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56137272c610 0x56137272ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.077+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56137272f010 con 0x56137272c610 2024-05-08T18:06:52.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.078+0000 7f2316315640 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56137272c610 0x56137272ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.078+0000 7f2316315640 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56137272c610 0x56137272ea00 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.078+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.078+0000 7f231881d800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:52.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.078+0000 7f231881d800 20 searching for the correct realm 2024-05-08T18:06:52.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.078+0000 7f231881d800 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x561372730fc0 0x561372733480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.078+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x561372733b50 con 0x561372730fc0 2024-05-08T18:06:52.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.078+0000 7f2315b14640 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x561372730fc0 0x561372733480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.079+0000 7f2315b14640 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x561372730fc0 0x561372733480 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.079+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f230c07f070 con 0x561372730fc0 2024-05-08T18:06:52.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.079+0000 7f2315b14640 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f230c074040 0x7f230c081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.098 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.079+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x561372733b50 con 0x7f230c074040 2024-05-08T18:06:52.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.079+0000 7f2316b16640 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f230c074040 0x7f230c081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.079+0000 7f2316b16640 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f230c074040 0x7f230c081410 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.080+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f231000a4a0 con 0x7f230c074040 2024-05-08T18:06:52.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.080+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56137272b8b0 con 0x56137272c610 2024-05-08T18:06:52.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.080+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.080+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56137272b8b0 con 0x561372727c20 2024-05-08T18:06:52.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.080+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f231000a4a0 con 0x561372727c20 2024-05-08T18:06:52.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.080+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56137272b8b0 con 0x561372727c20 2024-05-08T18:06:52.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.080+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f231000a4a0 con 0x561372727c20 2024-05-08T18:06:52.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.080+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56137272b8b0 con 0x561372727c20 2024-05-08T18:06:52.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.081+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f231000a4a0 con 0x561372727c20 2024-05-08T18:06:52.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.081+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56137272b8b0 con 0x56137272c610 2024-05-08T18:06:52.099 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.081+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v29'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.081+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230807a080 con 0x7f230c074040 2024-05-08T18:06:52.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.081+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f231000a4a0 con 0x7f230c074040 2024-05-08T18:06:52.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.081+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230807a080 con 0x561372730fc0 2024-05-08T18:06:52.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.082+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f230c07f070 con 0x561372730fc0 2024-05-08T18:06:52.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.082+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230807a080 con 0x561372727c20 2024-05-08T18:06:52.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.082+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f231000a4a0 con 0x561372727c20 2024-05-08T18:06:52.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.082+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230807a080 con 0x56137272c610 2024-05-08T18:06:52.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.082+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.082+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230807a080 con 0x561372727c20 2024-05-08T18:06:52.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.082+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f231000a4a0 con 0x561372727c20 2024-05-08T18:06:52.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.082+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230807a080 con 0x7f230c074040 2024-05-08T18:06:52.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.083+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2310032040 con 0x7f230c074040 2024-05-08T18:06:52.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.083+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230807a080 con 0x561372730fc0 2024-05-08T18:06:52.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.083+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f230c07f070 con 0x561372730fc0 2024-05-08T18:06:52.100 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.083+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x561372730fc0 2024-05-08T18:06:52.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.083+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f230c07f070 con 0x561372730fc0 2024-05-08T18:06:52.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.083+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x56137272c610 2024-05-08T18:06:52.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.084+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.084+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x561372730fc0 2024-05-08T18:06:52.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.084+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f230c07f070 con 0x561372730fc0 2024-05-08T18:06:52.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.084+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x561372730fc0 2024-05-08T18:06:52.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.084+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f230c07f070 con 0x561372730fc0 2024-05-08T18:06:52.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.084+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x561372727c20 2024-05-08T18:06:52.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.085+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.085+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x561372727c20 2024-05-08T18:06:52.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.085+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v29'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.085+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x56137272c610 2024-05-08T18:06:52.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.085+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.101 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.085+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x561372730fc0 2024-05-08T18:06:52.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.086+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f230c07f070 con 0x561372730fc0 2024-05-08T18:06:52.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.086+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x7f230c074040 2024-05-08T18:06:52.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.086+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2310032040 con 0x7f230c074040 2024-05-08T18:06:52.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.086+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x561372727c20 2024-05-08T18:06:52.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.086+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.086+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x56137272c610 2024-05-08T18:06:52.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.086+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.102 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.086+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x561372730fc0 2024-05-08T18:06:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.087+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f230c07f070 con 0x561372730fc0 2024-05-08T18:06:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.087+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x56137272c610 2024-05-08T18:06:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.087+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.087+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x7f230c074040 2024-05-08T18:06:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.087+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2310032040 con 0x7f230c074040 2024-05-08T18:06:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.087+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x561372730fc0 2024-05-08T18:06:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.087+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f230c07f070 con 0x561372730fc0 2024-05-08T18:06:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.087+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x56137272c610 2024-05-08T18:06:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.088+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.103 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.088+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x561372727c20 2024-05-08T18:06:52.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.088+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v29'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.088+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f230c08e040 con 0x7f230c074040 2024-05-08T18:06:52.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.088+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2310032040 con 0x7f230c074040 2024-05-08T18:06:52.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.088+0000 7f231881d800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:52.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.088+0000 7f231881d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:52.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.088+0000 7f231881d800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:52.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.088+0000 7f231881d800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:52.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.088+0000 7f231881d800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:52.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.088+0000 7f231881d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:52.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.088+0000 7f231881d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.088+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56137272f8d0 con 0x561372727c20 2024-05-08T18:06:52.104 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f231881d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f231881d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56137272f8d0 con 0x561372727c20 2024-05-08T18:06:52.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f231881d800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:52.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f231881d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56137272f8d0 con 0x561372730fc0 2024-05-08T18:06:52.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f230c07f070 con 0x561372730fc0 2024-05-08T18:06:52.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f231881d800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:52.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f231881d800 20 zone default found 2024-05-08T18:06:52.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f231881d800 4 Realm: () 2024-05-08T18:06:52.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f231881d800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:52.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f231881d800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:52.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f231881d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:52.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f231881d800 20 zonegroup default 2024-05-08T18:06:52.105 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f231881d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x561372730210 con 0x56137272c610 2024-05-08T18:06:52.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.089+0000 7f231881d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.090+0000 7f231881d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.090+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5613727359e0 con 0x561372730fc0 2024-05-08T18:06:52.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.090+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f230c07f070 con 0x561372730fc0 2024-05-08T18:06:52.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.090+0000 7f231881d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.090+0000 7f231881d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.090+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x561372735e90 con 0x561372730fc0 2024-05-08T18:06:52.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.090+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f230c07f070 con 0x561372730fc0 2024-05-08T18:06:52.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.090+0000 7f231881d800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.090+0000 7f231881d800 20 started sync module instance, tier type = 2024-05-08T18:06:52.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.090+0000 7f231881d800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:52.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.090+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372734e50 con 0x7f230c074040 2024-05-08T18:06:52.106 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.091+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2310032040 con 0x7f230c074040 2024-05-08T18:06:52.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.091+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94641524527952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372734e50 con 0x7f230c074040 2024-05-08T18:06:52.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.091+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372736970 con 0x561372727c20 2024-05-08T18:06:52.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.092+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94641524527952] v29'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2310032040 con 0x7f230c074040 2024-05-08T18:06:52.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.093+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.093+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94641524532320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372734e50 con 0x561372727c20 2024-05-08T18:06:52.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.093+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372737430 con 0x561372730fc0 2024-05-08T18:06:52.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.094+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94641524532320] v29'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.094+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f230c07f070 con 0x561372730fc0 2024-05-08T18:06:52.112 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.094+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94641524525472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372734dc0 con 0x561372730fc0 2024-05-08T18:06:52.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.094+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5613727382f0 con 0x7f230c074040 2024-05-08T18:06:52.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.095+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94641524525472] v29'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f230c07f070 con 0x561372730fc0 2024-05-08T18:06:52.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.095+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2310032040 con 0x7f230c074040 2024-05-08T18:06:52.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.095+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94641524538944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372738260 con 0x7f230c074040 2024-05-08T18:06:52.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.095+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372739060 con 0x561372727c20 2024-05-08T18:06:52.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.096+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94641524538944] v29'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2310032040 con 0x7f230c074040 2024-05-08T18:06:52.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.096+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.096+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94641524541008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372739060 con 0x561372727c20 2024-05-08T18:06:52.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.096+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372739a70 con 0x56137272c610 2024-05-08T18:06:52.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.097+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94641524541008] v29'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.097+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.097+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94641524544960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372739e60 con 0x56137272c610 2024-05-08T18:06:52.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.097+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56137273a860 con 0x56137272c610 2024-05-08T18:06:52.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.098+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94641524544960] v29'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.114 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.098+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.098+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94641524542064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56137273a470 con 0x56137272c610 2024-05-08T18:06:52.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.098+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56137273b600 con 0x56137272c610 2024-05-08T18:06:52.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.099+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94641524542064] v29'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.100+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.100+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94641524550208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56137273b5c0 con 0x56137272c610 2024-05-08T18:06:52.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.100+0000 7f231881d800 20 add_watcher() i=0 2024-05-08T18:06:52.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.100+0000 7f231881d800 20 add_watcher() i=1 2024-05-08T18:06:52.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.100+0000 7f231881d800 20 add_watcher() i=2 2024-05-08T18:06:52.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.100+0000 7f231881d800 20 add_watcher() i=3 2024-05-08T18:06:52.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.100+0000 7f231881d800 20 add_watcher() i=4 2024-05-08T18:06:52.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.100+0000 7f231881d800 20 add_watcher() i=5 2024-05-08T18:06:52.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.100+0000 7f231881d800 20 add_watcher() i=6 2024-05-08T18:06:52.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.101+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94641524550208] v29'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.101+0000 7f231881d800 20 add_watcher() i=7 2024-05-08T18:06:52.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.101+0000 7f231881d800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.101+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56137273ea80 con 0x561372727c20 2024-05-08T18:06:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.101+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.101+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94641524552416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56137273bf00 con 0x561372727c20 2024-05-08T18:06:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.102+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94641524552416] v29'257 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.102+0000 7f21ccfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.102+0000 7f231881d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.102+0000 7f231881d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.102+0000 7f231881d800 5 note: GC not initialized 2024-05-08T18:06:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.103+0000 7f21477fe640 20 reqs_thread_entry: start 2024-05-08T18:06:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.103+0000 7f231881d800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.103+0000 7f231881d800 20 Filter name: none 2024-05-08T18:06:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.103+0000 7f21457fa640 20 reqs_thread_entry: start 2024-05-08T18:06:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.103+0000 7f231881d800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-05-08T18:06:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.103+0000 7f231881d800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.103+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x561372771ff0 con 0x56137272c610 2024-05-08T18:06:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.103+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(62 ada [call out=48b,read 0~460 out=460b,getxattrs out=75b] v0'0 uv7 ondisk = 0) ==== 231+0+583 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.103+0000 7f231881d800 20 rados_obj.operate() r=0 bl.length=460 2024-05-08T18:06:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.103+0000 7f231881d800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-05-08T18:06:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.103+0000 7f231881d800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.103+0000 7f231881d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.103+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~490 in=490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372778660 con 0x56137272c610 2024-05-08T18:06:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.105+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~490] v29'8 uv8 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.105+0000 7f231881d800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.105+0000 7f231881d800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.105+0000 7f231881d800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-05-08T18:06:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.105+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94641524801568 in=678b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x561372773c60 con 0x561372727c20 2024-05-08T18:06:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.105+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== watch-notify(notify (1) cookie 94641524541008 notify 124554051620 ret 0) ==== 708+0+0 (crc 0 0 0) 0x7f231003d040 con 0x561372727c20 2024-05-08T18:06:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.105+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 notify.4 [notify cookie 94641524801568 out=8b] v0'0 uv258 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.105+0000 7f21f9ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051620 cookie 94641524541008 notifier 4865 bl.length()=666 2024-05-08T18:06:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.105+0000 7f21f9ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:52.117 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.105+0000 7f21f9ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.105+0000 7f21f9ffb640 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f21dc003ea0 con 0x561372727c20 2024-05-08T18:06:52.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.105+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== watch-notify(notify_complete (2) cookie 94641524801568 notify 124554051620 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f2310018ba0 con 0x561372727c20 2024-05-08T18:06:52.118 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.105+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.118 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-05-08T18:06:51.092020Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"user-info-without-keys","perm":"read"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:06:51.092006Z","tags":[],"group_ids":[]} 2024-05-08T18:06:52.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.106+0000 7f21ccfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:52.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.106+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94641524552416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5613727737b0 con 0x561372727c20 2024-05-08T18:06:52.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.107+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94641524552416] v29'258 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.107+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94641524527952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56137273b2b0 con 0x7f230c074040 2024-05-08T18:06:52.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.108+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94641524527952] v29'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2310032040 con 0x7f230c074040 2024-05-08T18:06:52.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.108+0000 7f231881d800 20 remove_watcher() i=0 2024-05-08T18:06:52.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.108+0000 7f231881d800 2 removed watcher, disabling cache 2024-05-08T18:06:52.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.108+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94641524532320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372735350 con 0x561372727c20 2024-05-08T18:06:52.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.109+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94641524532320] v29'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.109+0000 7f231881d800 20 remove_watcher() i=1 2024-05-08T18:06:52.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.109+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94641524525472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372735350 con 0x561372730fc0 2024-05-08T18:06:52.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.110+0000 7f2315b14640 1 -- 172.21.15.50:0/3841305823 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94641524525472] v29'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f230c07f070 con 0x561372730fc0 2024-05-08T18:06:52.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.110+0000 7f231881d800 20 remove_watcher() i=2 2024-05-08T18:06:52.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.110+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94641524538944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5613727349a0 con 0x7f230c074040 2024-05-08T18:06:52.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.111+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94641524538944] v29'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2310032040 con 0x7f230c074040 2024-05-08T18:06:52.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.111+0000 7f231881d800 20 remove_watcher() i=3 2024-05-08T18:06:52.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.111+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94641524541008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372735350 con 0x561372727c20 2024-05-08T18:06:52.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.112+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94641524541008] v29'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2310032040 con 0x561372727c20 2024-05-08T18:06:52.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.112+0000 7f231881d800 20 remove_watcher() i=4 2024-05-08T18:06:52.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.112+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94641524544960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372737e40 con 0x56137272c610 2024-05-08T18:06:52.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.113+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94641524544960] v29'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.114+0000 7f231881d800 20 remove_watcher() i=5 2024-05-08T18:06:52.126 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.114+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94641524542064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5613727349a0 con 0x56137272c610 2024-05-08T18:06:52.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.114+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94641524542064] v29'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.114+0000 7f231881d800 20 remove_watcher() i=6 2024-05-08T18:06:52.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.114+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94641524550208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x561372735350 con 0x56137272c610 2024-05-08T18:06:52.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f2316315640 1 -- 172.21.15.50:0/3841305823 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94641524550208] v29'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f230802e070 con 0x56137272c610 2024-05-08T18:06:52.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 20 remove_watcher() i=7 2024-05-08T18:06:52.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56137272c610 msgr2=0x56137272ea00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56137272c610 0x56137272ea00 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561372727c20 msgr2=0x56137272a0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x561372727c20 0x56137272a0e0 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f230c074040 msgr2=0x7f230c081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f230c074040 0x7f230c081410 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x561372730fc0 msgr2=0x561372733480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x561372730fc0 0x561372733480 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f21e003bba0 msgr2=0x7f21e003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f21e003bba0 0x7f21e003e060 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f230c01d050 tx=0x7f230c04f000 comp rx=0 tx=0).stop 2024-05-08T18:06:52.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ee830 msgr2=0x56137271c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ee830 0x56137271c650 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f230803ba20 tx=0x7f230804dfc0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f2316b16640 1 -- 172.21.15.50:0/3841305823 reap_dead start 2024-05-08T18:06:52.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 shutdown_connections 2024-05-08T18:06:52.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 --2- 172.21.15.50:0/3841305823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ee830 0x56137271c650 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 >> 172.21.15.50:0/3841305823 conn(0x56137270f3d0 msgr2=0x56137270ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 shutdown_connections 2024-05-08T18:06:52.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 -- 172.21.15.50:0/3841305823 wait complete. 2024-05-08T18:06:52.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 -- 172.21.15.50:0/2422211559 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f21f003bba0 msgr2=0x7f21f003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 --2- 172.21.15.50:0/2422211559 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f21f003bba0 0x7f21f003e060 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x5613726ebef0 tx=0x7f230c005ac0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 -- 172.21.15.50:0/2422211559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561372705fe0 msgr2=0x561372706400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 --2- 172.21.15.50:0/2422211559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561372705fe0 0x561372706400 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f23080174b0 tx=0x7f2308050510 comp rx=0 tx=0).stop 2024-05-08T18:06:52.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 -- 172.21.15.50:0/2422211559 shutdown_connections 2024-05-08T18:06:52.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 --2- 172.21.15.50:0/2422211559 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f21f003bba0 0x7f21f003e060 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 --2- 172.21.15.50:0/2422211559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561372705fe0 0x561372706400 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 -- 172.21.15.50:0/2422211559 >> 172.21.15.50:0/2422211559 conn(0x5613726fa080 msgr2=0x5613726fa480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 -- 172.21.15.50:0/2422211559 shutdown_connections 2024-05-08T18:06:52.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.116+0000 7f231881d800 1 -- 172.21.15.50:0/2422211559 wait complete. 2024-05-08T18:06:52.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.117+0000 7f231881d800 1 -- 172.21.15.50:0/1039387382 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5613726f1290 msgr2=0x5613726f3680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.117+0000 7f231881d800 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5613726f1290 0x5613726f3680 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.117+0000 7f231881d800 1 -- 172.21.15.50:0/1039387382 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5613725f2cd0 msgr2=0x5613726eea00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.117+0000 7f231881d800 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5613725f2cd0 0x5613726eea00 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.117+0000 7f231881d800 1 -- 172.21.15.50:0/1039387382 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5613726f5180 msgr2=0x5613726f7570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.117+0000 7f231881d800 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5613726f5180 0x5613726f7570 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.117+0000 7f231881d800 1 -- 172.21.15.50:0/1039387382 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f21fc03bba0 msgr2=0x7f21fc03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.117+0000 7f231881d800 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f21fc03bba0 0x7f21fc03e060 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f230c054000 tx=0x7f230c04e000 comp rx=0 tx=0).stop 2024-05-08T18:06:52.129 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.117+0000 7f231881d800 1 -- 172.21.15.50:0/1039387382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ef1c0 msgr2=0x5613726e77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.117+0000 7f231881d800 1 --2- 172.21.15.50:0/1039387382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5613725ef1c0 0x5613726e77c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f2308000f60 tx=0x7f2308000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.117+0000 7f2316b16640 1 -- 172.21.15.50:0/1039387382 reap_dead start 2024-05-08T18:06:52.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.117+0000 7f231881d800 1 -- 172.21.15.50:0/1039387382 shutdown_connections 2024-05-08T18:06:52.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.117+0000 7f231881d800 1 -- 172.21.15.50:0/1039387382 >> 172.21.15.50:0/1039387382 conn(0x5613725e8af0 msgr2=0x5613726d9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.117+0000 7f231881d800 1 -- 172.21.15.50:0/1039387382 shutdown_connections 2024-05-08T18:06:52.130 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.117+0000 7f231881d800 1 -- 172.21.15.50:0/1039387382 wait complete. 2024-05-08T18:06:52.130 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': 'read'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:52.130 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': 'read'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:52.130 ERROR:root:0 2024-05-08T18:06:52.131 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:52.135 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:52.136 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=*'] 2024-05-08T18:06:52.136 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=*'] 2024-05-08T18:06:52.136 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps 'user-info-without-keys=*' 2024-05-08T18:06:52.183 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:52.184 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:52.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.186+0000 7fadbcf2f800 1 Processor -- start 2024-05-08T18:06:52.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.186+0000 7fadbcf2f800 1 -- start start 2024-05-08T18:06:52.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.186+0000 7fadbcf2f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55673bfa2b00 0x55673bfa4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.186+0000 7fadbcf2f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55673bfa54c0 con 0x55673bf9e710 2024-05-08T18:06:52.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.186+0000 7fadbcf2f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x55673be3ee20 con 0x55673bfa2b00 2024-05-08T18:06:52.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.187+0000 7fadb83c9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55673bfa2b00 0x55673bfa4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.187+0000 7fadb8bca640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x55673bf9e710 0x55673bf9eb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50872/0 (socket says 172.21.15.50:50872) 2024-05-08T18:06:52.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.187+0000 7fadb8bca640 1 -- 172.21.15.50:0/2859261723 learned_addr learned my addr 172.21.15.50:0/2859261723 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:52.185 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.187+0000 7fadb83c9640 1 -- 172.21.15.50:0/2859261723 >> v1:172.21.15.46:6789/0 conn(0x55673bf9e710 legacy=0x55673bf9eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.186 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.187+0000 7fadb83c9640 1 -- 172.21.15.50:0/2859261723 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55673bfa66f0 con 0x55673bfa2b00 2024-05-08T18:06:52.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.189+0000 7fadb83c9640 1 --2- 172.21.15.50:0/2859261723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bfa2b00 0x55673bfa4ef0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fadac00a470 tx=0x7fadac01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2e9d418bf5bbc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.189+0000 7fadb7bc8640 1 -- 172.21.15.50:0/2859261723 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadac057070 con 0x55673bfa2b00 2024-05-08T18:06:52.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.189+0000 7fadb7bc8640 1 -- 172.21.15.50:0/2859261723 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadac052570 con 0x55673bfa2b00 2024-05-08T18:06:52.187 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.189+0000 7fadb7bc8640 1 -- 172.21.15.50:0/2859261723 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadac052870 con 0x55673bfa2b00 2024-05-08T18:06:52.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.190+0000 7fadbcf2f800 1 -- 172.21.15.50:0/2859261723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bfa2b00 msgr2=0x55673bfa4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.190+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/2859261723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bfa2b00 0x55673bfa4ef0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fadac00a470 tx=0x7fadac01ce90 comp rx=0 tx=0).stop 2024-05-08T18:06:52.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.190+0000 7fadbcf2f800 1 -- 172.21.15.50:0/2859261723 shutdown_connections 2024-05-08T18:06:52.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.190+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/2859261723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bfa2b00 0x55673bfa4ef0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.190+0000 7fadbcf2f800 1 -- 172.21.15.50:0/2859261723 >> 172.21.15.50:0/2859261723 conn(0x55673bf99c90 msgr2=0x55673bf9c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.190+0000 7fadbcf2f800 1 -- 172.21.15.50:0/2859261723 shutdown_connections 2024-05-08T18:06:52.188 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.190+0000 7fadbcf2f800 1 -- 172.21.15.50:0/2859261723 wait complete. 2024-05-08T18:06:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.205+0000 7fadbcf2f800 1 Processor -- start 2024-05-08T18:06:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.205+0000 7fadbcf2f800 1 -- start start 2024-05-08T18:06:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.205+0000 7fadbcf2f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bf9e710 0x55673bf13f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.205+0000 7fadbcf2f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55673bee9a30 con 0x55673bf9e710 2024-05-08T18:06:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.206+0000 7fadb8bca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bf9e710 0x55673bf13f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.206+0000 7fadb8bca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bf9e710 0x55673bf13f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46488/0 (socket says 172.21.15.50:46488) 2024-05-08T18:06:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.206+0000 7fadb8bca640 1 -- 172.21.15.50:0/4023088691 learned_addr learned my addr 172.21.15.50:0/4023088691 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.204 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.206+0000 7fadb8bca640 1 -- 172.21.15.50:0/4023088691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55673bf124c0 con 0x55673bf9e710 2024-05-08T18:06:52.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.206+0000 7fadb8bca640 1 --2- 172.21.15.50:0/4023088691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bf9e710 0x55673bf13f00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fada8001e40 tx=0x7fada8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5283f2955f6f373 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.206+0000 7faca6e0d640 1 -- 172.21.15.50:0/4023088691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada8012020 con 0x55673bf9e710 2024-05-08T18:06:52.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.206+0000 7faca6e0d640 1 -- 172.21.15.50:0/4023088691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada800e5a0 con 0x55673bf9e710 2024-05-08T18:06:52.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.206+0000 7faca6e0d640 1 -- 172.21.15.50:0/4023088691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada800e8a0 con 0x55673bf9e710 2024-05-08T18:06:52.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.207+0000 7fadbcf2f800 1 -- 172.21.15.50:0/4023088691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bf9e710 msgr2=0x55673bf13f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.207+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/4023088691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bf9e710 0x55673bf13f00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fada8001e40 tx=0x7fada8001e70 comp rx=0 tx=0).stop 2024-05-08T18:06:52.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.207+0000 7fadbcf2f800 1 -- 172.21.15.50:0/4023088691 shutdown_connections 2024-05-08T18:06:52.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.207+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/4023088691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bf9e710 0x55673bf13f00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.213 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.207+0000 7fadbcf2f800 1 -- 172.21.15.50:0/4023088691 >> 172.21.15.50:0/4023088691 conn(0x55673bf99c90 msgr2=0x55673c08b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.207+0000 7fadbcf2f800 1 -- 172.21.15.50:0/4023088691 shutdown_connections 2024-05-08T18:06:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.207+0000 7fadbcf2f800 1 -- 172.21.15.50:0/4023088691 wait complete. 2024-05-08T18:06:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.208+0000 7fadbcf2f800 1 Processor -- start 2024-05-08T18:06:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.208+0000 7fadbcf2f800 1 -- start start 2024-05-08T18:06:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.208+0000 7fadbcf2f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bf9e710 0x55673c099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.208+0000 7fadbcf2f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55673bfa5990 con 0x55673bf9e710 2024-05-08T18:06:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.208+0000 7fadb8bca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bf9e710 0x55673c099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.208+0000 7fadb8bca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bf9e710 0x55673c099980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46498/0 (socket says 172.21.15.50:46498) 2024-05-08T18:06:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.208+0000 7fadb8bca640 1 -- 172.21.15.50:0/1630509441 learned_addr learned my addr 172.21.15.50:0/1630509441 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.208+0000 7fadb8bca640 1 -- 172.21.15.50:0/1630509441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55673c099ec0 con 0x55673bf9e710 2024-05-08T18:06:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.208+0000 7fadb8bca640 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bf9e710 0x55673c099980 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fada8000c00 tx=0x7fada8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.209+0000 7faca560a640 1 -- 172.21.15.50:0/1630509441 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada8012030 con 0x55673bf9e710 2024-05-08T18:06:52.214 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.209+0000 7faca560a640 1 -- 172.21.15.50:0/1630509441 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada800ee50 con 0x55673bf9e710 2024-05-08T18:06:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.209+0000 7fadbcf2f800 1 -- 172.21.15.50:0/1630509441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55673c09a150 con 0x55673bf9e710 2024-05-08T18:06:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.209+0000 7faca560a640 1 -- 172.21.15.50:0/1630509441 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada801a940 con 0x55673bf9e710 2024-05-08T18:06:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.209+0000 7fadbcf2f800 1 -- 172.21.15.50:0/1630509441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55673c09cdb0 con 0x55673bf9e710 2024-05-08T18:06:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.209+0000 7faca560a640 1 -- 172.21.15.50:0/1630509441 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fada8015020 con 0x55673bf9e710 2024-05-08T18:06:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.210+0000 7faca560a640 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7faca003bb50 0x7faca003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.210+0000 7faca560a640 1 -- 172.21.15.50:0/1630509441 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fada8020030 con 0x55673bf9e710 2024-05-08T18:06:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.210+0000 7fadb83c9640 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7faca003bb50 0x7faca003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.210+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55673bfa2b00 0x55673c0a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.210+0000 7fadbcf2f800 1 -- 172.21.15.50:0/1630509441 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c0a1100 con 0x55673bfa2b00 2024-05-08T18:06:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.210+0000 7fadb93cb640 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55673bfa2b00 0x55673c0a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.210+0000 7fadb83c9640 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7faca003bb50 0x7faca003e010 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fadac054000 tx=0x7fadac04e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.210+0000 7fadb93cb640 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55673bfa2b00 0x55673c0a0b70 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.211+0000 7fadb93cb640 1 -- 172.21.15.50:0/1630509441 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fadb000a4a0 con 0x55673bfa2b00 2024-05-08T18:06:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.211+0000 7fadbcf2f800 1 -- 172.21.15.50:0/1630509441 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c09e140 con 0x55673bfa2b00 2024-05-08T18:06:52.215 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.211+0000 7fadb93cb640 1 -- 172.21.15.50:0/1630509441 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fadb000a4a0 con 0x55673bfa2b00 2024-05-08T18:06:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.211+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55673c0a3450 0x55673c0a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.211+0000 7fadbcf2f800 1 -- 172.21.15.50:0/1630509441 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c0a5f30 con 0x55673c0a3450 2024-05-08T18:06:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.211+0000 7fadb8bca640 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55673c0a3450 0x55673c0a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.212+0000 7fadb8bca640 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55673c0a3450 0x55673c0a5840 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.212+0000 7fadb8bca640 1 -- 172.21.15.50:0/1630509441 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fada803fd70 con 0x55673c0a3450 2024-05-08T18:06:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.212+0000 7fadbcf2f800 1 -- 172.21.15.50:0/1630509441 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c0a22f0 con 0x55673bfa2b00 2024-05-08T18:06:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.212+0000 7fadb93cb640 1 -- 172.21.15.50:0/1630509441 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fadb000a4a0 con 0x55673bfa2b00 2024-05-08T18:06:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.213+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55673c0a7390 0x55673c0a9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.213+0000 7fadbcf2f800 1 -- 172.21.15.50:0/1630509441 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c0a9e80 con 0x55673c0a7390 2024-05-08T18:06:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.213+0000 7fadb83c9640 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55673c0a7390 0x55673c0a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.213+0000 7fadb83c9640 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55673c0a7390 0x55673c0a9780 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.213+0000 7fadb83c9640 1 -- 172.21.15.50:0/1630509441 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fadac052940 con 0x55673c0a7390 2024-05-08T18:06:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.214+0000 7fadbcf2f800 1 Processor -- start 2024-05-08T18:06:52.216 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.214+0000 7fadbcf2f800 1 -- start start 2024-05-08T18:06:52.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.214+0000 7fadbcf2f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0b81f0 0x55673c0b8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.214+0000 7fadbcf2f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55673be90330 con 0x55673c0b81f0 2024-05-08T18:06:52.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.214+0000 7fadb8bca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0b81f0 0x55673c0b8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.214+0000 7fadb8bca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0b81f0 0x55673c0b8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46512/0 (socket says 172.21.15.50:46512) 2024-05-08T18:06:52.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.214+0000 7fadb8bca640 1 -- 172.21.15.50:0/423462255 learned_addr learned my addr 172.21.15.50:0/423462255 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.214+0000 7fadb8bca640 1 -- 172.21.15.50:0/423462255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55673c0b8b50 con 0x55673c0b81f0 2024-05-08T18:06:52.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.214+0000 7fadb8bca640 1 --2- 172.21.15.50:0/423462255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0b81f0 0x55673c0b8610 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fada80511b0 tx=0x7fada8043500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.214+0000 7fac9e7fc640 1 -- 172.21.15.50:0/423462255 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada8018c90 con 0x55673c0b81f0 2024-05-08T18:06:52.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.215+0000 7fac9e7fc640 1 -- 172.21.15.50:0/423462255 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada8047070 con 0x55673c0b81f0 2024-05-08T18:06:52.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.215+0000 7fadbcf2f800 1 -- 172.21.15.50:0/423462255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55673c0b95c0 con 0x55673c0b81f0 2024-05-08T18:06:52.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.215+0000 7fac9e7fc640 1 -- 172.21.15.50:0/423462255 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada803daf0 con 0x55673c0b81f0 2024-05-08T18:06:52.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.215+0000 7fadbcf2f800 1 -- 172.21.15.50:0/423462255 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55673c0b95c0 con 0x55673c0b81f0 2024-05-08T18:06:52.217 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.215+0000 7fac9e7fc640 1 -- 172.21.15.50:0/423462255 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fada803dc90 con 0x55673c0b81f0 2024-05-08T18:06:52.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.215+0000 7fac9e7fc640 1 --2- 172.21.15.50:0/423462255 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac9003bba0 0x7fac9003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.216+0000 7fac9e7fc640 1 -- 172.21.15.50:0/423462255 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fada8038d90 con 0x55673c0b81f0 2024-05-08T18:06:52.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.216+0000 7fadb83c9640 1 --2- 172.21.15.50:0/423462255 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac9003bba0 0x7fac9003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.234 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.216+0000 7fadbcf2f800 1 Processor -- start 2024-05-08T18:06:52.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.216+0000 7fadbcf2f800 1 -- start start 2024-05-08T18:06:52.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.216+0000 7fadb83c9640 1 --2- 172.21.15.50:0/423462255 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac9003bba0 0x7fac9003e060 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55673c0a1a00 tx=0x7fadac005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.216+0000 7fadbcf2f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0c5160 0x55673c0c5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.216+0000 7fadbcf2f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55673bf12840 con 0x55673c0c5160 2024-05-08T18:06:52.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.216+0000 7fadb8bca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0c5160 0x55673c0c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.216+0000 7fadb8bca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0c5160 0x55673c0c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46528/0 (socket says 172.21.15.50:46528) 2024-05-08T18:06:52.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.216+0000 7fadb8bca640 1 -- 172.21.15.50:0/2347367975 learned_addr learned my addr 172.21.15.50:0/2347367975 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.216+0000 7fadb8bca640 1 -- 172.21.15.50:0/2347367975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55673c0c5b50 con 0x55673c0c5160 2024-05-08T18:06:52.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.217+0000 7fadb8bca640 1 --2- 172.21.15.50:0/2347367975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0c5160 0x55673c0c5580 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fada8036fd0 tx=0x7fada80529f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c44fc5e7915ad77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.217+0000 7fac8ffff640 1 -- 172.21.15.50:0/2347367975 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada8056030 con 0x55673c0c5160 2024-05-08T18:06:52.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.217+0000 7fac8ffff640 1 -- 172.21.15.50:0/2347367975 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada804a350 con 0x55673c0c5160 2024-05-08T18:06:52.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.217+0000 7fac8ffff640 1 -- 172.21.15.50:0/2347367975 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada80522a0 con 0x55673c0c5160 2024-05-08T18:06:52.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.217+0000 7fadbcf2f800 1 -- 172.21.15.50:0/2347367975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0c5160 msgr2=0x55673c0c5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.217+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/2347367975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0c5160 0x55673c0c5580 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fada8036fd0 tx=0x7fada80529f0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.235 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.217+0000 7fadbcf2f800 1 -- 172.21.15.50:0/2347367975 shutdown_connections 2024-05-08T18:06:52.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.217+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/2347367975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0c5160 0x55673c0c5580 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.217+0000 7fadbcf2f800 1 -- 172.21.15.50:0/2347367975 >> 172.21.15.50:0/2347367975 conn(0x55673c0c14c0 msgr2=0x55673c0bf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.217+0000 7fadbcf2f800 1 -- 172.21.15.50:0/2347367975 shutdown_connections 2024-05-08T18:06:52.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.217+0000 7fadbcf2f800 1 -- 172.21.15.50:0/2347367975 wait complete. 2024-05-08T18:06:52.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.218+0000 7fadbcf2f800 1 Processor -- start 2024-05-08T18:06:52.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.218+0000 7fadbcf2f800 1 -- start start 2024-05-08T18:06:52.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.218+0000 7fadbcf2f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0c5160 0x55673c0cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.218+0000 7fadbcf2f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55673c0c6200 con 0x55673c0c5160 2024-05-08T18:06:52.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.218+0000 7fadb8bca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0c5160 0x55673c0cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.218+0000 7fadb8bca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0c5160 0x55673c0cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46536/0 (socket says 172.21.15.50:46536) 2024-05-08T18:06:52.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.218+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 learned_addr learned my addr 172.21.15.50:0/3804925659 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.218+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55673c0cca20 con 0x55673c0c5160 2024-05-08T18:06:52.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.218+0000 7fadb8bca640 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0c5160 0x55673c0cc4e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fada8039000 tx=0x7fada800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.219+0000 7fac8e7fc640 1 -- 172.21.15.50:0/3804925659 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada80227f0 con 0x55673c0c5160 2024-05-08T18:06:52.236 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.219+0000 7fac8e7fc640 1 -- 172.21.15.50:0/3804925659 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada8043620 con 0x55673c0c5160 2024-05-08T18:06:52.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.219+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55673c0cca20 con 0x55673c0c5160 2024-05-08T18:06:52.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.219+0000 7fac8e7fc640 1 -- 172.21.15.50:0/3804925659 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fada8043900 con 0x55673c0c5160 2024-05-08T18:06:52.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.219+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55673c0ccec0 con 0x55673c0c5160 2024-05-08T18:06:52.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.219+0000 7fac8e7fc640 1 -- 172.21.15.50:0/3804925659 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fada806a070 con 0x55673c0c5160 2024-05-08T18:06:52.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.220+0000 7fac8e7fc640 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac8403bba0 0x7fac8403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.220+0000 7fac8e7fc640 1 -- 172.21.15.50:0/3804925659 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fada8057070 con 0x55673c0c5160 2024-05-08T18:06:52.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.220+0000 7fadb83c9640 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac8403bba0 0x7fac8403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.220+0000 7fadb83c9640 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac8403bba0 0x7fac8403e060 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fadac052fd0 tx=0x7fadac04f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.220+0000 7fadbcf2f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.220+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55673c0d7a20 0x55673c0d9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.220+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c0da630 con 0x55673c0d7a20 2024-05-08T18:06:52.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.220+0000 7fadb93cb640 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55673c0d7a20 0x55673c0d9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.221+0000 7fadb93cb640 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55673c0d7a20 0x55673c0d9ee0 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.237 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.221+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fadb000a4a0 con 0x55673c0d7a20 2024-05-08T18:06:52.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.221+0000 7fadbcf2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.221+0000 7fadbcf2f800 20 realm 2024-05-08T18:06:52.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.221+0000 7fadbcf2f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.221+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c0daa40 con 0x55673c0d7a20 2024-05-08T18:06:52.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.221+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fadb000a4a0 con 0x55673c0d7a20 2024-05-08T18:06:52.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.221+0000 7fadbcf2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.221+0000 7fadbcf2f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:52.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.221+0000 7fadbcf2f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.221+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c0db310 con 0x55673c0d7a20 2024-05-08T18:06:52.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.222+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fadb000a4a0 con 0x55673c0d7a20 2024-05-08T18:06:52.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.222+0000 7fadbcf2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.222+0000 7fadbcf2f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.238 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.222+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c0db6b0 con 0x55673c0d7a20 2024-05-08T18:06:52.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.222+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fadb000a4a0 con 0x55673c0d7a20 2024-05-08T18:06:52.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.222+0000 7fadbcf2f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:52.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.222+0000 7fadbcf2f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.222+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55673c0dc410 0x55673c0de800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.222+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c0dee10 con 0x55673c0dc410 2024-05-08T18:06:52.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.222+0000 7fadb8bca640 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55673c0dc410 0x55673c0de800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.222+0000 7fadb8bca640 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55673c0dc410 0x55673c0de800 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.223+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.223+0000 7fadbcf2f800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:52.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.223+0000 7fadbcf2f800 20 searching for the correct realm 2024-05-08T18:06:52.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.223+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55673c0e0dc0 0x55673c0e3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.223+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e3950 con 0x55673c0e0dc0 2024-05-08T18:06:52.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.223+0000 7fadb83c9640 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55673c0e0dc0 0x55673c0e3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.223+0000 7fadb83c9640 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55673c0e0dc0 0x55673c0e3280 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.239 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.223+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadac081070 con 0x55673c0e0dc0 2024-05-08T18:06:52.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.223+0000 7fadb83c9640 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fadac076040 0x7fadac083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.223+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e3950 con 0x7fadac076040 2024-05-08T18:06:52.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.223+0000 7fadb93cb640 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fadac076040 0x7fadac083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.224+0000 7fadb93cb640 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fadac076040 0x7fadac083410 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.224+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadb000a4a0 con 0x7fadac076040 2024-05-08T18:06:52.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.224+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55673c0db6b0 con 0x55673c0dc410 2024-05-08T18:06:52.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.224+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.224+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55673c0db6b0 con 0x55673c0d7a20 2024-05-08T18:06:52.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.225+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadb000a4a0 con 0x55673c0d7a20 2024-05-08T18:06:52.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.225+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55673c0db6b0 con 0x55673c0d7a20 2024-05-08T18:06:52.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.225+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadb000a4a0 con 0x55673c0d7a20 2024-05-08T18:06:52.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.225+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55673c0db6b0 con 0x55673c0d7a20 2024-05-08T18:06:52.240 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.225+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadb000a4a0 con 0x55673c0d7a20 2024-05-08T18:06:52.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.225+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55673c0db6b0 con 0x55673c0dc410 2024-05-08T18:06:52.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.225+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v29'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.225+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55673c0db6b0 con 0x7fadac076040 2024-05-08T18:06:52.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.225+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadb000a4a0 con 0x7fadac076040 2024-05-08T18:06:52.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.226+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55673c0db6b0 con 0x55673c0e0dc0 2024-05-08T18:06:52.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.226+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadac081070 con 0x55673c0e0dc0 2024-05-08T18:06:52.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.226+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55673c0db6b0 con 0x55673c0d7a20 2024-05-08T18:06:52.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.226+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadb000a4a0 con 0x55673c0d7a20 2024-05-08T18:06:52.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.226+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55673c0db6b0 con 0x55673c0dc410 2024-05-08T18:06:52.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.227+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.227+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55673c0db6b0 con 0x55673c0d7a20 2024-05-08T18:06:52.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.227+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadb000a4a0 con 0x55673c0d7a20 2024-05-08T18:06:52.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.227+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55673c0db6b0 con 0x7fadac076040 2024-05-08T18:06:52.241 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.227+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fadb0032040 con 0x7fadac076040 2024-05-08T18:06:52.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.227+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55673c0db6b0 con 0x55673c0e0dc0 2024-05-08T18:06:52.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.227+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadac081070 con 0x55673c0e0dc0 2024-05-08T18:06:52.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.227+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x55673c0e0dc0 2024-05-08T18:06:52.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.228+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadac081070 con 0x55673c0e0dc0 2024-05-08T18:06:52.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.228+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x55673c0dc410 2024-05-08T18:06:52.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.228+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.228+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x55673c0e0dc0 2024-05-08T18:06:52.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.228+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadac081070 con 0x55673c0e0dc0 2024-05-08T18:06:52.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.228+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x55673c0e0dc0 2024-05-08T18:06:52.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.229+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fadac081070 con 0x55673c0e0dc0 2024-05-08T18:06:52.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.229+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x55673c0d7a20 2024-05-08T18:06:52.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.229+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.229+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x55673c0d7a20 2024-05-08T18:06:52.242 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.229+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v29'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.229+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x55673c0dc410 2024-05-08T18:06:52.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.229+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.229+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x55673c0e0dc0 2024-05-08T18:06:52.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.230+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadac081070 con 0x55673c0e0dc0 2024-05-08T18:06:52.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.230+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x7fadac076040 2024-05-08T18:06:52.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.230+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadb0032040 con 0x7fadac076040 2024-05-08T18:06:52.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.230+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x55673c0d7a20 2024-05-08T18:06:52.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.230+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.230+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x55673c0dc410 2024-05-08T18:06:52.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.230+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.230+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x55673c0e0dc0 2024-05-08T18:06:52.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.231+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadac081070 con 0x55673c0e0dc0 2024-05-08T18:06:52.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.231+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x55673c0dc410 2024-05-08T18:06:52.243 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.231+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.231+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x7fadac076040 2024-05-08T18:06:52.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.231+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadb0032040 con 0x7fadac076040 2024-05-08T18:06:52.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.231+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x55673c0e0dc0 2024-05-08T18:06:52.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.231+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadac081070 con 0x55673c0e0dc0 2024-05-08T18:06:52.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.231+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x55673c0dc410 2024-05-08T18:06:52.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.232+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.232+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x55673c0d7a20 2024-05-08T18:06:52.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.232+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v29'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.232+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7fadac090040 con 0x7fadac076040 2024-05-08T18:06:52.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.232+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fadb0032040 con 0x7fadac076040 2024-05-08T18:06:52.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.232+0000 7fadbcf2f800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:52.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.232+0000 7fadbcf2f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:52.244 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.232+0000 7fadbcf2f800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:52.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.232+0000 7fadbcf2f800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:52.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.232+0000 7fadbcf2f800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:52.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.232+0000 7fadbcf2f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:52.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.232+0000 7fadbcf2f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.232+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c0df6d0 con 0x55673c0d7a20 2024-05-08T18:06:52.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.232+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.232+0000 7fadbcf2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadbcf2f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c0df6d0 con 0x55673c0d7a20 2024-05-08T18:06:52.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadbcf2f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:52.245 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadbcf2f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c0df6d0 con 0x55673c0e0dc0 2024-05-08T18:06:52.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fadac081070 con 0x55673c0e0dc0 2024-05-08T18:06:52.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadbcf2f800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:52.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadbcf2f800 20 zone default found 2024-05-08T18:06:52.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadbcf2f800 4 Realm: () 2024-05-08T18:06:52.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadbcf2f800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:52.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadbcf2f800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:52.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadbcf2f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:52.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadbcf2f800 20 zonegroup default 2024-05-08T18:06:52.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadbcf2f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c0dffd0 con 0x55673c0dc410 2024-05-08T18:06:52.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.233+0000 7fadbcf2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.246 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.234+0000 7fadbcf2f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.234+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e57e0 con 0x55673c0e0dc0 2024-05-08T18:06:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.234+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fadac081070 con 0x55673c0e0dc0 2024-05-08T18:06:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.234+0000 7fadbcf2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.234+0000 7fadbcf2f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.234+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e5c70 con 0x55673c0e0dc0 2024-05-08T18:06:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.234+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fadac081070 con 0x55673c0e0dc0 2024-05-08T18:06:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.234+0000 7fadbcf2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.234+0000 7fadbcf2f800 20 started sync module instance, tier type = 2024-05-08T18:06:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.234+0000 7fadbcf2f800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.234+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e4c50 con 0x7fadac076040 2024-05-08T18:06:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.235+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadb0032040 con 0x7fadac076040 2024-05-08T18:06:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.235+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93901877563760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e4c50 con 0x7fadac076040 2024-05-08T18:06:52.247 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.235+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e66f0 con 0x55673c0d7a20 2024-05-08T18:06:52.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.236+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93901877563760] v29'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadb0032040 con 0x7fadac076040 2024-05-08T18:06:52.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.236+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.236+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93901877568544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e4c50 con 0x55673c0d7a20 2024-05-08T18:06:52.248 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.236+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e7160 con 0x55673c0e0dc0 2024-05-08T18:06:52.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.237+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93901877568544] v29'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.237+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadac081070 con 0x55673c0e0dc0 2024-05-08T18:06:52.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.237+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93901877561248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e4bc0 con 0x55673c0e0dc0 2024-05-08T18:06:52.253 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.238+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e8020 con 0x7fadac076040 2024-05-08T18:06:52.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.239+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadb0032040 con 0x7fadac076040 2024-05-08T18:06:52.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.239+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93901877561248] v29'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadac081070 con 0x55673c0e0dc0 2024-05-08T18:06:52.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.239+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93901877574512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e7f90 con 0x7fadac076040 2024-05-08T18:06:52.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.239+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e8de0 con 0x55673c0d7a20 2024-05-08T18:06:52.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.240+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93901877574512] v29'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadb0032040 con 0x7fadac076040 2024-05-08T18:06:52.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.240+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.240+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93901877576576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e8de0 con 0x55673c0d7a20 2024-05-08T18:06:52.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.240+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e97f0 con 0x55673c0dc410 2024-05-08T18:06:52.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.241+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93901877576576] v29'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.241+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.241+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93901877580608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e9be0 con 0x55673c0dc410 2024-05-08T18:06:52.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.241+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0ea5e0 con 0x55673c0dc410 2024-05-08T18:06:52.254 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.242+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93901877580608] v29'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.242+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.242+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93901877577632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0eae80 con 0x55673c0dc410 2024-05-08T18:06:52.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.242+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0eb840 con 0x55673c0dc410 2024-05-08T18:06:52.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.243+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93901877577632] v29'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.243+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.243+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93901877586864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0eb840 con 0x55673c0dc410 2024-05-08T18:06:52.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.243+0000 7fadbcf2f800 20 add_watcher() i=0 2024-05-08T18:06:52.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.243+0000 7fadbcf2f800 20 add_watcher() i=1 2024-05-08T18:06:52.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.243+0000 7fadbcf2f800 20 add_watcher() i=2 2024-05-08T18:06:52.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.243+0000 7fadbcf2f800 20 add_watcher() i=3 2024-05-08T18:06:52.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.243+0000 7fadbcf2f800 20 add_watcher() i=4 2024-05-08T18:06:52.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.243+0000 7fadbcf2f800 20 add_watcher() i=5 2024-05-08T18:06:52.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.243+0000 7fadbcf2f800 20 add_watcher() i=6 2024-05-08T18:06:52.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.244+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93901877586864] v29'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.255 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.244+0000 7fadbcf2f800 20 add_watcher() i=7 2024-05-08T18:06:52.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.244+0000 7fadbcf2f800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:52.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.244+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c0eeb50 con 0x55673c0d7a20 2024-05-08T18:06:52.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.244+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.244+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93901877589248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0ebdd0 con 0x55673c0d7a20 2024-05-08T18:06:52.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.245+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93901877589248] v29'259 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.246+0000 7fac6efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:52.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.246+0000 7fadbcf2f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:52.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.246+0000 7fadbcf2f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:52.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.246+0000 7fadbcf2f800 5 note: GC not initialized 2024-05-08T18:06:52.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.246+0000 7fac6dfe3640 20 reqs_thread_entry: start 2024-05-08T18:06:52.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.246+0000 7fadbcf2f800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:52.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.246+0000 7fadbcf2f800 20 Filter name: none 2024-05-08T18:06:52.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.246+0000 7fabdf7fe640 20 reqs_thread_entry: start 2024-05-08T18:06:52.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.246+0000 7fadbcf2f800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-05-08T18:06:52.256 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.246+0000 7fadbcf2f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.246+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c122110 con 0x55673c0dc410 2024-05-08T18:06:52.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.247+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(62 ada [call out=48b,read 0~490 out=490b,getxattrs out=75b] v0'0 uv8 ondisk = 0) ==== 231+0+613 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.247+0000 7fadbcf2f800 20 rados_obj.operate() r=0 bl.length=490 2024-05-08T18:06:52.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.247+0000 7fadbcf2f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-05-08T18:06:52.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.247+0000 7fadbcf2f800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.247+0000 7fadbcf2f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:52.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.247+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~490 in=490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c1288d0 con 0x55673c0dc410 2024-05-08T18:06:52.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.248+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~490] v29'9 uv9 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.248+0000 7fadbcf2f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:52.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.248+0000 7fadbcf2f800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.248+0000 7fadbcf2f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-05-08T18:06:52.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.248+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 93901877838480 in=678b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55673c123ef0 con 0x55673c0d7a20 2024-05-08T18:06:52.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.248+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== watch-notify(notify (1) cookie 93901877576576 notify 124554051621 ret 0) ==== 708+0+0 (crc 0 0 0) 0x7fadb003d040 con 0x55673c0d7a20 2024-05-08T18:06:52.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.248+0000 7fac8ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051621 cookie 93901877576576 notifier 4872 bl.length()=666 2024-05-08T18:06:52.257 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.248+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 notify.4 [notify cookie 93901877838480 out=8b] v0'0 uv261 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.248+0000 7fac8ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:52.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.248+0000 7fac8ffff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.249+0000 7fac8ffff640 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7fac7c003ea0 con 0x55673c0d7a20 2024-05-08T18:06:52.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.249+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.258 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.249+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== watch-notify(notify_complete (2) cookie 93901877838480 notify 124554051621 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fadb0018ba0 con 0x55673c0d7a20 2024-05-08T18:06:52.258 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-05-08T18:06:51.092020Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"user-info-without-keys","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:06:51.092006Z","tags":[],"group_ids":[]} 2024-05-08T18:06:52.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.250+0000 7fac6efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:52.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.250+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93901877589248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0edd00 con 0x55673c0d7a20 2024-05-08T18:06:52.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.251+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 93901877589248] v29'260 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.251+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93901877563760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0eb500 con 0x7fadac076040 2024-05-08T18:06:52.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.252+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 93901877563760] v29'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadb0032040 con 0x7fadac076040 2024-05-08T18:06:52.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.252+0000 7fadbcf2f800 20 remove_watcher() i=0 2024-05-08T18:06:52.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.252+0000 7fadbcf2f800 2 removed watcher, disabling cache 2024-05-08T18:06:52.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.252+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93901877568544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0eb500 con 0x55673c0d7a20 2024-05-08T18:06:52.263 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.253+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 93901877568544] v29'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.253+0000 7fadbcf2f800 20 remove_watcher() i=1 2024-05-08T18:06:52.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.253+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93901877561248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0eb500 con 0x55673c0e0dc0 2024-05-08T18:06:52.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.254+0000 7fadb83c9640 1 -- 172.21.15.50:0/3804925659 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 93901877561248] v29'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadac081070 con 0x55673c0e0dc0 2024-05-08T18:06:52.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.254+0000 7fadbcf2f800 20 remove_watcher() i=2 2024-05-08T18:06:52.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.254+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93901877574512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e47a0 con 0x7fadac076040 2024-05-08T18:06:52.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.255+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 93901877574512] v29'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadb0032040 con 0x7fadac076040 2024-05-08T18:06:52.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.255+0000 7fadbcf2f800 20 remove_watcher() i=3 2024-05-08T18:06:52.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.255+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93901877576576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0eb500 con 0x55673c0d7a20 2024-05-08T18:06:52.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.256+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 93901877576576] v29'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fadb0032040 con 0x55673c0d7a20 2024-05-08T18:06:52.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.256+0000 7fadbcf2f800 20 remove_watcher() i=4 2024-05-08T18:06:52.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.256+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93901877580608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e7b70 con 0x55673c0dc410 2024-05-08T18:06:52.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.257+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 93901877580608] v29'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.257+0000 7fadbcf2f800 20 remove_watcher() i=5 2024-05-08T18:06:52.264 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.257+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93901877577632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0e47a0 con 0x55673c0dc410 2024-05-08T18:06:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.258+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 93901877577632] v29'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.258+0000 7fadbcf2f800 20 remove_watcher() i=6 2024-05-08T18:06:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.258+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93901877586864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55673c0eb500 con 0x55673c0dc410 2024-05-08T18:06:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadb8bca640 1 -- 172.21.15.50:0/3804925659 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 93901877586864] v29'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fada8073020 con 0x55673c0dc410 2024-05-08T18:06:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 20 remove_watcher() i=7 2024-05-08T18:06:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55673c0dc410 msgr2=0x55673c0de800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55673c0dc410 0x55673c0de800 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55673c0d7a20 msgr2=0x55673c0d9ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55673c0d7a20 0x55673c0d9ee0 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fadac076040 msgr2=0x7fadac083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7fadac076040 0x7fadac083410 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55673c0e0dc0 msgr2=0x55673c0e3280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55673c0e0dc0 0x55673c0e3280 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac8403bba0 msgr2=0x7fac8403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac8403bba0 0x7fac8403e060 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fadac052fd0 tx=0x7fadac04f000 comp rx=0 tx=0).stop 2024-05-08T18:06:52.265 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0c5160 msgr2=0x55673c0cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadb93cb640 1 -- 172.21.15.50:0/3804925659 reap_dead start 2024-05-08T18:06:52.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0c5160 0x55673c0cc4e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fada8039000 tx=0x7fada800a340 comp rx=0 tx=0).stop 2024-05-08T18:06:52.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 shutdown_connections 2024-05-08T18:06:52.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/3804925659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0c5160 0x55673c0cc4e0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 >> 172.21.15.50:0/3804925659 conn(0x55673c0c14c0 msgr2=0x55673c0beb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 shutdown_connections 2024-05-08T18:06:52.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.259+0000 7fadbcf2f800 1 -- 172.21.15.50:0/3804925659 wait complete. 2024-05-08T18:06:52.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 -- 172.21.15.50:0/423462255 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac9003bba0 msgr2=0x7fac9003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/423462255 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac9003bba0 0x7fac9003e060 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55673c0a1a00 tx=0x7fadac005a00 comp rx=0 tx=0).stop 2024-05-08T18:06:52.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 -- 172.21.15.50:0/423462255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0b81f0 msgr2=0x55673c0b8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/423462255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0b81f0 0x55673c0b8610 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fada80511b0 tx=0x7fada8043500 comp rx=0 tx=0).stop 2024-05-08T18:06:52.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 -- 172.21.15.50:0/423462255 shutdown_connections 2024-05-08T18:06:52.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/423462255 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fac9003bba0 0x7fac9003e060 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.266 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/423462255 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673c0b81f0 0x55673c0b8610 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 -- 172.21.15.50:0/423462255 >> 172.21.15.50:0/423462255 conn(0x55673c0ac290 msgr2=0x55673c0ac690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 -- 172.21.15.50:0/423462255 shutdown_connections 2024-05-08T18:06:52.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 -- 172.21.15.50:0/423462255 wait complete. 2024-05-08T18:06:52.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 -- 172.21.15.50:0/1630509441 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55673c0a3450 msgr2=0x55673c0a5840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55673c0a3450 0x55673c0a5840 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 -- 172.21.15.50:0/1630509441 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55673bfa2b00 msgr2=0x55673c0a0b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55673bfa2b00 0x55673c0a0b70 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 -- 172.21.15.50:0/1630509441 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55673c0a7390 msgr2=0x55673c0a9780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55673c0a7390 0x55673c0a9780 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 -- 172.21.15.50:0/1630509441 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7faca003bb50 msgr2=0x7faca003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7faca003bb50 0x7faca003e010 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fadac054000 tx=0x7fadac04e000 comp rx=0 tx=0).stop 2024-05-08T18:06:52.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 -- 172.21.15.50:0/1630509441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bf9e710 msgr2=0x55673c099980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 --2- 172.21.15.50:0/1630509441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55673bf9e710 0x55673c099980 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fada8000c00 tx=0x7fada8000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadb93cb640 1 -- 172.21.15.50:0/1630509441 reap_dead start 2024-05-08T18:06:52.267 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 -- 172.21.15.50:0/1630509441 shutdown_connections 2024-05-08T18:06:52.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.260+0000 7fadbcf2f800 1 -- 172.21.15.50:0/1630509441 >> 172.21.15.50:0/1630509441 conn(0x55673bf99c90 msgr2=0x55673bf12da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.261+0000 7fadbcf2f800 1 -- 172.21.15.50:0/1630509441 shutdown_connections 2024-05-08T18:06:52.268 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.261+0000 7fadbcf2f800 1 -- 172.21.15.50:0/1630509441 wait complete. 2024-05-08T18:06:52.268 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:52.269 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:52.269 ERROR:root:0 2024-05-08T18:06:52.269 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:52.273 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:52.273 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-05-08T18:06:52.274 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-05-08T18:06:52.274 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps 'users=read, write' 2024-05-08T18:06:52.321 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:52.321 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:52.322 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.324+0000 7f13ada1f800 1 Processor -- start 2024-05-08T18:06:52.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.324+0000 7f13ada1f800 1 -- start start 2024-05-08T18:06:52.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.324+0000 7f13ada1f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x562aebd33c10 0x562aebd36000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.324+0000 7f13ada1f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562aebd36540 con 0x562aebd30100 2024-05-08T18:06:52.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.324+0000 7f13ada1f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x562aebbcfe20 con 0x562aebd33c10 2024-05-08T18:06:52.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.324+0000 7f13aad16640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x562aebd33c10 0x562aebd36000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.324+0000 7f13ab517640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x562aebd30100 0x562aebd324f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50888/0 (socket says 172.21.15.50:50888) 2024-05-08T18:06:52.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.324+0000 7f13ab517640 1 -- 172.21.15.50:0/3257606811 learned_addr learned my addr 172.21.15.50:0/3257606811 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:52.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.325+0000 7f13aad16640 1 -- 172.21.15.50:0/3257606811 >> v1:172.21.15.46:6789/0 conn(0x562aebd30100 legacy=0x562aebd324f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.325+0000 7f13aad16640 1 -- 172.21.15.50:0/3257606811 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aebd37720 con 0x562aebd33c10 2024-05-08T18:06:52.323 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.325+0000 7f13aa515640 1 -- 172.21.15.50:0/3257606811 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2053283897 0 0) 0x562aebd36540 con 0x562aebd30100 2024-05-08T18:06:52.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.327+0000 7f13aad16640 1 --2- 172.21.15.50:0/3257606811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd33c10 0x562aebd36000 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f13a000a470 tx=0x7f13a001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43871a46a780f7be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.327+0000 7f13aa515640 1 -- 172.21.15.50:0/3257606811 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13a0057070 con 0x562aebd33c10 2024-05-08T18:06:52.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.327+0000 7f13aa515640 1 -- 172.21.15.50:0/3257606811 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13a0052570 con 0x562aebd33c10 2024-05-08T18:06:52.325 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.327+0000 7f13aa515640 1 -- 172.21.15.50:0/3257606811 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13a0052870 con 0x562aebd33c10 2024-05-08T18:06:52.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.327+0000 7f13ada1f800 1 -- 172.21.15.50:0/3257606811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd33c10 msgr2=0x562aebd36000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.327+0000 7f13ada1f800 1 --2- 172.21.15.50:0/3257606811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd33c10 0x562aebd36000 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f13a000a470 tx=0x7f13a001ce90 comp rx=0 tx=0).stop 2024-05-08T18:06:52.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.327+0000 7f13ada1f800 1 -- 172.21.15.50:0/3257606811 shutdown_connections 2024-05-08T18:06:52.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.327+0000 7f13ada1f800 1 --2- 172.21.15.50:0/3257606811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd33c10 0x562aebd36000 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.327+0000 7f13ada1f800 1 -- 172.21.15.50:0/3257606811 >> 172.21.15.50:0/3257606811 conn(0x562aebd29b70 msgr2=0x562aebd2bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.327+0000 7f13ada1f800 1 -- 172.21.15.50:0/3257606811 shutdown_connections 2024-05-08T18:06:52.326 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.327+0000 7f13ada1f800 1 -- 172.21.15.50:0/3257606811 wait complete. 2024-05-08T18:06:52.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.343+0000 7f13ada1f800 1 Processor -- start 2024-05-08T18:06:52.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.343+0000 7f13ada1f800 1 -- start start 2024-05-08T18:06:52.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.343+0000 7f13ada1f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd30100 0x562aebe20a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.343+0000 7f13ada1f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562aebc7aa30 con 0x562aebd30100 2024-05-08T18:06:52.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.343+0000 7f13ab517640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd30100 0x562aebe20a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.343+0000 7f13ab517640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd30100 0x562aebe20a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46556/0 (socket says 172.21.15.50:46556) 2024-05-08T18:06:52.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.343+0000 7f13ab517640 1 -- 172.21.15.50:0/4220980334 learned_addr learned my addr 172.21.15.50:0/4220980334 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.342 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.344+0000 7f13ab517640 1 -- 172.21.15.50:0/4220980334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aebe21000 con 0x562aebd30100 2024-05-08T18:06:52.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.344+0000 7f13ab517640 1 --2- 172.21.15.50:0/4220980334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd30100 0x562aebe20a30 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f139c001e70 tx=0x7f139c00b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8682740ba319c357 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.344+0000 7f129960a640 1 -- 172.21.15.50:0/4220980334 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f139c008070 con 0x562aebd30100 2024-05-08T18:06:52.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.344+0000 7f129960a640 1 -- 172.21.15.50:0/4220980334 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139c00d500 con 0x562aebd30100 2024-05-08T18:06:52.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.344+0000 7f129960a640 1 -- 172.21.15.50:0/4220980334 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f139c00d8a0 con 0x562aebd30100 2024-05-08T18:06:52.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.345+0000 7f13ada1f800 1 -- 172.21.15.50:0/4220980334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd30100 msgr2=0x562aebe20a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.345+0000 7f13ada1f800 1 --2- 172.21.15.50:0/4220980334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd30100 0x562aebe20a30 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f139c001e70 tx=0x7f139c00b6a0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.345+0000 7f13ada1f800 1 -- 172.21.15.50:0/4220980334 shutdown_connections 2024-05-08T18:06:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.345+0000 7f13ada1f800 1 --2- 172.21.15.50:0/4220980334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd30100 0x562aebe20a30 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.345+0000 7f13ada1f800 1 -- 172.21.15.50:0/4220980334 >> 172.21.15.50:0/4220980334 conn(0x562aebd29b70 msgr2=0x562aebe1c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.345+0000 7f13ada1f800 1 -- 172.21.15.50:0/4220980334 shutdown_connections 2024-05-08T18:06:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.345+0000 7f13ada1f800 1 -- 172.21.15.50:0/4220980334 wait complete. 2024-05-08T18:06:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.345+0000 7f13ada1f800 1 Processor -- start 2024-05-08T18:06:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.345+0000 7f13ada1f800 1 -- start start 2024-05-08T18:06:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.345+0000 7f13ada1f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd30100 0x562aebe285a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.345+0000 7f13ada1f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562aebd36a10 con 0x562aebd30100 2024-05-08T18:06:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.345+0000 7f13ab517640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd30100 0x562aebe285a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.345+0000 7f13ab517640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd30100 0x562aebe285a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46564/0 (socket says 172.21.15.50:46564) 2024-05-08T18:06:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.345+0000 7f13ab517640 1 -- 172.21.15.50:0/2065303129 learned_addr learned my addr 172.21.15.50:0/2065303129 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.346 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.346+0000 7f13ab517640 1 -- 172.21.15.50:0/2065303129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aebe28ae0 con 0x562aebd30100 2024-05-08T18:06:52.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.346+0000 7f13ab517640 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd30100 0x562aebe285a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x562aebe217c0 tx=0x7f139c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.346+0000 7f1297e07640 1 -- 172.21.15.50:0/2065303129 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f139c00f980 con 0x562aebd30100 2024-05-08T18:06:52.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.346+0000 7f1297e07640 1 -- 172.21.15.50:0/2065303129 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139c00fb20 con 0x562aebd30100 2024-05-08T18:06:52.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.346+0000 7f13ada1f800 1 -- 172.21.15.50:0/2065303129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562aebe28d70 con 0x562aebd30100 2024-05-08T18:06:52.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.346+0000 7f1297e07640 1 -- 172.21.15.50:0/2065303129 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f139c00fe00 con 0x562aebd30100 2024-05-08T18:06:52.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.346+0000 7f13ada1f800 1 -- 172.21.15.50:0/2065303129 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aebe2ba20 con 0x562aebd30100 2024-05-08T18:06:52.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.347+0000 7f1297e07640 1 -- 172.21.15.50:0/2065303129 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f139c017040 con 0x562aebd30100 2024-05-08T18:06:52.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.347+0000 7f1297e07640 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f129003bba0 0x7f129003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.347+0000 7f1297e07640 1 -- 172.21.15.50:0/2065303129 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f139c015020 con 0x562aebd30100 2024-05-08T18:06:52.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.347+0000 7f13aad16640 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f129003bba0 0x7f129003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.347+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562aebd33c10 0x562aebe2f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.347+0000 7f13ada1f800 1 -- 172.21.15.50:0/2065303129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe2fd20 con 0x562aebd33c10 2024-05-08T18:06:52.347 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.347+0000 7f13abd18640 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562aebd33c10 0x562aebe2f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.348+0000 7f13aad16640 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f129003bba0 0x7f129003e060 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f13a0054000 tx=0x7f13a004e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.348+0000 7f13abd18640 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562aebd33c10 0x562aebe2f7e0 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.348+0000 7f13abd18640 1 -- 172.21.15.50:0/2065303129 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f13a400a4a0 con 0x562aebd33c10 2024-05-08T18:06:52.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.348+0000 7f13ada1f800 1 -- 172.21.15.50:0/2065303129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe2cd90 con 0x562aebd33c10 2024-05-08T18:06:52.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.348+0000 7f13abd18640 1 -- 172.21.15.50:0/2065303129 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f13a400a4a0 con 0x562aebd33c10 2024-05-08T18:06:52.355 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.348+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562aebe320a0 0x562aebe34490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.348+0000 7f13ada1f800 1 -- 172.21.15.50:0/2065303129 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe34b30 con 0x562aebe320a0 2024-05-08T18:06:52.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.349+0000 7f13ab517640 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562aebe320a0 0x562aebe34490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.349+0000 7f13ab517640 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562aebe320a0 0x562aebe34490 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.349+0000 7f13ab517640 1 -- 172.21.15.50:0/2065303129 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f139c02d070 con 0x562aebe320a0 2024-05-08T18:06:52.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.349+0000 7f13ada1f800 1 -- 172.21.15.50:0/2065303129 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe30f10 con 0x562aebd33c10 2024-05-08T18:06:52.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.349+0000 7f13abd18640 1 -- 172.21.15.50:0/2065303129 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f13a400a4a0 con 0x562aebd33c10 2024-05-08T18:06:52.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.350+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562aebe35f90 0x562aebe38380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.350+0000 7f13ada1f800 1 -- 172.21.15.50:0/2065303129 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe38a70 con 0x562aebe35f90 2024-05-08T18:06:52.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.350+0000 7f13aad16640 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562aebe35f90 0x562aebe38380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.350+0000 7f13aad16640 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562aebe35f90 0x562aebe38380 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.350+0000 7f13aad16640 1 -- 172.21.15.50:0/2065303129 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f13a0052940 con 0x562aebe35f90 2024-05-08T18:06:52.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.351+0000 7f13ada1f800 1 Processor -- start 2024-05-08T18:06:52.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.351+0000 7f13ada1f800 1 -- start start 2024-05-08T18:06:52.356 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.351+0000 7f13ada1f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebe46d40 0x562aebe47160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.351+0000 7f13ada1f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562aebc21330 con 0x562aebe46d40 2024-05-08T18:06:52.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.351+0000 7f13ab517640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebe46d40 0x562aebe47160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.351+0000 7f13ab517640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebe46d40 0x562aebe47160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46570/0 (socket says 172.21.15.50:46570) 2024-05-08T18:06:52.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.351+0000 7f13ab517640 1 -- 172.21.15.50:0/1947331147 learned_addr learned my addr 172.21.15.50:0/1947331147 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.351+0000 7f13ab517640 1 -- 172.21.15.50:0/1947331147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aebe476a0 con 0x562aebe46d40 2024-05-08T18:06:52.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.351+0000 7f13ab517640 1 --2- 172.21.15.50:0/1947331147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebe46d40 0x562aebe47160 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f139c051a80 tx=0x7f139c01ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.352+0000 7f1295602640 1 -- 172.21.15.50:0/1947331147 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f139c04a350 con 0x562aebe46d40 2024-05-08T18:06:52.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.352+0000 7f1295602640 1 -- 172.21.15.50:0/1947331147 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139c0501b0 con 0x562aebe46d40 2024-05-08T18:06:52.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.352+0000 7f13ada1f800 1 -- 172.21.15.50:0/1947331147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562aebe48110 con 0x562aebe46d40 2024-05-08T18:06:52.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.352+0000 7f1295602640 1 -- 172.21.15.50:0/1947331147 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f139c053070 con 0x562aebe46d40 2024-05-08T18:06:52.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.352+0000 7f13ada1f800 1 -- 172.21.15.50:0/1947331147 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aebe48610 con 0x562aebe46d40 2024-05-08T18:06:52.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.352+0000 7f1295602640 1 -- 172.21.15.50:0/1947331147 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f139c051b70 con 0x562aebe46d40 2024-05-08T18:06:52.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.353+0000 7f1295602640 1 --2- 172.21.15.50:0/1947331147 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f128403bba0 0x7f128403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.357 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.353+0000 7f1295602640 1 -- 172.21.15.50:0/1947331147 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f139c03cad0 con 0x562aebe46d40 2024-05-08T18:06:52.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.353+0000 7f13aad16640 1 --2- 172.21.15.50:0/1947331147 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f128403bba0 0x7f128403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.353+0000 7f13ada1f800 1 Processor -- start 2024-05-08T18:06:52.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.353+0000 7f13ada1f800 1 -- start start 2024-05-08T18:06:52.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.353+0000 7f13aad16640 1 --2- 172.21.15.50:0/1947331147 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f128403bba0 0x7f128403e060 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x562aebe2ccd0 tx=0x7f13a0005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.353+0000 7f13ada1f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd2f790 0x562aebd2fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.353+0000 7f13ada1f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562aebe21380 con 0x562aebd2f790 2024-05-08T18:06:52.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.353+0000 7f13ab517640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd2f790 0x562aebd2fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.353+0000 7f13ab517640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd2f790 0x562aebd2fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46580/0 (socket says 172.21.15.50:46580) 2024-05-08T18:06:52.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.353+0000 7f13ab517640 1 -- 172.21.15.50:0/2658902527 learned_addr learned my addr 172.21.15.50:0/2658902527 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.354+0000 7f13ab517640 1 -- 172.21.15.50:0/2658902527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aebd2df80 con 0x562aebd2f790 2024-05-08T18:06:52.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.354+0000 7f13ab517640 1 --2- 172.21.15.50:0/2658902527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd2f790 0x562aebd2fbb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f139c03ae10 tx=0x7f139c02f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dee992f1f5db1b7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.354+0000 7f128effd640 1 -- 172.21.15.50:0/2658902527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f139c02f5a0 con 0x562aebd2f790 2024-05-08T18:06:52.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.354+0000 7f128effd640 1 -- 172.21.15.50:0/2658902527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139c02e350 con 0x562aebd2f790 2024-05-08T18:06:52.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.354+0000 7f128effd640 1 -- 172.21.15.50:0/2658902527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f139c053210 con 0x562aebd2f790 2024-05-08T18:06:52.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.354+0000 7f13ada1f800 1 -- 172.21.15.50:0/2658902527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd2f790 msgr2=0x562aebd2fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.354+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2658902527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd2f790 0x562aebd2fbb0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f139c03ae10 tx=0x7f139c02f800 comp rx=0 tx=0).stop 2024-05-08T18:06:52.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.355+0000 7f13ada1f800 1 -- 172.21.15.50:0/2658902527 shutdown_connections 2024-05-08T18:06:52.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.355+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2658902527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd2f790 0x562aebd2fbb0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.355+0000 7f13ada1f800 1 -- 172.21.15.50:0/2658902527 >> 172.21.15.50:0/2658902527 conn(0x562aebe50320 msgr2=0x562aebe4e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.355+0000 7f13ada1f800 1 -- 172.21.15.50:0/2658902527 shutdown_connections 2024-05-08T18:06:52.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.355+0000 7f13ada1f800 1 -- 172.21.15.50:0/2658902527 wait complete. 2024-05-08T18:06:52.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.355+0000 7f13ada1f800 1 Processor -- start 2024-05-08T18:06:52.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.355+0000 7f13ada1f800 1 -- start start 2024-05-08T18:06:52.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.355+0000 7f13ada1f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd2f790 0x562aebe5d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.355+0000 7f13ada1f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562aebd2e390 con 0x562aebd2f790 2024-05-08T18:06:52.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.355+0000 7f13ab517640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd2f790 0x562aebe5d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.355+0000 7f13ab517640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd2f790 0x562aebe5d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46596/0 (socket says 172.21.15.50:46596) 2024-05-08T18:06:52.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.355+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 learned_addr learned my addr 172.21.15.50:0/2191611277 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.356+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562aebe5dbb0 con 0x562aebd2f790 2024-05-08T18:06:52.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.356+0000 7f13ab517640 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd2f790 0x562aebe5d670 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f139c02f790 tx=0x7f139c03ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.356+0000 7f128d7fa640 1 -- 172.21.15.50:0/2191611277 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f139c06b020 con 0x562aebd2f790 2024-05-08T18:06:52.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.356+0000 7f128d7fa640 1 -- 172.21.15.50:0/2191611277 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139c019470 con 0x562aebd2f790 2024-05-08T18:06:52.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.356+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562aebe5dbb0 con 0x562aebd2f790 2024-05-08T18:06:52.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.356+0000 7f128d7fa640 1 -- 172.21.15.50:0/2191611277 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f139c062040 con 0x562aebd2f790 2024-05-08T18:06:52.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.356+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562aebe5dbb0 con 0x562aebd2f790 2024-05-08T18:06:52.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.357+0000 7f128d7fa640 1 -- 172.21.15.50:0/2191611277 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f139c06e020 con 0x562aebd2f790 2024-05-08T18:06:52.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.357+0000 7f128d7fa640 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f127403bb50 0x7f127403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.357+0000 7f128d7fa640 1 -- 172.21.15.50:0/2191611277 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f139c059070 con 0x562aebd2f790 2024-05-08T18:06:52.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.357+0000 7f13aad16640 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f127403bb50 0x7f127403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.358+0000 7f13aad16640 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f127403bb50 0x7f127403e010 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f13a0052fd0 tx=0x7f13a004f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.358+0000 7f13ada1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.358+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562aebe68d80 0x562aebe6b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.358+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe6b990 con 0x562aebe68d80 2024-05-08T18:06:52.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.358+0000 7f13abd18640 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562aebe68d80 0x562aebe6b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.358+0000 7f13abd18640 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562aebe68d80 0x562aebe6b240 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f13a400a4a0 con 0x562aebe68d80 2024-05-08T18:06:52.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13ada1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13ada1f800 20 realm 2024-05-08T18:06:52.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13ada1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe6bda0 con 0x562aebe68d80 2024-05-08T18:06:52.382 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f13a400a4a0 con 0x562aebe68d80 2024-05-08T18:06:52.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13ada1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13ada1f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:52.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13ada1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe6c670 con 0x562aebe68d80 2024-05-08T18:06:52.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f13a400a4a0 con 0x562aebe68d80 2024-05-08T18:06:52.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13ada1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13ada1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe6ca10 con 0x562aebe68d80 2024-05-08T18:06:52.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f13a400a4a0 con 0x562aebe68d80 2024-05-08T18:06:52.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13ada1f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:52.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13ada1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562aebe6d770 0x562aebe6fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.359+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe70170 con 0x562aebe6d770 2024-05-08T18:06:52.383 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.360+0000 7f13ab517640 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562aebe6d770 0x562aebe6fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.360+0000 7f13ab517640 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562aebe6d770 0x562aebe6fb60 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.360+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.360+0000 7f13ada1f800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:52.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.360+0000 7f13ada1f800 20 searching for the correct realm 2024-05-08T18:06:52.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.360+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562aebe72120 0x562aebe745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.360+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x562aebe74cb0 con 0x562aebe72120 2024-05-08T18:06:52.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.361+0000 7f13aad16640 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562aebe72120 0x562aebe745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.361+0000 7f13aad16640 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562aebe72120 0x562aebe745e0 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.361+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a0081070 con 0x562aebe72120 2024-05-08T18:06:52.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.361+0000 7f13aad16640 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f13a0076040 0x7f13a0083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.361+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x562aebe74cb0 con 0x7f13a0076040 2024-05-08T18:06:52.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.361+0000 7f13abd18640 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f13a0076040 0x7f13a0083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.361+0000 7f13abd18640 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f13a0076040 0x7f13a0083410 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.384 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.362+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a400a4a0 con 0x7f13a0076040 2024-05-08T18:06:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.362+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x562aebe6ca10 con 0x562aebe6d770 2024-05-08T18:06:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.362+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.362+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x562aebe6ca10 con 0x562aebe68d80 2024-05-08T18:06:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.362+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a400a4a0 con 0x562aebe68d80 2024-05-08T18:06:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.362+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x562aebe6ca10 con 0x562aebe68d80 2024-05-08T18:06:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.363+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a400a4a0 con 0x562aebe68d80 2024-05-08T18:06:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.363+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x562aebe6ca10 con 0x562aebe68d80 2024-05-08T18:06:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.363+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a400a4a0 con 0x562aebe68d80 2024-05-08T18:06:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.363+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x562aebe6ca10 con 0x562aebe6d770 2024-05-08T18:06:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.363+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v29'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.363+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x562aebe6ca10 con 0x7f13a0076040 2024-05-08T18:06:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.363+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a400a4a0 con 0x7f13a0076040 2024-05-08T18:06:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.363+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x562aebe6ca10 con 0x562aebe72120 2024-05-08T18:06:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.363+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a0081070 con 0x562aebe72120 2024-05-08T18:06:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.364+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x562aebe6ca10 con 0x562aebe68d80 2024-05-08T18:06:52.385 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.364+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a400a4a0 con 0x562aebe68d80 2024-05-08T18:06:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.364+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x562aebe6ca10 con 0x562aebe6d770 2024-05-08T18:06:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.364+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.364+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x562aebe6ca10 con 0x562aebe68d80 2024-05-08T18:06:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.364+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a400a4a0 con 0x562aebe68d80 2024-05-08T18:06:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.364+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x562aebe6ca10 con 0x7f13a0076040 2024-05-08T18:06:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.365+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f13a4032040 con 0x7f13a0076040 2024-05-08T18:06:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.365+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x562aebe6ca10 con 0x562aebe72120 2024-05-08T18:06:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.365+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a0081070 con 0x562aebe72120 2024-05-08T18:06:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.365+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x562aebe72120 2024-05-08T18:06:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.365+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a0081070 con 0x562aebe72120 2024-05-08T18:06:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.365+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x562aebe6d770 2024-05-08T18:06:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.365+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.365+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x562aebe72120 2024-05-08T18:06:52.386 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.365+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a0081070 con 0x562aebe72120 2024-05-08T18:06:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.366+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x562aebe72120 2024-05-08T18:06:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.366+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f13a0081070 con 0x562aebe72120 2024-05-08T18:06:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.366+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x562aebe68d80 2024-05-08T18:06:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.366+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.366+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x562aebe68d80 2024-05-08T18:06:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.366+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v29'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.366+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x562aebe6d770 2024-05-08T18:06:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.367+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.367+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x562aebe72120 2024-05-08T18:06:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.367+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a0081070 con 0x562aebe72120 2024-05-08T18:06:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.367+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x7f13a0076040 2024-05-08T18:06:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.367+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a4032040 con 0x7f13a0076040 2024-05-08T18:06:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.367+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x562aebe68d80 2024-05-08T18:06:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.367+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.367+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x562aebe6d770 2024-05-08T18:06:52.387 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.368+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.368+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x562aebe72120 2024-05-08T18:06:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.368+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a0081070 con 0x562aebe72120 2024-05-08T18:06:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.368+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x562aebe6d770 2024-05-08T18:06:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.368+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.368+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x7f13a0076040 2024-05-08T18:06:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.369+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a4032040 con 0x7f13a0076040 2024-05-08T18:06:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.369+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x562aebe72120 2024-05-08T18:06:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.369+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a0081070 con 0x562aebe72120 2024-05-08T18:06:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.369+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x562aebe6d770 2024-05-08T18:06:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.369+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.369+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x562aebe68d80 2024-05-08T18:06:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.369+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v29'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.369+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f13a0090040 con 0x7f13a0076040 2024-05-08T18:06:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f13a4032040 con 0x7f13a0076040 2024-05-08T18:06:52.388 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13ada1f800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:52.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13ada1f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:52.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13ada1f800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:52.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13ada1f800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:52.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13ada1f800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:52.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13ada1f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:52.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13ada1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe70a30 con 0x562aebe68d80 2024-05-08T18:06:52.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13ada1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13ada1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe70a30 con 0x562aebe68d80 2024-05-08T18:06:52.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13ada1f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:52.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13ada1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe70a30 con 0x562aebe72120 2024-05-08T18:06:52.389 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f13a0081070 con 0x562aebe72120 2024-05-08T18:06:52.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.370+0000 7f13ada1f800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:52.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 20 zone default found 2024-05-08T18:06:52.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 4 Realm: () 2024-05-08T18:06:52.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:52.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:52.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:52.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 20 zonegroup default 2024-05-08T18:06:52.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe71390 con 0x562aebe6d770 2024-05-08T18:06:52.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.390 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe76ba0 con 0x562aebe72120 2024-05-08T18:06:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f13a0081070 con 0x562aebe72120 2024-05-08T18:06:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe77030 con 0x562aebe72120 2024-05-08T18:06:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f13a0081070 con 0x562aebe72120 2024-05-08T18:06:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 20 started sync module instance, tier type = 2024-05-08T18:06:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.371+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe75fb0 con 0x7f13a0076040 2024-05-08T18:06:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.372+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f13a4032040 con 0x7f13a0076040 2024-05-08T18:06:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.372+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94742346425520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe75fb0 con 0x7f13a0076040 2024-05-08T18:06:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.372+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe77ab0 con 0x562aebe68d80 2024-05-08T18:06:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.373+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94742346425520] v29'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f13a4032040 con 0x7f13a0076040 2024-05-08T18:06:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.374+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.391 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.374+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94742346430432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe75fb0 con 0x562aebe68d80 2024-05-08T18:06:52.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.374+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe78530 con 0x562aebe72120 2024-05-08T18:06:52.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.375+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f13a0081070 con 0x562aebe72120 2024-05-08T18:06:52.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.375+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94742346430432] v29'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.375+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94742346423040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe75f20 con 0x562aebe72120 2024-05-08T18:06:52.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.375+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe79420 con 0x7f13a0076040 2024-05-08T18:06:52.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.376+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94742346423040] v29'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f13a0081070 con 0x562aebe72120 2024-05-08T18:06:52.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.376+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f13a4032040 con 0x7f13a0076040 2024-05-08T18:06:52.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.376+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94742346436464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe79840 con 0x7f13a0076040 2024-05-08T18:06:52.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.376+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe7a0f0 con 0x562aebe68d80 2024-05-08T18:06:52.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.377+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94742346436464] v29'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f13a4032040 con 0x7f13a0076040 2024-05-08T18:06:52.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.377+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.377+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94742346437520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe79c60 con 0x562aebe68d80 2024-05-08T18:06:52.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.377+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe7ab40 con 0x562aebe6d770 2024-05-08T18:06:52.392 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.378+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94742346437520] v29'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.378+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.378+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94742346438576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe79bd0 con 0x562aebe6d770 2024-05-08T18:06:52.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.378+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe7ba00 con 0x562aebe6d770 2024-05-08T18:06:52.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.379+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94742346438576] v29'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.379+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.379+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94742346446160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe7be20 con 0x562aebe6d770 2024-05-08T18:06:52.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.379+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe7c7e0 con 0x562aebe6d770 2024-05-08T18:06:52.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.380+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94742346446160] v29'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.380+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.380+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94742346447696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe7c7e0 con 0x562aebe6d770 2024-05-08T18:06:52.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.380+0000 7f13ada1f800 20 add_watcher() i=0 2024-05-08T18:06:52.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.380+0000 7f13ada1f800 20 add_watcher() i=1 2024-05-08T18:06:52.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.380+0000 7f13ada1f800 20 add_watcher() i=2 2024-05-08T18:06:52.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.380+0000 7f13ada1f800 20 add_watcher() i=3 2024-05-08T18:06:52.393 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.380+0000 7f13ada1f800 20 add_watcher() i=4 2024-05-08T18:06:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.380+0000 7f13ada1f800 20 add_watcher() i=5 2024-05-08T18:06:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.380+0000 7f13ada1f800 20 add_watcher() i=6 2024-05-08T18:06:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.382+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94742346447696] v29'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.382+0000 7f13ada1f800 20 add_watcher() i=7 2024-05-08T18:06:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.382+0000 7f13ada1f800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.382+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebe7faf0 con 0x562aebe68d80 2024-05-08T18:06:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.382+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.382+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94742346450080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe7cd70 con 0x562aebe68d80 2024-05-08T18:06:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.383+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94742346450080] v29'261 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.383+0000 7f1261fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.383+0000 7f13ada1f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.383+0000 7f13ada1f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:52.394 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.383+0000 7f13ada1f800 5 note: GC not initialized 2024-05-08T18:06:52.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.383+0000 7f1260fe1640 20 reqs_thread_entry: start 2024-05-08T18:06:52.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.383+0000 7f13ada1f800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:52.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.384+0000 7f13ada1f800 20 Filter name: none 2024-05-08T18:06:52.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.384+0000 7f11da7fc640 20 reqs_thread_entry: start 2024-05-08T18:06:52.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.384+0000 7f13ada1f800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-05-08T18:06:52.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.384+0000 7f13ada1f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.395 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.384+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebeb3080 con 0x562aebe6d770 2024-05-08T18:06:52.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.384+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(62 ada [call out=48b,read 0~490 out=490b,getxattrs out=75b] v0'0 uv9 ondisk = 0) ==== 231+0+613 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.384+0000 7f13ada1f800 20 rados_obj.operate() r=0 bl.length=490 2024-05-08T18:06:52.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.384+0000 7f13ada1f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-05-08T18:06:52.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.384+0000 7f13ada1f800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.384+0000 7f13ada1f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:52.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.385+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~503 in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebeb9890 con 0x562aebe6d770 2024-05-08T18:06:52.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.386+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~503] v29'10 uv10 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.386+0000 7f13ada1f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:52.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.386+0000 7f13ada1f800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.386+0000 7f13ada1f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-05-08T18:06:52.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.386+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94742346682816 in=691b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x562aebeb6cd0 con 0x562aebe68d80 2024-05-08T18:06:52.397 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.386+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== watch-notify(notify (1) cookie 94742346437520 notify 124554051622 ret 0) ==== 721+0+0 (crc 0 0 0) 0x7f13a403d040 con 0x562aebe68d80 2024-05-08T18:06:52.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.386+0000 7f128effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051622 cookie 94742346437520 notifier 4879 bl.length()=679 2024-05-08T18:06:52.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.386+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 notify.4 [notify cookie 94742346682816 out=8b] v0'0 uv264 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.386+0000 7f128effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:52.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.386+0000 7f128effd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.386+0000 7f128effd640 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f1270003ea0 con 0x562aebe68d80 2024-05-08T18:06:52.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.386+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.398 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.386+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== watch-notify(notify_complete (2) cookie 94742346682816 notify 124554051622 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f13a4018870 con 0x562aebe68d80 2024-05-08T18:06:52.398 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-05-08T18:06:51.092020Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"user-info-without-keys","perm":"*"},{"type":"users","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:06:51.092006Z","tags":[],"group_ids":[]} 2024-05-08T18:06:52.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.387+0000 7f1261fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:52.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.387+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94742346450080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebeb2410 con 0x562aebe68d80 2024-05-08T18:06:52.403 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.388+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94742346450080] v29'262 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.388+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94742346425520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebeb2410 con 0x7f13a0076040 2024-05-08T18:06:52.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.389+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94742346425520] v29'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f13a4032040 con 0x7f13a0076040 2024-05-08T18:06:52.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.389+0000 7f13ada1f800 20 remove_watcher() i=0 2024-05-08T18:06:52.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.389+0000 7f13ada1f800 2 removed watcher, disabling cache 2024-05-08T18:06:52.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.389+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94742346430432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebeb2410 con 0x562aebe68d80 2024-05-08T18:06:52.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.391+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94742346430432] v29'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.391+0000 7f13ada1f800 20 remove_watcher() i=1 2024-05-08T18:06:52.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.391+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94742346423040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebeb2410 con 0x562aebe72120 2024-05-08T18:06:52.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.392+0000 7f13aad16640 1 -- 172.21.15.50:0/2191611277 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94742346423040] v29'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f13a0081070 con 0x562aebe72120 2024-05-08T18:06:52.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.392+0000 7f13ada1f800 20 remove_watcher() i=2 2024-05-08T18:06:52.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.392+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94742346436464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebeb2410 con 0x7f13a0076040 2024-05-08T18:06:52.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.393+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94742346436464] v29'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f13a4032040 con 0x7f13a0076040 2024-05-08T18:06:52.404 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.393+0000 7f13ada1f800 20 remove_watcher() i=3 2024-05-08T18:06:52.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.393+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94742346437520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebeb2410 con 0x562aebe68d80 2024-05-08T18:06:52.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.393+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94742346437520] v29'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f13a4032040 con 0x562aebe68d80 2024-05-08T18:06:52.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.394+0000 7f13ada1f800 20 remove_watcher() i=4 2024-05-08T18:06:52.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.394+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94742346438576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebe764b0 con 0x562aebe6d770 2024-05-08T18:06:52.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.395+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94742346438576] v29'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.395+0000 7f13ada1f800 20 remove_watcher() i=5 2024-05-08T18:06:52.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.395+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94742346446160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebeb2410 con 0x562aebe6d770 2024-05-08T18:06:52.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.395+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94742346446160] v29'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.396+0000 7f13ada1f800 20 remove_watcher() i=6 2024-05-08T18:06:52.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.396+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94742346447696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x562aebeb2410 con 0x562aebe6d770 2024-05-08T18:06:52.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ab517640 1 -- 172.21.15.50:0/2191611277 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94742346447696] v29'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f139c065040 con 0x562aebe6d770 2024-05-08T18:06:52.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 20 remove_watcher() i=7 2024-05-08T18:06:52.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562aebe6d770 msgr2=0x562aebe6fb60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.405 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562aebe6d770 0x562aebe6fb60 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562aebe68d80 msgr2=0x562aebe6b240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562aebe68d80 0x562aebe6b240 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f13a0076040 msgr2=0x7f13a0083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f13a0076040 0x7f13a0083410 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562aebe72120 msgr2=0x562aebe745e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562aebe72120 0x562aebe745e0 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f127403bb50 msgr2=0x7f127403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f127403bb50 0x7f127403e010 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f13a0052fd0 tx=0x7f13a004f000 comp rx=0 tx=0).stop 2024-05-08T18:06:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd2f790 msgr2=0x562aebe5d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13abd18640 1 -- 172.21.15.50:0/2191611277 reap_dead start 2024-05-08T18:06:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd2f790 0x562aebe5d670 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f139c02f790 tx=0x7f139c03ad70 comp rx=0 tx=0).stop 2024-05-08T18:06:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 shutdown_connections 2024-05-08T18:06:52.406 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2191611277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd2f790 0x562aebe5d670 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 >> 172.21.15.50:0/2191611277 conn(0x562aebe50320 msgr2=0x562aebe4d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 shutdown_connections 2024-05-08T18:06:52.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 -- 172.21.15.50:0/2191611277 wait complete. 2024-05-08T18:06:52.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 -- 172.21.15.50:0/1947331147 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f128403bba0 msgr2=0x7f128403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 --2- 172.21.15.50:0/1947331147 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f128403bba0 0x7f128403e060 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x562aebe2ccd0 tx=0x7f13a0005a00 comp rx=0 tx=0).stop 2024-05-08T18:06:52.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 -- 172.21.15.50:0/1947331147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebe46d40 msgr2=0x562aebe47160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.397+0000 7f13ada1f800 1 --2- 172.21.15.50:0/1947331147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebe46d40 0x562aebe47160 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f139c051a80 tx=0x7f139c01ab20 comp rx=0 tx=0).stop 2024-05-08T18:06:52.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 -- 172.21.15.50:0/1947331147 shutdown_connections 2024-05-08T18:06:52.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 --2- 172.21.15.50:0/1947331147 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f128403bba0 0x7f128403e060 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 --2- 172.21.15.50:0/1947331147 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebe46d40 0x562aebe47160 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 -- 172.21.15.50:0/1947331147 >> 172.21.15.50:0/1947331147 conn(0x562aebe3ade0 msgr2=0x562aebe3b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 -- 172.21.15.50:0/1947331147 shutdown_connections 2024-05-08T18:06:52.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 -- 172.21.15.50:0/1947331147 wait complete. 2024-05-08T18:06:52.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 -- 172.21.15.50:0/2065303129 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562aebe320a0 msgr2=0x562aebe34490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.407 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x562aebe320a0 0x562aebe34490 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 -- 172.21.15.50:0/2065303129 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562aebd33c10 msgr2=0x562aebe2f7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x562aebd33c10 0x562aebe2f7e0 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 -- 172.21.15.50:0/2065303129 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562aebe35f90 msgr2=0x562aebe38380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x562aebe35f90 0x562aebe38380 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 -- 172.21.15.50:0/2065303129 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f129003bba0 msgr2=0x7f129003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f129003bba0 0x7f129003e060 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f13a0054000 tx=0x7f13a004e000 comp rx=0 tx=0).stop 2024-05-08T18:06:52.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 -- 172.21.15.50:0/2065303129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd30100 msgr2=0x562aebe285a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 --2- 172.21.15.50:0/2065303129 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562aebd30100 0x562aebe285a0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x562aebe217c0 tx=0x7f139c000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13abd18640 1 -- 172.21.15.50:0/2065303129 reap_dead start 2024-05-08T18:06:52.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 -- 172.21.15.50:0/2065303129 shutdown_connections 2024-05-08T18:06:52.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 -- 172.21.15.50:0/2065303129 >> 172.21.15.50:0/2065303129 conn(0x562aebd29b70 msgr2=0x562aebe1ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 -- 172.21.15.50:0/2065303129 shutdown_connections 2024-05-08T18:06:52.408 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.398+0000 7f13ada1f800 1 -- 172.21.15.50:0/2065303129 wait complete. 2024-05-08T18:06:52.409 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:52.409 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:52.409 ERROR:root:0 2024-05-08T18:06:52.410 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:52.414 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:52.414 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'ada', '--caps', 'users=read, write; user-info-without-keys=*'] 2024-05-08T18:06:52.414 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'ada', '--caps', 'users=read, write; user-info-without-keys=*'] 2024-05-08T18:06:52.415 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid ada --caps 'users=read, write; user-info-without-keys=*' 2024-05-08T18:06:52.462 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:52.462 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:52.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.465+0000 7ff77bb2b800 1 Processor -- start 2024-05-08T18:06:52.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.465+0000 7ff77bb2b800 1 -- start start 2024-05-08T18:06:52.463 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.465+0000 7ff77bb2b800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56118f57cb10 0x56118f585610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.465+0000 7ff77bb2b800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56118f4f0010 con 0x56118f57afd0 2024-05-08T18:06:52.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.465+0000 7ff77bb2b800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x56118f41ce20 con 0x56118f57cb10 2024-05-08T18:06:52.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.465+0000 7ff77774b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x56118f57afd0 0x56118f57b3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50896/0 (socket says 172.21.15.50:50896) 2024-05-08T18:06:52.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.465+0000 7ff77774b640 1 -- 172.21.15.50:0/1718553523 learned_addr learned my addr 172.21.15.50:0/1718553523 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:52.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.465+0000 7ff776f4a640 1 --2- 172.21.15.50:0/1718553523 >> v2:172.21.15.46:3300/0 conn(0x56118f57cb10 0x56118f585610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.465+0000 7ff776749640 1 -- 172.21.15.50:0/1718553523 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3599759621 0 0) 0x56118f4f0010 con 0x56118f57afd0 2024-05-08T18:06:52.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.465+0000 7ff776749640 1 -- 172.21.15.50:0/1718553523 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff760001e10 con 0x56118f57afd0 2024-05-08T18:06:52.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.465+0000 7ff776f4a640 1 -- 172.21.15.50:0/1718553523 >> v1:172.21.15.46:6789/0 conn(0x56118f57afd0 legacy=0x56118f57b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.464 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.465+0000 7ff776f4a640 1 -- 172.21.15.50:0/1718553523 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118f4f01e0 con 0x56118f57cb10 2024-05-08T18:06:52.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.467+0000 7ff776f4a640 1 --2- 172.21.15.50:0/1718553523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57cb10 0x56118f585610 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00a470 tx=0x7ff76c01cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d057f46fbb851ced server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.467+0000 7ff776749640 1 -- 172.21.15.50:0/1718553523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff76c057070 con 0x56118f57cb10 2024-05-08T18:06:52.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.467+0000 7ff776749640 1 -- 172.21.15.50:0/1718553523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76c052570 con 0x56118f57cb10 2024-05-08T18:06:52.465 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.467+0000 7ff776749640 1 -- 172.21.15.50:0/1718553523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff76c052870 con 0x56118f57cb10 2024-05-08T18:06:52.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.468+0000 7ff77bb2b800 1 -- 172.21.15.50:0/1718553523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57cb10 msgr2=0x56118f585610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.468+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/1718553523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57cb10 0x56118f585610 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00a470 tx=0x7ff76c01cdb0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.468+0000 7ff77bb2b800 1 -- 172.21.15.50:0/1718553523 shutdown_connections 2024-05-08T18:06:52.466 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.468+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/1718553523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57cb10 0x56118f585610 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.468+0000 7ff77bb2b800 1 -- 172.21.15.50:0/1718553523 >> 172.21.15.50:0/1718553523 conn(0x56118f576af0 msgr2=0x56118f578f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.468+0000 7ff77bb2b800 1 -- 172.21.15.50:0/1718553523 shutdown_connections 2024-05-08T18:06:52.467 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.468+0000 7ff77bb2b800 1 -- 172.21.15.50:0/1718553523 wait complete. 2024-05-08T18:06:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.483+0000 7ff77bb2b800 1 Processor -- start 2024-05-08T18:06:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.483+0000 7ff77bb2b800 1 -- start start 2024-05-08T18:06:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.483+0000 7ff77bb2b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57afd0 0x56118f66fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.483+0000 7ff77bb2b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56118f4c7a30 con 0x56118f57afd0 2024-05-08T18:06:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.484+0000 7ff77774b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57afd0 0x56118f66fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.484+0000 7ff77774b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57afd0 0x56118f66fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46606/0 (socket says 172.21.15.50:46606) 2024-05-08T18:06:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.484+0000 7ff77774b640 1 -- 172.21.15.50:0/719234145 learned_addr learned my addr 172.21.15.50:0/719234145 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.484+0000 7ff77774b640 1 -- 172.21.15.50:0/719234145 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118f6701f0 con 0x56118f57afd0 2024-05-08T18:06:52.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.484+0000 7ff77774b640 1 --2- 172.21.15.50:0/719234145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57afd0 0x56118f66fc20 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff768006fa0 tx=0x7ff768001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9097159d1f642c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.484+0000 7ff66560a640 1 -- 172.21.15.50:0/719234145 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff768004520 con 0x56118f57afd0 2024-05-08T18:06:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.484+0000 7ff66560a640 1 -- 172.21.15.50:0/719234145 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7680046c0 con 0x56118f57afd0 2024-05-08T18:06:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.484+0000 7ff66560a640 1 -- 172.21.15.50:0/719234145 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff768004a20 con 0x56118f57afd0 2024-05-08T18:06:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.485+0000 7ff77bb2b800 1 -- 172.21.15.50:0/719234145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57afd0 msgr2=0x56118f66fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.485+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/719234145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57afd0 0x56118f66fc20 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff768006fa0 tx=0x7ff768001b20 comp rx=0 tx=0).stop 2024-05-08T18:06:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.485+0000 7ff77bb2b800 1 -- 172.21.15.50:0/719234145 shutdown_connections 2024-05-08T18:06:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.485+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/719234145 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57afd0 0x56118f66fc20 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.485+0000 7ff77bb2b800 1 -- 172.21.15.50:0/719234145 >> 172.21.15.50:0/719234145 conn(0x56118f576af0 msgr2=0x56118f66b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.485+0000 7ff77bb2b800 1 -- 172.21.15.50:0/719234145 shutdown_connections 2024-05-08T18:06:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.485+0000 7ff77bb2b800 1 -- 172.21.15.50:0/719234145 wait complete. 2024-05-08T18:06:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.485+0000 7ff77bb2b800 1 Processor -- start 2024-05-08T18:06:52.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.485+0000 7ff77bb2b800 1 -- start start 2024-05-08T18:06:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.486+0000 7ff77bb2b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57afd0 0x56118f677770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.486+0000 7ff77bb2b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56118f585e20 con 0x56118f57afd0 2024-05-08T18:06:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.486+0000 7ff77774b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57afd0 0x56118f677770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.486+0000 7ff77774b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57afd0 0x56118f677770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46616/0 (socket says 172.21.15.50:46616) 2024-05-08T18:06:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.486+0000 7ff77774b640 1 -- 172.21.15.50:0/666452284 learned_addr learned my addr 172.21.15.50:0/666452284 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.486+0000 7ff77774b640 1 -- 172.21.15.50:0/666452284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118f677cb0 con 0x56118f57afd0 2024-05-08T18:06:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.486+0000 7ff77774b640 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57afd0 0x56118f677770 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x56118f6709b0 tx=0x7ff768004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.486+0000 7ff663e07640 1 -- 172.21.15.50:0/666452284 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff768022020 con 0x56118f57afd0 2024-05-08T18:06:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.487+0000 7ff663e07640 1 -- 172.21.15.50:0/666452284 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7680148b0 con 0x56118f57afd0 2024-05-08T18:06:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.487+0000 7ff77bb2b800 1 -- 172.21.15.50:0/666452284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56118f677f40 con 0x56118f57afd0 2024-05-08T18:06:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.487+0000 7ff663e07640 1 -- 172.21.15.50:0/666452284 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff768014b90 con 0x56118f57afd0 2024-05-08T18:06:52.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.487+0000 7ff77bb2b800 1 -- 172.21.15.50:0/666452284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118f67abf0 con 0x56118f57afd0 2024-05-08T18:06:52.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.487+0000 7ff663e07640 1 -- 172.21.15.50:0/666452284 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff768025020 con 0x56118f57afd0 2024-05-08T18:06:52.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.488+0000 7ff663e07640 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff65c03bb50 0x7ff65c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.488+0000 7ff663e07640 1 -- 172.21.15.50:0/666452284 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7ff768018030 con 0x56118f57afd0 2024-05-08T18:06:52.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.488+0000 7ff776f4a640 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff65c03bb50 0x7ff65c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.494 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.488+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56118f57cb10 0x56118f67e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.488+0000 7ff77bb2b800 1 -- 172.21.15.50:0/666452284 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f67ef40 con 0x56118f57cb10 2024-05-08T18:06:52.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.488+0000 7ff777f4c640 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56118f57cb10 0x56118f67e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.488+0000 7ff776f4a640 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff65c03bb50 0x7ff65c03e010 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff76c054000 tx=0x7ff76c04e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.488+0000 7ff777f4c640 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56118f57cb10 0x56118f67e9b0 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.488+0000 7ff777f4c640 1 -- 172.21.15.50:0/666452284 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff7700083e0 con 0x56118f57cb10 2024-05-08T18:06:52.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.489+0000 7ff77bb2b800 1 -- 172.21.15.50:0/666452284 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f67bf60 con 0x56118f57cb10 2024-05-08T18:06:52.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.489+0000 7ff777f4c640 1 -- 172.21.15.50:0/666452284 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff7700083e0 con 0x56118f57cb10 2024-05-08T18:06:52.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.489+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56118f681270 0x56118f683660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.489+0000 7ff77bb2b800 1 -- 172.21.15.50:0/666452284 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f683d50 con 0x56118f681270 2024-05-08T18:06:52.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.489+0000 7ff77774b640 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56118f681270 0x56118f683660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.489+0000 7ff77774b640 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56118f681270 0x56118f683660 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.490+0000 7ff77774b640 1 -- 172.21.15.50:0/666452284 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff76801b070 con 0x56118f681270 2024-05-08T18:06:52.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.490+0000 7ff77bb2b800 1 -- 172.21.15.50:0/666452284 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f6800e0 con 0x56118f57cb10 2024-05-08T18:06:52.495 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.490+0000 7ff777f4c640 1 -- 172.21.15.50:0/666452284 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff7700083e0 con 0x56118f57cb10 2024-05-08T18:06:52.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.490+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56118f6851b0 0x56118f6875a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.490+0000 7ff77bb2b800 1 -- 172.21.15.50:0/666452284 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f687c60 con 0x56118f6851b0 2024-05-08T18:06:52.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.490+0000 7ff776f4a640 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56118f6851b0 0x56118f6875a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.490+0000 7ff776f4a640 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56118f6851b0 0x56118f6875a0 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.491+0000 7ff776f4a640 1 -- 172.21.15.50:0/666452284 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff76c052940 con 0x56118f6851b0 2024-05-08T18:06:52.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.491+0000 7ff77bb2b800 1 Processor -- start 2024-05-08T18:06:52.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.491+0000 7ff77bb2b800 1 -- start start 2024-05-08T18:06:52.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.491+0000 7ff77bb2b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f695fa0 0x56118f6963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.491+0000 7ff77bb2b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56118f46e330 con 0x56118f695fa0 2024-05-08T18:06:52.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.491+0000 7ff77774b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f695fa0 0x56118f6963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.491+0000 7ff77774b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f695fa0 0x56118f6963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46632/0 (socket says 172.21.15.50:46632) 2024-05-08T18:06:52.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.491+0000 7ff77774b640 1 -- 172.21.15.50:0/2102360487 learned_addr learned my addr 172.21.15.50:0/2102360487 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.496 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.492+0000 7ff77774b640 1 -- 172.21.15.50:0/2102360487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118f696900 con 0x56118f695fa0 2024-05-08T18:06:52.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.492+0000 7ff77774b640 1 --2- 172.21.15.50:0/2102360487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f695fa0 0x56118f6963c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff7680513e0 tx=0x7ff7680108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.492+0000 7ff661602640 1 -- 172.21.15.50:0/2102360487 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff76801d490 con 0x56118f695fa0 2024-05-08T18:06:52.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.492+0000 7ff661602640 1 -- 172.21.15.50:0/2102360487 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff76804a1a0 con 0x56118f695fa0 2024-05-08T18:06:52.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.492+0000 7ff77bb2b800 1 -- 172.21.15.50:0/2102360487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56118f697370 con 0x56118f695fa0 2024-05-08T18:06:52.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.492+0000 7ff661602640 1 -- 172.21.15.50:0/2102360487 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff76804a480 con 0x56118f695fa0 2024-05-08T18:06:52.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.492+0000 7ff77bb2b800 1 -- 172.21.15.50:0/2102360487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118f697870 con 0x56118f695fa0 2024-05-08T18:06:52.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.493+0000 7ff661602640 1 -- 172.21.15.50:0/2102360487 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff768010070 con 0x56118f695fa0 2024-05-08T18:06:52.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.493+0000 7ff661602640 1 --2- 172.21.15.50:0/2102360487 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff65003bb50 0x7ff65003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.493+0000 7ff661602640 1 -- 172.21.15.50:0/2102360487 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7ff768039d30 con 0x56118f695fa0 2024-05-08T18:06:52.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.493+0000 7ff776f4a640 1 --2- 172.21.15.50:0/2102360487 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff65003bb50 0x7ff65003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.493+0000 7ff77bb2b800 1 Processor -- start 2024-05-08T18:06:52.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.493+0000 7ff77bb2b800 1 -- start start 2024-05-08T18:06:52.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.493+0000 7ff776f4a640 1 --2- 172.21.15.50:0/2102360487 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff65003bb50 0x7ff65003e010 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x56118f67f820 tx=0x7ff76c005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.497 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.494+0000 7ff77bb2b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f6a30e0 0x56118f6a3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.494+0000 7ff77bb2b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56118f670570 con 0x56118f6a30e0 2024-05-08T18:06:52.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.494+0000 7ff77774b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f6a30e0 0x56118f6a3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.494+0000 7ff77774b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f6a30e0 0x56118f6a3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46638/0 (socket says 172.21.15.50:46638) 2024-05-08T18:06:52.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.494+0000 7ff77774b640 1 -- 172.21.15.50:0/3995403441 learned_addr learned my addr 172.21.15.50:0/3995403441 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.494+0000 7ff77774b640 1 -- 172.21.15.50:0/3995403441 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118f6a3ad0 con 0x56118f6a30e0 2024-05-08T18:06:52.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.494+0000 7ff77774b640 1 --2- 172.21.15.50:0/3995403441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f6a30e0 0x56118f6a3500 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff768010a40 tx=0x7ff76804c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999bd2af23dd1e57 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.494+0000 7ff65affd640 1 -- 172.21.15.50:0/3995403441 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff768043e00 con 0x56118f6a30e0 2024-05-08T18:06:52.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.494+0000 7ff65affd640 1 -- 172.21.15.50:0/3995403441 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7680514d0 con 0x56118f6a30e0 2024-05-08T18:06:52.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.494+0000 7ff65affd640 1 -- 172.21.15.50:0/3995403441 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff76804b850 con 0x56118f6a30e0 2024-05-08T18:06:52.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.495+0000 7ff77bb2b800 1 -- 172.21.15.50:0/3995403441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f6a30e0 msgr2=0x56118f6a3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.495+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/3995403441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f6a30e0 0x56118f6a3500 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff768010a40 tx=0x7ff76804c100 comp rx=0 tx=0).stop 2024-05-08T18:06:52.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.495+0000 7ff77bb2b800 1 -- 172.21.15.50:0/3995403441 shutdown_connections 2024-05-08T18:06:52.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.495+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/3995403441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f6a30e0 0x56118f6a3500 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.498 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.495+0000 7ff77bb2b800 1 -- 172.21.15.50:0/3995403441 >> 172.21.15.50:0/3995403441 conn(0x56118f69f440 msgr2=0x56118f69d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.495+0000 7ff77bb2b800 1 -- 172.21.15.50:0/3995403441 shutdown_connections 2024-05-08T18:06:52.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.495+0000 7ff77bb2b800 1 -- 172.21.15.50:0/3995403441 wait complete. 2024-05-08T18:06:52.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.495+0000 7ff77bb2b800 1 Processor -- start 2024-05-08T18:06:52.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.495+0000 7ff77bb2b800 1 -- start start 2024-05-08T18:06:52.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.495+0000 7ff77bb2b800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f6a30e0 0x56118f6aa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.495+0000 7ff77bb2b800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56118f6a4180 con 0x56118f6a30e0 2024-05-08T18:06:52.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.495+0000 7ff77774b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f6a30e0 0x56118f6aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.496+0000 7ff77774b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f6a30e0 0x56118f6aa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46646/0 (socket says 172.21.15.50:46646) 2024-05-08T18:06:52.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.496+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 learned_addr learned my addr 172.21.15.50:0/778996298 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.496+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56118f6aaa60 con 0x56118f6a30e0 2024-05-08T18:06:52.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.496+0000 7ff77774b640 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f6a30e0 0x56118f6aa520 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff76801dad0 tx=0x7ff76804c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.496+0000 7ff6597fa640 1 -- 172.21.15.50:0/778996298 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff76806e020 con 0x56118f6a30e0 2024-05-08T18:06:52.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.496+0000 7ff6597fa640 1 -- 172.21.15.50:0/778996298 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff768043e00 con 0x56118f6a30e0 2024-05-08T18:06:52.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.496+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56118f6aaa60 con 0x56118f6a30e0 2024-05-08T18:06:52.499 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.496+0000 7ff6597fa640 1 -- 172.21.15.50:0/778996298 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff768063040 con 0x56118f6a30e0 2024-05-08T18:06:52.500 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.496+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56118f6aaa60 con 0x56118f6a30e0 2024-05-08T18:06:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.497+0000 7ff6597fa640 1 -- 172.21.15.50:0/778996298 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff768071020 con 0x56118f6a30e0 2024-05-08T18:06:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.497+0000 7ff6597fa640 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff64003bb50 0x7ff64003e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.497+0000 7ff6597fa640 1 -- 172.21.15.50:0/778996298 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7ff768074d20 con 0x56118f6a30e0 2024-05-08T18:06:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.497+0000 7ff776f4a640 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff64003bb50 0x7ff64003e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.498+0000 7ff776f4a640 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff64003bb50 0x7ff64003e010 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff76c052fd0 tx=0x7ff76c04f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.498+0000 7ff77bb2b800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.498+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56118f6b5c70 0x56118f6b8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.498+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b8880 con 0x56118f6b5c70 2024-05-08T18:06:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.498+0000 7ff777f4c640 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56118f6b5c70 0x56118f6b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.498+0000 7ff777f4c640 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56118f6b5c70 0x56118f6b8130 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff7700083e0 con 0x56118f6b5c70 2024-05-08T18:06:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff77bb2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.513 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff77bb2b800 20 realm 2024-05-08T18:06:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff77bb2b800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b8cb0 con 0x56118f6b5c70 2024-05-08T18:06:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff7700083e0 con 0x56118f6b5c70 2024-05-08T18:06:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff77bb2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff77bb2b800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff77bb2b800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b9580 con 0x56118f6b5c70 2024-05-08T18:06:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff7700083e0 con 0x56118f6b5c70 2024-05-08T18:06:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff77bb2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff77bb2b800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b9920 con 0x56118f6b5c70 2024-05-08T18:06:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff7700083e0 con 0x56118f6b5c70 2024-05-08T18:06:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff77bb2b800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:52.514 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff77bb2b800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56118f6ba2e0 0x56118f6bc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f6bcce0 con 0x56118f6ba2e0 2024-05-08T18:06:52.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.499+0000 7ff77774b640 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56118f6ba2e0 0x56118f6bc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.500+0000 7ff77774b640 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56118f6ba2e0 0x56118f6bc6d0 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.500+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.500+0000 7ff77bb2b800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:52.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.500+0000 7ff77bb2b800 20 searching for the correct realm 2024-05-08T18:06:52.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.500+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56118f6bec70 0x56118f6c1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.500+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c1800 con 0x56118f6bec70 2024-05-08T18:06:52.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.500+0000 7ff776f4a640 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56118f6bec70 0x56118f6c1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.500+0000 7ff776f4a640 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56118f6bec70 0x56118f6c1130 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.501+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff76c081070 con 0x56118f6bec70 2024-05-08T18:06:52.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.501+0000 7ff776f4a640 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff76c076040 0x7ff76c083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.515 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.501+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c1800 con 0x7ff76c076040 2024-05-08T18:06:52.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.501+0000 7ff777f4c640 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff76c076040 0x7ff76c083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.501+0000 7ff777f4c640 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff76c076040 0x7ff76c083410 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.501+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff7700083e0 con 0x7ff76c076040 2024-05-08T18:06:52.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.501+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b9580 con 0x56118f6ba2e0 2024-05-08T18:06:52.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.502+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.502+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b9580 con 0x56118f6b5c70 2024-05-08T18:06:52.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.502+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff7700083e0 con 0x56118f6b5c70 2024-05-08T18:06:52.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.502+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b9580 con 0x56118f6b5c70 2024-05-08T18:06:52.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.502+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.502+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b9580 con 0x56118f6b5c70 2024-05-08T18:06:52.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.502+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.502+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b9580 con 0x56118f6ba2e0 2024-05-08T18:06:52.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.503+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v29'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.516 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.503+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b9580 con 0x7ff76c076040 2024-05-08T18:06:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.503+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff770026040 con 0x7ff76c076040 2024-05-08T18:06:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.503+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b9580 con 0x56118f6bec70 2024-05-08T18:06:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.503+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff76c081070 con 0x56118f6bec70 2024-05-08T18:06:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.503+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b9580 con 0x56118f6b5c70 2024-05-08T18:06:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.503+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.503+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b9580 con 0x56118f6ba2e0 2024-05-08T18:06:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.504+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.504+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b9580 con 0x56118f6b5c70 2024-05-08T18:06:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.504+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.504+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b9580 con 0x7ff76c076040 2024-05-08T18:06:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.504+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff770026040 con 0x7ff76c076040 2024-05-08T18:06:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.504+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b9580 con 0x56118f6bec70 2024-05-08T18:06:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.504+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff76c081070 con 0x56118f6bec70 2024-05-08T18:06:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.504+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x56118f6bec70 2024-05-08T18:06:52.517 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.505+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff76c081070 con 0x56118f6bec70 2024-05-08T18:06:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.505+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x56118f6ba2e0 2024-05-08T18:06:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.505+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.505+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x56118f6bec70 2024-05-08T18:06:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.505+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff76c081070 con 0x56118f6bec70 2024-05-08T18:06:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.505+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x56118f6bec70 2024-05-08T18:06:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.505+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ff76c081070 con 0x56118f6bec70 2024-05-08T18:06:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.505+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x56118f6b5c70 2024-05-08T18:06:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.506+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.506+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x56118f6b5c70 2024-05-08T18:06:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.506+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v29'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.506+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x56118f6ba2e0 2024-05-08T18:06:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.506+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.506+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x56118f6bec70 2024-05-08T18:06:52.518 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.506+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff76c081070 con 0x56118f6bec70 2024-05-08T18:06:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.506+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x7ff76c076040 2024-05-08T18:06:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.507+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff770026040 con 0x7ff76c076040 2024-05-08T18:06:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.507+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x56118f6b5c70 2024-05-08T18:06:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.507+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.507+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x56118f6ba2e0 2024-05-08T18:06:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.507+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.507+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x56118f6bec70 2024-05-08T18:06:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.507+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff76c081070 con 0x56118f6bec70 2024-05-08T18:06:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.507+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x56118f6ba2e0 2024-05-08T18:06:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.508+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.508+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x7ff76c076040 2024-05-08T18:06:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.508+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff770026040 con 0x7ff76c076040 2024-05-08T18:06:52.519 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.508+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x56118f6bec70 2024-05-08T18:06:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.508+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff76c081070 con 0x56118f6bec70 2024-05-08T18:06:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.508+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x56118f6ba2e0 2024-05-08T18:06:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.508+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.508+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x56118f6b5c70 2024-05-08T18:06:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v29'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7ff76c090040 con 0x7ff76c076040 2024-05-08T18:06:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff770026040 con 0x7ff76c076040 2024-05-08T18:06:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff77bb2b800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff77bb2b800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff77bb2b800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff77bb2b800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff77bb2b800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff77bb2b800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:52.520 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff77bb2b800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f6bd080 con 0x56118f6b5c70 2024-05-08T18:06:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff77bb2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff77bb2b800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f6bd080 con 0x56118f6b5c70 2024-05-08T18:06:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff77bb2b800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff77bb2b800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.509+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f6bd080 con 0x56118f6bec70 2024-05-08T18:06:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.510+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff76c081070 con 0x56118f6bec70 2024-05-08T18:06:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.510+0000 7ff77bb2b800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.510+0000 7ff77bb2b800 20 zone default found 2024-05-08T18:06:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.510+0000 7ff77bb2b800 4 Realm: () 2024-05-08T18:06:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.510+0000 7ff77bb2b800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:52.521 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.510+0000 7ff77bb2b800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.510+0000 7ff77bb2b800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.510+0000 7ff77bb2b800 20 zonegroup default 2024-05-08T18:06:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.510+0000 7ff77bb2b800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.510+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f6bda00 con 0x56118f6ba2e0 2024-05-08T18:06:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.510+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.510+0000 7ff77bb2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.510+0000 7ff77bb2b800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.510+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c36f0 con 0x56118f6bec70 2024-05-08T18:06:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.510+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff76c081070 con 0x56118f6bec70 2024-05-08T18:06:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.511+0000 7ff77bb2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.511+0000 7ff77bb2b800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.511+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c3b50 con 0x56118f6bec70 2024-05-08T18:06:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.511+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff76c081070 con 0x56118f6bec70 2024-05-08T18:06:52.522 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.511+0000 7ff77bb2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.511+0000 7ff77bb2b800 20 started sync module instance, tier type = 2024-05-08T18:06:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.511+0000 7ff77bb2b800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.511+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c2b00 con 0x7ff76c076040 2024-05-08T18:06:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.512+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff770026040 con 0x7ff76c076040 2024-05-08T18:06:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.512+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94633420664880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c2b00 con 0x7ff76c076040 2024-05-08T18:06:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.512+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c4630 con 0x56118f6b5c70 2024-05-08T18:06:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.513+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94633420664880] v29'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff770026040 con 0x7ff76c076040 2024-05-08T18:06:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.513+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.514+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94633420669216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c2b00 con 0x56118f6b5c70 2024-05-08T18:06:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.514+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c50f0 con 0x56118f6bec70 2024-05-08T18:06:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.515+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94633420669216] v29'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.515+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff76c081070 con 0x56118f6bec70 2024-05-08T18:06:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.515+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94633420662352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c2a70 con 0x56118f6bec70 2024-05-08T18:06:52.523 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.515+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c5fb0 con 0x7ff76c076040 2024-05-08T18:06:52.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.516+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94633420662352] v29'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff76c081070 con 0x56118f6bec70 2024-05-08T18:06:52.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.516+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff770026040 con 0x7ff76c076040 2024-05-08T18:06:52.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.516+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94633420675840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c63d0 con 0x7ff76c076040 2024-05-08T18:06:52.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.516+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c6d90 con 0x56118f6b5c70 2024-05-08T18:06:52.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.517+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.517+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94633420679392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c71b0 con 0x56118f6b5c70 2024-05-08T18:06:52.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.517+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94633420675840] v29'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff770026040 con 0x7ff76c076040 2024-05-08T18:06:52.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.517+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c7c50 con 0x56118f6ba2e0 2024-05-08T18:06:52.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.518+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94633420679392] v29'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.529 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.518+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.518+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94633420676896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c71b0 con 0x56118f6ba2e0 2024-05-08T18:06:52.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.518+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c8520 con 0x56118f6ba2e0 2024-05-08T18:06:52.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.519+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94633420676896] v29'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.519+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.519+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94633420680448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c8d70 con 0x56118f6ba2e0 2024-05-08T18:06:52.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.519+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c9730 con 0x56118f6ba2e0 2024-05-08T18:06:52.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.520+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94633420680448] v29'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.520+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.521+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94633420688032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c9730 con 0x56118f6ba2e0 2024-05-08T18:06:52.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.521+0000 7ff77bb2b800 20 add_watcher() i=0 2024-05-08T18:06:52.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.521+0000 7ff77bb2b800 20 add_watcher() i=1 2024-05-08T18:06:52.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.521+0000 7ff77bb2b800 20 add_watcher() i=2 2024-05-08T18:06:52.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.521+0000 7ff77bb2b800 20 add_watcher() i=3 2024-05-08T18:06:52.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.521+0000 7ff77bb2b800 20 add_watcher() i=4 2024-05-08T18:06:52.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.521+0000 7ff77bb2b800 20 add_watcher() i=5 2024-05-08T18:06:52.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.521+0000 7ff77bb2b800 20 add_watcher() i=6 2024-05-08T18:06:52.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.522+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94633420688032] v29'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.522+0000 7ff77bb2b800 20 add_watcher() i=7 2024-05-08T18:06:52.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.522+0000 7ff77bb2b800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:52.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.522+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f6cca40 con 0x56118f6b5c70 2024-05-08T18:06:52.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.522+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.522+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94633420690416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c9cc0 con 0x56118f6b5c70 2024-05-08T18:06:52.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.523+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94633420690416] v29'263 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.523+0000 7ff62dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:52.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.523+0000 7ff77bb2b800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:52.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.523+0000 7ff77bb2b800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:52.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.523+0000 7ff77bb2b800 5 note: GC not initialized 2024-05-08T18:06:52.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.524+0000 7ff62cfe1640 20 reqs_thread_entry: start 2024-05-08T18:06:52.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.524+0000 7ff77bb2b800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:52.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.524+0000 7ff77bb2b800 20 Filter name: none 2024-05-08T18:06:52.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.524+0000 7ff5a67fc640 20 reqs_thread_entry: start 2024-05-08T18:06:52.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.524+0000 7ff77bb2b800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-05-08T18:06:52.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.524+0000 7ff77bb2b800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.524+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f6ffff0 con 0x56118f6ba2e0 2024-05-08T18:06:52.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.525+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(62 ada [call out=48b,read 0~503 out=503b,getxattrs out=75b] v0'0 uv10 ondisk = 0) ==== 231+0+626 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.525+0000 7ff77bb2b800 20 rados_obj.operate() r=0 bl.length=503 2024-05-08T18:06:52.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.525+0000 7ff77bb2b800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-05-08T18:06:52.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.525+0000 7ff77bb2b800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.525+0000 7ff77bb2b800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:52.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.525+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~460 in=460b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f706a60 con 0x56118f6ba2e0 2024-05-08T18:06:52.532 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.526+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~460] v29'11 uv11 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.526+0000 7ff77bb2b800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:52.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.526+0000 7ff77bb2b800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.526+0000 7ff77bb2b800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-05-08T18:06:52.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.526+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94633420940320 in=648b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56118f702080 con 0x56118f6b5c70 2024-05-08T18:06:52.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.526+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== watch-notify(notify (1) cookie 94633420679392 notify 124554051623 ret 0) ==== 678+0+0 (crc 0 0 0) 0x7ff770010040 con 0x56118f6b5c70 2024-05-08T18:06:52.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.526+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 notify.4 [notify cookie 94633420940320 out=8b] v0'0 uv267 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.526+0000 7ff65a7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051623 cookie 94633420679392 notifier 4886 bl.length()=636 2024-05-08T18:06:52.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.526+0000 7ff65a7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:52.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.526+0000 7ff65a7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.526+0000 7ff65a7fc640 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7ff5ac003430 con 0x56118f6b5c70 2024-05-08T18:06:52.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.527+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== watch-notify(notify_complete (2) cookie 94633420940320 notify 124554051623 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7ff77000a840 con 0x56118f6b5c70 2024-05-08T18:06:52.533 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.527+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.534 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-05-08T18:06:51.092020Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:06:51.092006Z","tags":[],"group_ids":[]} 2024-05-08T18:06:52.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.528+0000 7ff62dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:52.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.528+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94633420690416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6b1bd0 con 0x56118f6b5c70 2024-05-08T18:06:52.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.529+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94633420690416] v29'264 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.529+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94633420664880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c93f0 con 0x7ff76c076040 2024-05-08T18:06:52.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.530+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94633420664880] v29'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff770026040 con 0x7ff76c076040 2024-05-08T18:06:52.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.530+0000 7ff77bb2b800 20 remove_watcher() i=0 2024-05-08T18:06:52.539 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.530+0000 7ff77bb2b800 2 removed watcher, disabling cache 2024-05-08T18:06:52.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.530+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94633420669216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c93f0 con 0x56118f6b5c70 2024-05-08T18:06:52.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.531+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94633420669216] v29'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.531+0000 7ff77bb2b800 20 remove_watcher() i=1 2024-05-08T18:06:52.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.531+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94633420662352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c4120 con 0x56118f6bec70 2024-05-08T18:06:52.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.532+0000 7ff776f4a640 1 -- 172.21.15.50:0/778996298 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94633420662352] v29'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff76c081070 con 0x56118f6bec70 2024-05-08T18:06:52.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.532+0000 7ff77bb2b800 20 remove_watcher() i=2 2024-05-08T18:06:52.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.532+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94633420675840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c93f0 con 0x7ff76c076040 2024-05-08T18:06:52.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.533+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94633420675840] v29'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff770026040 con 0x7ff76c076040 2024-05-08T18:06:52.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.533+0000 7ff77bb2b800 20 remove_watcher() i=3 2024-05-08T18:06:52.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.533+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94633420679392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c2650 con 0x56118f6b5c70 2024-05-08T18:06:52.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.534+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94633420679392] v29'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff770026040 con 0x56118f6b5c70 2024-05-08T18:06:52.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.534+0000 7ff77bb2b800 20 remove_watcher() i=4 2024-05-08T18:06:52.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.534+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94633420676896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c4120 con 0x56118f6ba2e0 2024-05-08T18:06:52.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.535+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94633420676896] v29'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.535+0000 7ff77bb2b800 20 remove_watcher() i=5 2024-05-08T18:06:52.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.535+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94633420680448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c93f0 con 0x56118f6ba2e0 2024-05-08T18:06:52.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.536+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94633420680448] v29'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.536+0000 7ff77bb2b800 20 remove_watcher() i=6 2024-05-08T18:06:52.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.536+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94633420688032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56118f6c2650 con 0x56118f6ba2e0 2024-05-08T18:06:52.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77774b640 1 -- 172.21.15.50:0/778996298 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94633420688032] v29'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff76805b030 con 0x56118f6ba2e0 2024-05-08T18:06:52.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 20 remove_watcher() i=7 2024-05-08T18:06:52.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56118f6ba2e0 msgr2=0x56118f6bc6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56118f6ba2e0 0x56118f6bc6d0 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56118f6b5c70 msgr2=0x56118f6b8130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56118f6b5c70 0x56118f6b8130 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff76c076040 msgr2=0x7ff76c083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff76c076040 0x7ff76c083410 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56118f6bec70 msgr2=0x56118f6c1130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56118f6bec70 0x56118f6c1130 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff64003bb50 msgr2=0x7ff64003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff64003bb50 0x7ff64003e010 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff76c052fd0 tx=0x7ff76c04f000 comp rx=0 tx=0).stop 2024-05-08T18:06:52.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff777f4c640 1 -- 172.21.15.50:0/778996298 reap_dead start 2024-05-08T18:06:52.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f6a30e0 msgr2=0x56118f6aa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f6a30e0 0x56118f6aa520 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff76801dad0 tx=0x7ff76804c180 comp rx=0 tx=0).stop 2024-05-08T18:06:52.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 shutdown_connections 2024-05-08T18:06:52.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/778996298 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f6a30e0 0x56118f6aa520 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 >> 172.21.15.50:0/778996298 conn(0x56118f69f440 msgr2=0x56118f69caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 shutdown_connections 2024-05-08T18:06:52.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.537+0000 7ff77bb2b800 1 -- 172.21.15.50:0/778996298 wait complete. 2024-05-08T18:06:52.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 -- 172.21.15.50:0/2102360487 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff65003bb50 msgr2=0x7ff65003e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/2102360487 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff65003bb50 0x7ff65003e010 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x56118f67f820 tx=0x7ff76c005a00 comp rx=0 tx=0).stop 2024-05-08T18:06:52.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 -- 172.21.15.50:0/2102360487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f695fa0 msgr2=0x56118f6963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/2102360487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f695fa0 0x56118f6963c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff7680513e0 tx=0x7ff7680108f0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 -- 172.21.15.50:0/2102360487 shutdown_connections 2024-05-08T18:06:52.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/2102360487 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff65003bb50 0x7ff65003e010 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/2102360487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f695fa0 0x56118f6963c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 -- 172.21.15.50:0/2102360487 >> 172.21.15.50:0/2102360487 conn(0x56118f68a040 msgr2=0x56118f68a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 -- 172.21.15.50:0/2102360487 shutdown_connections 2024-05-08T18:06:52.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 -- 172.21.15.50:0/2102360487 wait complete. 2024-05-08T18:06:52.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 -- 172.21.15.50:0/666452284 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56118f681270 msgr2=0x56118f683660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56118f681270 0x56118f683660 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 -- 172.21.15.50:0/666452284 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56118f57cb10 msgr2=0x56118f67e9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56118f57cb10 0x56118f67e9b0 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 -- 172.21.15.50:0/666452284 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56118f6851b0 msgr2=0x56118f6875a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56118f6851b0 0x56118f6875a0 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 -- 172.21.15.50:0/666452284 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff65c03bb50 msgr2=0x7ff65c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.543 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff65c03bb50 0x7ff65c03e010 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff76c054000 tx=0x7ff76c04e000 comp rx=0 tx=0).stop 2024-05-08T18:06:52.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 -- 172.21.15.50:0/666452284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57afd0 msgr2=0x56118f677770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 --2- 172.21.15.50:0/666452284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56118f57afd0 0x56118f677770 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x56118f6709b0 tx=0x7ff768004460 comp rx=0 tx=0).stop 2024-05-08T18:06:52.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff777f4c640 1 -- 172.21.15.50:0/666452284 reap_dead start 2024-05-08T18:06:52.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 -- 172.21.15.50:0/666452284 shutdown_connections 2024-05-08T18:06:52.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 -- 172.21.15.50:0/666452284 >> 172.21.15.50:0/666452284 conn(0x56118f576af0 msgr2=0x56118f669d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 -- 172.21.15.50:0/666452284 shutdown_connections 2024-05-08T18:06:52.544 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.538+0000 7ff77bb2b800 1 -- 172.21.15.50:0/666452284 wait complete. 2024-05-08T18:06:52.546 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:52.546 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:52.546 ERROR:root:0 2024-05-08T18:06:52.546 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=write'] 2024-05-08T18:06:52.546 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=write'] 2024-05-08T18:06:52.547 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps user-info-without-keys=write 2024-05-08T18:06:52.593 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:52.594 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:52.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.596+0000 7f482efaf800 1 Processor -- start 2024-05-08T18:06:52.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.597+0000 7f482efaf800 1 -- start start 2024-05-08T18:06:52.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.597+0000 7f482efaf800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5632ebb35b80 0x5632ebb37f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.597+0000 7f482efaf800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5632ebb38540 con 0x5632ebb2f960 2024-05-08T18:06:52.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.597+0000 7f482efaf800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x5632eb9d1e20 con 0x5632ebb35b80 2024-05-08T18:06:52.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.597+0000 7f482c2ad640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x5632ebb35b80 0x5632ebb37f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.597+0000 7f482caae640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5632ebb2f960 0x5632ebb2fd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50908/0 (socket says 172.21.15.50:50908) 2024-05-08T18:06:52.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.597+0000 7f482caae640 1 -- 172.21.15.50:0/316690474 learned_addr learned my addr 172.21.15.50:0/316690474 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:52.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.597+0000 7f482c2ad640 1 -- 172.21.15.50:0/316690474 >> v1:172.21.15.46:6789/0 conn(0x5632ebb2f960 legacy=0x5632ebb2fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.597+0000 7f482c2ad640 1 -- 172.21.15.50:0/316690474 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ebb39720 con 0x5632ebb35b80 2024-05-08T18:06:52.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.599+0000 7f482c2ad640 1 --2- 172.21.15.50:0/316690474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb35b80 0x5632ebb37f70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f482000a470 tx=0x7f482001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c77a96bf64aa2aa server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.599+0000 7f482baac640 1 -- 172.21.15.50:0/316690474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4820057070 con 0x5632ebb35b80 2024-05-08T18:06:52.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.599+0000 7f482baac640 1 -- 172.21.15.50:0/316690474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4820052570 con 0x5632ebb35b80 2024-05-08T18:06:52.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.599+0000 7f482baac640 1 -- 172.21.15.50:0/316690474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4820052870 con 0x5632ebb35b80 2024-05-08T18:06:52.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.600+0000 7f482efaf800 1 -- 172.21.15.50:0/316690474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb35b80 msgr2=0x5632ebb37f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.600+0000 7f482efaf800 1 --2- 172.21.15.50:0/316690474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb35b80 0x5632ebb37f70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f482000a470 tx=0x7f482001ce90 comp rx=0 tx=0).stop 2024-05-08T18:06:52.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.600+0000 7f482efaf800 1 -- 172.21.15.50:0/316690474 shutdown_connections 2024-05-08T18:06:52.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.600+0000 7f482efaf800 1 --2- 172.21.15.50:0/316690474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb35b80 0x5632ebb37f70 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.600+0000 7f482efaf800 1 -- 172.21.15.50:0/316690474 >> 172.21.15.50:0/316690474 conn(0x5632ebb2b2f0 msgr2=0x5632ebb2d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.600+0000 7f482efaf800 1 -- 172.21.15.50:0/316690474 shutdown_connections 2024-05-08T18:06:52.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.600+0000 7f482efaf800 1 -- 172.21.15.50:0/316690474 wait complete. 2024-05-08T18:06:52.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.615+0000 7f482efaf800 1 Processor -- start 2024-05-08T18:06:52.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.615+0000 7f482efaf800 1 -- start start 2024-05-08T18:06:52.613 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.615+0000 7f482efaf800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb2f960 0x5632ebc22b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.615+0000 7f482efaf800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5632eba7ca30 con 0x5632ebb2f960 2024-05-08T18:06:52.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.615+0000 7f482caae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb2f960 0x5632ebc22b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.615+0000 7f482caae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb2f960 0x5632ebc22b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46672/0 (socket says 172.21.15.50:46672) 2024-05-08T18:06:52.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.615+0000 7f482caae640 1 -- 172.21.15.50:0/1760933600 learned_addr learned my addr 172.21.15.50:0/1760933600 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.614 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.616+0000 7f482caae640 1 -- 172.21.15.50:0/1760933600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ebc23150 con 0x5632ebb2f960 2024-05-08T18:06:52.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.616+0000 7f482caae640 1 --2- 172.21.15.50:0/1760933600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb2f960 0x5632ebc22b80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f481c001e10 tx=0x7f481c00b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d8b0a77f0c204c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.616+0000 7f471ae0d640 1 -- 172.21.15.50:0/1760933600 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f481c012020 con 0x5632ebb2f960 2024-05-08T18:06:52.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.616+0000 7f471ae0d640 1 -- 172.21.15.50:0/1760933600 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f481c00e5a0 con 0x5632ebb2f960 2024-05-08T18:06:52.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.616+0000 7f471ae0d640 1 -- 172.21.15.50:0/1760933600 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f481c00e8a0 con 0x5632ebb2f960 2024-05-08T18:06:52.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.617+0000 7f482efaf800 1 -- 172.21.15.50:0/1760933600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb2f960 msgr2=0x5632ebc22b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.617+0000 7f482efaf800 1 --2- 172.21.15.50:0/1760933600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb2f960 0x5632ebc22b80 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f481c001e10 tx=0x7f481c00b780 comp rx=0 tx=0).stop 2024-05-08T18:06:52.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.617+0000 7f482efaf800 1 -- 172.21.15.50:0/1760933600 shutdown_connections 2024-05-08T18:06:52.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.617+0000 7f482efaf800 1 --2- 172.21.15.50:0/1760933600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb2f960 0x5632ebc22b80 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.617+0000 7f482efaf800 1 -- 172.21.15.50:0/1760933600 >> 172.21.15.50:0/1760933600 conn(0x5632ebb2b2f0 msgr2=0x5632ebc1e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.617+0000 7f482efaf800 1 -- 172.21.15.50:0/1760933600 shutdown_connections 2024-05-08T18:06:52.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.617+0000 7f482efaf800 1 -- 172.21.15.50:0/1760933600 wait complete. 2024-05-08T18:06:52.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.617+0000 7f482efaf800 1 Processor -- start 2024-05-08T18:06:52.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.617+0000 7f482efaf800 1 -- start start 2024-05-08T18:06:52.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.617+0000 7f482efaf800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb2f960 0x5632ebb32240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.617+0000 7f482efaf800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5632ebb389e0 con 0x5632ebb2f960 2024-05-08T18:06:52.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.618+0000 7f482caae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb2f960 0x5632ebb32240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.618+0000 7f482caae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb2f960 0x5632ebb32240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46686/0 (socket says 172.21.15.50:46686) 2024-05-08T18:06:52.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.618+0000 7f482caae640 1 -- 172.21.15.50:0/3561446656 learned_addr learned my addr 172.21.15.50:0/3561446656 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.618+0000 7f482caae640 1 -- 172.21.15.50:0/3561446656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ebb32780 con 0x5632ebb2f960 2024-05-08T18:06:52.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.618+0000 7f482caae640 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb2f960 0x5632ebb32240 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x5632ebc23910 tx=0x7f481c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.618+0000 7f471960a640 1 -- 172.21.15.50:0/3561446656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f481c012030 con 0x5632ebb2f960 2024-05-08T18:06:52.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.618+0000 7f471960a640 1 -- 172.21.15.50:0/3561446656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f481c001470 con 0x5632ebb2f960 2024-05-08T18:06:52.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.618+0000 7f482efaf800 1 -- 172.21.15.50:0/3561446656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5632ebb30980 con 0x5632ebb2f960 2024-05-08T18:06:52.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.618+0000 7f471960a640 1 -- 172.21.15.50:0/3561446656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f481c01a7a0 con 0x5632ebb2f960 2024-05-08T18:06:52.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.618+0000 7f482efaf800 1 -- 172.21.15.50:0/3561446656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632ebb30ea0 con 0x5632ebb2f960 2024-05-08T18:06:52.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.619+0000 7f471960a640 1 -- 172.21.15.50:0/3561446656 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f481c015020 con 0x5632ebb2f960 2024-05-08T18:06:52.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.619+0000 7f471960a640 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f471403bb50 0x7f471403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.619+0000 7f471960a640 1 -- 172.21.15.50:0/3561446656 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f481c020030 con 0x5632ebb2f960 2024-05-08T18:06:52.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.619+0000 7f482c2ad640 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f471403bb50 0x7f471403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.620+0000 7f482efaf800 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632ebb35b80 0x5632ebc33b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.620+0000 7f482efaf800 1 -- 172.21.15.50:0/3561446656 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc340b0 con 0x5632ebb35b80 2024-05-08T18:06:52.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.620+0000 7f482d2af640 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632ebb35b80 0x5632ebc33b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.620+0000 7f482c2ad640 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f471403bb50 0x7f471403e010 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4820054000 tx=0x7f482004e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.620+0000 7f482d2af640 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632ebb35b80 0x5632ebc33b70 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.620+0000 7f482d2af640 1 -- 172.21.15.50:0/3561446656 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f482400a4a0 con 0x5632ebb35b80 2024-05-08T18:06:52.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.620+0000 7f482efaf800 1 -- 172.21.15.50:0/3561446656 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebb313a0 con 0x5632ebb35b80 2024-05-08T18:06:52.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.620+0000 7f482d2af640 1 -- 172.21.15.50:0/3561446656 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f482400a4a0 con 0x5632ebb35b80 2024-05-08T18:06:52.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.621+0000 7f482efaf800 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632ebc363c0 0x5632ebc387b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.621+0000 7f482efaf800 1 -- 172.21.15.50:0/3561446656 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc38e50 con 0x5632ebc363c0 2024-05-08T18:06:52.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.621+0000 7f482caae640 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632ebc363c0 0x5632ebc387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.621+0000 7f482caae640 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632ebc363c0 0x5632ebc387b0 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.621+0000 7f482caae640 1 -- 172.21.15.50:0/3561446656 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f481c018430 con 0x5632ebc363c0 2024-05-08T18:06:52.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.621+0000 7f482efaf800 1 -- 172.21.15.50:0/3561446656 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc35260 con 0x5632ebb35b80 2024-05-08T18:06:52.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.622+0000 7f482d2af640 1 -- 172.21.15.50:0/3561446656 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f482400a4a0 con 0x5632ebb35b80 2024-05-08T18:06:52.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.622+0000 7f482efaf800 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632ebc3a2b0 0x5632ebc3c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.622+0000 7f482efaf800 1 -- 172.21.15.50:0/3561446656 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc3cda0 con 0x5632ebc3a2b0 2024-05-08T18:06:52.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.622+0000 7f482c2ad640 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632ebc3a2b0 0x5632ebc3c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.622+0000 7f482c2ad640 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632ebc3a2b0 0x5632ebc3c6a0 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.622+0000 7f482c2ad640 1 -- 172.21.15.50:0/3561446656 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4820052a40 con 0x5632ebc3a2b0 2024-05-08T18:06:52.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.623+0000 7f482efaf800 1 Processor -- start 2024-05-08T18:06:52.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.623+0000 7f482efaf800 1 -- start start 2024-05-08T18:06:52.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.623+0000 7f482efaf800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc4b0d0 0x5632ebc4b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.623+0000 7f482efaf800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5632eba23330 con 0x5632ebc4b0d0 2024-05-08T18:06:52.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.623+0000 7f482caae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc4b0d0 0x5632ebc4b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.623+0000 7f482caae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc4b0d0 0x5632ebc4b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46690/0 (socket says 172.21.15.50:46690) 2024-05-08T18:06:52.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.623+0000 7f482caae640 1 -- 172.21.15.50:0/4123254081 learned_addr learned my addr 172.21.15.50:0/4123254081 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.623+0000 7f482caae640 1 -- 172.21.15.50:0/4123254081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ebc4ba30 con 0x5632ebc4b0d0 2024-05-08T18:06:52.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.624+0000 7f482caae640 1 --2- 172.21.15.50:0/4123254081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc4b0d0 0x5632ebc4b4f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f481c050f80 tx=0x7f481c01af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.624+0000 7f47127fc640 1 -- 172.21.15.50:0/4123254081 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f481c00cc70 con 0x5632ebc4b0d0 2024-05-08T18:06:52.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.624+0000 7f47127fc640 1 -- 172.21.15.50:0/4123254081 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f481c047030 con 0x5632ebc4b0d0 2024-05-08T18:06:52.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.624+0000 7f482efaf800 1 -- 172.21.15.50:0/4123254081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5632ebc4c4a0 con 0x5632ebc4b0d0 2024-05-08T18:06:52.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.624+0000 7f47127fc640 1 -- 172.21.15.50:0/4123254081 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f481c016570 con 0x5632ebc4b0d0 2024-05-08T18:06:52.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.624+0000 7f482efaf800 1 -- 172.21.15.50:0/4123254081 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632ebc4c4a0 con 0x5632ebc4b0d0 2024-05-08T18:06:52.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.625+0000 7f47127fc640 1 -- 172.21.15.50:0/4123254081 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f481c016710 con 0x5632ebc4b0d0 2024-05-08T18:06:52.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.625+0000 7f47127fc640 1 --2- 172.21.15.50:0/4123254081 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f470403bbf0 0x7f470403e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.625+0000 7f47127fc640 1 -- 172.21.15.50:0/4123254081 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f481c03ba70 con 0x5632ebc4b0d0 2024-05-08T18:06:52.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.625+0000 7f482c2ad640 1 --2- 172.21.15.50:0/4123254081 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f470403bbf0 0x7f470403e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.625+0000 7f482efaf800 1 Processor -- start 2024-05-08T18:06:52.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.625+0000 7f482efaf800 1 -- start start 2024-05-08T18:06:52.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.625+0000 7f482c2ad640 1 --2- 172.21.15.50:0/4123254081 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f470403bbf0 0x7f470403e0b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5632ebb312c0 tx=0x7f4820005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.625+0000 7f482efaf800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc580e0 0x5632ebc58500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.625+0000 7f482efaf800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5632ebc234d0 con 0x5632ebc580e0 2024-05-08T18:06:52.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.626+0000 7f482caae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc580e0 0x5632ebc58500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.626+0000 7f482caae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc580e0 0x5632ebc58500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46698/0 (socket says 172.21.15.50:46698) 2024-05-08T18:06:52.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.626+0000 7f482caae640 1 -- 172.21.15.50:0/689660205 learned_addr learned my addr 172.21.15.50:0/689660205 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.626+0000 7f482caae640 1 -- 172.21.15.50:0/689660205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ebc58ad0 con 0x5632ebc580e0 2024-05-08T18:06:52.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.626+0000 7f482caae640 1 --2- 172.21.15.50:0/689660205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc580e0 0x5632ebc58500 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f481c016fb0 tx=0x7f481c0256d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a2e894a01e9090 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.626+0000 7f4703fff640 1 -- 172.21.15.50:0/689660205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f481c023d10 con 0x5632ebc580e0 2024-05-08T18:06:52.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.626+0000 7f4703fff640 1 -- 172.21.15.50:0/689660205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f481c023eb0 con 0x5632ebc580e0 2024-05-08T18:06:52.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.626+0000 7f4703fff640 1 -- 172.21.15.50:0/689660205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f481c049420 con 0x5632ebc580e0 2024-05-08T18:06:52.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.627+0000 7f482efaf800 1 -- 172.21.15.50:0/689660205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc580e0 msgr2=0x5632ebc58500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.627+0000 7f482efaf800 1 --2- 172.21.15.50:0/689660205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc580e0 0x5632ebc58500 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f481c016fb0 tx=0x7f481c0256d0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.627+0000 7f482efaf800 1 -- 172.21.15.50:0/689660205 shutdown_connections 2024-05-08T18:06:52.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.627+0000 7f482efaf800 1 --2- 172.21.15.50:0/689660205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc580e0 0x5632ebc58500 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.627+0000 7f482efaf800 1 -- 172.21.15.50:0/689660205 >> 172.21.15.50:0/689660205 conn(0x5632ebc543b0 msgr2=0x5632ebc52410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.627+0000 7f482efaf800 1 -- 172.21.15.50:0/689660205 shutdown_connections 2024-05-08T18:06:52.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.627+0000 7f482efaf800 1 -- 172.21.15.50:0/689660205 wait complete. 2024-05-08T18:06:52.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.627+0000 7f482efaf800 1 Processor -- start 2024-05-08T18:06:52.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.627+0000 7f482efaf800 1 -- start start 2024-05-08T18:06:52.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.627+0000 7f482efaf800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc580e0 0x5632ebc5f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.627+0000 7f482efaf800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5632ebc591a0 con 0x5632ebc580e0 2024-05-08T18:06:52.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.628+0000 7f482caae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc580e0 0x5632ebc5f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.628+0000 7f482caae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc580e0 0x5632ebc5f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46708/0 (socket says 172.21.15.50:46708) 2024-05-08T18:06:52.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.628+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 learned_addr learned my addr 172.21.15.50:0/1821204534 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.628+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632ebc5f9c0 con 0x5632ebc580e0 2024-05-08T18:06:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.628+0000 7f482caae640 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc580e0 0x5632ebc5f480 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f481c016fb0 tx=0x7f481c044ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.628+0000 7f47027fc640 1 -- 172.21.15.50:0/1821204534 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f481c06e070 con 0x5632ebc580e0 2024-05-08T18:06:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.628+0000 7f47027fc640 1 -- 172.21.15.50:0/1821204534 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f481c04a100 con 0x5632ebc580e0 2024-05-08T18:06:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.628+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5632ebc5f9c0 con 0x5632ebc580e0 2024-05-08T18:06:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.628+0000 7f47027fc640 1 -- 172.21.15.50:0/1821204534 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f481c049660 con 0x5632ebc580e0 2024-05-08T18:06:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.628+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632ebc5fe60 con 0x5632ebc580e0 2024-05-08T18:06:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.629+0000 7f47027fc640 1 -- 172.21.15.50:0/1821204534 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f481c071070 con 0x5632ebc580e0 2024-05-08T18:06:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.629+0000 7f47027fc640 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f46f803bba0 0x7f46f803e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.629+0000 7f47027fc640 1 -- 172.21.15.50:0/1821204534 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f481c05a070 con 0x5632ebc580e0 2024-05-08T18:06:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.629+0000 7f482c2ad640 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f46f803bba0 0x7f46f803e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.630+0000 7f482c2ad640 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f46f803bba0 0x7f46f803e060 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4820052fd0 tx=0x7f482004f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.630+0000 7f482efaf800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.630+0000 7f482efaf800 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632ebc6aab0 0x5632ebc6cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.638 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.630+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc6d6c0 con 0x5632ebc6aab0 2024-05-08T18:06:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.630+0000 7f482d2af640 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632ebc6aab0 0x5632ebc6cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.631+0000 7f482d2af640 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632ebc6aab0 0x5632ebc6cf70 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.631+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f482400a4a0 con 0x5632ebc6aab0 2024-05-08T18:06:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.631+0000 7f482efaf800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.631+0000 7f482efaf800 20 realm 2024-05-08T18:06:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.631+0000 7f482efaf800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.631+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc6dad0 con 0x5632ebc6aab0 2024-05-08T18:06:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.631+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f482400a4a0 con 0x5632ebc6aab0 2024-05-08T18:06:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.631+0000 7f482efaf800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.631+0000 7f482efaf800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.631+0000 7f482efaf800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.631+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc6e3a0 con 0x5632ebc6aab0 2024-05-08T18:06:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.631+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f482400a4a0 con 0x5632ebc6aab0 2024-05-08T18:06:52.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.631+0000 7f482efaf800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.631+0000 7f482efaf800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.631+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc6e740 con 0x5632ebc6aab0 2024-05-08T18:06:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.632+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f482400a4a0 con 0x5632ebc6aab0 2024-05-08T18:06:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.632+0000 7f482efaf800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.632+0000 7f482efaf800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.632+0000 7f482efaf800 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632ebc6f4a0 0x5632ebc71890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.632+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc71ea0 con 0x5632ebc6f4a0 2024-05-08T18:06:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.632+0000 7f482caae640 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632ebc6f4a0 0x5632ebc71890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.632+0000 7f482caae640 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632ebc6f4a0 0x5632ebc71890 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.632+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.632+0000 7f482efaf800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.632+0000 7f482efaf800 20 searching for the correct realm 2024-05-08T18:06:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.632+0000 7f482efaf800 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632ebc73e50 0x5632ebc76310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.633+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc769e0 con 0x5632ebc73e50 2024-05-08T18:06:52.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.633+0000 7f482c2ad640 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632ebc73e50 0x5632ebc76310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.633+0000 7f482c2ad640 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632ebc73e50 0x5632ebc76310 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.633+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4820081070 con 0x5632ebc73e50 2024-05-08T18:06:52.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.633+0000 7f482c2ad640 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f4820076040 0x7f4820083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.633+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc769e0 con 0x7f4820076040 2024-05-08T18:06:52.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.633+0000 7f482d2af640 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f4820076040 0x7f4820083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.634+0000 7f482d2af640 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f4820076040 0x7f4820083410 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.634+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f482400a4a0 con 0x7f4820076040 2024-05-08T18:06:52.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.634+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc6e740 con 0x5632ebc6f4a0 2024-05-08T18:06:52.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.634+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.634+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc6e740 con 0x5632ebc6aab0 2024-05-08T18:06:52.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.634+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f482400a4a0 con 0x5632ebc6aab0 2024-05-08T18:06:52.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.634+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc6e740 con 0x5632ebc6aab0 2024-05-08T18:06:52.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.635+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f482400a4a0 con 0x5632ebc6aab0 2024-05-08T18:06:52.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.635+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc6e740 con 0x5632ebc6aab0 2024-05-08T18:06:52.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.635+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f482400a4a0 con 0x5632ebc6aab0 2024-05-08T18:06:52.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.635+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc6e740 con 0x5632ebc6f4a0 2024-05-08T18:06:52.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.635+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v29'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.635+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc6e740 con 0x7f4820076040 2024-05-08T18:06:52.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.635+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f482400a4a0 con 0x7f4820076040 2024-05-08T18:06:52.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.635+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc6e740 con 0x5632ebc73e50 2024-05-08T18:06:52.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.635+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4820081070 con 0x5632ebc73e50 2024-05-08T18:06:52.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.636+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc6e740 con 0x5632ebc6aab0 2024-05-08T18:06:52.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.636+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f482400a4a0 con 0x5632ebc6aab0 2024-05-08T18:06:52.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.636+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc6e740 con 0x5632ebc6f4a0 2024-05-08T18:06:52.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.636+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.636+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc6e740 con 0x5632ebc6aab0 2024-05-08T18:06:52.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.636+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f482400a4a0 con 0x5632ebc6aab0 2024-05-08T18:06:52.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.636+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc6e740 con 0x7f4820076040 2024-05-08T18:06:52.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.636+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f4824032040 con 0x7f4820076040 2024-05-08T18:06:52.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.636+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc6e740 con 0x5632ebc73e50 2024-05-08T18:06:52.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.637+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4820081070 con 0x5632ebc73e50 2024-05-08T18:06:52.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.637+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x5632ebc73e50 2024-05-08T18:06:52.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.637+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4820081070 con 0x5632ebc73e50 2024-05-08T18:06:52.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.637+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x5632ebc6f4a0 2024-05-08T18:06:52.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.637+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.637+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x5632ebc73e50 2024-05-08T18:06:52.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.637+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4820081070 con 0x5632ebc73e50 2024-05-08T18:06:52.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.637+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x5632ebc73e50 2024-05-08T18:06:52.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.638+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f4820081070 con 0x5632ebc73e50 2024-05-08T18:06:52.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.638+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x5632ebc6aab0 2024-05-08T18:06:52.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.638+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.638+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x5632ebc6aab0 2024-05-08T18:06:52.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.638+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v29'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.638+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x5632ebc6f4a0 2024-05-08T18:06:52.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.638+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.638+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x5632ebc73e50 2024-05-08T18:06:52.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.639+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4820081070 con 0x5632ebc73e50 2024-05-08T18:06:52.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.639+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x7f4820076040 2024-05-08T18:06:52.655 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.639+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4824032040 con 0x7f4820076040 2024-05-08T18:06:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.639+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x5632ebc6aab0 2024-05-08T18:06:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.639+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.639+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x5632ebc6f4a0 2024-05-08T18:06:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.639+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.639+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x5632ebc73e50 2024-05-08T18:06:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.640+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4820081070 con 0x5632ebc73e50 2024-05-08T18:06:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.640+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x5632ebc6f4a0 2024-05-08T18:06:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.640+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.640+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x7f4820076040 2024-05-08T18:06:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.640+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4824032040 con 0x7f4820076040 2024-05-08T18:06:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.640+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x5632ebc73e50 2024-05-08T18:06:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.640+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4820081070 con 0x5632ebc73e50 2024-05-08T18:06:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.640+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x5632ebc6f4a0 2024-05-08T18:06:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.641+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.641+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x5632ebc6aab0 2024-05-08T18:06:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.641+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v29'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.641+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f4820090040 con 0x7f4820076040 2024-05-08T18:06:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.641+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4824032040 con 0x7f4820076040 2024-05-08T18:06:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.641+0000 7f482efaf800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.641+0000 7f482efaf800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.641+0000 7f482efaf800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.641+0000 7f482efaf800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.641+0000 7f482efaf800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.641+0000 7f482efaf800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.641+0000 7f482efaf800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.641+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc72760 con 0x5632ebc6aab0 2024-05-08T18:06:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc72760 con 0x5632ebc6aab0 2024-05-08T18:06:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc72760 con 0x5632ebc73e50 2024-05-08T18:06:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f4820081070 con 0x5632ebc73e50 2024-05-08T18:06:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 20 zone default found 2024-05-08T18:06:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 4 Realm: () 2024-05-08T18:06:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 20 zonegroup default 2024-05-08T18:06:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc730f0 con 0x5632ebc6f4a0 2024-05-08T18:06:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.642+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc72ca0 con 0x5632ebc73e50 2024-05-08T18:06:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.643+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4820081070 con 0x5632ebc73e50 2024-05-08T18:06:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.643+0000 7f482efaf800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.643+0000 7f482efaf800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.643+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc78d90 con 0x5632ebc73e50 2024-05-08T18:06:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.643+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4820081070 con 0x5632ebc73e50 2024-05-08T18:06:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.643+0000 7f482efaf800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.643+0000 7f482efaf800 20 started sync module instance, tier type = 2024-05-08T18:06:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.643+0000 7f482efaf800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.643+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc78180 con 0x7f4820076040 2024-05-08T18:06:52.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.644+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4824032040 con 0x7f4820076040 2024-05-08T18:06:52.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.644+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94776704075440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc78180 con 0x7f4820076040 2024-05-08T18:06:52.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.644+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc79810 con 0x5632ebc6aab0 2024-05-08T18:06:52.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.645+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94776704075440] v29'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4824032040 con 0x7f4820076040 2024-05-08T18:06:52.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.645+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.645+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94776704079168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc78180 con 0x5632ebc6aab0 2024-05-08T18:06:52.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.645+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc7a2a0 con 0x5632ebc73e50 2024-05-08T18:06:52.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.646+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94776704079168] v29'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.646+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4820081070 con 0x5632ebc73e50 2024-05-08T18:06:52.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.646+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94776704072912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc780f0 con 0x5632ebc73e50 2024-05-08T18:06:52.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.647+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc7b160 con 0x7f4820076040 2024-05-08T18:06:52.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.647+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94776704072912] v29'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4820081070 con 0x5632ebc73e50 2024-05-08T18:06:52.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.648+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4824032040 con 0x7f4820076040 2024-05-08T18:06:52.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.648+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94776704085168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc7b580 con 0x7f4820076040 2024-05-08T18:06:52.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.648+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc7bf40 con 0x5632ebc6aab0 2024-05-08T18:06:52.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.649+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.649+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94776704088720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc7c360 con 0x5632ebc6aab0 2024-05-08T18:06:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.649+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc7cda0 con 0x5632ebc6f4a0 2024-05-08T18:06:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.649+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94776704085168] v29'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4824032040 con 0x7f4820076040 2024-05-08T18:06:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.650+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94776704088720] v29'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.650+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.650+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94776704086224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc7c360 con 0x5632ebc6f4a0 2024-05-08T18:06:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.650+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc7d670 con 0x5632ebc6f4a0 2024-05-08T18:06:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.651+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(56 notify.6 [create] v29'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.651+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94776704096784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc7e2e0 con 0x5632ebc6f4a0 2024-05-08T18:06:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.651+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc7ed30 con 0x5632ebc6f4a0 2024-05-08T18:06:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.651+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94776704086224] v29'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.652+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94776704096784] v29'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.652+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.661 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.652+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94776704097840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc7bef0 con 0x5632ebc6f4a0 2024-05-08T18:06:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.652+0000 7f482efaf800 20 add_watcher() i=0 2024-05-08T18:06:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.652+0000 7f482efaf800 20 add_watcher() i=1 2024-05-08T18:06:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.652+0000 7f482efaf800 20 add_watcher() i=2 2024-05-08T18:06:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.652+0000 7f482efaf800 20 add_watcher() i=3 2024-05-08T18:06:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.652+0000 7f482efaf800 20 add_watcher() i=4 2024-05-08T18:06:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.652+0000 7f482efaf800 20 add_watcher() i=5 2024-05-08T18:06:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.652+0000 7f482efaf800 20 add_watcher() i=6 2024-05-08T18:06:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.653+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94776704097840] v29'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.653+0000 7f482efaf800 20 add_watcher() i=7 2024-05-08T18:06:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.653+0000 7f482efaf800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.653+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc81fd0 con 0x5632ebc6aab0 2024-05-08T18:06:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.653+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.654+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94776704100384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc7f4c0 con 0x5632ebc6aab0 2024-05-08T18:06:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.654+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94776704100384] v29'265 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.655+0000 7f46e2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.655+0000 7f482efaf800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.655+0000 7f482efaf800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.655+0000 7f482efaf800 5 note: GC not initialized 2024-05-08T18:06:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.655+0000 7f46e1fe3640 20 reqs_thread_entry: start 2024-05-08T18:06:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.655+0000 7f482efaf800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.655+0000 7f482efaf800 20 Filter name: none 2024-05-08T18:06:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.655+0000 7f46537fe640 20 reqs_thread_entry: start 2024-05-08T18:06:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.655+0000 7f482efaf800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-05-08T18:06:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.655+0000 7f482efaf800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.655+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebcb54f0 con 0x5632ebc6f4a0 2024-05-08T18:06:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.656+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(62 ada [call out=48b,read 0~460 out=460b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+583 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.656+0000 7f482efaf800 20 rados_obj.operate() r=0 bl.length=460 2024-05-08T18:06:52.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.656+0000 7f482efaf800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-05-08T18:06:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.656+0000 7f482efaf800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.656+0000 7f482efaf800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:52.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.656+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~490 in=490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebcbbbd0 con 0x5632ebc6f4a0 2024-05-08T18:06:52.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.658+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~490] v29'12 uv12 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.658+0000 7f482efaf800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:52.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.658+0000 7f482efaf800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.658+0000 7f482efaf800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-05-08T18:06:52.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.658+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94776704349584 in=678b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5632ebcb7210 con 0x5632ebc6aab0 2024-05-08T18:06:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.658+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== watch-notify(notify (1) cookie 94776704088720 notify 124554051624 ret 0) ==== 708+0+0 (crc 0 0 0) 0x7f482403d040 con 0x5632ebc6aab0 2024-05-08T18:06:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.658+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 notify.4 [notify cookie 94776704349584 out=8b] v0'0 uv270 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.658+0000 7f4703fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051624 cookie 94776704088720 notifier 4893 bl.length()=666 2024-05-08T18:06:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.658+0000 7f4703fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.658+0000 7f4703fff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.658+0000 7f4703fff640 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f46f0003ea0 con 0x5632ebc6aab0 2024-05-08T18:06:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.658+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.658+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== watch-notify(notify_complete (2) cookie 94776704349584 notify 124554051624 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f4824018ba0 con 0x5632ebc6aab0 2024-05-08T18:06:52.666 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-05-08T18:06:51.092020Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"user-info-without-keys","perm":"write"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:06:51.092006Z","tags":[],"group_ids":[]} 2024-05-08T18:06:52.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.659+0000 7f46e2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:52.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.659+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94776704100384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc816b0 con 0x5632ebc6aab0 2024-05-08T18:06:52.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.660+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94776704100384] v29'266 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.660+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94776704075440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc7e820 con 0x7f4820076040 2024-05-08T18:06:52.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.661+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94776704075440] v29'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4824032040 con 0x7f4820076040 2024-05-08T18:06:52.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.661+0000 7f482efaf800 20 remove_watcher() i=0 2024-05-08T18:06:52.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.661+0000 7f482efaf800 2 removed watcher, disabling cache 2024-05-08T18:06:52.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.661+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94776704079168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc7e820 con 0x5632ebc6aab0 2024-05-08T18:06:52.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.663+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94776704079168] v29'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.663+0000 7f482efaf800 20 remove_watcher() i=1 2024-05-08T18:06:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.663+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94776704072912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc79540 con 0x5632ebc73e50 2024-05-08T18:06:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.664+0000 7f482c2ad640 1 -- 172.21.15.50:0/1821204534 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94776704072912] v29'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4820081070 con 0x5632ebc73e50 2024-05-08T18:06:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.664+0000 7f482efaf800 20 remove_watcher() i=2 2024-05-08T18:06:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.664+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94776704085168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc7e820 con 0x7f4820076040 2024-05-08T18:06:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.665+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94776704085168] v29'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4824032040 con 0x7f4820076040 2024-05-08T18:06:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.665+0000 7f482efaf800 20 remove_watcher() i=3 2024-05-08T18:06:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.665+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94776704088720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc77cd0 con 0x5632ebc6aab0 2024-05-08T18:06:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.666+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94776704088720] v29'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4824032040 con 0x5632ebc6aab0 2024-05-08T18:06:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.666+0000 7f482efaf800 20 remove_watcher() i=4 2024-05-08T18:06:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.666+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94776704086224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc79540 con 0x5632ebc6f4a0 2024-05-08T18:06:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.667+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94776704086224] v29'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.667+0000 7f482efaf800 20 remove_watcher() i=5 2024-05-08T18:06:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.667+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94776704096784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc7e820 con 0x5632ebc6f4a0 2024-05-08T18:06:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.668+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94776704096784] v29'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.668+0000 7f482efaf800 20 remove_watcher() i=6 2024-05-08T18:06:52.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.668+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94776704097840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5632ebc77cd0 con 0x5632ebc6f4a0 2024-05-08T18:06:52.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482caae640 1 -- 172.21.15.50:0/1821204534 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94776704097840] v29'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f481c070030 con 0x5632ebc6f4a0 2024-05-08T18:06:52.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 20 remove_watcher() i=7 2024-05-08T18:06:52.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632ebc6f4a0 msgr2=0x5632ebc71890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632ebc6f4a0 0x5632ebc71890 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632ebc6aab0 msgr2=0x5632ebc6cf70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632ebc6aab0 0x5632ebc6cf70 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f4820076040 msgr2=0x7f4820083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f4820076040 0x7f4820083410 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632ebc73e50 msgr2=0x5632ebc76310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632ebc73e50 0x5632ebc76310 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f46f803bba0 msgr2=0x7f46f803e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f46f803bba0 0x7f46f803e060 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4820052fd0 tx=0x7f482004f000 comp rx=0 tx=0).stop 2024-05-08T18:06:52.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc580e0 msgr2=0x5632ebc5f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 1 --2- 172.21.15.50:0/1821204534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc580e0 0x5632ebc5f480 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f481c016fb0 tx=0x7f481c044ee0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 reap_dead start 2024-05-08T18:06:52.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482d2af640 1 -- 172.21.15.50:0/1821204534 reap_dead start 2024-05-08T18:06:52.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 shutdown_connections 2024-05-08T18:06:52.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 >> 172.21.15.50:0/1821204534 conn(0x5632ebc543b0 msgr2=0x5632ebc51a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 shutdown_connections 2024-05-08T18:06:52.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.669+0000 7f482efaf800 1 -- 172.21.15.50:0/1821204534 wait complete. 2024-05-08T18:06:52.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 -- 172.21.15.50:0/4123254081 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f470403bbf0 msgr2=0x7f470403e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 --2- 172.21.15.50:0/4123254081 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f470403bbf0 0x7f470403e0b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x5632ebb312c0 tx=0x7f4820005a00 comp rx=0 tx=0).stop 2024-05-08T18:06:52.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 -- 172.21.15.50:0/4123254081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc4b0d0 msgr2=0x5632ebc4b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 --2- 172.21.15.50:0/4123254081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc4b0d0 0x5632ebc4b4f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f481c050f80 tx=0x7f481c01af10 comp rx=0 tx=0).stop 2024-05-08T18:06:52.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 -- 172.21.15.50:0/4123254081 shutdown_connections 2024-05-08T18:06:52.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 --2- 172.21.15.50:0/4123254081 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f470403bbf0 0x7f470403e0b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 --2- 172.21.15.50:0/4123254081 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebc4b0d0 0x5632ebc4b4f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 -- 172.21.15.50:0/4123254081 >> 172.21.15.50:0/4123254081 conn(0x5632ebc3f170 msgr2=0x5632ebc3f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 -- 172.21.15.50:0/4123254081 shutdown_connections 2024-05-08T18:06:52.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 -- 172.21.15.50:0/4123254081 wait complete. 2024-05-08T18:06:52.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 -- 172.21.15.50:0/3561446656 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632ebc363c0 msgr2=0x5632ebc387b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x5632ebc363c0 0x5632ebc387b0 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 -- 172.21.15.50:0/3561446656 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632ebb35b80 msgr2=0x5632ebc33b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x5632ebb35b80 0x5632ebc33b70 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 -- 172.21.15.50:0/3561446656 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632ebc3a2b0 msgr2=0x5632ebc3c6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x5632ebc3a2b0 0x5632ebc3c6a0 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 -- 172.21.15.50:0/3561446656 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f471403bb50 msgr2=0x7f471403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f471403bb50 0x7f471403e010 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4820054000 tx=0x7f482004e000 comp rx=0 tx=0).stop 2024-05-08T18:06:52.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 -- 172.21.15.50:0/3561446656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb2f960 msgr2=0x5632ebb32240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 --2- 172.21.15.50:0/3561446656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5632ebb2f960 0x5632ebb32240 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x5632ebc23910 tx=0x7f481c000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482d2af640 1 -- 172.21.15.50:0/3561446656 reap_dead start 2024-05-08T18:06:52.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 -- 172.21.15.50:0/3561446656 shutdown_connections 2024-05-08T18:06:52.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.670+0000 7f482efaf800 1 -- 172.21.15.50:0/3561446656 >> 172.21.15.50:0/3561446656 conn(0x5632ebb2b2f0 msgr2=0x5632ebc1cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.671+0000 7f482efaf800 1 -- 172.21.15.50:0/3561446656 shutdown_connections 2024-05-08T18:06:52.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.671+0000 7f482efaf800 1 -- 172.21.15.50:0/3561446656 wait complete. 2024-05-08T18:06:52.679 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': 'write'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:52.679 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': 'write'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:52.679 ERROR:root:0 2024-05-08T18:06:52.679 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:52.684 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'AccessDenied', 'Message': '', 'RequestId': 'tx00000eebdcf89a4152144-00663bbf3c-4221-default', 'HostId': '4221-default-default'} 2024-05-08T18:06:52.684 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'ada', '--caps', 'user-info-without-keys=write'] 2024-05-08T18:06:52.684 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'ada', '--caps', 'user-info-without-keys=write'] 2024-05-08T18:06:52.684 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid ada --caps user-info-without-keys=write 2024-05-08T18:06:52.732 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:52.732 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:52.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.735+0000 7ff15171f800 1 Processor -- start 2024-05-08T18:06:52.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.735+0000 7ff15171f800 1 -- start start 2024-05-08T18:06:52.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.735+0000 7ff15171f800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x557778136950 0x557778136d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.735+0000 7ff15171f800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5577781373a0 con 0x557778139990 2024-05-08T18:06:52.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.735+0000 7ff15171f800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x557777fd8e20 con 0x557778136950 2024-05-08T18:06:52.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.735+0000 7ff14d3e0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x557778136950 0x557778136d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.735+0000 7ff14d3e0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x557778136950 0x557778136d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46716/0 (socket says 172.21.15.50:46716) 2024-05-08T18:06:52.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.735+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3399663874 learned_addr learned my addr 172.21.15.50:0/3399663874 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.735+0000 7ff14cbdf640 1 --1- 172.21.15.50:0/3399663874 >> v1:172.21.15.46:6789/0 conn(0x557778139990 0x557778139e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50920/0 (socket says 172.21.15.50:50920) 2024-05-08T18:06:52.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.735+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3399663874 >> v1:172.21.15.46:6789/0 conn(0x557778139990 legacy=0x557778139e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.736+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3399663874 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777813a550 con 0x557778136950 2024-05-08T18:06:52.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.737+0000 7ff14d3e0640 1 --2- 172.21.15.50:0/3399663874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 0x557778136d70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff14000a520 tx=0x7ff14001d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d1262696fb61a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.738+0000 7ff13ffff640 1 -- 172.21.15.50:0/3399663874 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff140057070 con 0x557778136950 2024-05-08T18:06:52.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.738+0000 7ff13ffff640 1 -- 172.21.15.50:0/3399663874 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff140052570 con 0x557778136950 2024-05-08T18:06:52.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.738+0000 7ff13ffff640 1 -- 172.21.15.50:0/3399663874 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff140052870 con 0x557778136950 2024-05-08T18:06:52.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.738+0000 7ff15171f800 1 -- 172.21.15.50:0/3399663874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 msgr2=0x557778136d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.738+0000 7ff15171f800 1 --2- 172.21.15.50:0/3399663874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 0x557778136d70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff14000a520 tx=0x7ff14001d010 comp rx=0 tx=0).stop 2024-05-08T18:06:52.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.738+0000 7ff15171f800 1 -- 172.21.15.50:0/3399663874 shutdown_connections 2024-05-08T18:06:52.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.738+0000 7ff15171f800 1 --2- 172.21.15.50:0/3399663874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 0x557778136d70 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.738+0000 7ff15171f800 1 -- 172.21.15.50:0/3399663874 >> 172.21.15.50:0/3399663874 conn(0x5577781322f0 msgr2=0x557778134710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.738+0000 7ff15171f800 1 -- 172.21.15.50:0/3399663874 shutdown_connections 2024-05-08T18:06:52.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.738+0000 7ff15171f800 1 -- 172.21.15.50:0/3399663874 wait complete. 2024-05-08T18:06:52.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.753+0000 7ff15171f800 1 Processor -- start 2024-05-08T18:06:52.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.754+0000 7ff15171f800 1 -- start start 2024-05-08T18:06:52.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.754+0000 7ff15171f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 0x557778229c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.754+0000 7ff15171f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557778083a30 con 0x557778136950 2024-05-08T18:06:52.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.754+0000 7ff14d3e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 0x557778229c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.754+0000 7ff14d3e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 0x557778229c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46728/0 (socket says 172.21.15.50:46728) 2024-05-08T18:06:52.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.754+0000 7ff14d3e0640 1 -- 172.21.15.50:0/1465939421 learned_addr learned my addr 172.21.15.50:0/1465939421 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.754+0000 7ff14d3e0640 1 -- 172.21.15.50:0/1465939421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777822a260 con 0x557778136950 2024-05-08T18:06:52.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.754+0000 7ff14d3e0640 1 --2- 172.21.15.50:0/1465939421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 0x557778229c90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ff140055040 tx=0x7ff14004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c700f827055983a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.755+0000 7ff03b606640 1 -- 172.21.15.50:0/1465939421 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff140057040 con 0x557778136950 2024-05-08T18:06:52.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.755+0000 7ff03b606640 1 -- 172.21.15.50:0/1465939421 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff140052870 con 0x557778136950 2024-05-08T18:06:52.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.755+0000 7ff03b606640 1 -- 172.21.15.50:0/1465939421 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff140052b90 con 0x557778136950 2024-05-08T18:06:52.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.755+0000 7ff15171f800 1 -- 172.21.15.50:0/1465939421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 msgr2=0x557778229c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.755+0000 7ff15171f800 1 --2- 172.21.15.50:0/1465939421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 0x557778229c90 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ff140055040 tx=0x7ff14004c000 comp rx=0 tx=0).stop 2024-05-08T18:06:52.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.755+0000 7ff15171f800 1 -- 172.21.15.50:0/1465939421 shutdown_connections 2024-05-08T18:06:52.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.755+0000 7ff15171f800 1 --2- 172.21.15.50:0/1465939421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 0x557778229c90 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.755+0000 7ff15171f800 1 -- 172.21.15.50:0/1465939421 >> 172.21.15.50:0/1465939421 conn(0x5577781322f0 msgr2=0x5577782255b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.755+0000 7ff15171f800 1 -- 172.21.15.50:0/1465939421 shutdown_connections 2024-05-08T18:06:52.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.755+0000 7ff15171f800 1 -- 172.21.15.50:0/1465939421 wait complete. 2024-05-08T18:06:52.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.756+0000 7ff15171f800 1 Processor -- start 2024-05-08T18:06:52.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.756+0000 7ff15171f800 1 -- start start 2024-05-08T18:06:52.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.756+0000 7ff15171f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 0x5577782317a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.756+0000 7ff15171f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557778141e00 con 0x557778136950 2024-05-08T18:06:52.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.756+0000 7ff14d3e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 0x5577782317a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.756+0000 7ff14d3e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 0x5577782317a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46736/0 (socket says 172.21.15.50:46736) 2024-05-08T18:06:52.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.756+0000 7ff14d3e0640 1 -- 172.21.15.50:0/2649711790 learned_addr learned my addr 172.21.15.50:0/2649711790 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.757+0000 7ff14d3e0640 1 -- 172.21.15.50:0/2649711790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557778231ce0 con 0x557778136950 2024-05-08T18:06:52.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.757+0000 7ff14d3e0640 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 0x5577782317a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff140052540 tx=0x7ff140003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.757+0000 7ff039e03640 1 -- 172.21.15.50:0/2649711790 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff140073070 con 0x557778136950 2024-05-08T18:06:52.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.757+0000 7ff039e03640 1 -- 172.21.15.50:0/2649711790 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff14005d570 con 0x557778136950 2024-05-08T18:06:52.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.757+0000 7ff039e03640 1 -- 172.21.15.50:0/2649711790 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff140057070 con 0x557778136950 2024-05-08T18:06:52.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.757+0000 7ff15171f800 1 -- 172.21.15.50:0/2649711790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557778231f70 con 0x557778136950 2024-05-08T18:06:52.757 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.757+0000 7ff15171f800 1 -- 172.21.15.50:0/2649711790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557778234bd0 con 0x557778136950 2024-05-08T18:06:52.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.758+0000 7ff039e03640 1 -- 172.21.15.50:0/2649711790 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff140076070 con 0x557778136950 2024-05-08T18:06:52.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.758+0000 7ff039e03640 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff03403bba0 0x7ff03403e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.758+0000 7ff039e03640 1 -- 172.21.15.50:0/2649711790 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7ff1400941d0 con 0x557778136950 2024-05-08T18:06:52.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.758+0000 7ff14cbdf640 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff03403bba0 0x7ff03403e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.758+0000 7ff15171f800 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557778139990 0x557778238990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.758+0000 7ff15171f800 1 -- 172.21.15.50:0/2649711790 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x557778238ed0 con 0x557778139990 2024-05-08T18:06:52.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.758+0000 7ff14dbe1640 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557778139990 0x557778238990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.759+0000 7ff14cbdf640 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff03403bba0 0x7ff03403e060 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff144001e30 tx=0x7ff144001390 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.759+0000 7ff14dbe1640 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557778139990 0x557778238990 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.759+0000 7ff14dbe1640 1 -- 172.21.15.50:0/2649711790 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff14800a4a0 con 0x557778139990 2024-05-08T18:06:52.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.759+0000 7ff15171f800 1 -- 172.21.15.50:0/2649711790 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x557778235f60 con 0x557778139990 2024-05-08T18:06:52.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.759+0000 7ff14dbe1640 1 -- 172.21.15.50:0/2649711790 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff14800a4a0 con 0x557778139990 2024-05-08T18:06:52.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.759+0000 7ff15171f800 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55777823b1c0 0x55777823d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.759+0000 7ff15171f800 1 -- 172.21.15.50:0/2649711790 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55777823dc80 con 0x55777823b1c0 2024-05-08T18:06:52.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.760+0000 7ff14d3e0640 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55777823b1c0 0x55777823d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.760+0000 7ff14d3e0640 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55777823b1c0 0x55777823d5b0 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.760+0000 7ff14d3e0640 1 -- 172.21.15.50:0/2649711790 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff140075030 con 0x55777823b1c0 2024-05-08T18:06:52.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.761+0000 7ff15171f800 1 -- 172.21.15.50:0/2649711790 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55777823a060 con 0x557778139990 2024-05-08T18:06:52.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.761+0000 7ff14dbe1640 1 -- 172.21.15.50:0/2649711790 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff14800a4a0 con 0x557778139990 2024-05-08T18:06:52.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.761+0000 7ff15171f800 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55777823f0c0 0x5577782414b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.761+0000 7ff15171f800 1 -- 172.21.15.50:0/2649711790 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x557778241bf0 con 0x55777823f0c0 2024-05-08T18:06:52.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.761+0000 7ff14cbdf640 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55777823f0c0 0x5577782414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.761+0000 7ff14cbdf640 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55777823f0c0 0x5577782414b0 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.761+0000 7ff14cbdf640 1 -- 172.21.15.50:0/2649711790 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff14400bd50 con 0x55777823f0c0 2024-05-08T18:06:52.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.762+0000 7ff15171f800 1 Processor -- start 2024-05-08T18:06:52.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.762+0000 7ff15171f800 1 -- start start 2024-05-08T18:06:52.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.762+0000 7ff15171f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778139510 0x557778137980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.762+0000 7ff15171f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55777802a330 con 0x557778139510 2024-05-08T18:06:52.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.762+0000 7ff14d3e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778139510 0x557778137980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.762+0000 7ff14d3e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778139510 0x557778137980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46742/0 (socket says 172.21.15.50:46742) 2024-05-08T18:06:52.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.762+0000 7ff14d3e0640 1 -- 172.21.15.50:0/2782532314 learned_addr learned my addr 172.21.15.50:0/2782532314 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.763+0000 7ff14d3e0640 1 -- 172.21.15.50:0/2782532314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557778137ec0 con 0x557778139510 2024-05-08T18:06:52.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.763+0000 7ff14d3e0640 1 --2- 172.21.15.50:0/2782532314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778139510 0x557778137980 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff14006d000 tx=0x7ff14008bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.763+0000 7ff032ffd640 1 -- 172.21.15.50:0/2782532314 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff140092600 con 0x557778139510 2024-05-08T18:06:52.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.763+0000 7ff032ffd640 1 -- 172.21.15.50:0/2782532314 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff140076de0 con 0x557778139510 2024-05-08T18:06:52.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.763+0000 7ff15171f800 1 -- 172.21.15.50:0/2782532314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5577781380f0 con 0x557778139510 2024-05-08T18:06:52.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.763+0000 7ff032ffd640 1 -- 172.21.15.50:0/2782532314 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff14005cb30 con 0x557778139510 2024-05-08T18:06:52.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.763+0000 7ff15171f800 1 -- 172.21.15.50:0/2782532314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577781385f0 con 0x557778139510 2024-05-08T18:06:52.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.764+0000 7ff032ffd640 1 -- 172.21.15.50:0/2782532314 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff140084070 con 0x557778139510 2024-05-08T18:06:52.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.764+0000 7ff032ffd640 1 --2- 172.21.15.50:0/2782532314 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff02803bd40 0x7ff02803e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.764+0000 7ff032ffd640 1 -- 172.21.15.50:0/2782532314 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7ff14005e030 con 0x557778139510 2024-05-08T18:06:52.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.764+0000 7ff14cbdf640 1 --2- 172.21.15.50:0/2782532314 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff02803bd40 0x7ff02803e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.764+0000 7ff15171f800 1 Processor -- start 2024-05-08T18:06:52.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.764+0000 7ff15171f800 1 -- start start 2024-05-08T18:06:52.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.765+0000 7ff14cbdf640 1 --2- 172.21.15.50:0/2782532314 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff02803bd40 0x7ff02803e200 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x557778239340 tx=0x7ff14400f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.765+0000 7ff15171f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777825f2e0 0x55777825f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.765+0000 7ff15171f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55777822a520 con 0x55777825f2e0 2024-05-08T18:06:52.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.765+0000 7ff14d3e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777825f2e0 0x55777825f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.765+0000 7ff14d3e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777825f2e0 0x55777825f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46744/0 (socket says 172.21.15.50:46744) 2024-05-08T18:06:52.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.765+0000 7ff14d3e0640 1 -- 172.21.15.50:0/1596154099 learned_addr learned my addr 172.21.15.50:0/1596154099 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.765+0000 7ff14d3e0640 1 -- 172.21.15.50:0/1596154099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55777825fcd0 con 0x55777825f2e0 2024-05-08T18:06:52.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.765+0000 7ff14d3e0640 1 --2- 172.21.15.50:0/1596154099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777825f2e0 0x55777825f700 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff140006800 tx=0x7ff14006c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8be8ec96a2a3408 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.765+0000 7ff030ff9640 1 -- 172.21.15.50:0/1596154099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff14006cce0 con 0x55777825f2e0 2024-05-08T18:06:52.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.765+0000 7ff030ff9640 1 -- 172.21.15.50:0/1596154099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff140078d10 con 0x55777825f2e0 2024-05-08T18:06:52.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.765+0000 7ff030ff9640 1 -- 172.21.15.50:0/1596154099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff140076360 con 0x55777825f2e0 2024-05-08T18:06:52.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.766+0000 7ff15171f800 1 -- 172.21.15.50:0/1596154099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777825f2e0 msgr2=0x55777825f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.766+0000 7ff15171f800 1 --2- 172.21.15.50:0/1596154099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777825f2e0 0x55777825f700 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff140006800 tx=0x7ff14006c780 comp rx=0 tx=0).stop 2024-05-08T18:06:52.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.766+0000 7ff15171f800 1 -- 172.21.15.50:0/1596154099 shutdown_connections 2024-05-08T18:06:52.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.766+0000 7ff15171f800 1 --2- 172.21.15.50:0/1596154099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777825f2e0 0x55777825f700 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.766+0000 7ff15171f800 1 -- 172.21.15.50:0/1596154099 >> 172.21.15.50:0/1596154099 conn(0x55777825b5b0 msgr2=0x557778259610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.766+0000 7ff15171f800 1 -- 172.21.15.50:0/1596154099 shutdown_connections 2024-05-08T18:06:52.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.766+0000 7ff15171f800 1 -- 172.21.15.50:0/1596154099 wait complete. 2024-05-08T18:06:52.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.766+0000 7ff15171f800 1 Processor -- start 2024-05-08T18:06:52.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.766+0000 7ff15171f800 1 -- start start 2024-05-08T18:06:52.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.766+0000 7ff15171f800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777825f2e0 0x5577782665e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.766+0000 7ff15171f800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557778260330 con 0x55777825f2e0 2024-05-08T18:06:52.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.766+0000 7ff14d3e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777825f2e0 0x5577782665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.766+0000 7ff14d3e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777825f2e0 0x5577782665e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46752/0 (socket says 172.21.15.50:46752) 2024-05-08T18:06:52.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.766+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 learned_addr learned my addr 172.21.15.50:0/3108203556 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.767+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557778266b20 con 0x55777825f2e0 2024-05-08T18:06:52.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.767+0000 7ff14d3e0640 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777825f2e0 0x5577782665e0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff140076f90 tx=0x7ff1400a3070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.767+0000 7ff01effd640 1 -- 172.21.15.50:0/3108203556 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1400b7020 con 0x55777825f2e0 2024-05-08T18:06:52.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.767+0000 7ff01effd640 1 -- 172.21.15.50:0/3108203556 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff14007f170 con 0x55777825f2e0 2024-05-08T18:06:52.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.767+0000 7ff01effd640 1 -- 172.21.15.50:0/3108203556 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1400948b0 con 0x55777825f2e0 2024-05-08T18:06:52.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.767+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557778266b20 con 0x55777825f2e0 2024-05-08T18:06:52.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.767+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557778266b20 con 0x55777825f2e0 2024-05-08T18:06:52.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.768+0000 7ff01effd640 1 -- 172.21.15.50:0/3108203556 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff1400ba020 con 0x55777825f2e0 2024-05-08T18:06:52.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.768+0000 7ff01effd640 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff01803bb50 0x7ff01803e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.768+0000 7ff01effd640 1 -- 172.21.15.50:0/3108203556 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7ff1400bdd20 con 0x55777825f2e0 2024-05-08T18:06:52.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.768+0000 7ff14cbdf640 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff01803bb50 0x7ff01803e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.769+0000 7ff15171f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.769+0000 7ff15171f800 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557778271d60 0x557778274220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.777 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.769+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x557778274970 con 0x557778271d60 2024-05-08T18:06:52.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.769+0000 7ff14cbdf640 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff01803bb50 0x7ff01803e010 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x557778241ff0 tx=0x7ff144010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.769+0000 7ff14dbe1640 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557778271d60 0x557778274220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.769+0000 7ff14dbe1640 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557778271d60 0x557778274220 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff14800a4a0 con 0x557778271d60 2024-05-08T18:06:52.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff15171f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff15171f800 20 realm 2024-05-08T18:06:52.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff15171f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x557778274d80 con 0x557778271d60 2024-05-08T18:06:52.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff14800a4a0 con 0x557778271d60 2024-05-08T18:06:52.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff15171f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff15171f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:52.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff15171f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.778 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x557778275650 con 0x557778271d60 2024-05-08T18:06:52.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff14800a4a0 con 0x557778271d60 2024-05-08T18:06:52.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff15171f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff15171f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778271d60 2024-05-08T18:06:52.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff14800a4a0 con 0x557778271d60 2024-05-08T18:06:52.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff15171f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:52.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff15171f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff15171f800 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557778276750 0x557778278b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.770+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x557778279100 con 0x557778276750 2024-05-08T18:06:52.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.771+0000 7ff14d3e0640 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557778276750 0x557778278b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.771+0000 7ff14d3e0640 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557778276750 0x557778278b40 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.771+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.771+0000 7ff15171f800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:52.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.771+0000 7ff15171f800 20 searching for the correct realm 2024-05-08T18:06:52.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.771+0000 7ff15171f800 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55777827b0b0 0x55777827d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.771+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55777827dc40 con 0x55777827b0b0 2024-05-08T18:06:52.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.771+0000 7ff14cbdf640 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55777827b0b0 0x55777827d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.772+0000 7ff14cbdf640 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55777827b0b0 0x55777827d570 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.772+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff144037040 con 0x55777827b0b0 2024-05-08T18:06:52.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.772+0000 7ff14cbdf640 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff14400f070 0x7ff14402e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.772+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x55777827dc40 con 0x7ff14400f070 2024-05-08T18:06:52.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.772+0000 7ff14dbe1640 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff14400f070 0x7ff14402e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.772+0000 7ff14dbe1640 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff14400f070 0x7ff14402e910 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.773+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff14800a4a0 con 0x7ff14400f070 2024-05-08T18:06:52.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.773+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778276750 2024-05-08T18:06:52.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.773+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.773+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778271d60 2024-05-08T18:06:52.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.773+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff14800a4a0 con 0x557778271d60 2024-05-08T18:06:52.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.773+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778271d60 2024-05-08T18:06:52.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.773+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff14800a4a0 con 0x557778271d60 2024-05-08T18:06:52.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.773+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778271d60 2024-05-08T18:06:52.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.774+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff14800a4a0 con 0x557778271d60 2024-05-08T18:06:52.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.774+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778276750 2024-05-08T18:06:52.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.774+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v29'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.774+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x7ff14400f070 2024-05-08T18:06:52.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.774+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff14800a4a0 con 0x7ff14400f070 2024-05-08T18:06:52.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.774+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x55777827b0b0 2024-05-08T18:06:52.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.774+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff144037040 con 0x55777827b0b0 2024-05-08T18:06:52.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.774+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778271d60 2024-05-08T18:06:52.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.775+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff14800a4a0 con 0x557778271d60 2024-05-08T18:06:52.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.775+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778276750 2024-05-08T18:06:52.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.775+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.775+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778271d60 2024-05-08T18:06:52.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.775+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff14800a4a0 con 0x557778271d60 2024-05-08T18:06:52.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.775+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x7ff14400f070 2024-05-08T18:06:52.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.775+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff148032040 con 0x7ff14400f070 2024-05-08T18:06:52.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.775+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x55777827b0b0 2024-05-08T18:06:52.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.776+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff144037040 con 0x55777827b0b0 2024-05-08T18:06:52.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.776+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x55777827b0b0 2024-05-08T18:06:52.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.776+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff144037040 con 0x55777827b0b0 2024-05-08T18:06:52.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.776+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778276750 2024-05-08T18:06:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.777+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.777+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x55777827b0b0 2024-05-08T18:06:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.777+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff144037040 con 0x55777827b0b0 2024-05-08T18:06:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.777+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x55777827b0b0 2024-05-08T18:06:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.777+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ff144037040 con 0x55777827b0b0 2024-05-08T18:06:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.777+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778271d60 2024-05-08T18:06:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.777+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.777+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778271d60 2024-05-08T18:06:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.777+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v29'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.777+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778276750 2024-05-08T18:06:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.778+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.793 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.778+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x55777827b0b0 2024-05-08T18:06:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.778+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff144037040 con 0x55777827b0b0 2024-05-08T18:06:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.778+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x7ff14400f070 2024-05-08T18:06:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.778+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff148032040 con 0x7ff14400f070 2024-05-08T18:06:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.778+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778271d60 2024-05-08T18:06:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.779+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.779+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778276750 2024-05-08T18:06:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.779+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.779+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x55777827b0b0 2024-05-08T18:06:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.779+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff144037040 con 0x55777827b0b0 2024-05-08T18:06:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.779+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778276750 2024-05-08T18:06:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.779+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.779+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x7ff14400f070 2024-05-08T18:06:52.794 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.780+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff148032040 con 0x7ff14400f070 2024-05-08T18:06:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.780+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x55777827b0b0 2024-05-08T18:06:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.780+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff144037040 con 0x55777827b0b0 2024-05-08T18:06:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.780+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778276750 2024-05-08T18:06:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.780+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.780+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x557778271d60 2024-05-08T18:06:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.780+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v29'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x5577782759f0 con 0x7ff14400f070 2024-05-08T18:06:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff148032040 con 0x7ff14400f070 2024-05-08T18:06:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff15171f800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff15171f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff15171f800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff15171f800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff15171f800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:52.795 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff15171f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff15171f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5577782799c0 con 0x557778271d60 2024-05-08T18:06:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff15171f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff15171f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5577782799c0 con 0x557778271d60 2024-05-08T18:06:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff15171f800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff15171f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.781+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5577782799c0 con 0x55777827b0b0 2024-05-08T18:06:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff144037040 con 0x55777827b0b0 2024-05-08T18:06:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff15171f800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff15171f800 20 zone default found 2024-05-08T18:06:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff15171f800 4 Realm: () 2024-05-08T18:06:52.796 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff15171f800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff15171f800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff15171f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff15171f800 20 zonegroup default 2024-05-08T18:06:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff15171f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x55777827a360 con 0x557778276750 2024-05-08T18:06:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff15171f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff15171f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x557778279ec0 con 0x55777827b0b0 2024-05-08T18:06:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff144037040 con 0x55777827b0b0 2024-05-08T18:06:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff15171f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff15171f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x557778280020 con 0x55777827b0b0 2024-05-08T18:06:52.797 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.782+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff144037040 con 0x55777827b0b0 2024-05-08T18:06:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.783+0000 7ff15171f800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.783+0000 7ff15171f800 20 started sync module instance, tier type = 2024-05-08T18:06:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.783+0000 7ff15171f800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.783+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55777827f3e0 con 0x7ff14400f070 2024-05-08T18:06:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.784+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff148032040 con 0x7ff14400f070 2024-05-08T18:06:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.784+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93971605354720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55777827f3e0 con 0x7ff14400f070 2024-05-08T18:06:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.784+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557778280ae0 con 0x557778271d60 2024-05-08T18:06:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.785+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93971605354720] v29'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff148032040 con 0x7ff14400f070 2024-05-08T18:06:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.785+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.785+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93971605358064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55777827f3e0 con 0x557778271d60 2024-05-08T18:06:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.785+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557778281560 con 0x55777827b0b0 2024-05-08T18:06:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.786+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff144037040 con 0x55777827b0b0 2024-05-08T18:06:52.798 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.786+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93971605358064] v29'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.786+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93971605360752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557778281e30 con 0x55777827b0b0 2024-05-08T18:06:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.786+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x55777827ef50 con 0x7ff14400f070 2024-05-08T18:06:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.787+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93971605360752] v29'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff144037040 con 0x55777827b0b0 2024-05-08T18:06:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.787+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff148032040 con 0x7ff14400f070 2024-05-08T18:06:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.787+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93971605363072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5577782828d0 con 0x7ff14400f070 2024-05-08T18:06:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.787+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557778283290 con 0x557778271d60 2024-05-08T18:06:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.788+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.788+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93971605368288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5577782836b0 con 0x557778271d60 2024-05-08T18:06:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.788+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557778284130 con 0x557778276750 2024-05-08T18:06:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.788+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93971605363072] v29'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff148032040 con 0x7ff14400f070 2024-05-08T18:06:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.789+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93971605368288] v29'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.789+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.789+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93971605365792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5577782836b0 con 0x557778276750 2024-05-08T18:06:52.799 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.789+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5577782849d0 con 0x557778276750 2024-05-08T18:06:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.790+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93971605365792] v29'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.790+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.790+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93971605369344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557778285220 con 0x557778276750 2024-05-08T18:06:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.790+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557778285be0 con 0x557778276750 2024-05-08T18:06:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.791+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 93971605369344] v29'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.792+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.792+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93971605378864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557778286000 con 0x557778276750 2024-05-08T18:06:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.792+0000 7ff15171f800 20 add_watcher() i=0 2024-05-08T18:06:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.792+0000 7ff15171f800 20 add_watcher() i=1 2024-05-08T18:06:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.792+0000 7ff15171f800 20 add_watcher() i=2 2024-05-08T18:06:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.792+0000 7ff15171f800 20 add_watcher() i=3 2024-05-08T18:06:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.792+0000 7ff15171f800 20 add_watcher() i=4 2024-05-08T18:06:52.800 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.792+0000 7ff15171f800 20 add_watcher() i=5 2024-05-08T18:06:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.792+0000 7ff15171f800 20 add_watcher() i=6 2024-05-08T18:06:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.792+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93971605378864] v29'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.793+0000 7ff15171f800 20 add_watcher() i=7 2024-05-08T18:06:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.793+0000 7ff15171f800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.793+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5577782892a0 con 0x557778271d60 2024-05-08T18:06:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.793+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.793+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93971605380288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557778286590 con 0x557778271d60 2024-05-08T18:06:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.794+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93971605380288] v29'267 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.794+0000 7ff003fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.794+0000 7ff15171f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.794+0000 7ff15171f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.794+0000 7ff15171f800 5 note: GC not initialized 2024-05-08T18:06:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.794+0000 7ff002fe5640 20 reqs_thread_entry: start 2024-05-08T18:06:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.795+0000 7ff15171f800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:52.801 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.795+0000 7ff15171f800 20 Filter name: none 2024-05-08T18:06:52.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.795+0000 7ff000fe1640 20 reqs_thread_entry: start 2024-05-08T18:06:52.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.795+0000 7ff15171f800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-05-08T18:06:52.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.795+0000 7ff15171f800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.802 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.795+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5577782bc790 con 0x557778276750 2024-05-08T18:06:52.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.795+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(62 ada [call out=48b,read 0~490 out=490b,getxattrs out=75b] v0'0 uv12 ondisk = 0) ==== 231+0+613 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.803 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.795+0000 7ff15171f800 20 rados_obj.operate() r=0 bl.length=490 2024-05-08T18:06:52.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.795+0000 7ff15171f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-05-08T18:06:52.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.796+0000 7ff15171f800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.796+0000 7ff15171f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:52.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.796+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~460 in=460b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5577782c2f10 con 0x557778276750 2024-05-08T18:06:52.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.797+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~460] v29'13 uv13 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.797+0000 7ff15171f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:52.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.797+0000 7ff15171f800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.797+0000 7ff15171f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-05-08T18:06:52.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.797+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 93971605629136 in=648b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x5577782be550 con 0x557778271d60 2024-05-08T18:06:52.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.797+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== watch-notify(notify (1) cookie 93971605368288 notify 124554051625 ret 0) ==== 678+0+0 (crc 0 0 0) 0x7ff14803d040 con 0x557778271d60 2024-05-08T18:06:52.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.797+0000 7ff030ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051625 cookie 93971605368288 notifier 4900 bl.length()=636 2024-05-08T18:06:52.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.797+0000 7ff030ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:52.804 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.797+0000 7ff030ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.797+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 notify.4 [notify cookie 93971605629136 out=8b] v0'0 uv273 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.797+0000 7ff030ff9640 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7ff010003ea0 con 0x557778271d60 2024-05-08T18:06:52.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.798+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.805 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.798+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== watch-notify(notify_complete (2) cookie 93971605629136 notify 124554051625 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7ff148018d10 con 0x557778271d60 2024-05-08T18:06:52.805 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-05-08T18:06:51.092020Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:06:51.092006Z","tags":[],"group_ids":[]} 2024-05-08T18:06:52.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.799+0000 7ff003fe7640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:52.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.799+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93971605380288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5577782c1e00 con 0x557778271d60 2024-05-08T18:06:52.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.800+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 93971605380288] v29'268 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.800+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93971605354720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557778285cc0 con 0x7ff14400f070 2024-05-08T18:06:52.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.801+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 93971605354720] v29'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff148032040 con 0x7ff14400f070 2024-05-08T18:06:52.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.801+0000 7ff15171f800 20 remove_watcher() i=0 2024-05-08T18:06:52.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.801+0000 7ff15171f800 2 removed watcher, disabling cache 2024-05-08T18:06:52.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.801+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93971605358064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557778285cc0 con 0x557778271d60 2024-05-08T18:06:52.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.802+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 93971605358064] v29'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.802+0000 7ff15171f800 20 remove_watcher() i=1 2024-05-08T18:06:52.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.802+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93971605360752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x5577782805f0 con 0x55777827b0b0 2024-05-08T18:06:52.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.803+0000 7ff14cbdf640 1 -- 172.21.15.50:0/3108203556 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 93971605360752] v29'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff144037040 con 0x55777827b0b0 2024-05-08T18:06:52.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.803+0000 7ff15171f800 20 remove_watcher() i=2 2024-05-08T18:06:52.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.803+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93971605363072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557778285cc0 con 0x7ff14400f070 2024-05-08T18:06:52.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.804+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 93971605363072] v29'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff148032040 con 0x7ff14400f070 2024-05-08T18:06:52.811 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.804+0000 7ff15171f800 20 remove_watcher() i=3 2024-05-08T18:06:52.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.804+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93971605368288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557778281980 con 0x557778271d60 2024-05-08T18:06:52.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.805+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 93971605368288] v29'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff148032040 con 0x557778271d60 2024-05-08T18:06:52.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.805+0000 7ff15171f800 20 remove_watcher() i=4 2024-05-08T18:06:52.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.805+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93971605365792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557778285cc0 con 0x557778276750 2024-05-08T18:06:52.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.806+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 93971605365792] v29'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.806+0000 7ff15171f800 20 remove_watcher() i=5 2024-05-08T18:06:52.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.806+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93971605369344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557778282de0 con 0x557778276750 2024-05-08T18:06:52.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.807+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 93971605369344] v29'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.807+0000 7ff15171f800 20 remove_watcher() i=6 2024-05-08T18:06:52.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.807+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93971605378864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x557778281980 con 0x557778276750 2024-05-08T18:06:52.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.808+0000 7ff14d3e0640 1 -- 172.21.15.50:0/3108203556 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 93971605378864] v29'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff140083030 con 0x557778276750 2024-05-08T18:06:52.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.808+0000 7ff15171f800 20 remove_watcher() i=7 2024-05-08T18:06:52.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.808+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557778276750 msgr2=0x557778278b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.808+0000 7ff15171f800 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x557778276750 0x557778278b40 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.812 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.808+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557778271d60 msgr2=0x557778274220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.808+0000 7ff15171f800 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557778271d60 0x557778274220 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.808+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff14400f070 msgr2=0x7ff14402e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.808+0000 7ff15171f800 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff14400f070 0x7ff14402e910 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.808+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55777827b0b0 msgr2=0x55777827d570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.808+0000 7ff15171f800 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55777827b0b0 0x55777827d570 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff01803bb50 msgr2=0x7ff01803e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff01803bb50 0x7ff01803e010 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x557778241ff0 tx=0x7ff144010000 comp rx=0 tx=0).stop 2024-05-08T18:06:52.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777825f2e0 msgr2=0x5577782665e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 --2- 172.21.15.50:0/3108203556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55777825f2e0 0x5577782665e0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff140076f90 tx=0x7ff1400a3070 comp rx=0 tx=0).stop 2024-05-08T18:06:52.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 reap_dead start 2024-05-08T18:06:52.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff14dbe1640 1 -- 172.21.15.50:0/3108203556 reap_dead start 2024-05-08T18:06:52.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 shutdown_connections 2024-05-08T18:06:52.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 >> 172.21.15.50:0/3108203556 conn(0x55777825b5b0 msgr2=0x557778258c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 shutdown_connections 2024-05-08T18:06:52.813 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 -- 172.21.15.50:0/3108203556 wait complete. 2024-05-08T18:06:52.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 -- 172.21.15.50:0/2782532314 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff02803bd40 msgr2=0x7ff02803e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 --2- 172.21.15.50:0/2782532314 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff02803bd40 0x7ff02803e200 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x557778239340 tx=0x7ff14400f000 comp rx=0 tx=0).stop 2024-05-08T18:06:52.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 -- 172.21.15.50:0/2782532314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778139510 msgr2=0x557778137980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 --2- 172.21.15.50:0/2782532314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778139510 0x557778137980 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff14006d000 tx=0x7ff14008bd80 comp rx=0 tx=0).stop 2024-05-08T18:06:52.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 -- 172.21.15.50:0/2782532314 shutdown_connections 2024-05-08T18:06:52.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 --2- 172.21.15.50:0/2782532314 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff02803bd40 0x7ff02803e200 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 --2- 172.21.15.50:0/2782532314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778139510 0x557778137980 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 -- 172.21.15.50:0/2782532314 >> 172.21.15.50:0/2782532314 conn(0x557778244000 msgr2=0x557778244400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 -- 172.21.15.50:0/2782532314 shutdown_connections 2024-05-08T18:06:52.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.809+0000 7ff15171f800 1 -- 172.21.15.50:0/2782532314 wait complete. 2024-05-08T18:06:52.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.810+0000 7ff15171f800 1 -- 172.21.15.50:0/2649711790 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55777823b1c0 msgr2=0x55777823d5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.810+0000 7ff15171f800 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x55777823b1c0 0x55777823d5b0 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.810+0000 7ff15171f800 1 -- 172.21.15.50:0/2649711790 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557778139990 msgr2=0x557778238990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.814 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.810+0000 7ff15171f800 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x557778139990 0x557778238990 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.810+0000 7ff15171f800 1 -- 172.21.15.50:0/2649711790 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55777823f0c0 msgr2=0x5577782414b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.810+0000 7ff15171f800 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x55777823f0c0 0x5577782414b0 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.810+0000 7ff15171f800 1 -- 172.21.15.50:0/2649711790 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff03403bba0 msgr2=0x7ff03403e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.810+0000 7ff15171f800 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff03403bba0 0x7ff03403e060 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff144001e30 tx=0x7ff144001390 comp rx=0 tx=0).stop 2024-05-08T18:06:52.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.810+0000 7ff15171f800 1 -- 172.21.15.50:0/2649711790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 msgr2=0x5577782317a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.810+0000 7ff15171f800 1 --2- 172.21.15.50:0/2649711790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557778136950 0x5577782317a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff140052540 tx=0x7ff140003020 comp rx=0 tx=0).stop 2024-05-08T18:06:52.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.810+0000 7ff14dbe1640 1 -- 172.21.15.50:0/2649711790 reap_dead start 2024-05-08T18:06:52.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.810+0000 7ff15171f800 1 -- 172.21.15.50:0/2649711790 shutdown_connections 2024-05-08T18:06:52.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.810+0000 7ff15171f800 1 -- 172.21.15.50:0/2649711790 >> 172.21.15.50:0/2649711790 conn(0x5577781322f0 msgr2=0x557778223db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.810+0000 7ff15171f800 1 -- 172.21.15.50:0/2649711790 shutdown_connections 2024-05-08T18:06:52.815 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.810+0000 7ff15171f800 1 -- 172.21.15.50:0/2649711790 wait complete. 2024-05-08T18:06:52.817 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:52.817 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:52.817 ERROR:root:0 2024-05-08T18:06:52.817 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-05-08T18:06:52.817 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-05-08T18:06:52.818 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps 'users=read, write' 2024-05-08T18:06:52.865 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:52.866 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:52.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.868+0000 7f7d87f33800 1 Processor -- start 2024-05-08T18:06:52.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.868+0000 7f7d87f33800 1 -- start start 2024-05-08T18:06:52.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.869+0000 7f7d87f33800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55617697acd0 0x55617697d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.869+0000 7f7d87f33800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55617697d600 con 0x5561769771c0 2024-05-08T18:06:52.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.869+0000 7f7d87f33800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x556176816e20 con 0x55617697acd0 2024-05-08T18:06:52.867 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.869+0000 7f7d833ed640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55617697acd0 0x55617697d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.869+0000 7f7d833ed640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x55617697acd0 0x55617697d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46756/0 (socket says 172.21.15.50:46756) 2024-05-08T18:06:52.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.869+0000 7f7d83bee640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x5561769771c0 0x5561769795b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50930/0 (socket says 172.21.15.50:50930) 2024-05-08T18:06:52.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.869+0000 7f7d833ed640 1 -- 172.21.15.50:0/3736577737 learned_addr learned my addr 172.21.15.50:0/3736577737 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.869+0000 7f7d833ed640 1 -- 172.21.15.50:0/3736577737 >> v1:172.21.15.46:6789/0 conn(0x5561769771c0 legacy=0x5561769795b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.868 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.869+0000 7f7d833ed640 1 -- 172.21.15.50:0/3736577737 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55617697e7e0 con 0x55617697acd0 2024-05-08T18:06:52.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.871+0000 7f7d833ed640 1 --2- 172.21.15.50:0/3736577737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55617697acd0 0x55617697d0c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800a520 tx=0x7f7d7801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff4b621b5f3e49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.871+0000 7f7d82bec640 1 -- 172.21.15.50:0/3736577737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d78057070 con 0x55617697acd0 2024-05-08T18:06:52.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.871+0000 7f7d82bec640 1 -- 172.21.15.50:0/3736577737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d78052570 con 0x55617697acd0 2024-05-08T18:06:52.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.871+0000 7f7d82bec640 1 -- 172.21.15.50:0/3736577737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d7804ae60 con 0x55617697acd0 2024-05-08T18:06:52.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.872+0000 7f7d87f33800 1 -- 172.21.15.50:0/3736577737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55617697acd0 msgr2=0x55617697d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.872+0000 7f7d87f33800 1 --2- 172.21.15.50:0/3736577737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55617697acd0 0x55617697d0c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800a520 tx=0x7f7d7801d010 comp rx=0 tx=0).stop 2024-05-08T18:06:52.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.872+0000 7f7d87f33800 1 -- 172.21.15.50:0/3736577737 shutdown_connections 2024-05-08T18:06:52.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.872+0000 7f7d87f33800 1 --2- 172.21.15.50:0/3736577737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55617697acd0 0x55617697d0c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.872+0000 7f7d87f33800 1 -- 172.21.15.50:0/3736577737 >> 172.21.15.50:0/3736577737 conn(0x556176970af0 msgr2=0x556176972f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.872+0000 7f7d87f33800 1 -- 172.21.15.50:0/3736577737 shutdown_connections 2024-05-08T18:06:52.870 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.872+0000 7f7d87f33800 1 -- 172.21.15.50:0/3736577737 wait complete. 2024-05-08T18:06:52.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.887+0000 7f7d87f33800 1 Processor -- start 2024-05-08T18:06:52.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.887+0000 7f7d87f33800 1 -- start start 2024-05-08T18:06:52.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.887+0000 7f7d87f33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561769771c0 0x556176a67c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.887+0000 7f7d87f33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5561768c1a30 con 0x5561769771c0 2024-05-08T18:06:52.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.887+0000 7f7d83bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561769771c0 0x556176a67c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.887+0000 7f7d83bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561769771c0 0x556176a67c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46760/0 (socket says 172.21.15.50:46760) 2024-05-08T18:06:52.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.887+0000 7f7d83bee640 1 -- 172.21.15.50:0/3400402656 learned_addr learned my addr 172.21.15.50:0/3400402656 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.888+0000 7f7d83bee640 1 -- 172.21.15.50:0/3400402656 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556176a681f0 con 0x5561769771c0 2024-05-08T18:06:52.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.888+0000 7f7d83bee640 1 --2- 172.21.15.50:0/3400402656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561769771c0 0x556176a67c20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400b260 tx=0x7f7d7400b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a7fe290da13c6d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.888+0000 7f7c71e0b640 1 -- 172.21.15.50:0/3400402656 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d7400e5c0 con 0x5561769771c0 2024-05-08T18:06:52.888 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.888+0000 7f7c71e0b640 1 -- 172.21.15.50:0/3400402656 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d74006070 con 0x5561769771c0 2024-05-08T18:06:52.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.888+0000 7f7c71e0b640 1 -- 172.21.15.50:0/3400402656 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d7400e890 con 0x5561769771c0 2024-05-08T18:06:52.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.889+0000 7f7d87f33800 1 -- 172.21.15.50:0/3400402656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561769771c0 msgr2=0x556176a67c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.889+0000 7f7d87f33800 1 --2- 172.21.15.50:0/3400402656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561769771c0 0x556176a67c20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400b260 tx=0x7f7d7400b720 comp rx=0 tx=0).stop 2024-05-08T18:06:52.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.889+0000 7f7d87f33800 1 -- 172.21.15.50:0/3400402656 shutdown_connections 2024-05-08T18:06:52.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.889+0000 7f7d87f33800 1 --2- 172.21.15.50:0/3400402656 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561769771c0 0x556176a67c20 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.889+0000 7f7d87f33800 1 -- 172.21.15.50:0/3400402656 >> 172.21.15.50:0/3400402656 conn(0x556176970af0 msgr2=0x556176a63540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.889+0000 7f7d87f33800 1 -- 172.21.15.50:0/3400402656 shutdown_connections 2024-05-08T18:06:52.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.889+0000 7f7d87f33800 1 -- 172.21.15.50:0/3400402656 wait complete. 2024-05-08T18:06:52.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.889+0000 7f7d87f33800 1 Processor -- start 2024-05-08T18:06:52.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.889+0000 7f7d87f33800 1 -- start start 2024-05-08T18:06:52.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.889+0000 7f7d87f33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561769771c0 0x556176a6f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.889+0000 7f7d87f33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55617697dad0 con 0x5561769771c0 2024-05-08T18:06:52.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.890+0000 7f7d83bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561769771c0 0x556176a6f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.890+0000 7f7d83bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561769771c0 0x556176a6f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46776/0 (socket says 172.21.15.50:46776) 2024-05-08T18:06:52.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.890+0000 7f7d83bee640 1 -- 172.21.15.50:0/1573924993 learned_addr learned my addr 172.21.15.50:0/1573924993 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.890+0000 7f7d83bee640 1 -- 172.21.15.50:0/1573924993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556176a6fd00 con 0x5561769771c0 2024-05-08T18:06:52.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.890+0000 7f7d83bee640 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561769771c0 0x556176a6f7c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400b760 tx=0x7f7d74000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.890+0000 7f7c6bfff640 1 -- 172.21.15.50:0/1573924993 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d7400fa30 con 0x5561769771c0 2024-05-08T18:06:52.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.890+0000 7f7c6bfff640 1 -- 172.21.15.50:0/1573924993 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d7400fbd0 con 0x5561769771c0 2024-05-08T18:06:52.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.890+0000 7f7d87f33800 1 -- 172.21.15.50:0/1573924993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556176a6ff90 con 0x5561769771c0 2024-05-08T18:06:52.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.890+0000 7f7c6bfff640 1 -- 172.21.15.50:0/1573924993 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d7401a480 con 0x5561769771c0 2024-05-08T18:06:52.890 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.890+0000 7f7d87f33800 1 -- 172.21.15.50:0/1573924993 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556176a72bf0 con 0x5561769771c0 2024-05-08T18:06:52.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.892+0000 7f7c6bfff640 1 -- 172.21.15.50:0/1573924993 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7d74017070 con 0x5561769771c0 2024-05-08T18:06:52.893 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.892+0000 7f7c6bfff640 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7c6c03bba0 0x7f7c6c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.892+0000 7f7c6bfff640 1 -- 172.21.15.50:0/1573924993 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f7d74015020 con 0x5561769771c0 2024-05-08T18:06:52.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.892+0000 7f7d833ed640 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7c6c03bba0 0x7f7c6c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.892+0000 7f7d87f33800 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55617697acd0 0x556176a769b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.892+0000 7f7d87f33800 1 -- 172.21.15.50:0/1573924993 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176a76f40 con 0x55617697acd0 2024-05-08T18:06:52.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.892+0000 7f7d843ef640 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55617697acd0 0x556176a769b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.892+0000 7f7d833ed640 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7c6c03bba0 0x7f7c6c03e060 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7d78054000 tx=0x7f7d7804e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.893+0000 7f7d843ef640 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55617697acd0 0x556176a769b0 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.893+0000 7f7d843ef640 1 -- 172.21.15.50:0/1573924993 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7d7c00a4a0 con 0x55617697acd0 2024-05-08T18:06:52.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.893+0000 7f7d87f33800 1 -- 172.21.15.50:0/1573924993 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176a73f80 con 0x55617697acd0 2024-05-08T18:06:52.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.893+0000 7f7d843ef640 1 -- 172.21.15.50:0/1573924993 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7d7c00a4a0 con 0x55617697acd0 2024-05-08T18:06:52.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.893+0000 7f7d87f33800 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556176a79290 0x556176a7b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.893+0000 7f7d87f33800 1 -- 172.21.15.50:0/1573924993 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176a7bd70 con 0x556176a79290 2024-05-08T18:06:52.894 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.893+0000 7f7d83bee640 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556176a79290 0x556176a7b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.894+0000 7f7d83bee640 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556176a79290 0x556176a7b680 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.894+0000 7f7d83bee640 1 -- 172.21.15.50:0/1573924993 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7d7402e070 con 0x556176a79290 2024-05-08T18:06:52.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.894+0000 7f7d87f33800 1 -- 172.21.15.50:0/1573924993 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176a78130 con 0x55617697acd0 2024-05-08T18:06:52.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.894+0000 7f7d843ef640 1 -- 172.21.15.50:0/1573924993 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7d7c00a4a0 con 0x55617697acd0 2024-05-08T18:06:52.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.894+0000 7f7d87f33800 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556176a7d1d0 0x556176a7f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.894+0000 7f7d87f33800 1 -- 172.21.15.50:0/1573924993 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176a7fcc0 con 0x556176a7d1d0 2024-05-08T18:06:52.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.894+0000 7f7d833ed640 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556176a7d1d0 0x556176a7f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.895+0000 7f7d833ed640 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556176a7d1d0 0x556176a7f5c0 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.895+0000 7f7d833ed640 1 -- 172.21.15.50:0/1573924993 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7d7805a040 con 0x556176a7d1d0 2024-05-08T18:06:52.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.895+0000 7f7d87f33800 1 Processor -- start 2024-05-08T18:06:52.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.895+0000 7f7d87f33800 1 -- start start 2024-05-08T18:06:52.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.896+0000 7f7d87f33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a8e030 0x556176a8e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.896+0000 7f7d87f33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556176868330 con 0x556176a8e030 2024-05-08T18:06:52.895 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.896+0000 7f7d83bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a8e030 0x556176a8e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.896+0000 7f7d83bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a8e030 0x556176a8e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46784/0 (socket says 172.21.15.50:46784) 2024-05-08T18:06:52.896 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.896+0000 7f7d83bee640 1 -- 172.21.15.50:0/2485375523 learned_addr learned my addr 172.21.15.50:0/2485375523 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.896+0000 7f7d83bee640 1 -- 172.21.15.50:0/2485375523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556176a8e990 con 0x556176a8e030 2024-05-08T18:06:52.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.896+0000 7f7d83bee640 1 --2- 172.21.15.50:0/2485375523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a8e030 0x556176a8e450 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7d74051890 tx=0x7f7d7401beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.896+0000 7f7c697fa640 1 -- 172.21.15.50:0/2485375523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d74046070 con 0x556176a8e030 2024-05-08T18:06:52.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.896+0000 7f7c697fa640 1 -- 172.21.15.50:0/2485375523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d74051ce0 con 0x556176a8e030 2024-05-08T18:06:52.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.896+0000 7f7d87f33800 1 -- 172.21.15.50:0/2485375523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556176a8f400 con 0x556176a8e030 2024-05-08T18:06:52.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.896+0000 7f7c697fa640 1 -- 172.21.15.50:0/2485375523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d7401a620 con 0x556176a8e030 2024-05-08T18:06:52.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.896+0000 7f7d87f33800 1 -- 172.21.15.50:0/2485375523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556176a8f900 con 0x556176a8e030 2024-05-08T18:06:52.904 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.897+0000 7f7c697fa640 1 -- 172.21.15.50:0/2485375523 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7d7403e040 con 0x556176a8e030 2024-05-08T18:06:52.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.897+0000 7f7c697fa640 1 --2- 172.21.15.50:0/2485375523 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7c5c03bba0 0x7f7c5c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.897+0000 7f7c697fa640 1 -- 172.21.15.50:0/2485375523 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f7d7403b040 con 0x556176a8e030 2024-05-08T18:06:52.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.897+0000 7f7d833ed640 1 --2- 172.21.15.50:0/2485375523 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7c5c03bba0 0x7f7c5c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.898+0000 7f7d87f33800 1 Processor -- start 2024-05-08T18:06:52.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.898+0000 7f7d87f33800 1 -- start start 2024-05-08T18:06:52.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.898+0000 7f7d833ed640 1 --2- 172.21.15.50:0/2485375523 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7c5c03bba0 0x7f7c5c03e060 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x556176a77840 tx=0x7f7d78005ac0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.898+0000 7f7d87f33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a9b0e0 0x556176a9b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.898+0000 7f7d87f33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556176a68570 con 0x556176a9b0e0 2024-05-08T18:06:52.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.898+0000 7f7d83bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a9b0e0 0x556176a9b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.898+0000 7f7d83bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a9b0e0 0x556176a9b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46794/0 (socket says 172.21.15.50:46794) 2024-05-08T18:06:52.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.898+0000 7f7d83bee640 1 -- 172.21.15.50:0/592729096 learned_addr learned my addr 172.21.15.50:0/592729096 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.898+0000 7f7d83bee640 1 -- 172.21.15.50:0/592729096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556176a9baf0 con 0x556176a9b0e0 2024-05-08T18:06:52.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.899+0000 7f7d83bee640 1 --2- 172.21.15.50:0/592729096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a9b0e0 0x556176a9b500 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f7d7402f700 tx=0x7f7d74049ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318ad14e5b3c624f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.905 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.899+0000 7f7c62ffd640 1 -- 172.21.15.50:0/592729096 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d7403fcd0 con 0x556176a9b0e0 2024-05-08T18:06:52.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.899+0000 7f7c62ffd640 1 -- 172.21.15.50:0/592729096 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d740193e0 con 0x556176a9b0e0 2024-05-08T18:06:52.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.899+0000 7f7c62ffd640 1 -- 172.21.15.50:0/592729096 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d7401b3d0 con 0x556176a9b0e0 2024-05-08T18:06:52.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.899+0000 7f7d87f33800 1 -- 172.21.15.50:0/592729096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a9b0e0 msgr2=0x556176a9b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.899+0000 7f7d87f33800 1 --2- 172.21.15.50:0/592729096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a9b0e0 0x556176a9b500 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f7d7402f700 tx=0x7f7d74049ab0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.899+0000 7f7d87f33800 1 -- 172.21.15.50:0/592729096 shutdown_connections 2024-05-08T18:06:52.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.899+0000 7f7d87f33800 1 --2- 172.21.15.50:0/592729096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a9b0e0 0x556176a9b500 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.899+0000 7f7d87f33800 1 -- 172.21.15.50:0/592729096 >> 172.21.15.50:0/592729096 conn(0x556176a97420 msgr2=0x556176a95480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.899+0000 7f7d87f33800 1 -- 172.21.15.50:0/592729096 shutdown_connections 2024-05-08T18:06:52.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.899+0000 7f7d87f33800 1 -- 172.21.15.50:0/592729096 wait complete. 2024-05-08T18:06:52.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.900+0000 7f7d87f33800 1 Processor -- start 2024-05-08T18:06:52.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.900+0000 7f7d87f33800 1 -- start start 2024-05-08T18:06:52.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.900+0000 7f7d87f33800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a9b0e0 0x556176aa2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.900+0000 7f7d87f33800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556176a9c210 con 0x556176a9b0e0 2024-05-08T18:06:52.906 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.900+0000 7f7d83bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a9b0e0 0x556176aa2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.900+0000 7f7d83bee640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a9b0e0 0x556176aa2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46806/0 (socket says 172.21.15.50:46806) 2024-05-08T18:06:52.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.900+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 learned_addr learned my addr 172.21.15.50:0/99299945 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:52.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.900+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556176aa29a0 con 0x556176a9b0e0 2024-05-08T18:06:52.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.900+0000 7f7d83bee640 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a9b0e0 0x556176aa2460 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7d7404abf0 tx=0x7f7d7404bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.901+0000 7f7c617fa640 1 -- 172.21.15.50:0/99299945 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d7406d070 con 0x556176a9b0e0 2024-05-08T18:06:52.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.901+0000 7f7c617fa640 1 -- 172.21.15.50:0/99299945 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d7404be80 con 0x556176a9b0e0 2024-05-08T18:06:52.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.901+0000 7f7c617fa640 1 -- 172.21.15.50:0/99299945 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7d740109b0 con 0x556176a9b0e0 2024-05-08T18:06:52.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.901+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556176aa29a0 con 0x556176a9b0e0 2024-05-08T18:06:52.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.901+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556176aa2e40 con 0x556176a9b0e0 2024-05-08T18:06:52.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.902+0000 7f7c617fa640 1 -- 172.21.15.50:0/99299945 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7d74070070 con 0x556176a9b0e0 2024-05-08T18:06:52.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.902+0000 7f7c617fa640 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7c4c03bba0 0x7f7c4c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.902+0000 7f7c617fa640 1 -- 172.21.15.50:0/99299945 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f7d74051040 con 0x556176a9b0e0 2024-05-08T18:06:52.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.902+0000 7f7d833ed640 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7c4c03bba0 0x7f7c4c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.907 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.902+0000 7f7d87f33800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.902+0000 7f7d87f33800 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556176aadbf0 0x556176ab00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.902+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176ab0800 con 0x556176aadbf0 2024-05-08T18:06:52.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.903+0000 7f7d843ef640 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556176aadbf0 0x556176ab00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.903+0000 7f7d833ed640 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7c4c03bba0 0x7f7c4c03e060 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7d78055040 tx=0x7f7d7804f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.903+0000 7f7d843ef640 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556176aadbf0 0x556176ab00b0 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.903+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7d7c00a4a0 con 0x556176aadbf0 2024-05-08T18:06:52.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.903+0000 7f7d87f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.903+0000 7f7d87f33800 20 realm 2024-05-08T18:06:52.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.903+0000 7f7d87f33800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.903+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176ab0c10 con 0x556176aadbf0 2024-05-08T18:06:52.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.903+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7d7c00a4a0 con 0x556176aadbf0 2024-05-08T18:06:52.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.903+0000 7f7d87f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.903+0000 7f7d87f33800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:52.908 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.903+0000 7f7d87f33800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.903+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176ab1030 con 0x556176aadbf0 2024-05-08T18:06:52.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.904+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7d7c00a4a0 con 0x556176aadbf0 2024-05-08T18:06:52.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.904+0000 7f7d87f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.904+0000 7f7d87f33800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.904+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176ab18d0 con 0x556176aadbf0 2024-05-08T18:06:52.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.904+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7d7c00a4a0 con 0x556176aadbf0 2024-05-08T18:06:52.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.904+0000 7f7d87f33800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:52.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.904+0000 7f7d87f33800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.904+0000 7f7d87f33800 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556176ab22f0 0x556176ab46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.904+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176ab4dd0 con 0x556176ab22f0 2024-05-08T18:06:52.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.904+0000 7f7d83bee640 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556176ab22f0 0x556176ab46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.904+0000 7f7d83bee640 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556176ab22f0 0x556176ab46e0 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.905+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.905+0000 7f7d87f33800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:52.909 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.905+0000 7f7d87f33800 20 searching for the correct realm 2024-05-08T18:06:52.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.905+0000 7f7d87f33800 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556176ab6da0 0x556176ab9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.905+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556176ab9930 con 0x556176ab6da0 2024-05-08T18:06:52.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.905+0000 7f7d833ed640 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556176ab6da0 0x556176ab9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.905+0000 7f7d833ed640 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556176ab6da0 0x556176ab9260 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.905+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7807f070 con 0x556176ab6da0 2024-05-08T18:06:52.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.905+0000 7f7d833ed640 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7d78074040 0x7f7d78081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:52.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.905+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556176ab9930 con 0x7f7d78074040 2024-05-08T18:06:52.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.906+0000 7f7d843ef640 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7d78074040 0x7f7d78081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:52.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.906+0000 7f7d843ef640 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7d78074040 0x7f7d78081410 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:52.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.906+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7c00a4a0 con 0x7f7d78074040 2024-05-08T18:06:52.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.906+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556176ab18d0 con 0x556176ab22f0 2024-05-08T18:06:52.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.906+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.906+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556176ab18d0 con 0x556176aadbf0 2024-05-08T18:06:52.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.907+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7c00a4a0 con 0x556176aadbf0 2024-05-08T18:06:52.920 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.907+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556176ab18d0 con 0x556176aadbf0 2024-05-08T18:06:52.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.907+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7c00a4a0 con 0x556176aadbf0 2024-05-08T18:06:52.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.907+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556176ab18d0 con 0x556176aadbf0 2024-05-08T18:06:52.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.907+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7c00a4a0 con 0x556176aadbf0 2024-05-08T18:06:52.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.907+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556176ab18d0 con 0x556176ab22f0 2024-05-08T18:06:52.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.907+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v29'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.907+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556176ab18d0 con 0x7f7d78074040 2024-05-08T18:06:52.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.908+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7c00a4a0 con 0x7f7d78074040 2024-05-08T18:06:52.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.908+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556176ab18d0 con 0x556176ab6da0 2024-05-08T18:06:52.921 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.908+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7807f070 con 0x556176ab6da0 2024-05-08T18:06:52.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.908+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556176ab18d0 con 0x556176aadbf0 2024-05-08T18:06:52.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.908+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7c00a4a0 con 0x556176aadbf0 2024-05-08T18:06:52.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.908+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556176ab18d0 con 0x556176ab22f0 2024-05-08T18:06:52.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.908+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.908+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556176ab18d0 con 0x556176aadbf0 2024-05-08T18:06:52.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.909+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7c00a4a0 con 0x556176aadbf0 2024-05-08T18:06:52.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.909+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556176ab18d0 con 0x7f7d78074040 2024-05-08T18:06:52.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.909+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7d7c032040 con 0x7f7d78074040 2024-05-08T18:06:52.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.909+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x556176ab18d0 con 0x556176ab6da0 2024-05-08T18:06:52.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.909+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7807f070 con 0x556176ab6da0 2024-05-08T18:06:52.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.909+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x556176ab6da0 2024-05-08T18:06:52.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.909+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7807f070 con 0x556176ab6da0 2024-05-08T18:06:52.922 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.910+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x556176ab22f0 2024-05-08T18:06:52.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.910+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.910+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x556176ab6da0 2024-05-08T18:06:52.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.910+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7807f070 con 0x556176ab6da0 2024-05-08T18:06:52.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.910+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x556176ab6da0 2024-05-08T18:06:52.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.910+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7d7807f070 con 0x556176ab6da0 2024-05-08T18:06:52.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.910+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x556176aadbf0 2024-05-08T18:06:52.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.911+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.911+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x556176aadbf0 2024-05-08T18:06:52.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.911+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v29'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.911+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x556176ab22f0 2024-05-08T18:06:52.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.911+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.911+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x556176ab6da0 2024-05-08T18:06:52.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.912+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7807f070 con 0x556176ab6da0 2024-05-08T18:06:52.923 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.912+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x7f7d78074040 2024-05-08T18:06:52.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.912+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7c032040 con 0x7f7d78074040 2024-05-08T18:06:52.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.912+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x556176aadbf0 2024-05-08T18:06:52.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.912+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.912+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x556176ab22f0 2024-05-08T18:06:52.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.913+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.913+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x556176ab6da0 2024-05-08T18:06:52.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.913+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7807f070 con 0x556176ab6da0 2024-05-08T18:06:52.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.913+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x556176ab22f0 2024-05-08T18:06:52.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.913+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.913+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x7f7d78074040 2024-05-08T18:06:52.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.913+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7c032040 con 0x7f7d78074040 2024-05-08T18:06:52.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.913+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x556176ab6da0 2024-05-08T18:06:52.924 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.914+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7807f070 con 0x556176ab6da0 2024-05-08T18:06:52.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.914+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x556176ab22f0 2024-05-08T18:06:52.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.914+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.914+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x556176aadbf0 2024-05-08T18:06:52.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.914+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v29'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.914+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f7d7808e040 con 0x7f7d78074040 2024-05-08T18:06:52.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.914+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7d7c032040 con 0x7f7d78074040 2024-05-08T18:06:52.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:52.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:52.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:52.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:52.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:52.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:52.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176ab5690 con 0x556176aadbf0 2024-05-08T18:06:52.925 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176ab5690 con 0x556176aadbf0 2024-05-08T18:06:52.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:52.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176ab5690 con 0x556176ab6da0 2024-05-08T18:06:52.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7d7807f070 con 0x556176ab6da0 2024-05-08T18:06:52.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:52.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 20 zone default found 2024-05-08T18:06:52.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 4 Realm: () 2024-05-08T18:06:52.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:52.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:52.926 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:52.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 20 zonegroup default 2024-05-08T18:06:52.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.915+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176ab6070 con 0x556176ab22f0 2024-05-08T18:06:52.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.916+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.916+0000 7f7d87f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.916+0000 7f7d87f33800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.916+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176ab5c40 con 0x556176ab6da0 2024-05-08T18:06:52.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.916+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7d7807f070 con 0x556176ab6da0 2024-05-08T18:06:52.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.916+0000 7f7d87f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.916+0000 7f7d87f33800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.916+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176abbcc0 con 0x556176ab6da0 2024-05-08T18:06:52.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.916+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7d7807f070 con 0x556176ab6da0 2024-05-08T18:06:52.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.916+0000 7f7d87f33800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:52.927 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.916+0000 7f7d87f33800 20 started sync module instance, tier type = 2024-05-08T18:06:52.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.916+0000 7f7d87f33800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:52.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.916+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176abb0d0 con 0x7f7d78074040 2024-05-08T18:06:52.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.917+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7c032040 con 0x7f7d78074040 2024-05-08T18:06:52.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.917+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93877091153392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176abb0d0 con 0x7f7d78074040 2024-05-08T18:06:52.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.917+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176abc7c0 con 0x556176aadbf0 2024-05-08T18:06:52.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.918+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93877091153392] v29'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7c032040 con 0x7f7d78074040 2024-05-08T18:06:52.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.918+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.918+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93877091156624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176abb0d0 con 0x556176aadbf0 2024-05-08T18:06:52.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.918+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176abd250 con 0x556176ab6da0 2024-05-08T18:06:52.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.920+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 93877091156624] v29'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.920+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7807f070 con 0x556176ab6da0 2024-05-08T18:06:52.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.920+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93877091150880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176abb040 con 0x556176ab6da0 2024-05-08T18:06:52.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.920+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176abe110 con 0x7f7d78074040 2024-05-08T18:06:52.928 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.921+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93877091150880] v29'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7807f070 con 0x556176ab6da0 2024-05-08T18:06:52.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.921+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7c032040 con 0x7f7d78074040 2024-05-08T18:06:52.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.921+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93877091163232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176abe080 con 0x7f7d78074040 2024-05-08T18:06:52.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.921+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176abeed0 con 0x556176aadbf0 2024-05-08T18:06:52.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.922+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.922+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93877091166640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176abeed0 con 0x556176aadbf0 2024-05-08T18:06:52.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.922+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176abfe30 con 0x556176ab22f0 2024-05-08T18:06:52.929 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.922+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93877091163232] v29'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7c032040 con 0x7f7d78074040 2024-05-08T18:06:52.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.923+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.923+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93877091165440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176ac0220 con 0x556176ab22f0 2024-05-08T18:06:52.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.923+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176ac0c20 con 0x556176ab22f0 2024-05-08T18:06:52.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.923+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93877091166640] v29'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.924+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93877091165440] v29'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.924+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.924+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93877091168960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176ac1010 con 0x556176ab22f0 2024-05-08T18:06:52.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.924+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176ac1860 con 0x556176ab22f0 2024-05-08T18:06:52.933 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.925+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(58 notify.7 [create] v29'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.925+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93877091177872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176ac1fe0 con 0x556176ab22f0 2024-05-08T18:06:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.925+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93877091168960] v29'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.925+0000 7f7d87f33800 20 add_watcher() i=0 2024-05-08T18:06:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.925+0000 7f7d87f33800 20 add_watcher() i=1 2024-05-08T18:06:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.925+0000 7f7d87f33800 20 add_watcher() i=2 2024-05-08T18:06:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.925+0000 7f7d87f33800 20 add_watcher() i=3 2024-05-08T18:06:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.925+0000 7f7d87f33800 20 add_watcher() i=4 2024-05-08T18:06:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.925+0000 7f7d87f33800 20 add_watcher() i=5 2024-05-08T18:06:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.925+0000 7f7d87f33800 20 add_watcher() i=6 2024-05-08T18:06:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.926+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 93877091177872] v29'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.926+0000 7f7d87f33800 20 add_watcher() i=7 2024-05-08T18:06:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.926+0000 7f7d87f33800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:52.934 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.926+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176ac07f0 con 0x556176aadbf0 2024-05-08T18:06:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.926+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.926+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93877091179120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176ac28e0 con 0x556176aadbf0 2024-05-08T18:06:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.927+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93877091179120] v29'269 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.928+0000 7f7c39fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.928+0000 7f7d87f33800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.928+0000 7f7d87f33800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.928+0000 7f7d87f33800 5 note: GC not initialized 2024-05-08T18:06:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.928+0000 7f7c38fe1640 20 reqs_thread_entry: start 2024-05-08T18:06:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.928+0000 7f7d87f33800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.928+0000 7f7d87f33800 20 Filter name: none 2024-05-08T18:06:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.928+0000 7f7bb27fc640 20 reqs_thread_entry: start 2024-05-08T18:06:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.928+0000 7f7d87f33800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-05-08T18:06:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.928+0000 7f7d87f33800 20 rados->read ofs=0 len=0 2024-05-08T18:06:52.935 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.928+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176af8610 con 0x556176ab22f0 2024-05-08T18:06:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.929+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(62 ada [call out=48b,read 0~460 out=460b,getxattrs out=75b] v0'0 uv13 ondisk = 0) ==== 231+0+583 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.929+0000 7f7d87f33800 20 rados_obj.operate() r=0 bl.length=460 2024-05-08T18:06:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.929+0000 7f7d87f33800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-05-08T18:06:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.929+0000 7f7d87f33800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.929+0000 7f7d87f33800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.929+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~473 in=473b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176afec40 con 0x556176ab22f0 2024-05-08T18:06:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.930+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~473] v29'14 uv14 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.930+0000 7f7d87f33800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.930+0000 7f7d87f33800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.930+0000 7f7d87f33800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-05-08T18:06:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.930+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 93877091427840 in=661b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x556176afa260 con 0x556176aadbf0 2024-05-08T18:06:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.931+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== watch-notify(notify (1) cookie 93877091166640 notify 124554051626 ret 0) ==== 691+0+0 (crc 0 0 0) 0x7f7d7c03d040 con 0x556176aadbf0 2024-05-08T18:06:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.931+0000 7f7c62ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051626 cookie 93877091166640 notifier 4907 bl.length()=649 2024-05-08T18:06:52.936 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.931+0000 7f7c62ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:52.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.931+0000 7f7c62ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:52.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.931+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 notify.4 [notify cookie 93877091427840 out=8b] v0'0 uv276 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.931+0000 7f7c62ffd640 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f7c48003ea0 con 0x556176aadbf0 2024-05-08T18:06:52.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.931+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.931+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== watch-notify(notify_complete (2) cookie 93877091427840 notify 124554051626 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f7d7c018b90 con 0x556176aadbf0 2024-05-08T18:06:52.937 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-05-08T18:06:51.092020Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"users","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:06:51.092006Z","tags":[],"group_ids":[]} 2024-05-08T18:06:52.942 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.932+0000 7f7c39fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:52.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.932+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93877091179120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176afdee0 con 0x556176aadbf0 2024-05-08T18:06:52.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.933+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 93877091179120] v29'270 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.933+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93877091153392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176afdee0 con 0x7f7d78074040 2024-05-08T18:06:52.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.934+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 93877091153392] v29'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7c032040 con 0x7f7d78074040 2024-05-08T18:06:52.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.934+0000 7f7d87f33800 20 remove_watcher() i=0 2024-05-08T18:06:52.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.934+0000 7f7d87f33800 2 removed watcher, disabling cache 2024-05-08T18:06:52.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.934+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93877091156624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176afdee0 con 0x556176aadbf0 2024-05-08T18:06:52.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.935+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 93877091156624] v29'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.935+0000 7f7d87f33800 20 remove_watcher() i=1 2024-05-08T18:06:52.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.935+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93877091150880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176afdee0 con 0x556176ab6da0 2024-05-08T18:06:52.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.936+0000 7f7d833ed640 1 -- 172.21.15.50:0/99299945 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 93877091150880] v29'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7807f070 con 0x556176ab6da0 2024-05-08T18:06:52.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.936+0000 7f7d87f33800 20 remove_watcher() i=2 2024-05-08T18:06:52.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.936+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93877091163232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176afdee0 con 0x7f7d78074040 2024-05-08T18:06:52.943 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.937+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 93877091163232] v29'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7c032040 con 0x7f7d78074040 2024-05-08T18:06:52.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.937+0000 7f7d87f33800 20 remove_watcher() i=3 2024-05-08T18:06:52.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.937+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93877091166640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176afdee0 con 0x556176aadbf0 2024-05-08T18:06:52.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.938+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 93877091166640] v29'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7c032040 con 0x556176aadbf0 2024-05-08T18:06:52.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.938+0000 7f7d87f33800 20 remove_watcher() i=4 2024-05-08T18:06:52.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.938+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93877091165440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176afdee0 con 0x556176ab22f0 2024-05-08T18:06:52.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.939+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 93877091165440] v29'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.939+0000 7f7d87f33800 20 remove_watcher() i=5 2024-05-08T18:06:52.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.939+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93877091168960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176afdee0 con 0x556176ab22f0 2024-05-08T18:06:52.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.940+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 93877091168960] v29'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.940+0000 7f7d87f33800 20 remove_watcher() i=6 2024-05-08T18:06:52.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.940+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93877091177872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x556176abac20 con 0x556176ab22f0 2024-05-08T18:06:52.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d83bee640 1 -- 172.21.15.50:0/99299945 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 93877091177872] v29'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7d7406f030 con 0x556176ab22f0 2024-05-08T18:06:52.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d87f33800 20 remove_watcher() i=7 2024-05-08T18:06:52.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556176ab22f0 msgr2=0x556176ab46e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.944 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d87f33800 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556176ab22f0 0x556176ab46e0 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556176aadbf0 msgr2=0x556176ab00b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d87f33800 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x556176aadbf0 0x556176ab00b0 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7d78074040 msgr2=0x7f7d78081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d87f33800 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f7d78074040 0x7f7d78081410 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556176ab6da0 msgr2=0x556176ab9260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d87f33800 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556176ab6da0 0x556176ab9260 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7c4c03bba0 msgr2=0x7f7c4c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d87f33800 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7c4c03bba0 0x7f7c4c03e060 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7d78055040 tx=0x7f7d7804f000 comp rx=0 tx=0).stop 2024-05-08T18:06:52.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a9b0e0 msgr2=0x556176aa2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d87f33800 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a9b0e0 0x556176aa2460 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7d7404abf0 tx=0x7f7d7404bc10 comp rx=0 tx=0).stop 2024-05-08T18:06:52.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d843ef640 1 -- 172.21.15.50:0/99299945 reap_dead start 2024-05-08T18:06:52.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 shutdown_connections 2024-05-08T18:06:52.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d87f33800 1 --2- 172.21.15.50:0/99299945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a9b0e0 0x556176aa2460 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.941+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 >> 172.21.15.50:0/99299945 conn(0x556176a97420 msgr2=0x556176a94a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 shutdown_connections 2024-05-08T18:06:52.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 -- 172.21.15.50:0/99299945 wait complete. 2024-05-08T18:06:52.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 -- 172.21.15.50:0/2485375523 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7c5c03bba0 msgr2=0x7f7c5c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 --2- 172.21.15.50:0/2485375523 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7c5c03bba0 0x7f7c5c03e060 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x556176a77840 tx=0x7f7d78005ac0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 -- 172.21.15.50:0/2485375523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a8e030 msgr2=0x556176a8e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 --2- 172.21.15.50:0/2485375523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a8e030 0x556176a8e450 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f7d74051890 tx=0x7f7d7401beb0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 -- 172.21.15.50:0/2485375523 shutdown_connections 2024-05-08T18:06:52.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 --2- 172.21.15.50:0/2485375523 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7c5c03bba0 0x7f7c5c03e060 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 --2- 172.21.15.50:0/2485375523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556176a8e030 0x556176a8e450 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 -- 172.21.15.50:0/2485375523 >> 172.21.15.50:0/2485375523 conn(0x556176a820d0 msgr2=0x556176a824d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 -- 172.21.15.50:0/2485375523 shutdown_connections 2024-05-08T18:06:52.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 -- 172.21.15.50:0/2485375523 wait complete. 2024-05-08T18:06:52.946 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 -- 172.21.15.50:0/1573924993 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556176a79290 msgr2=0x556176a7b680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x556176a79290 0x556176a7b680 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 -- 172.21.15.50:0/1573924993 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55617697acd0 msgr2=0x556176a769b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x55617697acd0 0x556176a769b0 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 -- 172.21.15.50:0/1573924993 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556176a7d1d0 msgr2=0x556176a7f5c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x556176a7d1d0 0x556176a7f5c0 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 -- 172.21.15.50:0/1573924993 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7c6c03bba0 msgr2=0x7f7c6c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7c6c03bba0 0x7f7c6c03e060 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7d78054000 tx=0x7f7d7804e000 comp rx=0 tx=0).stop 2024-05-08T18:06:52.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 -- 172.21.15.50:0/1573924993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561769771c0 msgr2=0x556176a6f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:52.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 --2- 172.21.15.50:0/1573924993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5561769771c0 0x556176a6f7c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400b760 tx=0x7f7d74000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:52.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d843ef640 1 -- 172.21.15.50:0/1573924993 reap_dead start 2024-05-08T18:06:52.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.942+0000 7f7d87f33800 1 -- 172.21.15.50:0/1573924993 shutdown_connections 2024-05-08T18:06:52.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.943+0000 7f7d87f33800 1 -- 172.21.15.50:0/1573924993 >> 172.21.15.50:0/1573924993 conn(0x556176970af0 msgr2=0x556176a61d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:52.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.943+0000 7f7d87f33800 1 -- 172.21.15.50:0/1573924993 shutdown_connections 2024-05-08T18:06:52.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:52.943+0000 7f7d87f33800 1 -- 172.21.15.50:0/1573924993 wait complete. 2024-05-08T18:06:52.950 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:52.950 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:52.951 ERROR:root:0 2024-05-08T18:06:52.951 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-05-08T18:06:52.951 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-05-08T18:06:52.951 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps 'users=read, write' 2024-05-08T18:06:52.998 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:06:52.999 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:06:53.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.001+0000 7f21a9128800 1 Processor -- start 2024-05-08T18:06:53.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.001+0000 7f21a9128800 1 -- start start 2024-05-08T18:06:53.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.001+0000 7f21a9128800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56073c60eb00 0x56073c610ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:53.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.001+0000 7f21a9128800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56073c6114c0 con 0x56073c60a710 2024-05-08T18:06:53.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.001+0000 7f21a9128800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x56073c4aae20 con 0x56073c60eb00 2024-05-08T18:06:53.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.002+0000 7f21a459f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x56073c60eb00 0x56073c610ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:53.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.002+0000 7f21a4da0640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x56073c60a710 0x56073c60ab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:50944/0 (socket says 172.21.15.50:50944) 2024-05-08T18:06:53.000 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.002+0000 7f21a4da0640 1 -- 172.21.15.50:0/2893156799 learned_addr learned my addr 172.21.15.50:0/2893156799 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:06:53.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.002+0000 7f21a459f640 1 -- 172.21.15.50:0/2893156799 >> v1:172.21.15.46:6789/0 conn(0x56073c60a710 legacy=0x56073c60ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.002+0000 7f21a3d9e640 1 -- 172.21.15.50:0/2893156799 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3751050377 0 0) 0x56073c6114c0 con 0x56073c60a710 2024-05-08T18:06:53.001 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.002+0000 7f21a459f640 1 -- 172.21.15.50:0/2893156799 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073c6126f0 con 0x56073c60eb00 2024-05-08T18:06:53.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.004+0000 7f21a459f640 1 --2- 172.21.15.50:0/2893156799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60eb00 0x56073c610ef0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f218c00a470 tx=0x7f218c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51be202feba9b03 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:53.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.004+0000 7f21a3d9e640 1 -- 172.21.15.50:0/2893156799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f218c057070 con 0x56073c60eb00 2024-05-08T18:06:53.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.004+0000 7f21a3d9e640 1 -- 172.21.15.50:0/2893156799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f218c052570 con 0x56073c60eb00 2024-05-08T18:06:53.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.004+0000 7f21a3d9e640 1 -- 172.21.15.50:0/2893156799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f218c052870 con 0x56073c60eb00 2024-05-08T18:06:53.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.005+0000 7f21a9128800 1 -- 172.21.15.50:0/2893156799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60eb00 msgr2=0x56073c610ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.005+0000 7f21a9128800 1 --2- 172.21.15.50:0/2893156799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60eb00 0x56073c610ef0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f218c00a470 tx=0x7f218c01ce90 comp rx=0 tx=0).stop 2024-05-08T18:06:53.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.005+0000 7f21a9128800 1 -- 172.21.15.50:0/2893156799 shutdown_connections 2024-05-08T18:06:53.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.005+0000 7f21a9128800 1 --2- 172.21.15.50:0/2893156799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60eb00 0x56073c610ef0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:53.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.005+0000 7f21a9128800 1 -- 172.21.15.50:0/2893156799 >> 172.21.15.50:0/2893156799 conn(0x56073c605c90 msgr2=0x56073c6080d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:53.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.005+0000 7f21a9128800 1 -- 172.21.15.50:0/2893156799 shutdown_connections 2024-05-08T18:06:53.003 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.005+0000 7f21a9128800 1 -- 172.21.15.50:0/2893156799 wait complete. 2024-05-08T18:06:53.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.020+0000 7f21a9128800 1 Processor -- start 2024-05-08T18:06:53.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.020+0000 7f21a9128800 1 -- start start 2024-05-08T18:06:53.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.020+0000 7f21a9128800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60a710 0x56073c6fb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:53.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.020+0000 7f21a9128800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56073c555a30 con 0x56073c60a710 2024-05-08T18:06:53.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.020+0000 7f21a4da0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60a710 0x56073c6fb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:53.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.020+0000 7f21a4da0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60a710 0x56073c6fb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46828/0 (socket says 172.21.15.50:46828) 2024-05-08T18:06:53.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.020+0000 7f21a4da0640 1 -- 172.21.15.50:0/529837256 learned_addr learned my addr 172.21.15.50:0/529837256 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:53.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.021+0000 7f21a4da0640 1 -- 172.21.15.50:0/529837256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073c6fbf30 con 0x56073c60a710 2024-05-08T18:06:53.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.021+0000 7f21a4da0640 1 --2- 172.21.15.50:0/529837256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60a710 0x56073c6fb960 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2194001ea0 tx=0x7f219400b6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd03537416ae305f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:53.019 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.021+0000 7f2092e0d640 1 -- 172.21.15.50:0/529837256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2194008070 con 0x56073c60a710 2024-05-08T18:06:53.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.021+0000 7f2092e0d640 1 -- 172.21.15.50:0/529837256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f219400d500 con 0x56073c60a710 2024-05-08T18:06:53.020 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.021+0000 7f2092e0d640 1 -- 172.21.15.50:0/529837256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f219400d880 con 0x56073c60a710 2024-05-08T18:06:53.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.022+0000 7f21a9128800 1 -- 172.21.15.50:0/529837256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60a710 msgr2=0x56073c6fb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.022+0000 7f21a9128800 1 --2- 172.21.15.50:0/529837256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60a710 0x56073c6fb960 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2194001ea0 tx=0x7f219400b6a0 comp rx=0 tx=0).stop 2024-05-08T18:06:53.023 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.022+0000 7f21a9128800 1 -- 172.21.15.50:0/529837256 shutdown_connections 2024-05-08T18:06:53.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.022+0000 7f21a9128800 1 --2- 172.21.15.50:0/529837256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60a710 0x56073c6fb960 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:53.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.022+0000 7f21a9128800 1 -- 172.21.15.50:0/529837256 >> 172.21.15.50:0/529837256 conn(0x56073c605c90 msgr2=0x56073c6f7280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:53.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.022+0000 7f21a9128800 1 -- 172.21.15.50:0/529837256 shutdown_connections 2024-05-08T18:06:53.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.022+0000 7f21a9128800 1 -- 172.21.15.50:0/529837256 wait complete. 2024-05-08T18:06:53.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.022+0000 7f21a9128800 1 Processor -- start 2024-05-08T18:06:53.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.022+0000 7f21a9128800 1 -- start start 2024-05-08T18:06:53.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.022+0000 7f21a9128800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60a710 0x56073c703500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:53.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.022+0000 7f21a9128800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56073c611990 con 0x56073c60a710 2024-05-08T18:06:53.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.022+0000 7f21a4da0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60a710 0x56073c703500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:53.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.022+0000 7f21a4da0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60a710 0x56073c703500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46840/0 (socket says 172.21.15.50:46840) 2024-05-08T18:06:53.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.022+0000 7f21a4da0640 1 -- 172.21.15.50:0/1616594968 learned_addr learned my addr 172.21.15.50:0/1616594968 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:53.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.023+0000 7f21a4da0640 1 -- 172.21.15.50:0/1616594968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073c703a40 con 0x56073c60a710 2024-05-08T18:06:53.024 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.023+0000 7f21a4da0640 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60a710 0x56073c703500 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f21940012f0 tx=0x7f2194000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:53.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.023+0000 7f209160a640 1 -- 172.21.15.50:0/1616594968 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f219400f9d0 con 0x56073c60a710 2024-05-08T18:06:53.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.023+0000 7f209160a640 1 -- 172.21.15.50:0/1616594968 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f219400fb70 con 0x56073c60a710 2024-05-08T18:06:53.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.023+0000 7f209160a640 1 -- 172.21.15.50:0/1616594968 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f219400fe50 con 0x56073c60a710 2024-05-08T18:06:53.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.023+0000 7f21a9128800 1 -- 172.21.15.50:0/1616594968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56073c703cd0 con 0x56073c60a710 2024-05-08T18:06:53.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.023+0000 7f21a9128800 1 -- 172.21.15.50:0/1616594968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56073c706980 con 0x56073c60a710 2024-05-08T18:06:53.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.024+0000 7f209160a640 1 -- 172.21.15.50:0/1616594968 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2194017040 con 0x56073c60a710 2024-05-08T18:06:53.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.024+0000 7f209160a640 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f208c03bd40 0x7f208c03e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:53.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.024+0000 7f209160a640 1 -- 172.21.15.50:0/1616594968 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f2194015020 con 0x56073c60a710 2024-05-08T18:06:53.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.024+0000 7f21a459f640 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f208c03bd40 0x7f208c03e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:53.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.024+0000 7f21a9128800 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56073c60eb00 0x56073c70a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:53.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.024+0000 7f21a9128800 1 -- 172.21.15.50:0/1616594968 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c70acc0 con 0x56073c60eb00 2024-05-08T18:06:53.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.025+0000 7f21a55a1640 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56073c60eb00 0x56073c70a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:53.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.025+0000 7f21a459f640 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f208c03bd40 0x7f208c03e200 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f218c054000 tx=0x7f218c04e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:53.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.025+0000 7f21a55a1640 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56073c60eb00 0x56073c70a740 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:53.025 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.025+0000 7f21a55a1640 1 -- 172.21.15.50:0/1616594968 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f219c00a4a0 con 0x56073c60eb00 2024-05-08T18:06:53.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.025+0000 7f21a9128800 1 -- 172.21.15.50:0/1616594968 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c707d10 con 0x56073c60eb00 2024-05-08T18:06:53.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.025+0000 7f21a55a1640 1 -- 172.21.15.50:0/1616594968 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f219c00a4a0 con 0x56073c60eb00 2024-05-08T18:06:53.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.025+0000 7f21a9128800 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56073c70d010 0x56073c70f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:53.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.025+0000 7f21a9128800 1 -- 172.21.15.50:0/1616594968 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c70faa0 con 0x56073c70d010 2024-05-08T18:06:53.026 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.026+0000 7f21a4da0640 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56073c70d010 0x56073c70f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:53.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.026+0000 7f21a4da0640 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56073c70d010 0x56073c70f400 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:53.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.026+0000 7f21a4da0640 1 -- 172.21.15.50:0/1616594968 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f219402d070 con 0x56073c70d010 2024-05-08T18:06:53.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.026+0000 7f21a9128800 1 -- 172.21.15.50:0/1616594968 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c70beb0 con 0x56073c60eb00 2024-05-08T18:06:53.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.027+0000 7f21a55a1640 1 -- 172.21.15.50:0/1616594968 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f219c00a4a0 con 0x56073c60eb00 2024-05-08T18:06:53.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.027+0000 7f21a9128800 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56073c710f00 0x56073c7132f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:53.032 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.027+0000 7f21a9128800 1 -- 172.21.15.50:0/1616594968 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c7139f0 con 0x56073c710f00 2024-05-08T18:06:53.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.027+0000 7f21a459f640 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56073c710f00 0x56073c7132f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:53.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.027+0000 7f21a459f640 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56073c710f00 0x56073c7132f0 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:53.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.027+0000 7f21a459f640 1 -- 172.21.15.50:0/1616594968 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f218c0528e0 con 0x56073c710f00 2024-05-08T18:06:53.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.028+0000 7f21a9128800 1 Processor -- start 2024-05-08T18:06:53.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.028+0000 7f21a9128800 1 -- start start 2024-05-08T18:06:53.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.028+0000 7f21a9128800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c721d20 0x56073c722140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:53.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.028+0000 7f21a9128800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56073c4fc330 con 0x56073c721d20 2024-05-08T18:06:53.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.028+0000 7f21a4da0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c721d20 0x56073c722140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:53.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.028+0000 7f21a4da0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c721d20 0x56073c722140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46844/0 (socket says 172.21.15.50:46844) 2024-05-08T18:06:53.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.028+0000 7f21a4da0640 1 -- 172.21.15.50:0/784416036 learned_addr learned my addr 172.21.15.50:0/784416036 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:53.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.028+0000 7f21a4da0640 1 -- 172.21.15.50:0/784416036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073c722680 con 0x56073c721d20 2024-05-08T18:06:53.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.028+0000 7f21a4da0640 1 --2- 172.21.15.50:0/784416036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c721d20 0x56073c722140 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2194000c00 tx=0x7f219403f300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:53.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.029+0000 7f208a7fc640 1 -- 172.21.15.50:0/784416036 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f219404a370 con 0x56073c721d20 2024-05-08T18:06:53.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.029+0000 7f208a7fc640 1 -- 172.21.15.50:0/784416036 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2194047070 con 0x56073c721d20 2024-05-08T18:06:53.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.029+0000 7f21a9128800 1 -- 172.21.15.50:0/784416036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56073c7230f0 con 0x56073c721d20 2024-05-08T18:06:53.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.029+0000 7f208a7fc640 1 -- 172.21.15.50:0/784416036 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2194018ba0 con 0x56073c721d20 2024-05-08T18:06:53.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.029+0000 7f21a9128800 1 -- 172.21.15.50:0/784416036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56073c7235f0 con 0x56073c721d20 2024-05-08T18:06:53.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.029+0000 7f208a7fc640 1 -- 172.21.15.50:0/784416036 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2194049020 con 0x56073c721d20 2024-05-08T18:06:53.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.030+0000 7f208a7fc640 1 --2- 172.21.15.50:0/784416036 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f207c03bba0 0x7f207c03e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:53.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.030+0000 7f208a7fc640 1 -- 172.21.15.50:0/784416036 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f219403e1d0 con 0x56073c721d20 2024-05-08T18:06:53.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.030+0000 7f21a459f640 1 --2- 172.21.15.50:0/784416036 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f207c03bba0 0x7f207c03e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:53.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.030+0000 7f21a9128800 1 Processor -- start 2024-05-08T18:06:53.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.030+0000 7f21a9128800 1 -- start start 2024-05-08T18:06:53.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.030+0000 7f21a459f640 1 --2- 172.21.15.50:0/784416036 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f207c03bba0 0x7f207c03e060 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x56073c707c30 tx=0x7f218c005a00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:53.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.030+0000 7f21a9128800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c57fcc0 0x56073c57e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:53.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.030+0000 7f21a9128800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56073c6fc2b0 con 0x56073c57fcc0 2024-05-08T18:06:53.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.031+0000 7f21a4da0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c57fcc0 0x56073c57e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:53.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.031+0000 7f21a4da0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c57fcc0 0x56073c57e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46860/0 (socket says 172.21.15.50:46860) 2024-05-08T18:06:53.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.031+0000 7f21a4da0640 1 -- 172.21.15.50:0/2754471946 learned_addr learned my addr 172.21.15.50:0/2754471946 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:53.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.031+0000 7f21a4da0640 1 -- 172.21.15.50:0/2754471946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073c580170 con 0x56073c57fcc0 2024-05-08T18:06:53.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.031+0000 7f21a4da0640 1 --2- 172.21.15.50:0/2754471946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c57fcc0 0x56073c57e3b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f219400f690 tx=0x7f2194026e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb8d5a9e2c42041 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:53.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.031+0000 7f207bfff640 1 -- 172.21.15.50:0/2754471946 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2194057030 con 0x56073c57fcc0 2024-05-08T18:06:53.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.031+0000 7f207bfff640 1 -- 172.21.15.50:0/2754471946 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2194026430 con 0x56073c57fcc0 2024-05-08T18:06:53.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.031+0000 7f207bfff640 1 -- 172.21.15.50:0/2754471946 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2194030480 con 0x56073c57fcc0 2024-05-08T18:06:53.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.032+0000 7f21a9128800 1 -- 172.21.15.50:0/2754471946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c57fcc0 msgr2=0x56073c57e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.032+0000 7f21a9128800 1 --2- 172.21.15.50:0/2754471946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c57fcc0 0x56073c57e3b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f219400f690 tx=0x7f2194026e00 comp rx=0 tx=0).stop 2024-05-08T18:06:53.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.032+0000 7f21a9128800 1 -- 172.21.15.50:0/2754471946 shutdown_connections 2024-05-08T18:06:53.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.032+0000 7f21a9128800 1 --2- 172.21.15.50:0/2754471946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c57fcc0 0x56073c57e3b0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:53.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.032+0000 7f21a9128800 1 -- 172.21.15.50:0/2754471946 >> 172.21.15.50:0/2754471946 conn(0x56073c72b2a0 msgr2=0x56073c729300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:53.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.032+0000 7f21a9128800 1 -- 172.21.15.50:0/2754471946 shutdown_connections 2024-05-08T18:06:53.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.032+0000 7f21a9128800 1 -- 172.21.15.50:0/2754471946 wait complete. 2024-05-08T18:06:53.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.032+0000 7f21a9128800 1 Processor -- start 2024-05-08T18:06:53.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.032+0000 7f21a9128800 1 -- start start 2024-05-08T18:06:53.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.032+0000 7f21a9128800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c57fcc0 0x56073c7384c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:53.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.032+0000 7f21a9128800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56073c57e8f0 con 0x56073c57fcc0 2024-05-08T18:06:53.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.032+0000 7f21a4da0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c57fcc0 0x56073c7384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:53.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.032+0000 7f21a4da0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c57fcc0 0x56073c7384c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:46862/0 (socket says 172.21.15.50:46862) 2024-05-08T18:06:53.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.032+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 learned_addr learned my addr 172.21.15.50:0/3652608195 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:06:53.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.033+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073c738a00 con 0x56073c57fcc0 2024-05-08T18:06:53.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.033+0000 7f21a4da0640 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c57fcc0 0x56073c7384c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f219400f690 tx=0x7f2194038ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:53.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.033+0000 7f207a7fc640 1 -- 172.21.15.50:0/3652608195 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2194043960 con 0x56073c57fcc0 2024-05-08T18:06:53.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.033+0000 7f207a7fc640 1 -- 172.21.15.50:0/3652608195 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2194019470 con 0x56073c57fcc0 2024-05-08T18:06:53.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.033+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56073c738a00 con 0x56073c57fcc0 2024-05-08T18:06:53.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.033+0000 7f207a7fc640 1 -- 172.21.15.50:0/3652608195 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2194019750 con 0x56073c57fcc0 2024-05-08T18:06:53.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.033+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56073c738a00 con 0x56073c57fcc0 2024-05-08T18:06:53.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.034+0000 7f207a7fc640 1 -- 172.21.15.50:0/3652608195 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f21940393f0 con 0x56073c57fcc0 2024-05-08T18:06:53.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.034+0000 7f207a7fc640 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f207003bba0 0x7f207003e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:53.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.034+0000 7f207a7fc640 1 -- 172.21.15.50:0/3652608195 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f2194068070 con 0x56073c57fcc0 2024-05-08T18:06:53.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.034+0000 7f21a459f640 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f207003bba0 0x7f207003e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:53.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.035+0000 7f21a9128800 20 rados->read ofs=0 len=0 2024-05-08T18:06:53.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.035+0000 7f21a9128800 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56073c743c60 0x56073c746120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:53.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.035+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c746870 con 0x56073c743c60 2024-05-08T18:06:53.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.035+0000 7f21a459f640 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f207003bba0 0x7f207003e060 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f218c052fd0 tx=0x7f218c04f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:53.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.035+0000 7f21a55a1640 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56073c743c60 0x56073c746120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:53.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.036+0000 7f21a55a1640 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56073c743c60 0x56073c746120 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:53.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.036+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f219c00a4a0 con 0x56073c743c60 2024-05-08T18:06:53.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.036+0000 7f21a9128800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:53.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.036+0000 7f21a9128800 20 realm 2024-05-08T18:06:53.050 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.036+0000 7f21a9128800 20 rados->read ofs=0 len=0 2024-05-08T18:06:53.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.036+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c746c80 con 0x56073c743c60 2024-05-08T18:06:53.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.036+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f219c00a4a0 con 0x56073c743c60 2024-05-08T18:06:53.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.036+0000 7f21a9128800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:53.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.036+0000 7f21a9128800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:06:53.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.036+0000 7f21a9128800 20 rados->read ofs=0 len=0 2024-05-08T18:06:53.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.036+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c747550 con 0x56073c743c60 2024-05-08T18:06:53.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.036+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f219c00a4a0 con 0x56073c743c60 2024-05-08T18:06:53.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.036+0000 7f21a9128800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:53.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.036+0000 7f21a9128800 20 rados->read ofs=0 len=0 2024-05-08T18:06:53.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.036+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c7478f0 con 0x56073c743c60 2024-05-08T18:06:53.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.037+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f219c00a4a0 con 0x56073c743c60 2024-05-08T18:06:53.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.037+0000 7f21a9128800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:53.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.037+0000 7f21a9128800 20 rados->read ofs=0 len=0 2024-05-08T18:06:53.051 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.037+0000 7f21a9128800 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56073c748650 0x56073c74aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:53.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.037+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c74b050 con 0x56073c748650 2024-05-08T18:06:53.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.037+0000 7f21a4da0640 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56073c748650 0x56073c74aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:53.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.037+0000 7f21a4da0640 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56073c748650 0x56073c74aa40 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:53.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.038+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.038+0000 7f21a9128800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:06:53.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.038+0000 7f21a9128800 20 searching for the correct realm 2024-05-08T18:06:53.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.038+0000 7f21a9128800 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56073c74d000 0x56073c74f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:53.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.038+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56073c74fb90 con 0x56073c74d000 2024-05-08T18:06:53.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.038+0000 7f21a459f640 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56073c74d000 0x56073c74f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:53.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.038+0000 7f21a459f640 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56073c74d000 0x56073c74f4c0 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:53.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.038+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f218c081070 con 0x56073c74d000 2024-05-08T18:06:53.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.038+0000 7f21a459f640 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f218c076040 0x7f218c05e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:06:53.052 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.038+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56073c74fb90 con 0x7f218c076040 2024-05-08T18:06:53.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.039+0000 7f21a55a1640 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f218c076040 0x7f218c05e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:06:53.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.039+0000 7f21a55a1640 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f218c076040 0x7f218c05e480 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:06:53.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.039+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219c00a4a0 con 0x7f218c076040 2024-05-08T18:06:53.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.039+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56073c7478f0 con 0x56073c748650 2024-05-08T18:06:53.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.039+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.039+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56073c7478f0 con 0x56073c743c60 2024-05-08T18:06:53.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.040+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219c00a4a0 con 0x56073c743c60 2024-05-08T18:06:53.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.040+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56073c7478f0 con 0x56073c743c60 2024-05-08T18:06:53.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.040+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219c00a4a0 con 0x56073c743c60 2024-05-08T18:06:53.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.040+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56073c7478f0 con 0x56073c743c60 2024-05-08T18:06:53.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.040+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219c00a4a0 con 0x56073c743c60 2024-05-08T18:06:53.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.040+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56073c7478f0 con 0x56073c748650 2024-05-08T18:06:53.053 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.040+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v29'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.040+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56073c7478f0 con 0x7f218c076040 2024-05-08T18:06:53.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.041+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219c00a4a0 con 0x7f218c076040 2024-05-08T18:06:53.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.041+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56073c7478f0 con 0x56073c74d000 2024-05-08T18:06:53.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.041+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f218c081070 con 0x56073c74d000 2024-05-08T18:06:53.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.041+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56073c7478f0 con 0x56073c743c60 2024-05-08T18:06:53.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.041+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219c00a4a0 con 0x56073c743c60 2024-05-08T18:06:53.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.041+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56073c7478f0 con 0x56073c748650 2024-05-08T18:06:53.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.042+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.042+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56073c7478f0 con 0x56073c743c60 2024-05-08T18:06:53.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.042+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219c00a4a0 con 0x56073c743c60 2024-05-08T18:06:53.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.042+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56073c7478f0 con 0x7f218c076040 2024-05-08T18:06:53.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.042+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f219c032040 con 0x7f218c076040 2024-05-08T18:06:53.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.042+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x56073c7478f0 con 0x56073c74d000 2024-05-08T18:06:53.054 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.043+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f218c081070 con 0x56073c74d000 2024-05-08T18:06:53.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.043+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x56073c74d000 2024-05-08T18:06:53.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.043+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f218c081070 con 0x56073c74d000 2024-05-08T18:06:53.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.043+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x56073c748650 2024-05-08T18:06:53.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.043+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.043+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x56073c74d000 2024-05-08T18:06:53.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.043+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f218c081070 con 0x56073c74d000 2024-05-08T18:06:53.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.043+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x56073c74d000 2024-05-08T18:06:53.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.044+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f218c081070 con 0x56073c74d000 2024-05-08T18:06:53.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.044+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x56073c743c60 2024-05-08T18:06:53.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.044+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.044+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x56073c743c60 2024-05-08T18:06:53.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.044+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v29'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.055 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.044+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x56073c748650 2024-05-08T18:06:53.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.044+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.044+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x56073c74d000 2024-05-08T18:06:53.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.045+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f218c081070 con 0x56073c74d000 2024-05-08T18:06:53.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.045+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x7f218c076040 2024-05-08T18:06:53.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.045+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219c032040 con 0x7f218c076040 2024-05-08T18:06:53.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.045+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x56073c743c60 2024-05-08T18:06:53.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.045+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.045+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x56073c748650 2024-05-08T18:06:53.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.045+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.045+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x56073c74d000 2024-05-08T18:06:53.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.046+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f218c081070 con 0x56073c74d000 2024-05-08T18:06:53.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.046+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x56073c748650 2024-05-08T18:06:53.056 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.046+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.046+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x7f218c076040 2024-05-08T18:06:53.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.046+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219c032040 con 0x7f218c076040 2024-05-08T18:06:53.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.046+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x56073c74d000 2024-05-08T18:06:53.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.046+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f218c081070 con 0x56073c74d000 2024-05-08T18:06:53.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.046+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x56073c748650 2024-05-08T18:06:53.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.047+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.047+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x56073c743c60 2024-05-08T18:06:53.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.047+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v29'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.047+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x7f218c090040 con 0x7f218c076040 2024-05-08T18:06:53.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.047+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f219c032040 con 0x7f218c076040 2024-05-08T18:06:53.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.047+0000 7f21a9128800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:06:53.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.047+0000 7f21a9128800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:06:53.057 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.047+0000 7f21a9128800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:06:53.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.047+0000 7f21a9128800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:06:53.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.047+0000 7f21a9128800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:06:53.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.047+0000 7f21a9128800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:06:53.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.047+0000 7f21a9128800 20 rados->read ofs=0 len=0 2024-05-08T18:06:53.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.047+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c74b910 con 0x56073c743c60 2024-05-08T18:06:53.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a9128800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:53.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a9128800 20 rados->read ofs=0 len=0 2024-05-08T18:06:53.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c74b910 con 0x56073c743c60 2024-05-08T18:06:53.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a9128800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:06:53.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a9128800 20 rados->read ofs=0 len=0 2024-05-08T18:06:53.058 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c74b910 con 0x56073c74d000 2024-05-08T18:06:53.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f218c081070 con 0x56073c74d000 2024-05-08T18:06:53.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a9128800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:06:53.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a9128800 20 zone default found 2024-05-08T18:06:53.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a9128800 4 Realm: () 2024-05-08T18:06:53.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a9128800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:06:53.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a9128800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:06:53.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a9128800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:06:53.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a9128800 20 zonegroup default 2024-05-08T18:06:53.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a9128800 20 rados->read ofs=0 len=0 2024-05-08T18:06:53.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.048+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c74c210 con 0x56073c748650 2024-05-08T18:06:53.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.049+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.049+0000 7f21a9128800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:53.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.049+0000 7f21a9128800 20 rados->read ofs=0 len=0 2024-05-08T18:06:53.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.049+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c751a20 con 0x56073c74d000 2024-05-08T18:06:53.059 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.049+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f218c081070 con 0x56073c74d000 2024-05-08T18:06:53.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.049+0000 7f21a9128800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:53.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.049+0000 7f21a9128800 20 rados->read ofs=0 len=0 2024-05-08T18:06:53.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.049+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c751eb0 con 0x56073c74d000 2024-05-08T18:06:53.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.049+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f218c081070 con 0x56073c74d000 2024-05-08T18:06:53.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.049+0000 7f21a9128800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:06:53.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.049+0000 7f21a9128800 20 started sync module instance, tier type = 2024-05-08T18:06:53.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.049+0000 7f21a9128800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:06:53.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.049+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c750e90 con 0x7f218c076040 2024-05-08T18:06:53.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.050+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219c032040 con 0x7f218c076040 2024-05-08T18:06:53.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.050+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94589079065536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c750e90 con 0x7f218c076040 2024-05-08T18:06:53.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.050+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c7529b0 con 0x56073c743c60 2024-05-08T18:06:53.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.051+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.060 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.051+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94589079069824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c7529b0 con 0x56073c743c60 2024-05-08T18:06:53.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.051+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c7539c0 con 0x56073c74d000 2024-05-08T18:06:53.061 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.051+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94589079065536] v29'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219c032040 con 0x7f218c076040 2024-05-08T18:06:53.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.052+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f218c081070 con 0x56073c74d000 2024-05-08T18:06:53.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.052+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94589079073904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c753db0 con 0x56073c74d000 2024-05-08T18:06:53.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.052+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c7547f0 con 0x7f218c076040 2024-05-08T18:06:53.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.053+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94589079069824] v29'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.053+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94589079073904] v29'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f218c081070 con 0x56073c74d000 2024-05-08T18:06:53.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.053+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219c032040 con 0x7f218c076040 2024-05-08T18:06:53.066 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.053+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94589079063008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c752d50 con 0x7f218c076040 2024-05-08T18:06:53.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.053+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c755090 con 0x56073c743c60 2024-05-08T18:06:53.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.054+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.054+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94589079079824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c755460 con 0x56073c743c60 2024-05-08T18:06:53.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.054+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c755f80 con 0x56073c748650 2024-05-08T18:06:53.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.055+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94589079063008] v29'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219c032040 con 0x7f218c076040 2024-05-08T18:06:53.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.055+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.056+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94589079083600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c756370 con 0x56073c748650 2024-05-08T18:06:53.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.056+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c756d70 con 0x56073c748650 2024-05-08T18:06:53.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.056+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94589079079824] v29'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.056+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94589079083600] v29'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.057+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(56 notify.6 [create] v29'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.057+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94589079080880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c757160 con 0x56073c748650 2024-05-08T18:06:53.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.057+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c7579b0 con 0x56073c748650 2024-05-08T18:06:53.067 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.058+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(58 notify.7 [create] v29'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.058+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94589079080880] v29'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.058+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94589079090016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c757df0 con 0x56073c748650 2024-05-08T18:06:53.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.058+0000 7f21a9128800 20 add_watcher() i=0 2024-05-08T18:06:53.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.058+0000 7f21a9128800 20 add_watcher() i=1 2024-05-08T18:06:53.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.058+0000 7f21a9128800 20 add_watcher() i=2 2024-05-08T18:06:53.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.058+0000 7f21a9128800 20 add_watcher() i=3 2024-05-08T18:06:53.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.058+0000 7f21a9128800 20 add_watcher() i=4 2024-05-08T18:06:53.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.058+0000 7f21a9128800 20 add_watcher() i=5 2024-05-08T18:06:53.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.058+0000 7f21a9128800 20 add_watcher() i=6 2024-05-08T18:06:53.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.058+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94589079090016] v29'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.059+0000 7f21a9128800 20 add_watcher() i=7 2024-05-08T18:06:53.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.059+0000 7f21a9128800 2 all 8 watchers are set, enabling cache 2024-05-08T18:06:53.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.059+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c7568e0 con 0x56073c743c60 2024-05-08T18:06:53.068 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.059+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.059+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94589079091328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c7568e0 con 0x56073c743c60 2024-05-08T18:06:53.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.060+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94589079091328] v29'271 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.060+0000 7f205afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:53.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.060+0000 7f21a9128800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:06:53.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.060+0000 7f21a9128800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:06:53.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.060+0000 7f21a9128800 5 note: GC not initialized 2024-05-08T18:06:53.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.060+0000 7f2059fe3640 20 reqs_thread_entry: start 2024-05-08T18:06:53.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.060+0000 7f21a9128800 20 init_complete bucket index max shards: 11 2024-05-08T18:06:53.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.061+0000 7f21a9128800 20 Filter name: none 2024-05-08T18:06:53.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.061+0000 7f1fcb7fe640 20 reqs_thread_entry: start 2024-05-08T18:06:53.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.061+0000 7f21a9128800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-05-08T18:06:53.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.061+0000 7f21a9128800 20 rados->read ofs=0 len=0 2024-05-08T18:06:53.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.061+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c78e310 con 0x56073c748650 2024-05-08T18:06:53.069 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.061+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(62 ada [call out=48b,read 0~473 out=473b,getxattrs out=75b] v0'0 uv14 ondisk = 0) ==== 231+0+596 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.061+0000 7f21a9128800 20 rados_obj.operate() r=0 bl.length=473 2024-05-08T18:06:53.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.061+0000 7f21a9128800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-05-08T18:06:53.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.061+0000 7f21a9128800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:53.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.061+0000 7f21a9128800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-05-08T18:06:53.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.061+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~473 in=473b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c794cc0 con 0x56073c748650 2024-05-08T18:06:53.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.062+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~473] v29'15 uv15 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.062+0000 7f21a9128800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:53.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.062+0000 7f21a9128800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:53.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.062+0000 7f21a9128800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-05-08T18:06:53.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.062+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94589079340672 in=661b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x56073c7902c0 con 0x56073c743c60 2024-05-08T18:06:53.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.063+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== watch-notify(notify (1) cookie 94589079079824 notify 124554051627 ret 0) ==== 691+0+0 (crc 0 0 0) 0x7f219c03d040 con 0x56073c743c60 2024-05-08T18:06:53.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.063+0000 7f207bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 124554051627 cookie 94589079079824 notifier 4914 bl.length()=649 2024-05-08T18:06:53.070 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.063+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(64 notify.4 [notify cookie 94589079340672 out=8b] v0'0 uv279 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.063+0000 7f207bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-05-08T18:06:53.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.063+0000 7f207bfff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-05-08T18:06:53.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.063+0000 7f207bfff640 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x7f2068003ea0 con 0x56073c743c60 2024-05-08T18:06:53.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.063+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.071 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.063+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== watch-notify(notify_complete (2) cookie 94589079340672 notify 124554051627 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f219c018b90 con 0x56073c743c60 2024-05-08T18:06:53.071 INFO:teuthology.orchestra.run.smithi050.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-05-08T18:06:51.092020Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"users","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-05-08T18:06:51.092006Z","tags":[],"group_ids":[]} 2024-05-08T18:06:53.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.064+0000 7f205afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:06:53.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.064+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94589079091328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c793f20 con 0x56073c743c60 2024-05-08T18:06:53.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.065+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94589079091328] v29'272 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.065+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94589079065536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c793f20 con 0x7f218c076040 2024-05-08T18:06:53.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.066+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94589079065536] v29'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219c032040 con 0x7f218c076040 2024-05-08T18:06:53.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.066+0000 7f21a9128800 20 remove_watcher() i=0 2024-05-08T18:06:53.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.066+0000 7f21a9128800 2 removed watcher, disabling cache 2024-05-08T18:06:53.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.066+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94589079069824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c793f20 con 0x56073c743c60 2024-05-08T18:06:53.077 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.067+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94589079069824] v29'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.067+0000 7f21a9128800 20 remove_watcher() i=1 2024-05-08T18:06:53.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.067+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94589079073904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c793f20 con 0x56073c74d000 2024-05-08T18:06:53.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.068+0000 7f21a459f640 1 -- 172.21.15.50:0/3652608195 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94589079073904] v29'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f218c081070 con 0x56073c74d000 2024-05-08T18:06:53.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.068+0000 7f21a9128800 20 remove_watcher() i=2 2024-05-08T18:06:53.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.068+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94589079063008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c7513c0 con 0x7f218c076040 2024-05-08T18:06:53.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.069+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94589079063008] v29'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219c032040 con 0x7f218c076040 2024-05-08T18:06:53.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.069+0000 7f21a9128800 20 remove_watcher() i=3 2024-05-08T18:06:53.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.069+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94589079079824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c7513c0 con 0x56073c743c60 2024-05-08T18:06:53.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.070+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94589079079824] v29'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219c032040 con 0x56073c743c60 2024-05-08T18:06:53.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.070+0000 7f21a9128800 20 remove_watcher() i=4 2024-05-08T18:06:53.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.070+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94589079083600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c754b90 con 0x56073c748650 2024-05-08T18:06:53.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.071+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94589079083600] v29'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.071+0000 7f21a9128800 20 remove_watcher() i=5 2024-05-08T18:06:53.078 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.071+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94589079080880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c7513c0 con 0x56073c748650 2024-05-08T18:06:53.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.072+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94589079080880] v29'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.072+0000 7f21a9128800 20 remove_watcher() i=6 2024-05-08T18:06:53.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.072+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94589079090016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x56073c7513c0 con 0x56073c748650 2024-05-08T18:06:53.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.073+0000 7f21a4da0640 1 -- 172.21.15.50:0/3652608195 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94589079090016] v29'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f219402d070 con 0x56073c748650 2024-05-08T18:06:53.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.073+0000 7f21a9128800 20 remove_watcher() i=7 2024-05-08T18:06:53.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56073c748650 msgr2=0x56073c74aa40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56073c748650 0x56073c74aa40 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:53.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56073c743c60 msgr2=0x56073c746120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56073c743c60 0x56073c746120 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:53.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f218c076040 msgr2=0x7f218c05e480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f218c076040 0x7f218c05e480 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:53.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56073c74d000 msgr2=0x56073c74f4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.079 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56073c74d000 0x56073c74f4c0 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:53.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f207003bba0 msgr2=0x7f207003e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f207003bba0 0x7f207003e060 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f218c052fd0 tx=0x7f218c04f000 comp rx=0 tx=0).stop 2024-05-08T18:06:53.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a55a1640 1 -- 172.21.15.50:0/3652608195 reap_dead start 2024-05-08T18:06:53.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c57fcc0 msgr2=0x56073c7384c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c57fcc0 0x56073c7384c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f219400f690 tx=0x7f2194038ee0 comp rx=0 tx=0).stop 2024-05-08T18:06:53.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 shutdown_connections 2024-05-08T18:06:53.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 --2- 172.21.15.50:0/3652608195 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c57fcc0 0x56073c7384c0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:53.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 >> 172.21.15.50:0/3652608195 conn(0x56073c72b2a0 msgr2=0x56073c728900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:53.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 shutdown_connections 2024-05-08T18:06:53.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 -- 172.21.15.50:0/3652608195 wait complete. 2024-05-08T18:06:53.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 -- 172.21.15.50:0/784416036 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f207c03bba0 msgr2=0x7f207c03e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 --2- 172.21.15.50:0/784416036 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f207c03bba0 0x7f207c03e060 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x56073c707c30 tx=0x7f218c005a00 comp rx=0 tx=0).stop 2024-05-08T18:06:53.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 -- 172.21.15.50:0/784416036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c721d20 msgr2=0x56073c722140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 --2- 172.21.15.50:0/784416036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c721d20 0x56073c722140 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2194000c00 tx=0x7f219403f300 comp rx=0 tx=0).stop 2024-05-08T18:06:53.080 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 -- 172.21.15.50:0/784416036 shutdown_connections 2024-05-08T18:06:53.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 --2- 172.21.15.50:0/784416036 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f207c03bba0 0x7f207c03e060 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:53.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 --2- 172.21.15.50:0/784416036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c721d20 0x56073c722140 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:53.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 -- 172.21.15.50:0/784416036 >> 172.21.15.50:0/784416036 conn(0x56073c715dc0 msgr2=0x56073c7161c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:53.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 -- 172.21.15.50:0/784416036 shutdown_connections 2024-05-08T18:06:53.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.074+0000 7f21a9128800 1 -- 172.21.15.50:0/784416036 wait complete. 2024-05-08T18:06:53.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.075+0000 7f21a9128800 1 -- 172.21.15.50:0/1616594968 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56073c70d010 msgr2=0x56073c70f400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.075+0000 7f21a9128800 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x56073c70d010 0x56073c70f400 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:53.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.075+0000 7f21a9128800 1 -- 172.21.15.50:0/1616594968 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56073c60eb00 msgr2=0x56073c70a740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.075+0000 7f21a9128800 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x56073c60eb00 0x56073c70a740 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:53.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.075+0000 7f21a9128800 1 -- 172.21.15.50:0/1616594968 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56073c710f00 msgr2=0x56073c7132f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.075+0000 7f21a9128800 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x56073c710f00 0x56073c7132f0 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:06:53.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.075+0000 7f21a9128800 1 -- 172.21.15.50:0/1616594968 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f208c03bd40 msgr2=0x7f208c03e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.075+0000 7f21a9128800 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f208c03bd40 0x7f208c03e200 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f218c054000 tx=0x7f218c04e000 comp rx=0 tx=0).stop 2024-05-08T18:06:53.081 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.075+0000 7f21a9128800 1 -- 172.21.15.50:0/1616594968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60a710 msgr2=0x56073c703500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:06:53.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.075+0000 7f21a9128800 1 --2- 172.21.15.50:0/1616594968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56073c60a710 0x56073c703500 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f21940012f0 tx=0x7f2194000ed0 comp rx=0 tx=0).stop 2024-05-08T18:06:53.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.075+0000 7f21a55a1640 1 -- 172.21.15.50:0/1616594968 reap_dead start 2024-05-08T18:06:53.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.075+0000 7f21a9128800 1 -- 172.21.15.50:0/1616594968 shutdown_connections 2024-05-08T18:06:53.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.075+0000 7f21a9128800 1 -- 172.21.15.50:0/1616594968 >> 172.21.15.50:0/1616594968 conn(0x56073c605c90 msgr2=0x56073c6f5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:06:53.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.075+0000 7f21a9128800 1 -- 172.21.15.50:0/1616594968 shutdown_connections 2024-05-08T18:06:53.082 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:06:53.075+0000 7f21a9128800 1 -- 172.21.15.50:0/1616594968 wait complete. 2024-05-08T18:06:53.083 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:53.083 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-05-08T18:06:51.092020Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-05-08T18:06:51.092006Z', 'tags': [], 'group_ids': []} 2024-05-08T18:06:53.083 ERROR:root:0 2024-05-08T18:06:53.083 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': True} 2024-05-08T18:06:53.090 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:53.090 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:53.094 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:53.094 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': 'false'} 2024-05-08T18:06:53.100 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:53.100 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:53.103 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:53.104 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'create'] {'uid': 'foo', 'access-key': 'p5YnriCv1nAtykxBrupQ', 'secret-key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'} 2024-05-08T18:06:53.112 INFO:tasks.radosgw_admin_rest: json result: [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True}] 2024-05-08T18:06:53.112 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:53.115 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:53.115 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'rm'] {'uid': 'foo', 'access-key': 'p5YnriCv1nAtykxBrupQ'} 2024-05-08T18:06:53.122 INFO:tasks.radosgw_admin_rest: empty result 2024-05-08T18:06:53.122 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:53.126 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:53.126 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'create'] {'subuser': 'foo:foo1', 'secret-key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'key-type': 'swift'} 2024-05-08T18:06:53.133 INFO:tasks.radosgw_admin_rest: json result: [{'id': 'foo:foo1', 'permissions': ''}] 2024-05-08T18:06:53.133 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:53.137 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:53.137 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'create'] {'subuser': 'foo:foo2', 'secret-key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'key-type': 'swift'} 2024-05-08T18:06:53.145 INFO:tasks.radosgw_admin_rest: json result: [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}] 2024-05-08T18:06:53.146 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:53.150 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:53.150 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'rm'] {'subuser': 'foo:foo1', 'key-type': 'swift'} 2024-05-08T18:06:53.156 INFO:tasks.radosgw_admin_rest: empty result 2024-05-08T18:06:53.157 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:53.160 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:53.160 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'rm'] {'subuser': 'foo:foo1'} 2024-05-08T18:06:53.166 INFO:tasks.radosgw_admin_rest: empty result 2024-05-08T18:06:53.166 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:53.170 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:53.170 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'rm'] {'subuser': 'foo:foo2', 'key-type': 'swift', '{purge-keys': True} 2024-05-08T18:06:53.177 INFO:tasks.radosgw_admin_rest: empty result 2024-05-08T18:06:53.177 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:06:53.181 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:53.181 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo'} 2024-05-08T18:06:53.185 INFO:tasks.radosgw_admin_rest: json result: [] 2024-05-08T18:06:53.186 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo', 'stats': True} 2024-05-08T18:06:53.190 INFO:tasks.radosgw_admin_rest: json result: [] 2024-05-08T18:06:53.201 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo'} 2024-05-08T18:06:53.206 INFO:tasks.radosgw_admin_rest: json result: ['myfoo'] 2024-05-08T18:06:53.206 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-05-08T18:06:53.210 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': '36133b23-abaa-446a-bf67-ceb1d1559ceb', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11', 'marker': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-05-08T18:06:53.200375Z', 'creation_time': '2024-05-08T18:06:53.197550Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-05-08T18:06:53.211 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo', 'stats': True} 2024-05-08T18:06:53.215 INFO:tasks.radosgw_admin_rest: json result: [{'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': '36133b23-abaa-446a-bf67-ceb1d1559ceb', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11', 'marker': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-05-08T18:06:53.200375Z', 'creation_time': '2024-05-08T18:06:53.197550Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}] 2024-05-08T18:06:53.220 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-05-08T18:06:53.224 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': '36133b23-abaa-446a-bf67-ceb1d1559ceb', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11', 'marker': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-05-08T18:06:53.200375Z', 'creation_time': '2024-05-08T18:06:53.197550Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-05-08T18:06:53.224 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'doesnotexist'} 2024-05-08T18:06:53.228 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchBucket', 'Message': '', 'RequestId': 'tx000002ea1d356c9ed4f47-00663bbf3d-4221-default', 'HostId': '4221-default-default'} 2024-05-08T18:06:53.233 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'unlink'] {'uid': 'foo', 'bucket': 'myfoo'} 2024-05-08T18:06:53.239 INFO:tasks.radosgw_admin_rest: empty result 2024-05-08T18:06:53.239 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'fud', 'display-name': 'Fud', 'access-key': 'p5YnriCv1nAtykxBrupQ', 'secret-key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'max-buckets': '1'} 2024-05-08T18:06:53.249 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'fud', 'tenant': '', 'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:06:53.303 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'link'] {'uid': 'fud', 'bucket': 'myfoo', 'bucket-id': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11'} 2024-05-08T18:06:53.314 INFO:tasks.radosgw_admin_rest: empty result 2024-05-08T18:06:53.316 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'link'] {'uid': 'foo', 'bucket': 'myfoo', 'bucket-id': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11'} 2024-05-08T18:06:53.326 INFO:tasks.radosgw_admin_rest: empty result 2024-05-08T18:06:53.326 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'fud'} 2024-05-08T18:06:53.335 INFO:tasks.radosgw_admin_rest: empty result 2024-05-08T18:06:53.342 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['object', 'rm'] {'bucket': 'myfoo', 'object': 'four'} 2024-05-08T18:06:53.349 INFO:tasks.radosgw_admin_rest: empty result 2024-05-08T18:06:53.349 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-05-08T18:06:53.354 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': '36133b23-abaa-446a-bf67-ceb1d1559ceb', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11', 'marker': '1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-05-08T18:06:53.324154Z', 'creation_time': '2024-05-08T18:06:53.197550Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-05-08T18:07:53.400 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'categories': 'delete_obj'} 2024-05-08T18:07:53.420 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'anonymous', 'categories': [], 's3select': {}}]}, {'user': 'bar', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'bar', 'categories': [], 's3select': {}}]}, {'user': 'bud', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'bud', 'categories': [], 's3select': {}}]}, {'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 's3select': {}}, {'bucket': 'useless-bucket', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}, {'user': 'fud', 'buckets': [{'bucket': 'myfoo', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'fud', 'categories': [], 's3select': {}}, {'bucket': 'myfoo7', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'fud', 'categories': [], 's3select': {}}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'bar', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'bud', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'fud', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-05-08T18:07:53.420 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] None 2024-05-08T18:07:53.436 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}, {'user': 'bar', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'bar', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}, {'user': 'bud', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'bud', 'categories': [{'category': 'get_obj', 'bytes_sent': 16, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}, {'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'useless-bucket', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}, {'user': 'fud', 'buckets': [{'bucket': 'myfoo', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'fud', 'categories': [{'category': 'put_obj', 'bytes_sent': 234, 'bytes_received': 0, 'ops': 1, 'successful_ops': 0}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'myfoo7', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'fud', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'get_obj', 'bytes_sent': 8, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'bar', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 0, 'bytes_received': 8, 'ops': 2, 'successful_ops': 2, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'bud', 'categories': [{'category': 'get_obj', 'bytes_sent': 16, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 'total': {'bytes_sent': 16, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 10, 'successful_ops': 10, 'bytes_processed': 0, 'bytes_returned': 0}}, {'user': 'fud', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'get_obj', 'bytes_sent': 8, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 234, 'bytes_received': 8, 'ops': 2, 'successful_ops': 1}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 242, 'bytes_received': 8, 'ops': 5, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-05-08T18:07:53.437 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo'} 2024-05-08T18:07:53.441 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}, {'bucket': 'useless-bucket', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}], 's3select': {'bytes_processed': 0, 'bytes_returned': 0}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 10, 'successful_ops': 10, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-05-08T18:07:53.442 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'create_bucket'} 2024-05-08T18:07:53.447 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}, {'bucket': 'useless-bucket', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-05-08T18:07:53.447 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'put_obj'} 2024-05-08T18:07:53.452 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}], 's3select': {}}, {'bucket': 'useless-bucket', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-05-08T18:07:53.452 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'delete_obj'} 2024-05-08T18:07:53.456 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 's3select': {}}, {'bucket': 'useless-bucket', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-05-08T18:07:53.457 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'delete_bucket'} 2024-05-08T18:07:53.461 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [], 's3select': {}}, {'bucket': 'useless-bucket', 'time': '2024-05-08T18:00:00.000000Z', 'epoch': 1715191200, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 's3select': {}}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1, 'bytes_processed': 0, 'bytes_returned': 0}}]} 2024-05-08T18:07:53.461 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'trim'] {'uid': 'foo'} 2024-05-08T18:07:53.468 INFO:tasks.radosgw_admin_rest: empty result 2024-05-08T18:07:53.468 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo'} 2024-05-08T18:07:53.472 INFO:tasks.radosgw_admin_rest: json result: {'entries': [], 'summary': []} 2024-05-08T18:07:53.473 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': True} 2024-05-08T18:07:53.482 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:07:53.486 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': 'false'} 2024-05-08T18:07:53.495 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'foo', 'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:07:53.737 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo'} 2024-05-08T18:07:53.742 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'BucketAlreadyExists', 'Message': '', 'RequestId': 'tx00000d0e0a175d39174da-00663bbf79-4221-default', 'HostId': '4221-default-default'} 2024-05-08T18:07:53.840 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['policy', 'show'] {'bucket': 'myfoo', 'object': b'bb3aec0fdcdbc2974890f805c585d432'} 2024-05-08T18:07:53.845 INFO:tasks.radosgw_admin_rest: json result: {'acl': {'acl_user_map': [{'user': 'foo', 'acl': 15}], 'acl_group_map': [], 'grant_map': [{'id': 'foo', 'grant': {'type': {'type': 0}, 'id': 'foo', 'name': 'Foo', 'permission': {'flags': 15}}}]}, 'owner': {'id': 'foo', 'display_name': 'Foo'}} 2024-05-08T18:07:53.850 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['policy', 'show'] {'bucket': 'myfoo', 'object': b'bb3aec0fdcdbc2974890f805c585d432'} 2024-05-08T18:07:53.854 INFO:tasks.radosgw_admin_rest: json result: {'acl': {'acl_user_map': [{'user': 'foo', 'acl': 15}], 'acl_group_map': [{'group': 1, 'acl': 1}], 'grant_map': [{'id': '', 'grant': {'type': {'type': 2}, 'group': 1, 'permission': {'flags': 1}}}, {'id': 'foo', 'grant': {'type': {'type': 0}, 'id': 'foo', 'name': 'Foo', 'permission': {'flags': 15}}}]}, 'owner': {'id': 'foo', 'display_name': 'Foo'}} 2024-05-08T18:07:53.966 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'rm'] {'bucket': 'myfoo', 'purge-objects': True} 2024-05-08T18:07:54.003 INFO:tasks.radosgw_admin_rest: empty result 2024-05-08T18:07:54.003 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['caps', 'add'] {'uid': 'foo', 'user-caps': 'usage=read'} 2024-05-08T18:07:54.009 INFO:tasks.radosgw_admin_rest: json result: [{'type': 'usage', 'perm': 'read'}] 2024-05-08T18:07:54.009 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['caps', 'rm'] {'uid': 'foo', 'user-caps': 'usage=read'} 2024-05-08T18:07:54.014 INFO:tasks.radosgw_admin_rest: json result: [] 2024-05-08T18:07:54.024 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo'} 2024-05-08T18:07:54.028 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'BucketAlreadyExists', 'Message': '', 'RequestId': 'tx000002c5c011fe72c640c-00663bbf7a-4221-default', 'HostId': '4221-default-default'} 2024-05-08T18:07:54.047 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo', 'purge-data': True} 2024-05-08T18:07:54.077 INFO:tasks.radosgw_admin_rest: empty result 2024-05-08T18:07:54.078 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-05-08T18:07:54.082 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchUser', 'Message': '', 'RequestId': 'tx0000042ccbac892c3e292-00663bbf7a-4221-default', 'HostId': '4221-default-default'} 2024-05-08T18:07:54.082 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['info', ''] None 2024-05-08T18:07:54.085 INFO:tasks.radosgw_admin_rest: json result: {'info': {'storage_backends': [{'name': 'rados', 'cluster_id': 'd8088d1f-616e-46b1-995a-85e572267d38'}]}} 2024-05-08T18:07:54.086 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'ratelimit_user', 'display-name': 'Foo', 'email': 'foo@foo.com', 'access-key': '9te6NH5mcdcq0Tc5i8i1', 'secret-key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'max-buckets': '1000'} 2024-05-08T18:07:54.096 INFO:tasks.radosgw_admin_rest: json result: {'full_user_id': 'ratelimit_user', 'tenant': '', 'user_id': 'ratelimit_user', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'ratelimit_user', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-05-08T18:07:54.096 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user'} 2024-05-08T18:07:54.099 INFO:tasks.radosgw_admin_rest: json result: {'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-05-08T18:07:54.100 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_userstring'} 2024-05-08T18:07:54.104 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx000007ca04e460e667bf5-00663bbf7a-4221-default', 'HostId': '4221-default-default'} 2024-05-08T18:07:54.104 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user'} 2024-05-08T18:07:54.108 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx00000b1c006a324ea17a1-00663bbf7a-4221-default', 'HostId': '4221-default-default'} 2024-05-08T18:07:54.118 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket'} 2024-05-08T18:07:54.122 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-05-08T18:07:54.122 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucketstring'} 2024-05-08T18:07:54.126 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx000004a1cab3c97eac79a-00663bbf7a-4221-default', 'HostId': '4221-default-default'} 2024-05-08T18:07:54.126 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket'} 2024-05-08T18:07:54.130 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx000007771c35cbddcd2a5-00663bbf7a-4221-default', 'HostId': '4221-default-default'} 2024-05-08T18:07:54.130 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'global': 'true'} 2024-05-08T18:07:54.135 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}, 'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}, 'anonymous_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-05-08T18:07:54.135 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_userstring', 'enabled': 'true'} 2024-05-08T18:07:54.139 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx000003ce52265dad6e53e-00663bbf7a-4221-default', 'HostId': '4221-default-default'} 2024-05-08T18:07:54.139 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user'} 2024-05-08T18:07:54.142 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx0000047fbbddaa0ed0117-00663bbf7a-4221-default', 'HostId': '4221-default-default'} 2024-05-08T18:07:54.143 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucketstring', 'enabled': 'true'} 2024-05-08T18:07:54.146 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx00000757b2a107c225b04-00663bbf7a-4221-default', 'HostId': '4221-default-default'} 2024-05-08T18:07:54.147 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'enabled': 'true'} 2024-05-08T18:07:54.150 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx000001af82867c8694360-00663bbf7a-4221-default', 'HostId': '4221-default-default'} 2024-05-08T18:07:54.150 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user', 'enabled': 'true', 'max-read-bytes': '2'} 2024-05-08T18:07:54.159 INFO:tasks.radosgw_admin_rest: empty result 2024-05-08T18:07:54.159 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user'} 2024-05-08T18:07:54.163 INFO:tasks.radosgw_admin_rest: json result: {'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 2, 'max_write_bytes': 0, 'enabled': True}} 2024-05-08T18:07:54.163 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket', 'enabled': 'true', 'max-write-bytes': '2'} 2024-05-08T18:07:54.169 INFO:tasks.radosgw_admin_rest: empty result 2024-05-08T18:07:54.170 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket'} 2024-05-08T18:07:54.173 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 2, 'enabled': True}} 2024-05-08T18:07:54.173 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'global': 'true', 'enabled': 'true'} 2024-05-08T18:07:54.179 INFO:tasks.radosgw_admin_rest: empty result 2024-05-08T18:07:54.183 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2024-05-08T18:07:54.194 DEBUG:tasks.rgw.client.0:waiting for process to exit 2024-05-08T18:07:54.194 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T18:07:54.195 INFO:tasks.rgw.client.0.smithi050.stdout:2024-05-08T18:07:54.196+0000 7fc3dd70d640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term radosgw --rgw-frontends beast port=80 -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground (PID: 34485) UID: 0 2024-05-08T18:07:54.195 INFO:tasks.rgw.client.0.smithi050.stdout:2024-05-08T18:07:54.196+0000 7fc3e340f880 -1 shutting down 2024-05-08T18:07:55.739 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~0s 2024-05-08T18:08:00.295 INFO:tasks.rgw.client.0:Stopped 2024-05-08T18:08:00.296 DEBUG:teuthology.orchestra.run.smithi050:> rm -f /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock 2024-05-08T18:08:00.316 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/vault-root-token 2024-05-08T18:08:00.388 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /home/ubuntu/cephtest/url_file 2024-05-08T18:08:00.459 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process', '--include-all'] 2024-05-08T18:08:00.459 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process', '--include-all'] 2024-05-08T18:08:00.459 DEBUG:teuthology.orchestra.run.smithi050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all 2024-05-08T18:08:00.549 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setuser ceph since I am not root 2024-05-08T18:08:00.549 INFO:teuthology.orchestra.run.smithi050.stderr:ignoring --setgroup ceph since I am not root 2024-05-08T18:08:00.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.552+0000 7ff47680e800 1 Processor -- start 2024-05-08T18:08:00.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.552+0000 7ff47680e800 1 -- start start 2024-05-08T18:08:00.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.552+0000 7ff47680e800 1 --2- >> v2:172.21.15.46:3300/0 conn(0x564ec90d8550 0x564ec90d8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.552+0000 7ff47680e800 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x564ec90d8ff0 con 0x564ec90d97b0 2024-05-08T18:08:00.550 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.552+0000 7ff47680e800 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x564ec8f79e20 con 0x564ec90d8550 2024-05-08T18:08:00.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.552+0000 7ff474306640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x564ec90d8550 0x564ec90d8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.552+0000 7ff473b05640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x564ec90d97b0 0x564ec90d9c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.50:40246/0 (socket says 172.21.15.50:40246) 2024-05-08T18:08:00.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.552+0000 7ff473b05640 1 -- 172.21.15.50:0/3949158321 learned_addr learned my addr 172.21.15.50:0/3949158321 (peer_addr_for_me v1:172.21.15.50:0/0) 2024-05-08T18:08:00.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.553+0000 7ff474306640 1 -- 172.21.15.50:0/3949158321 >> v1:172.21.15.46:6789/0 conn(0x564ec90d97b0 legacy=0x564ec90d9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.551 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.553+0000 7ff474306640 1 -- 172.21.15.50:0/3949158321 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ec90da370 con 0x564ec90d8550 2024-05-08T18:08:00.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.555+0000 7ff474306640 1 --2- 172.21.15.50:0/3949158321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 0x564ec90d8970 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff46400a600 tx=0x7ff46401ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7accd566d06b2afb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.555+0000 7ff473304640 1 -- 172.21.15.50:0/3949158321 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff46400a200 con 0x564ec90d8550 2024-05-08T18:08:00.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.555+0000 7ff473304640 1 -- 172.21.15.50:0/3949158321 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff46400a3a0 con 0x564ec90d8550 2024-05-08T18:08:00.553 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.555+0000 7ff473304640 1 -- 172.21.15.50:0/3949158321 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff4640086e0 con 0x564ec90d8550 2024-05-08T18:08:00.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.555+0000 7ff47680e800 1 -- 172.21.15.50:0/3949158321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 msgr2=0x564ec90d8970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.555+0000 7ff47680e800 1 --2- 172.21.15.50:0/3949158321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 0x564ec90d8970 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff46400a600 tx=0x7ff46401ed70 comp rx=0 tx=0).stop 2024-05-08T18:08:00.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.555+0000 7ff47680e800 1 -- 172.21.15.50:0/3949158321 shutdown_connections 2024-05-08T18:08:00.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.555+0000 7ff47680e800 1 --2- 172.21.15.50:0/3949158321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 0x564ec90d8970 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:00.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.555+0000 7ff47680e800 1 -- 172.21.15.50:0/3949158321 >> 172.21.15.50:0/3949158321 conn(0x564ec90d3b50 msgr2=0x564ec90d5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:00.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.556+0000 7ff47680e800 1 -- 172.21.15.50:0/3949158321 shutdown_connections 2024-05-08T18:08:00.554 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.556+0000 7ff47680e800 1 -- 172.21.15.50:0/3949158321 wait complete. 2024-05-08T18:08:00.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.573+0000 7ff47680e800 1 Processor -- start 2024-05-08T18:08:00.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.573+0000 7ff47680e800 1 -- start start 2024-05-08T18:08:00.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.573+0000 7ff47680e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 0x564ec91cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.571 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.573+0000 7ff47680e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ec9024a10 con 0x564ec90d8550 2024-05-08T18:08:00.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.574+0000 7ff474306640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 0x564ec91cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.574+0000 7ff474306640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 0x564ec91cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57178/0 (socket says 172.21.15.50:57178) 2024-05-08T18:08:00.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.574+0000 7ff474306640 1 -- 172.21.15.50:0/438777345 learned_addr learned my addr 172.21.15.50:0/438777345 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:08:00.572 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.574+0000 7ff474306640 1 -- 172.21.15.50:0/438777345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ec91cd420 con 0x564ec90d8550 2024-05-08T18:08:00.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.574+0000 7ff474306640 1 --2- 172.21.15.50:0/438777345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 0x564ec91cce50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff464056040 tx=0x7ff46404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31ef393bd9a3bc5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.574+0000 7ff36260c640 1 -- 172.21.15.50:0/438777345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff46405d570 con 0x564ec90d8550 2024-05-08T18:08:00.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.574+0000 7ff36260c640 1 -- 172.21.15.50:0/438777345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff464062070 con 0x564ec90d8550 2024-05-08T18:08:00.573 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.574+0000 7ff36260c640 1 -- 172.21.15.50:0/438777345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff46405d880 con 0x564ec90d8550 2024-05-08T18:08:00.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.575+0000 7ff47680e800 1 -- 172.21.15.50:0/438777345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 msgr2=0x564ec91cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.575+0000 7ff47680e800 1 --2- 172.21.15.50:0/438777345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 0x564ec91cce50 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7ff464056040 tx=0x7ff46404e000 comp rx=0 tx=0).stop 2024-05-08T18:08:00.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.575+0000 7ff47680e800 1 -- 172.21.15.50:0/438777345 shutdown_connections 2024-05-08T18:08:00.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.575+0000 7ff47680e800 1 --2- 172.21.15.50:0/438777345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 0x564ec91cce50 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:00.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.575+0000 7ff47680e800 1 -- 172.21.15.50:0/438777345 >> 172.21.15.50:0/438777345 conn(0x564ec90d3b50 msgr2=0x564ec91c4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:00.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.575+0000 7ff47680e800 1 -- 172.21.15.50:0/438777345 shutdown_connections 2024-05-08T18:08:00.575 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.575+0000 7ff47680e800 1 -- 172.21.15.50:0/438777345 wait complete. 2024-05-08T18:08:00.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.576+0000 7ff47680e800 1 Processor -- start 2024-05-08T18:08:00.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.576+0000 7ff47680e800 1 -- start start 2024-05-08T18:08:00.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.576+0000 7ff47680e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 0x564ec91d4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.576+0000 7ff47680e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ec904bc30 con 0x564ec90d8550 2024-05-08T18:08:00.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.576+0000 7ff474306640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 0x564ec91d4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.576+0000 7ff474306640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 0x564ec91d4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57182/0 (socket says 172.21.15.50:57182) 2024-05-08T18:08:00.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.576+0000 7ff474306640 1 -- 172.21.15.50:0/2926117503 learned_addr learned my addr 172.21.15.50:0/2926117503 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:08:00.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.577+0000 7ff474306640 1 -- 172.21.15.50:0/2926117503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ec91d4e50 con 0x564ec90d8550 2024-05-08T18:08:00.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.577+0000 7ff474306640 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 0x564ec91d4910 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff46401da60 tx=0x7ff464002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.577+0000 7ff360e09640 1 -- 172.21.15.50:0/2926117503 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff46406f420 con 0x564ec90d8550 2024-05-08T18:08:00.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.577+0000 7ff360e09640 1 -- 172.21.15.50:0/2926117503 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff46406b070 con 0x564ec90d8550 2024-05-08T18:08:00.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.577+0000 7ff47680e800 1 -- 172.21.15.50:0/2926117503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564ec91d50e0 con 0x564ec90d8550 2024-05-08T18:08:00.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.577+0000 7ff360e09640 1 -- 172.21.15.50:0/2926117503 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff46406f750 con 0x564ec90d8550 2024-05-08T18:08:00.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.577+0000 7ff47680e800 1 -- 172.21.15.50:0/2926117503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ec91d7cf0 con 0x564ec90d8550 2024-05-08T18:08:00.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.578+0000 7ff360e09640 1 -- 172.21.15.50:0/2926117503 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff464067400 con 0x564ec90d8550 2024-05-08T18:08:00.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.578+0000 7ff360e09640 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff35c03bb50 0x7ff35c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.578+0000 7ff360e09640 1 -- 172.21.15.50:0/2926117503 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7ff4640948b0 con 0x564ec90d8550 2024-05-08T18:08:00.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.578+0000 7ff473b05640 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff35c03bb50 0x7ff35c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.578+0000 7ff47680e800 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ec90d97b0 0x564ec91dbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.578+0000 7ff47680e800 1 -- 172.21.15.50:0/2926117503 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec91dc030 con 0x564ec90d97b0 2024-05-08T18:08:00.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.579+0000 7ff474b07640 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ec90d97b0 0x564ec91dbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.579+0000 7ff473b05640 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff35c03bb50 0x7ff35c03e010 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff468001d70 tx=0x7ff4680012d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.579+0000 7ff474b07640 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ec90d97b0 0x564ec91dbab0 crc :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.579+0000 7ff474b07640 1 -- 172.21.15.50:0/2926117503 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff46c00a4a0 con 0x564ec90d97b0 2024-05-08T18:08:00.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.579+0000 7ff47680e800 1 -- 172.21.15.50:0/2926117503 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec91dc960 con 0x564ec90d97b0 2024-05-08T18:08:00.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.580+0000 7ff474b07640 1 -- 172.21.15.50:0/2926117503 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff46c00a4a0 con 0x564ec90d97b0 2024-05-08T18:08:00.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.580+0000 7ff47680e800 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ec91de270 0x564ec91e0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.580+0000 7ff47680e800 1 -- 172.21.15.50:0/2926117503 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:3 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec91e0ea0 con 0x564ec91de270 2024-05-08T18:08:00.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.580+0000 7ff474306640 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ec91de270 0x564ec91e0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.580+0000 7ff474306640 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ec91de270 0x564ec91e0710 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.581+0000 7ff474306640 1 -- 172.21.15.50:0/2926117503 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(3 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff464090040 con 0x564ec91de270 2024-05-08T18:08:00.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.581+0000 7ff47680e800 1 -- 172.21.15.50:0/2926117503 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec91dd1b0 con 0x564ec90d97b0 2024-05-08T18:08:00.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.581+0000 7ff474b07640 1 -- 172.21.15.50:0/2926117503 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff46c00a4a0 con 0x564ec90d97b0 2024-05-08T18:08:00.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.581+0000 7ff47680e800 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ec91e2300 0x564ec91e46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.581+0000 7ff47680e800 1 -- 172.21.15.50:0/2926117503 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:5 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec91e4d90 con 0x564ec91e2300 2024-05-08T18:08:00.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.581+0000 7ff473b05640 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ec91e2300 0x564ec91e46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.582+0000 7ff473b05640 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ec91e2300 0x564ec91e46f0 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.582+0000 7ff473b05640 1 -- 172.21.15.50:0/2926117503 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(5 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff468009050 con 0x564ec91e2300 2024-05-08T18:08:00.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.582+0000 7ff47680e800 1 Processor -- start 2024-05-08T18:08:00.582 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.583+0000 7ff47680e800 1 -- start start 2024-05-08T18:08:00.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.583+0000 7ff47680e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec91f30d0 0x564ec91f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.583+0000 7ff47680e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ec8fcb330 con 0x564ec91f30d0 2024-05-08T18:08:00.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.583+0000 7ff474306640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec91f30d0 0x564ec91f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.583+0000 7ff474306640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec91f30d0 0x564ec91f34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57184/0 (socket says 172.21.15.50:57184) 2024-05-08T18:08:00.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.583+0000 7ff474306640 1 -- 172.21.15.50:0/742178631 learned_addr learned my addr 172.21.15.50:0/742178631 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:08:00.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.583+0000 7ff474306640 1 -- 172.21.15.50:0/742178631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ec91f3a30 con 0x564ec91f30d0 2024-05-08T18:08:00.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.583+0000 7ff474306640 1 --2- 172.21.15.50:0/742178631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec91f30d0 0x564ec91f34f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff464053560 tx=0x7ff46405ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.584+0000 7ff359ffb640 1 -- 172.21.15.50:0/742178631 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff46407f040 con 0x564ec91f30d0 2024-05-08T18:08:00.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.584+0000 7ff359ffb640 1 -- 172.21.15.50:0/742178631 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4640073f0 con 0x564ec91f30d0 2024-05-08T18:08:00.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.584+0000 7ff47680e800 1 -- 172.21.15.50:0/742178631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564ec91f44a0 con 0x564ec91f30d0 2024-05-08T18:08:00.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.584+0000 7ff359ffb640 1 -- 172.21.15.50:0/742178631 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff46408d040 con 0x564ec91f30d0 2024-05-08T18:08:00.583 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.584+0000 7ff47680e800 1 -- 172.21.15.50:0/742178631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ec91f49a0 con 0x564ec91f30d0 2024-05-08T18:08:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.585+0000 7ff359ffb640 1 -- 172.21.15.50:0/742178631 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff464094bd0 con 0x564ec91f30d0 2024-05-08T18:08:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.585+0000 7ff359ffb640 1 --2- 172.21.15.50:0/742178631 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff34c03bb50 0x7ff34c03e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.585+0000 7ff359ffb640 1 -- 172.21.15.50:0/742178631 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7ff464099ac0 con 0x564ec91f30d0 2024-05-08T18:08:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.585+0000 7ff473b05640 1 --2- 172.21.15.50:0/742178631 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff34c03bb50 0x7ff34c03e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.585+0000 7ff47680e800 1 Processor -- start 2024-05-08T18:08:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.585+0000 7ff47680e800 1 -- start start 2024-05-08T18:08:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.585+0000 7ff473b05640 1 --2- 172.21.15.50:0/742178631 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff34c03bb50 0x7ff34c03e010 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff4680016b0 tx=0x7ff46800f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.586+0000 7ff47680e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec9200240 0x564ec9200660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.586+0000 7ff47680e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ec91cd6e0 con 0x564ec9200240 2024-05-08T18:08:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.586+0000 7ff474306640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec9200240 0x564ec9200660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.587 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.586+0000 7ff474306640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec9200240 0x564ec9200660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57194/0 (socket says 172.21.15.50:57194) 2024-05-08T18:08:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.586+0000 7ff474306640 1 -- 172.21.15.50:0/2200202588 learned_addr learned my addr 172.21.15.50:0/2200202588 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:08:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.586+0000 7ff474306640 1 -- 172.21.15.50:0/2200202588 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ec9200c30 con 0x564ec9200240 2024-05-08T18:08:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.586+0000 7ff474306640 1 --2- 172.21.15.50:0/2200202588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec9200240 0x564ec9200660 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff464086000 tx=0x7ff464053800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cb6b0362937a70 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.586+0000 7ff3537fe640 1 -- 172.21.15.50:0/2200202588 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff464054900 con 0x564ec9200240 2024-05-08T18:08:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.586+0000 7ff3537fe640 1 -- 172.21.15.50:0/2200202588 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4640a9070 con 0x564ec9200240 2024-05-08T18:08:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.586+0000 7ff3537fe640 1 -- 172.21.15.50:0/2200202588 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff464094050 con 0x564ec9200240 2024-05-08T18:08:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.587+0000 7ff47680e800 1 -- 172.21.15.50:0/2200202588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec9200240 msgr2=0x564ec9200660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.587+0000 7ff47680e800 1 --2- 172.21.15.50:0/2200202588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec9200240 0x564ec9200660 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff464086000 tx=0x7ff464053800 comp rx=0 tx=0).stop 2024-05-08T18:08:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.587+0000 7ff47680e800 1 -- 172.21.15.50:0/2200202588 shutdown_connections 2024-05-08T18:08:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.587+0000 7ff47680e800 1 --2- 172.21.15.50:0/2200202588 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec9200240 0x564ec9200660 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.587+0000 7ff47680e800 1 -- 172.21.15.50:0/2200202588 >> 172.21.15.50:0/2200202588 conn(0x564ec91fc550 msgr2=0x564ec91fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.587+0000 7ff47680e800 1 -- 172.21.15.50:0/2200202588 shutdown_connections 2024-05-08T18:08:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.587+0000 7ff47680e800 1 -- 172.21.15.50:0/2200202588 wait complete. 2024-05-08T18:08:00.588 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.587+0000 7ff47680e800 1 Processor -- start 2024-05-08T18:08:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.587+0000 7ff47680e800 1 -- start start 2024-05-08T18:08:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.588+0000 7ff47680e800 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec9200240 0x564ec9207650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.588+0000 7ff47680e800 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564ec92012c0 con 0x564ec9200240 2024-05-08T18:08:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.588+0000 7ff474306640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec9200240 0x564ec9207650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.588+0000 7ff474306640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec9200240 0x564ec9207650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.50:57206/0 (socket says 172.21.15.50:57206) 2024-05-08T18:08:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.588+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 learned_addr learned my addr 172.21.15.50:0/189896415 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-05-08T18:08:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.588+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ec9207b90 con 0x564ec9200240 2024-05-08T18:08:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.588+0000 7ff474306640 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec9200240 0x564ec9207650 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff4640539c0 tx=0x7ff464064f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.588+0000 7ff351ffb640 1 -- 172.21.15.50:0/189896415 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff46405e030 con 0x564ec9200240 2024-05-08T18:08:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.588+0000 7ff351ffb640 1 -- 172.21.15.50:0/189896415 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff46407d160 con 0x564ec9200240 2024-05-08T18:08:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.588+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564ec9207b90 con 0x564ec9200240 2024-05-08T18:08:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.589+0000 7ff351ffb640 1 -- 172.21.15.50:0/189896415 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff464066d00 con 0x564ec9200240 2024-05-08T18:08:00.589 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.589+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ec9207b90 con 0x564ec9200240 2024-05-08T18:08:00.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.589+0000 7ff351ffb640 1 -- 172.21.15.50:0/189896415 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff4640a2040 con 0x564ec9200240 2024-05-08T18:08:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.590+0000 7ff351ffb640 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff34403bb50 0x7ff34403e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.590+0000 7ff351ffb640 1 -- 172.21.15.50:0/189896415 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7ff46409f040 con 0x564ec9200240 2024-05-08T18:08:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.590+0000 7ff473b05640 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff34403bb50 0x7ff34403e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.590+0000 7ff47680e800 20 rados->read ofs=0 len=0 2024-05-08T18:08:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.590+0000 7ff47680e800 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ec9212b60 0x564ec9215020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.591+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9215770 con 0x564ec9212b60 2024-05-08T18:08:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.591+0000 7ff473b05640 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff34403bb50 0x7ff34403e010 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff46800efd0 tx=0x7ff468010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.591+0000 7ff474b07640 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ec9212b60 0x564ec9215020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.591+0000 7ff474b07640 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ec9212b60 0x564ec9215020 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.591+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff46c00a4a0 con 0x564ec9212b60 2024-05-08T18:08:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.591+0000 7ff47680e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:08:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.591+0000 7ff47680e800 20 realm 2024-05-08T18:08:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.591+0000 7ff47680e800 20 rados->read ofs=0 len=0 2024-05-08T18:08:00.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.591+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9215b80 con 0x564ec9212b60 2024-05-08T18:08:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.591+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff46c00a4a0 con 0x564ec9212b60 2024-05-08T18:08:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.592+0000 7ff47680e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:08:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.592+0000 7ff47680e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-05-08T18:08:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.592+0000 7ff47680e800 20 rados->read ofs=0 len=0 2024-05-08T18:08:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.592+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9216450 con 0x564ec9212b60 2024-05-08T18:08:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.592+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff46c00a4a0 con 0x564ec9212b60 2024-05-08T18:08:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.592+0000 7ff47680e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:08:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.592+0000 7ff47680e800 20 rados->read ofs=0 len=0 2024-05-08T18:08:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.592+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9212b60 2024-05-08T18:08:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.592+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff46c00a4a0 con 0x564ec9212b60 2024-05-08T18:08:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.592+0000 7ff47680e800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:08:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.592+0000 7ff47680e800 20 rados->read ofs=0 len=0 2024-05-08T18:08:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.592+0000 7ff47680e800 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ec9217550 0x564ec9219940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.592+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:5 5.c 5:37b712a3:::zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9219f00 con 0x564ec9217550 2024-05-08T18:08:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.592+0000 7ff474306640 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ec9217550 0x564ec9219940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.596 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.593+0000 7ff474306640 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ec9217550 0x564ec9219940 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.593+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== osd_op_reply(5 zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.593+0000 7ff47680e800 20 rados_obj.operate() r=0 bl.length=993 2024-05-08T18:08:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.593+0000 7ff47680e800 20 searching for the correct realm 2024-05-08T18:08:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.593+0000 7ff47680e800 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ec921beb0 0x564ec921e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.593+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec921ea40 con 0x564ec921beb0 2024-05-08T18:08:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.593+0000 7ff473b05640 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ec921beb0 0x564ec921e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.594+0000 7ff473b05640 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ec921beb0 0x564ec921e370 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.594+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== osd_op_reply(6 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.594+0000 7ff473b05640 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff468037040 0x7ff468036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.594+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec921ea40 con 0x7ff468037040 2024-05-08T18:08:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.594+0000 7ff474b07640 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff468037040 0x7ff468036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.594+0000 7ff474b07640 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff468037040 0x7ff468036410 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.595+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== osd_op_reply(7 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff46c00a4a0 con 0x7ff468037040 2024-05-08T18:08:00.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.595+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9217550 2024-05-08T18:08:00.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.595+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== osd_op_reply(8 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.595+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9212b60 2024-05-08T18:08:00.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.595+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 5 ==== osd_op_reply(9 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff46c00a4a0 con 0x564ec9212b60 2024-05-08T18:08:00.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.595+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9212b60 2024-05-08T18:08:00.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.596+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 6 ==== osd_op_reply(10 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff46c00a4a0 con 0x564ec9212b60 2024-05-08T18:08:00.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.596+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9212b60 2024-05-08T18:08:00.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.596+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 7 ==== osd_op_reply(11 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff46c00a4a0 con 0x564ec9212b60 2024-05-08T18:08:00.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.596+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9217550 2024-05-08T18:08:00.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.596+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 3 ==== osd_op_reply(12 [pgnls start_epoch 29 out=107b] v29'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.596+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x7ff468037040 2024-05-08T18:08:00.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.597+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== osd_op_reply(13 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff46c00a4a0 con 0x7ff468037040 2024-05-08T18:08:00.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.597+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec921beb0 2024-05-08T18:08:00.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.597+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== osd_op_reply(14 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.599 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.597+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9212b60 2024-05-08T18:08:00.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.597+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 8 ==== osd_op_reply(15 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff46c00a4a0 con 0x564ec9212b60 2024-05-08T18:08:00.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.597+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9217550 2024-05-08T18:08:00.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.597+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 4 ==== osd_op_reply(16 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.597+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9212b60 2024-05-08T18:08:00.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.598+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 9 ==== osd_op_reply(17 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff46c00a4a0 con 0x564ec9212b60 2024-05-08T18:08:00.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.598+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x7ff468037040 2024-05-08T18:08:00.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.598+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 3 ==== osd_op_reply(18 [pgnls start_epoch 29 out=79b] v20'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.606 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.598+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec921beb0 2024-05-08T18:08:00.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.598+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 3 ==== osd_op_reply(19 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.598+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec921beb0 2024-05-08T18:08:00.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.599+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 4 ==== osd_op_reply(20 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.599+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9217550 2024-05-08T18:08:00.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.599+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 5 ==== osd_op_reply(21 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.599+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec921beb0 2024-05-08T18:08:00.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.599+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 5 ==== osd_op_reply(22 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.599+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec921beb0 2024-05-08T18:08:00.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.599+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 6 ==== osd_op_reply(23 [pgnls start_epoch 29 out=112b] v20'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.599+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9212b60 2024-05-08T18:08:00.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.600+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 10 ==== osd_op_reply(24 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.600+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9212b60 2024-05-08T18:08:00.607 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.600+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 11 ==== osd_op_reply(25 [pgnls start_epoch 29 out=74b] v29'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.600+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9217550 2024-05-08T18:08:00.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.600+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 6 ==== osd_op_reply(26 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.600+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec921beb0 2024-05-08T18:08:00.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.601+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 7 ==== osd_op_reply(27 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.601+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x7ff468037040 2024-05-08T18:08:00.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.601+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 4 ==== osd_op_reply(28 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.601+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9212b60 2024-05-08T18:08:00.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.601+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 12 ==== osd_op_reply(29 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.601+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9217550 2024-05-08T18:08:00.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.602+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 7 ==== osd_op_reply(30 [pgnls start_epoch 29 out=82b] v29'1 uv1 ondisk = 1) ==== 144+0+82 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.602+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec921beb0 2024-05-08T18:08:00.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.602+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 8 ==== osd_op_reply(31 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.608 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.602+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9217550 2024-05-08T18:08:00.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.602+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 8 ==== osd_op_reply(32 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.602+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x7ff468037040 2024-05-08T18:08:00.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.602+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 5 ==== osd_op_reply(33 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.602+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec921beb0 2024-05-08T18:08:00.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.603+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 9 ==== osd_op_reply(34 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.603+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9217550 2024-05-08T18:08:00.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.603+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 9 ==== osd_op_reply(35 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.603+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x564ec9212b60 2024-05-08T18:08:00.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.603+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 13 ==== osd_op_reply(36 [pgnls start_epoch 29 out=115b] v29'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.603+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 29 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e29) -- 0x564ec92167f0 con 0x7ff468037040 2024-05-08T18:08:00.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 6 ==== osd_op_reply(37 [pgnls start_epoch 29 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff47680e800 20 RGWRados::pool_iterate: got zone_info.1c041ffd-e40a-486a-88e5-3547b85128ea 2024-05-08T18:08:00.609 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff47680e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-05-08T18:08:00.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff47680e800 20 RGWRados::pool_iterate: got zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb 2024-05-08T18:08:00.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff47680e800 20 RGWRados::pool_iterate: got default.zone. 2024-05-08T18:08:00.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff47680e800 20 RGWRados::pool_iterate: got period_config.default 2024-05-08T18:08:00.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff47680e800 20 RGWRados::pool_iterate: got zone_names.default 2024-05-08T18:08:00.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff47680e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-05-08T18:08:00.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff47680e800 20 rados->read ofs=0 len=0 2024-05-08T18:08:00.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec921a7c0 con 0x564ec9212b60 2024-05-08T18:08:00.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff47680e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:08:00.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff47680e800 20 rados->read ofs=0 len=0 2024-05-08T18:08:00.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec921a7c0 con 0x564ec9212b60 2024-05-08T18:08:00.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 15 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.610 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff47680e800 20 rados_obj.operate() r=0 bl.length=46 2024-05-08T18:08:00.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff47680e800 20 rados->read ofs=0 len=0 2024-05-08T18:08:00.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.604+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:40 5.11 5:8fd93cf7:::zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec921a7c0 con 0x564ec921beb0 2024-05-08T18:08:00.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.605+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 10 ==== osd_op_reply(40 zonegroup_info.36133b23-abaa-446a-bf67-ceb1d1559ceb [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.605+0000 7ff47680e800 20 rados_obj.operate() r=0 bl.length=436 2024-05-08T18:08:00.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.605+0000 7ff47680e800 20 zone default found 2024-05-08T18:08:00.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.605+0000 7ff47680e800 4 Realm: () 2024-05-08T18:08:00.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.605+0000 7ff47680e800 4 ZoneGroup: default (36133b23-abaa-446a-bf67-ceb1d1559ceb) 2024-05-08T18:08:00.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.605+0000 7ff47680e800 4 Zone: default (1c041ffd-e40a-486a-88e5-3547b85128ea) 2024-05-08T18:08:00.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.605+0000 7ff47680e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-05-08T18:08:00.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.605+0000 7ff47680e800 20 zonegroup default 2024-05-08T18:08:00.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.605+0000 7ff47680e800 20 rados->read ofs=0 len=0 2024-05-08T18:08:00.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.605+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec921b100 con 0x564ec9217550 2024-05-08T18:08:00.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.605+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 10 ==== osd_op_reply(41 period_config.default [read 0~187 out=187b] v0'0 uv1 ondisk = 0) ==== 165+0+187 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.605+0000 7ff47680e800 20 rados_obj.operate() r=0 bl.length=187 2024-05-08T18:08:00.611 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.605+0000 7ff47680e800 20 rados->read ofs=0 len=0 2024-05-08T18:08:00.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.605+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9220d50 con 0x564ec921beb0 2024-05-08T18:08:00.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.606+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.606+0000 7ff47680e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:08:00.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.606+0000 7ff47680e800 20 rados->read ofs=0 len=0 2024-05-08T18:08:00.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.606+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec92210f0 con 0x564ec921beb0 2024-05-08T18:08:00.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.606+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.606+0000 7ff47680e800 20 rados_obj.operate() r=-2 bl.length=0 2024-05-08T18:08:00.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.606+0000 7ff47680e800 20 started sync module instance, tier type = 2024-05-08T18:08:00.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.606+0000 7ff47680e800 20 started zone id=1c041ffd-e40a-486a-88e5-3547b85128ea (name=default) with tier type = 2024-05-08T18:08:00.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.606+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9220190 con 0x7ff468037040 2024-05-08T18:08:00.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.607+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 7 ==== osd_op_reply(44 notify.0 [create] v29'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.607+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94896381887872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9220190 con 0x7ff468037040 2024-05-08T18:08:00.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.607+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9221980 con 0x564ec9212b60 2024-05-08T18:08:00.612 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.608+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94896381887872] v29'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.609+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 16 ==== osd_op_reply(46 notify.1 [create] v29'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.609+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94896381891728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9220190 con 0x564ec9212b60 2024-05-08T18:08:00.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.609+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec92223c0 con 0x564ec921beb0 2024-05-08T18:08:00.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.610+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 17 ==== osd_op_reply(47 notify.1 [watch watch cookie 94896381891728] v29'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.610+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 13 ==== osd_op_reply(48 notify.2 [create] v29'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.610+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94896381885664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9220100 con 0x564ec921beb0 2024-05-08T18:08:00.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.610+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9223230 con 0x7ff468037040 2024-05-08T18:08:00.616 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.611+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94896381885664] v29'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.611+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 9 ==== osd_op_reply(50 notify.3 [create] v29'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.611+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94896381898112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9223650 con 0x7ff468037040 2024-05-08T18:08:00.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.611+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9223f00 con 0x564ec9212b60 2024-05-08T18:08:00.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.613+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94896381898112] v29'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.613+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 18 ==== osd_op_reply(52 notify.4 [create] v29'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.613+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94896381899168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9223a70 con 0x564ec9212b60 2024-05-08T18:08:00.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.613+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9224950 con 0x564ec9217550 2024-05-08T18:08:00.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.614+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94896381899168] v29'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.614+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 11 ==== osd_op_reply(54 notify.5 [create] v29'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.614+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94896381900224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec92239e0 con 0x564ec9217550 2024-05-08T18:08:00.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.614+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9225810 con 0x564ec9217550 2024-05-08T18:08:00.617 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.615+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 12 ==== osd_op_reply(56 notify.6 [create] v29'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.615+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94896381909936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9226480 con 0x564ec9217550 2024-05-08T18:08:00.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.615+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9226e40 con 0x564ec9217550 2024-05-08T18:08:00.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.615+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94896381900224] v29'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.616+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 14 ==== osd_op_reply(57 notify.6 [watch watch cookie 94896381909936] v29'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.616+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 15 ==== osd_op_reply(58 notify.7 [create] v29'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.617+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94896381906160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9226e40 con 0x564ec9217550 2024-05-08T18:08:00.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.617+0000 7ff47680e800 20 add_watcher() i=0 2024-05-08T18:08:00.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.617+0000 7ff47680e800 20 add_watcher() i=1 2024-05-08T18:08:00.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.617+0000 7ff47680e800 20 add_watcher() i=2 2024-05-08T18:08:00.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.617+0000 7ff47680e800 20 add_watcher() i=3 2024-05-08T18:08:00.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.617+0000 7ff47680e800 20 add_watcher() i=4 2024-05-08T18:08:00.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.617+0000 7ff47680e800 20 add_watcher() i=5 2024-05-08T18:08:00.618 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.617+0000 7ff47680e800 20 add_watcher() i=6 2024-05-08T18:08:00.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.618+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 16 ==== osd_op_reply(59 notify.7 [watch watch cookie 94896381906160] v29'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.618+0000 7ff47680e800 20 add_watcher() i=7 2024-05-08T18:08:00.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.618+0000 7ff47680e800 2 all 8 watchers are set, enabling cache 2024-05-08T18:08:00.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.618+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec922a010 con 0x564ec9212b60 2024-05-08T18:08:00.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.618+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 20 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.619 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.618+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94896381913856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec92273d0 con 0x564ec9212b60 2024-05-08T18:08:00.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.619+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 21 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94896381913856] v29'274 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.619+0000 7ff32a7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:08:00.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.619+0000 7ff47680e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-05-08T18:08:00.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.619+0000 7ff47680e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-05-08T18:08:00.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.619+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9229810 con 0x564ec9212b60 2024-05-08T18:08:00.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.621+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 22 ==== osd_op_reply(62 gc.0 [create,call,call,call] v29'38 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.620 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.621+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec92330d0 con 0x564ec9217550 2024-05-08T18:08:00.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.622+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 17 ==== osd_op_reply(63 gc.1 [create,call,call,call] v29'32 uv22 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.622+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec92330d0 con 0x564ec921beb0 2024-05-08T18:08:00.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.623+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v29'31 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.621 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.623+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9233920 con 0x564ec9212b60 2024-05-08T18:08:00.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.624+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 23 ==== osd_op_reply(65 gc.3 [create,call,call,call] v29'275 uv164 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.622 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.624+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9233d40 con 0x7ff468037040 2024-05-08T18:08:00.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.625+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v29'83 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.623 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.625+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec92342b0 con 0x564ec921beb0 2024-05-08T18:08:00.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.626+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v29'55 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.624 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.626+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9234760 con 0x564ec9212b60 2024-05-08T18:08:00.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.627+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 24 ==== osd_op_reply(68 gc.6 [create,call,call,call] v29'34 uv23 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.625 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.627+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9234620 con 0x564ec9217550 2024-05-08T18:08:00.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.628+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 18 ==== osd_op_reply(69 gc.7 [create,call,call,call] v29'55 uv36 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.626 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.628+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9234620 con 0x7ff468037040 2024-05-08T18:08:00.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.630+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v29'31 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.627 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.630+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec92346b0 con 0x564ec921beb0 2024-05-08T18:08:00.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.631+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v29'38 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.628 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.631+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9234b60 con 0x7ff468037040 2024-05-08T18:08:00.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.631+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v29'84 uv53 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.629 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.632+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9234b60 con 0x564ec9217550 2024-05-08T18:08:00.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.633+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 19 ==== osd_op_reply(73 gc.11 [create,call,call,call] v29'42 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.631 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.633+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9234b60 con 0x7ff468037040 2024-05-08T18:08:00.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.634+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v29'32 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.632 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.634+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9234ad0 con 0x564ec9217550 2024-05-08T18:08:00.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.635+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 20 ==== osd_op_reply(75 gc.13 [create,call,call,call] v29'62 uv42 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.633 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.635+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9234aa0 con 0x564ec9212b60 2024-05-08T18:08:00.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.636+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 25 ==== osd_op_reply(76 gc.14 [create,call,call,call] v29'276 uv166 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.634 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.636+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec92348d0 con 0x564ec921beb0 2024-05-08T18:08:00.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.638+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v29'78 uv50 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.635 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.638+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9234960 con 0x7ff468037040 2024-05-08T18:08:00.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.638+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v29'85 uv55 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.636 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.638+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9234960 con 0x564ec921beb0 2024-05-08T18:08:00.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.639+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v29'32 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.637 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.639+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9235000 con 0x564ec9212b60 2024-05-08T18:08:00.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.641+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 26 ==== osd_op_reply(80 gc.18 [create,call,call,call] v29'39 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.639 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.641+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9235000 con 0x564ec9212b60 2024-05-08T18:08:00.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.642+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 27 ==== osd_op_reply(81 gc.19 [create,call,call,call] v29'277 uv162 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.640 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.642+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9235000 con 0x564ec921beb0 2024-05-08T18:08:00.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.643+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v29'79 uv46 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.641 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.643+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9235000 con 0x7ff468037040 2024-05-08T18:08:00.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.644+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v29'41 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.642 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.644+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9235000 con 0x564ec921beb0 2024-05-08T18:08:00.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.645+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v29'33 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.643 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.645+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9235000 con 0x564ec921beb0 2024-05-08T18:08:00.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.646+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v29'37 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.646+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9235000 con 0x564ec921beb0 2024-05-08T18:08:00.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.647+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v29'29 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.647+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9235000 con 0x564ec921beb0 2024-05-08T18:08:00.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.648+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v29'56 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.648+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9235000 con 0x564ec9217550 2024-05-08T18:08:00.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.650+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 21 ==== osd_op_reply(88 gc.26 [create,call,call,call] v29'33 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.650+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9235000 con 0x564ec921beb0 2024-05-08T18:08:00.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.651+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v29'33 uv21 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.649 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.651+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9235000 con 0x564ec9217550 2024-05-08T18:08:00.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.652+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 22 ==== osd_op_reply(90 gc.28 [create,call,call,call] v29'56 uv34 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.652+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9235000 con 0x564ec921beb0 2024-05-08T18:08:00.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.653+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v29'80 uv48 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.651 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.653+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9235000 con 0x564ec9217550 2024-05-08T18:08:00.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.654+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 23 ==== osd_op_reply(92 gc.30 [create,call,call,call] v29'63 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.652 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.654+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9235000 con 0x7ff468037040 2024-05-08T18:08:00.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.655+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v29'37 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.656+0000 7ff3297e2640 20 reqs_thread_entry: start 2024-05-08T18:08:00.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.656+0000 7ff47680e800 20 init_complete bucket index max shards: 11 2024-05-08T18:08:00.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.656+0000 7ff47680e800 20 Filter name: none 2024-05-08T18:08:00.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.656+0000 7ff29affd640 20 reqs_thread_entry: start 2024-05-08T18:08:00.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.656+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-05-08T18:08:00.654 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.656+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:94 6.7 6:e07a6548:gc::gc.15:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec92615a0 con 0x564ec921beb0 2024-05-08T18:08:00.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.657+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 27 ==== osd_op_reply(94 gc.15 [call] v29'81 uv81 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.657+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:95 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec92615a0 con 0x564ec921beb0 2024-05-08T18:08:00.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.657+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 28 ==== osd_op_reply(95 gc.15 [call out=15b] v0'0 uv81 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.657+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.657+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:96 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9260590 con 0x564ec921beb0 2024-05-08T18:08:00.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.658+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 29 ==== osd_op_reply(96 gc.15 [call out=24b] v0'0 uv81 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.658+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:97 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec92615a0 con 0x564ec921beb0 2024-05-08T18:08:00.656 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.658+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 30 ==== osd_op_reply(97 gc.15 [call out=15b] v0'0 uv81 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.658+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.658+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:98 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec92615a0 con 0x564ec921beb0 2024-05-08T18:08:00.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.658+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 31 ==== osd_op_reply(98 gc.15 [call out=15b] v0'0 uv81 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.658+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.658+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:99 6.7 6:e07a6548:gc::gc.15:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec92615a0 con 0x564ec921beb0 2024-05-08T18:08:00.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.659+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 32 ==== osd_op_reply(99 gc.15 [call] v29'82 uv82 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.657 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.659+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-05-08T18:08:00.658 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.659+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:100 6.3 6:c623b173:gc::gc.16:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec92615a0 con 0x7ff468037040 2024-05-08T18:08:00.659 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.660+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 18 ==== osd_op_reply(100 gc.16 [call] v29'86 uv86 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.660+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:101 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec92615a0 con 0x7ff468037040 2024-05-08T18:08:00.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.661+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 19 ==== osd_op_reply(101 gc.16 [call out=15b] v0'0 uv86 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.661+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.661+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:102 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9260590 con 0x7ff468037040 2024-05-08T18:08:00.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.661+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 20 ==== osd_op_reply(102 gc.16 [call out=24b] v0'0 uv86 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.661+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:103 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec92615a0 con 0x7ff468037040 2024-05-08T18:08:00.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.661+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 21 ==== osd_op_reply(103 gc.16 [call out=15b] v0'0 uv86 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.661+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.661+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:104 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec92615a0 con 0x7ff468037040 2024-05-08T18:08:00.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.661+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 22 ==== osd_op_reply(104 gc.16 [call out=15b] v0'0 uv86 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.661+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.661+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:105 6.3 6:c623b173:gc::gc.16:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec92615a0 con 0x7ff468037040 2024-05-08T18:08:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.662+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 23 ==== osd_op_reply(105 gc.16 [call] v29'87 uv87 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.662+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-05-08T18:08:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.662+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:106 6.16 6:6849fd06:gc::gc.17:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec92615a0 con 0x564ec921beb0 2024-05-08T18:08:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.663+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 33 ==== osd_op_reply(106 gc.17 [call] v29'33 uv33 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.663+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:107 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec92615a0 con 0x564ec921beb0 2024-05-08T18:08:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 34 ==== osd_op_reply(107 gc.17 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:108 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9260590 con 0x564ec921beb0 2024-05-08T18:08:00.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 35 ==== osd_op_reply(108 gc.17 [call out=24b] v0'0 uv33 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:109 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec92615a0 con 0x564ec921beb0 2024-05-08T18:08:00.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 36 ==== osd_op_reply(109 gc.17 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:110 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec92615a0 con 0x564ec921beb0 2024-05-08T18:08:00.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 37 ==== osd_op_reply(110 gc.17 [call out=1716b] v0'0 uv33 ondisk = 0) ==== 149+0+1716 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-05-08T18:08:00.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff47680e800 20 garbage collection: RGWGC::process iterating over entry tag='1c041ffd-e40a-486a-88e5-3547b85128ea.4221.17957606207564689355', time=2024-05-08T18:08:08.738997+0000, chain.objs.size()=7 2024-05-08T18:08:00.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff47680e800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_1 2024-05-08T18:08:00.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:111 3.2bs0 3:d723cf5b:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_1:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec925f880 con 0x564ec9212b60 2024-05-08T18:08:00.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff47680e800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_2 2024-05-08T18:08:00.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:112 3.31s0 3:8da563c7:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_2:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9264700 con 0x564ec9217550 2024-05-08T18:08:00.664 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff47680e800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_3 2024-05-08T18:08:00.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:113 3.22s0 3:44ad95d4:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_3:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9267270 con 0x564ec921beb0 2024-05-08T18:08:00.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff47680e800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_4 2024-05-08T18:08:00.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:114 3.8s0 3:107e5f4d:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_4:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec925f880 con 0x564ec921beb0 2024-05-08T18:08:00.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.664+0000 7ff47680e800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_5 2024-05-08T18:08:00.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.665+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:115 3.35s0 3:acf44c85:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_5:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9267270 con 0x564ec9212b60 2024-05-08T18:08:00.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.665+0000 7ff47680e800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_6 2024-05-08T18:08:00.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.665+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:116 3.23s0 3:c4c1a3ce:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_6:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9267270 con 0x564ec9217550 2024-05-08T18:08:00.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.665+0000 7ff47680e800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_7 2024-05-08T18:08:00.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.665+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:117 3.ds0 3:b262ba2f:::1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_7:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9270b70 con 0x564ec921beb0 2024-05-08T18:08:00.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.667+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 38 ==== osd_op_reply(113 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_3 [call] v29'6 uv6 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.666 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.668+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 28 ==== osd_op_reply(111 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_1 [call] v29'4 uv4 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.669+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 39 ==== osd_op_reply(117 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_7 [call] v29'4 uv4 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.669+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 40 ==== osd_op_reply(114 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_4 [call] v29'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.669+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 24 ==== osd_op_reply(112 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_2 [call] v29'6 uv6 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.667 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.669+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 25 ==== osd_op_reply(116 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_6 [call] v29'6 uv6 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.670+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 29 ==== osd_op_reply(115 1c041ffd-e40a-486a-88e5-3547b85128ea.4221.11__shadow_.FrNn1MVFSc4c6lk7RKB6T7959c7Iumf_5 [call] v29'4 uv4 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.670+0000 7ff47680e800 5 garbage collection: RGWGC::process removing entries, marker: 2024-05-08T18:08:00.668 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.670+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:118 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_remove_entries in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266b80 con 0x564ec921beb0 2024-05-08T18:08:00.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.672+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 41 ==== osd_op_reply(118 gc.17 [call] v29'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.670 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.672+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:119 6.16 6:6849fd06:gc::gc.17:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266b30 con 0x564ec921beb0 2024-05-08T18:08:00.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.673+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 42 ==== osd_op_reply(119 gc.17 [call] v29'35 uv35 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.673+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-05-08T18:08:00.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.673+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:120 6.d 6:b019ea83:gc::gc.18:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266b30 con 0x564ec9212b60 2024-05-08T18:08:00.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.674+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 30 ==== osd_op_reply(120 gc.18 [call] v29'40 uv40 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.674+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:121 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266b30 con 0x564ec9212b60 2024-05-08T18:08:00.672 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.675+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 31 ==== osd_op_reply(121 gc.18 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.675+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.675+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:122 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265b20 con 0x564ec9212b60 2024-05-08T18:08:00.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.675+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 32 ==== osd_op_reply(122 gc.18 [call out=24b] v0'0 uv40 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.675+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:123 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266b30 con 0x564ec9212b60 2024-05-08T18:08:00.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.675+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 33 ==== osd_op_reply(123 gc.18 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.675+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.675+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:124 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266b30 con 0x564ec9212b60 2024-05-08T18:08:00.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.676+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 34 ==== osd_op_reply(124 gc.18 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.676+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.676+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:125 6.d 6:b019ea83:gc::gc.18:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266b30 con 0x564ec9212b60 2024-05-08T18:08:00.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.677+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 35 ==== osd_op_reply(125 gc.18 [call] v29'41 uv41 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.677+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-05-08T18:08:00.675 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.677+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:126 6.4 6:24c8e1ae:gc::gc.19:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266b30 con 0x564ec9212b60 2024-05-08T18:08:00.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.678+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 36 ==== osd_op_reply(126 gc.19 [call] v29'278 uv278 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.678+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:127 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266ae0 con 0x564ec9212b60 2024-05-08T18:08:00.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.678+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 37 ==== osd_op_reply(127 gc.19 [call out=15b] v0'0 uv278 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.678+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.676 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.678+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:128 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265ab0 con 0x564ec9212b60 2024-05-08T18:08:00.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.678+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 38 ==== osd_op_reply(128 gc.19 [call out=24b] v0'0 uv278 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.678+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:129 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.679+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 39 ==== osd_op_reply(129 gc.19 [call out=15b] v0'0 uv278 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.679+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.679+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:130 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.679+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 40 ==== osd_op_reply(130 gc.19 [call out=15b] v0'0 uv278 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.679+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.679+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:131 6.4 6:24c8e1ae:gc::gc.19:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.680+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 41 ==== osd_op_reply(131 gc.19 [call] v29'279 uv279 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.680+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-05-08T18:08:00.678 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.680+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:132 6.7 6:e622f76a:gc::gc.20:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.681+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 43 ==== osd_op_reply(132 gc.20 [call] v29'83 uv83 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.681+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:133 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.682+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 44 ==== osd_op_reply(133 gc.20 [call out=15b] v0'0 uv83 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.682+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.682+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:134 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec921beb0 2024-05-08T18:08:00.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.682+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 45 ==== osd_op_reply(134 gc.20 [call out=24b] v0'0 uv83 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.682+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:135 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.682+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 46 ==== osd_op_reply(135 gc.20 [call out=15b] v0'0 uv83 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.682+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.682+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:136 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.682+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 47 ==== osd_op_reply(136 gc.20 [call out=15b] v0'0 uv83 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.682+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.681 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.682+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:137 6.7 6:e622f76a:gc::gc.20:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.684+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 48 ==== osd_op_reply(137 gc.20 [call] v29'84 uv84 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.684+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-05-08T18:08:00.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.684+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:138 6.a 6:5127e642:gc::gc.21:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.685+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 24 ==== osd_op_reply(138 gc.21 [call] v29'42 uv42 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.682 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.685+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:139 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.685+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 25 ==== osd_op_reply(139 gc.21 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.685+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.685+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:140 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x7ff468037040 2024-05-08T18:08:00.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.685+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 26 ==== osd_op_reply(140 gc.21 [call out=24b] v0'0 uv42 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.685+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:141 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.685+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 27 ==== osd_op_reply(141 gc.21 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.685+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.685+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:142 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.686+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 28 ==== osd_op_reply(142 gc.21 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.686+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.686+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:143 6.a 6:5127e642:gc::gc.21:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.686+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 29 ==== osd_op_reply(143 gc.21 [call] v29'43 uv43 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.686+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-05-08T18:08:00.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.686+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:144 6.11 6:8eb99c53:gc::gc.22:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.687+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 49 ==== osd_op_reply(144 gc.22 [call] v29'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.687+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:145 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.688+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 50 ==== osd_op_reply(145 gc.22 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.688+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.688+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:146 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec921beb0 2024-05-08T18:08:00.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.688+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 51 ==== osd_op_reply(146 gc.22 [call out=24b] v0'0 uv34 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.688+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:147 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.688+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 52 ==== osd_op_reply(147 gc.22 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.688+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.688+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:148 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.688+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 53 ==== osd_op_reply(148 gc.22 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.688+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.688 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.688+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:149 6.11 6:8eb99c53:gc::gc.22:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.689+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 54 ==== osd_op_reply(149 gc.22 [call] v29'35 uv35 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.689+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-05-08T18:08:00.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.689+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:150 6.8 6:17d7e8ed:gc::gc.23:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.690+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 55 ==== osd_op_reply(150 gc.23 [call] v29'38 uv38 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.690+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:151 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.691+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 56 ==== osd_op_reply(151 gc.23 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.691+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.691+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:152 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec921beb0 2024-05-08T18:08:00.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.691+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 57 ==== osd_op_reply(152 gc.23 [call out=24b] v0'0 uv38 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.691+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:153 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.691+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 58 ==== osd_op_reply(153 gc.23 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.691+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.691+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:154 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.690 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.691+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 59 ==== osd_op_reply(154 gc.23 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.691+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.691 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.691+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:155 6.8 6:17d7e8ed:gc::gc.23:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.692+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 60 ==== osd_op_reply(155 gc.23 [call] v29'39 uv39 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.692+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-05-08T18:08:00.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.692+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:156 6.1b 6:db9d130d:gc::gc.24:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.693+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 61 ==== osd_op_reply(156 gc.24 [call] v29'30 uv30 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.693+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:157 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.693+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 62 ==== osd_op_reply(157 gc.24 [call out=15b] v0'0 uv30 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.693+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.693+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:158 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec921beb0 2024-05-08T18:08:00.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.694+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 63 ==== osd_op_reply(158 gc.24 [call out=24b] v0'0 uv30 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.694+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:159 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.694+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 64 ==== osd_op_reply(159 gc.24 [call out=15b] v0'0 uv30 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.694+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.694+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:160 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.693 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.694+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 65 ==== osd_op_reply(160 gc.24 [call out=15b] v0'0 uv30 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.694+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.694+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:161 6.1b 6:db9d130d:gc::gc.24:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.695+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 66 ==== osd_op_reply(161 gc.24 [call] v29'31 uv31 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.695+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-05-08T18:08:00.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.695+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:162 6.9 6:921ecdb1:gc::gc.25:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.696+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 67 ==== osd_op_reply(162 gc.25 [call] v29'57 uv57 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.696+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:163 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.696+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 68 ==== osd_op_reply(163 gc.25 [call out=15b] v0'0 uv57 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.696+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.696+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:164 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec921beb0 2024-05-08T18:08:00.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.697+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 69 ==== osd_op_reply(164 gc.25 [call out=24b] v0'0 uv57 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.697+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:165 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.697+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 70 ==== osd_op_reply(165 gc.25 [call out=15b] v0'0 uv57 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.697+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.697+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:166 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.697+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 71 ==== osd_op_reply(166 gc.25 [call out=15b] v0'0 uv57 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.697+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.697+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:167 6.9 6:921ecdb1:gc::gc.25:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.698+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 72 ==== osd_op_reply(167 gc.25 [call] v29'58 uv58 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.698+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-05-08T18:08:00.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.698+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:168 6.1f 6:fde4dd55:gc::gc.26:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.699+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 26 ==== osd_op_reply(168 gc.26 [call] v29'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.699+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:169 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.700+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 27 ==== osd_op_reply(169 gc.26 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.700+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.700+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:170 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.700+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 28 ==== osd_op_reply(170 gc.26 [call out=24b] v0'0 uv34 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.700+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:171 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.700+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 29 ==== osd_op_reply(171 gc.26 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.699 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.700+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.700+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:172 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.700+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 30 ==== osd_op_reply(172 gc.26 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.700+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.700+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:173 6.1f 6:fde4dd55:gc::gc.26:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.701+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 31 ==== osd_op_reply(173 gc.26 [call] v29'35 uv35 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.701+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-05-08T18:08:00.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.701+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:174 6.1a 6:5909e671:gc::gc.27:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.702+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 73 ==== osd_op_reply(174 gc.27 [call] v29'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.701 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.702+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:175 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.703+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 74 ==== osd_op_reply(175 gc.27 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.703+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.703+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:176 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec921beb0 2024-05-08T18:08:00.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.703+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 75 ==== osd_op_reply(176 gc.27 [call out=24b] v0'0 uv34 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.703+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:177 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.703+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 76 ==== osd_op_reply(177 gc.27 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.703+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.703+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:178 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.703+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 77 ==== osd_op_reply(178 gc.27 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.703+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.702 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.703+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:179 6.1a 6:5909e671:gc::gc.27:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.704+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 78 ==== osd_op_reply(179 gc.27 [call] v29'35 uv35 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.704+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-05-08T18:08:00.703 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.704+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:180 6.1d 6:b88e3271:gc::gc.28:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.706+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 32 ==== osd_op_reply(180 gc.28 [call] v29'57 uv57 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.704 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.706+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:181 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.706+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 33 ==== osd_op_reply(181 gc.28 [call out=15b] v0'0 uv57 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.706+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.706+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:182 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.706+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 34 ==== osd_op_reply(182 gc.28 [call out=24b] v0'0 uv57 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.706+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:183 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.707+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 35 ==== osd_op_reply(183 gc.28 [call out=15b] v0'0 uv57 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.707+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.707+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:184 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.705 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.707+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 36 ==== osd_op_reply(184 gc.28 [call out=15b] v0'0 uv57 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.707+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.707+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:185 6.1d 6:b88e3271:gc::gc.28:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.708+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 37 ==== osd_op_reply(185 gc.28 [call] v29'58 uv58 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.708+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-05-08T18:08:00.706 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.708+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:186 6.7 6:e47c9322:gc::gc.29:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.709+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 79 ==== osd_op_reply(186 gc.29 [call] v29'85 uv85 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.709+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:187 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.709+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 80 ==== osd_op_reply(187 gc.29 [call out=15b] v0'0 uv85 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.709+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.709+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:188 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec921beb0 2024-05-08T18:08:00.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.710+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 81 ==== osd_op_reply(188 gc.29 [call out=24b] v0'0 uv85 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.710+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:189 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.708 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.710+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 82 ==== osd_op_reply(189 gc.29 [call out=15b] v0'0 uv85 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.710+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.710+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:190 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.710+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 83 ==== osd_op_reply(190 gc.29 [call out=15b] v0'0 uv85 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.710+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.710+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:191 6.7 6:e47c9322:gc::gc.29:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.711+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 84 ==== osd_op_reply(191 gc.29 [call] v29'86 uv86 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.711+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-05-08T18:08:00.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.711+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:192 6.0 6:013d6c33:gc::gc.30:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.712+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 38 ==== osd_op_reply(192 gc.30 [call] v29'64 uv64 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.712+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:193 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.713+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 39 ==== osd_op_reply(193 gc.30 [call out=15b] v0'0 uv64 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.713+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.713+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:194 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.713+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 40 ==== osd_op_reply(194 gc.30 [call out=24b] v0'0 uv64 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.713+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:195 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.713+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 41 ==== osd_op_reply(195 gc.30 [call out=15b] v0'0 uv64 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.713+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.713+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:196 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.713+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 42 ==== osd_op_reply(196 gc.30 [call out=15b] v0'0 uv64 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.714+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.714+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:197 6.0 6:013d6c33:gc::gc.30:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.714+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 43 ==== osd_op_reply(197 gc.30 [call] v29'65 uv65 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.714+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-05-08T18:08:00.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.714+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:198 6.f 6:f24cb4a8:gc::gc.31:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.716+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 30 ==== osd_op_reply(198 gc.31 [call] v29'38 uv38 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.716+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:199 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.716+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 31 ==== osd_op_reply(199 gc.31 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.716+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.716+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:200 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x7ff468037040 2024-05-08T18:08:00.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.716+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 32 ==== osd_op_reply(200 gc.31 [call out=24b] v0'0 uv38 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.716+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:201 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.717+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 33 ==== osd_op_reply(201 gc.31 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.715 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.717+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.717+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:202 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.717+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 34 ==== osd_op_reply(202 gc.31 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.717+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.717+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:203 6.f 6:f24cb4a8:gc::gc.31:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.718+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 35 ==== osd_op_reply(203 gc.31 [call] v29'39 uv39 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.718+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-05-08T18:08:00.717 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.718+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:204 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.719+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 42 ==== osd_op_reply(204 gc.0 [call] v29'39 uv39 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.719+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:205 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.720+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 43 ==== osd_op_reply(205 gc.0 [call out=15b] v0'0 uv39 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.718 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.720+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.720+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:206 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9212b60 2024-05-08T18:08:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.720+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 44 ==== osd_op_reply(206 gc.0 [call out=24b] v0'0 uv39 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.720+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:207 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.720+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 45 ==== osd_op_reply(207 gc.0 [call out=15b] v0'0 uv39 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.720+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.720+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:208 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.720+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 46 ==== osd_op_reply(208 gc.0 [call out=15b] v0'0 uv39 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.720+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.719 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.720+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:209 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.721+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 47 ==== osd_op_reply(209 gc.0 [call] v29'40 uv40 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.721+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-05-08T18:08:00.720 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.721+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:210 6.18 6:1e85421f:gc::gc.1:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.721 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.722+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 44 ==== osd_op_reply(210 gc.1 [call] v29'33 uv33 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.722+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:211 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.723+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 45 ==== osd_op_reply(211 gc.1 [call out=15b] v0'0 uv33 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.723+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.723+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:212 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.723+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 46 ==== osd_op_reply(212 gc.1 [call out=24b] v0'0 uv33 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.723+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:213 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.723+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 47 ==== osd_op_reply(213 gc.1 [call out=15b] v0'0 uv33 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.723+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.723+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:214 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.723+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 48 ==== osd_op_reply(214 gc.1 [call out=15b] v0'0 uv33 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.723+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.722 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.723+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:215 6.18 6:1e85421f:gc::gc.1:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.725+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 49 ==== osd_op_reply(215 gc.1 [call] v29'34 uv34 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.725+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-05-08T18:08:00.723 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.725+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:216 6.b 6:d242335b:gc::gc.2:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.726+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 85 ==== osd_op_reply(216 gc.2 [call] v29'32 uv32 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.726+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:217 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.726+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 86 ==== osd_op_reply(217 gc.2 [call out=15b] v0'0 uv32 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.726+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.726+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:218 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec921beb0 2024-05-08T18:08:00.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.726+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 87 ==== osd_op_reply(218 gc.2 [call out=24b] v0'0 uv32 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.726+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:219 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.726+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 88 ==== osd_op_reply(219 gc.2 [call out=15b] v0'0 uv32 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.726+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.727+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:220 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.725 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.727+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 89 ==== osd_op_reply(220 gc.2 [call out=15b] v0'0 uv32 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.727+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.726 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.727+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:221 6.b 6:d242335b:gc::gc.2:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.728+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 90 ==== osd_op_reply(221 gc.2 [call] v29'33 uv33 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.728+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-05-08T18:08:00.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.728+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:222 6.4 6:2166d8e1:gc::gc.3:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.729+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 48 ==== osd_op_reply(222 gc.3 [call] v29'280 uv280 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.729+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:223 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.729+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 49 ==== osd_op_reply(223 gc.3 [call out=15b] v0'0 uv280 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.729+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.727 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.729+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:224 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9212b60 2024-05-08T18:08:00.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.729+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 50 ==== osd_op_reply(224 gc.3 [call out=24b] v0'0 uv280 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.729+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:225 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.730+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 51 ==== osd_op_reply(225 gc.3 [call out=15b] v0'0 uv280 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.730+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.730+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:226 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.730+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 52 ==== osd_op_reply(226 gc.3 [call out=15b] v0'0 uv280 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.728 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.730+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.730+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:227 6.4 6:2166d8e1:gc::gc.3:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.731+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 53 ==== osd_op_reply(227 gc.3 [call] v29'281 uv281 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.731+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-05-08T18:08:00.729 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.731+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:228 6.3 6:c60da066:gc::gc.4:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.732+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 36 ==== osd_op_reply(228 gc.4 [call] v29'88 uv88 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.732+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:229 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.732+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 37 ==== osd_op_reply(229 gc.4 [call out=15b] v0'0 uv88 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.732+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.732+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:230 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x7ff468037040 2024-05-08T18:08:00.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.732+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 38 ==== osd_op_reply(230 gc.4 [call out=24b] v0'0 uv88 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.732+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:231 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.733+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 39 ==== osd_op_reply(231 gc.4 [call out=15b] v0'0 uv88 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.731 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.733+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.733+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:232 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.733+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 40 ==== osd_op_reply(232 gc.4 [call out=15b] v0'0 uv88 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.733+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.733+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:233 6.3 6:c60da066:gc::gc.4:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.734+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 41 ==== osd_op_reply(233 gc.4 [call] v29'89 uv89 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.732 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.734+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-05-08T18:08:00.733 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.734+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:234 6.9 6:954be8d2:gc::gc.5:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.735+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 91 ==== osd_op_reply(234 gc.5 [call] v29'59 uv59 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.735+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:235 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.735+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 92 ==== osd_op_reply(235 gc.5 [call out=15b] v0'0 uv59 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.735+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.734 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.735+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:236 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec921beb0 2024-05-08T18:08:00.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.736+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 93 ==== osd_op_reply(236 gc.5 [call out=24b] v0'0 uv59 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.736+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:237 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.736+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 94 ==== osd_op_reply(237 gc.5 [call out=15b] v0'0 uv59 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.736+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.736+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:238 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.736+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 95 ==== osd_op_reply(238 gc.5 [call out=15b] v0'0 uv59 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.736+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.736+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:239 6.9 6:954be8d2:gc::gc.5:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.737+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 96 ==== osd_op_reply(239 gc.5 [call] v29'60 uv60 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.737+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-05-08T18:08:00.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.737+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:240 6.e 6:748722ff:gc::gc.6:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.738+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 54 ==== osd_op_reply(240 gc.6 [call] v29'35 uv35 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.739+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:241 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.739+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 55 ==== osd_op_reply(241 gc.6 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.739+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.739+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:242 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9212b60 2024-05-08T18:08:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.739+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 56 ==== osd_op_reply(242 gc.6 [call out=24b] v0'0 uv35 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.739+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:243 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.739+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 57 ==== osd_op_reply(243 gc.6 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.739+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.739+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:244 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.740+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 58 ==== osd_op_reply(244 gc.6 [call out=15b] v0'0 uv35 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.740+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.740+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:245 6.e 6:748722ff:gc::gc.6:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.741+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 59 ==== osd_op_reply(245 gc.6 [call] v29'36 uv36 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.741+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-05-08T18:08:00.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.741+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:246 6.1d 6:beaa4be4:gc::gc.7:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.742+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 50 ==== osd_op_reply(246 gc.7 [call] v29'59 uv59 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.740 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.742+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:247 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.742+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 51 ==== osd_op_reply(247 gc.7 [call out=15b] v0'0 uv59 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.742+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.742+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:248 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.743+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 52 ==== osd_op_reply(248 gc.7 [call out=24b] v0'0 uv59 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.741 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.743+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:249 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.743+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 53 ==== osd_op_reply(249 gc.7 [call out=15b] v0'0 uv59 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.743+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.743+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:250 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.743+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 54 ==== osd_op_reply(250 gc.7 [call out=15b] v0'0 uv59 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.743+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.742 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.743+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:251 6.1d 6:beaa4be4:gc::gc.7:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.744+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 55 ==== osd_op_reply(251 gc.7 [call] v29'60 uv60 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.745+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-05-08T18:08:00.743 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.745+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:252 6.12 6:4f78e9a2:gc::gc.8:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.746+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 42 ==== osd_op_reply(252 gc.8 [call] v29'32 uv32 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.746+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:253 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.746+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 43 ==== osd_op_reply(253 gc.8 [call out=15b] v0'0 uv32 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.746+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.746+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:254 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x7ff468037040 2024-05-08T18:08:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.746+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 44 ==== osd_op_reply(254 gc.8 [call out=24b] v0'0 uv32 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.746+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:255 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.746+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 45 ==== osd_op_reply(255 gc.8 [call out=15b] v0'0 uv32 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.746+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.746+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:256 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.745 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.747+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 46 ==== osd_op_reply(256 gc.8 [call out=15b] v0'0 uv32 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.747+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.747+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:257 6.12 6:4f78e9a2:gc::gc.8:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.748+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 47 ==== osd_op_reply(257 gc.8 [call] v29'33 uv33 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.746 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.748+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-05-08T18:08:00.747 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.748+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:258 6.1c 6:3e426232:gc::gc.9:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.749+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 97 ==== osd_op_reply(258 gc.9 [call] v29'39 uv39 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.749+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:259 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.749+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 98 ==== osd_op_reply(259 gc.9 [call out=15b] v0'0 uv39 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.749+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.748 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.749+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:260 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec921beb0 2024-05-08T18:08:00.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.750+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 99 ==== osd_op_reply(260 gc.9 [call out=24b] v0'0 uv39 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.750+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:261 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.750+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 100 ==== osd_op_reply(261 gc.9 [call out=15b] v0'0 uv39 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.750+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.750+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:262 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.750+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 101 ==== osd_op_reply(262 gc.9 [call out=15b] v0'0 uv39 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.750+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.749 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.750+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:263 6.1c 6:3e426232:gc::gc.9:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec921beb0 2024-05-08T18:08:00.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.751+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 102 ==== osd_op_reply(263 gc.9 [call] v29'40 uv40 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.751+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-05-08T18:08:00.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.751+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:264 6.3 6:c31f41d7:gc::gc.10:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.752+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 48 ==== osd_op_reply(264 gc.10 [call] v29'90 uv90 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.750 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.752+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:265 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.752+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 49 ==== osd_op_reply(265 gc.10 [call out=15b] v0'0 uv90 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.753+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.753+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:266 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x7ff468037040 2024-05-08T18:08:00.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.753+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 50 ==== osd_op_reply(266 gc.10 [call out=24b] v0'0 uv90 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.753+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:267 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.753+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 51 ==== osd_op_reply(267 gc.10 [call out=15b] v0'0 uv90 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.753+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.753+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:268 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.753+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 52 ==== osd_op_reply(268 gc.10 [call out=15b] v0'0 uv90 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.754+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.752 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.754+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:269 6.3 6:c31f41d7:gc::gc.10:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.755+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 53 ==== osd_op_reply(269 gc.10 [call] v29'91 uv91 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.755+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-05-08T18:08:00.753 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.755+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:270 6.13 6:c8333aab:gc::gc.11:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.756+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 56 ==== osd_op_reply(270 gc.11 [call] v29'43 uv43 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.756+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:271 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.756+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 57 ==== osd_op_reply(271 gc.11 [call out=15b] v0'0 uv43 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.756+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.756+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:272 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.756+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 58 ==== osd_op_reply(272 gc.11 [call out=24b] v0'0 uv43 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.757+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:273 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.757+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 59 ==== osd_op_reply(273 gc.11 [call out=15b] v0'0 uv43 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.757+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.755 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.757+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:274 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.757+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 60 ==== osd_op_reply(274 gc.11 [call out=15b] v0'0 uv43 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.757+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.757+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:275 6.13 6:c8333aab:gc::gc.11:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.758+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 61 ==== osd_op_reply(275 gc.11 [call] v29'44 uv44 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.758+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-05-08T18:08:00.756 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.758+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:276 6.15 6:ab569f81:gc::gc.12:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.759+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 54 ==== osd_op_reply(276 gc.12 [call] v29'33 uv33 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.759+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:277 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.759+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 55 ==== osd_op_reply(277 gc.12 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.759+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.759+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:278 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x7ff468037040 2024-05-08T18:08:00.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.760+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 56 ==== osd_op_reply(278 gc.12 [call out=24b] v0'0 uv33 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.760+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:279 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x7ff468037040 2024-05-08T18:08:00.758 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.760+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 57 ==== osd_op_reply(279 gc.12 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.760+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.760+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:280 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x7ff468037040 2024-05-08T18:08:00.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.760+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 58 ==== osd_op_reply(280 gc.12 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.760+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.760+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:281 6.15 6:ab569f81:gc::gc.12:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x7ff468037040 2024-05-08T18:08:00.759 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.761+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 59 ==== osd_op_reply(281 gc.12 [call] v29'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.761+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-05-08T18:08:00.760 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.761+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:282 6.0 6:0667bb13:gc::gc.13:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.762+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 62 ==== osd_op_reply(282 gc.13 [call] v29'66 uv66 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.762+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:283 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.762+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 63 ==== osd_op_reply(283 gc.13 [call out=15b] v0'0 uv66 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.761 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.762+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.762+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:284 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.763+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 64 ==== osd_op_reply(284 gc.13 [call out=24b] v0'0 uv66 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.763+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:285 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.763+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 65 ==== osd_op_reply(285 gc.13 [call out=15b] v0'0 uv66 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.763+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.763+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:286 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9217550 2024-05-08T18:08:00.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.763+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 66 ==== osd_op_reply(286 gc.13 [call out=15b] v0'0 uv66 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.763+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.762 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.763+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:287 6.0 6:0667bb13:gc::gc.13:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9217550 2024-05-08T18:08:00.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.765+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 67 ==== osd_op_reply(287 gc.13 [call] v29'67 uv67 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.765+0000 7ff47680e800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-05-08T18:08:00.763 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.765+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:288 6.4 6:227f63b5:gc::gc.14:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.766+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 60 ==== osd_op_reply(288 gc.14 [call] v29'282 uv282 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.766+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:289 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.766+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 61 ==== osd_op_reply(289 gc.14 [call out=15b] v0'0 uv282 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.766+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.764 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.766+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:290 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9265a90 con 0x564ec9212b60 2024-05-08T18:08:00.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.766+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 62 ==== osd_op_reply(290 gc.14 [call out=24b] v0'0 uv282 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.766+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:291 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.766+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 63 ==== osd_op_reply(291 gc.14 [call out=15b] v0'0 uv282 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.767+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-05-08T18:08:00.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.767+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:292 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.767+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 64 ==== osd_op_reply(292 gc.14 [call out=15b] v0'0 uv282 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.765 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.767+0000 7ff47680e800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-05-08T18:08:00.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.767+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:293 6.4 6:227f63b5:gc::gc.14:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9266aa0 con 0x564ec9212b60 2024-05-08T18:08:00.766 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.768+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 65 ==== osd_op_reply(293 gc.14 [call] v29'283 uv283 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.769+0000 7ff32a7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-05-08T18:08:00.767 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.769+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:294 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94896381913856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec925eac0 con 0x564ec9212b60 2024-05-08T18:08:00.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.770+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 66 ==== osd_op_reply(294 data_loggenerations_metadata [watch unwatch cookie 94896381913856] v29'284 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.768 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.770+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:295 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94896381887872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9226b00 con 0x7ff468037040 2024-05-08T18:08:00.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.771+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 60 ==== osd_op_reply(295 notify.0 [watch unwatch cookie 94896381887872] v29'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.771+0000 7ff47680e800 20 remove_watcher() i=0 2024-05-08T18:08:00.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.771+0000 7ff47680e800 2 removed watcher, disabling cache 2024-05-08T18:08:00.769 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.771+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:296 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94896381891728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9226b00 con 0x564ec9212b60 2024-05-08T18:08:00.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.772+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 67 ==== osd_op_reply(296 notify.1 [watch unwatch cookie 94896381891728] v29'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.772+0000 7ff47680e800 20 remove_watcher() i=1 2024-05-08T18:08:00.770 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.772+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- osd_op(unknown.0.0:297 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94896381885664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9221490 con 0x564ec921beb0 2024-05-08T18:08:00.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.773+0000 7ff473b05640 1 -- 172.21.15.50:0/189896415 <== osd.3 v2:172.21.15.50:6800/3130492744 103 ==== osd_op_reply(297 notify.2 [watch unwatch cookie 94896381885664] v29'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff468009050 con 0x564ec921beb0 2024-05-08T18:08:00.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.773+0000 7ff47680e800 20 remove_watcher() i=2 2024-05-08T18:08:00.771 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.773+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- osd_op(unknown.0.0:298 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94896381898112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9226b00 con 0x7ff468037040 2024-05-08T18:08:00.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.774+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.2 v2:172.21.15.50:6811/3039646617 61 ==== osd_op_reply(298 notify.3 [watch unwatch cookie 94896381898112] v29'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x7ff468037040 2024-05-08T18:08:00.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.774+0000 7ff47680e800 20 remove_watcher() i=3 2024-05-08T18:08:00.772 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.774+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- osd_op(unknown.0.0:299 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94896381899168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec921fce0 con 0x564ec9212b60 2024-05-08T18:08:00.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.776+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 <== osd.1 v2:172.21.15.50:6808/2141923372 68 ==== osd_op_reply(299 notify.4 [watch unwatch cookie 94896381899168] v29'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff46c032040 con 0x564ec9212b60 2024-05-08T18:08:00.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.776+0000 7ff47680e800 20 remove_watcher() i=4 2024-05-08T18:08:00.774 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.776+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:300 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94896381900224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9221490 con 0x564ec9217550 2024-05-08T18:08:00.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.777+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 68 ==== osd_op_reply(300 notify.5 [watch unwatch cookie 94896381900224] v29'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.777+0000 7ff47680e800 20 remove_watcher() i=5 2024-05-08T18:08:00.775 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.777+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:301 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94896381909936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec9226b00 con 0x564ec9217550 2024-05-08T18:08:00.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.778+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 69 ==== osd_op_reply(301 notify.6 [watch unwatch cookie 94896381909936] v29'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.778+0000 7ff47680e800 20 remove_watcher() i=6 2024-05-08T18:08:00.776 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.778+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- osd_op(unknown.0.0:302 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94896381906160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29) -- 0x564ec921fce0 con 0x564ec9217550 2024-05-08T18:08:00.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff474306640 1 -- 172.21.15.50:0/189896415 <== osd.0 v2:172.21.15.46:6800/337143975 70 ==== osd_op_reply(302 notify.7 [watch unwatch cookie 94896381906160] v29'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff4640bf020 con 0x564ec9217550 2024-05-08T18:08:00.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff47680e800 20 remove_watcher() i=7 2024-05-08T18:08:00.779 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ec9217550 msgr2=0x564ec9219940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff47680e800 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ec9217550 0x564ec9219940 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:00.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ec9212b60 msgr2=0x564ec9215020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff47680e800 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ec9212b60 0x564ec9215020 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:00.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff468037040 msgr2=0x7ff468036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff47680e800 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7ff468037040 0x7ff468036410 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:00.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ec921beb0 msgr2=0x564ec921e370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff47680e800 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ec921beb0 0x564ec921e370 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:00.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff34403bb50 msgr2=0x7ff34403e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff47680e800 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff34403bb50 0x7ff34403e010 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff46800efd0 tx=0x7ff468010000 comp rx=0 tx=0).stop 2024-05-08T18:08:00.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec9200240 msgr2=0x564ec9207650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff47680e800 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec9200240 0x564ec9207650 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff4640539c0 tx=0x7ff464064f20 comp rx=0 tx=0).stop 2024-05-08T18:08:00.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff474b07640 1 -- 172.21.15.50:0/189896415 reap_dead start 2024-05-08T18:08:00.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 shutdown_connections 2024-05-08T18:08:00.780 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff47680e800 1 --2- 172.21.15.50:0/189896415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec9200240 0x564ec9207650 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:00.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.779+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 >> 172.21.15.50:0/189896415 conn(0x564ec91fc550 msgr2=0x564ec91f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:00.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 shutdown_connections 2024-05-08T18:08:00.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 -- 172.21.15.50:0/189896415 wait complete. 2024-05-08T18:08:00.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 -- 172.21.15.50:0/742178631 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff34c03bb50 msgr2=0x7ff34c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 --2- 172.21.15.50:0/742178631 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff34c03bb50 0x7ff34c03e010 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff4680016b0 tx=0x7ff46800f000 comp rx=0 tx=0).stop 2024-05-08T18:08:00.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 -- 172.21.15.50:0/742178631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec91f30d0 msgr2=0x564ec91f34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 --2- 172.21.15.50:0/742178631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec91f30d0 0x564ec91f34f0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff464053560 tx=0x7ff46405ef20 comp rx=0 tx=0).stop 2024-05-08T18:08:00.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 -- 172.21.15.50:0/742178631 shutdown_connections 2024-05-08T18:08:00.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 --2- 172.21.15.50:0/742178631 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff34c03bb50 0x7ff34c03e010 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:00.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 --2- 172.21.15.50:0/742178631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec91f30d0 0x564ec91f34f0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:00.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 -- 172.21.15.50:0/742178631 >> 172.21.15.50:0/742178631 conn(0x564ec91e7170 msgr2=0x564ec91e7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:00.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 -- 172.21.15.50:0/742178631 shutdown_connections 2024-05-08T18:08:00.781 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 -- 172.21.15.50:0/742178631 wait complete. 2024-05-08T18:08:00.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 -- 172.21.15.50:0/2926117503 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ec91de270 msgr2=0x564ec91e0710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x564ec91de270 0x564ec91e0710 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:00.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 -- 172.21.15.50:0/2926117503 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ec90d97b0 msgr2=0x564ec91dbab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x564ec90d97b0 0x564ec91dbab0 crc :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:00.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 -- 172.21.15.50:0/2926117503 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ec91e2300 msgr2=0x564ec91e46f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x564ec91e2300 0x564ec91e46f0 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:00.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 -- 172.21.15.50:0/2926117503 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff35c03bb50 msgr2=0x7ff35c03e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff35c03bb50 0x7ff35c03e010 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff468001d70 tx=0x7ff4680012d0 comp rx=0 tx=0).stop 2024-05-08T18:08:00.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.780+0000 7ff47680e800 1 -- 172.21.15.50:0/2926117503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 msgr2=0x564ec91d4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.781+0000 7ff47680e800 1 --2- 172.21.15.50:0/2926117503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564ec90d8550 0x564ec91d4910 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff46401da60 tx=0x7ff464002fb0 comp rx=0 tx=0).stop 2024-05-08T18:08:00.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.781+0000 7ff474b07640 1 -- 172.21.15.50:0/2926117503 reap_dead start 2024-05-08T18:08:00.782 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.781+0000 7ff47680e800 1 -- 172.21.15.50:0/2926117503 shutdown_connections 2024-05-08T18:08:00.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.781+0000 7ff47680e800 1 -- 172.21.15.50:0/2926117503 >> 172.21.15.50:0/2926117503 conn(0x564ec90d3b50 msgr2=0x564ec91c2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:00.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.781+0000 7ff47680e800 1 -- 172.21.15.50:0/2926117503 shutdown_connections 2024-05-08T18:08:00.783 INFO:teuthology.orchestra.run.smithi050.stderr:2024-05-08T18:08:00.781+0000 7ff47680e800 1 -- 172.21.15.50:0/2926117503 wait complete. 2024-05-08T18:08:00.788 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-08T18:08:00.798 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T18:08:00.798 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T18:08:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.907+0000 7f2c5647e640 1 Processor -- start 2024-05-08T18:08:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.907+0000 7f2c5647e640 1 -- start start 2024-05-08T18:08:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.907+0000 7f2c5647e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2c50130480 0x7f2c50132870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.907+0000 7f2c5647e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c501144f0 con 0x7f2c50129ea0 2024-05-08T18:08:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.907+0000 7f2c5647e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f2c50129ca0 con 0x7f2c50130480 2024-05-08T18:08:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.907+0000 7f2c47fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2c50130480 0x7f2c50132870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.907+0000 7f2c4ffff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2c50129ea0 0x7f2c5012a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45148/0 (socket says 172.21.15.46:45148) 2024-05-08T18:08:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.907+0000 7f2c4ffff640 1 -- 172.21.15.46:0/687589326 learned_addr learned my addr 172.21.15.46:0/687589326 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:08:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.908+0000 7f2c47fff640 1 -- 172.21.15.46:0/687589326 >> v1:172.21.15.46:6789/0 conn(0x7f2c50129ea0 legacy=0x7f2c5012a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.908+0000 7f2c47fff640 1 -- 172.21.15.46:0/687589326 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c5009acc0 con 0x7f2c50130480 2024-05-08T18:08:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.908+0000 7f2c47fff640 1 --2- 172.21.15.46:0/687589326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c50130480 0x7f2c50132870 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009720 tx=0x7f2c3802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9295c811080417 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.908+0000 7f2c4f7fe640 1 -- 172.21.15.46:0/687589326 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c380316d0 con 0x7f2c50130480 2024-05-08T18:08:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.909+0000 7f2c4f7fe640 1 -- 172.21.15.46:0/687589326 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c38037070 con 0x7f2c50130480 2024-05-08T18:08:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.909+0000 7f2c4f7fe640 1 -- 172.21.15.46:0/687589326 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c380319c0 con 0x7f2c50130480 2024-05-08T18:08:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.909+0000 7f2c5647e640 1 -- 172.21.15.46:0/687589326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c50130480 msgr2=0x7f2c50132870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.909+0000 7f2c5647e640 1 --2- 172.21.15.46:0/687589326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c50130480 0x7f2c50132870 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f2c38009720 tx=0x7f2c3802b840 comp rx=0 tx=0).stop 2024-05-08T18:08:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.909+0000 7f2c5647e640 1 -- 172.21.15.46:0/687589326 shutdown_connections 2024-05-08T18:08:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.909+0000 7f2c5647e640 1 --2- 172.21.15.46:0/687589326 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c50130480 0x7f2c50132870 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.909+0000 7f2c5647e640 1 -- 172.21.15.46:0/687589326 >> 172.21.15.46:0/687589326 conn(0x7f2c5009d7a0 msgr2=0x7f2c5009dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.909+0000 7f2c5647e640 1 -- 172.21.15.46:0/687589326 shutdown_connections 2024-05-08T18:08:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.909+0000 7f2c5647e640 1 -- 172.21.15.46:0/687589326 wait complete. 2024-05-08T18:08:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.910+0000 7f2c5647e640 1 Processor -- start 2024-05-08T18:08:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.910+0000 7f2c5647e640 1 -- start start 2024-05-08T18:08:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.910+0000 7f2c5647e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c50129ea0 0x7f2c501c1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.910+0000 7f2c5647e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2c50113c10 con 0x7f2c50129ea0 2024-05-08T18:08:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.911+0000 7f2c4ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c50129ea0 0x7f2c501c1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.911+0000 7f2c4ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c50129ea0 0x7f2c501c1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52002/0 (socket says 172.21.15.46:52002) 2024-05-08T18:08:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.911+0000 7f2c4ffff640 1 -- 172.21.15.46:0/1848463641 learned_addr learned my addr 172.21.15.46:0/1848463641 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:08:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.911+0000 7f2c4ffff640 1 -- 172.21.15.46:0/1848463641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c50123ef0 con 0x7f2c50129ea0 2024-05-08T18:08:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.911+0000 7f2c4ffff640 1 --2- 172.21.15.46:0/1848463641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c50129ea0 0x7f2c501c1d10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000c480 tx=0x7f2c4000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.911+0000 7f2c4d7fa640 1 -- 172.21.15.46:0/1848463641 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c40017070 con 0x7f2c50129ea0 2024-05-08T18:08:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.911+0000 7f2c4d7fa640 1 -- 172.21.15.46:0/1848463641 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c40005c10 con 0x7f2c50129ea0 2024-05-08T18:08:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.911+0000 7f2c5647e640 1 -- 172.21.15.46:0/1848463641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c50126a40 con 0x7f2c50129ea0 2024-05-08T18:08:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.911+0000 7f2c4d7fa640 1 -- 172.21.15.46:0/1848463641 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c40007070 con 0x7f2c50129ea0 2024-05-08T18:08:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.911+0000 7f2c5647e640 1 -- 172.21.15.46:0/1848463641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c50117870 con 0x7f2c50129ea0 2024-05-08T18:08:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.912+0000 7f2c4d7fa640 1 -- 172.21.15.46:0/1848463641 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f2c4001a070 con 0x7f2c50129ea0 2024-05-08T18:08:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.912+0000 7f2c5647e640 1 -- 172.21.15.46:0/1848463641 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c500640a0 con 0x7f2c50129ea0 2024-05-08T18:08:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.912+0000 7f2c4d7fa640 1 --2- 172.21.15.46:0/1848463641 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2c3003d2b0 0x7f2c3003f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.912+0000 7f2c4d7fa640 1 -- 172.21.15.46:0/1848463641 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f2c400510f0 con 0x7f2c50129ea0 2024-05-08T18:08:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.913+0000 7f2c47fff640 1 --2- 172.21.15.46:0/1848463641 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2c3003d2b0 0x7f2c3003f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.913+0000 7f2c47fff640 1 --2- 172.21.15.46:0/1848463641 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2c3003d2b0 0x7f2c3003f770 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2c38036000 tx=0x7f2c38008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:00.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:00.917+0000 7f2c4d7fa640 1 -- 172.21.15.46:0/1848463641 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f2c4001ac60 con 0x7f2c50129ea0 2024-05-08T18:08:01.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.105+0000 7f2c5647e640 1 -- 172.21.15.46:0/1848463641 --> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2c501240c0 con 0x7f2c3003d2b0 2024-05-08T18:08:01.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.122+0000 7f2c4d7fa640 1 -- 172.21.15.46:0/1848463641 <== mgr.4098 v2:172.21.15.50:6824/2602185226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665903 (secure 0 0 0) 0x7f2c501240c0 con 0x7f2c3003d2b0 2024-05-08T18:08:01.128 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:08:01.134 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T18:08:01.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.127+0000 7f2c5647e640 1 -- 172.21.15.46:0/1848463641 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2c3003d2b0 msgr2=0x7f2c3003f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.127+0000 7f2c5647e640 1 --2- 172.21.15.46:0/1848463641 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2c3003d2b0 0x7f2c3003f770 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2c38036000 tx=0x7f2c38008030 comp rx=0 tx=0).stop 2024-05-08T18:08:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.127+0000 7f2c5647e640 1 -- 172.21.15.46:0/1848463641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c50129ea0 msgr2=0x7f2c501c1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.127+0000 7f2c5647e640 1 --2- 172.21.15.46:0/1848463641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c50129ea0 0x7f2c501c1d10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2c4000c480 tx=0x7f2c4000c940 comp rx=0 tx=0).stop 2024-05-08T18:08:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.127+0000 7f2c5647e640 1 -- 172.21.15.46:0/1848463641 shutdown_connections 2024-05-08T18:08:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.127+0000 7f2c5647e640 1 --2- 172.21.15.46:0/1848463641 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f2c3003d2b0 0x7f2c3003f770 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.127+0000 7f2c5647e640 1 --2- 172.21.15.46:0/1848463641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c50129ea0 0x7f2c501c1d10 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.127+0000 7f2c5647e640 1 -- 172.21.15.46:0/1848463641 >> 172.21.15.46:0/1848463641 conn(0x7f2c5009d7a0 msgr2=0x7f2c501b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.127+0000 7f2c5647e640 1 -- 172.21.15.46:0/1848463641 shutdown_connections 2024-05-08T18:08:01.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.127+0000 7f2c5647e640 1 -- 172.21.15.46:0/1848463641 wait complete. 2024-05-08T18:08:01.138 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-05-08T18:07:59.727131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26416886,"num_objects":227,"num_object_clones":0,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":227,"num_whiteouts":0,"num_read":4834,"num_read_kb":4416,"num_write":2046,"num_write_kb":60584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4431,"ondisk_log_size":4431,"up":658,"acting":658,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":658,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":186032,"kb_used_data":36368,"kb_used_omap":6,"kb_used_meta":149625,"kb_avail":377301328,"statfs":{"total":386547056640,"available":386356559872,"internally_reserved":0,"allocated":37240832,"data_stored":36146252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":153216790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25815784,"num_objects":9,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":497,"num_read_kb":452,"num_write":384,"num_write_kb":29371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003443"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866950+0000","last_change":"2024-05-08T18:04:43.836900+0000","last_active":"2024-05-08T18:05:30.866950+0000","last_peered":"2024-05-08T18:05:30.866950+0000","last_clean":"2024-05-08T18:05:30.866950+0000","last_became_active":"2024-05-08T18:04:43.836496+0000","last_became_peered":"2024-05-08T18:04:43.836496+0000","last_unstale":"2024-05-08T18:05:30.866950+0000","last_undegraded":"2024-05-08T18:05:30.866950+0000","last_fullsized":"2024-05-08T18:05:30.866950+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:22.454546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'6","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.860763+0000","last_change":"2024-05-08T18:04:45.812340+0000","last_active":"2024-05-08T18:07:53.860763+0000","last_peered":"2024-05-08T18:07:53.860763+0000","last_clean":"2024-05-08T18:07:53.860763+0000","last_became_active":"2024-05-08T18:04:45.811368+0000","last_became_peered":"2024-05-08T18:04:45.811368+0000","last_unstale":"2024-05-08T18:07:53.860763+0000","last_undegraded":"2024-05-08T18:07:53.860763+0000","last_fullsized":"2024-05-08T18:07:53.860763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:36:45.459800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.206267+0000","last_change":"2024-05-08T18:04:43.835466+0000","last_active":"2024-05-08T18:06:50.206267+0000","last_peered":"2024-05-08T18:06:50.206267+0000","last_clean":"2024-05-08T18:06:50.206267+0000","last_became_active":"2024-05-08T18:04:43.834612+0000","last_became_peered":"2024-05-08T18:04:43.834612+0000","last_unstale":"2024-05-08T18:06:50.206267+0000","last_undegraded":"2024-05-08T18:06:50.206267+0000","last_fullsized":"2024-05-08T18:06:50.206267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:46.422109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"29'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.546966+0000","last_change":"2024-05-08T18:04:45.828749+0000","last_active":"2024-05-08T18:06:09.546966+0000","last_peered":"2024-05-08T18:06:09.546966+0000","last_clean":"2024-05-08T18:06:09.546966+0000","last_became_active":"2024-05-08T18:04:45.827965+0000","last_became_peered":"2024-05-08T18:04:45.827965+0000","last_unstale":"2024-05-08T18:06:09.546966+0000","last_undegraded":"2024-05-08T18:06:09.546966+0000","last_fullsized":"2024-05-08T18:06:09.546966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:56:23.117814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861383+0000","last_change":"2024-05-08T18:04:43.843806+0000","last_active":"2024-05-08T18:05:30.861383+0000","last_peered":"2024-05-08T18:05:30.861383+0000","last_clean":"2024-05-08T18:05:30.861383+0000","last_became_active":"2024-05-08T18:04:43.837456+0000","last_became_peered":"2024-05-08T18:04:43.837456+0000","last_unstale":"2024-05-08T18:05:30.861383+0000","last_undegraded":"2024-05-08T18:05:30.861383+0000","last_fullsized":"2024-05-08T18:05:30.861383+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:00:24.839225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"29'12","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063738+0000","last_change":"2024-05-08T18:04:45.825315+0000","last_active":"2024-05-08T18:07:54.063738+0000","last_peered":"2024-05-08T18:07:54.063738+0000","last_clean":"2024-05-08T18:07:54.063738+0000","last_became_active":"2024-05-08T18:04:45.825022+0000","last_became_peered":"2024-05-08T18:04:45.825022+0000","last_unstale":"2024-05-08T18:07:54.063738+0000","last_undegraded":"2024-05-08T18:07:54.063738+0000","last_fullsized":"2024-05-08T18:07:54.063738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:45:53.925354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863324+0000","last_change":"2024-05-08T18:04:43.842478+0000","last_active":"2024-05-08T18:05:30.863324+0000","last_peered":"2024-05-08T18:05:30.863324+0000","last_clean":"2024-05-08T18:05:30.863324+0000","last_became_active":"2024-05-08T18:04:43.842077+0000","last_became_peered":"2024-05-08T18:04:43.842077+0000","last_unstale":"2024-05-08T18:05:30.863324+0000","last_undegraded":"2024-05-08T18:05:30.863324+0000","last_fullsized":"2024-05-08T18:05:30.863324+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:06.338736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"29'16","reported_seq":70,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.035049+0000","last_change":"2024-05-08T18:04:45.824870+0000","last_active":"2024-05-08T18:07:54.035049+0000","last_peered":"2024-05-08T18:07:54.035049+0000","last_clean":"2024-05-08T18:07:54.035049+0000","last_became_active":"2024-05-08T18:04:45.824581+0000","last_became_peered":"2024-05-08T18:04:45.824581+0000","last_unstale":"2024-05-08T18:07:54.035049+0000","last_undegraded":"2024-05-08T18:07:54.035049+0000","last_fullsized":"2024-05-08T18:07:54.035049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:48.934445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859690+0000","last_change":"2024-05-08T18:04:43.833461+0000","last_active":"2024-05-08T18:05:30.859690+0000","last_peered":"2024-05-08T18:05:30.859690+0000","last_clean":"2024-05-08T18:05:30.859690+0000","last_became_active":"2024-05-08T18:04:43.832722+0000","last_became_peered":"2024-05-08T18:04:43.832722+0000","last_unstale":"2024-05-08T18:05:30.859690+0000","last_undegraded":"2024-05-08T18:05:30.859690+0000","last_fullsized":"2024-05-08T18:05:30.859690+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:11:15.182813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"29'8","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.738600+0000","last_change":"2024-05-08T18:04:45.808678+0000","last_active":"2024-05-08T18:06:09.738600+0000","last_peered":"2024-05-08T18:06:09.738600+0000","last_clean":"2024-05-08T18:06:09.738600+0000","last_became_active":"2024-05-08T18:04:45.808383+0000","last_became_peered":"2024-05-08T18:04:45.808383+0000","last_unstale":"2024-05-08T18:06:09.738600+0000","last_undegraded":"2024-05-08T18:06:09.738600+0000","last_fullsized":"2024-05-08T18:06:09.738600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:02.599918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862756+0000","last_change":"2024-05-08T18:04:43.841284+0000","last_active":"2024-05-08T18:05:30.862756+0000","last_peered":"2024-05-08T18:05:30.862756+0000","last_clean":"2024-05-08T18:05:30.862756+0000","last_became_active":"2024-05-08T18:04:43.840851+0000","last_became_peered":"2024-05-08T18:04:43.840851+0000","last_unstale":"2024-05-08T18:05:30.862756+0000","last_undegraded":"2024-05-08T18:05:30.862756+0000","last_fullsized":"2024-05-08T18:05:30.862756+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:10.360847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063767+0000","last_change":"2024-05-08T18:04:45.829481+0000","last_active":"2024-05-08T18:07:54.063767+0000","last_peered":"2024-05-08T18:07:54.063767+0000","last_clean":"2024-05-08T18:07:54.063767+0000","last_became_active":"2024-05-08T18:04:45.828334+0000","last_became_peered":"2024-05-08T18:04:45.828334+0000","last_unstale":"2024-05-08T18:07:54.063767+0000","last_undegraded":"2024-05-08T18:07:54.063767+0000","last_fullsized":"2024-05-08T18:07:54.063767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:20:31.411889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.978496+0000","last_change":"2024-05-08T18:04:43.833826+0000","last_active":"2024-05-08T18:07:53.978496+0000","last_peered":"2024-05-08T18:07:53.978496+0000","last_clean":"2024-05-08T18:07:53.978496+0000","last_became_active":"2024-05-08T18:04:43.833437+0000","last_became_peered":"2024-05-08T18:04:43.833437+0000","last_unstale":"2024-05-08T18:07:53.978496+0000","last_undegraded":"2024-05-08T18:07:53.978496+0000","last_fullsized":"2024-05-08T18:07:53.978496+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:46.740116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.868015+0000","last_change":"2024-05-08T18:04:45.812158+0000","last_active":"2024-05-08T18:05:30.868015+0000","last_peered":"2024-05-08T18:05:30.868015+0000","last_clean":"2024-05-08T18:05:30.868015+0000","last_became_active":"2024-05-08T18:04:45.811884+0000","last_became_peered":"2024-05-08T18:04:45.811884+0000","last_unstale":"2024-05-08T18:05:30.868015+0000","last_undegraded":"2024-05-08T18:05:30.868015+0000","last_fullsized":"2024-05-08T18:05:30.868015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:30.458103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858546+0000","last_change":"2024-05-08T18:04:43.821005+0000","last_active":"2024-05-08T18:05:30.858546+0000","last_peered":"2024-05-08T18:05:30.858546+0000","last_clean":"2024-05-08T18:05:30.858546+0000","last_became_active":"2024-05-08T18:04:43.820312+0000","last_became_peered":"2024-05-08T18:04:43.820312+0000","last_unstale":"2024-05-08T18:05:30.858546+0000","last_undegraded":"2024-05-08T18:05:30.858546+0000","last_fullsized":"2024-05-08T18:05:30.858546+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:40:40.407240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"29'7","reported_seq":61,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070848+0000","last_change":"2024-05-08T18:04:45.823448+0000","last_active":"2024-05-08T18:07:54.070848+0000","last_peered":"2024-05-08T18:07:54.070848+0000","last_clean":"2024-05-08T18:07:54.070848+0000","last_became_active":"2024-05-08T18:04:45.822620+0000","last_became_peered":"2024-05-08T18:04:45.822620+0000","last_unstale":"2024-05-08T18:07:54.070848+0000","last_undegraded":"2024-05-08T18:07:54.070848+0000","last_fullsized":"2024-05-08T18:07:54.070848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:08.184602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"29'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995728+0000","last_change":"2024-05-08T18:04:45.826814+0000","last_active":"2024-05-08T18:07:53.995728+0000","last_peered":"2024-05-08T18:07:53.995728+0000","last_clean":"2024-05-08T18:07:53.995728+0000","last_became_active":"2024-05-08T18:04:45.826493+0000","last_became_peered":"2024-05-08T18:04:45.826493+0000","last_unstale":"2024-05-08T18:07:53.995728+0000","last_undegraded":"2024-05-08T18:07:53.995728+0000","last_fullsized":"2024-05-08T18:07:53.995728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:12.401514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866321+0000","last_change":"2024-05-08T18:04:43.838700+0000","last_active":"2024-05-08T18:05:30.866321+0000","last_peered":"2024-05-08T18:05:30.866321+0000","last_clean":"2024-05-08T18:05:30.866321+0000","last_became_active":"2024-05-08T18:04:43.838332+0000","last_became_peered":"2024-05-08T18:04:43.838332+0000","last_unstale":"2024-05-08T18:05:30.866321+0000","last_undegraded":"2024-05-08T18:05:30.866321+0000","last_fullsized":"2024-05-08T18:05:30.866321+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:01:49.620924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"29'4","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399796+0000","last_change":"2024-05-08T18:04:45.807732+0000","last_active":"2024-05-08T18:06:53.399796+0000","last_peered":"2024-05-08T18:06:53.399796+0000","last_clean":"2024-05-08T18:06:53.399796+0000","last_became_active":"2024-05-08T18:04:45.807438+0000","last_became_peered":"2024-05-08T18:04:45.807438+0000","last_unstale":"2024-05-08T18:06:53.399796+0000","last_undegraded":"2024-05-08T18:06:53.399796+0000","last_fullsized":"2024-05-08T18:06:53.399796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:27:20.114031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867876+0000","last_change":"2024-05-08T18:04:43.843599+0000","last_active":"2024-05-08T18:05:30.867876+0000","last_peered":"2024-05-08T18:05:30.867876+0000","last_clean":"2024-05-08T18:05:30.867876+0000","last_became_active":"2024-05-08T18:04:43.842741+0000","last_became_peered":"2024-05-08T18:04:43.842741+0000","last_unstale":"2024-05-08T18:05:30.867876+0000","last_undegraded":"2024-05-08T18:05:30.867876+0000","last_fullsized":"2024-05-08T18:05:30.867876+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:54.711833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"29'11","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.002969+0000","last_change":"2024-05-08T18:04:45.811141+0000","last_active":"2024-05-08T18:07:54.002969+0000","last_peered":"2024-05-08T18:07:54.002969+0000","last_clean":"2024-05-08T18:07:54.002969+0000","last_became_active":"2024-05-08T18:04:45.810827+0000","last_became_peered":"2024-05-08T18:04:45.810827+0000","last_unstale":"2024-05-08T18:07:54.002969+0000","last_undegraded":"2024-05-08T18:07:54.002969+0000","last_fullsized":"2024-05-08T18:07:54.002969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:26.952570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"29'3","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746208+0000","last_change":"2024-05-08T18:04:43.838115+0000","last_active":"2024-05-08T18:07:08.746208+0000","last_peered":"2024-05-08T18:07:08.746208+0000","last_clean":"2024-05-08T18:07:08.746208+0000","last_became_active":"2024-05-08T18:04:43.837706+0000","last_became_peered":"2024-05-08T18:04:43.837706+0000","last_unstale":"2024-05-08T18:07:08.746208+0000","last_undegraded":"2024-05-08T18:07:08.746208+0000","last_fullsized":"2024-05-08T18:07:08.746208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:26.918679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"29'8","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.738522+0000","last_change":"2024-05-08T18:04:45.822292+0000","last_active":"2024-05-08T18:06:09.738522+0000","last_peered":"2024-05-08T18:06:09.738522+0000","last_clean":"2024-05-08T18:06:09.738522+0000","last_became_active":"2024-05-08T18:04:45.822000+0000","last_became_peered":"2024-05-08T18:04:45.822000+0000","last_unstale":"2024-05-08T18:06:09.738522+0000","last_undegraded":"2024-05-08T18:06:09.738522+0000","last_fullsized":"2024-05-08T18:06:09.738522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:08.184602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866478+0000","last_change":"2024-05-08T18:04:43.841486+0000","last_active":"2024-05-08T18:05:30.866478+0000","last_peered":"2024-05-08T18:05:30.866478+0000","last_clean":"2024-05-08T18:05:30.866478+0000","last_became_active":"2024-05-08T18:04:43.841063+0000","last_became_peered":"2024-05-08T18:04:43.841063+0000","last_unstale":"2024-05-08T18:05:30.866478+0000","last_undegraded":"2024-05-08T18:05:30.866478+0000","last_fullsized":"2024-05-08T18:05:30.866478+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:05:33.776915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"29'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.116184+0000","last_change":"2024-05-08T18:04:45.824006+0000","last_active":"2024-05-08T18:07:54.116184+0000","last_peered":"2024-05-08T18:07:54.116184+0000","last_clean":"2024-05-08T18:07:54.116184+0000","last_became_active":"2024-05-08T18:04:45.823611+0000","last_became_peered":"2024-05-08T18:04:45.823611+0000","last_unstale":"2024-05-08T18:07:54.116184+0000","last_undegraded":"2024-05-08T18:07:54.116184+0000","last_fullsized":"2024-05-08T18:07:54.116184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:02.599918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860254+0000","last_change":"2024-05-08T18:04:43.835685+0000","last_active":"2024-05-08T18:05:30.860254+0000","last_peered":"2024-05-08T18:05:30.860254+0000","last_clean":"2024-05-08T18:05:30.860254+0000","last_became_active":"2024-05-08T18:04:43.834456+0000","last_became_peered":"2024-05-08T18:04:43.834456+0000","last_unstale":"2024-05-08T18:05:30.860254+0000","last_undegraded":"2024-05-08T18:05:30.860254+0000","last_fullsized":"2024-05-08T18:05:30.860254+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:50.043665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399432+0000","last_change":"2024-05-08T18:04:45.808763+0000","last_active":"2024-05-08T18:06:53.399432+0000","last_peered":"2024-05-08T18:06:53.399432+0000","last_clean":"2024-05-08T18:06:53.399432+0000","last_became_active":"2024-05-08T18:04:45.808416+0000","last_became_peered":"2024-05-08T18:04:45.808416+0000","last_unstale":"2024-05-08T18:06:53.399432+0000","last_undegraded":"2024-05-08T18:06:53.399432+0000","last_fullsized":"2024-05-08T18:06:53.399432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:35.780903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'4","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858633+0000","last_change":"2024-05-08T18:04:43.845458+0000","last_active":"2024-05-08T18:05:30.858633+0000","last_peered":"2024-05-08T18:05:30.858633+0000","last_clean":"2024-05-08T18:05:30.858633+0000","last_became_active":"2024-05-08T18:04:43.844702+0000","last_became_peered":"2024-05-08T18:04:43.844702+0000","last_unstale":"2024-05-08T18:05:30.858633+0000","last_undegraded":"2024-05-08T18:05:30.858633+0000","last_fullsized":"2024-05-08T18:05:30.858633+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:58.229253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"29'35","reported_seq":99,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.016701+0000","last_change":"2024-05-08T18:04:53.849775+0000","last_active":"2024-05-08T18:07:54.016701+0000","last_peered":"2024-05-08T18:07:54.016701+0000","last_clean":"2024-05-08T18:07:54.016701+0000","last_became_active":"2024-05-08T18:04:53.849475+0000","last_became_peered":"2024-05-08T18:04:53.849475+0000","last_unstale":"2024-05-08T18:07:54.016701+0000","last_undegraded":"2024-05-08T18:07:54.016701+0000","last_fullsized":"2024-05-08T18:07:54.016701+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:12.918578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"29'12","reported_seq":66,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995839+0000","last_change":"2024-05-08T18:04:45.824289+0000","last_active":"2024-05-08T18:07:53.995839+0000","last_peered":"2024-05-08T18:07:53.995839+0000","last_clean":"2024-05-08T18:07:53.995839+0000","last_became_active":"2024-05-08T18:04:45.824005+0000","last_became_peered":"2024-05-08T18:04:45.824005+0000","last_unstale":"2024-05-08T18:07:53.995839+0000","last_undegraded":"2024-05-08T18:07:53.995839+0000","last_fullsized":"2024-05-08T18:07:53.995839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:45.743430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859173+0000","last_change":"2024-05-08T18:04:43.834249+0000","last_active":"2024-05-08T18:05:30.859173+0000","last_peered":"2024-05-08T18:05:30.859173+0000","last_clean":"2024-05-08T18:05:30.859173+0000","last_became_active":"2024-05-08T18:04:43.833822+0000","last_became_peered":"2024-05-08T18:04:43.833822+0000","last_unstale":"2024-05-08T18:05:30.859173+0000","last_undegraded":"2024-05-08T18:05:30.859173+0000","last_fullsized":"2024-05-08T18:05:30.859173+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:00:37.867739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859597+0000","last_change":"2024-05-08T18:04:47.820083+0000","last_active":"2024-05-08T18:05:30.859597+0000","last_peered":"2024-05-08T18:05:30.859597+0000","last_clean":"2024-05-08T18:05:30.859597+0000","last_became_active":"2024-05-08T18:04:47.819682+0000","last_became_peered":"2024-05-08T18:04:47.819682+0000","last_unstale":"2024-05-08T18:05:30.859597+0000","last_undegraded":"2024-05-08T18:05:30.859597+0000","last_fullsized":"2024-05-08T18:05:30.859597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:00:55.772655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"29'31","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.748532+0000","last_change":"2024-05-08T18:04:49.833877+0000","last_active":"2024-05-08T18:07:18.748532+0000","last_peered":"2024-05-08T18:07:18.748532+0000","last_clean":"2024-05-08T18:07:18.748532+0000","last_became_active":"2024-05-08T18:04:49.832148+0000","last_became_peered":"2024-05-08T18:04:49.832148+0000","last_unstale":"2024-05-08T18:07:18.748532+0000","last_undegraded":"2024-05-08T18:07:18.748532+0000","last_fullsized":"2024-05-08T18:07:18.748532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:20:21.388280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5807,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":31,"num_read_kb":25,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866106+0000","last_change":"2024-05-08T18:04:51.845281+0000","last_active":"2024-05-08T18:05:30.866106+0000","last_peered":"2024-05-08T18:05:30.866106+0000","last_clean":"2024-05-08T18:05:30.866106+0000","last_became_active":"2024-05-08T18:04:51.844901+0000","last_became_peered":"2024-05-08T18:04:51.844901+0000","last_unstale":"2024-05-08T18:05:30.866106+0000","last_undegraded":"2024-05-08T18:05:30.866106+0000","last_fullsized":"2024-05-08T18:05:30.866106+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:46.002263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"29'11","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.552238+0000","last_change":"2024-05-08T18:04:53.849933+0000","last_active":"2024-05-08T18:06:09.552238+0000","last_peered":"2024-05-08T18:06:09.552238+0000","last_clean":"2024-05-08T18:06:09.552238+0000","last_became_active":"2024-05-08T18:04:53.849579+0000","last_became_peered":"2024-05-08T18:04:53.849579+0000","last_unstale":"2024-05-08T18:06:09.552238+0000","last_undegraded":"2024-05-08T18:06:09.552238+0000","last_fullsized":"2024-05-08T18:06:09.552238+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:42:45.987685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"29'6","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.064004+0000","last_change":"2024-05-08T18:04:45.826976+0000","last_active":"2024-05-08T18:07:54.064004+0000","last_peered":"2024-05-08T18:07:54.064004+0000","last_clean":"2024-05-08T18:07:54.064004+0000","last_became_active":"2024-05-08T18:04:45.825162+0000","last_became_peered":"2024-05-08T18:04:45.825162+0000","last_unstale":"2024-05-08T18:07:54.064004+0000","last_undegraded":"2024-05-08T18:07:54.064004+0000","last_fullsized":"2024-05-08T18:07:54.064004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:36.088260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862438+0000","last_change":"2024-05-08T18:04:43.827844+0000","last_active":"2024-05-08T18:05:30.862438+0000","last_peered":"2024-05-08T18:05:30.862438+0000","last_clean":"2024-05-08T18:05:30.862438+0000","last_became_active":"2024-05-08T18:04:43.827374+0000","last_became_peered":"2024-05-08T18:04:43.827374+0000","last_unstale":"2024-05-08T18:05:30.862438+0000","last_undegraded":"2024-05-08T18:05:30.862438+0000","last_fullsized":"2024-05-08T18:05:30.862438+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:26.918679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:33.732471+0000","last_change":"2024-05-08T18:05:30.868535+0000","last_active":"2024-05-08T18:05:33.732471+0000","last_peered":"2024-05-08T18:05:33.732471+0000","last_clean":"2024-05-08T18:05:33.732471+0000","last_became_active":"2024-05-08T18:05:30.868191+0000","last_became_peered":"2024-05-08T18:05:30.868191+0000","last_unstale":"2024-05-08T18:05:33.732471+0000","last_undegraded":"2024-05-08T18:05:33.732471+0000","last_fullsized":"2024-05-08T18:05:33.732471+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:19.468033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"27'6","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862185+0000","last_change":"2024-05-08T18:04:49.826617+0000","last_active":"2024-05-08T18:05:30.862185+0000","last_peered":"2024-05-08T18:05:30.862185+0000","last_clean":"2024-05-08T18:05:30.862185+0000","last_became_active":"2024-05-08T18:04:49.825265+0000","last_became_peered":"2024-05-08T18:04:49.825265+0000","last_unstale":"2024-05-08T18:05:30.862185+0000","last_undegraded":"2024-05-08T18:05:30.862185+0000","last_fullsized":"2024-05-08T18:05:30.862185+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:35.867914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862381+0000","last_change":"2024-05-08T18:04:51.834892+0000","last_active":"2024-05-08T18:05:30.862381+0000","last_peered":"2024-05-08T18:05:30.862381+0000","last_clean":"2024-05-08T18:05:30.862381+0000","last_became_active":"2024-05-08T18:04:51.833523+0000","last_became_peered":"2024-05-08T18:04:51.833523+0000","last_unstale":"2024-05-08T18:05:30.862381+0000","last_undegraded":"2024-05-08T18:05:30.862381+0000","last_fullsized":"2024-05-08T18:05:30.862381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:01.176258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'2","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858040+0000","last_change":"2024-05-08T18:04:53.840760+0000","last_active":"2024-05-08T18:05:30.858040+0000","last_peered":"2024-05-08T18:05:30.858040+0000","last_clean":"2024-05-08T18:05:30.858040+0000","last_became_active":"2024-05-08T18:04:53.840395+0000","last_became_peered":"2024-05-08T18:04:53.840395+0000","last_unstale":"2024-05-08T18:05:30.858040+0000","last_undegraded":"2024-05-08T18:05:30.858040+0000","last_fullsized":"2024-05-08T18:05:30.858040+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:54.850231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:33.732213+0000","last_change":"2024-05-08T18:05:30.876470+0000","last_active":"2024-05-08T18:05:33.732213+0000","last_peered":"2024-05-08T18:05:33.732213+0000","last_clean":"2024-05-08T18:05:33.732213+0000","last_became_active":"2024-05-08T18:05:30.869162+0000","last_became_peered":"2024-05-08T18:05:30.869162+0000","last_unstale":"2024-05-08T18:05:33.732213+0000","last_undegraded":"2024-05-08T18:05:33.732213+0000","last_fullsized":"2024-05-08T18:05:33.732213+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:04.889530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":45,"num_write":24,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857600+0000","last_change":"2024-05-08T18:04:43.844847+0000","last_active":"2024-05-08T18:05:30.857600+0000","last_peered":"2024-05-08T18:05:30.857600+0000","last_clean":"2024-05-08T18:05:30.857600+0000","last_became_active":"2024-05-08T18:04:43.838666+0000","last_became_peered":"2024-05-08T18:04:43.838666+0000","last_unstale":"2024-05-08T18:05:30.857600+0000","last_undegraded":"2024-05-08T18:05:30.857600+0000","last_fullsized":"2024-05-08T18:05:30.857600+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:32.496387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861628+0000","last_change":"2024-05-08T18:04:47.819743+0000","last_active":"2024-05-08T18:05:30.861628+0000","last_peered":"2024-05-08T18:05:30.861628+0000","last_clean":"2024-05-08T18:05:30.861628+0000","last_became_active":"2024-05-08T18:04:47.819169+0000","last_became_peered":"2024-05-08T18:04:47.819169+0000","last_unstale":"2024-05-08T18:05:30.861628+0000","last_undegraded":"2024-05-08T18:05:30.861628+0000","last_fullsized":"2024-05-08T18:05:30.861628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:15:57.213130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"29'12","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.746941+0000","last_change":"2024-05-08T18:04:49.831839+0000","last_active":"2024-05-08T18:07:13.746941+0000","last_peered":"2024-05-08T18:07:13.746941+0000","last_clean":"2024-05-08T18:07:13.746941+0000","last_became_active":"2024-05-08T18:04:49.831500+0000","last_became_peered":"2024-05-08T18:04:49.831500+0000","last_unstale":"2024-05-08T18:07:13.746941+0000","last_undegraded":"2024-05-08T18:07:13.746941+0000","last_fullsized":"2024-05-08T18:07:13.746941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:19:01.714463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"29'282","reported_seq":381,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903208+0000","last_change":"2024-05-08T18:04:51.849859+0000","last_active":"2024-05-08T18:07:52.903208+0000","last_peered":"2024-05-08T18:07:52.903208+0000","last_clean":"2024-05-08T18:07:52.903208+0000","last_became_active":"2024-05-08T18:04:51.849301+0000","last_became_peered":"2024-05-08T18:04:51.849301+0000","last_unstale":"2024-05-08T18:07:52.903208+0000","last_undegraded":"2024-05-08T18:07:52.903208+0000","last_fullsized":"2024-05-08T18:07:52.903208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:02:26.756919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862433+0000","last_change":"2024-05-08T18:04:53.840922+0000","last_active":"2024-05-08T18:05:30.862433+0000","last_peered":"2024-05-08T18:05:30.862433+0000","last_clean":"2024-05-08T18:05:30.862433+0000","last_became_active":"2024-05-08T18:04:53.840559+0000","last_became_peered":"2024-05-08T18:04:53.840559+0000","last_unstale":"2024-05-08T18:05:30.862433+0000","last_undegraded":"2024-05-08T18:05:30.862433+0000","last_fullsized":"2024-05-08T18:05:30.862433+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:24.814022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"29'6","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.035260+0000","last_change":"2024-05-08T18:04:45.812045+0000","last_active":"2024-05-08T18:07:54.035260+0000","last_peered":"2024-05-08T18:07:54.035260+0000","last_clean":"2024-05-08T18:07:54.035260+0000","last_became_active":"2024-05-08T18:04:45.811570+0000","last_became_peered":"2024-05-08T18:04:45.811570+0000","last_unstale":"2024-05-08T18:07:54.035260+0000","last_undegraded":"2024-05-08T18:07:54.035260+0000","last_fullsized":"2024-05-08T18:07:54.035260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:22:19.585204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.987908+0000","last_change":"2024-05-08T18:04:43.838356+0000","last_active":"2024-05-08T18:07:53.987908+0000","last_peered":"2024-05-08T18:07:53.987908+0000","last_clean":"2024-05-08T18:07:53.987908+0000","last_became_active":"2024-05-08T18:04:43.836410+0000","last_became_peered":"2024-05-08T18:04:43.836410+0000","last_unstale":"2024-05-08T18:07:53.987908+0000","last_undegraded":"2024-05-08T18:07:53.987908+0000","last_fullsized":"2024-05-08T18:07:53.987908+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:59:26.742382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860150+0000","last_change":"2024-05-08T18:04:47.819091+0000","last_active":"2024-05-08T18:05:30.860150+0000","last_peered":"2024-05-08T18:05:30.860150+0000","last_clean":"2024-05-08T18:05:30.860150+0000","last_became_active":"2024-05-08T18:04:47.818797+0000","last_became_peered":"2024-05-08T18:04:47.818797+0000","last_unstale":"2024-05-08T18:05:30.860150+0000","last_undegraded":"2024-05-08T18:05:30.860150+0000","last_fullsized":"2024-05-08T18:05:30.860150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:44.738079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"29'31","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.948644+0000","last_change":"2024-05-08T18:04:49.831750+0000","last_active":"2024-05-08T18:06:51.948644+0000","last_peered":"2024-05-08T18:06:51.948644+0000","last_clean":"2024-05-08T18:06:51.948644+0000","last_became_active":"2024-05-08T18:04:49.831379+0000","last_became_peered":"2024-05-08T18:04:49.831379+0000","last_unstale":"2024-05-08T18:06:51.948644+0000","last_undegraded":"2024-05-08T18:06:51.948644+0000","last_fullsized":"2024-05-08T18:06:51.948644+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:15:59.220647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"29'282","reported_seq":441,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.067044+0000","last_change":"2024-05-08T18:04:51.827294+0000","last_active":"2024-05-08T18:07:54.067044+0000","last_peered":"2024-05-08T18:07:54.067044+0000","last_clean":"2024-05-08T18:07:54.067044+0000","last_became_active":"2024-05-08T18:04:51.826926+0000","last_became_peered":"2024-05-08T18:04:51.826926+0000","last_unstale":"2024-05-08T18:07:54.067044+0000","last_undegraded":"2024-05-08T18:07:54.067044+0000","last_fullsized":"2024-05-08T18:07:54.067044+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:00:59.786865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'6","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858148+0000","last_change":"2024-05-08T18:04:45.827616+0000","last_active":"2024-05-08T18:05:30.858148+0000","last_peered":"2024-05-08T18:05:30.858148+0000","last_clean":"2024-05-08T18:05:30.858148+0000","last_became_active":"2024-05-08T18:04:45.827351+0000","last_became_peered":"2024-05-08T18:04:45.827351+0000","last_unstale":"2024-05-08T18:05:30.858148+0000","last_undegraded":"2024-05-08T18:05:30.858148+0000","last_fullsized":"2024-05-08T18:05:30.858148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:31.853297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866835+0000","last_change":"2024-05-08T18:04:43.839205+0000","last_active":"2024-05-08T18:05:30.866835+0000","last_peered":"2024-05-08T18:05:30.866835+0000","last_clean":"2024-05-08T18:05:30.866835+0000","last_became_active":"2024-05-08T18:04:43.838840+0000","last_became_peered":"2024-05-08T18:04:43.838840+0000","last_unstale":"2024-05-08T18:05:30.866835+0000","last_undegraded":"2024-05-08T18:05:30.866835+0000","last_fullsized":"2024-05-08T18:05:30.866835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:50.043665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,0],"acting":[2,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861049+0000","last_change":"2024-05-08T18:04:47.825571+0000","last_active":"2024-05-08T18:05:30.861049+0000","last_peered":"2024-05-08T18:05:30.861049+0000","last_clean":"2024-05-08T18:05:30.861049+0000","last_became_active":"2024-05-08T18:04:47.825215+0000","last_became_peered":"2024-05-08T18:04:47.825215+0000","last_unstale":"2024-05-08T18:05:30.861049+0000","last_undegraded":"2024-05-08T18:05:30.861049+0000","last_fullsized":"2024-05-08T18:05:30.861049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:47.750530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"29'40","reported_seq":87,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.957612+0000","last_change":"2024-05-08T18:04:49.828257+0000","last_active":"2024-05-08T18:06:51.957612+0000","last_peered":"2024-05-08T18:06:51.957612+0000","last_clean":"2024-05-08T18:06:51.957612+0000","last_became_active":"2024-05-08T18:04:49.827954+0000","last_became_peered":"2024-05-08T18:04:49.827954+0000","last_unstale":"2024-05-08T18:06:51.957612+0000","last_undegraded":"2024-05-08T18:06:51.957612+0000","last_fullsized":"2024-05-08T18:06:51.957612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:43.565272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859378+0000","last_change":"2024-05-08T18:04:51.831489+0000","last_active":"2024-05-08T18:05:30.859378+0000","last_peered":"2024-05-08T18:05:30.859378+0000","last_clean":"2024-05-08T18:05:30.859378+0000","last_became_active":"2024-05-08T18:04:51.831149+0000","last_became_peered":"2024-05-08T18:04:51.831149+0000","last_unstale":"2024-05-08T18:05:30.859378+0000","last_undegraded":"2024-05-08T18:05:30.859378+0000","last_fullsized":"2024-05-08T18:05:30.859378+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:59.859356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"29'5","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.735436+0000","last_change":"2024-05-08T18:04:53.840960+0000","last_active":"2024-05-08T18:06:09.735436+0000","last_peered":"2024-05-08T18:06:09.735436+0000","last_clean":"2024-05-08T18:06:09.735436+0000","last_became_active":"2024-05-08T18:04:53.840652+0000","last_became_peered":"2024-05-08T18:04:53.840652+0000","last_unstale":"2024-05-08T18:06:09.735436+0000","last_undegraded":"2024-05-08T18:06:09.735436+0000","last_fullsized":"2024-05-08T18:06:09.735436+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:02:50.273794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"27'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861806+0000","last_change":"2024-05-08T18:04:45.804412+0000","last_active":"2024-05-08T18:05:30.861806+0000","last_peered":"2024-05-08T18:05:30.861806+0000","last_clean":"2024-05-08T18:05:30.861806+0000","last_became_active":"2024-05-08T18:04:45.804124+0000","last_became_peered":"2024-05-08T18:04:45.804124+0000","last_unstale":"2024-05-08T18:05:30.861806+0000","last_undegraded":"2024-05-08T18:05:30.861806+0000","last_fullsized":"2024-05-08T18:05:30.861806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:08.184602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867328+0000","last_change":"2024-05-08T18:04:43.843177+0000","last_active":"2024-05-08T18:05:30.867328+0000","last_peered":"2024-05-08T18:05:30.867328+0000","last_clean":"2024-05-08T18:05:30.867328+0000","last_became_active":"2024-05-08T18:04:43.842126+0000","last_became_peered":"2024-05-08T18:04:43.842126+0000","last_unstale":"2024-05-08T18:05:30.867328+0000","last_undegraded":"2024-05-08T18:05:30.867328+0000","last_fullsized":"2024-05-08T18:05:30.867328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:41:01.395720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858650+0000","last_change":"2024-05-08T18:04:47.823007+0000","last_active":"2024-05-08T18:05:30.858650+0000","last_peered":"2024-05-08T18:05:30.858650+0000","last_clean":"2024-05-08T18:05:30.858650+0000","last_became_active":"2024-05-08T18:04:47.822688+0000","last_became_peered":"2024-05-08T18:04:47.822688+0000","last_unstale":"2024-05-08T18:05:30.858650+0000","last_undegraded":"2024-05-08T18:05:30.858650+0000","last_fullsized":"2024-05-08T18:05:30.858650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:32.662502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"29'30","reported_seq":72,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.938361+0000","last_change":"2024-05-08T18:04:49.832010+0000","last_active":"2024-05-08T18:06:51.938361+0000","last_peered":"2024-05-08T18:06:51.938361+0000","last_clean":"2024-05-08T18:06:51.938361+0000","last_became_active":"2024-05-08T18:04:49.829892+0000","last_became_peered":"2024-05-08T18:04:49.829892+0000","last_unstale":"2024-05-08T18:06:51.938361+0000","last_undegraded":"2024-05-08T18:06:51.938361+0000","last_fullsized":"2024-05-08T18:06:51.938361+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:13:16.535751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858848+0000","last_change":"2024-05-08T18:04:51.833183+0000","last_active":"2024-05-08T18:05:30.858848+0000","last_peered":"2024-05-08T18:05:30.858848+0000","last_clean":"2024-05-08T18:05:30.858848+0000","last_became_active":"2024-05-08T18:04:51.832810+0000","last_became_peered":"2024-05-08T18:04:51.832810+0000","last_unstale":"2024-05-08T18:05:30.858848+0000","last_undegraded":"2024-05-08T18:05:30.858848+0000","last_fullsized":"2024-05-08T18:05:30.858848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:02.130044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"29'3","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:03.972536+0000","last_change":"2024-05-08T18:04:53.841201+0000","last_active":"2024-05-08T18:06:03.972536+0000","last_peered":"2024-05-08T18:06:03.972536+0000","last_clean":"2024-05-08T18:06:03.972536+0000","last_became_active":"2024-05-08T18:04:53.840894+0000","last_became_peered":"2024-05-08T18:04:53.840894+0000","last_unstale":"2024-05-08T18:06:03.972536+0000","last_undegraded":"2024-05-08T18:06:03.972536+0000","last_fullsized":"2024-05-08T18:06:03.972536+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:22:10.520507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"29'9","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.867093+0000","last_change":"2024-05-08T18:04:45.817096+0000","last_active":"2024-05-08T18:07:53.867093+0000","last_peered":"2024-05-08T18:07:53.867093+0000","last_clean":"2024-05-08T18:07:53.867093+0000","last_became_active":"2024-05-08T18:04:45.816817+0000","last_became_peered":"2024-05-08T18:04:45.816817+0000","last_unstale":"2024-05-08T18:07:53.867093+0000","last_undegraded":"2024-05-08T18:07:53.867093+0000","last_fullsized":"2024-05-08T18:07:53.867093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:48.747375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"29'3","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.984852+0000","last_change":"2024-05-08T18:04:43.844652+0000","last_active":"2024-05-08T18:07:53.984852+0000","last_peered":"2024-05-08T18:07:53.984852+0000","last_clean":"2024-05-08T18:07:53.984852+0000","last_became_active":"2024-05-08T18:04:43.838781+0000","last_became_peered":"2024-05-08T18:04:43.838781+0000","last_unstale":"2024-05-08T18:07:53.984852+0000","last_undegraded":"2024-05-08T18:07:53.984852+0000","last_fullsized":"2024-05-08T18:07:53.984852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:22:17.577945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":668544,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858935+0000","last_change":"2024-05-08T18:04:47.819192+0000","last_active":"2024-05-08T18:05:30.858935+0000","last_peered":"2024-05-08T18:05:30.858935+0000","last_clean":"2024-05-08T18:05:30.858935+0000","last_became_active":"2024-05-08T18:04:47.818871+0000","last_became_peered":"2024-05-08T18:04:47.818871+0000","last_unstale":"2024-05-08T18:05:30.858935+0000","last_undegraded":"2024-05-08T18:05:30.858935+0000","last_fullsized":"2024-05-08T18:05:30.858935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:20.855010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"29'36","reported_seq":81,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.959548+0000","last_change":"2024-05-08T18:04:49.829583+0000","last_active":"2024-05-08T18:06:51.959548+0000","last_peered":"2024-05-08T18:06:51.959548+0000","last_clean":"2024-05-08T18:06:51.959548+0000","last_became_active":"2024-05-08T18:04:49.829270+0000","last_became_peered":"2024-05-08T18:04:49.829270+0000","last_unstale":"2024-05-08T18:06:51.959548+0000","last_undegraded":"2024-05-08T18:06:51.959548+0000","last_fullsized":"2024-05-08T18:06:51.959548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:11:17.784122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"29'282","reported_seq":348,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.181903+0000","last_change":"2024-05-08T18:04:51.845064+0000","last_active":"2024-05-08T18:07:54.181903+0000","last_peered":"2024-05-08T18:07:54.181903+0000","last_clean":"2024-05-08T18:07:54.181903+0000","last_became_active":"2024-05-08T18:04:51.844529+0000","last_became_peered":"2024-05-08T18:04:51.844529+0000","last_unstale":"2024-05-08T18:07:54.181903+0000","last_undegraded":"2024-05-08T18:07:54.181903+0000","last_fullsized":"2024-05-08T18:07:54.181903+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:46.002263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"29'2","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.862719+0000","last_change":"2024-05-08T18:04:53.841062+0000","last_active":"2024-05-08T18:07:53.862719+0000","last_peered":"2024-05-08T18:07:53.862719+0000","last_clean":"2024-05-08T18:07:53.862719+0000","last_became_active":"2024-05-08T18:04:53.840767+0000","last_became_peered":"2024-05-08T18:04:53.840767+0000","last_unstale":"2024-05-08T18:07:53.862719+0000","last_undegraded":"2024-05-08T18:07:53.862719+0000","last_fullsized":"2024-05-08T18:07:53.862719+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:54:38.757755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"29'6","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995942+0000","last_change":"2024-05-08T18:04:45.814276+0000","last_active":"2024-05-08T18:07:53.995942+0000","last_peered":"2024-05-08T18:07:53.995942+0000","last_clean":"2024-05-08T18:07:53.995942+0000","last_became_active":"2024-05-08T18:04:45.813735+0000","last_became_peered":"2024-05-08T18:04:45.813735+0000","last_unstale":"2024-05-08T18:07:53.995942+0000","last_undegraded":"2024-05-08T18:07:53.995942+0000","last_fullsized":"2024-05-08T18:07:53.995942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:18.463624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"27'4","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859087+0000","last_change":"2024-05-08T18:04:43.848140+0000","last_active":"2024-05-08T18:05:30.859087+0000","last_peered":"2024-05-08T18:05:30.859087+0000","last_clean":"2024-05-08T18:05:30.859087+0000","last_became_active":"2024-05-08T18:04:43.839047+0000","last_became_peered":"2024-05-08T18:04:43.839047+0000","last_unstale":"2024-05-08T18:05:30.859087+0000","last_undegraded":"2024-05-08T18:05:30.859087+0000","last_fullsized":"2024-05-08T18:05:30.859087+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:24.945310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858315+0000","last_change":"2024-05-08T18:04:47.819319+0000","last_active":"2024-05-08T18:05:30.858315+0000","last_peered":"2024-05-08T18:05:30.858315+0000","last_clean":"2024-05-08T18:05:30.858315+0000","last_became_active":"2024-05-08T18:04:47.818972+0000","last_became_peered":"2024-05-08T18:04:47.818972+0000","last_unstale":"2024-05-08T18:05:30.858315+0000","last_undegraded":"2024-05-08T18:05:30.858315+0000","last_fullsized":"2024-05-08T18:05:30.858315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:09:51.843752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"29'54","reported_seq":110,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.961509+0000","last_change":"2024-05-08T18:04:49.832104+0000","last_active":"2024-05-08T18:06:51.961509+0000","last_peered":"2024-05-08T18:06:51.961509+0000","last_clean":"2024-05-08T18:06:51.961509+0000","last_became_active":"2024-05-08T18:04:49.830424+0000","last_became_peered":"2024-05-08T18:04:49.830424+0000","last_unstale":"2024-05-08T18:06:51.961509+0000","last_undegraded":"2024-05-08T18:06:51.961509+0000","last_fullsized":"2024-05-08T18:06:51.961509+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:50.646336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858392+0000","last_change":"2024-05-08T18:04:51.830631+0000","last_active":"2024-05-08T18:05:30.858392+0000","last_peered":"2024-05-08T18:05:30.858392+0000","last_clean":"2024-05-08T18:05:30.858392+0000","last_became_active":"2024-05-08T18:04:51.830383+0000","last_became_peered":"2024-05-08T18:04:51.830383+0000","last_unstale":"2024-05-08T18:05:30.858392+0000","last_undegraded":"2024-05-08T18:05:30.858392+0000","last_fullsized":"2024-05-08T18:05:30.858392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:01.176258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"29'20","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.158011+0000","last_change":"2024-05-08T18:04:53.848105+0000","last_active":"2024-05-08T18:07:54.158011+0000","last_peered":"2024-05-08T18:07:54.158011+0000","last_clean":"2024-05-08T18:07:54.158011+0000","last_became_active":"2024-05-08T18:04:53.847842+0000","last_became_peered":"2024-05-08T18:04:53.847842+0000","last_unstale":"2024-05-08T18:07:54.158011+0000","last_undegraded":"2024-05-08T18:07:54.158011+0000","last_fullsized":"2024-05-08T18:07:54.158011+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:03:57.205665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":48,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"29'6","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.064006+0000","last_change":"2024-05-08T18:04:45.821729+0000","last_active":"2024-05-08T18:07:54.064006+0000","last_peered":"2024-05-08T18:07:54.064006+0000","last_clean":"2024-05-08T18:07:54.064006+0000","last_became_active":"2024-05-08T18:04:45.821412+0000","last_became_peered":"2024-05-08T18:04:45.821412+0000","last_unstale":"2024-05-08T18:07:54.064006+0000","last_undegraded":"2024-05-08T18:07:54.064006+0000","last_fullsized":"2024-05-08T18:07:54.064006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:26.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859201+0000","last_change":"2024-05-08T18:04:43.835922+0000","last_active":"2024-05-08T18:05:30.859201+0000","last_peered":"2024-05-08T18:05:30.859201+0000","last_clean":"2024-05-08T18:05:30.859201+0000","last_became_active":"2024-05-08T18:04:43.834723+0000","last_became_peered":"2024-05-08T18:04:43.834723+0000","last_unstale":"2024-05-08T18:05:30.859201+0000","last_undegraded":"2024-05-08T18:05:30.859201+0000","last_fullsized":"2024-05-08T18:05:30.859201+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:19.700373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864185+0000","last_change":"2024-05-08T18:04:47.823171+0000","last_active":"2024-05-08T18:05:30.864185+0000","last_peered":"2024-05-08T18:05:30.864185+0000","last_clean":"2024-05-08T18:05:30.864185+0000","last_became_active":"2024-05-08T18:04:47.822865+0000","last_became_peered":"2024-05-08T18:04:47.822865+0000","last_unstale":"2024-05-08T18:05:30.864185+0000","last_undegraded":"2024-05-08T18:05:30.864185+0000","last_fullsized":"2024-05-08T18:05:30.864185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:22:57.464984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"29'33","reported_seq":76,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.942615+0000","last_change":"2024-05-08T18:04:49.824918+0000","last_active":"2024-05-08T18:06:51.942615+0000","last_peered":"2024-05-08T18:06:51.942615+0000","last_clean":"2024-05-08T18:06:51.942615+0000","last_became_active":"2024-05-08T18:04:49.824610+0000","last_became_peered":"2024-05-08T18:04:49.824610+0000","last_unstale":"2024-05-08T18:06:51.942615+0000","last_undegraded":"2024-05-08T18:06:51.942615+0000","last_fullsized":"2024-05-08T18:06:51.942615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:56:27.132432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864191+0000","last_change":"2024-05-08T18:04:51.849355+0000","last_active":"2024-05-08T18:05:30.864191+0000","last_peered":"2024-05-08T18:05:30.864191+0000","last_clean":"2024-05-08T18:05:30.864191+0000","last_became_active":"2024-05-08T18:04:51.848793+0000","last_became_peered":"2024-05-08T18:04:51.848793+0000","last_unstale":"2024-05-08T18:05:30.864191+0000","last_undegraded":"2024-05-08T18:05:30.864191+0000","last_fullsized":"2024-05-08T18:05:30.864191+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:40:15.186526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"29'5","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744070+0000","last_change":"2024-05-08T18:04:53.849242+0000","last_active":"2024-05-08T18:06:53.744070+0000","last_peered":"2024-05-08T18:06:53.744070+0000","last_clean":"2024-05-08T18:06:53.744070+0000","last_became_active":"2024-05-08T18:04:53.848945+0000","last_became_peered":"2024-05-08T18:04:53.848945+0000","last_unstale":"2024-05-08T18:06:53.744070+0000","last_undegraded":"2024-05-08T18:06:53.744070+0000","last_fullsized":"2024-05-08T18:06:53.744070+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:00.234436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"29'10","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.772802+0000","last_change":"2024-05-08T18:04:45.829691+0000","last_active":"2024-05-08T18:07:53.772802+0000","last_peered":"2024-05-08T18:07:53.772802+0000","last_clean":"2024-05-08T18:07:53.772802+0000","last_became_active":"2024-05-08T18:04:45.829120+0000","last_became_peered":"2024-05-08T18:04:45.829120+0000","last_unstale":"2024-05-08T18:07:53.772802+0000","last_undegraded":"2024-05-08T18:07:53.772802+0000","last_fullsized":"2024-05-08T18:07:53.772802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:29.119368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":57,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859093+0000","last_change":"2024-05-08T18:04:43.835329+0000","last_active":"2024-05-08T18:05:30.859093+0000","last_peered":"2024-05-08T18:05:30.859093+0000","last_clean":"2024-05-08T18:05:30.859093+0000","last_became_active":"2024-05-08T18:04:43.834536+0000","last_became_peered":"2024-05-08T18:04:43.834536+0000","last_unstale":"2024-05-08T18:05:30.859093+0000","last_undegraded":"2024-05-08T18:05:30.859093+0000","last_fullsized":"2024-05-08T18:05:30.859093+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:56.567490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"29'4","reported_seq":226,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.038721+0000","last_change":"2024-05-08T18:04:47.819789+0000","last_active":"2024-05-08T18:06:53.038721+0000","last_peered":"2024-05-08T18:06:53.038721+0000","last_clean":"2024-05-08T18:06:53.038721+0000","last_became_active":"2024-05-08T18:04:47.819085+0000","last_became_peered":"2024-05-08T18:04:47.819085+0000","last_unstale":"2024-05-08T18:06:53.038721+0000","last_undegraded":"2024-05-08T18:06:53.038721+0000","last_fullsized":"2024-05-08T18:06:53.038721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:24.812721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":298,"num_read_kb":298,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"29'36","reported_seq":81,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.967858+0000","last_change":"2024-05-08T18:04:49.825997+0000","last_active":"2024-05-08T18:06:51.967858+0000","last_peered":"2024-05-08T18:06:51.967858+0000","last_clean":"2024-05-08T18:06:51.967858+0000","last_became_active":"2024-05-08T18:04:49.825613+0000","last_became_peered":"2024-05-08T18:04:49.825613+0000","last_unstale":"2024-05-08T18:06:51.967858+0000","last_undegraded":"2024-05-08T18:06:51.967858+0000","last_fullsized":"2024-05-08T18:06:51.967858+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:49.758047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"29'282","reported_seq":431,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.162304+0000","last_change":"2024-05-08T18:04:51.831317+0000","last_active":"2024-05-08T18:07:54.162304+0000","last_peered":"2024-05-08T18:07:54.162304+0000","last_clean":"2024-05-08T18:07:54.162304+0000","last_became_active":"2024-05-08T18:04:51.830924+0000","last_became_peered":"2024-05-08T18:04:51.830924+0000","last_unstale":"2024-05-08T18:07:54.162304+0000","last_undegraded":"2024-05-08T18:07:54.162304+0000","last_fullsized":"2024-05-08T18:07:54.162304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:23.880081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"29'14","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.739659+0000","last_change":"2024-05-08T18:04:53.841349+0000","last_active":"2024-05-08T18:06:58.739659+0000","last_peered":"2024-05-08T18:06:58.739659+0000","last_clean":"2024-05-08T18:06:58.739659+0000","last_became_active":"2024-05-08T18:04:53.840708+0000","last_became_peered":"2024-05-08T18:04:53.840708+0000","last_unstale":"2024-05-08T18:06:58.739659+0000","last_undegraded":"2024-05-08T18:06:58.739659+0000","last_fullsized":"2024-05-08T18:06:58.739659+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:57:33.051338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063870+0000","last_change":"2024-05-08T18:04:45.812281+0000","last_active":"2024-05-08T18:07:54.063870+0000","last_peered":"2024-05-08T18:07:54.063870+0000","last_clean":"2024-05-08T18:07:54.063870+0000","last_became_active":"2024-05-08T18:04:45.811926+0000","last_became_peered":"2024-05-08T18:04:45.811926+0000","last_unstale":"2024-05-08T18:07:54.063870+0000","last_undegraded":"2024-05-08T18:07:54.063870+0000","last_fullsized":"2024-05-08T18:07:54.063870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:26.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"29'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.728795+0000","last_change":"2024-05-08T18:04:43.844418+0000","last_active":"2024-05-08T18:06:09.728795+0000","last_peered":"2024-05-08T18:06:09.728795+0000","last_clean":"2024-05-08T18:06:09.728795+0000","last_became_active":"2024-05-08T18:04:43.843934+0000","last_became_peered":"2024-05-08T18:04:43.843934+0000","last_unstale":"2024-05-08T18:06:09.728795+0000","last_undegraded":"2024-05-08T18:06:09.728795+0000","last_fullsized":"2024-05-08T18:06:09.728795+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:30.825989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"29'3","reported_seq":413,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.049002+0000","last_change":"2024-05-08T18:04:47.818222+0000","last_active":"2024-05-08T18:06:53.049002+0000","last_peered":"2024-05-08T18:06:53.049002+0000","last_clean":"2024-05-08T18:06:53.049002+0000","last_became_active":"2024-05-08T18:04:47.817851+0000","last_became_peered":"2024-05-08T18:04:47.817851+0000","last_unstale":"2024-05-08T18:06:53.049002+0000","last_undegraded":"2024-05-08T18:06:53.049002+0000","last_fullsized":"2024-05-08T18:06:53.049002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:45:55.932454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":381,"num_read_kb":381,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"27'10","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863960+0000","last_change":"2024-05-08T18:04:49.832191+0000","last_active":"2024-05-08T18:05:30.863960+0000","last_peered":"2024-05-08T18:05:30.863960+0000","last_clean":"2024-05-08T18:05:30.863960+0000","last_became_active":"2024-05-08T18:04:49.830714+0000","last_became_peered":"2024-05-08T18:04:49.830714+0000","last_unstale":"2024-05-08T18:05:30.863960+0000","last_undegraded":"2024-05-08T18:05:30.863960+0000","last_fullsized":"2024-05-08T18:05:30.863960+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:14:01.011620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"29'282","reported_seq":468,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.120234+0000","last_change":"2024-05-08T18:04:51.832589+0000","last_active":"2024-05-08T18:07:54.120234+0000","last_peered":"2024-05-08T18:07:54.120234+0000","last_clean":"2024-05-08T18:07:54.120234+0000","last_became_active":"2024-05-08T18:04:51.831508+0000","last_became_peered":"2024-05-08T18:04:51.831508+0000","last_unstale":"2024-05-08T18:07:54.120234+0000","last_undegraded":"2024-05-08T18:07:54.120234+0000","last_fullsized":"2024-05-08T18:07:54.120234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:15.559137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'4","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.179451+0000","last_change":"2024-05-08T18:04:53.840524+0000","last_active":"2024-05-08T18:06:53.179451+0000","last_peered":"2024-05-08T18:06:53.179451+0000","last_clean":"2024-05-08T18:06:53.179451+0000","last_became_active":"2024-05-08T18:04:53.840215+0000","last_became_peered":"2024-05-08T18:04:53.840215+0000","last_unstale":"2024-05-08T18:06:53.179451+0000","last_undegraded":"2024-05-08T18:06:53.179451+0000","last_fullsized":"2024-05-08T18:06:53.179451+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:39.550898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'2","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866626+0000","last_change":"2024-05-08T18:04:45.819074+0000","last_active":"2024-05-08T18:05:30.866626+0000","last_peered":"2024-05-08T18:05:30.866626+0000","last_clean":"2024-05-08T18:05:30.866626+0000","last_became_active":"2024-05-08T18:04:45.818533+0000","last_became_peered":"2024-05-08T18:04:45.818533+0000","last_unstale":"2024-05-08T18:05:30.866626+0000","last_undegraded":"2024-05-08T18:05:30.866626+0000","last_fullsized":"2024-05-08T18:05:30.866626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:34.419882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"29'1","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744105+0000","last_change":"2024-05-08T18:04:43.848496+0000","last_active":"2024-05-08T18:06:53.744105+0000","last_peered":"2024-05-08T18:06:53.744105+0000","last_clean":"2024-05-08T18:06:53.744105+0000","last_became_active":"2024-05-08T18:04:43.846125+0000","last_became_peered":"2024-05-08T18:04:43.846125+0000","last_unstale":"2024-05-08T18:06:53.744105+0000","last_undegraded":"2024-05-08T18:06:53.744105+0000","last_fullsized":"2024-05-08T18:06:53.744105+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:32.125079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861388+0000","last_change":"2024-05-08T18:04:47.822964+0000","last_active":"2024-05-08T18:05:30.861388+0000","last_peered":"2024-05-08T18:05:30.861388+0000","last_clean":"2024-05-08T18:05:30.861388+0000","last_became_active":"2024-05-08T18:04:47.822242+0000","last_became_peered":"2024-05-08T18:04:47.822242+0000","last_unstale":"2024-05-08T18:05:30.861388+0000","last_undegraded":"2024-05-08T18:05:30.861388+0000","last_fullsized":"2024-05-08T18:05:30.861388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:55.845147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"29'38","reported_seq":84,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.954827+0000","last_change":"2024-05-08T18:04:49.826706+0000","last_active":"2024-05-08T18:06:51.954827+0000","last_peered":"2024-05-08T18:06:51.954827+0000","last_clean":"2024-05-08T18:06:51.954827+0000","last_became_active":"2024-05-08T18:04:49.825314+0000","last_became_peered":"2024-05-08T18:04:49.825314+0000","last_unstale":"2024-05-08T18:06:51.954827+0000","last_undegraded":"2024-05-08T18:06:51.954827+0000","last_fullsized":"2024-05-08T18:06:51.954827+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:21:18.445670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862981+0000","last_change":"2024-05-08T18:04:51.833093+0000","last_active":"2024-05-08T18:05:30.862981+0000","last_peered":"2024-05-08T18:05:30.862981+0000","last_clean":"2024-05-08T18:05:30.862981+0000","last_became_active":"2024-05-08T18:04:51.832808+0000","last_became_peered":"2024-05-08T18:04:51.832808+0000","last_unstale":"2024-05-08T18:05:30.862981+0000","last_undegraded":"2024-05-08T18:05:30.862981+0000","last_fullsized":"2024-05-08T18:05:30.862981+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:54.955755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"29'8","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.096584+0000","last_change":"2024-05-08T18:04:53.837583+0000","last_active":"2024-05-08T18:07:54.096584+0000","last_peered":"2024-05-08T18:07:54.096584+0000","last_clean":"2024-05-08T18:07:54.096584+0000","last_became_active":"2024-05-08T18:04:53.837292+0000","last_became_peered":"2024-05-08T18:04:53.837292+0000","last_unstale":"2024-05-08T18:07:54.096584+0000","last_undegraded":"2024-05-08T18:07:54.096584+0000","last_fullsized":"2024-05-08T18:07:54.096584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:15.652477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'4","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.035838+0000","last_change":"2024-05-08T18:04:45.829568+0000","last_active":"2024-05-08T18:07:54.035838+0000","last_peered":"2024-05-08T18:07:54.035838+0000","last_clean":"2024-05-08T18:07:54.035838+0000","last_became_active":"2024-05-08T18:04:45.828674+0000","last_became_peered":"2024-05-08T18:04:45.828674+0000","last_unstale":"2024-05-08T18:07:54.035838+0000","last_undegraded":"2024-05-08T18:07:54.035838+0000","last_fullsized":"2024-05-08T18:07:54.035838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:33.440280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":137,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859322+0000","last_change":"2024-05-08T18:04:35.552002+0000","last_active":"2024-05-08T18:05:30.859322+0000","last_peered":"2024-05-08T18:05:30.859322+0000","last_clean":"2024-05-08T18:05:30.859322+0000","last_became_active":"2024-05-08T18:04:35.551437+0000","last_became_peered":"2024-05-08T18:04:35.551437+0000","last_unstale":"2024-05-08T18:05:30.859322+0000","last_undegraded":"2024-05-08T18:05:30.859322+0000","last_fullsized":"2024-05-08T18:05:30.859322+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:58.749794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859862+0000","last_change":"2024-05-08T18:04:35.552341+0000","last_active":"2024-05-08T18:05:30.859862+0000","last_peered":"2024-05-08T18:05:30.859862+0000","last_clean":"2024-05-08T18:05:30.859862+0000","last_became_active":"2024-05-08T18:04:35.552017+0000","last_became_peered":"2024-05-08T18:04:35.552017+0000","last_unstale":"2024-05-08T18:05:30.859862+0000","last_undegraded":"2024-05-08T18:05:30.859862+0000","last_fullsized":"2024-05-08T18:05:30.859862+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:30.242269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858503+0000","last_change":"2024-05-08T18:04:43.837405+0000","last_active":"2024-05-08T18:05:30.858503+0000","last_peered":"2024-05-08T18:05:30.858503+0000","last_clean":"2024-05-08T18:05:30.858503+0000","last_became_active":"2024-05-08T18:04:43.837019+0000","last_became_peered":"2024-05-08T18:04:43.837019+0000","last_unstale":"2024-05-08T18:05:30.858503+0000","last_undegraded":"2024-05-08T18:05:30.858503+0000","last_fullsized":"2024-05-08T18:05:30.858503+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:42:03.247527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.871817+0000","last_change":"2024-05-08T18:05:30.871131+0000","last_active":"2024-05-08T18:05:30.871817+0000","last_peered":"2024-05-08T18:05:30.871817+0000","last_clean":"2024-05-08T18:05:30.871817+0000","last_became_active":"2024-05-08T18:05:30.870647+0000","last_became_peered":"2024-05-08T18:05:30.870647+0000","last_unstale":"2024-05-08T18:05:30.871817+0000","last_undegraded":"2024-05-08T18:05:30.871817+0000","last_fullsized":"2024-05-08T18:05:30.871817+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:48.638819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"29'77","reported_seq":146,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.965816+0000","last_change":"2024-05-08T18:04:49.832374+0000","last_active":"2024-05-08T18:06:51.965816+0000","last_peered":"2024-05-08T18:06:51.965816+0000","last_clean":"2024-05-08T18:06:51.965816+0000","last_became_active":"2024-05-08T18:04:49.830636+0000","last_became_peered":"2024-05-08T18:04:49.830636+0000","last_unstale":"2024-05-08T18:06:51.965816+0000","last_undegraded":"2024-05-08T18:06:51.965816+0000","last_fullsized":"2024-05-08T18:06:51.965816+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:23:12.330689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859612+0000","last_change":"2024-05-08T18:04:51.845249+0000","last_active":"2024-05-08T18:05:30.859612+0000","last_peered":"2024-05-08T18:05:30.859612+0000","last_clean":"2024-05-08T18:05:30.859612+0000","last_became_active":"2024-05-08T18:04:51.844737+0000","last_became_peered":"2024-05-08T18:04:51.844737+0000","last_unstale":"2024-05-08T18:05:30.859612+0000","last_undegraded":"2024-05-08T18:05:30.859612+0000","last_fullsized":"2024-05-08T18:05:30.859612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:08.922575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"29'85","reported_seq":166,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.117787+0000","last_change":"2024-05-08T18:04:53.839334+0000","last_active":"2024-05-08T18:07:54.117787+0000","last_peered":"2024-05-08T18:07:54.117787+0000","last_clean":"2024-05-08T18:07:54.117787+0000","last_became_active":"2024-05-08T18:04:53.838957+0000","last_became_peered":"2024-05-08T18:04:53.838957+0000","last_unstale":"2024-05-08T18:07:54.117787+0000","last_undegraded":"2024-05-08T18:07:54.117787+0000","last_fullsized":"2024-05-08T18:07:54.117787+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:25:57.828734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":216,"num_read_kb":214,"num_write":167,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"29'9","reported_seq":65,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070594+0000","last_change":"2024-05-08T18:04:45.826087+0000","last_active":"2024-05-08T18:07:54.070594+0000","last_peered":"2024-05-08T18:07:54.070594+0000","last_clean":"2024-05-08T18:07:54.070594+0000","last_became_active":"2024-05-08T18:04:45.825843+0000","last_became_peered":"2024-05-08T18:04:45.825843+0000","last_unstale":"2024-05-08T18:07:54.070594+0000","last_undegraded":"2024-05-08T18:07:54.070594+0000","last_fullsized":"2024-05-08T18:07:54.070594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:21:14.431053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857318+0000","last_change":"2024-05-08T18:04:35.552553+0000","last_active":"2024-05-08T18:05:30.857318+0000","last_peered":"2024-05-08T18:05:30.857318+0000","last_clean":"2024-05-08T18:05:30.857318+0000","last_became_active":"2024-05-08T18:04:35.552214+0000","last_became_peered":"2024-05-08T18:04:35.552214+0000","last_unstale":"2024-05-08T18:05:30.857318+0000","last_undegraded":"2024-05-08T18:05:30.857318+0000","last_fullsized":"2024-05-08T18:05:30.857318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:12:16.476925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858237+0000","last_change":"2024-05-08T18:04:43.821058+0000","last_active":"2024-05-08T18:05:30.858237+0000","last_peered":"2024-05-08T18:05:30.858237+0000","last_clean":"2024-05-08T18:05:30.858237+0000","last_became_active":"2024-05-08T18:04:43.820369+0000","last_became_peered":"2024-05-08T18:04:43.820369+0000","last_unstale":"2024-05-08T18:05:30.858237+0000","last_undegraded":"2024-05-08T18:05:30.858237+0000","last_fullsized":"2024-05-08T18:05:30.858237+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:01:49.620924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.871018+0000","last_change":"2024-05-08T18:05:30.870035+0000","last_active":"2024-05-08T18:05:30.871018+0000","last_peered":"2024-05-08T18:05:30.871018+0000","last_clean":"2024-05-08T18:05:30.871018+0000","last_became_active":"2024-05-08T18:05:30.869578+0000","last_became_peered":"2024-05-08T18:05:30.869578+0000","last_unstale":"2024-05-08T18:05:30.871018+0000","last_undegraded":"2024-05-08T18:05:30.871018+0000","last_fullsized":"2024-05-08T18:05:30.871018+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:51:48.737463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"29'273","reported_seq":476,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903545+0000","last_change":"2024-05-08T18:04:49.833379+0000","last_active":"2024-05-08T18:07:52.903545+0000","last_peered":"2024-05-08T18:07:52.903545+0000","last_clean":"2024-05-08T18:07:52.903545+0000","last_became_active":"2024-05-08T18:04:49.833053+0000","last_became_peered":"2024-05-08T18:04:49.833053+0000","last_unstale":"2024-05-08T18:07:52.903545+0000","last_undegraded":"2024-05-08T18:07:52.903545+0000","last_fullsized":"2024-05-08T18:07:52.903545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:21.873389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":357,"num_read_kb":342,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859300+0000","last_change":"2024-05-08T18:04:51.849383+0000","last_active":"2024-05-08T18:05:30.859300+0000","last_peered":"2024-05-08T18:05:30.859300+0000","last_clean":"2024-05-08T18:05:30.859300+0000","last_became_active":"2024-05-08T18:04:51.848066+0000","last_became_peered":"2024-05-08T18:04:51.848066+0000","last_unstale":"2024-05-08T18:05:30.859300+0000","last_undegraded":"2024-05-08T18:05:30.859300+0000","last_fullsized":"2024-05-08T18:05:30.859300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:21.450184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"29'3","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.550250+0000","last_change":"2024-05-08T18:04:53.843187+0000","last_active":"2024-05-08T18:06:09.550250+0000","last_peered":"2024-05-08T18:06:09.550250+0000","last_clean":"2024-05-08T18:06:09.550250+0000","last_became_active":"2024-05-08T18:04:53.842899+0000","last_became_peered":"2024-05-08T18:04:53.842899+0000","last_unstale":"2024-05-08T18:06:09.550250+0000","last_undegraded":"2024-05-08T18:06:09.550250+0000","last_fullsized":"2024-05-08T18:06:09.550250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:05:40.879119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"29'8","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063713+0000","last_change":"2024-05-08T18:04:45.812417+0000","last_active":"2024-05-08T18:07:54.063713+0000","last_peered":"2024-05-08T18:07:54.063713+0000","last_clean":"2024-05-08T18:07:54.063713+0000","last_became_active":"2024-05-08T18:04:45.812091+0000","last_became_peered":"2024-05-08T18:04:45.812091+0000","last_unstale":"2024-05-08T18:07:54.063713+0000","last_undegraded":"2024-05-08T18:07:54.063713+0000","last_fullsized":"2024-05-08T18:07:54.063713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:26.952570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863622+0000","last_change":"2024-05-08T18:04:35.551648+0000","last_active":"2024-05-08T18:05:30.863622+0000","last_peered":"2024-05-08T18:05:30.863622+0000","last_clean":"2024-05-08T18:05:30.863622+0000","last_became_active":"2024-05-08T18:04:35.551313+0000","last_became_peered":"2024-05-08T18:04:35.551313+0000","last_unstale":"2024-05-08T18:05:30.863622+0000","last_undegraded":"2024-05-08T18:05:30.863622+0000","last_fullsized":"2024-05-08T18:05:30.863622+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:34:32.994346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858040+0000","last_change":"2024-05-08T18:04:43.821180+0000","last_active":"2024-05-08T18:05:30.858040+0000","last_peered":"2024-05-08T18:05:30.858040+0000","last_clean":"2024-05-08T18:05:30.858040+0000","last_became_active":"2024-05-08T18:04:43.820335+0000","last_became_peered":"2024-05-08T18:04:43.820335+0000","last_unstale":"2024-05-08T18:05:30.858040+0000","last_undegraded":"2024-05-08T18:05:30.858040+0000","last_fullsized":"2024-05-08T18:05:30.858040+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:31.909506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"20'1","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866981+0000","last_change":"2024-05-08T18:04:47.824034+0000","last_active":"2024-05-08T18:05:30.866981+0000","last_peered":"2024-05-08T18:05:30.866981+0000","last_clean":"2024-05-08T18:05:30.866981+0000","last_became_active":"2024-05-08T18:04:47.823631+0000","last_became_peered":"2024-05-08T18:04:47.823631+0000","last_unstale":"2024-05-08T18:05:30.866981+0000","last_undegraded":"2024-05-08T18:05:30.866981+0000","last_fullsized":"2024-05-08T18:05:30.866981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:00.791290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"29'17","reported_seq":85,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903379+0000","last_change":"2024-05-08T18:04:49.828228+0000","last_active":"2024-05-08T18:07:52.903379+0000","last_peered":"2024-05-08T18:07:52.903379+0000","last_clean":"2024-05-08T18:07:52.903379+0000","last_became_active":"2024-05-08T18:04:49.827735+0000","last_became_peered":"2024-05-08T18:04:49.827735+0000","last_unstale":"2024-05-08T18:07:52.903379+0000","last_undegraded":"2024-05-08T18:07:52.903379+0000","last_fullsized":"2024-05-08T18:07:52.903379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:04:17.239231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861148+0000","last_change":"2024-05-08T18:04:51.844276+0000","last_active":"2024-05-08T18:05:30.861148+0000","last_peered":"2024-05-08T18:05:30.861148+0000","last_clean":"2024-05-08T18:05:30.861148+0000","last_became_active":"2024-05-08T18:04:51.843940+0000","last_became_peered":"2024-05-08T18:04:51.843940+0000","last_unstale":"2024-05-08T18:05:30.861148+0000","last_undegraded":"2024-05-08T18:05:30.861148+0000","last_fullsized":"2024-05-08T18:05:30.861148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:03:10.170093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"29'4","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.791011+0000","last_change":"2024-05-08T18:04:53.848676+0000","last_active":"2024-05-08T18:07:53.791011+0000","last_peered":"2024-05-08T18:07:53.791011+0000","last_clean":"2024-05-08T18:07:53.791011+0000","last_became_active":"2024-05-08T18:04:53.848255+0000","last_became_peered":"2024-05-08T18:04:53.848255+0000","last_unstale":"2024-05-08T18:07:53.791011+0000","last_undegraded":"2024-05-08T18:07:53.791011+0000","last_fullsized":"2024-05-08T18:07:53.791011+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:34:38.766674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"29'10","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063735+0000","last_change":"2024-05-08T18:04:45.807470+0000","last_active":"2024-05-08T18:07:54.063735+0000","last_peered":"2024-05-08T18:07:54.063735+0000","last_clean":"2024-05-08T18:07:54.063735+0000","last_became_active":"2024-05-08T18:04:45.807085+0000","last_became_peered":"2024-05-08T18:04:45.807085+0000","last_unstale":"2024-05-08T18:07:54.063735+0000","last_undegraded":"2024-05-08T18:07:54.063735+0000","last_fullsized":"2024-05-08T18:07:54.063735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:05.828431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857751+0000","last_change":"2024-05-08T18:04:35.551456+0000","last_active":"2024-05-08T18:05:30.857751+0000","last_peered":"2024-05-08T18:05:30.857751+0000","last_clean":"2024-05-08T18:05:30.857751+0000","last_became_active":"2024-05-08T18:04:35.551120+0000","last_became_peered":"2024-05-08T18:04:35.551120+0000","last_unstale":"2024-05-08T18:05:30.857751+0000","last_undegraded":"2024-05-08T18:05:30.857751+0000","last_fullsized":"2024-05-08T18:05:30.857751+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:14.181756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864168+0000","last_change":"2024-05-08T18:04:43.840439+0000","last_active":"2024-05-08T18:05:30.864168+0000","last_peered":"2024-05-08T18:05:30.864168+0000","last_clean":"2024-05-08T18:05:30.864168+0000","last_became_active":"2024-05-08T18:04:43.840036+0000","last_became_peered":"2024-05-08T18:04:43.840036+0000","last_unstale":"2024-05-08T18:05:30.864168+0000","last_undegraded":"2024-05-08T18:05:30.864168+0000","last_fullsized":"2024-05-08T18:05:30.864168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:31.909506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0,3],"acting":[2,1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859629+0000","last_change":"2024-05-08T18:04:47.823507+0000","last_active":"2024-05-08T18:05:30.859629+0000","last_peered":"2024-05-08T18:05:30.859629+0000","last_clean":"2024-05-08T18:05:30.859629+0000","last_became_active":"2024-05-08T18:04:47.823093+0000","last_became_peered":"2024-05-08T18:04:47.823093+0000","last_unstale":"2024-05-08T18:05:30.859629+0000","last_undegraded":"2024-05-08T18:05:30.859629+0000","last_fullsized":"2024-05-08T18:05:30.859629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:59.706946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"29'13","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.412397+0000","last_change":"2024-05-08T18:04:49.832657+0000","last_active":"2024-05-08T18:07:53.412397+0000","last_peered":"2024-05-08T18:07:53.412397+0000","last_clean":"2024-05-08T18:07:53.412397+0000","last_became_active":"2024-05-08T18:04:49.830586+0000","last_became_peered":"2024-05-08T18:04:49.830586+0000","last_unstale":"2024-05-08T18:07:53.412397+0000","last_undegraded":"2024-05-08T18:07:53.412397+0000","last_fullsized":"2024-05-08T18:07:53.412397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:01:11.727041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864596+0000","last_change":"2024-05-08T18:04:51.849539+0000","last_active":"2024-05-08T18:05:30.864596+0000","last_peered":"2024-05-08T18:05:30.864596+0000","last_clean":"2024-05-08T18:05:30.864596+0000","last_became_active":"2024-05-08T18:04:51.849137+0000","last_became_peered":"2024-05-08T18:04:51.849137+0000","last_unstale":"2024-05-08T18:05:30.864596+0000","last_undegraded":"2024-05-08T18:05:30.864596+0000","last_fullsized":"2024-05-08T18:05:30.864596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:05.655002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'4","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.159086+0000","last_change":"2024-05-08T18:04:53.851345+0000","last_active":"2024-05-08T18:06:53.159086+0000","last_peered":"2024-05-08T18:06:53.159086+0000","last_clean":"2024-05-08T18:06:53.159086+0000","last_became_active":"2024-05-08T18:04:53.851008+0000","last_became_peered":"2024-05-08T18:04:53.851008+0000","last_unstale":"2024-05-08T18:06:53.159086+0000","last_undegraded":"2024-05-08T18:06:53.159086+0000","last_fullsized":"2024-05-08T18:06:53.159086+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:43:02.228949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.865362+0000","last_change":"2024-05-08T18:04:45.820976+0000","last_active":"2024-05-08T18:05:30.865362+0000","last_peered":"2024-05-08T18:05:30.865362+0000","last_clean":"2024-05-08T18:05:30.865362+0000","last_became_active":"2024-05-08T18:04:45.819326+0000","last_became_peered":"2024-05-08T18:04:45.819326+0000","last_unstale":"2024-05-08T18:05:30.865362+0000","last_undegraded":"2024-05-08T18:05:30.865362+0000","last_fullsized":"2024-05-08T18:05:30.865362+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:22:19.585204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862814+0000","last_change":"2024-05-08T18:04:43.840661+0000","last_active":"2024-05-08T18:05:30.862814+0000","last_peered":"2024-05-08T18:05:30.862814+0000","last_clean":"2024-05-08T18:05:30.862814+0000","last_became_active":"2024-05-08T18:04:43.840236+0000","last_became_peered":"2024-05-08T18:04:43.840236+0000","last_unstale":"2024-05-08T18:05:30.862814+0000","last_undegraded":"2024-05-08T18:05:30.862814+0000","last_fullsized":"2024-05-08T18:05:30.862814+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:40:40.407240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859953+0000","last_change":"2024-05-08T18:04:35.552512+0000","last_active":"2024-05-08T18:05:30.859953+0000","last_peered":"2024-05-08T18:05:30.859953+0000","last_clean":"2024-05-08T18:05:30.859953+0000","last_became_active":"2024-05-08T18:04:35.552173+0000","last_became_peered":"2024-05-08T18:04:35.552173+0000","last_unstale":"2024-05-08T18:05:30.859953+0000","last_undegraded":"2024-05-08T18:05:30.859953+0000","last_fullsized":"2024-05-08T18:05:30.859953+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:46:49.443150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"29'1","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.738959+0000","last_change":"2024-05-08T18:04:47.820423+0000","last_active":"2024-05-08T18:06:53.738959+0000","last_peered":"2024-05-08T18:06:53.738959+0000","last_clean":"2024-05-08T18:06:53.738959+0000","last_became_active":"2024-05-08T18:04:47.819910+0000","last_became_peered":"2024-05-08T18:04:47.819910+0000","last_unstale":"2024-05-08T18:06:53.738959+0000","last_undegraded":"2024-05-08T18:06:53.738959+0000","last_fullsized":"2024-05-08T18:06:53.738959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:41:39.241804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"29'61","reported_seq":120,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.738898+0000","last_change":"2024-05-08T18:04:49.831684+0000","last_active":"2024-05-08T18:06:53.738898+0000","last_peered":"2024-05-08T18:06:53.738898+0000","last_clean":"2024-05-08T18:06:53.738898+0000","last_became_active":"2024-05-08T18:04:49.831360+0000","last_became_peered":"2024-05-08T18:04:49.831360+0000","last_unstale":"2024-05-08T18:06:53.738898+0000","last_undegraded":"2024-05-08T18:06:53.738898+0000","last_fullsized":"2024-05-08T18:06:53.738898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:33:52.278907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":59,"num_read_kb":47,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859470+0000","last_change":"2024-05-08T18:04:51.836834+0000","last_active":"2024-05-08T18:05:30.859470+0000","last_peered":"2024-05-08T18:05:30.859470+0000","last_clean":"2024-05-08T18:05:30.859470+0000","last_became_active":"2024-05-08T18:04:51.836441+0000","last_became_peered":"2024-05-08T18:04:51.836441+0000","last_unstale":"2024-05-08T18:05:30.859470+0000","last_undegraded":"2024-05-08T18:05:30.859470+0000","last_fullsized":"2024-05-08T18:05:30.859470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:42.109569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"29'9","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.337708+0000","last_change":"2024-05-08T18:04:53.838192+0000","last_active":"2024-05-08T18:06:53.337708+0000","last_peered":"2024-05-08T18:06:53.337708+0000","last_clean":"2024-05-08T18:06:53.337708+0000","last_became_active":"2024-05-08T18:04:53.837920+0000","last_became_peered":"2024-05-08T18:04:53.837920+0000","last_unstale":"2024-05-08T18:06:53.337708+0000","last_undegraded":"2024-05-08T18:06:53.337708+0000","last_fullsized":"2024-05-08T18:06:53.337708+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:53.793862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"29'15","reported_seq":77,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.042063+0000","last_change":"2024-05-08T18:04:45.821636+0000","last_active":"2024-05-08T18:07:54.042063+0000","last_peered":"2024-05-08T18:07:54.042063+0000","last_clean":"2024-05-08T18:07:54.042063+0000","last_became_active":"2024-05-08T18:04:45.821107+0000","last_became_peered":"2024-05-08T18:04:45.821107+0000","last_unstale":"2024-05-08T18:07:54.042063+0000","last_undegraded":"2024-05-08T18:07:54.042063+0000","last_fullsized":"2024-05-08T18:07:54.042063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:22:55.457884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861385+0000","last_change":"2024-05-08T18:04:43.828309+0000","last_active":"2024-05-08T18:05:30.861385+0000","last_peered":"2024-05-08T18:05:30.861385+0000","last_clean":"2024-05-08T18:05:30.861385+0000","last_became_active":"2024-05-08T18:04:43.827920+0000","last_became_peered":"2024-05-08T18:04:43.827920+0000","last_unstale":"2024-05-08T18:05:30.861385+0000","last_undegraded":"2024-05-08T18:05:30.861385+0000","last_fullsized":"2024-05-08T18:05:30.861385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:41:01.395720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860131+0000","last_change":"2024-05-08T18:04:35.551253+0000","last_active":"2024-05-08T18:05:30.860131+0000","last_peered":"2024-05-08T18:05:30.860131+0000","last_clean":"2024-05-08T18:05:30.860131+0000","last_became_active":"2024-05-08T18:04:35.550912+0000","last_became_peered":"2024-05-08T18:04:35.550912+0000","last_unstale":"2024-05-08T18:05:30.860131+0000","last_undegraded":"2024-05-08T18:05:30.860131+0000","last_fullsized":"2024-05-08T18:05:30.860131+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:49.303072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857486+0000","last_change":"2024-05-08T18:04:47.814015+0000","last_active":"2024-05-08T18:05:30.857486+0000","last_peered":"2024-05-08T18:05:30.857486+0000","last_clean":"2024-05-08T18:05:30.857486+0000","last_became_active":"2024-05-08T18:04:47.813717+0000","last_became_peered":"2024-05-08T18:04:47.813717+0000","last_unstale":"2024-05-08T18:05:30.857486+0000","last_undegraded":"2024-05-08T18:05:30.857486+0000","last_fullsized":"2024-05-08T18:05:30.857486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:41.988054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"27'14","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861597+0000","last_change":"2024-05-08T18:04:49.835869+0000","last_active":"2024-05-08T18:05:30.861597+0000","last_peered":"2024-05-08T18:05:30.861597+0000","last_clean":"2024-05-08T18:05:30.861597+0000","last_became_active":"2024-05-08T18:04:49.835451+0000","last_became_peered":"2024-05-08T18:04:49.835451+0000","last_unstale":"2024-05-08T18:05:30.861597+0000","last_undegraded":"2024-05-08T18:05:30.861597+0000","last_fullsized":"2024-05-08T18:05:30.861597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:33.133985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"29'282","reported_seq":369,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903220+0000","last_change":"2024-05-08T18:04:51.832417+0000","last_active":"2024-05-08T18:07:52.903220+0000","last_peered":"2024-05-08T18:07:52.903220+0000","last_clean":"2024-05-08T18:07:52.903220+0000","last_became_active":"2024-05-08T18:04:51.832109+0000","last_became_peered":"2024-05-08T18:04:51.832109+0000","last_unstale":"2024-05-08T18:07:52.903220+0000","last_undegraded":"2024-05-08T18:07:52.903220+0000","last_fullsized":"2024-05-08T18:07:52.903220+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:37.543160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860146+0000","last_change":"2024-05-08T18:04:53.844572+0000","last_active":"2024-05-08T18:05:30.860146+0000","last_peered":"2024-05-08T18:05:30.860146+0000","last_clean":"2024-05-08T18:05:30.860146+0000","last_became_active":"2024-05-08T18:04:53.842998+0000","last_became_peered":"2024-05-08T18:04:53.842998+0000","last_unstale":"2024-05-08T18:05:30.860146+0000","last_undegraded":"2024-05-08T18:05:30.860146+0000","last_fullsized":"2024-05-08T18:05:30.860146+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:50.643601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'4","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.775877+0000","last_change":"2024-05-08T18:04:45.830059+0000","last_active":"2024-05-08T18:07:53.775877+0000","last_peered":"2024-05-08T18:07:53.775877+0000","last_clean":"2024-05-08T18:07:53.775877+0000","last_became_active":"2024-05-08T18:04:45.829257+0000","last_became_peered":"2024-05-08T18:04:45.829257+0000","last_unstale":"2024-05-08T18:07:53.775877+0000","last_undegraded":"2024-05-08T18:07:53.775877+0000","last_fullsized":"2024-05-08T18:07:53.775877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:10:31.034888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"29'3","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.536792+0000","last_change":"2024-05-08T18:04:43.835453+0000","last_active":"2024-05-08T18:06:09.536792+0000","last_peered":"2024-05-08T18:06:09.536792+0000","last_clean":"2024-05-08T18:06:09.536792+0000","last_became_active":"2024-05-08T18:04:43.834610+0000","last_became_peered":"2024-05-08T18:04:43.834610+0000","last_unstale":"2024-05-08T18:06:09.536792+0000","last_undegraded":"2024-05-08T18:06:09.536792+0000","last_fullsized":"2024-05-08T18:06:09.536792+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:03.821172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859946+0000","last_change":"2024-05-08T18:04:35.551595+0000","last_active":"2024-05-08T18:05:30.859946+0000","last_peered":"2024-05-08T18:05:30.859946+0000","last_clean":"2024-05-08T18:05:30.859946+0000","last_became_active":"2024-05-08T18:04:35.551366+0000","last_became_peered":"2024-05-08T18:04:35.551366+0000","last_unstale":"2024-05-08T18:05:30.859946+0000","last_undegraded":"2024-05-08T18:05:30.859946+0000","last_fullsized":"2024-05-08T18:05:30.859946+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:00.951608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857461+0000","last_change":"2024-05-08T18:04:47.822209+0000","last_active":"2024-05-08T18:05:30.857461+0000","last_peered":"2024-05-08T18:05:30.857461+0000","last_clean":"2024-05-08T18:05:30.857461+0000","last_became_active":"2024-05-08T18:04:47.819866+0000","last_became_peered":"2024-05-08T18:04:47.819866+0000","last_unstale":"2024-05-08T18:05:30.857461+0000","last_undegraded":"2024-05-08T18:05:30.857461+0000","last_fullsized":"2024-05-08T18:05:30.857461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:28.233298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"27'10","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858934+0000","last_change":"2024-05-08T18:04:49.832338+0000","last_active":"2024-05-08T18:05:30.858934+0000","last_peered":"2024-05-08T18:05:30.858934+0000","last_clean":"2024-05-08T18:05:30.858934+0000","last_became_active":"2024-05-08T18:04:49.830452+0000","last_became_peered":"2024-05-08T18:04:49.830452+0000","last_unstale":"2024-05-08T18:05:30.858934+0000","last_undegraded":"2024-05-08T18:05:30.858934+0000","last_fullsized":"2024-05-08T18:05:30.858934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:49.779432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"29'282","reported_seq":399,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.172497+0000","last_change":"2024-05-08T18:04:51.831530+0000","last_active":"2024-05-08T18:07:54.172497+0000","last_peered":"2024-05-08T18:07:54.172497+0000","last_clean":"2024-05-08T18:07:54.172497+0000","last_became_active":"2024-05-08T18:04:51.831155+0000","last_became_peered":"2024-05-08T18:04:51.831155+0000","last_unstale":"2024-05-08T18:07:54.172497+0000","last_undegraded":"2024-05-08T18:07:54.172497+0000","last_fullsized":"2024-05-08T18:07:54.172497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:32.247507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'5","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.023861+0000","last_change":"2024-05-08T18:04:53.849164+0000","last_active":"2024-05-08T18:07:54.023861+0000","last_peered":"2024-05-08T18:07:54.023861+0000","last_clean":"2024-05-08T18:07:54.023861+0000","last_became_active":"2024-05-08T18:04:53.848862+0000","last_became_peered":"2024-05-08T18:04:53.848862+0000","last_unstale":"2024-05-08T18:07:54.023861+0000","last_undegraded":"2024-05-08T18:07:54.023861+0000","last_fullsized":"2024-05-08T18:07:54.023861+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:33:19.797257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"29'6","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.867036+0000","last_change":"2024-05-08T18:04:45.823169+0000","last_active":"2024-05-08T18:07:53.867036+0000","last_peered":"2024-05-08T18:07:53.867036+0000","last_clean":"2024-05-08T18:07:53.867036+0000","last_became_active":"2024-05-08T18:04:45.822824+0000","last_became_peered":"2024-05-08T18:04:45.822824+0000","last_unstale":"2024-05-08T18:07:53.867036+0000","last_undegraded":"2024-05-08T18:07:53.867036+0000","last_fullsized":"2024-05-08T18:07:53.867036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:34.503647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862506+0000","last_change":"2024-05-08T18:04:43.839633+0000","last_active":"2024-05-08T18:05:30.862506+0000","last_peered":"2024-05-08T18:05:30.862506+0000","last_clean":"2024-05-08T18:05:30.862506+0000","last_became_active":"2024-05-08T18:04:43.839240+0000","last_became_peered":"2024-05-08T18:04:43.839240+0000","last_unstale":"2024-05-08T18:05:30.862506+0000","last_undegraded":"2024-05-08T18:05:30.862506+0000","last_fullsized":"2024-05-08T18:05:30.862506+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:31.909506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864842+0000","last_change":"2024-05-08T18:04:35.551765+0000","last_active":"2024-05-08T18:05:30.864842+0000","last_peered":"2024-05-08T18:05:30.864842+0000","last_clean":"2024-05-08T18:05:30.864842+0000","last_became_active":"2024-05-08T18:04:35.551412+0000","last_became_peered":"2024-05-08T18:04:35.551412+0000","last_unstale":"2024-05-08T18:05:30.864842+0000","last_undegraded":"2024-05-08T18:05:30.864842+0000","last_fullsized":"2024-05-08T18:05:30.864842+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:37:38.776558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857179+0000","last_change":"2024-05-08T18:04:47.816250+0000","last_active":"2024-05-08T18:05:30.857179+0000","last_peered":"2024-05-08T18:05:30.857179+0000","last_clean":"2024-05-08T18:05:30.857179+0000","last_became_active":"2024-05-08T18:04:47.815947+0000","last_became_peered":"2024-05-08T18:04:47.815947+0000","last_unstale":"2024-05-08T18:05:30.857179+0000","last_undegraded":"2024-05-08T18:05:30.857179+0000","last_fullsized":"2024-05-08T18:05:30.857179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:16:57.162048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"29'82","reported_seq":154,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.952921+0000","last_change":"2024-05-08T18:04:49.834985+0000","last_active":"2024-05-08T18:06:51.952921+0000","last_peered":"2024-05-08T18:06:51.952921+0000","last_clean":"2024-05-08T18:06:51.952921+0000","last_became_active":"2024-05-08T18:04:49.834586+0000","last_became_peered":"2024-05-08T18:04:49.834586+0000","last_unstale":"2024-05-08T18:06:51.952921+0000","last_undegraded":"2024-05-08T18:06:51.952921+0000","last_fullsized":"2024-05-08T18:06:51.952921+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:26.820238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858181+0000","last_change":"2024-05-08T18:04:51.831135+0000","last_active":"2024-05-08T18:05:30.858181+0000","last_peered":"2024-05-08T18:05:30.858181+0000","last_clean":"2024-05-08T18:05:30.858181+0000","last_became_active":"2024-05-08T18:04:51.830760+0000","last_became_peered":"2024-05-08T18:04:51.830760+0000","last_unstale":"2024-05-08T18:05:30.858181+0000","last_undegraded":"2024-05-08T18:05:30.858181+0000","last_fullsized":"2024-05-08T18:05:30.858181+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:15.559137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.865409+0000","last_change":"2024-05-08T18:04:53.839069+0000","last_active":"2024-05-08T18:05:30.865409+0000","last_peered":"2024-05-08T18:05:30.865409+0000","last_clean":"2024-05-08T18:05:30.865409+0000","last_became_active":"2024-05-08T18:04:53.838600+0000","last_became_peered":"2024-05-08T18:04:53.838600+0000","last_unstale":"2024-05-08T18:05:30.865409+0000","last_undegraded":"2024-05-08T18:05:30.865409+0000","last_fullsized":"2024-05-08T18:05:30.865409+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:17.566875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"27'2","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866690+0000","last_change":"2024-05-08T18:04:53.839793+0000","last_active":"2024-05-08T18:05:30.866690+0000","last_peered":"2024-05-08T18:05:30.866690+0000","last_clean":"2024-05-08T18:05:30.866690+0000","last_became_active":"2024-05-08T18:04:53.839500+0000","last_became_peered":"2024-05-08T18:04:53.839500+0000","last_unstale":"2024-05-08T18:05:30.866690+0000","last_undegraded":"2024-05-08T18:05:30.866690+0000","last_fullsized":"2024-05-08T18:05:30.866690+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:19.575012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"29'8","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.219834+0000","last_change":"2024-05-08T18:04:45.819806+0000","last_active":"2024-05-08T18:06:50.219834+0000","last_peered":"2024-05-08T18:06:50.219834+0000","last_clean":"2024-05-08T18:06:50.219834+0000","last_became_active":"2024-05-08T18:04:45.819508+0000","last_became_peered":"2024-05-08T18:04:45.819508+0000","last_unstale":"2024-05-08T18:06:50.219834+0000","last_undegraded":"2024-05-08T18:06:50.219834+0000","last_fullsized":"2024-05-08T18:06:50.219834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:32.833248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859163+0000","last_change":"2024-05-08T18:04:43.844864+0000","last_active":"2024-05-08T18:05:30.859163+0000","last_peered":"2024-05-08T18:05:30.859163+0000","last_clean":"2024-05-08T18:05:30.859163+0000","last_became_active":"2024-05-08T18:04:43.837929+0000","last_became_peered":"2024-05-08T18:04:43.837929+0000","last_unstale":"2024-05-08T18:05:30.859163+0000","last_undegraded":"2024-05-08T18:05:30.859163+0000","last_fullsized":"2024-05-08T18:05:30.859163+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:06.177343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861102+0000","last_change":"2024-05-08T18:04:47.818763+0000","last_active":"2024-05-08T18:05:30.861102+0000","last_peered":"2024-05-08T18:05:30.861102+0000","last_clean":"2024-05-08T18:05:30.861102+0000","last_became_active":"2024-05-08T18:04:47.818443+0000","last_became_peered":"2024-05-08T18:04:47.818443+0000","last_unstale":"2024-05-08T18:05:30.861102+0000","last_undegraded":"2024-05-08T18:05:30.861102+0000","last_fullsized":"2024-05-08T18:05:30.861102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:15:57.213130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"29'32","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.958612+0000","last_change":"2024-05-08T18:04:49.834233+0000","last_active":"2024-05-08T18:06:51.958612+0000","last_peered":"2024-05-08T18:06:51.958612+0000","last_clean":"2024-05-08T18:06:51.958612+0000","last_became_active":"2024-05-08T18:04:49.833599+0000","last_became_peered":"2024-05-08T18:04:49.833599+0000","last_unstale":"2024-05-08T18:06:51.958612+0000","last_undegraded":"2024-05-08T18:06:51.958612+0000","last_fullsized":"2024-05-08T18:06:51.958612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:54:34.743324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858258+0000","last_change":"2024-05-08T18:04:51.844827+0000","last_active":"2024-05-08T18:05:30.858258+0000","last_peered":"2024-05-08T18:05:30.858258+0000","last_clean":"2024-05-08T18:05:30.858258+0000","last_became_active":"2024-05-08T18:04:51.844526+0000","last_became_peered":"2024-05-08T18:04:51.844526+0000","last_unstale":"2024-05-08T18:05:30.858258+0000","last_undegraded":"2024-05-08T18:05:30.858258+0000","last_fullsized":"2024-05-08T18:05:30.858258+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:13:54.471619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"29'11","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.639339+0000","last_change":"2024-05-08T18:04:53.850911+0000","last_active":"2024-05-08T18:06:51.639339+0000","last_peered":"2024-05-08T18:06:51.639339+0000","last_clean":"2024-05-08T18:06:51.639339+0000","last_became_active":"2024-05-08T18:04:53.850593+0000","last_became_peered":"2024-05-08T18:04:53.850593+0000","last_unstale":"2024-05-08T18:06:51.639339+0000","last_undegraded":"2024-05-08T18:06:51.639339+0000","last_fullsized":"2024-05-08T18:06:51.639339+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:11:21.798553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":62,"num_write":31,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"29'8","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399973+0000","last_change":"2024-05-08T18:04:45.819974+0000","last_active":"2024-05-08T18:06:53.399973+0000","last_peered":"2024-05-08T18:06:53.399973+0000","last_clean":"2024-05-08T18:06:53.399973+0000","last_became_active":"2024-05-08T18:04:45.818979+0000","last_became_peered":"2024-05-08T18:04:45.818979+0000","last_unstale":"2024-05-08T18:06:53.399973+0000","last_undegraded":"2024-05-08T18:06:53.399973+0000","last_fullsized":"2024-05-08T18:06:53.399973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:30.458103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863085+0000","last_change":"2024-05-08T18:04:43.836993+0000","last_active":"2024-05-08T18:05:30.863085+0000","last_peered":"2024-05-08T18:05:30.863085+0000","last_clean":"2024-05-08T18:05:30.863085+0000","last_became_active":"2024-05-08T18:04:43.836614+0000","last_became_peered":"2024-05-08T18:04:43.836614+0000","last_unstale":"2024-05-08T18:05:30.863085+0000","last_undegraded":"2024-05-08T18:05:30.863085+0000","last_fullsized":"2024-05-08T18:05:30.863085+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:05:33.776915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858802+0000","last_change":"2024-05-08T18:04:47.814847+0000","last_active":"2024-05-08T18:05:30.858802+0000","last_peered":"2024-05-08T18:05:30.858802+0000","last_clean":"2024-05-08T18:05:30.858802+0000","last_became_active":"2024-05-08T18:04:47.814561+0000","last_became_peered":"2024-05-08T18:04:47.814561+0000","last_unstale":"2024-05-08T18:05:30.858802+0000","last_undegraded":"2024-05-08T18:05:30.858802+0000","last_fullsized":"2024-05-08T18:05:30.858802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:58.115834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"29'7","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.433454+0000","last_change":"2024-05-08T18:04:49.825623+0000","last_active":"2024-05-08T18:07:53.433454+0000","last_peered":"2024-05-08T18:07:53.433454+0000","last_clean":"2024-05-08T18:07:53.433454+0000","last_became_active":"2024-05-08T18:04:49.825347+0000","last_became_peered":"2024-05-08T18:04:49.825347+0000","last_unstale":"2024-05-08T18:07:53.433454+0000","last_undegraded":"2024-05-08T18:07:53.433454+0000","last_fullsized":"2024-05-08T18:07:53.433454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:37.454897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863344+0000","last_change":"2024-05-08T18:04:51.843388+0000","last_active":"2024-05-08T18:05:30.863344+0000","last_peered":"2024-05-08T18:05:30.863344+0000","last_clean":"2024-05-08T18:05:30.863344+0000","last_became_active":"2024-05-08T18:04:51.843002+0000","last_became_peered":"2024-05-08T18:04:51.843002+0000","last_unstale":"2024-05-08T18:05:30.863344+0000","last_undegraded":"2024-05-08T18:05:30.863344+0000","last_fullsized":"2024-05-08T18:05:30.863344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:13.211841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859114+0000","last_change":"2024-05-08T18:04:53.840844+0000","last_active":"2024-05-08T18:05:30.859114+0000","last_peered":"2024-05-08T18:05:30.859114+0000","last_clean":"2024-05-08T18:05:30.859114+0000","last_became_active":"2024-05-08T18:04:53.840437+0000","last_became_peered":"2024-05-08T18:04:53.840437+0000","last_unstale":"2024-05-08T18:05:30.859114+0000","last_undegraded":"2024-05-08T18:05:30.859114+0000","last_fullsized":"2024-05-08T18:05:30.859114+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:42:52.195643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"29'8","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.861059+0000","last_change":"2024-05-08T18:05:30.868442+0000","last_active":"2024-05-08T18:07:53.861059+0000","last_peered":"2024-05-08T18:07:53.861059+0000","last_clean":"2024-05-08T18:07:53.861059+0000","last_became_active":"2024-05-08T18:05:30.868156+0000","last_became_peered":"2024-05-08T18:05:30.868156+0000","last_unstale":"2024-05-08T18:07:53.861059+0000","last_undegraded":"2024-05-08T18:07:53.861059+0000","last_fullsized":"2024-05-08T18:07:53.861059+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:44.461215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867524+0000","last_change":"2024-05-08T18:04:43.835856+0000","last_active":"2024-05-08T18:05:30.867524+0000","last_peered":"2024-05-08T18:05:30.867524+0000","last_clean":"2024-05-08T18:05:30.867524+0000","last_became_active":"2024-05-08T18:04:43.835435+0000","last_became_peered":"2024-05-08T18:04:43.835435+0000","last_unstale":"2024-05-08T18:05:30.867524+0000","last_undegraded":"2024-05-08T18:05:30.867524+0000","last_fullsized":"2024-05-08T18:05:30.867524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:47.971014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860152+0000","last_change":"2024-05-08T18:04:47.824614+0000","last_active":"2024-05-08T18:05:30.860152+0000","last_peered":"2024-05-08T18:05:30.860152+0000","last_clean":"2024-05-08T18:05:30.860152+0000","last_became_active":"2024-05-08T18:04:47.824140+0000","last_became_peered":"2024-05-08T18:04:47.824140+0000","last_unstale":"2024-05-08T18:05:30.860152+0000","last_undegraded":"2024-05-08T18:05:30.860152+0000","last_fullsized":"2024-05-08T18:05:30.860152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:36:47.466900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"29'41","reported_seq":100,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.004393+0000","last_change":"2024-05-08T18:04:49.824946+0000","last_active":"2024-05-08T18:07:54.004393+0000","last_peered":"2024-05-08T18:07:54.004393+0000","last_clean":"2024-05-08T18:07:54.004393+0000","last_became_active":"2024-05-08T18:04:49.824600+0000","last_became_peered":"2024-05-08T18:04:49.824600+0000","last_unstale":"2024-05-08T18:07:54.004393+0000","last_undegraded":"2024-05-08T18:07:54.004393+0000","last_fullsized":"2024-05-08T18:07:54.004393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:11.205149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":64,"num_write":46,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860821+0000","last_change":"2024-05-08T18:04:51.833599+0000","last_active":"2024-05-08T18:05:30.860821+0000","last_peered":"2024-05-08T18:05:30.860821+0000","last_clean":"2024-05-08T18:05:30.860821+0000","last_became_active":"2024-05-08T18:04:51.829937+0000","last_became_peered":"2024-05-08T18:04:51.829937+0000","last_unstale":"2024-05-08T18:05:30.860821+0000","last_undegraded":"2024-05-08T18:05:30.860821+0000","last_fullsized":"2024-05-08T18:05:30.860821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:02:48.266056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858578+0000","last_change":"2024-05-08T18:04:53.842981+0000","last_active":"2024-05-08T18:05:30.858578+0000","last_peered":"2024-05-08T18:05:30.858578+0000","last_clean":"2024-05-08T18:05:30.858578+0000","last_became_active":"2024-05-08T18:04:53.842610+0000","last_became_peered":"2024-05-08T18:04:53.842610+0000","last_unstale":"2024-05-08T18:05:30.858578+0000","last_undegraded":"2024-05-08T18:05:30.858578+0000","last_fullsized":"2024-05-08T18:05:30.858578+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:16:15.240226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857482+0000","last_change":"2024-05-08T18:04:45.829838+0000","last_active":"2024-05-08T18:05:30.857482+0000","last_peered":"2024-05-08T18:05:30.857482+0000","last_clean":"2024-05-08T18:05:30.857482+0000","last_became_active":"2024-05-08T18:04:45.829458+0000","last_became_peered":"2024-05-08T18:04:45.829458+0000","last_unstale":"2024-05-08T18:05:30.857482+0000","last_undegraded":"2024-05-08T18:05:30.857482+0000","last_fullsized":"2024-05-08T18:05:30.857482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:15:55.206029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"29'4","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.352042+0000","last_change":"2024-05-08T18:04:43.837089+0000","last_active":"2024-05-08T18:06:53.352042+0000","last_peered":"2024-05-08T18:06:53.352042+0000","last_clean":"2024-05-08T18:06:53.352042+0000","last_became_active":"2024-05-08T18:04:43.836633+0000","last_became_peered":"2024-05-08T18:04:43.836633+0000","last_unstale":"2024-05-08T18:06:53.352042+0000","last_undegraded":"2024-05-08T18:06:53.352042+0000","last_fullsized":"2024-05-08T18:06:53.352042+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:55.493870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"20'1","reported_seq":218,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.049308+0000","last_change":"2024-05-08T18:04:47.820110+0000","last_active":"2024-05-08T18:06:53.049308+0000","last_peered":"2024-05-08T18:06:53.049308+0000","last_clean":"2024-05-08T18:06:53.049308+0000","last_became_active":"2024-05-08T18:04:47.819771+0000","last_became_peered":"2024-05-08T18:04:47.819771+0000","last_unstale":"2024-05-08T18:06:53.049308+0000","last_undegraded":"2024-05-08T18:06:53.049308+0000","last_fullsized":"2024-05-08T18:06:53.049308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:42:21.961312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"29'30","reported_seq":72,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.944676+0000","last_change":"2024-05-08T18:04:49.824323+0000","last_active":"2024-05-08T18:06:51.944676+0000","last_peered":"2024-05-08T18:06:51.944676+0000","last_clean":"2024-05-08T18:06:51.944676+0000","last_became_active":"2024-05-08T18:04:49.824022+0000","last_became_peered":"2024-05-08T18:04:49.824022+0000","last_unstale":"2024-05-08T18:06:51.944676+0000","last_undegraded":"2024-05-08T18:06:51.944676+0000","last_fullsized":"2024-05-08T18:06:51.944676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:41:41.249321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866329+0000","last_change":"2024-05-08T18:04:51.826713+0000","last_active":"2024-05-08T18:05:30.866329+0000","last_peered":"2024-05-08T18:05:30.866329+0000","last_clean":"2024-05-08T18:05:30.866329+0000","last_became_active":"2024-05-08T18:04:51.826424+0000","last_became_peered":"2024-05-08T18:04:51.826424+0000","last_unstale":"2024-05-08T18:05:30.866329+0000","last_undegraded":"2024-05-08T18:05:30.866329+0000","last_fullsized":"2024-05-08T18:05:30.866329+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:33:54.285599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"29'20","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995883+0000","last_change":"2024-05-08T18:05:30.873315+0000","last_active":"2024-05-08T18:07:53.995883+0000","last_peered":"2024-05-08T18:07:53.995883+0000","last_clean":"2024-05-08T18:07:53.995883+0000","last_became_active":"2024-05-08T18:05:30.868409+0000","last_became_peered":"2024-05-08T18:05:30.868409+0000","last_unstale":"2024-05-08T18:07:53.995883+0000","last_undegraded":"2024-05-08T18:07:53.995883+0000","last_fullsized":"2024-05-08T18:07:53.995883+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:07.623428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857341+0000","last_change":"2024-05-08T18:04:43.844983+0000","last_active":"2024-05-08T18:05:30.857341+0000","last_peered":"2024-05-08T18:05:30.857341+0000","last_clean":"2024-05-08T18:05:30.857341+0000","last_became_active":"2024-05-08T18:04:43.838957+0000","last_became_peered":"2024-05-08T18:04:43.838957+0000","last_unstale":"2024-05-08T18:05:30.857341+0000","last_undegraded":"2024-05-08T18:05:30.857341+0000","last_fullsized":"2024-05-08T18:05:30.857341+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:42:05.210775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859885+0000","last_change":"2024-05-08T18:04:47.820891+0000","last_active":"2024-05-08T18:05:30.859885+0000","last_peered":"2024-05-08T18:05:30.859885+0000","last_clean":"2024-05-08T18:05:30.859885+0000","last_became_active":"2024-05-08T18:04:47.820256+0000","last_became_peered":"2024-05-08T18:04:47.820256+0000","last_unstale":"2024-05-08T18:05:30.859885+0000","last_undegraded":"2024-05-08T18:05:30.859885+0000","last_fullsized":"2024-05-08T18:05:30.859885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:04:15.231714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"29'54","reported_seq":110,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.964783+0000","last_change":"2024-05-08T18:04:49.825705+0000","last_active":"2024-05-08T18:06:51.964783+0000","last_peered":"2024-05-08T18:06:51.964783+0000","last_clean":"2024-05-08T18:06:51.964783+0000","last_became_active":"2024-05-08T18:04:49.825366+0000","last_became_peered":"2024-05-08T18:04:49.825366+0000","last_unstale":"2024-05-08T18:06:51.964783+0000","last_undegraded":"2024-05-08T18:06:51.964783+0000","last_fullsized":"2024-05-08T18:06:51.964783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:27:27.454431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862900+0000","last_change":"2024-05-08T18:04:51.827568+0000","last_active":"2024-05-08T18:05:30.862900+0000","last_peered":"2024-05-08T18:05:30.862900+0000","last_clean":"2024-05-08T18:05:30.862900+0000","last_became_active":"2024-05-08T18:04:51.827127+0000","last_became_peered":"2024-05-08T18:04:51.827127+0000","last_unstale":"2024-05-08T18:05:30.862900+0000","last_undegraded":"2024-05-08T18:05:30.862900+0000","last_fullsized":"2024-05-08T18:05:30.862900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:27:29.461124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"29'14","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.333941+0000","last_change":"2024-05-08T18:04:53.844087+0000","last_active":"2024-05-08T18:06:53.333941+0000","last_peered":"2024-05-08T18:06:53.333941+0000","last_clean":"2024-05-08T18:06:53.333941+0000","last_became_active":"2024-05-08T18:04:53.843772+0000","last_became_peered":"2024-05-08T18:04:53.843772+0000","last_unstale":"2024-05-08T18:06:53.333941+0000","last_undegraded":"2024-05-08T18:06:53.333941+0000","last_fullsized":"2024-05-08T18:06:53.333941+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:26.083815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.738215+0000","last_change":"2024-05-08T18:04:45.811909+0000","last_active":"2024-05-08T18:06:48.738215+0000","last_peered":"2024-05-08T18:06:48.738215+0000","last_clean":"2024-05-08T18:06:48.738215+0000","last_became_active":"2024-05-08T18:04:45.811517+0000","last_became_peered":"2024-05-08T18:04:45.811517+0000","last_unstale":"2024-05-08T18:06:48.738215+0000","last_undegraded":"2024-05-08T18:06:48.738215+0000","last_fullsized":"2024-05-08T18:06:48.738215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:32.141179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860711+0000","last_change":"2024-05-08T18:04:43.836313+0000","last_active":"2024-05-08T18:05:30.860711+0000","last_peered":"2024-05-08T18:05:30.860711+0000","last_clean":"2024-05-08T18:05:30.860711+0000","last_became_active":"2024-05-08T18:04:43.834767+0000","last_became_peered":"2024-05-08T18:04:43.834767+0000","last_unstale":"2024-05-08T18:05:30.860711+0000","last_undegraded":"2024-05-08T18:05:30.860711+0000","last_fullsized":"2024-05-08T18:05:30.860711+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:22.454546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0,2],"acting":[1,3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867042+0000","last_change":"2024-05-08T18:04:47.821012+0000","last_active":"2024-05-08T18:05:30.867042+0000","last_peered":"2024-05-08T18:05:30.867042+0000","last_clean":"2024-05-08T18:05:30.867042+0000","last_became_active":"2024-05-08T18:04:47.820662+0000","last_became_peered":"2024-05-08T18:04:47.820662+0000","last_unstale":"2024-05-08T18:05:30.867042+0000","last_undegraded":"2024-05-08T18:05:30.867042+0000","last_fullsized":"2024-05-08T18:05:30.867042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:29:01.485039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"29'37","reported_seq":82,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.401221+0000","last_change":"2024-05-08T18:04:49.831871+0000","last_active":"2024-05-08T18:06:53.401221+0000","last_peered":"2024-05-08T18:06:53.401221+0000","last_clean":"2024-05-08T18:06:53.401221+0000","last_became_active":"2024-05-08T18:04:49.830476+0000","last_became_peered":"2024-05-08T18:04:49.830476+0000","last_unstale":"2024-05-08T18:06:53.401221+0000","last_undegraded":"2024-05-08T18:06:53.401221+0000","last_fullsized":"2024-05-08T18:06:53.401221+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:46.223733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862181+0000","last_change":"2024-05-08T18:04:51.845269+0000","last_active":"2024-05-08T18:05:30.862181+0000","last_peered":"2024-05-08T18:05:30.862181+0000","last_clean":"2024-05-08T18:05:30.862181+0000","last_became_active":"2024-05-08T18:04:51.844912+0000","last_became_peered":"2024-05-08T18:04:51.844912+0000","last_unstale":"2024-05-08T18:05:30.862181+0000","last_undegraded":"2024-05-08T18:05:30.862181+0000","last_fullsized":"2024-05-08T18:05:30.862181+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:17.567274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"29'2","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.037376+0000","last_change":"2024-05-08T18:04:53.847736+0000","last_active":"2024-05-08T18:07:54.037376+0000","last_peered":"2024-05-08T18:07:54.037376+0000","last_clean":"2024-05-08T18:07:54.037376+0000","last_became_active":"2024-05-08T18:04:53.847413+0000","last_became_peered":"2024-05-08T18:04:53.847413+0000","last_unstale":"2024-05-08T18:07:54.037376+0000","last_undegraded":"2024-05-08T18:07:54.037376+0000","last_fullsized":"2024-05-08T18:07:54.037376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:18.059582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"29'12","reported_seq":60,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.041935+0000","last_change":"2024-05-08T18:04:45.830619+0000","last_active":"2024-05-08T18:07:54.041935+0000","last_peered":"2024-05-08T18:07:54.041935+0000","last_clean":"2024-05-08T18:07:54.041935+0000","last_became_active":"2024-05-08T18:04:45.830329+0000","last_became_peered":"2024-05-08T18:04:45.830329+0000","last_unstale":"2024-05-08T18:07:54.041935+0000","last_undegraded":"2024-05-08T18:07:54.041935+0000","last_fullsized":"2024-05-08T18:07:54.041935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:13:12.748032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861213+0000","last_change":"2024-05-08T18:04:43.835662+0000","last_active":"2024-05-08T18:05:30.861213+0000","last_peered":"2024-05-08T18:05:30.861213+0000","last_clean":"2024-05-08T18:05:30.861213+0000","last_became_active":"2024-05-08T18:04:43.835227+0000","last_became_peered":"2024-05-08T18:04:43.835227+0000","last_unstale":"2024-05-08T18:05:30.861213+0000","last_undegraded":"2024-05-08T18:05:30.861213+0000","last_fullsized":"2024-05-08T18:05:30.861213+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:10.360847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3,2],"acting":[1,0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.871559+0000","last_change":"2024-05-08T18:05:30.870636+0000","last_active":"2024-05-08T18:05:30.871559+0000","last_peered":"2024-05-08T18:05:30.871559+0000","last_clean":"2024-05-08T18:05:30.871559+0000","last_became_active":"2024-05-08T18:05:30.870155+0000","last_became_peered":"2024-05-08T18:05:30.870155+0000","last_unstale":"2024-05-08T18:05:30.871559+0000","last_undegraded":"2024-05-08T18:05:30.871559+0000","last_fullsized":"2024-05-08T18:05:30.871559+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:56.207001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"29'32","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.962760+0000","last_change":"2024-05-08T18:04:49.832968+0000","last_active":"2024-05-08T18:06:51.962760+0000","last_peered":"2024-05-08T18:06:51.962760+0000","last_clean":"2024-05-08T18:06:51.962760+0000","last_became_active":"2024-05-08T18:04:49.832649+0000","last_became_peered":"2024-05-08T18:04:49.832649+0000","last_unstale":"2024-05-08T18:06:51.962760+0000","last_undegraded":"2024-05-08T18:06:51.962760+0000","last_fullsized":"2024-05-08T18:06:51.962760+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:52.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859450+0000","last_change":"2024-05-08T18:04:51.833712+0000","last_active":"2024-05-08T18:05:30.859450+0000","last_peered":"2024-05-08T18:05:30.859450+0000","last_clean":"2024-05-08T18:05:30.859450+0000","last_became_active":"2024-05-08T18:04:51.833421+0000","last_became_peered":"2024-05-08T18:04:51.833421+0000","last_unstale":"2024-05-08T18:05:30.859450+0000","last_undegraded":"2024-05-08T18:05:30.859450+0000","last_fullsized":"2024-05-08T18:05:30.859450+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:48.752288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"29'5","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.394775+0000","last_change":"2024-05-08T18:04:53.848762+0000","last_active":"2024-05-08T18:06:53.394775+0000","last_peered":"2024-05-08T18:06:53.394775+0000","last_clean":"2024-05-08T18:06:53.394775+0000","last_became_active":"2024-05-08T18:04:53.848430+0000","last_became_peered":"2024-05-08T18:04:53.848430+0000","last_unstale":"2024-05-08T18:06:53.394775+0000","last_undegraded":"2024-05-08T18:06:53.394775+0000","last_fullsized":"2024-05-08T18:06:53.394775+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:48.070547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"29'10","reported_seq":66,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.218980+0000","last_change":"2024-05-08T18:04:45.807312+0000","last_active":"2024-05-08T18:06:50.218980+0000","last_peered":"2024-05-08T18:06:50.218980+0000","last_clean":"2024-05-08T18:06:50.218980+0000","last_became_active":"2024-05-08T18:04:45.806808+0000","last_became_peered":"2024-05-08T18:04:45.806808+0000","last_unstale":"2024-05-08T18:06:50.218980+0000","last_undegraded":"2024-05-08T18:06:50.218980+0000","last_fullsized":"2024-05-08T18:06:50.218980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:27:20.114031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859441+0000","last_change":"2024-05-08T18:04:43.833094+0000","last_active":"2024-05-08T18:05:30.859441+0000","last_peered":"2024-05-08T18:05:30.859441+0000","last_clean":"2024-05-08T18:05:30.859441+0000","last_became_active":"2024-05-08T18:04:43.832147+0000","last_became_peered":"2024-05-08T18:04:43.832147+0000","last_unstale":"2024-05-08T18:05:30.859441+0000","last_undegraded":"2024-05-08T18:05:30.859441+0000","last_fullsized":"2024-05-08T18:05:30.859441+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:49.361708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862397+0000","last_change":"2024-05-08T18:04:47.815796+0000","last_active":"2024-05-08T18:05:30.862397+0000","last_peered":"2024-05-08T18:05:30.862397+0000","last_clean":"2024-05-08T18:05:30.862397+0000","last_became_active":"2024-05-08T18:04:47.815219+0000","last_became_peered":"2024-05-08T18:04:47.815219+0000","last_unstale":"2024-05-08T18:05:30.862397+0000","last_undegraded":"2024-05-08T18:05:30.862397+0000","last_fullsized":"2024-05-08T18:05:30.862397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:23:20.608088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"27'10","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861853+0000","last_change":"2024-05-08T18:04:49.833674+0000","last_active":"2024-05-08T18:05:30.861853+0000","last_peered":"2024-05-08T18:05:30.861853+0000","last_clean":"2024-05-08T18:05:30.861853+0000","last_became_active":"2024-05-08T18:04:49.830793+0000","last_became_peered":"2024-05-08T18:04:49.830793+0000","last_unstale":"2024-05-08T18:05:30.861853+0000","last_undegraded":"2024-05-08T18:05:30.861853+0000","last_fullsized":"2024-05-08T18:05:30.861853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:54:53.163574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"29'282","reported_seq":366,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903209+0000","last_change":"2024-05-08T18:04:51.834224+0000","last_active":"2024-05-08T18:07:52.903209+0000","last_peered":"2024-05-08T18:07:52.903209+0000","last_clean":"2024-05-08T18:07:52.903209+0000","last_became_active":"2024-05-08T18:04:51.833866+0000","last_became_peered":"2024-05-08T18:04:51.833866+0000","last_unstale":"2024-05-08T18:07:52.903209+0000","last_undegraded":"2024-05-08T18:07:52.903209+0000","last_fullsized":"2024-05-08T18:07:52.903209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:21:20.452363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.856865+0000","last_change":"2024-05-08T18:04:53.848804+0000","last_active":"2024-05-08T18:05:30.856865+0000","last_peered":"2024-05-08T18:05:30.856865+0000","last_clean":"2024-05-08T18:05:30.856865+0000","last_became_active":"2024-05-08T18:04:53.848321+0000","last_became_peered":"2024-05-08T18:04:53.848321+0000","last_unstale":"2024-05-08T18:05:30.856865+0000","last_undegraded":"2024-05-08T18:05:30.856865+0000","last_fullsized":"2024-05-08T18:05:30.856865+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:34.662852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"29'4","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399620+0000","last_change":"2024-05-08T18:04:45.814018+0000","last_active":"2024-05-08T18:06:53.399620+0000","last_peered":"2024-05-08T18:06:53.399620+0000","last_clean":"2024-05-08T18:06:53.399620+0000","last_became_active":"2024-05-08T18:04:45.813424+0000","last_became_peered":"2024-05-08T18:04:45.813424+0000","last_unstale":"2024-05-08T18:06:53.399620+0000","last_undegraded":"2024-05-08T18:06:53.399620+0000","last_fullsized":"2024-05-08T18:06:53.399620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:42:05.254786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866726+0000","last_change":"2024-05-08T18:04:43.841603+0000","last_active":"2024-05-08T18:05:30.866726+0000","last_peered":"2024-05-08T18:05:30.866726+0000","last_clean":"2024-05-08T18:05:30.866726+0000","last_became_active":"2024-05-08T18:04:43.840502+0000","last_became_peered":"2024-05-08T18:04:43.840502+0000","last_unstale":"2024-05-08T18:05:30.866726+0000","last_undegraded":"2024-05-08T18:05:30.866726+0000","last_fullsized":"2024-05-08T18:05:30.866726+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:40:40.407240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860986+0000","last_change":"2024-05-08T18:04:47.819165+0000","last_active":"2024-05-08T18:05:30.860986+0000","last_peered":"2024-05-08T18:05:30.860986+0000","last_clean":"2024-05-08T18:05:30.860986+0000","last_became_active":"2024-05-08T18:04:47.818839+0000","last_became_peered":"2024-05-08T18:04:47.818839+0000","last_unstale":"2024-05-08T18:05:30.860986+0000","last_undegraded":"2024-05-08T18:05:30.860986+0000","last_fullsized":"2024-05-08T18:05:30.860986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:04:15.231714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"29'37","reported_seq":82,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.936372+0000","last_change":"2024-05-08T18:04:49.824708+0000","last_active":"2024-05-08T18:06:51.936372+0000","last_peered":"2024-05-08T18:06:51.936372+0000","last_clean":"2024-05-08T18:06:51.936372+0000","last_became_active":"2024-05-08T18:04:49.824405+0000","last_became_peered":"2024-05-08T18:04:49.824405+0000","last_unstale":"2024-05-08T18:06:51.936372+0000","last_undegraded":"2024-05-08T18:06:51.936372+0000","last_fullsized":"2024-05-08T18:06:51.936372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:52.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858238+0000","last_change":"2024-05-08T18:04:51.845116+0000","last_active":"2024-05-08T18:05:30.858238+0000","last_peered":"2024-05-08T18:05:30.858238+0000","last_clean":"2024-05-08T18:05:30.858238+0000","last_became_active":"2024-05-08T18:04:51.844757+0000","last_became_peered":"2024-05-08T18:04:51.844757+0000","last_unstale":"2024-05-08T18:05:30.858238+0000","last_undegraded":"2024-05-08T18:05:30.858238+0000","last_fullsized":"2024-05-08T18:05:30.858238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:17.567274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866299+0000","last_change":"2024-05-08T18:04:53.841236+0000","last_active":"2024-05-08T18:05:30.866299+0000","last_peered":"2024-05-08T18:05:30.866299+0000","last_clean":"2024-05-08T18:05:30.866299+0000","last_became_active":"2024-05-08T18:04:53.840956+0000","last_became_peered":"2024-05-08T18:04:53.840956+0000","last_unstale":"2024-05-08T18:05:30.866299+0000","last_undegraded":"2024-05-08T18:05:30.866299+0000","last_fullsized":"2024-05-08T18:05:30.866299+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:58.125427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"29'3","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.866543+0000","last_change":"2024-05-08T18:04:45.807309+0000","last_active":"2024-05-08T18:07:53.866543+0000","last_peered":"2024-05-08T18:07:53.866543+0000","last_clean":"2024-05-08T18:07:53.866543+0000","last_became_active":"2024-05-08T18:04:45.806560+0000","last_became_peered":"2024-05-08T18:04:45.806560+0000","last_unstale":"2024-05-08T18:07:53.866543+0000","last_undegraded":"2024-05-08T18:07:53.866543+0000","last_fullsized":"2024-05-08T18:07:53.866543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:11:17.190072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861114+0000","last_change":"2024-05-08T18:04:43.845626+0000","last_active":"2024-05-08T18:05:30.861114+0000","last_peered":"2024-05-08T18:05:30.861114+0000","last_clean":"2024-05-08T18:05:30.861114+0000","last_became_active":"2024-05-08T18:04:43.845214+0000","last_became_peered":"2024-05-08T18:04:43.845214+0000","last_unstale":"2024-05-08T18:05:30.861114+0000","last_undegraded":"2024-05-08T18:05:30.861114+0000","last_fullsized":"2024-05-08T18:05:30.861114+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:00.592659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.865365+0000","last_change":"2024-05-08T18:04:47.823059+0000","last_active":"2024-05-08T18:05:30.865365+0000","last_peered":"2024-05-08T18:05:30.865365+0000","last_clean":"2024-05-08T18:05:30.865365+0000","last_became_active":"2024-05-08T18:04:47.822748+0000","last_became_peered":"2024-05-08T18:04:47.822748+0000","last_unstale":"2024-05-08T18:05:30.865365+0000","last_undegraded":"2024-05-08T18:05:30.865365+0000","last_fullsized":"2024-05-08T18:05:30.865365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:20.980114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"29'31","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.937310+0000","last_change":"2024-05-08T18:04:49.831735+0000","last_active":"2024-05-08T18:06:51.937310+0000","last_peered":"2024-05-08T18:06:51.937310+0000","last_clean":"2024-05-08T18:06:51.937310+0000","last_became_active":"2024-05-08T18:04:49.831440+0000","last_became_peered":"2024-05-08T18:04:49.831440+0000","last_unstale":"2024-05-08T18:06:51.937310+0000","last_undegraded":"2024-05-08T18:06:51.937310+0000","last_fullsized":"2024-05-08T18:06:51.937310+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:40.102877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857159+0000","last_change":"2024-05-08T18:04:51.844443+0000","last_active":"2024-05-08T18:05:30.857159+0000","last_peered":"2024-05-08T18:05:30.857159+0000","last_clean":"2024-05-08T18:05:30.857159+0000","last_became_active":"2024-05-08T18:04:51.844125+0000","last_became_peered":"2024-05-08T18:04:51.844125+0000","last_unstale":"2024-05-08T18:05:30.857159+0000","last_undegraded":"2024-05-08T18:05:30.857159+0000","last_fullsized":"2024-05-08T18:05:30.857159+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:22.017553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862839+0000","last_change":"2024-05-08T18:04:53.849962+0000","last_active":"2024-05-08T18:05:30.862839+0000","last_peered":"2024-05-08T18:05:30.862839+0000","last_clean":"2024-05-08T18:05:30.862839+0000","last_became_active":"2024-05-08T18:04:53.849514+0000","last_became_peered":"2024-05-08T18:04:53.849514+0000","last_unstale":"2024-05-08T18:05:30.862839+0000","last_undegraded":"2024-05-08T18:05:30.862839+0000","last_fullsized":"2024-05-08T18:05:30.862839+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:31.875101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"29'4","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399430+0000","last_change":"2024-05-08T18:04:45.822495+0000","last_active":"2024-05-08T18:06:53.399430+0000","last_peered":"2024-05-08T18:06:53.399430+0000","last_clean":"2024-05-08T18:06:53.399430+0000","last_became_active":"2024-05-08T18:04:45.822204+0000","last_became_peered":"2024-05-08T18:04:45.822204+0000","last_unstale":"2024-05-08T18:06:53.399430+0000","last_undegraded":"2024-05-08T18:06:53.399430+0000","last_fullsized":"2024-05-08T18:06:53.399430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:06.207449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.058832+0000","last_change":"2024-05-08T18:04:43.835654+0000","last_active":"2024-05-08T18:07:54.058832+0000","last_peered":"2024-05-08T18:07:54.058832+0000","last_clean":"2024-05-08T18:07:54.058832+0000","last_became_active":"2024-05-08T18:04:43.834678+0000","last_became_peered":"2024-05-08T18:04:43.834678+0000","last_unstale":"2024-05-08T18:07:54.058832+0000","last_undegraded":"2024-05-08T18:07:54.058832+0000","last_fullsized":"2024-05-08T18:07:54.058832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:23:16.295599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859176+0000","last_change":"2024-05-08T18:04:47.820376+0000","last_active":"2024-05-08T18:05:30.859176+0000","last_peered":"2024-05-08T18:05:30.859176+0000","last_clean":"2024-05-08T18:05:30.859176+0000","last_became_active":"2024-05-08T18:04:47.820036+0000","last_became_peered":"2024-05-08T18:04:47.820036+0000","last_unstale":"2024-05-08T18:05:30.859176+0000","last_undegraded":"2024-05-08T18:05:30.859176+0000","last_fullsized":"2024-05-08T18:05:30.859176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:10:33.041988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"29'28","reported_seq":68,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.960416+0000","last_change":"2024-05-08T18:04:49.832032+0000","last_active":"2024-05-08T18:06:51.960416+0000","last_peered":"2024-05-08T18:06:51.960416+0000","last_clean":"2024-05-08T18:06:51.960416+0000","last_became_active":"2024-05-08T18:04:49.830882+0000","last_became_peered":"2024-05-08T18:04:49.830882+0000","last_unstale":"2024-05-08T18:06:51.960416+0000","last_undegraded":"2024-05-08T18:06:51.960416+0000","last_fullsized":"2024-05-08T18:06:51.960416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:21.475550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":40,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857634+0000","last_change":"2024-05-08T18:04:51.844109+0000","last_active":"2024-05-08T18:05:30.857634+0000","last_peered":"2024-05-08T18:05:30.857634+0000","last_clean":"2024-05-08T18:05:30.857634+0000","last_became_active":"2024-05-08T18:04:51.842734+0000","last_became_peered":"2024-05-08T18:04:51.842734+0000","last_unstale":"2024-05-08T18:05:30.857634+0000","last_undegraded":"2024-05-08T18:05:30.857634+0000","last_fullsized":"2024-05-08T18:05:30.857634+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:47.953187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'10","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.495078+0000","last_change":"2024-05-08T18:04:53.845139+0000","last_active":"2024-05-08T18:07:53.495078+0000","last_peered":"2024-05-08T18:07:53.495078+0000","last_clean":"2024-05-08T18:07:53.495078+0000","last_became_active":"2024-05-08T18:04:53.844891+0000","last_became_peered":"2024-05-08T18:04:53.844891+0000","last_unstale":"2024-05-08T18:07:53.495078+0000","last_undegraded":"2024-05-08T18:07:53.495078+0000","last_fullsized":"2024-05-08T18:07:53.495078+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:32.203551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"29'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.860618+0000","last_change":"2024-05-08T18:04:45.809216+0000","last_active":"2024-05-08T18:07:53.860618+0000","last_peered":"2024-05-08T18:07:53.860618+0000","last_clean":"2024-05-08T18:07:53.860618+0000","last_became_active":"2024-05-08T18:04:45.808936+0000","last_became_peered":"2024-05-08T18:04:45.808936+0000","last_unstale":"2024-05-08T18:07:53.860618+0000","last_undegraded":"2024-05-08T18:07:53.860618+0000","last_fullsized":"2024-05-08T18:07:53.860618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:51.368967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866160+0000","last_change":"2024-05-08T18:04:43.837820+0000","last_active":"2024-05-08T18:05:30.866160+0000","last_peered":"2024-05-08T18:05:30.866160+0000","last_clean":"2024-05-08T18:05:30.866160+0000","last_became_active":"2024-05-08T18:04:43.837070+0000","last_became_peered":"2024-05-08T18:04:43.837070+0000","last_unstale":"2024-05-08T18:05:30.866160+0000","last_undegraded":"2024-05-08T18:05:30.866160+0000","last_fullsized":"2024-05-08T18:05:30.866160+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:50.043665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"29'2","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.565208+0000","last_change":"2024-05-08T18:04:47.824489+0000","last_active":"2024-05-08T18:06:50.565208+0000","last_peered":"2024-05-08T18:06:50.565208+0000","last_clean":"2024-05-08T18:06:50.565208+0000","last_became_active":"2024-05-08T18:04:47.824172+0000","last_became_peered":"2024-05-08T18:04:47.824172+0000","last_unstale":"2024-05-08T18:06:50.565208+0000","last_undegraded":"2024-05-08T18:06:50.565208+0000","last_fullsized":"2024-05-08T18:06:50.565208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:14.408614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"29'32","reported_seq":76,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.418071+0000","last_change":"2024-05-08T18:04:49.832422+0000","last_active":"2024-05-08T18:07:53.418071+0000","last_peered":"2024-05-08T18:07:53.418071+0000","last_clean":"2024-05-08T18:07:53.418071+0000","last_became_active":"2024-05-08T18:04:49.831803+0000","last_became_peered":"2024-05-08T18:04:49.831803+0000","last_unstale":"2024-05-08T18:07:53.418071+0000","last_undegraded":"2024-05-08T18:07:53.418071+0000","last_fullsized":"2024-05-08T18:07:53.418071+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:03:53.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858576+0000","last_change":"2024-05-08T18:04:51.846422+0000","last_active":"2024-05-08T18:05:30.858576+0000","last_peered":"2024-05-08T18:05:30.858576+0000","last_clean":"2024-05-08T18:05:30.858576+0000","last_became_active":"2024-05-08T18:04:51.846225+0000","last_became_peered":"2024-05-08T18:04:51.846225+0000","last_unstale":"2024-05-08T18:05:30.858576+0000","last_undegraded":"2024-05-08T18:05:30.858576+0000","last_fullsized":"2024-05-08T18:05:30.858576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:42:25.975522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"29'4","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.972855+0000","last_change":"2024-05-08T18:04:53.843030+0000","last_active":"2024-05-08T18:06:51.972855+0000","last_peered":"2024-05-08T18:06:51.972855+0000","last_clean":"2024-05-08T18:06:51.972855+0000","last_became_active":"2024-05-08T18:04:53.842429+0000","last_became_peered":"2024-05-08T18:04:53.842429+0000","last_unstale":"2024-05-08T18:06:51.972855+0000","last_undegraded":"2024-05-08T18:06:51.972855+0000","last_fullsized":"2024-05-08T18:06:51.972855+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:30.303540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"27'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866744+0000","last_change":"2024-05-08T18:04:45.818828+0000","last_active":"2024-05-08T18:05:30.866744+0000","last_peered":"2024-05-08T18:05:30.866744+0000","last_clean":"2024-05-08T18:05:30.866744+0000","last_became_active":"2024-05-08T18:04:45.818161+0000","last_became_peered":"2024-05-08T18:04:45.818161+0000","last_unstale":"2024-05-08T18:05:30.866744+0000","last_undegraded":"2024-05-08T18:05:30.866744+0000","last_fullsized":"2024-05-08T18:05:30.866744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:59.477939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859517+0000","last_change":"2024-05-08T18:04:43.843055+0000","last_active":"2024-05-08T18:05:30.859517+0000","last_peered":"2024-05-08T18:05:30.859517+0000","last_clean":"2024-05-08T18:05:30.859517+0000","last_became_active":"2024-05-08T18:04:43.842483+0000","last_became_peered":"2024-05-08T18:04:43.842483+0000","last_unstale":"2024-05-08T18:05:30.859517+0000","last_undegraded":"2024-05-08T18:05:30.859517+0000","last_fullsized":"2024-05-08T18:05:30.859517+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:11.225102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"29'36","reported_seq":106,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995781+0000","last_change":"2024-05-08T18:04:45.808186+0000","last_active":"2024-05-08T18:07:53.995781+0000","last_peered":"2024-05-08T18:07:53.995781+0000","last_clean":"2024-05-08T18:07:53.995781+0000","last_became_active":"2024-05-08T18:04:45.807847+0000","last_became_peered":"2024-05-08T18:04:45.807847+0000","last_unstale":"2024-05-08T18:07:53.995781+0000","last_undegraded":"2024-05-08T18:07:53.995781+0000","last_fullsized":"2024-05-08T18:07:53.995781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:00:26.846485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":44,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858145+0000","last_change":"2024-05-08T18:04:43.836530+0000","last_active":"2024-05-08T18:05:30.858145+0000","last_peered":"2024-05-08T18:05:30.858145+0000","last_clean":"2024-05-08T18:05:30.858145+0000","last_became_active":"2024-05-08T18:04:43.836045+0000","last_became_peered":"2024-05-08T18:04:43.836045+0000","last_unstale":"2024-05-08T18:05:30.858145+0000","last_undegraded":"2024-05-08T18:05:30.858145+0000","last_fullsized":"2024-05-08T18:05:30.858145+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:22.454546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3,2],"acting":[0,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"29'11","reported_seq":68,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.782514+0000","last_change":"2024-05-08T18:04:45.826393+0000","last_active":"2024-05-08T18:07:53.782514+0000","last_peered":"2024-05-08T18:07:53.782514+0000","last_clean":"2024-05-08T18:07:53.782514+0000","last_became_active":"2024-05-08T18:04:45.825506+0000","last_became_peered":"2024-05-08T18:04:45.825506+0000","last_unstale":"2024-05-08T18:07:53.782514+0000","last_undegraded":"2024-05-08T18:07:53.782514+0000","last_fullsized":"2024-05-08T18:07:53.782514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:23:18.600988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":46,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"29'5","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.990901+0000","last_change":"2024-05-08T18:04:43.832854+0000","last_active":"2024-05-08T18:07:53.990901+0000","last_peered":"2024-05-08T18:07:53.990901+0000","last_clean":"2024-05-08T18:07:53.990901+0000","last_became_active":"2024-05-08T18:04:43.832406+0000","last_became_peered":"2024-05-08T18:04:43.832406+0000","last_unstale":"2024-05-08T18:07:53.990901+0000","last_undegraded":"2024-05-08T18:07:53.990901+0000","last_fullsized":"2024-05-08T18:07:53.990901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:16.456364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":29,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"29'2","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.774175+0000","last_change":"2024-05-08T18:04:45.807187+0000","last_active":"2024-05-08T18:07:53.774175+0000","last_peered":"2024-05-08T18:07:53.774175+0000","last_clean":"2024-05-08T18:07:53.774175+0000","last_became_active":"2024-05-08T18:04:45.806646+0000","last_became_peered":"2024-05-08T18:04:45.806646+0000","last_unstale":"2024-05-08T18:07:53.774175+0000","last_undegraded":"2024-05-08T18:07:53.774175+0000","last_fullsized":"2024-05-08T18:07:53.774175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:30.458103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"29'5","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.374758+0000","last_change":"2024-05-08T18:04:43.836158+0000","last_active":"2024-05-08T18:06:53.374758+0000","last_peered":"2024-05-08T18:06:53.374758+0000","last_clean":"2024-05-08T18:06:53.374758+0000","last_became_active":"2024-05-08T18:04:43.835163+0000","last_became_peered":"2024-05-08T18:04:43.835163+0000","last_unstale":"2024-05-08T18:06:53.374758+0000","last_undegraded":"2024-05-08T18:06:53.374758+0000","last_fullsized":"2024-05-08T18:06:53.374758+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:55.493870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":13,"num_write_kb":4746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,1],"acting":[0,3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"29'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.861341+0000","last_change":"2024-05-08T18:04:45.816663+0000","last_active":"2024-05-08T18:07:53.861341+0000","last_peered":"2024-05-08T18:07:53.861341+0000","last_clean":"2024-05-08T18:07:53.861341+0000","last_became_active":"2024-05-08T18:04:45.816391+0000","last_became_peered":"2024-05-08T18:04:45.816391+0000","last_unstale":"2024-05-08T18:07:53.861341+0000","last_undegraded":"2024-05-08T18:07:53.861341+0000","last_fullsized":"2024-05-08T18:07:53.861341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:02.599918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'4","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861292+0000","last_change":"2024-05-08T18:04:43.836083+0000","last_active":"2024-05-08T18:05:30.861292+0000","last_peered":"2024-05-08T18:05:30.861292+0000","last_clean":"2024-05-08T18:05:30.861292+0000","last_became_active":"2024-05-08T18:04:43.835268+0000","last_became_peered":"2024-05-08T18:04:43.835268+0000","last_unstale":"2024-05-08T18:05:30.861292+0000","last_undegraded":"2024-05-08T18:05:30.861292+0000","last_fullsized":"2024-05-08T18:05:30.861292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:47.971014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"29'12","reported_seq":77,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.035458+0000","last_change":"2024-05-08T18:04:45.820787+0000","last_active":"2024-05-08T18:07:54.035458+0000","last_peered":"2024-05-08T18:07:54.035458+0000","last_clean":"2024-05-08T18:07:54.035458+0000","last_became_active":"2024-05-08T18:04:45.819699+0000","last_became_peered":"2024-05-08T18:04:45.819699+0000","last_unstale":"2024-05-08T18:07:54.035458+0000","last_undegraded":"2024-05-08T18:07:54.035458+0000","last_fullsized":"2024-05-08T18:07:54.035458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:18.539896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862510+0000","last_change":"2024-05-08T18:04:43.837155+0000","last_active":"2024-05-08T18:05:30.862510+0000","last_peered":"2024-05-08T18:05:30.862510+0000","last_clean":"2024-05-08T18:05:30.862510+0000","last_became_active":"2024-05-08T18:04:43.836735+0000","last_became_peered":"2024-05-08T18:04:43.836735+0000","last_unstale":"2024-05-08T18:05:30.862510+0000","last_undegraded":"2024-05-08T18:05:30.862510+0000","last_fullsized":"2024-05-08T18:05:30.862510+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:54.711833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"29'14","reported_seq":70,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.219716+0000","last_change":"2024-05-08T18:04:45.812327+0000","last_active":"2024-05-08T18:06:50.219716+0000","last_peered":"2024-05-08T18:06:50.219716+0000","last_clean":"2024-05-08T18:06:50.219716+0000","last_became_active":"2024-05-08T18:04:45.811953+0000","last_became_peered":"2024-05-08T18:04:45.811953+0000","last_unstale":"2024-05-08T18:06:50.219716+0000","last_undegraded":"2024-05-08T18:06:50.219716+0000","last_fullsized":"2024-05-08T18:06:50.219716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:11:17.190072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867588+0000","last_change":"2024-05-08T18:04:43.843298+0000","last_active":"2024-05-08T18:05:30.867588+0000","last_peered":"2024-05-08T18:05:30.867588+0000","last_clean":"2024-05-08T18:05:30.867588+0000","last_became_active":"2024-05-08T18:04:43.842179+0000","last_became_peered":"2024-05-08T18:04:43.842179+0000","last_unstale":"2024-05-08T18:05:30.867588+0000","last_undegraded":"2024-05-08T18:05:30.867588+0000","last_fullsized":"2024-05-08T18:05:30.867588+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:55.493870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,0],"acting":[2,3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"29'8","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063691+0000","last_change":"2024-05-08T18:04:45.812893+0000","last_active":"2024-05-08T18:07:54.063691+0000","last_peered":"2024-05-08T18:07:54.063691+0000","last_clean":"2024-05-08T18:07:54.063691+0000","last_became_active":"2024-05-08T18:04:45.812637+0000","last_became_peered":"2024-05-08T18:04:45.812637+0000","last_unstale":"2024-05-08T18:07:54.063691+0000","last_undegraded":"2024-05-08T18:07:54.063691+0000","last_fullsized":"2024-05-08T18:07:54.063691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:18.973014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"29'6","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.981626+0000","last_change":"2024-05-08T18:04:43.834951+0000","last_active":"2024-05-08T18:07:53.981626+0000","last_peered":"2024-05-08T18:07:53.981626+0000","last_clean":"2024-05-08T18:07:53.981626+0000","last_became_active":"2024-05-08T18:04:43.834491+0000","last_became_peered":"2024-05-08T18:04:43.834491+0000","last_unstale":"2024-05-08T18:07:53.981626+0000","last_undegraded":"2024-05-08T18:07:53.981626+0000","last_fullsized":"2024-05-08T18:07:53.981626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:24:46.108691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"29'8","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.041264+0000","last_change":"2024-05-08T18:04:45.815948+0000","last_active":"2024-05-08T18:07:54.041264+0000","last_peered":"2024-05-08T18:07:54.041264+0000","last_clean":"2024-05-08T18:07:54.041264+0000","last_became_active":"2024-05-08T18:04:45.815677+0000","last_became_peered":"2024-05-08T18:04:45.815677+0000","last_unstale":"2024-05-08T18:07:54.041264+0000","last_undegraded":"2024-05-08T18:07:54.041264+0000","last_fullsized":"2024-05-08T18:07:54.041264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:16.870297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857605+0000","last_change":"2024-05-08T18:04:43.846054+0000","last_active":"2024-05-08T18:05:30.857605+0000","last_peered":"2024-05-08T18:05:30.857605+0000","last_clean":"2024-05-08T18:05:30.857605+0000","last_became_active":"2024-05-08T18:04:43.843826+0000","last_became_peered":"2024-05-08T18:04:43.843826+0000","last_unstale":"2024-05-08T18:05:30.857605+0000","last_undegraded":"2024-05-08T18:05:30.857605+0000","last_fullsized":"2024-05-08T18:05:30.857605+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:08:10.712708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399417+0000","last_change":"2024-05-08T18:04:45.827535+0000","last_active":"2024-05-08T18:06:53.399417+0000","last_peered":"2024-05-08T18:06:53.399417+0000","last_clean":"2024-05-08T18:06:53.399417+0000","last_became_active":"2024-05-08T18:04:45.827252+0000","last_became_peered":"2024-05-08T18:04:45.827252+0000","last_unstale":"2024-05-08T18:06:53.399417+0000","last_undegraded":"2024-05-08T18:06:53.399417+0000","last_fullsized":"2024-05-08T18:06:53.399417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:07.190532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858630+0000","last_change":"2024-05-08T18:04:43.820995+0000","last_active":"2024-05-08T18:05:30.858630+0000","last_peered":"2024-05-08T18:05:30.858630+0000","last_clean":"2024-05-08T18:05:30.858630+0000","last_became_active":"2024-05-08T18:04:43.820360+0000","last_became_peered":"2024-05-08T18:04:43.820360+0000","last_unstale":"2024-05-08T18:05:30.858630+0000","last_undegraded":"2024-05-08T18:05:30.858630+0000","last_fullsized":"2024-05-08T18:05:30.858630+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:54.711833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860974+0000","last_change":"2024-05-08T18:04:45.823656+0000","last_active":"2024-05-08T18:05:30.860974+0000","last_peered":"2024-05-08T18:05:30.860974+0000","last_clean":"2024-05-08T18:05:30.860974+0000","last_became_active":"2024-05-08T18:04:45.823385+0000","last_became_peered":"2024-05-08T18:04:45.823385+0000","last_unstale":"2024-05-08T18:05:30.860974+0000","last_undegraded":"2024-05-08T18:05:30.860974+0000","last_fullsized":"2024-05-08T18:05:30.860974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:26.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'6","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.755149+0000","last_change":"2024-05-08T18:04:43.836532+0000","last_active":"2024-05-08T18:07:53.755149+0000","last_peered":"2024-05-08T18:07:53.755149+0000","last_clean":"2024-05-08T18:07:53.755149+0000","last_became_active":"2024-05-08T18:04:43.835838+0000","last_became_peered":"2024-05-08T18:04:43.835838+0000","last_unstale":"2024-05-08T18:07:53.755149+0000","last_undegraded":"2024-05-08T18:07:53.755149+0000","last_fullsized":"2024-05-08T18:07:53.755149+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:11.225102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8,"num_write":30,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"29'10","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995916+0000","last_change":"2024-05-08T18:04:45.810992+0000","last_active":"2024-05-08T18:07:53.995916+0000","last_peered":"2024-05-08T18:07:53.995916+0000","last_clean":"2024-05-08T18:07:53.995916+0000","last_became_active":"2024-05-08T18:04:45.810703+0000","last_became_peered":"2024-05-08T18:04:45.810703+0000","last_unstale":"2024-05-08T18:07:53.995916+0000","last_undegraded":"2024-05-08T18:07:53.995916+0000","last_fullsized":"2024-05-08T18:07:53.995916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:05.828431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"29'3","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744531+0000","last_change":"2024-05-08T18:04:43.836123+0000","last_active":"2024-05-08T18:06:58.744531+0000","last_peered":"2024-05-08T18:06:58.744531+0000","last_clean":"2024-05-08T18:06:58.744531+0000","last_became_active":"2024-05-08T18:04:43.835518+0000","last_became_peered":"2024-05-08T18:04:43.835518+0000","last_unstale":"2024-05-08T18:06:58.744531+0000","last_undegraded":"2024-05-08T18:06:58.744531+0000","last_fullsized":"2024-05-08T18:06:58.744531+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:06.338736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,3],"acting":[1,2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"29'4","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.219429+0000","last_change":"2024-05-08T18:04:45.821376+0000","last_active":"2024-05-08T18:06:50.219429+0000","last_peered":"2024-05-08T18:06:50.219429+0000","last_clean":"2024-05-08T18:06:50.219429+0000","last_became_active":"2024-05-08T18:04:45.820746+0000","last_became_peered":"2024-05-08T18:04:45.820746+0000","last_unstale":"2024-05-08T18:06:50.219429+0000","last_undegraded":"2024-05-08T18:06:50.219429+0000","last_fullsized":"2024-05-08T18:06:50.219429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:32.833248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859364+0000","last_change":"2024-05-08T18:04:43.847998+0000","last_active":"2024-05-08T18:05:30.859364+0000","last_peered":"2024-05-08T18:05:30.859364+0000","last_clean":"2024-05-08T18:05:30.859364+0000","last_became_active":"2024-05-08T18:04:43.844404+0000","last_became_peered":"2024-05-08T18:04:43.844404+0000","last_unstale":"2024-05-08T18:05:30.859364+0000","last_undegraded":"2024-05-08T18:05:30.859364+0000","last_fullsized":"2024-05-08T18:05:30.859364+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:28.450844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"29'4","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.400051+0000","last_change":"2024-05-08T18:04:45.809582+0000","last_active":"2024-05-08T18:06:53.400051+0000","last_peered":"2024-05-08T18:06:53.400051+0000","last_clean":"2024-05-08T18:06:53.400051+0000","last_became_active":"2024-05-08T18:04:45.809271+0000","last_became_peered":"2024-05-08T18:04:45.809271+0000","last_unstale":"2024-05-08T18:06:53.400051+0000","last_undegraded":"2024-05-08T18:06:53.400051+0000","last_fullsized":"2024-05-08T18:06:53.400051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:32.141179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861152+0000","last_change":"2024-05-08T18:04:43.844366+0000","last_active":"2024-05-08T18:05:30.861152+0000","last_peered":"2024-05-08T18:05:30.861152+0000","last_clean":"2024-05-08T18:05:30.861152+0000","last_became_active":"2024-05-08T18:04:43.838125+0000","last_became_peered":"2024-05-08T18:04:43.838125+0000","last_unstale":"2024-05-08T18:05:30.861152+0000","last_undegraded":"2024-05-08T18:05:30.861152+0000","last_fullsized":"2024-05-08T18:05:30.861152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:24.149010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"29'15","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070567+0000","last_change":"2024-05-08T18:04:45.828761+0000","last_active":"2024-05-08T18:07:54.070567+0000","last_peered":"2024-05-08T18:07:54.070567+0000","last_clean":"2024-05-08T18:07:54.070567+0000","last_became_active":"2024-05-08T18:04:45.828488+0000","last_became_peered":"2024-05-08T18:04:45.828488+0000","last_unstale":"2024-05-08T18:07:54.070567+0000","last_undegraded":"2024-05-08T18:07:54.070567+0000","last_fullsized":"2024-05-08T18:07:54.070567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:27:23.439814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867753+0000","last_change":"2024-05-08T18:04:43.843455+0000","last_active":"2024-05-08T18:05:30.867753+0000","last_peered":"2024-05-08T18:05:30.867753+0000","last_clean":"2024-05-08T18:05:30.867753+0000","last_became_active":"2024-05-08T18:04:43.842670+0000","last_became_peered":"2024-05-08T18:04:43.842670+0000","last_unstale":"2024-05-08T18:05:30.867753+0000","last_undegraded":"2024-05-08T18:05:30.867753+0000","last_fullsized":"2024-05-08T18:05:30.867753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:01:49.620924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"29'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.547055+0000","last_change":"2024-05-08T18:04:45.812074+0000","last_active":"2024-05-08T18:06:09.547055+0000","last_peered":"2024-05-08T18:06:09.547055+0000","last_clean":"2024-05-08T18:06:09.547055+0000","last_became_active":"2024-05-08T18:04:45.810562+0000","last_became_peered":"2024-05-08T18:04:45.810562+0000","last_unstale":"2024-05-08T18:06:09.547055+0000","last_undegraded":"2024-05-08T18:06:09.547055+0000","last_fullsized":"2024-05-08T18:06:09.547055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:58.784190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864727+0000","last_change":"2024-05-08T18:04:43.838746+0000","last_active":"2024-05-08T18:05:30.864727+0000","last_peered":"2024-05-08T18:05:30.864727+0000","last_clean":"2024-05-08T18:05:30.864727+0000","last_became_active":"2024-05-08T18:04:43.835943+0000","last_became_peered":"2024-05-08T18:04:43.835943+0000","last_unstale":"2024-05-08T18:05:30.864727+0000","last_undegraded":"2024-05-08T18:05:30.864727+0000","last_fullsized":"2024-05-08T18:05:30.864727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:27:18.106771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857870+0000","last_change":"2024-05-08T18:04:43.836378+0000","last_active":"2024-05-08T18:05:30.857870+0000","last_peered":"2024-05-08T18:05:30.857870+0000","last_clean":"2024-05-08T18:05:30.857870+0000","last_became_active":"2024-05-08T18:04:43.836007+0000","last_became_peered":"2024-05-08T18:04:43.836007+0000","last_unstale":"2024-05-08T18:05:30.857870+0000","last_undegraded":"2024-05-08T18:05:30.857870+0000","last_fullsized":"2024-05-08T18:05:30.857870+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:57:00.868492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"29'8","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.219707+0000","last_change":"2024-05-08T18:04:45.812333+0000","last_active":"2024-05-08T18:06:50.219707+0000","last_peered":"2024-05-08T18:06:50.219707+0000","last_clean":"2024-05-08T18:06:50.219707+0000","last_became_active":"2024-05-08T18:04:45.811648+0000","last_became_peered":"2024-05-08T18:04:45.811648+0000","last_unstale":"2024-05-08T18:06:50.219707+0000","last_undegraded":"2024-05-08T18:06:50.219707+0000","last_fullsized":"2024-05-08T18:06:50.219707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:18.539896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":31,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"29'5","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.740580+0000","last_change":"2024-05-08T18:04:43.836500+0000","last_active":"2024-05-08T18:07:03.740580+0000","last_peered":"2024-05-08T18:07:03.740580+0000","last_clean":"2024-05-08T18:07:03.740580+0000","last_became_active":"2024-05-08T18:04:43.835712+0000","last_became_peered":"2024-05-08T18:04:43.835712+0000","last_unstale":"2024-05-08T18:07:03.740580+0000","last_undegraded":"2024-05-08T18:07:03.740580+0000","last_fullsized":"2024-05-08T18:07:03.740580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:06.338736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8,"num_write":21,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"29'10","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.775008+0000","last_change":"2024-05-08T18:04:45.812258+0000","last_active":"2024-05-08T18:07:53.775008+0000","last_peered":"2024-05-08T18:07:53.775008+0000","last_clean":"2024-05-08T18:07:53.775008+0000","last_became_active":"2024-05-08T18:04:45.811921+0000","last_became_peered":"2024-05-08T18:04:45.811921+0000","last_unstale":"2024-05-08T18:07:53.775008+0000","last_undegraded":"2024-05-08T18:07:53.775008+0000","last_fullsized":"2024-05-08T18:07:53.775008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:00:26.846485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2256,"ondisk_log_size":2256,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":13385,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1402,"num_read_kb":1188,"num_write":531,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":10826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1275,"ondisk_log_size":1275,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":961,"num_read_kb":961,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":25809792,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":138,"num_read_kb":87,"num_write":309,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34422784,"data_stored":34422784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":256,"acting":256,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705707,"num_pgs":179,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48332,"kb_used_data":8836,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323508,"statfs":{"total":96636764160,"available":96587272192,"internally_reserved":0,"allocated":9048064,"data_stored":8743050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705708,"num_pgs":153,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48848,"kb_used_data":9352,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322992,"statfs":{"total":96636764160,"available":96586743808,"internally_reserved":0,"allocated":9576448,"data_stored":9329849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705708,"num_pgs":157,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48872,"kb_used_data":9376,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322968,"statfs":{"total":96636764160,"available":96586719232,"internally_reserved":0,"allocated":9601024,"data_stored":9332048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39980,"kb_used_data":8804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331860,"statfs":{"total":96636764160,"available":96595824640,"internally_reserved":0,"allocated":9015296,"data_stored":8741305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":8605696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":8605696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":8605696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":8605696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":4269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T18:08:01.150 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T18:08:01.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.282+0000 7ff192b6e640 1 Processor -- start 2024-05-08T18:08:01.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.282+0000 7ff192b6e640 1 -- start start 2024-05-08T18:08:01.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.283+0000 7ff192b6e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff18c12c530 0x7ff18c12c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.283+0000 7ff192b6e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff18c12c230 con 0x7ff18c071d40 2024-05-08T18:08:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.283+0000 7ff192b6e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7ff18c09ad20 con 0x7ff18c12c530 2024-05-08T18:08:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.283+0000 7ff19090a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff18c12c530 0x7ff18c12c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.283+0000 7ff18bfff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff18c071d40 0x7ff18c072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45162/0 (socket says 172.21.15.46:45162) 2024-05-08T18:08:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.283+0000 7ff18bfff640 1 -- 172.21.15.46:0/3658319271 learned_addr learned my addr 172.21.15.46:0/3658319271 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:08:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.283+0000 7ff19090a640 1 -- 172.21.15.46:0/3658319271 >> v1:172.21.15.46:6789/0 conn(0x7ff18c071d40 legacy=0x7ff18c072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:01.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.283+0000 7ff19090a640 1 -- 172.21.15.46:0/3658319271 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18c073510 con 0x7ff18c12c530 2024-05-08T18:08:01.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.284+0000 7ff19090a640 1 --2- 172.21.15.46:0/3658319271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c12c530 0x7ff18c12c910 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff184009720 tx=0x7ff18402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635220353c3a1c71 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.284+0000 7ff18b7fe640 1 -- 172.21.15.46:0/3658319271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1840316d0 con 0x7ff18c12c530 2024-05-08T18:08:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.284+0000 7ff18b7fe640 1 -- 172.21.15.46:0/3658319271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff184037070 con 0x7ff18c12c530 2024-05-08T18:08:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.284+0000 7ff18b7fe640 1 -- 172.21.15.46:0/3658319271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff1840319c0 con 0x7ff18c12c530 2024-05-08T18:08:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.285+0000 7ff192b6e640 1 -- 172.21.15.46:0/3658319271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c12c530 msgr2=0x7ff18c12c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.285+0000 7ff192b6e640 1 --2- 172.21.15.46:0/3658319271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c12c530 0x7ff18c12c910 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff184009720 tx=0x7ff18402b840 comp rx=0 tx=0).stop 2024-05-08T18:08:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.285+0000 7ff192b6e640 1 -- 172.21.15.46:0/3658319271 shutdown_connections 2024-05-08T18:08:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.285+0000 7ff192b6e640 1 --2- 172.21.15.46:0/3658319271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c12c530 0x7ff18c12c910 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.285+0000 7ff192b6e640 1 -- 172.21.15.46:0/3658319271 >> 172.21.15.46:0/3658319271 conn(0x7ff18c09d7a0 msgr2=0x7ff18c09dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.285+0000 7ff192b6e640 1 -- 172.21.15.46:0/3658319271 shutdown_connections 2024-05-08T18:08:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.285+0000 7ff192b6e640 1 -- 172.21.15.46:0/3658319271 wait complete. 2024-05-08T18:08:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.286+0000 7ff192b6e640 1 Processor -- start 2024-05-08T18:08:01.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.286+0000 7ff192b6e640 1 -- start start 2024-05-08T18:08:01.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.286+0000 7ff192b6e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c071d40 0x7ff18c1c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:01.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.286+0000 7ff192b6e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff18c12cfd0 con 0x7ff18c071d40 2024-05-08T18:08:01.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.286+0000 7ff19090a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c071d40 0x7ff18c1c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:01.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.286+0000 7ff19090a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c071d40 0x7ff18c1c4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52012/0 (socket says 172.21.15.46:52012) 2024-05-08T18:08:01.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.286+0000 7ff19090a640 1 -- 172.21.15.46:0/2554298902 learned_addr learned my addr 172.21.15.46:0/2554298902 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:08:01.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.287+0000 7ff19090a640 1 -- 172.21.15.46:0/2554298902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18c071ad0 con 0x7ff18c071d40 2024-05-08T18:08:01.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.287+0000 7ff19090a640 1 --2- 172.21.15.46:0/2554298902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c071d40 0x7ff18c1c4060 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff184034040 tx=0x7ff1840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:01.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.287+0000 7ff1897fa640 1 -- 172.21.15.46:0/2554298902 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff184032770 con 0x7ff18c071d40 2024-05-08T18:08:01.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.287+0000 7ff1897fa640 1 -- 172.21.15.46:0/2554298902 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff184032930 con 0x7ff18c071d40 2024-05-08T18:08:01.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.287+0000 7ff192b6e640 1 -- 172.21.15.46:0/2554298902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff18c070ea0 con 0x7ff18c071d40 2024-05-08T18:08:01.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.287+0000 7ff1897fa640 1 -- 172.21.15.46:0/2554298902 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff184032c50 con 0x7ff18c071d40 2024-05-08T18:08:01.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.287+0000 7ff192b6e640 1 -- 172.21.15.46:0/2554298902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff18c0715c0 con 0x7ff18c071d40 2024-05-08T18:08:01.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.288+0000 7ff1897fa640 1 -- 172.21.15.46:0/2554298902 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7ff18404c060 con 0x7ff18c071d40 2024-05-08T18:08:01.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.288+0000 7ff1897fa640 1 --2- 172.21.15.46:0/2554298902 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff16c03d2b0 0x7ff16c03f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:01.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.288+0000 7ff1897fa640 1 -- 172.21.15.46:0/2554298902 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7ff18402f050 con 0x7ff18c071d40 2024-05-08T18:08:01.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.288+0000 7ff192b6e640 1 -- 172.21.15.46:0/2554298902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff18c117da0 con 0x7ff18c071d40 2024-05-08T18:08:01.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.288+0000 7ff18bfff640 1 --2- 172.21.15.46:0/2554298902 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff16c03d2b0 0x7ff16c03f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:01.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.289+0000 7ff18bfff640 1 --2- 172.21.15.46:0/2554298902 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff16c03d2b0 0x7ff16c03f770 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff17c005c00 tx=0x7ff17c0016b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:01.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.293+0000 7ff1897fa640 1 -- 172.21.15.46:0/2554298902 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7ff184040070 con 0x7ff18c071d40 2024-05-08T18:08:01.351 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 has been restored 2024-05-08T18:08:01.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.482+0000 7ff192b6e640 1 -- 172.21.15.46:0/2554298902 --> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff18c0640a0 con 0x7ff16c03d2b0 2024-05-08T18:08:01.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.498+0000 7ff1897fa640 1 -- 172.21.15.46:0/2554298902 <== mgr.4098 v2:172.21.15.50:6824/2602185226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665903 (secure 0 0 0) 0x7ff18c0640a0 con 0x7ff16c03d2b0 2024-05-08T18:08:01.504 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:08:01.509 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T18:08:01.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.503+0000 7ff192b6e640 1 -- 172.21.15.46:0/2554298902 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff16c03d2b0 msgr2=0x7ff16c03f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:01.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.503+0000 7ff192b6e640 1 --2- 172.21.15.46:0/2554298902 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff16c03d2b0 0x7ff16c03f770 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff17c005c00 tx=0x7ff17c0016b0 comp rx=0 tx=0).stop 2024-05-08T18:08:01.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.503+0000 7ff192b6e640 1 -- 172.21.15.46:0/2554298902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c071d40 msgr2=0x7ff18c1c4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:01.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.503+0000 7ff192b6e640 1 --2- 172.21.15.46:0/2554298902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c071d40 0x7ff18c1c4060 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff184034040 tx=0x7ff1840046c0 comp rx=0 tx=0).stop 2024-05-08T18:08:01.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.503+0000 7ff192b6e640 1 -- 172.21.15.46:0/2554298902 shutdown_connections 2024-05-08T18:08:01.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.503+0000 7ff192b6e640 1 --2- 172.21.15.46:0/2554298902 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7ff16c03d2b0 0x7ff16c03f770 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:01.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.503+0000 7ff192b6e640 1 --2- 172.21.15.46:0/2554298902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff18c071d40 0x7ff18c1c4060 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:01.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.503+0000 7ff192b6e640 1 -- 172.21.15.46:0/2554298902 >> 172.21.15.46:0/2554298902 conn(0x7ff18c09d7a0 msgr2=0x7ff18c06f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:01.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.504+0000 7ff192b6e640 1 -- 172.21.15.46:0/2554298902 shutdown_connections 2024-05-08T18:08:01.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.504+0000 7ff192b6e640 1 -- 172.21.15.46:0/2554298902 wait complete. 2024-05-08T18:08:01.512 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-05-08T18:07:59.727131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26416886,"num_objects":227,"num_object_clones":0,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":227,"num_whiteouts":0,"num_read":4834,"num_read_kb":4416,"num_write":2046,"num_write_kb":60584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4431,"ondisk_log_size":4431,"up":658,"acting":658,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":658,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":186032,"kb_used_data":36368,"kb_used_omap":6,"kb_used_meta":149625,"kb_avail":377301328,"statfs":{"total":386547056640,"available":386356559872,"internally_reserved":0,"allocated":37240832,"data_stored":36146252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":153216790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25815784,"num_objects":9,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":497,"num_read_kb":452,"num_write":384,"num_write_kb":29371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003443"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866950+0000","last_change":"2024-05-08T18:04:43.836900+0000","last_active":"2024-05-08T18:05:30.866950+0000","last_peered":"2024-05-08T18:05:30.866950+0000","last_clean":"2024-05-08T18:05:30.866950+0000","last_became_active":"2024-05-08T18:04:43.836496+0000","last_became_peered":"2024-05-08T18:04:43.836496+0000","last_unstale":"2024-05-08T18:05:30.866950+0000","last_undegraded":"2024-05-08T18:05:30.866950+0000","last_fullsized":"2024-05-08T18:05:30.866950+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:22.454546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'6","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.860763+0000","last_change":"2024-05-08T18:04:45.812340+0000","last_active":"2024-05-08T18:07:53.860763+0000","last_peered":"2024-05-08T18:07:53.860763+0000","last_clean":"2024-05-08T18:07:53.860763+0000","last_became_active":"2024-05-08T18:04:45.811368+0000","last_became_peered":"2024-05-08T18:04:45.811368+0000","last_unstale":"2024-05-08T18:07:53.860763+0000","last_undegraded":"2024-05-08T18:07:53.860763+0000","last_fullsized":"2024-05-08T18:07:53.860763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:36:45.459800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.206267+0000","last_change":"2024-05-08T18:04:43.835466+0000","last_active":"2024-05-08T18:06:50.206267+0000","last_peered":"2024-05-08T18:06:50.206267+0000","last_clean":"2024-05-08T18:06:50.206267+0000","last_became_active":"2024-05-08T18:04:43.834612+0000","last_became_peered":"2024-05-08T18:04:43.834612+0000","last_unstale":"2024-05-08T18:06:50.206267+0000","last_undegraded":"2024-05-08T18:06:50.206267+0000","last_fullsized":"2024-05-08T18:06:50.206267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:46.422109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"29'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.546966+0000","last_change":"2024-05-08T18:04:45.828749+0000","last_active":"2024-05-08T18:06:09.546966+0000","last_peered":"2024-05-08T18:06:09.546966+0000","last_clean":"2024-05-08T18:06:09.546966+0000","last_became_active":"2024-05-08T18:04:45.827965+0000","last_became_peered":"2024-05-08T18:04:45.827965+0000","last_unstale":"2024-05-08T18:06:09.546966+0000","last_undegraded":"2024-05-08T18:06:09.546966+0000","last_fullsized":"2024-05-08T18:06:09.546966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:56:23.117814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861383+0000","last_change":"2024-05-08T18:04:43.843806+0000","last_active":"2024-05-08T18:05:30.861383+0000","last_peered":"2024-05-08T18:05:30.861383+0000","last_clean":"2024-05-08T18:05:30.861383+0000","last_became_active":"2024-05-08T18:04:43.837456+0000","last_became_peered":"2024-05-08T18:04:43.837456+0000","last_unstale":"2024-05-08T18:05:30.861383+0000","last_undegraded":"2024-05-08T18:05:30.861383+0000","last_fullsized":"2024-05-08T18:05:30.861383+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:00:24.839225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"29'12","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063738+0000","last_change":"2024-05-08T18:04:45.825315+0000","last_active":"2024-05-08T18:07:54.063738+0000","last_peered":"2024-05-08T18:07:54.063738+0000","last_clean":"2024-05-08T18:07:54.063738+0000","last_became_active":"2024-05-08T18:04:45.825022+0000","last_became_peered":"2024-05-08T18:04:45.825022+0000","last_unstale":"2024-05-08T18:07:54.063738+0000","last_undegraded":"2024-05-08T18:07:54.063738+0000","last_fullsized":"2024-05-08T18:07:54.063738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:45:53.925354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863324+0000","last_change":"2024-05-08T18:04:43.842478+0000","last_active":"2024-05-08T18:05:30.863324+0000","last_peered":"2024-05-08T18:05:30.863324+0000","last_clean":"2024-05-08T18:05:30.863324+0000","last_became_active":"2024-05-08T18:04:43.842077+0000","last_became_peered":"2024-05-08T18:04:43.842077+0000","last_unstale":"2024-05-08T18:05:30.863324+0000","last_undegraded":"2024-05-08T18:05:30.863324+0000","last_fullsized":"2024-05-08T18:05:30.863324+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:06.338736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"29'16","reported_seq":70,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.035049+0000","last_change":"2024-05-08T18:04:45.824870+0000","last_active":"2024-05-08T18:07:54.035049+0000","last_peered":"2024-05-08T18:07:54.035049+0000","last_clean":"2024-05-08T18:07:54.035049+0000","last_became_active":"2024-05-08T18:04:45.824581+0000","last_became_peered":"2024-05-08T18:04:45.824581+0000","last_unstale":"2024-05-08T18:07:54.035049+0000","last_undegraded":"2024-05-08T18:07:54.035049+0000","last_fullsized":"2024-05-08T18:07:54.035049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:48.934445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859690+0000","last_change":"2024-05-08T18:04:43.833461+0000","last_active":"2024-05-08T18:05:30.859690+0000","last_peered":"2024-05-08T18:05:30.859690+0000","last_clean":"2024-05-08T18:05:30.859690+0000","last_became_active":"2024-05-08T18:04:43.832722+0000","last_became_peered":"2024-05-08T18:04:43.832722+0000","last_unstale":"2024-05-08T18:05:30.859690+0000","last_undegraded":"2024-05-08T18:05:30.859690+0000","last_fullsized":"2024-05-08T18:05:30.859690+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:11:15.182813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"29'8","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.738600+0000","last_change":"2024-05-08T18:04:45.808678+0000","last_active":"2024-05-08T18:06:09.738600+0000","last_peered":"2024-05-08T18:06:09.738600+0000","last_clean":"2024-05-08T18:06:09.738600+0000","last_became_active":"2024-05-08T18:04:45.808383+0000","last_became_peered":"2024-05-08T18:04:45.808383+0000","last_unstale":"2024-05-08T18:06:09.738600+0000","last_undegraded":"2024-05-08T18:06:09.738600+0000","last_fullsized":"2024-05-08T18:06:09.738600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:02.599918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862756+0000","last_change":"2024-05-08T18:04:43.841284+0000","last_active":"2024-05-08T18:05:30.862756+0000","last_peered":"2024-05-08T18:05:30.862756+0000","last_clean":"2024-05-08T18:05:30.862756+0000","last_became_active":"2024-05-08T18:04:43.840851+0000","last_became_peered":"2024-05-08T18:04:43.840851+0000","last_unstale":"2024-05-08T18:05:30.862756+0000","last_undegraded":"2024-05-08T18:05:30.862756+0000","last_fullsized":"2024-05-08T18:05:30.862756+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:10.360847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063767+0000","last_change":"2024-05-08T18:04:45.829481+0000","last_active":"2024-05-08T18:07:54.063767+0000","last_peered":"2024-05-08T18:07:54.063767+0000","last_clean":"2024-05-08T18:07:54.063767+0000","last_became_active":"2024-05-08T18:04:45.828334+0000","last_became_peered":"2024-05-08T18:04:45.828334+0000","last_unstale":"2024-05-08T18:07:54.063767+0000","last_undegraded":"2024-05-08T18:07:54.063767+0000","last_fullsized":"2024-05-08T18:07:54.063767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:20:31.411889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.978496+0000","last_change":"2024-05-08T18:04:43.833826+0000","last_active":"2024-05-08T18:07:53.978496+0000","last_peered":"2024-05-08T18:07:53.978496+0000","last_clean":"2024-05-08T18:07:53.978496+0000","last_became_active":"2024-05-08T18:04:43.833437+0000","last_became_peered":"2024-05-08T18:04:43.833437+0000","last_unstale":"2024-05-08T18:07:53.978496+0000","last_undegraded":"2024-05-08T18:07:53.978496+0000","last_fullsized":"2024-05-08T18:07:53.978496+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:46.740116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.868015+0000","last_change":"2024-05-08T18:04:45.812158+0000","last_active":"2024-05-08T18:05:30.868015+0000","last_peered":"2024-05-08T18:05:30.868015+0000","last_clean":"2024-05-08T18:05:30.868015+0000","last_became_active":"2024-05-08T18:04:45.811884+0000","last_became_peered":"2024-05-08T18:04:45.811884+0000","last_unstale":"2024-05-08T18:05:30.868015+0000","last_undegraded":"2024-05-08T18:05:30.868015+0000","last_fullsized":"2024-05-08T18:05:30.868015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:30.458103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858546+0000","last_change":"2024-05-08T18:04:43.821005+0000","last_active":"2024-05-08T18:05:30.858546+0000","last_peered":"2024-05-08T18:05:30.858546+0000","last_clean":"2024-05-08T18:05:30.858546+0000","last_became_active":"2024-05-08T18:04:43.820312+0000","last_became_peered":"2024-05-08T18:04:43.820312+0000","last_unstale":"2024-05-08T18:05:30.858546+0000","last_undegraded":"2024-05-08T18:05:30.858546+0000","last_fullsized":"2024-05-08T18:05:30.858546+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:40:40.407240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"29'7","reported_seq":61,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070848+0000","last_change":"2024-05-08T18:04:45.823448+0000","last_active":"2024-05-08T18:07:54.070848+0000","last_peered":"2024-05-08T18:07:54.070848+0000","last_clean":"2024-05-08T18:07:54.070848+0000","last_became_active":"2024-05-08T18:04:45.822620+0000","last_became_peered":"2024-05-08T18:04:45.822620+0000","last_unstale":"2024-05-08T18:07:54.070848+0000","last_undegraded":"2024-05-08T18:07:54.070848+0000","last_fullsized":"2024-05-08T18:07:54.070848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:08.184602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"29'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995728+0000","last_change":"2024-05-08T18:04:45.826814+0000","last_active":"2024-05-08T18:07:53.995728+0000","last_peered":"2024-05-08T18:07:53.995728+0000","last_clean":"2024-05-08T18:07:53.995728+0000","last_became_active":"2024-05-08T18:04:45.826493+0000","last_became_peered":"2024-05-08T18:04:45.826493+0000","last_unstale":"2024-05-08T18:07:53.995728+0000","last_undegraded":"2024-05-08T18:07:53.995728+0000","last_fullsized":"2024-05-08T18:07:53.995728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:12.401514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866321+0000","last_change":"2024-05-08T18:04:43.838700+0000","last_active":"2024-05-08T18:05:30.866321+0000","last_peered":"2024-05-08T18:05:30.866321+0000","last_clean":"2024-05-08T18:05:30.866321+0000","last_became_active":"2024-05-08T18:04:43.838332+0000","last_became_peered":"2024-05-08T18:04:43.838332+0000","last_unstale":"2024-05-08T18:05:30.866321+0000","last_undegraded":"2024-05-08T18:05:30.866321+0000","last_fullsized":"2024-05-08T18:05:30.866321+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:01:49.620924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"29'4","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399796+0000","last_change":"2024-05-08T18:04:45.807732+0000","last_active":"2024-05-08T18:06:53.399796+0000","last_peered":"2024-05-08T18:06:53.399796+0000","last_clean":"2024-05-08T18:06:53.399796+0000","last_became_active":"2024-05-08T18:04:45.807438+0000","last_became_peered":"2024-05-08T18:04:45.807438+0000","last_unstale":"2024-05-08T18:06:53.399796+0000","last_undegraded":"2024-05-08T18:06:53.399796+0000","last_fullsized":"2024-05-08T18:06:53.399796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:27:20.114031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867876+0000","last_change":"2024-05-08T18:04:43.843599+0000","last_active":"2024-05-08T18:05:30.867876+0000","last_peered":"2024-05-08T18:05:30.867876+0000","last_clean":"2024-05-08T18:05:30.867876+0000","last_became_active":"2024-05-08T18:04:43.842741+0000","last_became_peered":"2024-05-08T18:04:43.842741+0000","last_unstale":"2024-05-08T18:05:30.867876+0000","last_undegraded":"2024-05-08T18:05:30.867876+0000","last_fullsized":"2024-05-08T18:05:30.867876+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:54.711833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"29'11","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.002969+0000","last_change":"2024-05-08T18:04:45.811141+0000","last_active":"2024-05-08T18:07:54.002969+0000","last_peered":"2024-05-08T18:07:54.002969+0000","last_clean":"2024-05-08T18:07:54.002969+0000","last_became_active":"2024-05-08T18:04:45.810827+0000","last_became_peered":"2024-05-08T18:04:45.810827+0000","last_unstale":"2024-05-08T18:07:54.002969+0000","last_undegraded":"2024-05-08T18:07:54.002969+0000","last_fullsized":"2024-05-08T18:07:54.002969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:26.952570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"29'3","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746208+0000","last_change":"2024-05-08T18:04:43.838115+0000","last_active":"2024-05-08T18:07:08.746208+0000","last_peered":"2024-05-08T18:07:08.746208+0000","last_clean":"2024-05-08T18:07:08.746208+0000","last_became_active":"2024-05-08T18:04:43.837706+0000","last_became_peered":"2024-05-08T18:04:43.837706+0000","last_unstale":"2024-05-08T18:07:08.746208+0000","last_undegraded":"2024-05-08T18:07:08.746208+0000","last_fullsized":"2024-05-08T18:07:08.746208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:26.918679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"29'8","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.738522+0000","last_change":"2024-05-08T18:04:45.822292+0000","last_active":"2024-05-08T18:06:09.738522+0000","last_peered":"2024-05-08T18:06:09.738522+0000","last_clean":"2024-05-08T18:06:09.738522+0000","last_became_active":"2024-05-08T18:04:45.822000+0000","last_became_peered":"2024-05-08T18:04:45.822000+0000","last_unstale":"2024-05-08T18:06:09.738522+0000","last_undegraded":"2024-05-08T18:06:09.738522+0000","last_fullsized":"2024-05-08T18:06:09.738522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:08.184602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866478+0000","last_change":"2024-05-08T18:04:43.841486+0000","last_active":"2024-05-08T18:05:30.866478+0000","last_peered":"2024-05-08T18:05:30.866478+0000","last_clean":"2024-05-08T18:05:30.866478+0000","last_became_active":"2024-05-08T18:04:43.841063+0000","last_became_peered":"2024-05-08T18:04:43.841063+0000","last_unstale":"2024-05-08T18:05:30.866478+0000","last_undegraded":"2024-05-08T18:05:30.866478+0000","last_fullsized":"2024-05-08T18:05:30.866478+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:05:33.776915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"29'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.116184+0000","last_change":"2024-05-08T18:04:45.824006+0000","last_active":"2024-05-08T18:07:54.116184+0000","last_peered":"2024-05-08T18:07:54.116184+0000","last_clean":"2024-05-08T18:07:54.116184+0000","last_became_active":"2024-05-08T18:04:45.823611+0000","last_became_peered":"2024-05-08T18:04:45.823611+0000","last_unstale":"2024-05-08T18:07:54.116184+0000","last_undegraded":"2024-05-08T18:07:54.116184+0000","last_fullsized":"2024-05-08T18:07:54.116184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:02.599918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860254+0000","last_change":"2024-05-08T18:04:43.835685+0000","last_active":"2024-05-08T18:05:30.860254+0000","last_peered":"2024-05-08T18:05:30.860254+0000","last_clean":"2024-05-08T18:05:30.860254+0000","last_became_active":"2024-05-08T18:04:43.834456+0000","last_became_peered":"2024-05-08T18:04:43.834456+0000","last_unstale":"2024-05-08T18:05:30.860254+0000","last_undegraded":"2024-05-08T18:05:30.860254+0000","last_fullsized":"2024-05-08T18:05:30.860254+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:50.043665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399432+0000","last_change":"2024-05-08T18:04:45.808763+0000","last_active":"2024-05-08T18:06:53.399432+0000","last_peered":"2024-05-08T18:06:53.399432+0000","last_clean":"2024-05-08T18:06:53.399432+0000","last_became_active":"2024-05-08T18:04:45.808416+0000","last_became_peered":"2024-05-08T18:04:45.808416+0000","last_unstale":"2024-05-08T18:06:53.399432+0000","last_undegraded":"2024-05-08T18:06:53.399432+0000","last_fullsized":"2024-05-08T18:06:53.399432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:35.780903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'4","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858633+0000","last_change":"2024-05-08T18:04:43.845458+0000","last_active":"2024-05-08T18:05:30.858633+0000","last_peered":"2024-05-08T18:05:30.858633+0000","last_clean":"2024-05-08T18:05:30.858633+0000","last_became_active":"2024-05-08T18:04:43.844702+0000","last_became_peered":"2024-05-08T18:04:43.844702+0000","last_unstale":"2024-05-08T18:05:30.858633+0000","last_undegraded":"2024-05-08T18:05:30.858633+0000","last_fullsized":"2024-05-08T18:05:30.858633+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:58.229253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"29'35","reported_seq":99,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.016701+0000","last_change":"2024-05-08T18:04:53.849775+0000","last_active":"2024-05-08T18:07:54.016701+0000","last_peered":"2024-05-08T18:07:54.016701+0000","last_clean":"2024-05-08T18:07:54.016701+0000","last_became_active":"2024-05-08T18:04:53.849475+0000","last_became_peered":"2024-05-08T18:04:53.849475+0000","last_unstale":"2024-05-08T18:07:54.016701+0000","last_undegraded":"2024-05-08T18:07:54.016701+0000","last_fullsized":"2024-05-08T18:07:54.016701+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:12.918578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"29'12","reported_seq":66,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995839+0000","last_change":"2024-05-08T18:04:45.824289+0000","last_active":"2024-05-08T18:07:53.995839+0000","last_peered":"2024-05-08T18:07:53.995839+0000","last_clean":"2024-05-08T18:07:53.995839+0000","last_became_active":"2024-05-08T18:04:45.824005+0000","last_became_peered":"2024-05-08T18:04:45.824005+0000","last_unstale":"2024-05-08T18:07:53.995839+0000","last_undegraded":"2024-05-08T18:07:53.995839+0000","last_fullsized":"2024-05-08T18:07:53.995839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:45.743430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859173+0000","last_change":"2024-05-08T18:04:43.834249+0000","last_active":"2024-05-08T18:05:30.859173+0000","last_peered":"2024-05-08T18:05:30.859173+0000","last_clean":"2024-05-08T18:05:30.859173+0000","last_became_active":"2024-05-08T18:04:43.833822+0000","last_became_peered":"2024-05-08T18:04:43.833822+0000","last_unstale":"2024-05-08T18:05:30.859173+0000","last_undegraded":"2024-05-08T18:05:30.859173+0000","last_fullsized":"2024-05-08T18:05:30.859173+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:00:37.867739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859597+0000","last_change":"2024-05-08T18:04:47.820083+0000","last_active":"2024-05-08T18:05:30.859597+0000","last_peered":"2024-05-08T18:05:30.859597+0000","last_clean":"2024-05-08T18:05:30.859597+0000","last_became_active":"2024-05-08T18:04:47.819682+0000","last_became_peered":"2024-05-08T18:04:47.819682+0000","last_unstale":"2024-05-08T18:05:30.859597+0000","last_undegraded":"2024-05-08T18:05:30.859597+0000","last_fullsized":"2024-05-08T18:05:30.859597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:00:55.772655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"29'31","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.748532+0000","last_change":"2024-05-08T18:04:49.833877+0000","last_active":"2024-05-08T18:07:18.748532+0000","last_peered":"2024-05-08T18:07:18.748532+0000","last_clean":"2024-05-08T18:07:18.748532+0000","last_became_active":"2024-05-08T18:04:49.832148+0000","last_became_peered":"2024-05-08T18:04:49.832148+0000","last_unstale":"2024-05-08T18:07:18.748532+0000","last_undegraded":"2024-05-08T18:07:18.748532+0000","last_fullsized":"2024-05-08T18:07:18.748532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:20:21.388280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5807,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":31,"num_read_kb":25,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866106+0000","last_change":"2024-05-08T18:04:51.845281+0000","last_active":"2024-05-08T18:05:30.866106+0000","last_peered":"2024-05-08T18:05:30.866106+0000","last_clean":"2024-05-08T18:05:30.866106+0000","last_became_active":"2024-05-08T18:04:51.844901+0000","last_became_peered":"2024-05-08T18:04:51.844901+0000","last_unstale":"2024-05-08T18:05:30.866106+0000","last_undegraded":"2024-05-08T18:05:30.866106+0000","last_fullsized":"2024-05-08T18:05:30.866106+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:46.002263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"29'11","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.552238+0000","last_change":"2024-05-08T18:04:53.849933+0000","last_active":"2024-05-08T18:06:09.552238+0000","last_peered":"2024-05-08T18:06:09.552238+0000","last_clean":"2024-05-08T18:06:09.552238+0000","last_became_active":"2024-05-08T18:04:53.849579+0000","last_became_peered":"2024-05-08T18:04:53.849579+0000","last_unstale":"2024-05-08T18:06:09.552238+0000","last_undegraded":"2024-05-08T18:06:09.552238+0000","last_fullsized":"2024-05-08T18:06:09.552238+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:42:45.987685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"29'6","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.064004+0000","last_change":"2024-05-08T18:04:45.826976+0000","last_active":"2024-05-08T18:07:54.064004+0000","last_peered":"2024-05-08T18:07:54.064004+0000","last_clean":"2024-05-08T18:07:54.064004+0000","last_became_active":"2024-05-08T18:04:45.825162+0000","last_became_peered":"2024-05-08T18:04:45.825162+0000","last_unstale":"2024-05-08T18:07:54.064004+0000","last_undegraded":"2024-05-08T18:07:54.064004+0000","last_fullsized":"2024-05-08T18:07:54.064004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:36.088260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862438+0000","last_change":"2024-05-08T18:04:43.827844+0000","last_active":"2024-05-08T18:05:30.862438+0000","last_peered":"2024-05-08T18:05:30.862438+0000","last_clean":"2024-05-08T18:05:30.862438+0000","last_became_active":"2024-05-08T18:04:43.827374+0000","last_became_peered":"2024-05-08T18:04:43.827374+0000","last_unstale":"2024-05-08T18:05:30.862438+0000","last_undegraded":"2024-05-08T18:05:30.862438+0000","last_fullsized":"2024-05-08T18:05:30.862438+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:26.918679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:33.732471+0000","last_change":"2024-05-08T18:05:30.868535+0000","last_active":"2024-05-08T18:05:33.732471+0000","last_peered":"2024-05-08T18:05:33.732471+0000","last_clean":"2024-05-08T18:05:33.732471+0000","last_became_active":"2024-05-08T18:05:30.868191+0000","last_became_peered":"2024-05-08T18:05:30.868191+0000","last_unstale":"2024-05-08T18:05:33.732471+0000","last_undegraded":"2024-05-08T18:05:33.732471+0000","last_fullsized":"2024-05-08T18:05:33.732471+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:19.468033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"27'6","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862185+0000","last_change":"2024-05-08T18:04:49.826617+0000","last_active":"2024-05-08T18:05:30.862185+0000","last_peered":"2024-05-08T18:05:30.862185+0000","last_clean":"2024-05-08T18:05:30.862185+0000","last_became_active":"2024-05-08T18:04:49.825265+0000","last_became_peered":"2024-05-08T18:04:49.825265+0000","last_unstale":"2024-05-08T18:05:30.862185+0000","last_undegraded":"2024-05-08T18:05:30.862185+0000","last_fullsized":"2024-05-08T18:05:30.862185+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:35.867914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862381+0000","last_change":"2024-05-08T18:04:51.834892+0000","last_active":"2024-05-08T18:05:30.862381+0000","last_peered":"2024-05-08T18:05:30.862381+0000","last_clean":"2024-05-08T18:05:30.862381+0000","last_became_active":"2024-05-08T18:04:51.833523+0000","last_became_peered":"2024-05-08T18:04:51.833523+0000","last_unstale":"2024-05-08T18:05:30.862381+0000","last_undegraded":"2024-05-08T18:05:30.862381+0000","last_fullsized":"2024-05-08T18:05:30.862381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:01.176258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'2","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858040+0000","last_change":"2024-05-08T18:04:53.840760+0000","last_active":"2024-05-08T18:05:30.858040+0000","last_peered":"2024-05-08T18:05:30.858040+0000","last_clean":"2024-05-08T18:05:30.858040+0000","last_became_active":"2024-05-08T18:04:53.840395+0000","last_became_peered":"2024-05-08T18:04:53.840395+0000","last_unstale":"2024-05-08T18:05:30.858040+0000","last_undegraded":"2024-05-08T18:05:30.858040+0000","last_fullsized":"2024-05-08T18:05:30.858040+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:54.850231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:33.732213+0000","last_change":"2024-05-08T18:05:30.876470+0000","last_active":"2024-05-08T18:05:33.732213+0000","last_peered":"2024-05-08T18:05:33.732213+0000","last_clean":"2024-05-08T18:05:33.732213+0000","last_became_active":"2024-05-08T18:05:30.869162+0000","last_became_peered":"2024-05-08T18:05:30.869162+0000","last_unstale":"2024-05-08T18:05:33.732213+0000","last_undegraded":"2024-05-08T18:05:33.732213+0000","last_fullsized":"2024-05-08T18:05:33.732213+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:04.889530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":45,"num_write":24,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857600+0000","last_change":"2024-05-08T18:04:43.844847+0000","last_active":"2024-05-08T18:05:30.857600+0000","last_peered":"2024-05-08T18:05:30.857600+0000","last_clean":"2024-05-08T18:05:30.857600+0000","last_became_active":"2024-05-08T18:04:43.838666+0000","last_became_peered":"2024-05-08T18:04:43.838666+0000","last_unstale":"2024-05-08T18:05:30.857600+0000","last_undegraded":"2024-05-08T18:05:30.857600+0000","last_fullsized":"2024-05-08T18:05:30.857600+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:32.496387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861628+0000","last_change":"2024-05-08T18:04:47.819743+0000","last_active":"2024-05-08T18:05:30.861628+0000","last_peered":"2024-05-08T18:05:30.861628+0000","last_clean":"2024-05-08T18:05:30.861628+0000","last_became_active":"2024-05-08T18:04:47.819169+0000","last_became_peered":"2024-05-08T18:04:47.819169+0000","last_unstale":"2024-05-08T18:05:30.861628+0000","last_undegraded":"2024-05-08T18:05:30.861628+0000","last_fullsized":"2024-05-08T18:05:30.861628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:15:57.213130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"29'12","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.746941+0000","last_change":"2024-05-08T18:04:49.831839+0000","last_active":"2024-05-08T18:07:13.746941+0000","last_peered":"2024-05-08T18:07:13.746941+0000","last_clean":"2024-05-08T18:07:13.746941+0000","last_became_active":"2024-05-08T18:04:49.831500+0000","last_became_peered":"2024-05-08T18:04:49.831500+0000","last_unstale":"2024-05-08T18:07:13.746941+0000","last_undegraded":"2024-05-08T18:07:13.746941+0000","last_fullsized":"2024-05-08T18:07:13.746941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:19:01.714463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"29'282","reported_seq":381,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903208+0000","last_change":"2024-05-08T18:04:51.849859+0000","last_active":"2024-05-08T18:07:52.903208+0000","last_peered":"2024-05-08T18:07:52.903208+0000","last_clean":"2024-05-08T18:07:52.903208+0000","last_became_active":"2024-05-08T18:04:51.849301+0000","last_became_peered":"2024-05-08T18:04:51.849301+0000","last_unstale":"2024-05-08T18:07:52.903208+0000","last_undegraded":"2024-05-08T18:07:52.903208+0000","last_fullsized":"2024-05-08T18:07:52.903208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:02:26.756919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862433+0000","last_change":"2024-05-08T18:04:53.840922+0000","last_active":"2024-05-08T18:05:30.862433+0000","last_peered":"2024-05-08T18:05:30.862433+0000","last_clean":"2024-05-08T18:05:30.862433+0000","last_became_active":"2024-05-08T18:04:53.840559+0000","last_became_peered":"2024-05-08T18:04:53.840559+0000","last_unstale":"2024-05-08T18:05:30.862433+0000","last_undegraded":"2024-05-08T18:05:30.862433+0000","last_fullsized":"2024-05-08T18:05:30.862433+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:24.814022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"29'6","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.035260+0000","last_change":"2024-05-08T18:04:45.812045+0000","last_active":"2024-05-08T18:07:54.035260+0000","last_peered":"2024-05-08T18:07:54.035260+0000","last_clean":"2024-05-08T18:07:54.035260+0000","last_became_active":"2024-05-08T18:04:45.811570+0000","last_became_peered":"2024-05-08T18:04:45.811570+0000","last_unstale":"2024-05-08T18:07:54.035260+0000","last_undegraded":"2024-05-08T18:07:54.035260+0000","last_fullsized":"2024-05-08T18:07:54.035260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:22:19.585204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.987908+0000","last_change":"2024-05-08T18:04:43.838356+0000","last_active":"2024-05-08T18:07:53.987908+0000","last_peered":"2024-05-08T18:07:53.987908+0000","last_clean":"2024-05-08T18:07:53.987908+0000","last_became_active":"2024-05-08T18:04:43.836410+0000","last_became_peered":"2024-05-08T18:04:43.836410+0000","last_unstale":"2024-05-08T18:07:53.987908+0000","last_undegraded":"2024-05-08T18:07:53.987908+0000","last_fullsized":"2024-05-08T18:07:53.987908+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:59:26.742382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860150+0000","last_change":"2024-05-08T18:04:47.819091+0000","last_active":"2024-05-08T18:05:30.860150+0000","last_peered":"2024-05-08T18:05:30.860150+0000","last_clean":"2024-05-08T18:05:30.860150+0000","last_became_active":"2024-05-08T18:04:47.818797+0000","last_became_peered":"2024-05-08T18:04:47.818797+0000","last_unstale":"2024-05-08T18:05:30.860150+0000","last_undegraded":"2024-05-08T18:05:30.860150+0000","last_fullsized":"2024-05-08T18:05:30.860150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:44.738079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"29'31","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.948644+0000","last_change":"2024-05-08T18:04:49.831750+0000","last_active":"2024-05-08T18:06:51.948644+0000","last_peered":"2024-05-08T18:06:51.948644+0000","last_clean":"2024-05-08T18:06:51.948644+0000","last_became_active":"2024-05-08T18:04:49.831379+0000","last_became_peered":"2024-05-08T18:04:49.831379+0000","last_unstale":"2024-05-08T18:06:51.948644+0000","last_undegraded":"2024-05-08T18:06:51.948644+0000","last_fullsized":"2024-05-08T18:06:51.948644+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:15:59.220647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"29'282","reported_seq":441,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.067044+0000","last_change":"2024-05-08T18:04:51.827294+0000","last_active":"2024-05-08T18:07:54.067044+0000","last_peered":"2024-05-08T18:07:54.067044+0000","last_clean":"2024-05-08T18:07:54.067044+0000","last_became_active":"2024-05-08T18:04:51.826926+0000","last_became_peered":"2024-05-08T18:04:51.826926+0000","last_unstale":"2024-05-08T18:07:54.067044+0000","last_undegraded":"2024-05-08T18:07:54.067044+0000","last_fullsized":"2024-05-08T18:07:54.067044+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:00:59.786865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'6","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858148+0000","last_change":"2024-05-08T18:04:45.827616+0000","last_active":"2024-05-08T18:05:30.858148+0000","last_peered":"2024-05-08T18:05:30.858148+0000","last_clean":"2024-05-08T18:05:30.858148+0000","last_became_active":"2024-05-08T18:04:45.827351+0000","last_became_peered":"2024-05-08T18:04:45.827351+0000","last_unstale":"2024-05-08T18:05:30.858148+0000","last_undegraded":"2024-05-08T18:05:30.858148+0000","last_fullsized":"2024-05-08T18:05:30.858148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:31.853297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866835+0000","last_change":"2024-05-08T18:04:43.839205+0000","last_active":"2024-05-08T18:05:30.866835+0000","last_peered":"2024-05-08T18:05:30.866835+0000","last_clean":"2024-05-08T18:05:30.866835+0000","last_became_active":"2024-05-08T18:04:43.838840+0000","last_became_peered":"2024-05-08T18:04:43.838840+0000","last_unstale":"2024-05-08T18:05:30.866835+0000","last_undegraded":"2024-05-08T18:05:30.866835+0000","last_fullsized":"2024-05-08T18:05:30.866835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:50.043665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,0],"acting":[2,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861049+0000","last_change":"2024-05-08T18:04:47.825571+0000","last_active":"2024-05-08T18:05:30.861049+0000","last_peered":"2024-05-08T18:05:30.861049+0000","last_clean":"2024-05-08T18:05:30.861049+0000","last_became_active":"2024-05-08T18:04:47.825215+0000","last_became_peered":"2024-05-08T18:04:47.825215+0000","last_unstale":"2024-05-08T18:05:30.861049+0000","last_undegraded":"2024-05-08T18:05:30.861049+0000","last_fullsized":"2024-05-08T18:05:30.861049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:47.750530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"29'40","reported_seq":87,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.957612+0000","last_change":"2024-05-08T18:04:49.828257+0000","last_active":"2024-05-08T18:06:51.957612+0000","last_peered":"2024-05-08T18:06:51.957612+0000","last_clean":"2024-05-08T18:06:51.957612+0000","last_became_active":"2024-05-08T18:04:49.827954+0000","last_became_peered":"2024-05-08T18:04:49.827954+0000","last_unstale":"2024-05-08T18:06:51.957612+0000","last_undegraded":"2024-05-08T18:06:51.957612+0000","last_fullsized":"2024-05-08T18:06:51.957612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:43.565272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859378+0000","last_change":"2024-05-08T18:04:51.831489+0000","last_active":"2024-05-08T18:05:30.859378+0000","last_peered":"2024-05-08T18:05:30.859378+0000","last_clean":"2024-05-08T18:05:30.859378+0000","last_became_active":"2024-05-08T18:04:51.831149+0000","last_became_peered":"2024-05-08T18:04:51.831149+0000","last_unstale":"2024-05-08T18:05:30.859378+0000","last_undegraded":"2024-05-08T18:05:30.859378+0000","last_fullsized":"2024-05-08T18:05:30.859378+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:59.859356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"29'5","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.735436+0000","last_change":"2024-05-08T18:04:53.840960+0000","last_active":"2024-05-08T18:06:09.735436+0000","last_peered":"2024-05-08T18:06:09.735436+0000","last_clean":"2024-05-08T18:06:09.735436+0000","last_became_active":"2024-05-08T18:04:53.840652+0000","last_became_peered":"2024-05-08T18:04:53.840652+0000","last_unstale":"2024-05-08T18:06:09.735436+0000","last_undegraded":"2024-05-08T18:06:09.735436+0000","last_fullsized":"2024-05-08T18:06:09.735436+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:02:50.273794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"27'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861806+0000","last_change":"2024-05-08T18:04:45.804412+0000","last_active":"2024-05-08T18:05:30.861806+0000","last_peered":"2024-05-08T18:05:30.861806+0000","last_clean":"2024-05-08T18:05:30.861806+0000","last_became_active":"2024-05-08T18:04:45.804124+0000","last_became_peered":"2024-05-08T18:04:45.804124+0000","last_unstale":"2024-05-08T18:05:30.861806+0000","last_undegraded":"2024-05-08T18:05:30.861806+0000","last_fullsized":"2024-05-08T18:05:30.861806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:08.184602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867328+0000","last_change":"2024-05-08T18:04:43.843177+0000","last_active":"2024-05-08T18:05:30.867328+0000","last_peered":"2024-05-08T18:05:30.867328+0000","last_clean":"2024-05-08T18:05:30.867328+0000","last_became_active":"2024-05-08T18:04:43.842126+0000","last_became_peered":"2024-05-08T18:04:43.842126+0000","last_unstale":"2024-05-08T18:05:30.867328+0000","last_undegraded":"2024-05-08T18:05:30.867328+0000","last_fullsized":"2024-05-08T18:05:30.867328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:41:01.395720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858650+0000","last_change":"2024-05-08T18:04:47.823007+0000","last_active":"2024-05-08T18:05:30.858650+0000","last_peered":"2024-05-08T18:05:30.858650+0000","last_clean":"2024-05-08T18:05:30.858650+0000","last_became_active":"2024-05-08T18:04:47.822688+0000","last_became_peered":"2024-05-08T18:04:47.822688+0000","last_unstale":"2024-05-08T18:05:30.858650+0000","last_undegraded":"2024-05-08T18:05:30.858650+0000","last_fullsized":"2024-05-08T18:05:30.858650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:32.662502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"29'30","reported_seq":72,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.938361+0000","last_change":"2024-05-08T18:04:49.832010+0000","last_active":"2024-05-08T18:06:51.938361+0000","last_peered":"2024-05-08T18:06:51.938361+0000","last_clean":"2024-05-08T18:06:51.938361+0000","last_became_active":"2024-05-08T18:04:49.829892+0000","last_became_peered":"2024-05-08T18:04:49.829892+0000","last_unstale":"2024-05-08T18:06:51.938361+0000","last_undegraded":"2024-05-08T18:06:51.938361+0000","last_fullsized":"2024-05-08T18:06:51.938361+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:13:16.535751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858848+0000","last_change":"2024-05-08T18:04:51.833183+0000","last_active":"2024-05-08T18:05:30.858848+0000","last_peered":"2024-05-08T18:05:30.858848+0000","last_clean":"2024-05-08T18:05:30.858848+0000","last_became_active":"2024-05-08T18:04:51.832810+0000","last_became_peered":"2024-05-08T18:04:51.832810+0000","last_unstale":"2024-05-08T18:05:30.858848+0000","last_undegraded":"2024-05-08T18:05:30.858848+0000","last_fullsized":"2024-05-08T18:05:30.858848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:02.130044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"29'3","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:03.972536+0000","last_change":"2024-05-08T18:04:53.841201+0000","last_active":"2024-05-08T18:06:03.972536+0000","last_peered":"2024-05-08T18:06:03.972536+0000","last_clean":"2024-05-08T18:06:03.972536+0000","last_became_active":"2024-05-08T18:04:53.840894+0000","last_became_peered":"2024-05-08T18:04:53.840894+0000","last_unstale":"2024-05-08T18:06:03.972536+0000","last_undegraded":"2024-05-08T18:06:03.972536+0000","last_fullsized":"2024-05-08T18:06:03.972536+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:22:10.520507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"29'9","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.867093+0000","last_change":"2024-05-08T18:04:45.817096+0000","last_active":"2024-05-08T18:07:53.867093+0000","last_peered":"2024-05-08T18:07:53.867093+0000","last_clean":"2024-05-08T18:07:53.867093+0000","last_became_active":"2024-05-08T18:04:45.816817+0000","last_became_peered":"2024-05-08T18:04:45.816817+0000","last_unstale":"2024-05-08T18:07:53.867093+0000","last_undegraded":"2024-05-08T18:07:53.867093+0000","last_fullsized":"2024-05-08T18:07:53.867093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:48.747375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"29'3","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.984852+0000","last_change":"2024-05-08T18:04:43.844652+0000","last_active":"2024-05-08T18:07:53.984852+0000","last_peered":"2024-05-08T18:07:53.984852+0000","last_clean":"2024-05-08T18:07:53.984852+0000","last_became_active":"2024-05-08T18:04:43.838781+0000","last_became_peered":"2024-05-08T18:04:43.838781+0000","last_unstale":"2024-05-08T18:07:53.984852+0000","last_undegraded":"2024-05-08T18:07:53.984852+0000","last_fullsized":"2024-05-08T18:07:53.984852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:22:17.577945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":668544,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858935+0000","last_change":"2024-05-08T18:04:47.819192+0000","last_active":"2024-05-08T18:05:30.858935+0000","last_peered":"2024-05-08T18:05:30.858935+0000","last_clean":"2024-05-08T18:05:30.858935+0000","last_became_active":"2024-05-08T18:04:47.818871+0000","last_became_peered":"2024-05-08T18:04:47.818871+0000","last_unstale":"2024-05-08T18:05:30.858935+0000","last_undegraded":"2024-05-08T18:05:30.858935+0000","last_fullsized":"2024-05-08T18:05:30.858935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:20.855010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"29'36","reported_seq":81,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.959548+0000","last_change":"2024-05-08T18:04:49.829583+0000","last_active":"2024-05-08T18:06:51.959548+0000","last_peered":"2024-05-08T18:06:51.959548+0000","last_clean":"2024-05-08T18:06:51.959548+0000","last_became_active":"2024-05-08T18:04:49.829270+0000","last_became_peered":"2024-05-08T18:04:49.829270+0000","last_unstale":"2024-05-08T18:06:51.959548+0000","last_undegraded":"2024-05-08T18:06:51.959548+0000","last_fullsized":"2024-05-08T18:06:51.959548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:11:17.784122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"29'282","reported_seq":348,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.181903+0000","last_change":"2024-05-08T18:04:51.845064+0000","last_active":"2024-05-08T18:07:54.181903+0000","last_peered":"2024-05-08T18:07:54.181903+0000","last_clean":"2024-05-08T18:07:54.181903+0000","last_became_active":"2024-05-08T18:04:51.844529+0000","last_became_peered":"2024-05-08T18:04:51.844529+0000","last_unstale":"2024-05-08T18:07:54.181903+0000","last_undegraded":"2024-05-08T18:07:54.181903+0000","last_fullsized":"2024-05-08T18:07:54.181903+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:46.002263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"29'2","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.862719+0000","last_change":"2024-05-08T18:04:53.841062+0000","last_active":"2024-05-08T18:07:53.862719+0000","last_peered":"2024-05-08T18:07:53.862719+0000","last_clean":"2024-05-08T18:07:53.862719+0000","last_became_active":"2024-05-08T18:04:53.840767+0000","last_became_peered":"2024-05-08T18:04:53.840767+0000","last_unstale":"2024-05-08T18:07:53.862719+0000","last_undegraded":"2024-05-08T18:07:53.862719+0000","last_fullsized":"2024-05-08T18:07:53.862719+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:54:38.757755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"29'6","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995942+0000","last_change":"2024-05-08T18:04:45.814276+0000","last_active":"2024-05-08T18:07:53.995942+0000","last_peered":"2024-05-08T18:07:53.995942+0000","last_clean":"2024-05-08T18:07:53.995942+0000","last_became_active":"2024-05-08T18:04:45.813735+0000","last_became_peered":"2024-05-08T18:04:45.813735+0000","last_unstale":"2024-05-08T18:07:53.995942+0000","last_undegraded":"2024-05-08T18:07:53.995942+0000","last_fullsized":"2024-05-08T18:07:53.995942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:18.463624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"27'4","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859087+0000","last_change":"2024-05-08T18:04:43.848140+0000","last_active":"2024-05-08T18:05:30.859087+0000","last_peered":"2024-05-08T18:05:30.859087+0000","last_clean":"2024-05-08T18:05:30.859087+0000","last_became_active":"2024-05-08T18:04:43.839047+0000","last_became_peered":"2024-05-08T18:04:43.839047+0000","last_unstale":"2024-05-08T18:05:30.859087+0000","last_undegraded":"2024-05-08T18:05:30.859087+0000","last_fullsized":"2024-05-08T18:05:30.859087+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:24.945310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858315+0000","last_change":"2024-05-08T18:04:47.819319+0000","last_active":"2024-05-08T18:05:30.858315+0000","last_peered":"2024-05-08T18:05:30.858315+0000","last_clean":"2024-05-08T18:05:30.858315+0000","last_became_active":"2024-05-08T18:04:47.818972+0000","last_became_peered":"2024-05-08T18:04:47.818972+0000","last_unstale":"2024-05-08T18:05:30.858315+0000","last_undegraded":"2024-05-08T18:05:30.858315+0000","last_fullsized":"2024-05-08T18:05:30.858315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:09:51.843752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"29'54","reported_seq":110,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.961509+0000","last_change":"2024-05-08T18:04:49.832104+0000","last_active":"2024-05-08T18:06:51.961509+0000","last_peered":"2024-05-08T18:06:51.961509+0000","last_clean":"2024-05-08T18:06:51.961509+0000","last_became_active":"2024-05-08T18:04:49.830424+0000","last_became_peered":"2024-05-08T18:04:49.830424+0000","last_unstale":"2024-05-08T18:06:51.961509+0000","last_undegraded":"2024-05-08T18:06:51.961509+0000","last_fullsized":"2024-05-08T18:06:51.961509+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:50.646336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858392+0000","last_change":"2024-05-08T18:04:51.830631+0000","last_active":"2024-05-08T18:05:30.858392+0000","last_peered":"2024-05-08T18:05:30.858392+0000","last_clean":"2024-05-08T18:05:30.858392+0000","last_became_active":"2024-05-08T18:04:51.830383+0000","last_became_peered":"2024-05-08T18:04:51.830383+0000","last_unstale":"2024-05-08T18:05:30.858392+0000","last_undegraded":"2024-05-08T18:05:30.858392+0000","last_fullsized":"2024-05-08T18:05:30.858392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:01.176258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"29'20","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.158011+0000","last_change":"2024-05-08T18:04:53.848105+0000","last_active":"2024-05-08T18:07:54.158011+0000","last_peered":"2024-05-08T18:07:54.158011+0000","last_clean":"2024-05-08T18:07:54.158011+0000","last_became_active":"2024-05-08T18:04:53.847842+0000","last_became_peered":"2024-05-08T18:04:53.847842+0000","last_unstale":"2024-05-08T18:07:54.158011+0000","last_undegraded":"2024-05-08T18:07:54.158011+0000","last_fullsized":"2024-05-08T18:07:54.158011+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:03:57.205665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":48,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"29'6","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.064006+0000","last_change":"2024-05-08T18:04:45.821729+0000","last_active":"2024-05-08T18:07:54.064006+0000","last_peered":"2024-05-08T18:07:54.064006+0000","last_clean":"2024-05-08T18:07:54.064006+0000","last_became_active":"2024-05-08T18:04:45.821412+0000","last_became_peered":"2024-05-08T18:04:45.821412+0000","last_unstale":"2024-05-08T18:07:54.064006+0000","last_undegraded":"2024-05-08T18:07:54.064006+0000","last_fullsized":"2024-05-08T18:07:54.064006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:26.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859201+0000","last_change":"2024-05-08T18:04:43.835922+0000","last_active":"2024-05-08T18:05:30.859201+0000","last_peered":"2024-05-08T18:05:30.859201+0000","last_clean":"2024-05-08T18:05:30.859201+0000","last_became_active":"2024-05-08T18:04:43.834723+0000","last_became_peered":"2024-05-08T18:04:43.834723+0000","last_unstale":"2024-05-08T18:05:30.859201+0000","last_undegraded":"2024-05-08T18:05:30.859201+0000","last_fullsized":"2024-05-08T18:05:30.859201+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:19.700373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864185+0000","last_change":"2024-05-08T18:04:47.823171+0000","last_active":"2024-05-08T18:05:30.864185+0000","last_peered":"2024-05-08T18:05:30.864185+0000","last_clean":"2024-05-08T18:05:30.864185+0000","last_became_active":"2024-05-08T18:04:47.822865+0000","last_became_peered":"2024-05-08T18:04:47.822865+0000","last_unstale":"2024-05-08T18:05:30.864185+0000","last_undegraded":"2024-05-08T18:05:30.864185+0000","last_fullsized":"2024-05-08T18:05:30.864185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:22:57.464984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"29'33","reported_seq":76,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.942615+0000","last_change":"2024-05-08T18:04:49.824918+0000","last_active":"2024-05-08T18:06:51.942615+0000","last_peered":"2024-05-08T18:06:51.942615+0000","last_clean":"2024-05-08T18:06:51.942615+0000","last_became_active":"2024-05-08T18:04:49.824610+0000","last_became_peered":"2024-05-08T18:04:49.824610+0000","last_unstale":"2024-05-08T18:06:51.942615+0000","last_undegraded":"2024-05-08T18:06:51.942615+0000","last_fullsized":"2024-05-08T18:06:51.942615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:56:27.132432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864191+0000","last_change":"2024-05-08T18:04:51.849355+0000","last_active":"2024-05-08T18:05:30.864191+0000","last_peered":"2024-05-08T18:05:30.864191+0000","last_clean":"2024-05-08T18:05:30.864191+0000","last_became_active":"2024-05-08T18:04:51.848793+0000","last_became_peered":"2024-05-08T18:04:51.848793+0000","last_unstale":"2024-05-08T18:05:30.864191+0000","last_undegraded":"2024-05-08T18:05:30.864191+0000","last_fullsized":"2024-05-08T18:05:30.864191+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:40:15.186526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"29'5","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744070+0000","last_change":"2024-05-08T18:04:53.849242+0000","last_active":"2024-05-08T18:06:53.744070+0000","last_peered":"2024-05-08T18:06:53.744070+0000","last_clean":"2024-05-08T18:06:53.744070+0000","last_became_active":"2024-05-08T18:04:53.848945+0000","last_became_peered":"2024-05-08T18:04:53.848945+0000","last_unstale":"2024-05-08T18:06:53.744070+0000","last_undegraded":"2024-05-08T18:06:53.744070+0000","last_fullsized":"2024-05-08T18:06:53.744070+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:00.234436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"29'10","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.772802+0000","last_change":"2024-05-08T18:04:45.829691+0000","last_active":"2024-05-08T18:07:53.772802+0000","last_peered":"2024-05-08T18:07:53.772802+0000","last_clean":"2024-05-08T18:07:53.772802+0000","last_became_active":"2024-05-08T18:04:45.829120+0000","last_became_peered":"2024-05-08T18:04:45.829120+0000","last_unstale":"2024-05-08T18:07:53.772802+0000","last_undegraded":"2024-05-08T18:07:53.772802+0000","last_fullsized":"2024-05-08T18:07:53.772802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:29.119368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":57,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859093+0000","last_change":"2024-05-08T18:04:43.835329+0000","last_active":"2024-05-08T18:05:30.859093+0000","last_peered":"2024-05-08T18:05:30.859093+0000","last_clean":"2024-05-08T18:05:30.859093+0000","last_became_active":"2024-05-08T18:04:43.834536+0000","last_became_peered":"2024-05-08T18:04:43.834536+0000","last_unstale":"2024-05-08T18:05:30.859093+0000","last_undegraded":"2024-05-08T18:05:30.859093+0000","last_fullsized":"2024-05-08T18:05:30.859093+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:56.567490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"29'4","reported_seq":226,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.038721+0000","last_change":"2024-05-08T18:04:47.819789+0000","last_active":"2024-05-08T18:06:53.038721+0000","last_peered":"2024-05-08T18:06:53.038721+0000","last_clean":"2024-05-08T18:06:53.038721+0000","last_became_active":"2024-05-08T18:04:47.819085+0000","last_became_peered":"2024-05-08T18:04:47.819085+0000","last_unstale":"2024-05-08T18:06:53.038721+0000","last_undegraded":"2024-05-08T18:06:53.038721+0000","last_fullsized":"2024-05-08T18:06:53.038721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:24.812721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":298,"num_read_kb":298,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"29'36","reported_seq":81,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.967858+0000","last_change":"2024-05-08T18:04:49.825997+0000","last_active":"2024-05-08T18:06:51.967858+0000","last_peered":"2024-05-08T18:06:51.967858+0000","last_clean":"2024-05-08T18:06:51.967858+0000","last_became_active":"2024-05-08T18:04:49.825613+0000","last_became_peered":"2024-05-08T18:04:49.825613+0000","last_unstale":"2024-05-08T18:06:51.967858+0000","last_undegraded":"2024-05-08T18:06:51.967858+0000","last_fullsized":"2024-05-08T18:06:51.967858+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:49.758047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"29'282","reported_seq":431,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.162304+0000","last_change":"2024-05-08T18:04:51.831317+0000","last_active":"2024-05-08T18:07:54.162304+0000","last_peered":"2024-05-08T18:07:54.162304+0000","last_clean":"2024-05-08T18:07:54.162304+0000","last_became_active":"2024-05-08T18:04:51.830924+0000","last_became_peered":"2024-05-08T18:04:51.830924+0000","last_unstale":"2024-05-08T18:07:54.162304+0000","last_undegraded":"2024-05-08T18:07:54.162304+0000","last_fullsized":"2024-05-08T18:07:54.162304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:23.880081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"29'14","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.739659+0000","last_change":"2024-05-08T18:04:53.841349+0000","last_active":"2024-05-08T18:06:58.739659+0000","last_peered":"2024-05-08T18:06:58.739659+0000","last_clean":"2024-05-08T18:06:58.739659+0000","last_became_active":"2024-05-08T18:04:53.840708+0000","last_became_peered":"2024-05-08T18:04:53.840708+0000","last_unstale":"2024-05-08T18:06:58.739659+0000","last_undegraded":"2024-05-08T18:06:58.739659+0000","last_fullsized":"2024-05-08T18:06:58.739659+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:57:33.051338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063870+0000","last_change":"2024-05-08T18:04:45.812281+0000","last_active":"2024-05-08T18:07:54.063870+0000","last_peered":"2024-05-08T18:07:54.063870+0000","last_clean":"2024-05-08T18:07:54.063870+0000","last_became_active":"2024-05-08T18:04:45.811926+0000","last_became_peered":"2024-05-08T18:04:45.811926+0000","last_unstale":"2024-05-08T18:07:54.063870+0000","last_undegraded":"2024-05-08T18:07:54.063870+0000","last_fullsized":"2024-05-08T18:07:54.063870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:26.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"29'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.728795+0000","last_change":"2024-05-08T18:04:43.844418+0000","last_active":"2024-05-08T18:06:09.728795+0000","last_peered":"2024-05-08T18:06:09.728795+0000","last_clean":"2024-05-08T18:06:09.728795+0000","last_became_active":"2024-05-08T18:04:43.843934+0000","last_became_peered":"2024-05-08T18:04:43.843934+0000","last_unstale":"2024-05-08T18:06:09.728795+0000","last_undegraded":"2024-05-08T18:06:09.728795+0000","last_fullsized":"2024-05-08T18:06:09.728795+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:30.825989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"29'3","reported_seq":413,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.049002+0000","last_change":"2024-05-08T18:04:47.818222+0000","last_active":"2024-05-08T18:06:53.049002+0000","last_peered":"2024-05-08T18:06:53.049002+0000","last_clean":"2024-05-08T18:06:53.049002+0000","last_became_active":"2024-05-08T18:04:47.817851+0000","last_became_peered":"2024-05-08T18:04:47.817851+0000","last_unstale":"2024-05-08T18:06:53.049002+0000","last_undegraded":"2024-05-08T18:06:53.049002+0000","last_fullsized":"2024-05-08T18:06:53.049002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:45:55.932454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":381,"num_read_kb":381,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"27'10","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863960+0000","last_change":"2024-05-08T18:04:49.832191+0000","last_active":"2024-05-08T18:05:30.863960+0000","last_peered":"2024-05-08T18:05:30.863960+0000","last_clean":"2024-05-08T18:05:30.863960+0000","last_became_active":"2024-05-08T18:04:49.830714+0000","last_became_peered":"2024-05-08T18:04:49.830714+0000","last_unstale":"2024-05-08T18:05:30.863960+0000","last_undegraded":"2024-05-08T18:05:30.863960+0000","last_fullsized":"2024-05-08T18:05:30.863960+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:14:01.011620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"29'282","reported_seq":468,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.120234+0000","last_change":"2024-05-08T18:04:51.832589+0000","last_active":"2024-05-08T18:07:54.120234+0000","last_peered":"2024-05-08T18:07:54.120234+0000","last_clean":"2024-05-08T18:07:54.120234+0000","last_became_active":"2024-05-08T18:04:51.831508+0000","last_became_peered":"2024-05-08T18:04:51.831508+0000","last_unstale":"2024-05-08T18:07:54.120234+0000","last_undegraded":"2024-05-08T18:07:54.120234+0000","last_fullsized":"2024-05-08T18:07:54.120234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:15.559137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'4","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.179451+0000","last_change":"2024-05-08T18:04:53.840524+0000","last_active":"2024-05-08T18:06:53.179451+0000","last_peered":"2024-05-08T18:06:53.179451+0000","last_clean":"2024-05-08T18:06:53.179451+0000","last_became_active":"2024-05-08T18:04:53.840215+0000","last_became_peered":"2024-05-08T18:04:53.840215+0000","last_unstale":"2024-05-08T18:06:53.179451+0000","last_undegraded":"2024-05-08T18:06:53.179451+0000","last_fullsized":"2024-05-08T18:06:53.179451+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:39.550898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'2","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866626+0000","last_change":"2024-05-08T18:04:45.819074+0000","last_active":"2024-05-08T18:05:30.866626+0000","last_peered":"2024-05-08T18:05:30.866626+0000","last_clean":"2024-05-08T18:05:30.866626+0000","last_became_active":"2024-05-08T18:04:45.818533+0000","last_became_peered":"2024-05-08T18:04:45.818533+0000","last_unstale":"2024-05-08T18:05:30.866626+0000","last_undegraded":"2024-05-08T18:05:30.866626+0000","last_fullsized":"2024-05-08T18:05:30.866626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:34.419882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"29'1","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744105+0000","last_change":"2024-05-08T18:04:43.848496+0000","last_active":"2024-05-08T18:06:53.744105+0000","last_peered":"2024-05-08T18:06:53.744105+0000","last_clean":"2024-05-08T18:06:53.744105+0000","last_became_active":"2024-05-08T18:04:43.846125+0000","last_became_peered":"2024-05-08T18:04:43.846125+0000","last_unstale":"2024-05-08T18:06:53.744105+0000","last_undegraded":"2024-05-08T18:06:53.744105+0000","last_fullsized":"2024-05-08T18:06:53.744105+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:32.125079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861388+0000","last_change":"2024-05-08T18:04:47.822964+0000","last_active":"2024-05-08T18:05:30.861388+0000","last_peered":"2024-05-08T18:05:30.861388+0000","last_clean":"2024-05-08T18:05:30.861388+0000","last_became_active":"2024-05-08T18:04:47.822242+0000","last_became_peered":"2024-05-08T18:04:47.822242+0000","last_unstale":"2024-05-08T18:05:30.861388+0000","last_undegraded":"2024-05-08T18:05:30.861388+0000","last_fullsized":"2024-05-08T18:05:30.861388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:55.845147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"29'38","reported_seq":84,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.954827+0000","last_change":"2024-05-08T18:04:49.826706+0000","last_active":"2024-05-08T18:06:51.954827+0000","last_peered":"2024-05-08T18:06:51.954827+0000","last_clean":"2024-05-08T18:06:51.954827+0000","last_became_active":"2024-05-08T18:04:49.825314+0000","last_became_peered":"2024-05-08T18:04:49.825314+0000","last_unstale":"2024-05-08T18:06:51.954827+0000","last_undegraded":"2024-05-08T18:06:51.954827+0000","last_fullsized":"2024-05-08T18:06:51.954827+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:21:18.445670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862981+0000","last_change":"2024-05-08T18:04:51.833093+0000","last_active":"2024-05-08T18:05:30.862981+0000","last_peered":"2024-05-08T18:05:30.862981+0000","last_clean":"2024-05-08T18:05:30.862981+0000","last_became_active":"2024-05-08T18:04:51.832808+0000","last_became_peered":"2024-05-08T18:04:51.832808+0000","last_unstale":"2024-05-08T18:05:30.862981+0000","last_undegraded":"2024-05-08T18:05:30.862981+0000","last_fullsized":"2024-05-08T18:05:30.862981+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:54.955755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"29'8","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.096584+0000","last_change":"2024-05-08T18:04:53.837583+0000","last_active":"2024-05-08T18:07:54.096584+0000","last_peered":"2024-05-08T18:07:54.096584+0000","last_clean":"2024-05-08T18:07:54.096584+0000","last_became_active":"2024-05-08T18:04:53.837292+0000","last_became_peered":"2024-05-08T18:04:53.837292+0000","last_unstale":"2024-05-08T18:07:54.096584+0000","last_undegraded":"2024-05-08T18:07:54.096584+0000","last_fullsized":"2024-05-08T18:07:54.096584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:15.652477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'4","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.035838+0000","last_change":"2024-05-08T18:04:45.829568+0000","last_active":"2024-05-08T18:07:54.035838+0000","last_peered":"2024-05-08T18:07:54.035838+0000","last_clean":"2024-05-08T18:07:54.035838+0000","last_became_active":"2024-05-08T18:04:45.828674+0000","last_became_peered":"2024-05-08T18:04:45.828674+0000","last_unstale":"2024-05-08T18:07:54.035838+0000","last_undegraded":"2024-05-08T18:07:54.035838+0000","last_fullsized":"2024-05-08T18:07:54.035838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:33.440280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":137,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859322+0000","last_change":"2024-05-08T18:04:35.552002+0000","last_active":"2024-05-08T18:05:30.859322+0000","last_peered":"2024-05-08T18:05:30.859322+0000","last_clean":"2024-05-08T18:05:30.859322+0000","last_became_active":"2024-05-08T18:04:35.551437+0000","last_became_peered":"2024-05-08T18:04:35.551437+0000","last_unstale":"2024-05-08T18:05:30.859322+0000","last_undegraded":"2024-05-08T18:05:30.859322+0000","last_fullsized":"2024-05-08T18:05:30.859322+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:58.749794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859862+0000","last_change":"2024-05-08T18:04:35.552341+0000","last_active":"2024-05-08T18:05:30.859862+0000","last_peered":"2024-05-08T18:05:30.859862+0000","last_clean":"2024-05-08T18:05:30.859862+0000","last_became_active":"2024-05-08T18:04:35.552017+0000","last_became_peered":"2024-05-08T18:04:35.552017+0000","last_unstale":"2024-05-08T18:05:30.859862+0000","last_undegraded":"2024-05-08T18:05:30.859862+0000","last_fullsized":"2024-05-08T18:05:30.859862+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:30.242269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858503+0000","last_change":"2024-05-08T18:04:43.837405+0000","last_active":"2024-05-08T18:05:30.858503+0000","last_peered":"2024-05-08T18:05:30.858503+0000","last_clean":"2024-05-08T18:05:30.858503+0000","last_became_active":"2024-05-08T18:04:43.837019+0000","last_became_peered":"2024-05-08T18:04:43.837019+0000","last_unstale":"2024-05-08T18:05:30.858503+0000","last_undegraded":"2024-05-08T18:05:30.858503+0000","last_fullsized":"2024-05-08T18:05:30.858503+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:42:03.247527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.871817+0000","last_change":"2024-05-08T18:05:30.871131+0000","last_active":"2024-05-08T18:05:30.871817+0000","last_peered":"2024-05-08T18:05:30.871817+0000","last_clean":"2024-05-08T18:05:30.871817+0000","last_became_active":"2024-05-08T18:05:30.870647+0000","last_became_peered":"2024-05-08T18:05:30.870647+0000","last_unstale":"2024-05-08T18:05:30.871817+0000","last_undegraded":"2024-05-08T18:05:30.871817+0000","last_fullsized":"2024-05-08T18:05:30.871817+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:48.638819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"29'77","reported_seq":146,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.965816+0000","last_change":"2024-05-08T18:04:49.832374+0000","last_active":"2024-05-08T18:06:51.965816+0000","last_peered":"2024-05-08T18:06:51.965816+0000","last_clean":"2024-05-08T18:06:51.965816+0000","last_became_active":"2024-05-08T18:04:49.830636+0000","last_became_peered":"2024-05-08T18:04:49.830636+0000","last_unstale":"2024-05-08T18:06:51.965816+0000","last_undegraded":"2024-05-08T18:06:51.965816+0000","last_fullsized":"2024-05-08T18:06:51.965816+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:23:12.330689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859612+0000","last_change":"2024-05-08T18:04:51.845249+0000","last_active":"2024-05-08T18:05:30.859612+0000","last_peered":"2024-05-08T18:05:30.859612+0000","last_clean":"2024-05-08T18:05:30.859612+0000","last_became_active":"2024-05-08T18:04:51.844737+0000","last_became_peered":"2024-05-08T18:04:51.844737+0000","last_unstale":"2024-05-08T18:05:30.859612+0000","last_undegraded":"2024-05-08T18:05:30.859612+0000","last_fullsized":"2024-05-08T18:05:30.859612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:08.922575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"29'85","reported_seq":166,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.117787+0000","last_change":"2024-05-08T18:04:53.839334+0000","last_active":"2024-05-08T18:07:54.117787+0000","last_peered":"2024-05-08T18:07:54.117787+0000","last_clean":"2024-05-08T18:07:54.117787+0000","last_became_active":"2024-05-08T18:04:53.838957+0000","last_became_peered":"2024-05-08T18:04:53.838957+0000","last_unstale":"2024-05-08T18:07:54.117787+0000","last_undegraded":"2024-05-08T18:07:54.117787+0000","last_fullsized":"2024-05-08T18:07:54.117787+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:25:57.828734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":216,"num_read_kb":214,"num_write":167,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"29'9","reported_seq":65,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070594+0000","last_change":"2024-05-08T18:04:45.826087+0000","last_active":"2024-05-08T18:07:54.070594+0000","last_peered":"2024-05-08T18:07:54.070594+0000","last_clean":"2024-05-08T18:07:54.070594+0000","last_became_active":"2024-05-08T18:04:45.825843+0000","last_became_peered":"2024-05-08T18:04:45.825843+0000","last_unstale":"2024-05-08T18:07:54.070594+0000","last_undegraded":"2024-05-08T18:07:54.070594+0000","last_fullsized":"2024-05-08T18:07:54.070594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:21:14.431053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857318+0000","last_change":"2024-05-08T18:04:35.552553+0000","last_active":"2024-05-08T18:05:30.857318+0000","last_peered":"2024-05-08T18:05:30.857318+0000","last_clean":"2024-05-08T18:05:30.857318+0000","last_became_active":"2024-05-08T18:04:35.552214+0000","last_became_peered":"2024-05-08T18:04:35.552214+0000","last_unstale":"2024-05-08T18:05:30.857318+0000","last_undegraded":"2024-05-08T18:05:30.857318+0000","last_fullsized":"2024-05-08T18:05:30.857318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:12:16.476925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858237+0000","last_change":"2024-05-08T18:04:43.821058+0000","last_active":"2024-05-08T18:05:30.858237+0000","last_peered":"2024-05-08T18:05:30.858237+0000","last_clean":"2024-05-08T18:05:30.858237+0000","last_became_active":"2024-05-08T18:04:43.820369+0000","last_became_peered":"2024-05-08T18:04:43.820369+0000","last_unstale":"2024-05-08T18:05:30.858237+0000","last_undegraded":"2024-05-08T18:05:30.858237+0000","last_fullsized":"2024-05-08T18:05:30.858237+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:01:49.620924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.871018+0000","last_change":"2024-05-08T18:05:30.870035+0000","last_active":"2024-05-08T18:05:30.871018+0000","last_peered":"2024-05-08T18:05:30.871018+0000","last_clean":"2024-05-08T18:05:30.871018+0000","last_became_active":"2024-05-08T18:05:30.869578+0000","last_became_peered":"2024-05-08T18:05:30.869578+0000","last_unstale":"2024-05-08T18:05:30.871018+0000","last_undegraded":"2024-05-08T18:05:30.871018+0000","last_fullsized":"2024-05-08T18:05:30.871018+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:51:48.737463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"29'273","reported_seq":476,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903545+0000","last_change":"2024-05-08T18:04:49.833379+0000","last_active":"2024-05-08T18:07:52.903545+0000","last_peered":"2024-05-08T18:07:52.903545+0000","last_clean":"2024-05-08T18:07:52.903545+0000","last_became_active":"2024-05-08T18:04:49.833053+0000","last_became_peered":"2024-05-08T18:04:49.833053+0000","last_unstale":"2024-05-08T18:07:52.903545+0000","last_undegraded":"2024-05-08T18:07:52.903545+0000","last_fullsized":"2024-05-08T18:07:52.903545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:21.873389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":357,"num_read_kb":342,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859300+0000","last_change":"2024-05-08T18:04:51.849383+0000","last_active":"2024-05-08T18:05:30.859300+0000","last_peered":"2024-05-08T18:05:30.859300+0000","last_clean":"2024-05-08T18:05:30.859300+0000","last_became_active":"2024-05-08T18:04:51.848066+0000","last_became_peered":"2024-05-08T18:04:51.848066+0000","last_unstale":"2024-05-08T18:05:30.859300+0000","last_undegraded":"2024-05-08T18:05:30.859300+0000","last_fullsized":"2024-05-08T18:05:30.859300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:21.450184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"29'3","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.550250+0000","last_change":"2024-05-08T18:04:53.843187+0000","last_active":"2024-05-08T18:06:09.550250+0000","last_peered":"2024-05-08T18:06:09.550250+0000","last_clean":"2024-05-08T18:06:09.550250+0000","last_became_active":"2024-05-08T18:04:53.842899+0000","last_became_peered":"2024-05-08T18:04:53.842899+0000","last_unstale":"2024-05-08T18:06:09.550250+0000","last_undegraded":"2024-05-08T18:06:09.550250+0000","last_fullsized":"2024-05-08T18:06:09.550250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:05:40.879119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"29'8","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063713+0000","last_change":"2024-05-08T18:04:45.812417+0000","last_active":"2024-05-08T18:07:54.063713+0000","last_peered":"2024-05-08T18:07:54.063713+0000","last_clean":"2024-05-08T18:07:54.063713+0000","last_became_active":"2024-05-08T18:04:45.812091+0000","last_became_peered":"2024-05-08T18:04:45.812091+0000","last_unstale":"2024-05-08T18:07:54.063713+0000","last_undegraded":"2024-05-08T18:07:54.063713+0000","last_fullsized":"2024-05-08T18:07:54.063713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:26.952570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863622+0000","last_change":"2024-05-08T18:04:35.551648+0000","last_active":"2024-05-08T18:05:30.863622+0000","last_peered":"2024-05-08T18:05:30.863622+0000","last_clean":"2024-05-08T18:05:30.863622+0000","last_became_active":"2024-05-08T18:04:35.551313+0000","last_became_peered":"2024-05-08T18:04:35.551313+0000","last_unstale":"2024-05-08T18:05:30.863622+0000","last_undegraded":"2024-05-08T18:05:30.863622+0000","last_fullsized":"2024-05-08T18:05:30.863622+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:34:32.994346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858040+0000","last_change":"2024-05-08T18:04:43.821180+0000","last_active":"2024-05-08T18:05:30.858040+0000","last_peered":"2024-05-08T18:05:30.858040+0000","last_clean":"2024-05-08T18:05:30.858040+0000","last_became_active":"2024-05-08T18:04:43.820335+0000","last_became_peered":"2024-05-08T18:04:43.820335+0000","last_unstale":"2024-05-08T18:05:30.858040+0000","last_undegraded":"2024-05-08T18:05:30.858040+0000","last_fullsized":"2024-05-08T18:05:30.858040+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:31.909506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"20'1","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866981+0000","last_change":"2024-05-08T18:04:47.824034+0000","last_active":"2024-05-08T18:05:30.866981+0000","last_peered":"2024-05-08T18:05:30.866981+0000","last_clean":"2024-05-08T18:05:30.866981+0000","last_became_active":"2024-05-08T18:04:47.823631+0000","last_became_peered":"2024-05-08T18:04:47.823631+0000","last_unstale":"2024-05-08T18:05:30.866981+0000","last_undegraded":"2024-05-08T18:05:30.866981+0000","last_fullsized":"2024-05-08T18:05:30.866981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:00.791290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"29'17","reported_seq":85,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903379+0000","last_change":"2024-05-08T18:04:49.828228+0000","last_active":"2024-05-08T18:07:52.903379+0000","last_peered":"2024-05-08T18:07:52.903379+0000","last_clean":"2024-05-08T18:07:52.903379+0000","last_became_active":"2024-05-08T18:04:49.827735+0000","last_became_peered":"2024-05-08T18:04:49.827735+0000","last_unstale":"2024-05-08T18:07:52.903379+0000","last_undegraded":"2024-05-08T18:07:52.903379+0000","last_fullsized":"2024-05-08T18:07:52.903379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:04:17.239231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861148+0000","last_change":"2024-05-08T18:04:51.844276+0000","last_active":"2024-05-08T18:05:30.861148+0000","last_peered":"2024-05-08T18:05:30.861148+0000","last_clean":"2024-05-08T18:05:30.861148+0000","last_became_active":"2024-05-08T18:04:51.843940+0000","last_became_peered":"2024-05-08T18:04:51.843940+0000","last_unstale":"2024-05-08T18:05:30.861148+0000","last_undegraded":"2024-05-08T18:05:30.861148+0000","last_fullsized":"2024-05-08T18:05:30.861148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:03:10.170093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"29'4","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.791011+0000","last_change":"2024-05-08T18:04:53.848676+0000","last_active":"2024-05-08T18:07:53.791011+0000","last_peered":"2024-05-08T18:07:53.791011+0000","last_clean":"2024-05-08T18:07:53.791011+0000","last_became_active":"2024-05-08T18:04:53.848255+0000","last_became_peered":"2024-05-08T18:04:53.848255+0000","last_unstale":"2024-05-08T18:07:53.791011+0000","last_undegraded":"2024-05-08T18:07:53.791011+0000","last_fullsized":"2024-05-08T18:07:53.791011+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:34:38.766674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"29'10","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063735+0000","last_change":"2024-05-08T18:04:45.807470+0000","last_active":"2024-05-08T18:07:54.063735+0000","last_peered":"2024-05-08T18:07:54.063735+0000","last_clean":"2024-05-08T18:07:54.063735+0000","last_became_active":"2024-05-08T18:04:45.807085+0000","last_became_peered":"2024-05-08T18:04:45.807085+0000","last_unstale":"2024-05-08T18:07:54.063735+0000","last_undegraded":"2024-05-08T18:07:54.063735+0000","last_fullsized":"2024-05-08T18:07:54.063735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:05.828431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857751+0000","last_change":"2024-05-08T18:04:35.551456+0000","last_active":"2024-05-08T18:05:30.857751+0000","last_peered":"2024-05-08T18:05:30.857751+0000","last_clean":"2024-05-08T18:05:30.857751+0000","last_became_active":"2024-05-08T18:04:35.551120+0000","last_became_peered":"2024-05-08T18:04:35.551120+0000","last_unstale":"2024-05-08T18:05:30.857751+0000","last_undegraded":"2024-05-08T18:05:30.857751+0000","last_fullsized":"2024-05-08T18:05:30.857751+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:14.181756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864168+0000","last_change":"2024-05-08T18:04:43.840439+0000","last_active":"2024-05-08T18:05:30.864168+0000","last_peered":"2024-05-08T18:05:30.864168+0000","last_clean":"2024-05-08T18:05:30.864168+0000","last_became_active":"2024-05-08T18:04:43.840036+0000","last_became_peered":"2024-05-08T18:04:43.840036+0000","last_unstale":"2024-05-08T18:05:30.864168+0000","last_undegraded":"2024-05-08T18:05:30.864168+0000","last_fullsized":"2024-05-08T18:05:30.864168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:31.909506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0,3],"acting":[2,1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859629+0000","last_change":"2024-05-08T18:04:47.823507+0000","last_active":"2024-05-08T18:05:30.859629+0000","last_peered":"2024-05-08T18:05:30.859629+0000","last_clean":"2024-05-08T18:05:30.859629+0000","last_became_active":"2024-05-08T18:04:47.823093+0000","last_became_peered":"2024-05-08T18:04:47.823093+0000","last_unstale":"2024-05-08T18:05:30.859629+0000","last_undegraded":"2024-05-08T18:05:30.859629+0000","last_fullsized":"2024-05-08T18:05:30.859629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:59.706946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"29'13","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.412397+0000","last_change":"2024-05-08T18:04:49.832657+0000","last_active":"2024-05-08T18:07:53.412397+0000","last_peered":"2024-05-08T18:07:53.412397+0000","last_clean":"2024-05-08T18:07:53.412397+0000","last_became_active":"2024-05-08T18:04:49.830586+0000","last_became_peered":"2024-05-08T18:04:49.830586+0000","last_unstale":"2024-05-08T18:07:53.412397+0000","last_undegraded":"2024-05-08T18:07:53.412397+0000","last_fullsized":"2024-05-08T18:07:53.412397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:01:11.727041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864596+0000","last_change":"2024-05-08T18:04:51.849539+0000","last_active":"2024-05-08T18:05:30.864596+0000","last_peered":"2024-05-08T18:05:30.864596+0000","last_clean":"2024-05-08T18:05:30.864596+0000","last_became_active":"2024-05-08T18:04:51.849137+0000","last_became_peered":"2024-05-08T18:04:51.849137+0000","last_unstale":"2024-05-08T18:05:30.864596+0000","last_undegraded":"2024-05-08T18:05:30.864596+0000","last_fullsized":"2024-05-08T18:05:30.864596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:05.655002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'4","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.159086+0000","last_change":"2024-05-08T18:04:53.851345+0000","last_active":"2024-05-08T18:06:53.159086+0000","last_peered":"2024-05-08T18:06:53.159086+0000","last_clean":"2024-05-08T18:06:53.159086+0000","last_became_active":"2024-05-08T18:04:53.851008+0000","last_became_peered":"2024-05-08T18:04:53.851008+0000","last_unstale":"2024-05-08T18:06:53.159086+0000","last_undegraded":"2024-05-08T18:06:53.159086+0000","last_fullsized":"2024-05-08T18:06:53.159086+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:43:02.228949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.865362+0000","last_change":"2024-05-08T18:04:45.820976+0000","last_active":"2024-05-08T18:05:30.865362+0000","last_peered":"2024-05-08T18:05:30.865362+0000","last_clean":"2024-05-08T18:05:30.865362+0000","last_became_active":"2024-05-08T18:04:45.819326+0000","last_became_peered":"2024-05-08T18:04:45.819326+0000","last_unstale":"2024-05-08T18:05:30.865362+0000","last_undegraded":"2024-05-08T18:05:30.865362+0000","last_fullsized":"2024-05-08T18:05:30.865362+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:22:19.585204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862814+0000","last_change":"2024-05-08T18:04:43.840661+0000","last_active":"2024-05-08T18:05:30.862814+0000","last_peered":"2024-05-08T18:05:30.862814+0000","last_clean":"2024-05-08T18:05:30.862814+0000","last_became_active":"2024-05-08T18:04:43.840236+0000","last_became_peered":"2024-05-08T18:04:43.840236+0000","last_unstale":"2024-05-08T18:05:30.862814+0000","last_undegraded":"2024-05-08T18:05:30.862814+0000","last_fullsized":"2024-05-08T18:05:30.862814+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:40:40.407240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859953+0000","last_change":"2024-05-08T18:04:35.552512+0000","last_active":"2024-05-08T18:05:30.859953+0000","last_peered":"2024-05-08T18:05:30.859953+0000","last_clean":"2024-05-08T18:05:30.859953+0000","last_became_active":"2024-05-08T18:04:35.552173+0000","last_became_peered":"2024-05-08T18:04:35.552173+0000","last_unstale":"2024-05-08T18:05:30.859953+0000","last_undegraded":"2024-05-08T18:05:30.859953+0000","last_fullsized":"2024-05-08T18:05:30.859953+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:46:49.443150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"29'1","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.738959+0000","last_change":"2024-05-08T18:04:47.820423+0000","last_active":"2024-05-08T18:06:53.738959+0000","last_peered":"2024-05-08T18:06:53.738959+0000","last_clean":"2024-05-08T18:06:53.738959+0000","last_became_active":"2024-05-08T18:04:47.819910+0000","last_became_peered":"2024-05-08T18:04:47.819910+0000","last_unstale":"2024-05-08T18:06:53.738959+0000","last_undegraded":"2024-05-08T18:06:53.738959+0000","last_fullsized":"2024-05-08T18:06:53.738959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:41:39.241804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"29'61","reported_seq":120,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.738898+0000","last_change":"2024-05-08T18:04:49.831684+0000","last_active":"2024-05-08T18:06:53.738898+0000","last_peered":"2024-05-08T18:06:53.738898+0000","last_clean":"2024-05-08T18:06:53.738898+0000","last_became_active":"2024-05-08T18:04:49.831360+0000","last_became_peered":"2024-05-08T18:04:49.831360+0000","last_unstale":"2024-05-08T18:06:53.738898+0000","last_undegraded":"2024-05-08T18:06:53.738898+0000","last_fullsized":"2024-05-08T18:06:53.738898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:33:52.278907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":59,"num_read_kb":47,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859470+0000","last_change":"2024-05-08T18:04:51.836834+0000","last_active":"2024-05-08T18:05:30.859470+0000","last_peered":"2024-05-08T18:05:30.859470+0000","last_clean":"2024-05-08T18:05:30.859470+0000","last_became_active":"2024-05-08T18:04:51.836441+0000","last_became_peered":"2024-05-08T18:04:51.836441+0000","last_unstale":"2024-05-08T18:05:30.859470+0000","last_undegraded":"2024-05-08T18:05:30.859470+0000","last_fullsized":"2024-05-08T18:05:30.859470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:42.109569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"29'9","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.337708+0000","last_change":"2024-05-08T18:04:53.838192+0000","last_active":"2024-05-08T18:06:53.337708+0000","last_peered":"2024-05-08T18:06:53.337708+0000","last_clean":"2024-05-08T18:06:53.337708+0000","last_became_active":"2024-05-08T18:04:53.837920+0000","last_became_peered":"2024-05-08T18:04:53.837920+0000","last_unstale":"2024-05-08T18:06:53.337708+0000","last_undegraded":"2024-05-08T18:06:53.337708+0000","last_fullsized":"2024-05-08T18:06:53.337708+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:53.793862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"29'15","reported_seq":77,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.042063+0000","last_change":"2024-05-08T18:04:45.821636+0000","last_active":"2024-05-08T18:07:54.042063+0000","last_peered":"2024-05-08T18:07:54.042063+0000","last_clean":"2024-05-08T18:07:54.042063+0000","last_became_active":"2024-05-08T18:04:45.821107+0000","last_became_peered":"2024-05-08T18:04:45.821107+0000","last_unstale":"2024-05-08T18:07:54.042063+0000","last_undegraded":"2024-05-08T18:07:54.042063+0000","last_fullsized":"2024-05-08T18:07:54.042063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:22:55.457884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861385+0000","last_change":"2024-05-08T18:04:43.828309+0000","last_active":"2024-05-08T18:05:30.861385+0000","last_peered":"2024-05-08T18:05:30.861385+0000","last_clean":"2024-05-08T18:05:30.861385+0000","last_became_active":"2024-05-08T18:04:43.827920+0000","last_became_peered":"2024-05-08T18:04:43.827920+0000","last_unstale":"2024-05-08T18:05:30.861385+0000","last_undegraded":"2024-05-08T18:05:30.861385+0000","last_fullsized":"2024-05-08T18:05:30.861385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:41:01.395720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860131+0000","last_change":"2024-05-08T18:04:35.551253+0000","last_active":"2024-05-08T18:05:30.860131+0000","last_peered":"2024-05-08T18:05:30.860131+0000","last_clean":"2024-05-08T18:05:30.860131+0000","last_became_active":"2024-05-08T18:04:35.550912+0000","last_became_peered":"2024-05-08T18:04:35.550912+0000","last_unstale":"2024-05-08T18:05:30.860131+0000","last_undegraded":"2024-05-08T18:05:30.860131+0000","last_fullsized":"2024-05-08T18:05:30.860131+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:49.303072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857486+0000","last_change":"2024-05-08T18:04:47.814015+0000","last_active":"2024-05-08T18:05:30.857486+0000","last_peered":"2024-05-08T18:05:30.857486+0000","last_clean":"2024-05-08T18:05:30.857486+0000","last_became_active":"2024-05-08T18:04:47.813717+0000","last_became_peered":"2024-05-08T18:04:47.813717+0000","last_unstale":"2024-05-08T18:05:30.857486+0000","last_undegraded":"2024-05-08T18:05:30.857486+0000","last_fullsized":"2024-05-08T18:05:30.857486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:41.988054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"27'14","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861597+0000","last_change":"2024-05-08T18:04:49.835869+0000","last_active":"2024-05-08T18:05:30.861597+0000","last_peered":"2024-05-08T18:05:30.861597+0000","last_clean":"2024-05-08T18:05:30.861597+0000","last_became_active":"2024-05-08T18:04:49.835451+0000","last_became_peered":"2024-05-08T18:04:49.835451+0000","last_unstale":"2024-05-08T18:05:30.861597+0000","last_undegraded":"2024-05-08T18:05:30.861597+0000","last_fullsized":"2024-05-08T18:05:30.861597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:33.133985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"29'282","reported_seq":369,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903220+0000","last_change":"2024-05-08T18:04:51.832417+0000","last_active":"2024-05-08T18:07:52.903220+0000","last_peered":"2024-05-08T18:07:52.903220+0000","last_clean":"2024-05-08T18:07:52.903220+0000","last_became_active":"2024-05-08T18:04:51.832109+0000","last_became_peered":"2024-05-08T18:04:51.832109+0000","last_unstale":"2024-05-08T18:07:52.903220+0000","last_undegraded":"2024-05-08T18:07:52.903220+0000","last_fullsized":"2024-05-08T18:07:52.903220+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:37.543160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860146+0000","last_change":"2024-05-08T18:04:53.844572+0000","last_active":"2024-05-08T18:05:30.860146+0000","last_peered":"2024-05-08T18:05:30.860146+0000","last_clean":"2024-05-08T18:05:30.860146+0000","last_became_active":"2024-05-08T18:04:53.842998+0000","last_became_peered":"2024-05-08T18:04:53.842998+0000","last_unstale":"2024-05-08T18:05:30.860146+0000","last_undegraded":"2024-05-08T18:05:30.860146+0000","last_fullsized":"2024-05-08T18:05:30.860146+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:50.643601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'4","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.775877+0000","last_change":"2024-05-08T18:04:45.830059+0000","last_active":"2024-05-08T18:07:53.775877+0000","last_peered":"2024-05-08T18:07:53.775877+0000","last_clean":"2024-05-08T18:07:53.775877+0000","last_became_active":"2024-05-08T18:04:45.829257+0000","last_became_peered":"2024-05-08T18:04:45.829257+0000","last_unstale":"2024-05-08T18:07:53.775877+0000","last_undegraded":"2024-05-08T18:07:53.775877+0000","last_fullsized":"2024-05-08T18:07:53.775877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:10:31.034888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"29'3","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.536792+0000","last_change":"2024-05-08T18:04:43.835453+0000","last_active":"2024-05-08T18:06:09.536792+0000","last_peered":"2024-05-08T18:06:09.536792+0000","last_clean":"2024-05-08T18:06:09.536792+0000","last_became_active":"2024-05-08T18:04:43.834610+0000","last_became_peered":"2024-05-08T18:04:43.834610+0000","last_unstale":"2024-05-08T18:06:09.536792+0000","last_undegraded":"2024-05-08T18:06:09.536792+0000","last_fullsized":"2024-05-08T18:06:09.536792+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:03.821172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859946+0000","last_change":"2024-05-08T18:04:35.551595+0000","last_active":"2024-05-08T18:05:30.859946+0000","last_peered":"2024-05-08T18:05:30.859946+0000","last_clean":"2024-05-08T18:05:30.859946+0000","last_became_active":"2024-05-08T18:04:35.551366+0000","last_became_peered":"2024-05-08T18:04:35.551366+0000","last_unstale":"2024-05-08T18:05:30.859946+0000","last_undegraded":"2024-05-08T18:05:30.859946+0000","last_fullsized":"2024-05-08T18:05:30.859946+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:00.951608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857461+0000","last_change":"2024-05-08T18:04:47.822209+0000","last_active":"2024-05-08T18:05:30.857461+0000","last_peered":"2024-05-08T18:05:30.857461+0000","last_clean":"2024-05-08T18:05:30.857461+0000","last_became_active":"2024-05-08T18:04:47.819866+0000","last_became_peered":"2024-05-08T18:04:47.819866+0000","last_unstale":"2024-05-08T18:05:30.857461+0000","last_undegraded":"2024-05-08T18:05:30.857461+0000","last_fullsized":"2024-05-08T18:05:30.857461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:28.233298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"27'10","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858934+0000","last_change":"2024-05-08T18:04:49.832338+0000","last_active":"2024-05-08T18:05:30.858934+0000","last_peered":"2024-05-08T18:05:30.858934+0000","last_clean":"2024-05-08T18:05:30.858934+0000","last_became_active":"2024-05-08T18:04:49.830452+0000","last_became_peered":"2024-05-08T18:04:49.830452+0000","last_unstale":"2024-05-08T18:05:30.858934+0000","last_undegraded":"2024-05-08T18:05:30.858934+0000","last_fullsized":"2024-05-08T18:05:30.858934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:49.779432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"29'282","reported_seq":399,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.172497+0000","last_change":"2024-05-08T18:04:51.831530+0000","last_active":"2024-05-08T18:07:54.172497+0000","last_peered":"2024-05-08T18:07:54.172497+0000","last_clean":"2024-05-08T18:07:54.172497+0000","last_became_active":"2024-05-08T18:04:51.831155+0000","last_became_peered":"2024-05-08T18:04:51.831155+0000","last_unstale":"2024-05-08T18:07:54.172497+0000","last_undegraded":"2024-05-08T18:07:54.172497+0000","last_fullsized":"2024-05-08T18:07:54.172497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:32.247507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'5","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.023861+0000","last_change":"2024-05-08T18:04:53.849164+0000","last_active":"2024-05-08T18:07:54.023861+0000","last_peered":"2024-05-08T18:07:54.023861+0000","last_clean":"2024-05-08T18:07:54.023861+0000","last_became_active":"2024-05-08T18:04:53.848862+0000","last_became_peered":"2024-05-08T18:04:53.848862+0000","last_unstale":"2024-05-08T18:07:54.023861+0000","last_undegraded":"2024-05-08T18:07:54.023861+0000","last_fullsized":"2024-05-08T18:07:54.023861+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:33:19.797257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"29'6","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.867036+0000","last_change":"2024-05-08T18:04:45.823169+0000","last_active":"2024-05-08T18:07:53.867036+0000","last_peered":"2024-05-08T18:07:53.867036+0000","last_clean":"2024-05-08T18:07:53.867036+0000","last_became_active":"2024-05-08T18:04:45.822824+0000","last_became_peered":"2024-05-08T18:04:45.822824+0000","last_unstale":"2024-05-08T18:07:53.867036+0000","last_undegraded":"2024-05-08T18:07:53.867036+0000","last_fullsized":"2024-05-08T18:07:53.867036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:34.503647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862506+0000","last_change":"2024-05-08T18:04:43.839633+0000","last_active":"2024-05-08T18:05:30.862506+0000","last_peered":"2024-05-08T18:05:30.862506+0000","last_clean":"2024-05-08T18:05:30.862506+0000","last_became_active":"2024-05-08T18:04:43.839240+0000","last_became_peered":"2024-05-08T18:04:43.839240+0000","last_unstale":"2024-05-08T18:05:30.862506+0000","last_undegraded":"2024-05-08T18:05:30.862506+0000","last_fullsized":"2024-05-08T18:05:30.862506+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:31.909506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864842+0000","last_change":"2024-05-08T18:04:35.551765+0000","last_active":"2024-05-08T18:05:30.864842+0000","last_peered":"2024-05-08T18:05:30.864842+0000","last_clean":"2024-05-08T18:05:30.864842+0000","last_became_active":"2024-05-08T18:04:35.551412+0000","last_became_peered":"2024-05-08T18:04:35.551412+0000","last_unstale":"2024-05-08T18:05:30.864842+0000","last_undegraded":"2024-05-08T18:05:30.864842+0000","last_fullsized":"2024-05-08T18:05:30.864842+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:37:38.776558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857179+0000","last_change":"2024-05-08T18:04:47.816250+0000","last_active":"2024-05-08T18:05:30.857179+0000","last_peered":"2024-05-08T18:05:30.857179+0000","last_clean":"2024-05-08T18:05:30.857179+0000","last_became_active":"2024-05-08T18:04:47.815947+0000","last_became_peered":"2024-05-08T18:04:47.815947+0000","last_unstale":"2024-05-08T18:05:30.857179+0000","last_undegraded":"2024-05-08T18:05:30.857179+0000","last_fullsized":"2024-05-08T18:05:30.857179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:16:57.162048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"29'82","reported_seq":154,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.952921+0000","last_change":"2024-05-08T18:04:49.834985+0000","last_active":"2024-05-08T18:06:51.952921+0000","last_peered":"2024-05-08T18:06:51.952921+0000","last_clean":"2024-05-08T18:06:51.952921+0000","last_became_active":"2024-05-08T18:04:49.834586+0000","last_became_peered":"2024-05-08T18:04:49.834586+0000","last_unstale":"2024-05-08T18:06:51.952921+0000","last_undegraded":"2024-05-08T18:06:51.952921+0000","last_fullsized":"2024-05-08T18:06:51.952921+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:26.820238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858181+0000","last_change":"2024-05-08T18:04:51.831135+0000","last_active":"2024-05-08T18:05:30.858181+0000","last_peered":"2024-05-08T18:05:30.858181+0000","last_clean":"2024-05-08T18:05:30.858181+0000","last_became_active":"2024-05-08T18:04:51.830760+0000","last_became_peered":"2024-05-08T18:04:51.830760+0000","last_unstale":"2024-05-08T18:05:30.858181+0000","last_undegraded":"2024-05-08T18:05:30.858181+0000","last_fullsized":"2024-05-08T18:05:30.858181+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:15.559137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.865409+0000","last_change":"2024-05-08T18:04:53.839069+0000","last_active":"2024-05-08T18:05:30.865409+0000","last_peered":"2024-05-08T18:05:30.865409+0000","last_clean":"2024-05-08T18:05:30.865409+0000","last_became_active":"2024-05-08T18:04:53.838600+0000","last_became_peered":"2024-05-08T18:04:53.838600+0000","last_unstale":"2024-05-08T18:05:30.865409+0000","last_undegraded":"2024-05-08T18:05:30.865409+0000","last_fullsized":"2024-05-08T18:05:30.865409+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:17.566875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"27'2","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866690+0000","last_change":"2024-05-08T18:04:53.839793+0000","last_active":"2024-05-08T18:05:30.866690+0000","last_peered":"2024-05-08T18:05:30.866690+0000","last_clean":"2024-05-08T18:05:30.866690+0000","last_became_active":"2024-05-08T18:04:53.839500+0000","last_became_peered":"2024-05-08T18:04:53.839500+0000","last_unstale":"2024-05-08T18:05:30.866690+0000","last_undegraded":"2024-05-08T18:05:30.866690+0000","last_fullsized":"2024-05-08T18:05:30.866690+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:19.575012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"29'8","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.219834+0000","last_change":"2024-05-08T18:04:45.819806+0000","last_active":"2024-05-08T18:06:50.219834+0000","last_peered":"2024-05-08T18:06:50.219834+0000","last_clean":"2024-05-08T18:06:50.219834+0000","last_became_active":"2024-05-08T18:04:45.819508+0000","last_became_peered":"2024-05-08T18:04:45.819508+0000","last_unstale":"2024-05-08T18:06:50.219834+0000","last_undegraded":"2024-05-08T18:06:50.219834+0000","last_fullsized":"2024-05-08T18:06:50.219834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:32.833248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859163+0000","last_change":"2024-05-08T18:04:43.844864+0000","last_active":"2024-05-08T18:05:30.859163+0000","last_peered":"2024-05-08T18:05:30.859163+0000","last_clean":"2024-05-08T18:05:30.859163+0000","last_became_active":"2024-05-08T18:04:43.837929+0000","last_became_peered":"2024-05-08T18:04:43.837929+0000","last_unstale":"2024-05-08T18:05:30.859163+0000","last_undegraded":"2024-05-08T18:05:30.859163+0000","last_fullsized":"2024-05-08T18:05:30.859163+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:06.177343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861102+0000","last_change":"2024-05-08T18:04:47.818763+0000","last_active":"2024-05-08T18:05:30.861102+0000","last_peered":"2024-05-08T18:05:30.861102+0000","last_clean":"2024-05-08T18:05:30.861102+0000","last_became_active":"2024-05-08T18:04:47.818443+0000","last_became_peered":"2024-05-08T18:04:47.818443+0000","last_unstale":"2024-05-08T18:05:30.861102+0000","last_undegraded":"2024-05-08T18:05:30.861102+0000","last_fullsized":"2024-05-08T18:05:30.861102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:15:57.213130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"29'32","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.958612+0000","last_change":"2024-05-08T18:04:49.834233+0000","last_active":"2024-05-08T18:06:51.958612+0000","last_peered":"2024-05-08T18:06:51.958612+0000","last_clean":"2024-05-08T18:06:51.958612+0000","last_became_active":"2024-05-08T18:04:49.833599+0000","last_became_peered":"2024-05-08T18:04:49.833599+0000","last_unstale":"2024-05-08T18:06:51.958612+0000","last_undegraded":"2024-05-08T18:06:51.958612+0000","last_fullsized":"2024-05-08T18:06:51.958612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:54:34.743324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858258+0000","last_change":"2024-05-08T18:04:51.844827+0000","last_active":"2024-05-08T18:05:30.858258+0000","last_peered":"2024-05-08T18:05:30.858258+0000","last_clean":"2024-05-08T18:05:30.858258+0000","last_became_active":"2024-05-08T18:04:51.844526+0000","last_became_peered":"2024-05-08T18:04:51.844526+0000","last_unstale":"2024-05-08T18:05:30.858258+0000","last_undegraded":"2024-05-08T18:05:30.858258+0000","last_fullsized":"2024-05-08T18:05:30.858258+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:13:54.471619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"29'11","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.639339+0000","last_change":"2024-05-08T18:04:53.850911+0000","last_active":"2024-05-08T18:06:51.639339+0000","last_peered":"2024-05-08T18:06:51.639339+0000","last_clean":"2024-05-08T18:06:51.639339+0000","last_became_active":"2024-05-08T18:04:53.850593+0000","last_became_peered":"2024-05-08T18:04:53.850593+0000","last_unstale":"2024-05-08T18:06:51.639339+0000","last_undegraded":"2024-05-08T18:06:51.639339+0000","last_fullsized":"2024-05-08T18:06:51.639339+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:11:21.798553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":62,"num_write":31,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"29'8","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399973+0000","last_change":"2024-05-08T18:04:45.819974+0000","last_active":"2024-05-08T18:06:53.399973+0000","last_peered":"2024-05-08T18:06:53.399973+0000","last_clean":"2024-05-08T18:06:53.399973+0000","last_became_active":"2024-05-08T18:04:45.818979+0000","last_became_peered":"2024-05-08T18:04:45.818979+0000","last_unstale":"2024-05-08T18:06:53.399973+0000","last_undegraded":"2024-05-08T18:06:53.399973+0000","last_fullsized":"2024-05-08T18:06:53.399973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:30.458103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863085+0000","last_change":"2024-05-08T18:04:43.836993+0000","last_active":"2024-05-08T18:05:30.863085+0000","last_peered":"2024-05-08T18:05:30.863085+0000","last_clean":"2024-05-08T18:05:30.863085+0000","last_became_active":"2024-05-08T18:04:43.836614+0000","last_became_peered":"2024-05-08T18:04:43.836614+0000","last_unstale":"2024-05-08T18:05:30.863085+0000","last_undegraded":"2024-05-08T18:05:30.863085+0000","last_fullsized":"2024-05-08T18:05:30.863085+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:05:33.776915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858802+0000","last_change":"2024-05-08T18:04:47.814847+0000","last_active":"2024-05-08T18:05:30.858802+0000","last_peered":"2024-05-08T18:05:30.858802+0000","last_clean":"2024-05-08T18:05:30.858802+0000","last_became_active":"2024-05-08T18:04:47.814561+0000","last_became_peered":"2024-05-08T18:04:47.814561+0000","last_unstale":"2024-05-08T18:05:30.858802+0000","last_undegraded":"2024-05-08T18:05:30.858802+0000","last_fullsized":"2024-05-08T18:05:30.858802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:58.115834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"29'7","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.433454+0000","last_change":"2024-05-08T18:04:49.825623+0000","last_active":"2024-05-08T18:07:53.433454+0000","last_peered":"2024-05-08T18:07:53.433454+0000","last_clean":"2024-05-08T18:07:53.433454+0000","last_became_active":"2024-05-08T18:04:49.825347+0000","last_became_peered":"2024-05-08T18:04:49.825347+0000","last_unstale":"2024-05-08T18:07:53.433454+0000","last_undegraded":"2024-05-08T18:07:53.433454+0000","last_fullsized":"2024-05-08T18:07:53.433454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:37.454897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863344+0000","last_change":"2024-05-08T18:04:51.843388+0000","last_active":"2024-05-08T18:05:30.863344+0000","last_peered":"2024-05-08T18:05:30.863344+0000","last_clean":"2024-05-08T18:05:30.863344+0000","last_became_active":"2024-05-08T18:04:51.843002+0000","last_became_peered":"2024-05-08T18:04:51.843002+0000","last_unstale":"2024-05-08T18:05:30.863344+0000","last_undegraded":"2024-05-08T18:05:30.863344+0000","last_fullsized":"2024-05-08T18:05:30.863344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:13.211841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859114+0000","last_change":"2024-05-08T18:04:53.840844+0000","last_active":"2024-05-08T18:05:30.859114+0000","last_peered":"2024-05-08T18:05:30.859114+0000","last_clean":"2024-05-08T18:05:30.859114+0000","last_became_active":"2024-05-08T18:04:53.840437+0000","last_became_peered":"2024-05-08T18:04:53.840437+0000","last_unstale":"2024-05-08T18:05:30.859114+0000","last_undegraded":"2024-05-08T18:05:30.859114+0000","last_fullsized":"2024-05-08T18:05:30.859114+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:42:52.195643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"29'8","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.861059+0000","last_change":"2024-05-08T18:05:30.868442+0000","last_active":"2024-05-08T18:07:53.861059+0000","last_peered":"2024-05-08T18:07:53.861059+0000","last_clean":"2024-05-08T18:07:53.861059+0000","last_became_active":"2024-05-08T18:05:30.868156+0000","last_became_peered":"2024-05-08T18:05:30.868156+0000","last_unstale":"2024-05-08T18:07:53.861059+0000","last_undegraded":"2024-05-08T18:07:53.861059+0000","last_fullsized":"2024-05-08T18:07:53.861059+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:44.461215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867524+0000","last_change":"2024-05-08T18:04:43.835856+0000","last_active":"2024-05-08T18:05:30.867524+0000","last_peered":"2024-05-08T18:05:30.867524+0000","last_clean":"2024-05-08T18:05:30.867524+0000","last_became_active":"2024-05-08T18:04:43.835435+0000","last_became_peered":"2024-05-08T18:04:43.835435+0000","last_unstale":"2024-05-08T18:05:30.867524+0000","last_undegraded":"2024-05-08T18:05:30.867524+0000","last_fullsized":"2024-05-08T18:05:30.867524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:47.971014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860152+0000","last_change":"2024-05-08T18:04:47.824614+0000","last_active":"2024-05-08T18:05:30.860152+0000","last_peered":"2024-05-08T18:05:30.860152+0000","last_clean":"2024-05-08T18:05:30.860152+0000","last_became_active":"2024-05-08T18:04:47.824140+0000","last_became_peered":"2024-05-08T18:04:47.824140+0000","last_unstale":"2024-05-08T18:05:30.860152+0000","last_undegraded":"2024-05-08T18:05:30.860152+0000","last_fullsized":"2024-05-08T18:05:30.860152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:36:47.466900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"29'41","reported_seq":100,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.004393+0000","last_change":"2024-05-08T18:04:49.824946+0000","last_active":"2024-05-08T18:07:54.004393+0000","last_peered":"2024-05-08T18:07:54.004393+0000","last_clean":"2024-05-08T18:07:54.004393+0000","last_became_active":"2024-05-08T18:04:49.824600+0000","last_became_peered":"2024-05-08T18:04:49.824600+0000","last_unstale":"2024-05-08T18:07:54.004393+0000","last_undegraded":"2024-05-08T18:07:54.004393+0000","last_fullsized":"2024-05-08T18:07:54.004393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:11.205149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":64,"num_write":46,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860821+0000","last_change":"2024-05-08T18:04:51.833599+0000","last_active":"2024-05-08T18:05:30.860821+0000","last_peered":"2024-05-08T18:05:30.860821+0000","last_clean":"2024-05-08T18:05:30.860821+0000","last_became_active":"2024-05-08T18:04:51.829937+0000","last_became_peered":"2024-05-08T18:04:51.829937+0000","last_unstale":"2024-05-08T18:05:30.860821+0000","last_undegraded":"2024-05-08T18:05:30.860821+0000","last_fullsized":"2024-05-08T18:05:30.860821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:02:48.266056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858578+0000","last_change":"2024-05-08T18:04:53.842981+0000","last_active":"2024-05-08T18:05:30.858578+0000","last_peered":"2024-05-08T18:05:30.858578+0000","last_clean":"2024-05-08T18:05:30.858578+0000","last_became_active":"2024-05-08T18:04:53.842610+0000","last_became_peered":"2024-05-08T18:04:53.842610+0000","last_unstale":"2024-05-08T18:05:30.858578+0000","last_undegraded":"2024-05-08T18:05:30.858578+0000","last_fullsized":"2024-05-08T18:05:30.858578+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:16:15.240226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857482+0000","last_change":"2024-05-08T18:04:45.829838+0000","last_active":"2024-05-08T18:05:30.857482+0000","last_peered":"2024-05-08T18:05:30.857482+0000","last_clean":"2024-05-08T18:05:30.857482+0000","last_became_active":"2024-05-08T18:04:45.829458+0000","last_became_peered":"2024-05-08T18:04:45.829458+0000","last_unstale":"2024-05-08T18:05:30.857482+0000","last_undegraded":"2024-05-08T18:05:30.857482+0000","last_fullsized":"2024-05-08T18:05:30.857482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:15:55.206029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"29'4","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.352042+0000","last_change":"2024-05-08T18:04:43.837089+0000","last_active":"2024-05-08T18:06:53.352042+0000","last_peered":"2024-05-08T18:06:53.352042+0000","last_clean":"2024-05-08T18:06:53.352042+0000","last_became_active":"2024-05-08T18:04:43.836633+0000","last_became_peered":"2024-05-08T18:04:43.836633+0000","last_unstale":"2024-05-08T18:06:53.352042+0000","last_undegraded":"2024-05-08T18:06:53.352042+0000","last_fullsized":"2024-05-08T18:06:53.352042+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:55.493870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"20'1","reported_seq":218,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.049308+0000","last_change":"2024-05-08T18:04:47.820110+0000","last_active":"2024-05-08T18:06:53.049308+0000","last_peered":"2024-05-08T18:06:53.049308+0000","last_clean":"2024-05-08T18:06:53.049308+0000","last_became_active":"2024-05-08T18:04:47.819771+0000","last_became_peered":"2024-05-08T18:04:47.819771+0000","last_unstale":"2024-05-08T18:06:53.049308+0000","last_undegraded":"2024-05-08T18:06:53.049308+0000","last_fullsized":"2024-05-08T18:06:53.049308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:42:21.961312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"29'30","reported_seq":72,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.944676+0000","last_change":"2024-05-08T18:04:49.824323+0000","last_active":"2024-05-08T18:06:51.944676+0000","last_peered":"2024-05-08T18:06:51.944676+0000","last_clean":"2024-05-08T18:06:51.944676+0000","last_became_active":"2024-05-08T18:04:49.824022+0000","last_became_peered":"2024-05-08T18:04:49.824022+0000","last_unstale":"2024-05-08T18:06:51.944676+0000","last_undegraded":"2024-05-08T18:06:51.944676+0000","last_fullsized":"2024-05-08T18:06:51.944676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:41:41.249321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866329+0000","last_change":"2024-05-08T18:04:51.826713+0000","last_active":"2024-05-08T18:05:30.866329+0000","last_peered":"2024-05-08T18:05:30.866329+0000","last_clean":"2024-05-08T18:05:30.866329+0000","last_became_active":"2024-05-08T18:04:51.826424+0000","last_became_peered":"2024-05-08T18:04:51.826424+0000","last_unstale":"2024-05-08T18:05:30.866329+0000","last_undegraded":"2024-05-08T18:05:30.866329+0000","last_fullsized":"2024-05-08T18:05:30.866329+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:33:54.285599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"29'20","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995883+0000","last_change":"2024-05-08T18:05:30.873315+0000","last_active":"2024-05-08T18:07:53.995883+0000","last_peered":"2024-05-08T18:07:53.995883+0000","last_clean":"2024-05-08T18:07:53.995883+0000","last_became_active":"2024-05-08T18:05:30.868409+0000","last_became_peered":"2024-05-08T18:05:30.868409+0000","last_unstale":"2024-05-08T18:07:53.995883+0000","last_undegraded":"2024-05-08T18:07:53.995883+0000","last_fullsized":"2024-05-08T18:07:53.995883+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:07.623428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857341+0000","last_change":"2024-05-08T18:04:43.844983+0000","last_active":"2024-05-08T18:05:30.857341+0000","last_peered":"2024-05-08T18:05:30.857341+0000","last_clean":"2024-05-08T18:05:30.857341+0000","last_became_active":"2024-05-08T18:04:43.838957+0000","last_became_peered":"2024-05-08T18:04:43.838957+0000","last_unstale":"2024-05-08T18:05:30.857341+0000","last_undegraded":"2024-05-08T18:05:30.857341+0000","last_fullsized":"2024-05-08T18:05:30.857341+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:42:05.210775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859885+0000","last_change":"2024-05-08T18:04:47.820891+0000","last_active":"2024-05-08T18:05:30.859885+0000","last_peered":"2024-05-08T18:05:30.859885+0000","last_clean":"2024-05-08T18:05:30.859885+0000","last_became_active":"2024-05-08T18:04:47.820256+0000","last_became_peered":"2024-05-08T18:04:47.820256+0000","last_unstale":"2024-05-08T18:05:30.859885+0000","last_undegraded":"2024-05-08T18:05:30.859885+0000","last_fullsized":"2024-05-08T18:05:30.859885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:04:15.231714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"29'54","reported_seq":110,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.964783+0000","last_change":"2024-05-08T18:04:49.825705+0000","last_active":"2024-05-08T18:06:51.964783+0000","last_peered":"2024-05-08T18:06:51.964783+0000","last_clean":"2024-05-08T18:06:51.964783+0000","last_became_active":"2024-05-08T18:04:49.825366+0000","last_became_peered":"2024-05-08T18:04:49.825366+0000","last_unstale":"2024-05-08T18:06:51.964783+0000","last_undegraded":"2024-05-08T18:06:51.964783+0000","last_fullsized":"2024-05-08T18:06:51.964783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:27:27.454431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862900+0000","last_change":"2024-05-08T18:04:51.827568+0000","last_active":"2024-05-08T18:05:30.862900+0000","last_peered":"2024-05-08T18:05:30.862900+0000","last_clean":"2024-05-08T18:05:30.862900+0000","last_became_active":"2024-05-08T18:04:51.827127+0000","last_became_peered":"2024-05-08T18:04:51.827127+0000","last_unstale":"2024-05-08T18:05:30.862900+0000","last_undegraded":"2024-05-08T18:05:30.862900+0000","last_fullsized":"2024-05-08T18:05:30.862900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:27:29.461124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"29'14","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.333941+0000","last_change":"2024-05-08T18:04:53.844087+0000","last_active":"2024-05-08T18:06:53.333941+0000","last_peered":"2024-05-08T18:06:53.333941+0000","last_clean":"2024-05-08T18:06:53.333941+0000","last_became_active":"2024-05-08T18:04:53.843772+0000","last_became_peered":"2024-05-08T18:04:53.843772+0000","last_unstale":"2024-05-08T18:06:53.333941+0000","last_undegraded":"2024-05-08T18:06:53.333941+0000","last_fullsized":"2024-05-08T18:06:53.333941+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:26.083815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.738215+0000","last_change":"2024-05-08T18:04:45.811909+0000","last_active":"2024-05-08T18:06:48.738215+0000","last_peered":"2024-05-08T18:06:48.738215+0000","last_clean":"2024-05-08T18:06:48.738215+0000","last_became_active":"2024-05-08T18:04:45.811517+0000","last_became_peered":"2024-05-08T18:04:45.811517+0000","last_unstale":"2024-05-08T18:06:48.738215+0000","last_undegraded":"2024-05-08T18:06:48.738215+0000","last_fullsized":"2024-05-08T18:06:48.738215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:32.141179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860711+0000","last_change":"2024-05-08T18:04:43.836313+0000","last_active":"2024-05-08T18:05:30.860711+0000","last_peered":"2024-05-08T18:05:30.860711+0000","last_clean":"2024-05-08T18:05:30.860711+0000","last_became_active":"2024-05-08T18:04:43.834767+0000","last_became_peered":"2024-05-08T18:04:43.834767+0000","last_unstale":"2024-05-08T18:05:30.860711+0000","last_undegraded":"2024-05-08T18:05:30.860711+0000","last_fullsized":"2024-05-08T18:05:30.860711+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:22.454546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0,2],"acting":[1,3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867042+0000","last_change":"2024-05-08T18:04:47.821012+0000","last_active":"2024-05-08T18:05:30.867042+0000","last_peered":"2024-05-08T18:05:30.867042+0000","last_clean":"2024-05-08T18:05:30.867042+0000","last_became_active":"2024-05-08T18:04:47.820662+0000","last_became_peered":"2024-05-08T18:04:47.820662+0000","last_unstale":"2024-05-08T18:05:30.867042+0000","last_undegraded":"2024-05-08T18:05:30.867042+0000","last_fullsized":"2024-05-08T18:05:30.867042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:29:01.485039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"29'37","reported_seq":82,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.401221+0000","last_change":"2024-05-08T18:04:49.831871+0000","last_active":"2024-05-08T18:06:53.401221+0000","last_peered":"2024-05-08T18:06:53.401221+0000","last_clean":"2024-05-08T18:06:53.401221+0000","last_became_active":"2024-05-08T18:04:49.830476+0000","last_became_peered":"2024-05-08T18:04:49.830476+0000","last_unstale":"2024-05-08T18:06:53.401221+0000","last_undegraded":"2024-05-08T18:06:53.401221+0000","last_fullsized":"2024-05-08T18:06:53.401221+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:46.223733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862181+0000","last_change":"2024-05-08T18:04:51.845269+0000","last_active":"2024-05-08T18:05:30.862181+0000","last_peered":"2024-05-08T18:05:30.862181+0000","last_clean":"2024-05-08T18:05:30.862181+0000","last_became_active":"2024-05-08T18:04:51.844912+0000","last_became_peered":"2024-05-08T18:04:51.844912+0000","last_unstale":"2024-05-08T18:05:30.862181+0000","last_undegraded":"2024-05-08T18:05:30.862181+0000","last_fullsized":"2024-05-08T18:05:30.862181+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:17.567274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"29'2","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.037376+0000","last_change":"2024-05-08T18:04:53.847736+0000","last_active":"2024-05-08T18:07:54.037376+0000","last_peered":"2024-05-08T18:07:54.037376+0000","last_clean":"2024-05-08T18:07:54.037376+0000","last_became_active":"2024-05-08T18:04:53.847413+0000","last_became_peered":"2024-05-08T18:04:53.847413+0000","last_unstale":"2024-05-08T18:07:54.037376+0000","last_undegraded":"2024-05-08T18:07:54.037376+0000","last_fullsized":"2024-05-08T18:07:54.037376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:18.059582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"29'12","reported_seq":60,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.041935+0000","last_change":"2024-05-08T18:04:45.830619+0000","last_active":"2024-05-08T18:07:54.041935+0000","last_peered":"2024-05-08T18:07:54.041935+0000","last_clean":"2024-05-08T18:07:54.041935+0000","last_became_active":"2024-05-08T18:04:45.830329+0000","last_became_peered":"2024-05-08T18:04:45.830329+0000","last_unstale":"2024-05-08T18:07:54.041935+0000","last_undegraded":"2024-05-08T18:07:54.041935+0000","last_fullsized":"2024-05-08T18:07:54.041935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:13:12.748032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861213+0000","last_change":"2024-05-08T18:04:43.835662+0000","last_active":"2024-05-08T18:05:30.861213+0000","last_peered":"2024-05-08T18:05:30.861213+0000","last_clean":"2024-05-08T18:05:30.861213+0000","last_became_active":"2024-05-08T18:04:43.835227+0000","last_became_peered":"2024-05-08T18:04:43.835227+0000","last_unstale":"2024-05-08T18:05:30.861213+0000","last_undegraded":"2024-05-08T18:05:30.861213+0000","last_fullsized":"2024-05-08T18:05:30.861213+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:10.360847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3,2],"acting":[1,0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.871559+0000","last_change":"2024-05-08T18:05:30.870636+0000","last_active":"2024-05-08T18:05:30.871559+0000","last_peered":"2024-05-08T18:05:30.871559+0000","last_clean":"2024-05-08T18:05:30.871559+0000","last_became_active":"2024-05-08T18:05:30.870155+0000","last_became_peered":"2024-05-08T18:05:30.870155+0000","last_unstale":"2024-05-08T18:05:30.871559+0000","last_undegraded":"2024-05-08T18:05:30.871559+0000","last_fullsized":"2024-05-08T18:05:30.871559+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:56.207001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"29'32","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.962760+0000","last_change":"2024-05-08T18:04:49.832968+0000","last_active":"2024-05-08T18:06:51.962760+0000","last_peered":"2024-05-08T18:06:51.962760+0000","last_clean":"2024-05-08T18:06:51.962760+0000","last_became_active":"2024-05-08T18:04:49.832649+0000","last_became_peered":"2024-05-08T18:04:49.832649+0000","last_unstale":"2024-05-08T18:06:51.962760+0000","last_undegraded":"2024-05-08T18:06:51.962760+0000","last_fullsized":"2024-05-08T18:06:51.962760+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:52.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859450+0000","last_change":"2024-05-08T18:04:51.833712+0000","last_active":"2024-05-08T18:05:30.859450+0000","last_peered":"2024-05-08T18:05:30.859450+0000","last_clean":"2024-05-08T18:05:30.859450+0000","last_became_active":"2024-05-08T18:04:51.833421+0000","last_became_peered":"2024-05-08T18:04:51.833421+0000","last_unstale":"2024-05-08T18:05:30.859450+0000","last_undegraded":"2024-05-08T18:05:30.859450+0000","last_fullsized":"2024-05-08T18:05:30.859450+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:48.752288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"29'5","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.394775+0000","last_change":"2024-05-08T18:04:53.848762+0000","last_active":"2024-05-08T18:06:53.394775+0000","last_peered":"2024-05-08T18:06:53.394775+0000","last_clean":"2024-05-08T18:06:53.394775+0000","last_became_active":"2024-05-08T18:04:53.848430+0000","last_became_peered":"2024-05-08T18:04:53.848430+0000","last_unstale":"2024-05-08T18:06:53.394775+0000","last_undegraded":"2024-05-08T18:06:53.394775+0000","last_fullsized":"2024-05-08T18:06:53.394775+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:48.070547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"29'10","reported_seq":66,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.218980+0000","last_change":"2024-05-08T18:04:45.807312+0000","last_active":"2024-05-08T18:06:50.218980+0000","last_peered":"2024-05-08T18:06:50.218980+0000","last_clean":"2024-05-08T18:06:50.218980+0000","last_became_active":"2024-05-08T18:04:45.806808+0000","last_became_peered":"2024-05-08T18:04:45.806808+0000","last_unstale":"2024-05-08T18:06:50.218980+0000","last_undegraded":"2024-05-08T18:06:50.218980+0000","last_fullsized":"2024-05-08T18:06:50.218980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:27:20.114031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859441+0000","last_change":"2024-05-08T18:04:43.833094+0000","last_active":"2024-05-08T18:05:30.859441+0000","last_peered":"2024-05-08T18:05:30.859441+0000","last_clean":"2024-05-08T18:05:30.859441+0000","last_became_active":"2024-05-08T18:04:43.832147+0000","last_became_peered":"2024-05-08T18:04:43.832147+0000","last_unstale":"2024-05-08T18:05:30.859441+0000","last_undegraded":"2024-05-08T18:05:30.859441+0000","last_fullsized":"2024-05-08T18:05:30.859441+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:49.361708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862397+0000","last_change":"2024-05-08T18:04:47.815796+0000","last_active":"2024-05-08T18:05:30.862397+0000","last_peered":"2024-05-08T18:05:30.862397+0000","last_clean":"2024-05-08T18:05:30.862397+0000","last_became_active":"2024-05-08T18:04:47.815219+0000","last_became_peered":"2024-05-08T18:04:47.815219+0000","last_unstale":"2024-05-08T18:05:30.862397+0000","last_undegraded":"2024-05-08T18:05:30.862397+0000","last_fullsized":"2024-05-08T18:05:30.862397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:23:20.608088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"27'10","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861853+0000","last_change":"2024-05-08T18:04:49.833674+0000","last_active":"2024-05-08T18:05:30.861853+0000","last_peered":"2024-05-08T18:05:30.861853+0000","last_clean":"2024-05-08T18:05:30.861853+0000","last_became_active":"2024-05-08T18:04:49.830793+0000","last_became_peered":"2024-05-08T18:04:49.830793+0000","last_unstale":"2024-05-08T18:05:30.861853+0000","last_undegraded":"2024-05-08T18:05:30.861853+0000","last_fullsized":"2024-05-08T18:05:30.861853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:54:53.163574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"29'282","reported_seq":366,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903209+0000","last_change":"2024-05-08T18:04:51.834224+0000","last_active":"2024-05-08T18:07:52.903209+0000","last_peered":"2024-05-08T18:07:52.903209+0000","last_clean":"2024-05-08T18:07:52.903209+0000","last_became_active":"2024-05-08T18:04:51.833866+0000","last_became_peered":"2024-05-08T18:04:51.833866+0000","last_unstale":"2024-05-08T18:07:52.903209+0000","last_undegraded":"2024-05-08T18:07:52.903209+0000","last_fullsized":"2024-05-08T18:07:52.903209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:21:20.452363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.856865+0000","last_change":"2024-05-08T18:04:53.848804+0000","last_active":"2024-05-08T18:05:30.856865+0000","last_peered":"2024-05-08T18:05:30.856865+0000","last_clean":"2024-05-08T18:05:30.856865+0000","last_became_active":"2024-05-08T18:04:53.848321+0000","last_became_peered":"2024-05-08T18:04:53.848321+0000","last_unstale":"2024-05-08T18:05:30.856865+0000","last_undegraded":"2024-05-08T18:05:30.856865+0000","last_fullsized":"2024-05-08T18:05:30.856865+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:34.662852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"29'4","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399620+0000","last_change":"2024-05-08T18:04:45.814018+0000","last_active":"2024-05-08T18:06:53.399620+0000","last_peered":"2024-05-08T18:06:53.399620+0000","last_clean":"2024-05-08T18:06:53.399620+0000","last_became_active":"2024-05-08T18:04:45.813424+0000","last_became_peered":"2024-05-08T18:04:45.813424+0000","last_unstale":"2024-05-08T18:06:53.399620+0000","last_undegraded":"2024-05-08T18:06:53.399620+0000","last_fullsized":"2024-05-08T18:06:53.399620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:42:05.254786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866726+0000","last_change":"2024-05-08T18:04:43.841603+0000","last_active":"2024-05-08T18:05:30.866726+0000","last_peered":"2024-05-08T18:05:30.866726+0000","last_clean":"2024-05-08T18:05:30.866726+0000","last_became_active":"2024-05-08T18:04:43.840502+0000","last_became_peered":"2024-05-08T18:04:43.840502+0000","last_unstale":"2024-05-08T18:05:30.866726+0000","last_undegraded":"2024-05-08T18:05:30.866726+0000","last_fullsized":"2024-05-08T18:05:30.866726+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:40:40.407240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860986+0000","last_change":"2024-05-08T18:04:47.819165+0000","last_active":"2024-05-08T18:05:30.860986+0000","last_peered":"2024-05-08T18:05:30.860986+0000","last_clean":"2024-05-08T18:05:30.860986+0000","last_became_active":"2024-05-08T18:04:47.818839+0000","last_became_peered":"2024-05-08T18:04:47.818839+0000","last_unstale":"2024-05-08T18:05:30.860986+0000","last_undegraded":"2024-05-08T18:05:30.860986+0000","last_fullsized":"2024-05-08T18:05:30.860986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:04:15.231714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"29'37","reported_seq":82,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.936372+0000","last_change":"2024-05-08T18:04:49.824708+0000","last_active":"2024-05-08T18:06:51.936372+0000","last_peered":"2024-05-08T18:06:51.936372+0000","last_clean":"2024-05-08T18:06:51.936372+0000","last_became_active":"2024-05-08T18:04:49.824405+0000","last_became_peered":"2024-05-08T18:04:49.824405+0000","last_unstale":"2024-05-08T18:06:51.936372+0000","last_undegraded":"2024-05-08T18:06:51.936372+0000","last_fullsized":"2024-05-08T18:06:51.936372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:52.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858238+0000","last_change":"2024-05-08T18:04:51.845116+0000","last_active":"2024-05-08T18:05:30.858238+0000","last_peered":"2024-05-08T18:05:30.858238+0000","last_clean":"2024-05-08T18:05:30.858238+0000","last_became_active":"2024-05-08T18:04:51.844757+0000","last_became_peered":"2024-05-08T18:04:51.844757+0000","last_unstale":"2024-05-08T18:05:30.858238+0000","last_undegraded":"2024-05-08T18:05:30.858238+0000","last_fullsized":"2024-05-08T18:05:30.858238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:17.567274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866299+0000","last_change":"2024-05-08T18:04:53.841236+0000","last_active":"2024-05-08T18:05:30.866299+0000","last_peered":"2024-05-08T18:05:30.866299+0000","last_clean":"2024-05-08T18:05:30.866299+0000","last_became_active":"2024-05-08T18:04:53.840956+0000","last_became_peered":"2024-05-08T18:04:53.840956+0000","last_unstale":"2024-05-08T18:05:30.866299+0000","last_undegraded":"2024-05-08T18:05:30.866299+0000","last_fullsized":"2024-05-08T18:05:30.866299+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:58.125427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"29'3","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.866543+0000","last_change":"2024-05-08T18:04:45.807309+0000","last_active":"2024-05-08T18:07:53.866543+0000","last_peered":"2024-05-08T18:07:53.866543+0000","last_clean":"2024-05-08T18:07:53.866543+0000","last_became_active":"2024-05-08T18:04:45.806560+0000","last_became_peered":"2024-05-08T18:04:45.806560+0000","last_unstale":"2024-05-08T18:07:53.866543+0000","last_undegraded":"2024-05-08T18:07:53.866543+0000","last_fullsized":"2024-05-08T18:07:53.866543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:11:17.190072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861114+0000","last_change":"2024-05-08T18:04:43.845626+0000","last_active":"2024-05-08T18:05:30.861114+0000","last_peered":"2024-05-08T18:05:30.861114+0000","last_clean":"2024-05-08T18:05:30.861114+0000","last_became_active":"2024-05-08T18:04:43.845214+0000","last_became_peered":"2024-05-08T18:04:43.845214+0000","last_unstale":"2024-05-08T18:05:30.861114+0000","last_undegraded":"2024-05-08T18:05:30.861114+0000","last_fullsized":"2024-05-08T18:05:30.861114+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:00.592659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.865365+0000","last_change":"2024-05-08T18:04:47.823059+0000","last_active":"2024-05-08T18:05:30.865365+0000","last_peered":"2024-05-08T18:05:30.865365+0000","last_clean":"2024-05-08T18:05:30.865365+0000","last_became_active":"2024-05-08T18:04:47.822748+0000","last_became_peered":"2024-05-08T18:04:47.822748+0000","last_unstale":"2024-05-08T18:05:30.865365+0000","last_undegraded":"2024-05-08T18:05:30.865365+0000","last_fullsized":"2024-05-08T18:05:30.865365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:20.980114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"29'31","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.937310+0000","last_change":"2024-05-08T18:04:49.831735+0000","last_active":"2024-05-08T18:06:51.937310+0000","last_peered":"2024-05-08T18:06:51.937310+0000","last_clean":"2024-05-08T18:06:51.937310+0000","last_became_active":"2024-05-08T18:04:49.831440+0000","last_became_peered":"2024-05-08T18:04:49.831440+0000","last_unstale":"2024-05-08T18:06:51.937310+0000","last_undegraded":"2024-05-08T18:06:51.937310+0000","last_fullsized":"2024-05-08T18:06:51.937310+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:40.102877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857159+0000","last_change":"2024-05-08T18:04:51.844443+0000","last_active":"2024-05-08T18:05:30.857159+0000","last_peered":"2024-05-08T18:05:30.857159+0000","last_clean":"2024-05-08T18:05:30.857159+0000","last_became_active":"2024-05-08T18:04:51.844125+0000","last_became_peered":"2024-05-08T18:04:51.844125+0000","last_unstale":"2024-05-08T18:05:30.857159+0000","last_undegraded":"2024-05-08T18:05:30.857159+0000","last_fullsized":"2024-05-08T18:05:30.857159+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:22.017553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862839+0000","last_change":"2024-05-08T18:04:53.849962+0000","last_active":"2024-05-08T18:05:30.862839+0000","last_peered":"2024-05-08T18:05:30.862839+0000","last_clean":"2024-05-08T18:05:30.862839+0000","last_became_active":"2024-05-08T18:04:53.849514+0000","last_became_peered":"2024-05-08T18:04:53.849514+0000","last_unstale":"2024-05-08T18:05:30.862839+0000","last_undegraded":"2024-05-08T18:05:30.862839+0000","last_fullsized":"2024-05-08T18:05:30.862839+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:31.875101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"29'4","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399430+0000","last_change":"2024-05-08T18:04:45.822495+0000","last_active":"2024-05-08T18:06:53.399430+0000","last_peered":"2024-05-08T18:06:53.399430+0000","last_clean":"2024-05-08T18:06:53.399430+0000","last_became_active":"2024-05-08T18:04:45.822204+0000","last_became_peered":"2024-05-08T18:04:45.822204+0000","last_unstale":"2024-05-08T18:06:53.399430+0000","last_undegraded":"2024-05-08T18:06:53.399430+0000","last_fullsized":"2024-05-08T18:06:53.399430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:06.207449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.058832+0000","last_change":"2024-05-08T18:04:43.835654+0000","last_active":"2024-05-08T18:07:54.058832+0000","last_peered":"2024-05-08T18:07:54.058832+0000","last_clean":"2024-05-08T18:07:54.058832+0000","last_became_active":"2024-05-08T18:04:43.834678+0000","last_became_peered":"2024-05-08T18:04:43.834678+0000","last_unstale":"2024-05-08T18:07:54.058832+0000","last_undegraded":"2024-05-08T18:07:54.058832+0000","last_fullsized":"2024-05-08T18:07:54.058832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:23:16.295599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859176+0000","last_change":"2024-05-08T18:04:47.820376+0000","last_active":"2024-05-08T18:05:30.859176+0000","last_peered":"2024-05-08T18:05:30.859176+0000","last_clean":"2024-05-08T18:05:30.859176+0000","last_became_active":"2024-05-08T18:04:47.820036+0000","last_became_peered":"2024-05-08T18:04:47.820036+0000","last_unstale":"2024-05-08T18:05:30.859176+0000","last_undegraded":"2024-05-08T18:05:30.859176+0000","last_fullsized":"2024-05-08T18:05:30.859176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:10:33.041988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"29'28","reported_seq":68,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.960416+0000","last_change":"2024-05-08T18:04:49.832032+0000","last_active":"2024-05-08T18:06:51.960416+0000","last_peered":"2024-05-08T18:06:51.960416+0000","last_clean":"2024-05-08T18:06:51.960416+0000","last_became_active":"2024-05-08T18:04:49.830882+0000","last_became_peered":"2024-05-08T18:04:49.830882+0000","last_unstale":"2024-05-08T18:06:51.960416+0000","last_undegraded":"2024-05-08T18:06:51.960416+0000","last_fullsized":"2024-05-08T18:06:51.960416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:21.475550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":40,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857634+0000","last_change":"2024-05-08T18:04:51.844109+0000","last_active":"2024-05-08T18:05:30.857634+0000","last_peered":"2024-05-08T18:05:30.857634+0000","last_clean":"2024-05-08T18:05:30.857634+0000","last_became_active":"2024-05-08T18:04:51.842734+0000","last_became_peered":"2024-05-08T18:04:51.842734+0000","last_unstale":"2024-05-08T18:05:30.857634+0000","last_undegraded":"2024-05-08T18:05:30.857634+0000","last_fullsized":"2024-05-08T18:05:30.857634+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:47.953187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'10","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.495078+0000","last_change":"2024-05-08T18:04:53.845139+0000","last_active":"2024-05-08T18:07:53.495078+0000","last_peered":"2024-05-08T18:07:53.495078+0000","last_clean":"2024-05-08T18:07:53.495078+0000","last_became_active":"2024-05-08T18:04:53.844891+0000","last_became_peered":"2024-05-08T18:04:53.844891+0000","last_unstale":"2024-05-08T18:07:53.495078+0000","last_undegraded":"2024-05-08T18:07:53.495078+0000","last_fullsized":"2024-05-08T18:07:53.495078+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:32.203551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"29'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.860618+0000","last_change":"2024-05-08T18:04:45.809216+0000","last_active":"2024-05-08T18:07:53.860618+0000","last_peered":"2024-05-08T18:07:53.860618+0000","last_clean":"2024-05-08T18:07:53.860618+0000","last_became_active":"2024-05-08T18:04:45.808936+0000","last_became_peered":"2024-05-08T18:04:45.808936+0000","last_unstale":"2024-05-08T18:07:53.860618+0000","last_undegraded":"2024-05-08T18:07:53.860618+0000","last_fullsized":"2024-05-08T18:07:53.860618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:51.368967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866160+0000","last_change":"2024-05-08T18:04:43.837820+0000","last_active":"2024-05-08T18:05:30.866160+0000","last_peered":"2024-05-08T18:05:30.866160+0000","last_clean":"2024-05-08T18:05:30.866160+0000","last_became_active":"2024-05-08T18:04:43.837070+0000","last_became_peered":"2024-05-08T18:04:43.837070+0000","last_unstale":"2024-05-08T18:05:30.866160+0000","last_undegraded":"2024-05-08T18:05:30.866160+0000","last_fullsized":"2024-05-08T18:05:30.866160+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:50.043665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"29'2","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.565208+0000","last_change":"2024-05-08T18:04:47.824489+0000","last_active":"2024-05-08T18:06:50.565208+0000","last_peered":"2024-05-08T18:06:50.565208+0000","last_clean":"2024-05-08T18:06:50.565208+0000","last_became_active":"2024-05-08T18:04:47.824172+0000","last_became_peered":"2024-05-08T18:04:47.824172+0000","last_unstale":"2024-05-08T18:06:50.565208+0000","last_undegraded":"2024-05-08T18:06:50.565208+0000","last_fullsized":"2024-05-08T18:06:50.565208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:14.408614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"29'32","reported_seq":76,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.418071+0000","last_change":"2024-05-08T18:04:49.832422+0000","last_active":"2024-05-08T18:07:53.418071+0000","last_peered":"2024-05-08T18:07:53.418071+0000","last_clean":"2024-05-08T18:07:53.418071+0000","last_became_active":"2024-05-08T18:04:49.831803+0000","last_became_peered":"2024-05-08T18:04:49.831803+0000","last_unstale":"2024-05-08T18:07:53.418071+0000","last_undegraded":"2024-05-08T18:07:53.418071+0000","last_fullsized":"2024-05-08T18:07:53.418071+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:03:53.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858576+0000","last_change":"2024-05-08T18:04:51.846422+0000","last_active":"2024-05-08T18:05:30.858576+0000","last_peered":"2024-05-08T18:05:30.858576+0000","last_clean":"2024-05-08T18:05:30.858576+0000","last_became_active":"2024-05-08T18:04:51.846225+0000","last_became_peered":"2024-05-08T18:04:51.846225+0000","last_unstale":"2024-05-08T18:05:30.858576+0000","last_undegraded":"2024-05-08T18:05:30.858576+0000","last_fullsized":"2024-05-08T18:05:30.858576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:42:25.975522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"29'4","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.972855+0000","last_change":"2024-05-08T18:04:53.843030+0000","last_active":"2024-05-08T18:06:51.972855+0000","last_peered":"2024-05-08T18:06:51.972855+0000","last_clean":"2024-05-08T18:06:51.972855+0000","last_became_active":"2024-05-08T18:04:53.842429+0000","last_became_peered":"2024-05-08T18:04:53.842429+0000","last_unstale":"2024-05-08T18:06:51.972855+0000","last_undegraded":"2024-05-08T18:06:51.972855+0000","last_fullsized":"2024-05-08T18:06:51.972855+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:30.303540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"27'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866744+0000","last_change":"2024-05-08T18:04:45.818828+0000","last_active":"2024-05-08T18:05:30.866744+0000","last_peered":"2024-05-08T18:05:30.866744+0000","last_clean":"2024-05-08T18:05:30.866744+0000","last_became_active":"2024-05-08T18:04:45.818161+0000","last_became_peered":"2024-05-08T18:04:45.818161+0000","last_unstale":"2024-05-08T18:05:30.866744+0000","last_undegraded":"2024-05-08T18:05:30.866744+0000","last_fullsized":"2024-05-08T18:05:30.866744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:59.477939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859517+0000","last_change":"2024-05-08T18:04:43.843055+0000","last_active":"2024-05-08T18:05:30.859517+0000","last_peered":"2024-05-08T18:05:30.859517+0000","last_clean":"2024-05-08T18:05:30.859517+0000","last_became_active":"2024-05-08T18:04:43.842483+0000","last_became_peered":"2024-05-08T18:04:43.842483+0000","last_unstale":"2024-05-08T18:05:30.859517+0000","last_undegraded":"2024-05-08T18:05:30.859517+0000","last_fullsized":"2024-05-08T18:05:30.859517+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:11.225102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"29'36","reported_seq":106,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995781+0000","last_change":"2024-05-08T18:04:45.808186+0000","last_active":"2024-05-08T18:07:53.995781+0000","last_peered":"2024-05-08T18:07:53.995781+0000","last_clean":"2024-05-08T18:07:53.995781+0000","last_became_active":"2024-05-08T18:04:45.807847+0000","last_became_peered":"2024-05-08T18:04:45.807847+0000","last_unstale":"2024-05-08T18:07:53.995781+0000","last_undegraded":"2024-05-08T18:07:53.995781+0000","last_fullsized":"2024-05-08T18:07:53.995781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:00:26.846485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":44,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858145+0000","last_change":"2024-05-08T18:04:43.836530+0000","last_active":"2024-05-08T18:05:30.858145+0000","last_peered":"2024-05-08T18:05:30.858145+0000","last_clean":"2024-05-08T18:05:30.858145+0000","last_became_active":"2024-05-08T18:04:43.836045+0000","last_became_peered":"2024-05-08T18:04:43.836045+0000","last_unstale":"2024-05-08T18:05:30.858145+0000","last_undegraded":"2024-05-08T18:05:30.858145+0000","last_fullsized":"2024-05-08T18:05:30.858145+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:22.454546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3,2],"acting":[0,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"29'11","reported_seq":68,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.782514+0000","last_change":"2024-05-08T18:04:45.826393+0000","last_active":"2024-05-08T18:07:53.782514+0000","last_peered":"2024-05-08T18:07:53.782514+0000","last_clean":"2024-05-08T18:07:53.782514+0000","last_became_active":"2024-05-08T18:04:45.825506+0000","last_became_peered":"2024-05-08T18:04:45.825506+0000","last_unstale":"2024-05-08T18:07:53.782514+0000","last_undegraded":"2024-05-08T18:07:53.782514+0000","last_fullsized":"2024-05-08T18:07:53.782514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:23:18.600988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":46,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"29'5","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.990901+0000","last_change":"2024-05-08T18:04:43.832854+0000","last_active":"2024-05-08T18:07:53.990901+0000","last_peered":"2024-05-08T18:07:53.990901+0000","last_clean":"2024-05-08T18:07:53.990901+0000","last_became_active":"2024-05-08T18:04:43.832406+0000","last_became_peered":"2024-05-08T18:04:43.832406+0000","last_unstale":"2024-05-08T18:07:53.990901+0000","last_undegraded":"2024-05-08T18:07:53.990901+0000","last_fullsized":"2024-05-08T18:07:53.990901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:16.456364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":29,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"29'2","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.774175+0000","last_change":"2024-05-08T18:04:45.807187+0000","last_active":"2024-05-08T18:07:53.774175+0000","last_peered":"2024-05-08T18:07:53.774175+0000","last_clean":"2024-05-08T18:07:53.774175+0000","last_became_active":"2024-05-08T18:04:45.806646+0000","last_became_peered":"2024-05-08T18:04:45.806646+0000","last_unstale":"2024-05-08T18:07:53.774175+0000","last_undegraded":"2024-05-08T18:07:53.774175+0000","last_fullsized":"2024-05-08T18:07:53.774175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:30.458103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"29'5","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.374758+0000","last_change":"2024-05-08T18:04:43.836158+0000","last_active":"2024-05-08T18:06:53.374758+0000","last_peered":"2024-05-08T18:06:53.374758+0000","last_clean":"2024-05-08T18:06:53.374758+0000","last_became_active":"2024-05-08T18:04:43.835163+0000","last_became_peered":"2024-05-08T18:04:43.835163+0000","last_unstale":"2024-05-08T18:06:53.374758+0000","last_undegraded":"2024-05-08T18:06:53.374758+0000","last_fullsized":"2024-05-08T18:06:53.374758+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:55.493870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":13,"num_write_kb":4746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,1],"acting":[0,3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"29'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.861341+0000","last_change":"2024-05-08T18:04:45.816663+0000","last_active":"2024-05-08T18:07:53.861341+0000","last_peered":"2024-05-08T18:07:53.861341+0000","last_clean":"2024-05-08T18:07:53.861341+0000","last_became_active":"2024-05-08T18:04:45.816391+0000","last_became_peered":"2024-05-08T18:04:45.816391+0000","last_unstale":"2024-05-08T18:07:53.861341+0000","last_undegraded":"2024-05-08T18:07:53.861341+0000","last_fullsized":"2024-05-08T18:07:53.861341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:02.599918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'4","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861292+0000","last_change":"2024-05-08T18:04:43.836083+0000","last_active":"2024-05-08T18:05:30.861292+0000","last_peered":"2024-05-08T18:05:30.861292+0000","last_clean":"2024-05-08T18:05:30.861292+0000","last_became_active":"2024-05-08T18:04:43.835268+0000","last_became_peered":"2024-05-08T18:04:43.835268+0000","last_unstale":"2024-05-08T18:05:30.861292+0000","last_undegraded":"2024-05-08T18:05:30.861292+0000","last_fullsized":"2024-05-08T18:05:30.861292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:47.971014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"29'12","reported_seq":77,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.035458+0000","last_change":"2024-05-08T18:04:45.820787+0000","last_active":"2024-05-08T18:07:54.035458+0000","last_peered":"2024-05-08T18:07:54.035458+0000","last_clean":"2024-05-08T18:07:54.035458+0000","last_became_active":"2024-05-08T18:04:45.819699+0000","last_became_peered":"2024-05-08T18:04:45.819699+0000","last_unstale":"2024-05-08T18:07:54.035458+0000","last_undegraded":"2024-05-08T18:07:54.035458+0000","last_fullsized":"2024-05-08T18:07:54.035458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:18.539896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862510+0000","last_change":"2024-05-08T18:04:43.837155+0000","last_active":"2024-05-08T18:05:30.862510+0000","last_peered":"2024-05-08T18:05:30.862510+0000","last_clean":"2024-05-08T18:05:30.862510+0000","last_became_active":"2024-05-08T18:04:43.836735+0000","last_became_peered":"2024-05-08T18:04:43.836735+0000","last_unstale":"2024-05-08T18:05:30.862510+0000","last_undegraded":"2024-05-08T18:05:30.862510+0000","last_fullsized":"2024-05-08T18:05:30.862510+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:54.711833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"29'14","reported_seq":70,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.219716+0000","last_change":"2024-05-08T18:04:45.812327+0000","last_active":"2024-05-08T18:06:50.219716+0000","last_peered":"2024-05-08T18:06:50.219716+0000","last_clean":"2024-05-08T18:06:50.219716+0000","last_became_active":"2024-05-08T18:04:45.811953+0000","last_became_peered":"2024-05-08T18:04:45.811953+0000","last_unstale":"2024-05-08T18:06:50.219716+0000","last_undegraded":"2024-05-08T18:06:50.219716+0000","last_fullsized":"2024-05-08T18:06:50.219716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:11:17.190072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867588+0000","last_change":"2024-05-08T18:04:43.843298+0000","last_active":"2024-05-08T18:05:30.867588+0000","last_peered":"2024-05-08T18:05:30.867588+0000","last_clean":"2024-05-08T18:05:30.867588+0000","last_became_active":"2024-05-08T18:04:43.842179+0000","last_became_peered":"2024-05-08T18:04:43.842179+0000","last_unstale":"2024-05-08T18:05:30.867588+0000","last_undegraded":"2024-05-08T18:05:30.867588+0000","last_fullsized":"2024-05-08T18:05:30.867588+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:55.493870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,0],"acting":[2,3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"29'8","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063691+0000","last_change":"2024-05-08T18:04:45.812893+0000","last_active":"2024-05-08T18:07:54.063691+0000","last_peered":"2024-05-08T18:07:54.063691+0000","last_clean":"2024-05-08T18:07:54.063691+0000","last_became_active":"2024-05-08T18:04:45.812637+0000","last_became_peered":"2024-05-08T18:04:45.812637+0000","last_unstale":"2024-05-08T18:07:54.063691+0000","last_undegraded":"2024-05-08T18:07:54.063691+0000","last_fullsized":"2024-05-08T18:07:54.063691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:18.973014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"29'6","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.981626+0000","last_change":"2024-05-08T18:04:43.834951+0000","last_active":"2024-05-08T18:07:53.981626+0000","last_peered":"2024-05-08T18:07:53.981626+0000","last_clean":"2024-05-08T18:07:53.981626+0000","last_became_active":"2024-05-08T18:04:43.834491+0000","last_became_peered":"2024-05-08T18:04:43.834491+0000","last_unstale":"2024-05-08T18:07:53.981626+0000","last_undegraded":"2024-05-08T18:07:53.981626+0000","last_fullsized":"2024-05-08T18:07:53.981626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:24:46.108691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"29'8","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.041264+0000","last_change":"2024-05-08T18:04:45.815948+0000","last_active":"2024-05-08T18:07:54.041264+0000","last_peered":"2024-05-08T18:07:54.041264+0000","last_clean":"2024-05-08T18:07:54.041264+0000","last_became_active":"2024-05-08T18:04:45.815677+0000","last_became_peered":"2024-05-08T18:04:45.815677+0000","last_unstale":"2024-05-08T18:07:54.041264+0000","last_undegraded":"2024-05-08T18:07:54.041264+0000","last_fullsized":"2024-05-08T18:07:54.041264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:16.870297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857605+0000","last_change":"2024-05-08T18:04:43.846054+0000","last_active":"2024-05-08T18:05:30.857605+0000","last_peered":"2024-05-08T18:05:30.857605+0000","last_clean":"2024-05-08T18:05:30.857605+0000","last_became_active":"2024-05-08T18:04:43.843826+0000","last_became_peered":"2024-05-08T18:04:43.843826+0000","last_unstale":"2024-05-08T18:05:30.857605+0000","last_undegraded":"2024-05-08T18:05:30.857605+0000","last_fullsized":"2024-05-08T18:05:30.857605+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:08:10.712708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399417+0000","last_change":"2024-05-08T18:04:45.827535+0000","last_active":"2024-05-08T18:06:53.399417+0000","last_peered":"2024-05-08T18:06:53.399417+0000","last_clean":"2024-05-08T18:06:53.399417+0000","last_became_active":"2024-05-08T18:04:45.827252+0000","last_became_peered":"2024-05-08T18:04:45.827252+0000","last_unstale":"2024-05-08T18:06:53.399417+0000","last_undegraded":"2024-05-08T18:06:53.399417+0000","last_fullsized":"2024-05-08T18:06:53.399417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:07.190532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858630+0000","last_change":"2024-05-08T18:04:43.820995+0000","last_active":"2024-05-08T18:05:30.858630+0000","last_peered":"2024-05-08T18:05:30.858630+0000","last_clean":"2024-05-08T18:05:30.858630+0000","last_became_active":"2024-05-08T18:04:43.820360+0000","last_became_peered":"2024-05-08T18:04:43.820360+0000","last_unstale":"2024-05-08T18:05:30.858630+0000","last_undegraded":"2024-05-08T18:05:30.858630+0000","last_fullsized":"2024-05-08T18:05:30.858630+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:54.711833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860974+0000","last_change":"2024-05-08T18:04:45.823656+0000","last_active":"2024-05-08T18:05:30.860974+0000","last_peered":"2024-05-08T18:05:30.860974+0000","last_clean":"2024-05-08T18:05:30.860974+0000","last_became_active":"2024-05-08T18:04:45.823385+0000","last_became_peered":"2024-05-08T18:04:45.823385+0000","last_unstale":"2024-05-08T18:05:30.860974+0000","last_undegraded":"2024-05-08T18:05:30.860974+0000","last_fullsized":"2024-05-08T18:05:30.860974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:26.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'6","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.755149+0000","last_change":"2024-05-08T18:04:43.836532+0000","last_active":"2024-05-08T18:07:53.755149+0000","last_peered":"2024-05-08T18:07:53.755149+0000","last_clean":"2024-05-08T18:07:53.755149+0000","last_became_active":"2024-05-08T18:04:43.835838+0000","last_became_peered":"2024-05-08T18:04:43.835838+0000","last_unstale":"2024-05-08T18:07:53.755149+0000","last_undegraded":"2024-05-08T18:07:53.755149+0000","last_fullsized":"2024-05-08T18:07:53.755149+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:11.225102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8,"num_write":30,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"29'10","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995916+0000","last_change":"2024-05-08T18:04:45.810992+0000","last_active":"2024-05-08T18:07:53.995916+0000","last_peered":"2024-05-08T18:07:53.995916+0000","last_clean":"2024-05-08T18:07:53.995916+0000","last_became_active":"2024-05-08T18:04:45.810703+0000","last_became_peered":"2024-05-08T18:04:45.810703+0000","last_unstale":"2024-05-08T18:07:53.995916+0000","last_undegraded":"2024-05-08T18:07:53.995916+0000","last_fullsized":"2024-05-08T18:07:53.995916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:05.828431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"29'3","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744531+0000","last_change":"2024-05-08T18:04:43.836123+0000","last_active":"2024-05-08T18:06:58.744531+0000","last_peered":"2024-05-08T18:06:58.744531+0000","last_clean":"2024-05-08T18:06:58.744531+0000","last_became_active":"2024-05-08T18:04:43.835518+0000","last_became_peered":"2024-05-08T18:04:43.835518+0000","last_unstale":"2024-05-08T18:06:58.744531+0000","last_undegraded":"2024-05-08T18:06:58.744531+0000","last_fullsized":"2024-05-08T18:06:58.744531+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:06.338736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,3],"acting":[1,2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"29'4","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.219429+0000","last_change":"2024-05-08T18:04:45.821376+0000","last_active":"2024-05-08T18:06:50.219429+0000","last_peered":"2024-05-08T18:06:50.219429+0000","last_clean":"2024-05-08T18:06:50.219429+0000","last_became_active":"2024-05-08T18:04:45.820746+0000","last_became_peered":"2024-05-08T18:04:45.820746+0000","last_unstale":"2024-05-08T18:06:50.219429+0000","last_undegraded":"2024-05-08T18:06:50.219429+0000","last_fullsized":"2024-05-08T18:06:50.219429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:32.833248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859364+0000","last_change":"2024-05-08T18:04:43.847998+0000","last_active":"2024-05-08T18:05:30.859364+0000","last_peered":"2024-05-08T18:05:30.859364+0000","last_clean":"2024-05-08T18:05:30.859364+0000","last_became_active":"2024-05-08T18:04:43.844404+0000","last_became_peered":"2024-05-08T18:04:43.844404+0000","last_unstale":"2024-05-08T18:05:30.859364+0000","last_undegraded":"2024-05-08T18:05:30.859364+0000","last_fullsized":"2024-05-08T18:05:30.859364+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:28.450844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"29'4","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.400051+0000","last_change":"2024-05-08T18:04:45.809582+0000","last_active":"2024-05-08T18:06:53.400051+0000","last_peered":"2024-05-08T18:06:53.400051+0000","last_clean":"2024-05-08T18:06:53.400051+0000","last_became_active":"2024-05-08T18:04:45.809271+0000","last_became_peered":"2024-05-08T18:04:45.809271+0000","last_unstale":"2024-05-08T18:06:53.400051+0000","last_undegraded":"2024-05-08T18:06:53.400051+0000","last_fullsized":"2024-05-08T18:06:53.400051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:32.141179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861152+0000","last_change":"2024-05-08T18:04:43.844366+0000","last_active":"2024-05-08T18:05:30.861152+0000","last_peered":"2024-05-08T18:05:30.861152+0000","last_clean":"2024-05-08T18:05:30.861152+0000","last_became_active":"2024-05-08T18:04:43.838125+0000","last_became_peered":"2024-05-08T18:04:43.838125+0000","last_unstale":"2024-05-08T18:05:30.861152+0000","last_undegraded":"2024-05-08T18:05:30.861152+0000","last_fullsized":"2024-05-08T18:05:30.861152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:24.149010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"29'15","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070567+0000","last_change":"2024-05-08T18:04:45.828761+0000","last_active":"2024-05-08T18:07:54.070567+0000","last_peered":"2024-05-08T18:07:54.070567+0000","last_clean":"2024-05-08T18:07:54.070567+0000","last_became_active":"2024-05-08T18:04:45.828488+0000","last_became_peered":"2024-05-08T18:04:45.828488+0000","last_unstale":"2024-05-08T18:07:54.070567+0000","last_undegraded":"2024-05-08T18:07:54.070567+0000","last_fullsized":"2024-05-08T18:07:54.070567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:27:23.439814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867753+0000","last_change":"2024-05-08T18:04:43.843455+0000","last_active":"2024-05-08T18:05:30.867753+0000","last_peered":"2024-05-08T18:05:30.867753+0000","last_clean":"2024-05-08T18:05:30.867753+0000","last_became_active":"2024-05-08T18:04:43.842670+0000","last_became_peered":"2024-05-08T18:04:43.842670+0000","last_unstale":"2024-05-08T18:05:30.867753+0000","last_undegraded":"2024-05-08T18:05:30.867753+0000","last_fullsized":"2024-05-08T18:05:30.867753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:01:49.620924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"29'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.547055+0000","last_change":"2024-05-08T18:04:45.812074+0000","last_active":"2024-05-08T18:06:09.547055+0000","last_peered":"2024-05-08T18:06:09.547055+0000","last_clean":"2024-05-08T18:06:09.547055+0000","last_became_active":"2024-05-08T18:04:45.810562+0000","last_became_peered":"2024-05-08T18:04:45.810562+0000","last_unstale":"2024-05-08T18:06:09.547055+0000","last_undegraded":"2024-05-08T18:06:09.547055+0000","last_fullsized":"2024-05-08T18:06:09.547055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:58.784190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864727+0000","last_change":"2024-05-08T18:04:43.838746+0000","last_active":"2024-05-08T18:05:30.864727+0000","last_peered":"2024-05-08T18:05:30.864727+0000","last_clean":"2024-05-08T18:05:30.864727+0000","last_became_active":"2024-05-08T18:04:43.835943+0000","last_became_peered":"2024-05-08T18:04:43.835943+0000","last_unstale":"2024-05-08T18:05:30.864727+0000","last_undegraded":"2024-05-08T18:05:30.864727+0000","last_fullsized":"2024-05-08T18:05:30.864727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:27:18.106771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857870+0000","last_change":"2024-05-08T18:04:43.836378+0000","last_active":"2024-05-08T18:05:30.857870+0000","last_peered":"2024-05-08T18:05:30.857870+0000","last_clean":"2024-05-08T18:05:30.857870+0000","last_became_active":"2024-05-08T18:04:43.836007+0000","last_became_peered":"2024-05-08T18:04:43.836007+0000","last_unstale":"2024-05-08T18:05:30.857870+0000","last_undegraded":"2024-05-08T18:05:30.857870+0000","last_fullsized":"2024-05-08T18:05:30.857870+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:57:00.868492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"29'8","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.219707+0000","last_change":"2024-05-08T18:04:45.812333+0000","last_active":"2024-05-08T18:06:50.219707+0000","last_peered":"2024-05-08T18:06:50.219707+0000","last_clean":"2024-05-08T18:06:50.219707+0000","last_became_active":"2024-05-08T18:04:45.811648+0000","last_became_peered":"2024-05-08T18:04:45.811648+0000","last_unstale":"2024-05-08T18:06:50.219707+0000","last_undegraded":"2024-05-08T18:06:50.219707+0000","last_fullsized":"2024-05-08T18:06:50.219707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:18.539896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":31,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"29'5","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.740580+0000","last_change":"2024-05-08T18:04:43.836500+0000","last_active":"2024-05-08T18:07:03.740580+0000","last_peered":"2024-05-08T18:07:03.740580+0000","last_clean":"2024-05-08T18:07:03.740580+0000","last_became_active":"2024-05-08T18:04:43.835712+0000","last_became_peered":"2024-05-08T18:04:43.835712+0000","last_unstale":"2024-05-08T18:07:03.740580+0000","last_undegraded":"2024-05-08T18:07:03.740580+0000","last_fullsized":"2024-05-08T18:07:03.740580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:06.338736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8,"num_write":21,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"29'10","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.775008+0000","last_change":"2024-05-08T18:04:45.812258+0000","last_active":"2024-05-08T18:07:53.775008+0000","last_peered":"2024-05-08T18:07:53.775008+0000","last_clean":"2024-05-08T18:07:53.775008+0000","last_became_active":"2024-05-08T18:04:45.811921+0000","last_became_peered":"2024-05-08T18:04:45.811921+0000","last_unstale":"2024-05-08T18:07:53.775008+0000","last_undegraded":"2024-05-08T18:07:53.775008+0000","last_fullsized":"2024-05-08T18:07:53.775008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:00:26.846485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2256,"ondisk_log_size":2256,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":13385,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1402,"num_read_kb":1188,"num_write":531,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":10826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1275,"ondisk_log_size":1275,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":961,"num_read_kb":961,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":25809792,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":138,"num_read_kb":87,"num_write":309,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34422784,"data_stored":34422784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":256,"acting":256,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705707,"num_pgs":179,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48332,"kb_used_data":8836,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323508,"statfs":{"total":96636764160,"available":96587272192,"internally_reserved":0,"allocated":9048064,"data_stored":8743050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705708,"num_pgs":153,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48848,"kb_used_data":9352,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322992,"statfs":{"total":96636764160,"available":96586743808,"internally_reserved":0,"allocated":9576448,"data_stored":9329849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705708,"num_pgs":157,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48872,"kb_used_data":9376,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322968,"statfs":{"total":96636764160,"available":96586719232,"internally_reserved":0,"allocated":9601024,"data_stored":9332048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39980,"kb_used_data":8804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331860,"statfs":{"total":96636764160,"available":96595824640,"internally_reserved":0,"allocated":9015296,"data_stored":8741305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":8605696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":8605696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":8605696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":8605696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":4269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T18:08:01.521 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T18:08:01.521 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T18:08:01.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.657+0000 7f10cb408640 1 Processor -- start 2024-05-08T18:08:01.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.657+0000 7f10cb408640 1 -- start start 2024-05-08T18:08:01.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.658+0000 7f10cb408640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f10c41304f0 0x7f10c41328e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:01.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.658+0000 7f10cb408640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10c4129c00 con 0x7f10c412a0c0 2024-05-08T18:08:01.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.658+0000 7f10cb408640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f10c412abb0 con 0x7f10c41304f0 2024-05-08T18:08:01.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.658+0000 7f10c89a3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f10c41304f0 0x7f10c41328e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:01.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.658+0000 7f10c91a4640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f10c412a0c0 0x7f10c412a4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45174/0 (socket says 172.21.15.46:45174) 2024-05-08T18:08:01.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.658+0000 7f10c91a4640 1 -- 172.21.15.46:0/1923786126 learned_addr learned my addr 172.21.15.46:0/1923786126 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:08:01.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.658+0000 7f10c89a3640 1 -- 172.21.15.46:0/1923786126 >> v1:172.21.15.46:6789/0 conn(0x7f10c412a0c0 legacy=0x7f10c412a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:01.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.658+0000 7f10bbfff640 1 -- 172.21.15.46:0/1923786126 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3413854622 0 0) 0x7f10c4129c00 con 0x7f10c412a0c0 2024-05-08T18:08:01.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.658+0000 7f10c89a3640 1 -- 172.21.15.46:0/1923786126 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c409acc0 con 0x7f10c41304f0 2024-05-08T18:08:01.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.659+0000 7f10c89a3640 1 --2- 172.21.15.46:0/1923786126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c41304f0 0x7f10c41328e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009720 tx=0x7f10b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c920287f98085226 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:01.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.659+0000 7f10bbfff640 1 -- 172.21.15.46:0/1923786126 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10b4035020 con 0x7f10c41304f0 2024-05-08T18:08:01.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.659+0000 7f10bbfff640 1 -- 172.21.15.46:0/1923786126 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10b4031760 con 0x7f10c41304f0 2024-05-08T18:08:01.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.659+0000 7f10bbfff640 1 -- 172.21.15.46:0/1923786126 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10b4031a60 con 0x7f10c41304f0 2024-05-08T18:08:01.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.660+0000 7f10cb408640 1 -- 172.21.15.46:0/1923786126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c41304f0 msgr2=0x7f10c41328e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:01.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.660+0000 7f10cb408640 1 --2- 172.21.15.46:0/1923786126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c41304f0 0x7f10c41328e0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f10b4009720 tx=0x7f10b402b8d0 comp rx=0 tx=0).stop 2024-05-08T18:08:01.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.660+0000 7f10cb408640 1 -- 172.21.15.46:0/1923786126 shutdown_connections 2024-05-08T18:08:01.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.660+0000 7f10cb408640 1 --2- 172.21.15.46:0/1923786126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c41304f0 0x7f10c41328e0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:01.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.660+0000 7f10cb408640 1 -- 172.21.15.46:0/1923786126 >> 172.21.15.46:0/1923786126 conn(0x7f10c409d7a0 msgr2=0x7f10c409dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:01.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.660+0000 7f10cb408640 1 -- 172.21.15.46:0/1923786126 shutdown_connections 2024-05-08T18:08:01.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.660+0000 7f10cb408640 1 -- 172.21.15.46:0/1923786126 wait complete. 2024-05-08T18:08:01.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.660+0000 7f10cb408640 1 Processor -- start 2024-05-08T18:08:01.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.661+0000 7f10cb408640 1 -- start start 2024-05-08T18:08:01.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.661+0000 7f10cb408640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c412a0c0 0x7f10c4115060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:01.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.661+0000 7f10cb408640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f10c4114910 con 0x7f10c412a0c0 2024-05-08T18:08:01.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.661+0000 7f10c91a4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c412a0c0 0x7f10c4115060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:01.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.661+0000 7f10c91a4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c412a0c0 0x7f10c4115060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52038/0 (socket says 172.21.15.46:52038) 2024-05-08T18:08:01.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.661+0000 7f10c91a4640 1 -- 172.21.15.46:0/2991889071 learned_addr learned my addr 172.21.15.46:0/2991889071 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:08:01.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.661+0000 7f10c91a4640 1 -- 172.21.15.46:0/2991889071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c41175c0 con 0x7f10c412a0c0 2024-05-08T18:08:01.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.661+0000 7f10c91a4640 1 --2- 172.21.15.46:0/2991889071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c412a0c0 0x7f10c4115060 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f10c000c3d0 tx=0x7f10c000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:01.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.662+0000 7f10b9ffb640 1 -- 172.21.15.46:0/2991889071 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10c00074d0 con 0x7f10c412a0c0 2024-05-08T18:08:01.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.662+0000 7f10b9ffb640 1 -- 172.21.15.46:0/2991889071 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c00047e0 con 0x7f10c412a0c0 2024-05-08T18:08:01.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.662+0000 7f10b9ffb640 1 -- 172.21.15.46:0/2991889071 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10c0007670 con 0x7f10c412a0c0 2024-05-08T18:08:01.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.662+0000 7f10cb408640 1 -- 172.21.15.46:0/2991889071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c412d430 con 0x7f10c412a0c0 2024-05-08T18:08:01.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.662+0000 7f10cb408640 1 -- 172.21.15.46:0/2991889071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c41155a0 con 0x7f10c412a0c0 2024-05-08T18:08:01.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.663+0000 7f10b9ffb640 1 -- 172.21.15.46:0/2991889071 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f10c0011070 con 0x7f10c412a0c0 2024-05-08T18:08:01.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.663+0000 7f10b9ffb640 1 --2- 172.21.15.46:0/2991889071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f10a803d2b0 0x7f10a803f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:01.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.663+0000 7f10b9ffb640 1 -- 172.21.15.46:0/2991889071 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f10c0051990 con 0x7f10c412a0c0 2024-05-08T18:08:01.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.663+0000 7f10cb408640 1 -- 172.21.15.46:0/2991889071 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c40640a0 con 0x7f10c412a0c0 2024-05-08T18:08:01.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.663+0000 7f10c89a3640 1 --2- 172.21.15.46:0/2991889071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f10a803d2b0 0x7f10a803f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:01.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.664+0000 7f10c89a3640 1 --2- 172.21.15.46:0/2991889071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f10a803d2b0 0x7f10a803f770 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f10b4036000 tx=0x7f10b4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:01.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.668+0000 7f10b9ffb640 1 -- 172.21.15.46:0/2991889071 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f10c41175c0 con 0x7f10c412a0c0 2024-05-08T18:08:01.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.855+0000 7f10cb408640 1 -- 172.21.15.46:0/2991889071 --> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f10c4069d70 con 0x7f10a803d2b0 2024-05-08T18:08:01.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.872+0000 7f10b9ffb640 1 -- 172.21.15.46:0/2991889071 <== mgr.4098 v2:172.21.15.50:6824/2602185226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665903 (secure 0 0 0) 0x7f10c4069d70 con 0x7f10a803d2b0 2024-05-08T18:08:01.877 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:08:01.882 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T18:08:01.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.876+0000 7f10cb408640 1 -- 172.21.15.46:0/2991889071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f10a803d2b0 msgr2=0x7f10a803f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:01.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.876+0000 7f10cb408640 1 --2- 172.21.15.46:0/2991889071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f10a803d2b0 0x7f10a803f770 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f10b4036000 tx=0x7f10b4008030 comp rx=0 tx=0).stop 2024-05-08T18:08:01.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.876+0000 7f10cb408640 1 -- 172.21.15.46:0/2991889071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c412a0c0 msgr2=0x7f10c4115060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:01.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.876+0000 7f10cb408640 1 --2- 172.21.15.46:0/2991889071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c412a0c0 0x7f10c4115060 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f10c000c3d0 tx=0x7f10c000c890 comp rx=0 tx=0).stop 2024-05-08T18:08:01.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.877+0000 7f10cb408640 1 -- 172.21.15.46:0/2991889071 shutdown_connections 2024-05-08T18:08:01.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.877+0000 7f10cb408640 1 --2- 172.21.15.46:0/2991889071 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f10a803d2b0 0x7f10a803f770 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:01.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.877+0000 7f10cb408640 1 --2- 172.21.15.46:0/2991889071 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f10c412a0c0 0x7f10c4115060 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:01.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.877+0000 7f10cb408640 1 -- 172.21.15.46:0/2991889071 >> 172.21.15.46:0/2991889071 conn(0x7f10c409d7a0 msgr2=0x7f10c41b55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:01.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.877+0000 7f10cb408640 1 -- 172.21.15.46:0/2991889071 shutdown_connections 2024-05-08T18:08:01.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:01.877+0000 7f10cb408640 1 -- 172.21.15.46:0/2991889071 wait complete. 2024-05-08T18:08:01.885 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-05-08T18:08:01.727963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26416886,"num_objects":227,"num_object_clones":0,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":227,"num_whiteouts":0,"num_read":4834,"num_read_kb":4416,"num_write":2046,"num_write_kb":60584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4431,"ondisk_log_size":4431,"up":658,"acting":658,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":658,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":186032,"kb_used_data":36368,"kb_used_omap":6,"kb_used_meta":149625,"kb_avail":377301328,"statfs":{"total":386547056640,"available":386356559872,"internally_reserved":0,"allocated":37240832,"data_stored":36146252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":153216790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25815784,"num_objects":9,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":497,"num_read_kb":452,"num_write":384,"num_write_kb":29371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003905"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866950+0000","last_change":"2024-05-08T18:04:43.836900+0000","last_active":"2024-05-08T18:05:30.866950+0000","last_peered":"2024-05-08T18:05:30.866950+0000","last_clean":"2024-05-08T18:05:30.866950+0000","last_became_active":"2024-05-08T18:04:43.836496+0000","last_became_peered":"2024-05-08T18:04:43.836496+0000","last_unstale":"2024-05-08T18:05:30.866950+0000","last_undegraded":"2024-05-08T18:05:30.866950+0000","last_fullsized":"2024-05-08T18:05:30.866950+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:22.454546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'6","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.860763+0000","last_change":"2024-05-08T18:04:45.812340+0000","last_active":"2024-05-08T18:07:53.860763+0000","last_peered":"2024-05-08T18:07:53.860763+0000","last_clean":"2024-05-08T18:07:53.860763+0000","last_became_active":"2024-05-08T18:04:45.811368+0000","last_became_peered":"2024-05-08T18:04:45.811368+0000","last_unstale":"2024-05-08T18:07:53.860763+0000","last_undegraded":"2024-05-08T18:07:53.860763+0000","last_fullsized":"2024-05-08T18:07:53.860763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:36:45.459800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.206267+0000","last_change":"2024-05-08T18:04:43.835466+0000","last_active":"2024-05-08T18:06:50.206267+0000","last_peered":"2024-05-08T18:06:50.206267+0000","last_clean":"2024-05-08T18:06:50.206267+0000","last_became_active":"2024-05-08T18:04:43.834612+0000","last_became_peered":"2024-05-08T18:04:43.834612+0000","last_unstale":"2024-05-08T18:06:50.206267+0000","last_undegraded":"2024-05-08T18:06:50.206267+0000","last_fullsized":"2024-05-08T18:06:50.206267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:46.422109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"29'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.546966+0000","last_change":"2024-05-08T18:04:45.828749+0000","last_active":"2024-05-08T18:06:09.546966+0000","last_peered":"2024-05-08T18:06:09.546966+0000","last_clean":"2024-05-08T18:06:09.546966+0000","last_became_active":"2024-05-08T18:04:45.827965+0000","last_became_peered":"2024-05-08T18:04:45.827965+0000","last_unstale":"2024-05-08T18:06:09.546966+0000","last_undegraded":"2024-05-08T18:06:09.546966+0000","last_fullsized":"2024-05-08T18:06:09.546966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:56:23.117814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861383+0000","last_change":"2024-05-08T18:04:43.843806+0000","last_active":"2024-05-08T18:05:30.861383+0000","last_peered":"2024-05-08T18:05:30.861383+0000","last_clean":"2024-05-08T18:05:30.861383+0000","last_became_active":"2024-05-08T18:04:43.837456+0000","last_became_peered":"2024-05-08T18:04:43.837456+0000","last_unstale":"2024-05-08T18:05:30.861383+0000","last_undegraded":"2024-05-08T18:05:30.861383+0000","last_fullsized":"2024-05-08T18:05:30.861383+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:00:24.839225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"29'12","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063738+0000","last_change":"2024-05-08T18:04:45.825315+0000","last_active":"2024-05-08T18:07:54.063738+0000","last_peered":"2024-05-08T18:07:54.063738+0000","last_clean":"2024-05-08T18:07:54.063738+0000","last_became_active":"2024-05-08T18:04:45.825022+0000","last_became_peered":"2024-05-08T18:04:45.825022+0000","last_unstale":"2024-05-08T18:07:54.063738+0000","last_undegraded":"2024-05-08T18:07:54.063738+0000","last_fullsized":"2024-05-08T18:07:54.063738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:45:53.925354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863324+0000","last_change":"2024-05-08T18:04:43.842478+0000","last_active":"2024-05-08T18:05:30.863324+0000","last_peered":"2024-05-08T18:05:30.863324+0000","last_clean":"2024-05-08T18:05:30.863324+0000","last_became_active":"2024-05-08T18:04:43.842077+0000","last_became_peered":"2024-05-08T18:04:43.842077+0000","last_unstale":"2024-05-08T18:05:30.863324+0000","last_undegraded":"2024-05-08T18:05:30.863324+0000","last_fullsized":"2024-05-08T18:05:30.863324+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:06.338736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"29'16","reported_seq":70,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.035049+0000","last_change":"2024-05-08T18:04:45.824870+0000","last_active":"2024-05-08T18:07:54.035049+0000","last_peered":"2024-05-08T18:07:54.035049+0000","last_clean":"2024-05-08T18:07:54.035049+0000","last_became_active":"2024-05-08T18:04:45.824581+0000","last_became_peered":"2024-05-08T18:04:45.824581+0000","last_unstale":"2024-05-08T18:07:54.035049+0000","last_undegraded":"2024-05-08T18:07:54.035049+0000","last_fullsized":"2024-05-08T18:07:54.035049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:48.934445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859690+0000","last_change":"2024-05-08T18:04:43.833461+0000","last_active":"2024-05-08T18:05:30.859690+0000","last_peered":"2024-05-08T18:05:30.859690+0000","last_clean":"2024-05-08T18:05:30.859690+0000","last_became_active":"2024-05-08T18:04:43.832722+0000","last_became_peered":"2024-05-08T18:04:43.832722+0000","last_unstale":"2024-05-08T18:05:30.859690+0000","last_undegraded":"2024-05-08T18:05:30.859690+0000","last_fullsized":"2024-05-08T18:05:30.859690+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:11:15.182813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"29'8","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.738600+0000","last_change":"2024-05-08T18:04:45.808678+0000","last_active":"2024-05-08T18:06:09.738600+0000","last_peered":"2024-05-08T18:06:09.738600+0000","last_clean":"2024-05-08T18:06:09.738600+0000","last_became_active":"2024-05-08T18:04:45.808383+0000","last_became_peered":"2024-05-08T18:04:45.808383+0000","last_unstale":"2024-05-08T18:06:09.738600+0000","last_undegraded":"2024-05-08T18:06:09.738600+0000","last_fullsized":"2024-05-08T18:06:09.738600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:02.599918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862756+0000","last_change":"2024-05-08T18:04:43.841284+0000","last_active":"2024-05-08T18:05:30.862756+0000","last_peered":"2024-05-08T18:05:30.862756+0000","last_clean":"2024-05-08T18:05:30.862756+0000","last_became_active":"2024-05-08T18:04:43.840851+0000","last_became_peered":"2024-05-08T18:04:43.840851+0000","last_unstale":"2024-05-08T18:05:30.862756+0000","last_undegraded":"2024-05-08T18:05:30.862756+0000","last_fullsized":"2024-05-08T18:05:30.862756+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:10.360847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063767+0000","last_change":"2024-05-08T18:04:45.829481+0000","last_active":"2024-05-08T18:07:54.063767+0000","last_peered":"2024-05-08T18:07:54.063767+0000","last_clean":"2024-05-08T18:07:54.063767+0000","last_became_active":"2024-05-08T18:04:45.828334+0000","last_became_peered":"2024-05-08T18:04:45.828334+0000","last_unstale":"2024-05-08T18:07:54.063767+0000","last_undegraded":"2024-05-08T18:07:54.063767+0000","last_fullsized":"2024-05-08T18:07:54.063767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:20:31.411889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.978496+0000","last_change":"2024-05-08T18:04:43.833826+0000","last_active":"2024-05-08T18:07:53.978496+0000","last_peered":"2024-05-08T18:07:53.978496+0000","last_clean":"2024-05-08T18:07:53.978496+0000","last_became_active":"2024-05-08T18:04:43.833437+0000","last_became_peered":"2024-05-08T18:04:43.833437+0000","last_unstale":"2024-05-08T18:07:53.978496+0000","last_undegraded":"2024-05-08T18:07:53.978496+0000","last_fullsized":"2024-05-08T18:07:53.978496+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:46.740116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.868015+0000","last_change":"2024-05-08T18:04:45.812158+0000","last_active":"2024-05-08T18:05:30.868015+0000","last_peered":"2024-05-08T18:05:30.868015+0000","last_clean":"2024-05-08T18:05:30.868015+0000","last_became_active":"2024-05-08T18:04:45.811884+0000","last_became_peered":"2024-05-08T18:04:45.811884+0000","last_unstale":"2024-05-08T18:05:30.868015+0000","last_undegraded":"2024-05-08T18:05:30.868015+0000","last_fullsized":"2024-05-08T18:05:30.868015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:30.458103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858546+0000","last_change":"2024-05-08T18:04:43.821005+0000","last_active":"2024-05-08T18:05:30.858546+0000","last_peered":"2024-05-08T18:05:30.858546+0000","last_clean":"2024-05-08T18:05:30.858546+0000","last_became_active":"2024-05-08T18:04:43.820312+0000","last_became_peered":"2024-05-08T18:04:43.820312+0000","last_unstale":"2024-05-08T18:05:30.858546+0000","last_undegraded":"2024-05-08T18:05:30.858546+0000","last_fullsized":"2024-05-08T18:05:30.858546+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:40:40.407240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"29'7","reported_seq":61,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070848+0000","last_change":"2024-05-08T18:04:45.823448+0000","last_active":"2024-05-08T18:07:54.070848+0000","last_peered":"2024-05-08T18:07:54.070848+0000","last_clean":"2024-05-08T18:07:54.070848+0000","last_became_active":"2024-05-08T18:04:45.822620+0000","last_became_peered":"2024-05-08T18:04:45.822620+0000","last_unstale":"2024-05-08T18:07:54.070848+0000","last_undegraded":"2024-05-08T18:07:54.070848+0000","last_fullsized":"2024-05-08T18:07:54.070848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:08.184602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"29'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995728+0000","last_change":"2024-05-08T18:04:45.826814+0000","last_active":"2024-05-08T18:07:53.995728+0000","last_peered":"2024-05-08T18:07:53.995728+0000","last_clean":"2024-05-08T18:07:53.995728+0000","last_became_active":"2024-05-08T18:04:45.826493+0000","last_became_peered":"2024-05-08T18:04:45.826493+0000","last_unstale":"2024-05-08T18:07:53.995728+0000","last_undegraded":"2024-05-08T18:07:53.995728+0000","last_fullsized":"2024-05-08T18:07:53.995728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:12.401514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866321+0000","last_change":"2024-05-08T18:04:43.838700+0000","last_active":"2024-05-08T18:05:30.866321+0000","last_peered":"2024-05-08T18:05:30.866321+0000","last_clean":"2024-05-08T18:05:30.866321+0000","last_became_active":"2024-05-08T18:04:43.838332+0000","last_became_peered":"2024-05-08T18:04:43.838332+0000","last_unstale":"2024-05-08T18:05:30.866321+0000","last_undegraded":"2024-05-08T18:05:30.866321+0000","last_fullsized":"2024-05-08T18:05:30.866321+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:01:49.620924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"29'4","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399796+0000","last_change":"2024-05-08T18:04:45.807732+0000","last_active":"2024-05-08T18:06:53.399796+0000","last_peered":"2024-05-08T18:06:53.399796+0000","last_clean":"2024-05-08T18:06:53.399796+0000","last_became_active":"2024-05-08T18:04:45.807438+0000","last_became_peered":"2024-05-08T18:04:45.807438+0000","last_unstale":"2024-05-08T18:06:53.399796+0000","last_undegraded":"2024-05-08T18:06:53.399796+0000","last_fullsized":"2024-05-08T18:06:53.399796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:27:20.114031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867876+0000","last_change":"2024-05-08T18:04:43.843599+0000","last_active":"2024-05-08T18:05:30.867876+0000","last_peered":"2024-05-08T18:05:30.867876+0000","last_clean":"2024-05-08T18:05:30.867876+0000","last_became_active":"2024-05-08T18:04:43.842741+0000","last_became_peered":"2024-05-08T18:04:43.842741+0000","last_unstale":"2024-05-08T18:05:30.867876+0000","last_undegraded":"2024-05-08T18:05:30.867876+0000","last_fullsized":"2024-05-08T18:05:30.867876+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:54.711833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"29'11","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.002969+0000","last_change":"2024-05-08T18:04:45.811141+0000","last_active":"2024-05-08T18:07:54.002969+0000","last_peered":"2024-05-08T18:07:54.002969+0000","last_clean":"2024-05-08T18:07:54.002969+0000","last_became_active":"2024-05-08T18:04:45.810827+0000","last_became_peered":"2024-05-08T18:04:45.810827+0000","last_unstale":"2024-05-08T18:07:54.002969+0000","last_undegraded":"2024-05-08T18:07:54.002969+0000","last_fullsized":"2024-05-08T18:07:54.002969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:26.952570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"29'3","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746208+0000","last_change":"2024-05-08T18:04:43.838115+0000","last_active":"2024-05-08T18:07:08.746208+0000","last_peered":"2024-05-08T18:07:08.746208+0000","last_clean":"2024-05-08T18:07:08.746208+0000","last_became_active":"2024-05-08T18:04:43.837706+0000","last_became_peered":"2024-05-08T18:04:43.837706+0000","last_unstale":"2024-05-08T18:07:08.746208+0000","last_undegraded":"2024-05-08T18:07:08.746208+0000","last_fullsized":"2024-05-08T18:07:08.746208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:26.918679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"29'8","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.738522+0000","last_change":"2024-05-08T18:04:45.822292+0000","last_active":"2024-05-08T18:06:09.738522+0000","last_peered":"2024-05-08T18:06:09.738522+0000","last_clean":"2024-05-08T18:06:09.738522+0000","last_became_active":"2024-05-08T18:04:45.822000+0000","last_became_peered":"2024-05-08T18:04:45.822000+0000","last_unstale":"2024-05-08T18:06:09.738522+0000","last_undegraded":"2024-05-08T18:06:09.738522+0000","last_fullsized":"2024-05-08T18:06:09.738522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:08.184602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866478+0000","last_change":"2024-05-08T18:04:43.841486+0000","last_active":"2024-05-08T18:05:30.866478+0000","last_peered":"2024-05-08T18:05:30.866478+0000","last_clean":"2024-05-08T18:05:30.866478+0000","last_became_active":"2024-05-08T18:04:43.841063+0000","last_became_peered":"2024-05-08T18:04:43.841063+0000","last_unstale":"2024-05-08T18:05:30.866478+0000","last_undegraded":"2024-05-08T18:05:30.866478+0000","last_fullsized":"2024-05-08T18:05:30.866478+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:05:33.776915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"29'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.116184+0000","last_change":"2024-05-08T18:04:45.824006+0000","last_active":"2024-05-08T18:07:54.116184+0000","last_peered":"2024-05-08T18:07:54.116184+0000","last_clean":"2024-05-08T18:07:54.116184+0000","last_became_active":"2024-05-08T18:04:45.823611+0000","last_became_peered":"2024-05-08T18:04:45.823611+0000","last_unstale":"2024-05-08T18:07:54.116184+0000","last_undegraded":"2024-05-08T18:07:54.116184+0000","last_fullsized":"2024-05-08T18:07:54.116184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:02.599918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860254+0000","last_change":"2024-05-08T18:04:43.835685+0000","last_active":"2024-05-08T18:05:30.860254+0000","last_peered":"2024-05-08T18:05:30.860254+0000","last_clean":"2024-05-08T18:05:30.860254+0000","last_became_active":"2024-05-08T18:04:43.834456+0000","last_became_peered":"2024-05-08T18:04:43.834456+0000","last_unstale":"2024-05-08T18:05:30.860254+0000","last_undegraded":"2024-05-08T18:05:30.860254+0000","last_fullsized":"2024-05-08T18:05:30.860254+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:50.043665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399432+0000","last_change":"2024-05-08T18:04:45.808763+0000","last_active":"2024-05-08T18:06:53.399432+0000","last_peered":"2024-05-08T18:06:53.399432+0000","last_clean":"2024-05-08T18:06:53.399432+0000","last_became_active":"2024-05-08T18:04:45.808416+0000","last_became_peered":"2024-05-08T18:04:45.808416+0000","last_unstale":"2024-05-08T18:06:53.399432+0000","last_undegraded":"2024-05-08T18:06:53.399432+0000","last_fullsized":"2024-05-08T18:06:53.399432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:35.780903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'4","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858633+0000","last_change":"2024-05-08T18:04:43.845458+0000","last_active":"2024-05-08T18:05:30.858633+0000","last_peered":"2024-05-08T18:05:30.858633+0000","last_clean":"2024-05-08T18:05:30.858633+0000","last_became_active":"2024-05-08T18:04:43.844702+0000","last_became_peered":"2024-05-08T18:04:43.844702+0000","last_unstale":"2024-05-08T18:05:30.858633+0000","last_undegraded":"2024-05-08T18:05:30.858633+0000","last_fullsized":"2024-05-08T18:05:30.858633+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:58.229253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"29'35","reported_seq":99,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.016701+0000","last_change":"2024-05-08T18:04:53.849775+0000","last_active":"2024-05-08T18:07:54.016701+0000","last_peered":"2024-05-08T18:07:54.016701+0000","last_clean":"2024-05-08T18:07:54.016701+0000","last_became_active":"2024-05-08T18:04:53.849475+0000","last_became_peered":"2024-05-08T18:04:53.849475+0000","last_unstale":"2024-05-08T18:07:54.016701+0000","last_undegraded":"2024-05-08T18:07:54.016701+0000","last_fullsized":"2024-05-08T18:07:54.016701+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:12.918578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"29'12","reported_seq":66,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995839+0000","last_change":"2024-05-08T18:04:45.824289+0000","last_active":"2024-05-08T18:07:53.995839+0000","last_peered":"2024-05-08T18:07:53.995839+0000","last_clean":"2024-05-08T18:07:53.995839+0000","last_became_active":"2024-05-08T18:04:45.824005+0000","last_became_peered":"2024-05-08T18:04:45.824005+0000","last_unstale":"2024-05-08T18:07:53.995839+0000","last_undegraded":"2024-05-08T18:07:53.995839+0000","last_fullsized":"2024-05-08T18:07:53.995839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:45.743430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859173+0000","last_change":"2024-05-08T18:04:43.834249+0000","last_active":"2024-05-08T18:05:30.859173+0000","last_peered":"2024-05-08T18:05:30.859173+0000","last_clean":"2024-05-08T18:05:30.859173+0000","last_became_active":"2024-05-08T18:04:43.833822+0000","last_became_peered":"2024-05-08T18:04:43.833822+0000","last_unstale":"2024-05-08T18:05:30.859173+0000","last_undegraded":"2024-05-08T18:05:30.859173+0000","last_fullsized":"2024-05-08T18:05:30.859173+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:00:37.867739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859597+0000","last_change":"2024-05-08T18:04:47.820083+0000","last_active":"2024-05-08T18:05:30.859597+0000","last_peered":"2024-05-08T18:05:30.859597+0000","last_clean":"2024-05-08T18:05:30.859597+0000","last_became_active":"2024-05-08T18:04:47.819682+0000","last_became_peered":"2024-05-08T18:04:47.819682+0000","last_unstale":"2024-05-08T18:05:30.859597+0000","last_undegraded":"2024-05-08T18:05:30.859597+0000","last_fullsized":"2024-05-08T18:05:30.859597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:00:55.772655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"29'31","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.748532+0000","last_change":"2024-05-08T18:04:49.833877+0000","last_active":"2024-05-08T18:07:18.748532+0000","last_peered":"2024-05-08T18:07:18.748532+0000","last_clean":"2024-05-08T18:07:18.748532+0000","last_became_active":"2024-05-08T18:04:49.832148+0000","last_became_peered":"2024-05-08T18:04:49.832148+0000","last_unstale":"2024-05-08T18:07:18.748532+0000","last_undegraded":"2024-05-08T18:07:18.748532+0000","last_fullsized":"2024-05-08T18:07:18.748532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:20:21.388280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5807,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":31,"num_read_kb":25,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866106+0000","last_change":"2024-05-08T18:04:51.845281+0000","last_active":"2024-05-08T18:05:30.866106+0000","last_peered":"2024-05-08T18:05:30.866106+0000","last_clean":"2024-05-08T18:05:30.866106+0000","last_became_active":"2024-05-08T18:04:51.844901+0000","last_became_peered":"2024-05-08T18:04:51.844901+0000","last_unstale":"2024-05-08T18:05:30.866106+0000","last_undegraded":"2024-05-08T18:05:30.866106+0000","last_fullsized":"2024-05-08T18:05:30.866106+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:46.002263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"29'11","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.552238+0000","last_change":"2024-05-08T18:04:53.849933+0000","last_active":"2024-05-08T18:06:09.552238+0000","last_peered":"2024-05-08T18:06:09.552238+0000","last_clean":"2024-05-08T18:06:09.552238+0000","last_became_active":"2024-05-08T18:04:53.849579+0000","last_became_peered":"2024-05-08T18:04:53.849579+0000","last_unstale":"2024-05-08T18:06:09.552238+0000","last_undegraded":"2024-05-08T18:06:09.552238+0000","last_fullsized":"2024-05-08T18:06:09.552238+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:42:45.987685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"29'6","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.064004+0000","last_change":"2024-05-08T18:04:45.826976+0000","last_active":"2024-05-08T18:07:54.064004+0000","last_peered":"2024-05-08T18:07:54.064004+0000","last_clean":"2024-05-08T18:07:54.064004+0000","last_became_active":"2024-05-08T18:04:45.825162+0000","last_became_peered":"2024-05-08T18:04:45.825162+0000","last_unstale":"2024-05-08T18:07:54.064004+0000","last_undegraded":"2024-05-08T18:07:54.064004+0000","last_fullsized":"2024-05-08T18:07:54.064004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:36.088260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862438+0000","last_change":"2024-05-08T18:04:43.827844+0000","last_active":"2024-05-08T18:05:30.862438+0000","last_peered":"2024-05-08T18:05:30.862438+0000","last_clean":"2024-05-08T18:05:30.862438+0000","last_became_active":"2024-05-08T18:04:43.827374+0000","last_became_peered":"2024-05-08T18:04:43.827374+0000","last_unstale":"2024-05-08T18:05:30.862438+0000","last_undegraded":"2024-05-08T18:05:30.862438+0000","last_fullsized":"2024-05-08T18:05:30.862438+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:26.918679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:33.732471+0000","last_change":"2024-05-08T18:05:30.868535+0000","last_active":"2024-05-08T18:05:33.732471+0000","last_peered":"2024-05-08T18:05:33.732471+0000","last_clean":"2024-05-08T18:05:33.732471+0000","last_became_active":"2024-05-08T18:05:30.868191+0000","last_became_peered":"2024-05-08T18:05:30.868191+0000","last_unstale":"2024-05-08T18:05:33.732471+0000","last_undegraded":"2024-05-08T18:05:33.732471+0000","last_fullsized":"2024-05-08T18:05:33.732471+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:19.468033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"27'6","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862185+0000","last_change":"2024-05-08T18:04:49.826617+0000","last_active":"2024-05-08T18:05:30.862185+0000","last_peered":"2024-05-08T18:05:30.862185+0000","last_clean":"2024-05-08T18:05:30.862185+0000","last_became_active":"2024-05-08T18:04:49.825265+0000","last_became_peered":"2024-05-08T18:04:49.825265+0000","last_unstale":"2024-05-08T18:05:30.862185+0000","last_undegraded":"2024-05-08T18:05:30.862185+0000","last_fullsized":"2024-05-08T18:05:30.862185+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:35.867914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862381+0000","last_change":"2024-05-08T18:04:51.834892+0000","last_active":"2024-05-08T18:05:30.862381+0000","last_peered":"2024-05-08T18:05:30.862381+0000","last_clean":"2024-05-08T18:05:30.862381+0000","last_became_active":"2024-05-08T18:04:51.833523+0000","last_became_peered":"2024-05-08T18:04:51.833523+0000","last_unstale":"2024-05-08T18:05:30.862381+0000","last_undegraded":"2024-05-08T18:05:30.862381+0000","last_fullsized":"2024-05-08T18:05:30.862381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:01.176258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'2","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858040+0000","last_change":"2024-05-08T18:04:53.840760+0000","last_active":"2024-05-08T18:05:30.858040+0000","last_peered":"2024-05-08T18:05:30.858040+0000","last_clean":"2024-05-08T18:05:30.858040+0000","last_became_active":"2024-05-08T18:04:53.840395+0000","last_became_peered":"2024-05-08T18:04:53.840395+0000","last_unstale":"2024-05-08T18:05:30.858040+0000","last_undegraded":"2024-05-08T18:05:30.858040+0000","last_fullsized":"2024-05-08T18:05:30.858040+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:54.850231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:33.732213+0000","last_change":"2024-05-08T18:05:30.876470+0000","last_active":"2024-05-08T18:05:33.732213+0000","last_peered":"2024-05-08T18:05:33.732213+0000","last_clean":"2024-05-08T18:05:33.732213+0000","last_became_active":"2024-05-08T18:05:30.869162+0000","last_became_peered":"2024-05-08T18:05:30.869162+0000","last_unstale":"2024-05-08T18:05:33.732213+0000","last_undegraded":"2024-05-08T18:05:33.732213+0000","last_fullsized":"2024-05-08T18:05:33.732213+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:04.889530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":45,"num_write":24,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857600+0000","last_change":"2024-05-08T18:04:43.844847+0000","last_active":"2024-05-08T18:05:30.857600+0000","last_peered":"2024-05-08T18:05:30.857600+0000","last_clean":"2024-05-08T18:05:30.857600+0000","last_became_active":"2024-05-08T18:04:43.838666+0000","last_became_peered":"2024-05-08T18:04:43.838666+0000","last_unstale":"2024-05-08T18:05:30.857600+0000","last_undegraded":"2024-05-08T18:05:30.857600+0000","last_fullsized":"2024-05-08T18:05:30.857600+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:32.496387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861628+0000","last_change":"2024-05-08T18:04:47.819743+0000","last_active":"2024-05-08T18:05:30.861628+0000","last_peered":"2024-05-08T18:05:30.861628+0000","last_clean":"2024-05-08T18:05:30.861628+0000","last_became_active":"2024-05-08T18:04:47.819169+0000","last_became_peered":"2024-05-08T18:04:47.819169+0000","last_unstale":"2024-05-08T18:05:30.861628+0000","last_undegraded":"2024-05-08T18:05:30.861628+0000","last_fullsized":"2024-05-08T18:05:30.861628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:15:57.213130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"29'12","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.746941+0000","last_change":"2024-05-08T18:04:49.831839+0000","last_active":"2024-05-08T18:07:13.746941+0000","last_peered":"2024-05-08T18:07:13.746941+0000","last_clean":"2024-05-08T18:07:13.746941+0000","last_became_active":"2024-05-08T18:04:49.831500+0000","last_became_peered":"2024-05-08T18:04:49.831500+0000","last_unstale":"2024-05-08T18:07:13.746941+0000","last_undegraded":"2024-05-08T18:07:13.746941+0000","last_fullsized":"2024-05-08T18:07:13.746941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:19:01.714463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"29'282","reported_seq":381,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903208+0000","last_change":"2024-05-08T18:04:51.849859+0000","last_active":"2024-05-08T18:07:52.903208+0000","last_peered":"2024-05-08T18:07:52.903208+0000","last_clean":"2024-05-08T18:07:52.903208+0000","last_became_active":"2024-05-08T18:04:51.849301+0000","last_became_peered":"2024-05-08T18:04:51.849301+0000","last_unstale":"2024-05-08T18:07:52.903208+0000","last_undegraded":"2024-05-08T18:07:52.903208+0000","last_fullsized":"2024-05-08T18:07:52.903208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:02:26.756919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862433+0000","last_change":"2024-05-08T18:04:53.840922+0000","last_active":"2024-05-08T18:05:30.862433+0000","last_peered":"2024-05-08T18:05:30.862433+0000","last_clean":"2024-05-08T18:05:30.862433+0000","last_became_active":"2024-05-08T18:04:53.840559+0000","last_became_peered":"2024-05-08T18:04:53.840559+0000","last_unstale":"2024-05-08T18:05:30.862433+0000","last_undegraded":"2024-05-08T18:05:30.862433+0000","last_fullsized":"2024-05-08T18:05:30.862433+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:24.814022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"29'6","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.035260+0000","last_change":"2024-05-08T18:04:45.812045+0000","last_active":"2024-05-08T18:07:54.035260+0000","last_peered":"2024-05-08T18:07:54.035260+0000","last_clean":"2024-05-08T18:07:54.035260+0000","last_became_active":"2024-05-08T18:04:45.811570+0000","last_became_peered":"2024-05-08T18:04:45.811570+0000","last_unstale":"2024-05-08T18:07:54.035260+0000","last_undegraded":"2024-05-08T18:07:54.035260+0000","last_fullsized":"2024-05-08T18:07:54.035260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:22:19.585204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.987908+0000","last_change":"2024-05-08T18:04:43.838356+0000","last_active":"2024-05-08T18:07:53.987908+0000","last_peered":"2024-05-08T18:07:53.987908+0000","last_clean":"2024-05-08T18:07:53.987908+0000","last_became_active":"2024-05-08T18:04:43.836410+0000","last_became_peered":"2024-05-08T18:04:43.836410+0000","last_unstale":"2024-05-08T18:07:53.987908+0000","last_undegraded":"2024-05-08T18:07:53.987908+0000","last_fullsized":"2024-05-08T18:07:53.987908+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:59:26.742382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860150+0000","last_change":"2024-05-08T18:04:47.819091+0000","last_active":"2024-05-08T18:05:30.860150+0000","last_peered":"2024-05-08T18:05:30.860150+0000","last_clean":"2024-05-08T18:05:30.860150+0000","last_became_active":"2024-05-08T18:04:47.818797+0000","last_became_peered":"2024-05-08T18:04:47.818797+0000","last_unstale":"2024-05-08T18:05:30.860150+0000","last_undegraded":"2024-05-08T18:05:30.860150+0000","last_fullsized":"2024-05-08T18:05:30.860150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:44.738079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"29'31","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.948644+0000","last_change":"2024-05-08T18:04:49.831750+0000","last_active":"2024-05-08T18:06:51.948644+0000","last_peered":"2024-05-08T18:06:51.948644+0000","last_clean":"2024-05-08T18:06:51.948644+0000","last_became_active":"2024-05-08T18:04:49.831379+0000","last_became_peered":"2024-05-08T18:04:49.831379+0000","last_unstale":"2024-05-08T18:06:51.948644+0000","last_undegraded":"2024-05-08T18:06:51.948644+0000","last_fullsized":"2024-05-08T18:06:51.948644+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:15:59.220647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"29'282","reported_seq":441,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.067044+0000","last_change":"2024-05-08T18:04:51.827294+0000","last_active":"2024-05-08T18:07:54.067044+0000","last_peered":"2024-05-08T18:07:54.067044+0000","last_clean":"2024-05-08T18:07:54.067044+0000","last_became_active":"2024-05-08T18:04:51.826926+0000","last_became_peered":"2024-05-08T18:04:51.826926+0000","last_unstale":"2024-05-08T18:07:54.067044+0000","last_undegraded":"2024-05-08T18:07:54.067044+0000","last_fullsized":"2024-05-08T18:07:54.067044+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:00:59.786865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'6","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858148+0000","last_change":"2024-05-08T18:04:45.827616+0000","last_active":"2024-05-08T18:05:30.858148+0000","last_peered":"2024-05-08T18:05:30.858148+0000","last_clean":"2024-05-08T18:05:30.858148+0000","last_became_active":"2024-05-08T18:04:45.827351+0000","last_became_peered":"2024-05-08T18:04:45.827351+0000","last_unstale":"2024-05-08T18:05:30.858148+0000","last_undegraded":"2024-05-08T18:05:30.858148+0000","last_fullsized":"2024-05-08T18:05:30.858148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:31.853297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866835+0000","last_change":"2024-05-08T18:04:43.839205+0000","last_active":"2024-05-08T18:05:30.866835+0000","last_peered":"2024-05-08T18:05:30.866835+0000","last_clean":"2024-05-08T18:05:30.866835+0000","last_became_active":"2024-05-08T18:04:43.838840+0000","last_became_peered":"2024-05-08T18:04:43.838840+0000","last_unstale":"2024-05-08T18:05:30.866835+0000","last_undegraded":"2024-05-08T18:05:30.866835+0000","last_fullsized":"2024-05-08T18:05:30.866835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:50.043665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,0],"acting":[2,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861049+0000","last_change":"2024-05-08T18:04:47.825571+0000","last_active":"2024-05-08T18:05:30.861049+0000","last_peered":"2024-05-08T18:05:30.861049+0000","last_clean":"2024-05-08T18:05:30.861049+0000","last_became_active":"2024-05-08T18:04:47.825215+0000","last_became_peered":"2024-05-08T18:04:47.825215+0000","last_unstale":"2024-05-08T18:05:30.861049+0000","last_undegraded":"2024-05-08T18:05:30.861049+0000","last_fullsized":"2024-05-08T18:05:30.861049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:47.750530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"29'40","reported_seq":87,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.957612+0000","last_change":"2024-05-08T18:04:49.828257+0000","last_active":"2024-05-08T18:06:51.957612+0000","last_peered":"2024-05-08T18:06:51.957612+0000","last_clean":"2024-05-08T18:06:51.957612+0000","last_became_active":"2024-05-08T18:04:49.827954+0000","last_became_peered":"2024-05-08T18:04:49.827954+0000","last_unstale":"2024-05-08T18:06:51.957612+0000","last_undegraded":"2024-05-08T18:06:51.957612+0000","last_fullsized":"2024-05-08T18:06:51.957612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:43.565272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859378+0000","last_change":"2024-05-08T18:04:51.831489+0000","last_active":"2024-05-08T18:05:30.859378+0000","last_peered":"2024-05-08T18:05:30.859378+0000","last_clean":"2024-05-08T18:05:30.859378+0000","last_became_active":"2024-05-08T18:04:51.831149+0000","last_became_peered":"2024-05-08T18:04:51.831149+0000","last_unstale":"2024-05-08T18:05:30.859378+0000","last_undegraded":"2024-05-08T18:05:30.859378+0000","last_fullsized":"2024-05-08T18:05:30.859378+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:59.859356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"29'5","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.735436+0000","last_change":"2024-05-08T18:04:53.840960+0000","last_active":"2024-05-08T18:06:09.735436+0000","last_peered":"2024-05-08T18:06:09.735436+0000","last_clean":"2024-05-08T18:06:09.735436+0000","last_became_active":"2024-05-08T18:04:53.840652+0000","last_became_peered":"2024-05-08T18:04:53.840652+0000","last_unstale":"2024-05-08T18:06:09.735436+0000","last_undegraded":"2024-05-08T18:06:09.735436+0000","last_fullsized":"2024-05-08T18:06:09.735436+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:02:50.273794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"27'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861806+0000","last_change":"2024-05-08T18:04:45.804412+0000","last_active":"2024-05-08T18:05:30.861806+0000","last_peered":"2024-05-08T18:05:30.861806+0000","last_clean":"2024-05-08T18:05:30.861806+0000","last_became_active":"2024-05-08T18:04:45.804124+0000","last_became_peered":"2024-05-08T18:04:45.804124+0000","last_unstale":"2024-05-08T18:05:30.861806+0000","last_undegraded":"2024-05-08T18:05:30.861806+0000","last_fullsized":"2024-05-08T18:05:30.861806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:08.184602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867328+0000","last_change":"2024-05-08T18:04:43.843177+0000","last_active":"2024-05-08T18:05:30.867328+0000","last_peered":"2024-05-08T18:05:30.867328+0000","last_clean":"2024-05-08T18:05:30.867328+0000","last_became_active":"2024-05-08T18:04:43.842126+0000","last_became_peered":"2024-05-08T18:04:43.842126+0000","last_unstale":"2024-05-08T18:05:30.867328+0000","last_undegraded":"2024-05-08T18:05:30.867328+0000","last_fullsized":"2024-05-08T18:05:30.867328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:41:01.395720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858650+0000","last_change":"2024-05-08T18:04:47.823007+0000","last_active":"2024-05-08T18:05:30.858650+0000","last_peered":"2024-05-08T18:05:30.858650+0000","last_clean":"2024-05-08T18:05:30.858650+0000","last_became_active":"2024-05-08T18:04:47.822688+0000","last_became_peered":"2024-05-08T18:04:47.822688+0000","last_unstale":"2024-05-08T18:05:30.858650+0000","last_undegraded":"2024-05-08T18:05:30.858650+0000","last_fullsized":"2024-05-08T18:05:30.858650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:32.662502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"29'30","reported_seq":72,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.938361+0000","last_change":"2024-05-08T18:04:49.832010+0000","last_active":"2024-05-08T18:06:51.938361+0000","last_peered":"2024-05-08T18:06:51.938361+0000","last_clean":"2024-05-08T18:06:51.938361+0000","last_became_active":"2024-05-08T18:04:49.829892+0000","last_became_peered":"2024-05-08T18:04:49.829892+0000","last_unstale":"2024-05-08T18:06:51.938361+0000","last_undegraded":"2024-05-08T18:06:51.938361+0000","last_fullsized":"2024-05-08T18:06:51.938361+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:13:16.535751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858848+0000","last_change":"2024-05-08T18:04:51.833183+0000","last_active":"2024-05-08T18:05:30.858848+0000","last_peered":"2024-05-08T18:05:30.858848+0000","last_clean":"2024-05-08T18:05:30.858848+0000","last_became_active":"2024-05-08T18:04:51.832810+0000","last_became_peered":"2024-05-08T18:04:51.832810+0000","last_unstale":"2024-05-08T18:05:30.858848+0000","last_undegraded":"2024-05-08T18:05:30.858848+0000","last_fullsized":"2024-05-08T18:05:30.858848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:02.130044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"29'3","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:03.972536+0000","last_change":"2024-05-08T18:04:53.841201+0000","last_active":"2024-05-08T18:06:03.972536+0000","last_peered":"2024-05-08T18:06:03.972536+0000","last_clean":"2024-05-08T18:06:03.972536+0000","last_became_active":"2024-05-08T18:04:53.840894+0000","last_became_peered":"2024-05-08T18:04:53.840894+0000","last_unstale":"2024-05-08T18:06:03.972536+0000","last_undegraded":"2024-05-08T18:06:03.972536+0000","last_fullsized":"2024-05-08T18:06:03.972536+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:22:10.520507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"29'9","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.867093+0000","last_change":"2024-05-08T18:04:45.817096+0000","last_active":"2024-05-08T18:07:53.867093+0000","last_peered":"2024-05-08T18:07:53.867093+0000","last_clean":"2024-05-08T18:07:53.867093+0000","last_became_active":"2024-05-08T18:04:45.816817+0000","last_became_peered":"2024-05-08T18:04:45.816817+0000","last_unstale":"2024-05-08T18:07:53.867093+0000","last_undegraded":"2024-05-08T18:07:53.867093+0000","last_fullsized":"2024-05-08T18:07:53.867093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:48.747375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"29'3","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.984852+0000","last_change":"2024-05-08T18:04:43.844652+0000","last_active":"2024-05-08T18:07:53.984852+0000","last_peered":"2024-05-08T18:07:53.984852+0000","last_clean":"2024-05-08T18:07:53.984852+0000","last_became_active":"2024-05-08T18:04:43.838781+0000","last_became_peered":"2024-05-08T18:04:43.838781+0000","last_unstale":"2024-05-08T18:07:53.984852+0000","last_undegraded":"2024-05-08T18:07:53.984852+0000","last_fullsized":"2024-05-08T18:07:53.984852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:22:17.577945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":668544,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858935+0000","last_change":"2024-05-08T18:04:47.819192+0000","last_active":"2024-05-08T18:05:30.858935+0000","last_peered":"2024-05-08T18:05:30.858935+0000","last_clean":"2024-05-08T18:05:30.858935+0000","last_became_active":"2024-05-08T18:04:47.818871+0000","last_became_peered":"2024-05-08T18:04:47.818871+0000","last_unstale":"2024-05-08T18:05:30.858935+0000","last_undegraded":"2024-05-08T18:05:30.858935+0000","last_fullsized":"2024-05-08T18:05:30.858935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:20.855010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"29'36","reported_seq":81,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.959548+0000","last_change":"2024-05-08T18:04:49.829583+0000","last_active":"2024-05-08T18:06:51.959548+0000","last_peered":"2024-05-08T18:06:51.959548+0000","last_clean":"2024-05-08T18:06:51.959548+0000","last_became_active":"2024-05-08T18:04:49.829270+0000","last_became_peered":"2024-05-08T18:04:49.829270+0000","last_unstale":"2024-05-08T18:06:51.959548+0000","last_undegraded":"2024-05-08T18:06:51.959548+0000","last_fullsized":"2024-05-08T18:06:51.959548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:11:17.784122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"29'282","reported_seq":348,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.181903+0000","last_change":"2024-05-08T18:04:51.845064+0000","last_active":"2024-05-08T18:07:54.181903+0000","last_peered":"2024-05-08T18:07:54.181903+0000","last_clean":"2024-05-08T18:07:54.181903+0000","last_became_active":"2024-05-08T18:04:51.844529+0000","last_became_peered":"2024-05-08T18:04:51.844529+0000","last_unstale":"2024-05-08T18:07:54.181903+0000","last_undegraded":"2024-05-08T18:07:54.181903+0000","last_fullsized":"2024-05-08T18:07:54.181903+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:46.002263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"29'2","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.862719+0000","last_change":"2024-05-08T18:04:53.841062+0000","last_active":"2024-05-08T18:07:53.862719+0000","last_peered":"2024-05-08T18:07:53.862719+0000","last_clean":"2024-05-08T18:07:53.862719+0000","last_became_active":"2024-05-08T18:04:53.840767+0000","last_became_peered":"2024-05-08T18:04:53.840767+0000","last_unstale":"2024-05-08T18:07:53.862719+0000","last_undegraded":"2024-05-08T18:07:53.862719+0000","last_fullsized":"2024-05-08T18:07:53.862719+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:54:38.757755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"29'6","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995942+0000","last_change":"2024-05-08T18:04:45.814276+0000","last_active":"2024-05-08T18:07:53.995942+0000","last_peered":"2024-05-08T18:07:53.995942+0000","last_clean":"2024-05-08T18:07:53.995942+0000","last_became_active":"2024-05-08T18:04:45.813735+0000","last_became_peered":"2024-05-08T18:04:45.813735+0000","last_unstale":"2024-05-08T18:07:53.995942+0000","last_undegraded":"2024-05-08T18:07:53.995942+0000","last_fullsized":"2024-05-08T18:07:53.995942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:18.463624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"27'4","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859087+0000","last_change":"2024-05-08T18:04:43.848140+0000","last_active":"2024-05-08T18:05:30.859087+0000","last_peered":"2024-05-08T18:05:30.859087+0000","last_clean":"2024-05-08T18:05:30.859087+0000","last_became_active":"2024-05-08T18:04:43.839047+0000","last_became_peered":"2024-05-08T18:04:43.839047+0000","last_unstale":"2024-05-08T18:05:30.859087+0000","last_undegraded":"2024-05-08T18:05:30.859087+0000","last_fullsized":"2024-05-08T18:05:30.859087+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:24.945310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858315+0000","last_change":"2024-05-08T18:04:47.819319+0000","last_active":"2024-05-08T18:05:30.858315+0000","last_peered":"2024-05-08T18:05:30.858315+0000","last_clean":"2024-05-08T18:05:30.858315+0000","last_became_active":"2024-05-08T18:04:47.818972+0000","last_became_peered":"2024-05-08T18:04:47.818972+0000","last_unstale":"2024-05-08T18:05:30.858315+0000","last_undegraded":"2024-05-08T18:05:30.858315+0000","last_fullsized":"2024-05-08T18:05:30.858315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:09:51.843752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"29'54","reported_seq":110,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.961509+0000","last_change":"2024-05-08T18:04:49.832104+0000","last_active":"2024-05-08T18:06:51.961509+0000","last_peered":"2024-05-08T18:06:51.961509+0000","last_clean":"2024-05-08T18:06:51.961509+0000","last_became_active":"2024-05-08T18:04:49.830424+0000","last_became_peered":"2024-05-08T18:04:49.830424+0000","last_unstale":"2024-05-08T18:06:51.961509+0000","last_undegraded":"2024-05-08T18:06:51.961509+0000","last_fullsized":"2024-05-08T18:06:51.961509+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:50.646336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858392+0000","last_change":"2024-05-08T18:04:51.830631+0000","last_active":"2024-05-08T18:05:30.858392+0000","last_peered":"2024-05-08T18:05:30.858392+0000","last_clean":"2024-05-08T18:05:30.858392+0000","last_became_active":"2024-05-08T18:04:51.830383+0000","last_became_peered":"2024-05-08T18:04:51.830383+0000","last_unstale":"2024-05-08T18:05:30.858392+0000","last_undegraded":"2024-05-08T18:05:30.858392+0000","last_fullsized":"2024-05-08T18:05:30.858392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:01.176258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"29'20","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.158011+0000","last_change":"2024-05-08T18:04:53.848105+0000","last_active":"2024-05-08T18:07:54.158011+0000","last_peered":"2024-05-08T18:07:54.158011+0000","last_clean":"2024-05-08T18:07:54.158011+0000","last_became_active":"2024-05-08T18:04:53.847842+0000","last_became_peered":"2024-05-08T18:04:53.847842+0000","last_unstale":"2024-05-08T18:07:54.158011+0000","last_undegraded":"2024-05-08T18:07:54.158011+0000","last_fullsized":"2024-05-08T18:07:54.158011+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:03:57.205665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":48,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"29'6","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.064006+0000","last_change":"2024-05-08T18:04:45.821729+0000","last_active":"2024-05-08T18:07:54.064006+0000","last_peered":"2024-05-08T18:07:54.064006+0000","last_clean":"2024-05-08T18:07:54.064006+0000","last_became_active":"2024-05-08T18:04:45.821412+0000","last_became_peered":"2024-05-08T18:04:45.821412+0000","last_unstale":"2024-05-08T18:07:54.064006+0000","last_undegraded":"2024-05-08T18:07:54.064006+0000","last_fullsized":"2024-05-08T18:07:54.064006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:26.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859201+0000","last_change":"2024-05-08T18:04:43.835922+0000","last_active":"2024-05-08T18:05:30.859201+0000","last_peered":"2024-05-08T18:05:30.859201+0000","last_clean":"2024-05-08T18:05:30.859201+0000","last_became_active":"2024-05-08T18:04:43.834723+0000","last_became_peered":"2024-05-08T18:04:43.834723+0000","last_unstale":"2024-05-08T18:05:30.859201+0000","last_undegraded":"2024-05-08T18:05:30.859201+0000","last_fullsized":"2024-05-08T18:05:30.859201+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:19.700373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864185+0000","last_change":"2024-05-08T18:04:47.823171+0000","last_active":"2024-05-08T18:05:30.864185+0000","last_peered":"2024-05-08T18:05:30.864185+0000","last_clean":"2024-05-08T18:05:30.864185+0000","last_became_active":"2024-05-08T18:04:47.822865+0000","last_became_peered":"2024-05-08T18:04:47.822865+0000","last_unstale":"2024-05-08T18:05:30.864185+0000","last_undegraded":"2024-05-08T18:05:30.864185+0000","last_fullsized":"2024-05-08T18:05:30.864185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:22:57.464984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"29'33","reported_seq":76,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.942615+0000","last_change":"2024-05-08T18:04:49.824918+0000","last_active":"2024-05-08T18:06:51.942615+0000","last_peered":"2024-05-08T18:06:51.942615+0000","last_clean":"2024-05-08T18:06:51.942615+0000","last_became_active":"2024-05-08T18:04:49.824610+0000","last_became_peered":"2024-05-08T18:04:49.824610+0000","last_unstale":"2024-05-08T18:06:51.942615+0000","last_undegraded":"2024-05-08T18:06:51.942615+0000","last_fullsized":"2024-05-08T18:06:51.942615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:56:27.132432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864191+0000","last_change":"2024-05-08T18:04:51.849355+0000","last_active":"2024-05-08T18:05:30.864191+0000","last_peered":"2024-05-08T18:05:30.864191+0000","last_clean":"2024-05-08T18:05:30.864191+0000","last_became_active":"2024-05-08T18:04:51.848793+0000","last_became_peered":"2024-05-08T18:04:51.848793+0000","last_unstale":"2024-05-08T18:05:30.864191+0000","last_undegraded":"2024-05-08T18:05:30.864191+0000","last_fullsized":"2024-05-08T18:05:30.864191+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:40:15.186526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"29'5","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744070+0000","last_change":"2024-05-08T18:04:53.849242+0000","last_active":"2024-05-08T18:06:53.744070+0000","last_peered":"2024-05-08T18:06:53.744070+0000","last_clean":"2024-05-08T18:06:53.744070+0000","last_became_active":"2024-05-08T18:04:53.848945+0000","last_became_peered":"2024-05-08T18:04:53.848945+0000","last_unstale":"2024-05-08T18:06:53.744070+0000","last_undegraded":"2024-05-08T18:06:53.744070+0000","last_fullsized":"2024-05-08T18:06:53.744070+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:00.234436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"29'10","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.772802+0000","last_change":"2024-05-08T18:04:45.829691+0000","last_active":"2024-05-08T18:07:53.772802+0000","last_peered":"2024-05-08T18:07:53.772802+0000","last_clean":"2024-05-08T18:07:53.772802+0000","last_became_active":"2024-05-08T18:04:45.829120+0000","last_became_peered":"2024-05-08T18:04:45.829120+0000","last_unstale":"2024-05-08T18:07:53.772802+0000","last_undegraded":"2024-05-08T18:07:53.772802+0000","last_fullsized":"2024-05-08T18:07:53.772802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:29.119368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":57,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859093+0000","last_change":"2024-05-08T18:04:43.835329+0000","last_active":"2024-05-08T18:05:30.859093+0000","last_peered":"2024-05-08T18:05:30.859093+0000","last_clean":"2024-05-08T18:05:30.859093+0000","last_became_active":"2024-05-08T18:04:43.834536+0000","last_became_peered":"2024-05-08T18:04:43.834536+0000","last_unstale":"2024-05-08T18:05:30.859093+0000","last_undegraded":"2024-05-08T18:05:30.859093+0000","last_fullsized":"2024-05-08T18:05:30.859093+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:56.567490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"29'4","reported_seq":226,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.038721+0000","last_change":"2024-05-08T18:04:47.819789+0000","last_active":"2024-05-08T18:06:53.038721+0000","last_peered":"2024-05-08T18:06:53.038721+0000","last_clean":"2024-05-08T18:06:53.038721+0000","last_became_active":"2024-05-08T18:04:47.819085+0000","last_became_peered":"2024-05-08T18:04:47.819085+0000","last_unstale":"2024-05-08T18:06:53.038721+0000","last_undegraded":"2024-05-08T18:06:53.038721+0000","last_fullsized":"2024-05-08T18:06:53.038721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:24.812721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":298,"num_read_kb":298,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"29'36","reported_seq":81,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.967858+0000","last_change":"2024-05-08T18:04:49.825997+0000","last_active":"2024-05-08T18:06:51.967858+0000","last_peered":"2024-05-08T18:06:51.967858+0000","last_clean":"2024-05-08T18:06:51.967858+0000","last_became_active":"2024-05-08T18:04:49.825613+0000","last_became_peered":"2024-05-08T18:04:49.825613+0000","last_unstale":"2024-05-08T18:06:51.967858+0000","last_undegraded":"2024-05-08T18:06:51.967858+0000","last_fullsized":"2024-05-08T18:06:51.967858+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:49.758047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"29'282","reported_seq":431,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.162304+0000","last_change":"2024-05-08T18:04:51.831317+0000","last_active":"2024-05-08T18:07:54.162304+0000","last_peered":"2024-05-08T18:07:54.162304+0000","last_clean":"2024-05-08T18:07:54.162304+0000","last_became_active":"2024-05-08T18:04:51.830924+0000","last_became_peered":"2024-05-08T18:04:51.830924+0000","last_unstale":"2024-05-08T18:07:54.162304+0000","last_undegraded":"2024-05-08T18:07:54.162304+0000","last_fullsized":"2024-05-08T18:07:54.162304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:23.880081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"29'14","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.739659+0000","last_change":"2024-05-08T18:04:53.841349+0000","last_active":"2024-05-08T18:06:58.739659+0000","last_peered":"2024-05-08T18:06:58.739659+0000","last_clean":"2024-05-08T18:06:58.739659+0000","last_became_active":"2024-05-08T18:04:53.840708+0000","last_became_peered":"2024-05-08T18:04:53.840708+0000","last_unstale":"2024-05-08T18:06:58.739659+0000","last_undegraded":"2024-05-08T18:06:58.739659+0000","last_fullsized":"2024-05-08T18:06:58.739659+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:57:33.051338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063870+0000","last_change":"2024-05-08T18:04:45.812281+0000","last_active":"2024-05-08T18:07:54.063870+0000","last_peered":"2024-05-08T18:07:54.063870+0000","last_clean":"2024-05-08T18:07:54.063870+0000","last_became_active":"2024-05-08T18:04:45.811926+0000","last_became_peered":"2024-05-08T18:04:45.811926+0000","last_unstale":"2024-05-08T18:07:54.063870+0000","last_undegraded":"2024-05-08T18:07:54.063870+0000","last_fullsized":"2024-05-08T18:07:54.063870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:26.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"29'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.728795+0000","last_change":"2024-05-08T18:04:43.844418+0000","last_active":"2024-05-08T18:06:09.728795+0000","last_peered":"2024-05-08T18:06:09.728795+0000","last_clean":"2024-05-08T18:06:09.728795+0000","last_became_active":"2024-05-08T18:04:43.843934+0000","last_became_peered":"2024-05-08T18:04:43.843934+0000","last_unstale":"2024-05-08T18:06:09.728795+0000","last_undegraded":"2024-05-08T18:06:09.728795+0000","last_fullsized":"2024-05-08T18:06:09.728795+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:30.825989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"29'3","reported_seq":413,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.049002+0000","last_change":"2024-05-08T18:04:47.818222+0000","last_active":"2024-05-08T18:06:53.049002+0000","last_peered":"2024-05-08T18:06:53.049002+0000","last_clean":"2024-05-08T18:06:53.049002+0000","last_became_active":"2024-05-08T18:04:47.817851+0000","last_became_peered":"2024-05-08T18:04:47.817851+0000","last_unstale":"2024-05-08T18:06:53.049002+0000","last_undegraded":"2024-05-08T18:06:53.049002+0000","last_fullsized":"2024-05-08T18:06:53.049002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:45:55.932454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":381,"num_read_kb":381,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"27'10","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863960+0000","last_change":"2024-05-08T18:04:49.832191+0000","last_active":"2024-05-08T18:05:30.863960+0000","last_peered":"2024-05-08T18:05:30.863960+0000","last_clean":"2024-05-08T18:05:30.863960+0000","last_became_active":"2024-05-08T18:04:49.830714+0000","last_became_peered":"2024-05-08T18:04:49.830714+0000","last_unstale":"2024-05-08T18:05:30.863960+0000","last_undegraded":"2024-05-08T18:05:30.863960+0000","last_fullsized":"2024-05-08T18:05:30.863960+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:14:01.011620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"29'282","reported_seq":468,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.120234+0000","last_change":"2024-05-08T18:04:51.832589+0000","last_active":"2024-05-08T18:07:54.120234+0000","last_peered":"2024-05-08T18:07:54.120234+0000","last_clean":"2024-05-08T18:07:54.120234+0000","last_became_active":"2024-05-08T18:04:51.831508+0000","last_became_peered":"2024-05-08T18:04:51.831508+0000","last_unstale":"2024-05-08T18:07:54.120234+0000","last_undegraded":"2024-05-08T18:07:54.120234+0000","last_fullsized":"2024-05-08T18:07:54.120234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:15.559137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'4","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.179451+0000","last_change":"2024-05-08T18:04:53.840524+0000","last_active":"2024-05-08T18:06:53.179451+0000","last_peered":"2024-05-08T18:06:53.179451+0000","last_clean":"2024-05-08T18:06:53.179451+0000","last_became_active":"2024-05-08T18:04:53.840215+0000","last_became_peered":"2024-05-08T18:04:53.840215+0000","last_unstale":"2024-05-08T18:06:53.179451+0000","last_undegraded":"2024-05-08T18:06:53.179451+0000","last_fullsized":"2024-05-08T18:06:53.179451+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:39.550898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'2","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866626+0000","last_change":"2024-05-08T18:04:45.819074+0000","last_active":"2024-05-08T18:05:30.866626+0000","last_peered":"2024-05-08T18:05:30.866626+0000","last_clean":"2024-05-08T18:05:30.866626+0000","last_became_active":"2024-05-08T18:04:45.818533+0000","last_became_peered":"2024-05-08T18:04:45.818533+0000","last_unstale":"2024-05-08T18:05:30.866626+0000","last_undegraded":"2024-05-08T18:05:30.866626+0000","last_fullsized":"2024-05-08T18:05:30.866626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:34.419882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"29'1","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744105+0000","last_change":"2024-05-08T18:04:43.848496+0000","last_active":"2024-05-08T18:06:53.744105+0000","last_peered":"2024-05-08T18:06:53.744105+0000","last_clean":"2024-05-08T18:06:53.744105+0000","last_became_active":"2024-05-08T18:04:43.846125+0000","last_became_peered":"2024-05-08T18:04:43.846125+0000","last_unstale":"2024-05-08T18:06:53.744105+0000","last_undegraded":"2024-05-08T18:06:53.744105+0000","last_fullsized":"2024-05-08T18:06:53.744105+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:32.125079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861388+0000","last_change":"2024-05-08T18:04:47.822964+0000","last_active":"2024-05-08T18:05:30.861388+0000","last_peered":"2024-05-08T18:05:30.861388+0000","last_clean":"2024-05-08T18:05:30.861388+0000","last_became_active":"2024-05-08T18:04:47.822242+0000","last_became_peered":"2024-05-08T18:04:47.822242+0000","last_unstale":"2024-05-08T18:05:30.861388+0000","last_undegraded":"2024-05-08T18:05:30.861388+0000","last_fullsized":"2024-05-08T18:05:30.861388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:55.845147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"29'38","reported_seq":84,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.954827+0000","last_change":"2024-05-08T18:04:49.826706+0000","last_active":"2024-05-08T18:06:51.954827+0000","last_peered":"2024-05-08T18:06:51.954827+0000","last_clean":"2024-05-08T18:06:51.954827+0000","last_became_active":"2024-05-08T18:04:49.825314+0000","last_became_peered":"2024-05-08T18:04:49.825314+0000","last_unstale":"2024-05-08T18:06:51.954827+0000","last_undegraded":"2024-05-08T18:06:51.954827+0000","last_fullsized":"2024-05-08T18:06:51.954827+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:21:18.445670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862981+0000","last_change":"2024-05-08T18:04:51.833093+0000","last_active":"2024-05-08T18:05:30.862981+0000","last_peered":"2024-05-08T18:05:30.862981+0000","last_clean":"2024-05-08T18:05:30.862981+0000","last_became_active":"2024-05-08T18:04:51.832808+0000","last_became_peered":"2024-05-08T18:04:51.832808+0000","last_unstale":"2024-05-08T18:05:30.862981+0000","last_undegraded":"2024-05-08T18:05:30.862981+0000","last_fullsized":"2024-05-08T18:05:30.862981+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:54.955755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"29'8","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.096584+0000","last_change":"2024-05-08T18:04:53.837583+0000","last_active":"2024-05-08T18:07:54.096584+0000","last_peered":"2024-05-08T18:07:54.096584+0000","last_clean":"2024-05-08T18:07:54.096584+0000","last_became_active":"2024-05-08T18:04:53.837292+0000","last_became_peered":"2024-05-08T18:04:53.837292+0000","last_unstale":"2024-05-08T18:07:54.096584+0000","last_undegraded":"2024-05-08T18:07:54.096584+0000","last_fullsized":"2024-05-08T18:07:54.096584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:15.652477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'4","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.035838+0000","last_change":"2024-05-08T18:04:45.829568+0000","last_active":"2024-05-08T18:07:54.035838+0000","last_peered":"2024-05-08T18:07:54.035838+0000","last_clean":"2024-05-08T18:07:54.035838+0000","last_became_active":"2024-05-08T18:04:45.828674+0000","last_became_peered":"2024-05-08T18:04:45.828674+0000","last_unstale":"2024-05-08T18:07:54.035838+0000","last_undegraded":"2024-05-08T18:07:54.035838+0000","last_fullsized":"2024-05-08T18:07:54.035838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:33.440280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":137,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859322+0000","last_change":"2024-05-08T18:04:35.552002+0000","last_active":"2024-05-08T18:05:30.859322+0000","last_peered":"2024-05-08T18:05:30.859322+0000","last_clean":"2024-05-08T18:05:30.859322+0000","last_became_active":"2024-05-08T18:04:35.551437+0000","last_became_peered":"2024-05-08T18:04:35.551437+0000","last_unstale":"2024-05-08T18:05:30.859322+0000","last_undegraded":"2024-05-08T18:05:30.859322+0000","last_fullsized":"2024-05-08T18:05:30.859322+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:58.749794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859862+0000","last_change":"2024-05-08T18:04:35.552341+0000","last_active":"2024-05-08T18:05:30.859862+0000","last_peered":"2024-05-08T18:05:30.859862+0000","last_clean":"2024-05-08T18:05:30.859862+0000","last_became_active":"2024-05-08T18:04:35.552017+0000","last_became_peered":"2024-05-08T18:04:35.552017+0000","last_unstale":"2024-05-08T18:05:30.859862+0000","last_undegraded":"2024-05-08T18:05:30.859862+0000","last_fullsized":"2024-05-08T18:05:30.859862+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:30.242269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858503+0000","last_change":"2024-05-08T18:04:43.837405+0000","last_active":"2024-05-08T18:05:30.858503+0000","last_peered":"2024-05-08T18:05:30.858503+0000","last_clean":"2024-05-08T18:05:30.858503+0000","last_became_active":"2024-05-08T18:04:43.837019+0000","last_became_peered":"2024-05-08T18:04:43.837019+0000","last_unstale":"2024-05-08T18:05:30.858503+0000","last_undegraded":"2024-05-08T18:05:30.858503+0000","last_fullsized":"2024-05-08T18:05:30.858503+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:42:03.247527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.871817+0000","last_change":"2024-05-08T18:05:30.871131+0000","last_active":"2024-05-08T18:05:30.871817+0000","last_peered":"2024-05-08T18:05:30.871817+0000","last_clean":"2024-05-08T18:05:30.871817+0000","last_became_active":"2024-05-08T18:05:30.870647+0000","last_became_peered":"2024-05-08T18:05:30.870647+0000","last_unstale":"2024-05-08T18:05:30.871817+0000","last_undegraded":"2024-05-08T18:05:30.871817+0000","last_fullsized":"2024-05-08T18:05:30.871817+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:48.638819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"29'77","reported_seq":146,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.965816+0000","last_change":"2024-05-08T18:04:49.832374+0000","last_active":"2024-05-08T18:06:51.965816+0000","last_peered":"2024-05-08T18:06:51.965816+0000","last_clean":"2024-05-08T18:06:51.965816+0000","last_became_active":"2024-05-08T18:04:49.830636+0000","last_became_peered":"2024-05-08T18:04:49.830636+0000","last_unstale":"2024-05-08T18:06:51.965816+0000","last_undegraded":"2024-05-08T18:06:51.965816+0000","last_fullsized":"2024-05-08T18:06:51.965816+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:23:12.330689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859612+0000","last_change":"2024-05-08T18:04:51.845249+0000","last_active":"2024-05-08T18:05:30.859612+0000","last_peered":"2024-05-08T18:05:30.859612+0000","last_clean":"2024-05-08T18:05:30.859612+0000","last_became_active":"2024-05-08T18:04:51.844737+0000","last_became_peered":"2024-05-08T18:04:51.844737+0000","last_unstale":"2024-05-08T18:05:30.859612+0000","last_undegraded":"2024-05-08T18:05:30.859612+0000","last_fullsized":"2024-05-08T18:05:30.859612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:08.922575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"29'85","reported_seq":166,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.117787+0000","last_change":"2024-05-08T18:04:53.839334+0000","last_active":"2024-05-08T18:07:54.117787+0000","last_peered":"2024-05-08T18:07:54.117787+0000","last_clean":"2024-05-08T18:07:54.117787+0000","last_became_active":"2024-05-08T18:04:53.838957+0000","last_became_peered":"2024-05-08T18:04:53.838957+0000","last_unstale":"2024-05-08T18:07:54.117787+0000","last_undegraded":"2024-05-08T18:07:54.117787+0000","last_fullsized":"2024-05-08T18:07:54.117787+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:25:57.828734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":216,"num_read_kb":214,"num_write":167,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"29'9","reported_seq":65,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070594+0000","last_change":"2024-05-08T18:04:45.826087+0000","last_active":"2024-05-08T18:07:54.070594+0000","last_peered":"2024-05-08T18:07:54.070594+0000","last_clean":"2024-05-08T18:07:54.070594+0000","last_became_active":"2024-05-08T18:04:45.825843+0000","last_became_peered":"2024-05-08T18:04:45.825843+0000","last_unstale":"2024-05-08T18:07:54.070594+0000","last_undegraded":"2024-05-08T18:07:54.070594+0000","last_fullsized":"2024-05-08T18:07:54.070594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:21:14.431053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857318+0000","last_change":"2024-05-08T18:04:35.552553+0000","last_active":"2024-05-08T18:05:30.857318+0000","last_peered":"2024-05-08T18:05:30.857318+0000","last_clean":"2024-05-08T18:05:30.857318+0000","last_became_active":"2024-05-08T18:04:35.552214+0000","last_became_peered":"2024-05-08T18:04:35.552214+0000","last_unstale":"2024-05-08T18:05:30.857318+0000","last_undegraded":"2024-05-08T18:05:30.857318+0000","last_fullsized":"2024-05-08T18:05:30.857318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:12:16.476925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858237+0000","last_change":"2024-05-08T18:04:43.821058+0000","last_active":"2024-05-08T18:05:30.858237+0000","last_peered":"2024-05-08T18:05:30.858237+0000","last_clean":"2024-05-08T18:05:30.858237+0000","last_became_active":"2024-05-08T18:04:43.820369+0000","last_became_peered":"2024-05-08T18:04:43.820369+0000","last_unstale":"2024-05-08T18:05:30.858237+0000","last_undegraded":"2024-05-08T18:05:30.858237+0000","last_fullsized":"2024-05-08T18:05:30.858237+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:01:49.620924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.871018+0000","last_change":"2024-05-08T18:05:30.870035+0000","last_active":"2024-05-08T18:05:30.871018+0000","last_peered":"2024-05-08T18:05:30.871018+0000","last_clean":"2024-05-08T18:05:30.871018+0000","last_became_active":"2024-05-08T18:05:30.869578+0000","last_became_peered":"2024-05-08T18:05:30.869578+0000","last_unstale":"2024-05-08T18:05:30.871018+0000","last_undegraded":"2024-05-08T18:05:30.871018+0000","last_fullsized":"2024-05-08T18:05:30.871018+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:51:48.737463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"29'273","reported_seq":476,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903545+0000","last_change":"2024-05-08T18:04:49.833379+0000","last_active":"2024-05-08T18:07:52.903545+0000","last_peered":"2024-05-08T18:07:52.903545+0000","last_clean":"2024-05-08T18:07:52.903545+0000","last_became_active":"2024-05-08T18:04:49.833053+0000","last_became_peered":"2024-05-08T18:04:49.833053+0000","last_unstale":"2024-05-08T18:07:52.903545+0000","last_undegraded":"2024-05-08T18:07:52.903545+0000","last_fullsized":"2024-05-08T18:07:52.903545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:21.873389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":357,"num_read_kb":342,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859300+0000","last_change":"2024-05-08T18:04:51.849383+0000","last_active":"2024-05-08T18:05:30.859300+0000","last_peered":"2024-05-08T18:05:30.859300+0000","last_clean":"2024-05-08T18:05:30.859300+0000","last_became_active":"2024-05-08T18:04:51.848066+0000","last_became_peered":"2024-05-08T18:04:51.848066+0000","last_unstale":"2024-05-08T18:05:30.859300+0000","last_undegraded":"2024-05-08T18:05:30.859300+0000","last_fullsized":"2024-05-08T18:05:30.859300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:21.450184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"29'3","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.550250+0000","last_change":"2024-05-08T18:04:53.843187+0000","last_active":"2024-05-08T18:06:09.550250+0000","last_peered":"2024-05-08T18:06:09.550250+0000","last_clean":"2024-05-08T18:06:09.550250+0000","last_became_active":"2024-05-08T18:04:53.842899+0000","last_became_peered":"2024-05-08T18:04:53.842899+0000","last_unstale":"2024-05-08T18:06:09.550250+0000","last_undegraded":"2024-05-08T18:06:09.550250+0000","last_fullsized":"2024-05-08T18:06:09.550250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:05:40.879119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"29'8","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063713+0000","last_change":"2024-05-08T18:04:45.812417+0000","last_active":"2024-05-08T18:07:54.063713+0000","last_peered":"2024-05-08T18:07:54.063713+0000","last_clean":"2024-05-08T18:07:54.063713+0000","last_became_active":"2024-05-08T18:04:45.812091+0000","last_became_peered":"2024-05-08T18:04:45.812091+0000","last_unstale":"2024-05-08T18:07:54.063713+0000","last_undegraded":"2024-05-08T18:07:54.063713+0000","last_fullsized":"2024-05-08T18:07:54.063713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:26.952570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863622+0000","last_change":"2024-05-08T18:04:35.551648+0000","last_active":"2024-05-08T18:05:30.863622+0000","last_peered":"2024-05-08T18:05:30.863622+0000","last_clean":"2024-05-08T18:05:30.863622+0000","last_became_active":"2024-05-08T18:04:35.551313+0000","last_became_peered":"2024-05-08T18:04:35.551313+0000","last_unstale":"2024-05-08T18:05:30.863622+0000","last_undegraded":"2024-05-08T18:05:30.863622+0000","last_fullsized":"2024-05-08T18:05:30.863622+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:34:32.994346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858040+0000","last_change":"2024-05-08T18:04:43.821180+0000","last_active":"2024-05-08T18:05:30.858040+0000","last_peered":"2024-05-08T18:05:30.858040+0000","last_clean":"2024-05-08T18:05:30.858040+0000","last_became_active":"2024-05-08T18:04:43.820335+0000","last_became_peered":"2024-05-08T18:04:43.820335+0000","last_unstale":"2024-05-08T18:05:30.858040+0000","last_undegraded":"2024-05-08T18:05:30.858040+0000","last_fullsized":"2024-05-08T18:05:30.858040+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:31.909506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"20'1","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866981+0000","last_change":"2024-05-08T18:04:47.824034+0000","last_active":"2024-05-08T18:05:30.866981+0000","last_peered":"2024-05-08T18:05:30.866981+0000","last_clean":"2024-05-08T18:05:30.866981+0000","last_became_active":"2024-05-08T18:04:47.823631+0000","last_became_peered":"2024-05-08T18:04:47.823631+0000","last_unstale":"2024-05-08T18:05:30.866981+0000","last_undegraded":"2024-05-08T18:05:30.866981+0000","last_fullsized":"2024-05-08T18:05:30.866981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:00.791290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"29'17","reported_seq":85,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903379+0000","last_change":"2024-05-08T18:04:49.828228+0000","last_active":"2024-05-08T18:07:52.903379+0000","last_peered":"2024-05-08T18:07:52.903379+0000","last_clean":"2024-05-08T18:07:52.903379+0000","last_became_active":"2024-05-08T18:04:49.827735+0000","last_became_peered":"2024-05-08T18:04:49.827735+0000","last_unstale":"2024-05-08T18:07:52.903379+0000","last_undegraded":"2024-05-08T18:07:52.903379+0000","last_fullsized":"2024-05-08T18:07:52.903379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:04:17.239231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861148+0000","last_change":"2024-05-08T18:04:51.844276+0000","last_active":"2024-05-08T18:05:30.861148+0000","last_peered":"2024-05-08T18:05:30.861148+0000","last_clean":"2024-05-08T18:05:30.861148+0000","last_became_active":"2024-05-08T18:04:51.843940+0000","last_became_peered":"2024-05-08T18:04:51.843940+0000","last_unstale":"2024-05-08T18:05:30.861148+0000","last_undegraded":"2024-05-08T18:05:30.861148+0000","last_fullsized":"2024-05-08T18:05:30.861148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:03:10.170093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"29'4","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.791011+0000","last_change":"2024-05-08T18:04:53.848676+0000","last_active":"2024-05-08T18:07:53.791011+0000","last_peered":"2024-05-08T18:07:53.791011+0000","last_clean":"2024-05-08T18:07:53.791011+0000","last_became_active":"2024-05-08T18:04:53.848255+0000","last_became_peered":"2024-05-08T18:04:53.848255+0000","last_unstale":"2024-05-08T18:07:53.791011+0000","last_undegraded":"2024-05-08T18:07:53.791011+0000","last_fullsized":"2024-05-08T18:07:53.791011+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:34:38.766674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"29'10","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063735+0000","last_change":"2024-05-08T18:04:45.807470+0000","last_active":"2024-05-08T18:07:54.063735+0000","last_peered":"2024-05-08T18:07:54.063735+0000","last_clean":"2024-05-08T18:07:54.063735+0000","last_became_active":"2024-05-08T18:04:45.807085+0000","last_became_peered":"2024-05-08T18:04:45.807085+0000","last_unstale":"2024-05-08T18:07:54.063735+0000","last_undegraded":"2024-05-08T18:07:54.063735+0000","last_fullsized":"2024-05-08T18:07:54.063735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:05.828431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857751+0000","last_change":"2024-05-08T18:04:35.551456+0000","last_active":"2024-05-08T18:05:30.857751+0000","last_peered":"2024-05-08T18:05:30.857751+0000","last_clean":"2024-05-08T18:05:30.857751+0000","last_became_active":"2024-05-08T18:04:35.551120+0000","last_became_peered":"2024-05-08T18:04:35.551120+0000","last_unstale":"2024-05-08T18:05:30.857751+0000","last_undegraded":"2024-05-08T18:05:30.857751+0000","last_fullsized":"2024-05-08T18:05:30.857751+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:14.181756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864168+0000","last_change":"2024-05-08T18:04:43.840439+0000","last_active":"2024-05-08T18:05:30.864168+0000","last_peered":"2024-05-08T18:05:30.864168+0000","last_clean":"2024-05-08T18:05:30.864168+0000","last_became_active":"2024-05-08T18:04:43.840036+0000","last_became_peered":"2024-05-08T18:04:43.840036+0000","last_unstale":"2024-05-08T18:05:30.864168+0000","last_undegraded":"2024-05-08T18:05:30.864168+0000","last_fullsized":"2024-05-08T18:05:30.864168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:31.909506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0,3],"acting":[2,1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859629+0000","last_change":"2024-05-08T18:04:47.823507+0000","last_active":"2024-05-08T18:05:30.859629+0000","last_peered":"2024-05-08T18:05:30.859629+0000","last_clean":"2024-05-08T18:05:30.859629+0000","last_became_active":"2024-05-08T18:04:47.823093+0000","last_became_peered":"2024-05-08T18:04:47.823093+0000","last_unstale":"2024-05-08T18:05:30.859629+0000","last_undegraded":"2024-05-08T18:05:30.859629+0000","last_fullsized":"2024-05-08T18:05:30.859629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:59.706946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"29'13","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.412397+0000","last_change":"2024-05-08T18:04:49.832657+0000","last_active":"2024-05-08T18:07:53.412397+0000","last_peered":"2024-05-08T18:07:53.412397+0000","last_clean":"2024-05-08T18:07:53.412397+0000","last_became_active":"2024-05-08T18:04:49.830586+0000","last_became_peered":"2024-05-08T18:04:49.830586+0000","last_unstale":"2024-05-08T18:07:53.412397+0000","last_undegraded":"2024-05-08T18:07:53.412397+0000","last_fullsized":"2024-05-08T18:07:53.412397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:01:11.727041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864596+0000","last_change":"2024-05-08T18:04:51.849539+0000","last_active":"2024-05-08T18:05:30.864596+0000","last_peered":"2024-05-08T18:05:30.864596+0000","last_clean":"2024-05-08T18:05:30.864596+0000","last_became_active":"2024-05-08T18:04:51.849137+0000","last_became_peered":"2024-05-08T18:04:51.849137+0000","last_unstale":"2024-05-08T18:05:30.864596+0000","last_undegraded":"2024-05-08T18:05:30.864596+0000","last_fullsized":"2024-05-08T18:05:30.864596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:05.655002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'4","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.159086+0000","last_change":"2024-05-08T18:04:53.851345+0000","last_active":"2024-05-08T18:06:53.159086+0000","last_peered":"2024-05-08T18:06:53.159086+0000","last_clean":"2024-05-08T18:06:53.159086+0000","last_became_active":"2024-05-08T18:04:53.851008+0000","last_became_peered":"2024-05-08T18:04:53.851008+0000","last_unstale":"2024-05-08T18:06:53.159086+0000","last_undegraded":"2024-05-08T18:06:53.159086+0000","last_fullsized":"2024-05-08T18:06:53.159086+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:43:02.228949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.865362+0000","last_change":"2024-05-08T18:04:45.820976+0000","last_active":"2024-05-08T18:05:30.865362+0000","last_peered":"2024-05-08T18:05:30.865362+0000","last_clean":"2024-05-08T18:05:30.865362+0000","last_became_active":"2024-05-08T18:04:45.819326+0000","last_became_peered":"2024-05-08T18:04:45.819326+0000","last_unstale":"2024-05-08T18:05:30.865362+0000","last_undegraded":"2024-05-08T18:05:30.865362+0000","last_fullsized":"2024-05-08T18:05:30.865362+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:22:19.585204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862814+0000","last_change":"2024-05-08T18:04:43.840661+0000","last_active":"2024-05-08T18:05:30.862814+0000","last_peered":"2024-05-08T18:05:30.862814+0000","last_clean":"2024-05-08T18:05:30.862814+0000","last_became_active":"2024-05-08T18:04:43.840236+0000","last_became_peered":"2024-05-08T18:04:43.840236+0000","last_unstale":"2024-05-08T18:05:30.862814+0000","last_undegraded":"2024-05-08T18:05:30.862814+0000","last_fullsized":"2024-05-08T18:05:30.862814+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:40:40.407240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859953+0000","last_change":"2024-05-08T18:04:35.552512+0000","last_active":"2024-05-08T18:05:30.859953+0000","last_peered":"2024-05-08T18:05:30.859953+0000","last_clean":"2024-05-08T18:05:30.859953+0000","last_became_active":"2024-05-08T18:04:35.552173+0000","last_became_peered":"2024-05-08T18:04:35.552173+0000","last_unstale":"2024-05-08T18:05:30.859953+0000","last_undegraded":"2024-05-08T18:05:30.859953+0000","last_fullsized":"2024-05-08T18:05:30.859953+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:46:49.443150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"29'1","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.738959+0000","last_change":"2024-05-08T18:04:47.820423+0000","last_active":"2024-05-08T18:06:53.738959+0000","last_peered":"2024-05-08T18:06:53.738959+0000","last_clean":"2024-05-08T18:06:53.738959+0000","last_became_active":"2024-05-08T18:04:47.819910+0000","last_became_peered":"2024-05-08T18:04:47.819910+0000","last_unstale":"2024-05-08T18:06:53.738959+0000","last_undegraded":"2024-05-08T18:06:53.738959+0000","last_fullsized":"2024-05-08T18:06:53.738959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:41:39.241804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"29'61","reported_seq":120,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.738898+0000","last_change":"2024-05-08T18:04:49.831684+0000","last_active":"2024-05-08T18:06:53.738898+0000","last_peered":"2024-05-08T18:06:53.738898+0000","last_clean":"2024-05-08T18:06:53.738898+0000","last_became_active":"2024-05-08T18:04:49.831360+0000","last_became_peered":"2024-05-08T18:04:49.831360+0000","last_unstale":"2024-05-08T18:06:53.738898+0000","last_undegraded":"2024-05-08T18:06:53.738898+0000","last_fullsized":"2024-05-08T18:06:53.738898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:33:52.278907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":59,"num_read_kb":47,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859470+0000","last_change":"2024-05-08T18:04:51.836834+0000","last_active":"2024-05-08T18:05:30.859470+0000","last_peered":"2024-05-08T18:05:30.859470+0000","last_clean":"2024-05-08T18:05:30.859470+0000","last_became_active":"2024-05-08T18:04:51.836441+0000","last_became_peered":"2024-05-08T18:04:51.836441+0000","last_unstale":"2024-05-08T18:05:30.859470+0000","last_undegraded":"2024-05-08T18:05:30.859470+0000","last_fullsized":"2024-05-08T18:05:30.859470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:42.109569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"29'9","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.337708+0000","last_change":"2024-05-08T18:04:53.838192+0000","last_active":"2024-05-08T18:06:53.337708+0000","last_peered":"2024-05-08T18:06:53.337708+0000","last_clean":"2024-05-08T18:06:53.337708+0000","last_became_active":"2024-05-08T18:04:53.837920+0000","last_became_peered":"2024-05-08T18:04:53.837920+0000","last_unstale":"2024-05-08T18:06:53.337708+0000","last_undegraded":"2024-05-08T18:06:53.337708+0000","last_fullsized":"2024-05-08T18:06:53.337708+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:53.793862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"29'15","reported_seq":77,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.042063+0000","last_change":"2024-05-08T18:04:45.821636+0000","last_active":"2024-05-08T18:07:54.042063+0000","last_peered":"2024-05-08T18:07:54.042063+0000","last_clean":"2024-05-08T18:07:54.042063+0000","last_became_active":"2024-05-08T18:04:45.821107+0000","last_became_peered":"2024-05-08T18:04:45.821107+0000","last_unstale":"2024-05-08T18:07:54.042063+0000","last_undegraded":"2024-05-08T18:07:54.042063+0000","last_fullsized":"2024-05-08T18:07:54.042063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:22:55.457884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861385+0000","last_change":"2024-05-08T18:04:43.828309+0000","last_active":"2024-05-08T18:05:30.861385+0000","last_peered":"2024-05-08T18:05:30.861385+0000","last_clean":"2024-05-08T18:05:30.861385+0000","last_became_active":"2024-05-08T18:04:43.827920+0000","last_became_peered":"2024-05-08T18:04:43.827920+0000","last_unstale":"2024-05-08T18:05:30.861385+0000","last_undegraded":"2024-05-08T18:05:30.861385+0000","last_fullsized":"2024-05-08T18:05:30.861385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:41:01.395720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860131+0000","last_change":"2024-05-08T18:04:35.551253+0000","last_active":"2024-05-08T18:05:30.860131+0000","last_peered":"2024-05-08T18:05:30.860131+0000","last_clean":"2024-05-08T18:05:30.860131+0000","last_became_active":"2024-05-08T18:04:35.550912+0000","last_became_peered":"2024-05-08T18:04:35.550912+0000","last_unstale":"2024-05-08T18:05:30.860131+0000","last_undegraded":"2024-05-08T18:05:30.860131+0000","last_fullsized":"2024-05-08T18:05:30.860131+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:49.303072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857486+0000","last_change":"2024-05-08T18:04:47.814015+0000","last_active":"2024-05-08T18:05:30.857486+0000","last_peered":"2024-05-08T18:05:30.857486+0000","last_clean":"2024-05-08T18:05:30.857486+0000","last_became_active":"2024-05-08T18:04:47.813717+0000","last_became_peered":"2024-05-08T18:04:47.813717+0000","last_unstale":"2024-05-08T18:05:30.857486+0000","last_undegraded":"2024-05-08T18:05:30.857486+0000","last_fullsized":"2024-05-08T18:05:30.857486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:41.988054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"27'14","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861597+0000","last_change":"2024-05-08T18:04:49.835869+0000","last_active":"2024-05-08T18:05:30.861597+0000","last_peered":"2024-05-08T18:05:30.861597+0000","last_clean":"2024-05-08T18:05:30.861597+0000","last_became_active":"2024-05-08T18:04:49.835451+0000","last_became_peered":"2024-05-08T18:04:49.835451+0000","last_unstale":"2024-05-08T18:05:30.861597+0000","last_undegraded":"2024-05-08T18:05:30.861597+0000","last_fullsized":"2024-05-08T18:05:30.861597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:33.133985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"29'282","reported_seq":369,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903220+0000","last_change":"2024-05-08T18:04:51.832417+0000","last_active":"2024-05-08T18:07:52.903220+0000","last_peered":"2024-05-08T18:07:52.903220+0000","last_clean":"2024-05-08T18:07:52.903220+0000","last_became_active":"2024-05-08T18:04:51.832109+0000","last_became_peered":"2024-05-08T18:04:51.832109+0000","last_unstale":"2024-05-08T18:07:52.903220+0000","last_undegraded":"2024-05-08T18:07:52.903220+0000","last_fullsized":"2024-05-08T18:07:52.903220+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:37.543160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860146+0000","last_change":"2024-05-08T18:04:53.844572+0000","last_active":"2024-05-08T18:05:30.860146+0000","last_peered":"2024-05-08T18:05:30.860146+0000","last_clean":"2024-05-08T18:05:30.860146+0000","last_became_active":"2024-05-08T18:04:53.842998+0000","last_became_peered":"2024-05-08T18:04:53.842998+0000","last_unstale":"2024-05-08T18:05:30.860146+0000","last_undegraded":"2024-05-08T18:05:30.860146+0000","last_fullsized":"2024-05-08T18:05:30.860146+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:50.643601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'4","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.775877+0000","last_change":"2024-05-08T18:04:45.830059+0000","last_active":"2024-05-08T18:07:53.775877+0000","last_peered":"2024-05-08T18:07:53.775877+0000","last_clean":"2024-05-08T18:07:53.775877+0000","last_became_active":"2024-05-08T18:04:45.829257+0000","last_became_peered":"2024-05-08T18:04:45.829257+0000","last_unstale":"2024-05-08T18:07:53.775877+0000","last_undegraded":"2024-05-08T18:07:53.775877+0000","last_fullsized":"2024-05-08T18:07:53.775877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:10:31.034888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"29'3","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.536792+0000","last_change":"2024-05-08T18:04:43.835453+0000","last_active":"2024-05-08T18:06:09.536792+0000","last_peered":"2024-05-08T18:06:09.536792+0000","last_clean":"2024-05-08T18:06:09.536792+0000","last_became_active":"2024-05-08T18:04:43.834610+0000","last_became_peered":"2024-05-08T18:04:43.834610+0000","last_unstale":"2024-05-08T18:06:09.536792+0000","last_undegraded":"2024-05-08T18:06:09.536792+0000","last_fullsized":"2024-05-08T18:06:09.536792+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:03.821172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859946+0000","last_change":"2024-05-08T18:04:35.551595+0000","last_active":"2024-05-08T18:05:30.859946+0000","last_peered":"2024-05-08T18:05:30.859946+0000","last_clean":"2024-05-08T18:05:30.859946+0000","last_became_active":"2024-05-08T18:04:35.551366+0000","last_became_peered":"2024-05-08T18:04:35.551366+0000","last_unstale":"2024-05-08T18:05:30.859946+0000","last_undegraded":"2024-05-08T18:05:30.859946+0000","last_fullsized":"2024-05-08T18:05:30.859946+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:00.951608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857461+0000","last_change":"2024-05-08T18:04:47.822209+0000","last_active":"2024-05-08T18:05:30.857461+0000","last_peered":"2024-05-08T18:05:30.857461+0000","last_clean":"2024-05-08T18:05:30.857461+0000","last_became_active":"2024-05-08T18:04:47.819866+0000","last_became_peered":"2024-05-08T18:04:47.819866+0000","last_unstale":"2024-05-08T18:05:30.857461+0000","last_undegraded":"2024-05-08T18:05:30.857461+0000","last_fullsized":"2024-05-08T18:05:30.857461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:28.233298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"27'10","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858934+0000","last_change":"2024-05-08T18:04:49.832338+0000","last_active":"2024-05-08T18:05:30.858934+0000","last_peered":"2024-05-08T18:05:30.858934+0000","last_clean":"2024-05-08T18:05:30.858934+0000","last_became_active":"2024-05-08T18:04:49.830452+0000","last_became_peered":"2024-05-08T18:04:49.830452+0000","last_unstale":"2024-05-08T18:05:30.858934+0000","last_undegraded":"2024-05-08T18:05:30.858934+0000","last_fullsized":"2024-05-08T18:05:30.858934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:49.779432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"29'282","reported_seq":399,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.172497+0000","last_change":"2024-05-08T18:04:51.831530+0000","last_active":"2024-05-08T18:07:54.172497+0000","last_peered":"2024-05-08T18:07:54.172497+0000","last_clean":"2024-05-08T18:07:54.172497+0000","last_became_active":"2024-05-08T18:04:51.831155+0000","last_became_peered":"2024-05-08T18:04:51.831155+0000","last_unstale":"2024-05-08T18:07:54.172497+0000","last_undegraded":"2024-05-08T18:07:54.172497+0000","last_fullsized":"2024-05-08T18:07:54.172497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:32.247507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'5","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.023861+0000","last_change":"2024-05-08T18:04:53.849164+0000","last_active":"2024-05-08T18:07:54.023861+0000","last_peered":"2024-05-08T18:07:54.023861+0000","last_clean":"2024-05-08T18:07:54.023861+0000","last_became_active":"2024-05-08T18:04:53.848862+0000","last_became_peered":"2024-05-08T18:04:53.848862+0000","last_unstale":"2024-05-08T18:07:54.023861+0000","last_undegraded":"2024-05-08T18:07:54.023861+0000","last_fullsized":"2024-05-08T18:07:54.023861+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:33:19.797257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"29'6","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.867036+0000","last_change":"2024-05-08T18:04:45.823169+0000","last_active":"2024-05-08T18:07:53.867036+0000","last_peered":"2024-05-08T18:07:53.867036+0000","last_clean":"2024-05-08T18:07:53.867036+0000","last_became_active":"2024-05-08T18:04:45.822824+0000","last_became_peered":"2024-05-08T18:04:45.822824+0000","last_unstale":"2024-05-08T18:07:53.867036+0000","last_undegraded":"2024-05-08T18:07:53.867036+0000","last_fullsized":"2024-05-08T18:07:53.867036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:34.503647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862506+0000","last_change":"2024-05-08T18:04:43.839633+0000","last_active":"2024-05-08T18:05:30.862506+0000","last_peered":"2024-05-08T18:05:30.862506+0000","last_clean":"2024-05-08T18:05:30.862506+0000","last_became_active":"2024-05-08T18:04:43.839240+0000","last_became_peered":"2024-05-08T18:04:43.839240+0000","last_unstale":"2024-05-08T18:05:30.862506+0000","last_undegraded":"2024-05-08T18:05:30.862506+0000","last_fullsized":"2024-05-08T18:05:30.862506+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:31.909506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864842+0000","last_change":"2024-05-08T18:04:35.551765+0000","last_active":"2024-05-08T18:05:30.864842+0000","last_peered":"2024-05-08T18:05:30.864842+0000","last_clean":"2024-05-08T18:05:30.864842+0000","last_became_active":"2024-05-08T18:04:35.551412+0000","last_became_peered":"2024-05-08T18:04:35.551412+0000","last_unstale":"2024-05-08T18:05:30.864842+0000","last_undegraded":"2024-05-08T18:05:30.864842+0000","last_fullsized":"2024-05-08T18:05:30.864842+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:37:38.776558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857179+0000","last_change":"2024-05-08T18:04:47.816250+0000","last_active":"2024-05-08T18:05:30.857179+0000","last_peered":"2024-05-08T18:05:30.857179+0000","last_clean":"2024-05-08T18:05:30.857179+0000","last_became_active":"2024-05-08T18:04:47.815947+0000","last_became_peered":"2024-05-08T18:04:47.815947+0000","last_unstale":"2024-05-08T18:05:30.857179+0000","last_undegraded":"2024-05-08T18:05:30.857179+0000","last_fullsized":"2024-05-08T18:05:30.857179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:16:57.162048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"29'82","reported_seq":154,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.952921+0000","last_change":"2024-05-08T18:04:49.834985+0000","last_active":"2024-05-08T18:06:51.952921+0000","last_peered":"2024-05-08T18:06:51.952921+0000","last_clean":"2024-05-08T18:06:51.952921+0000","last_became_active":"2024-05-08T18:04:49.834586+0000","last_became_peered":"2024-05-08T18:04:49.834586+0000","last_unstale":"2024-05-08T18:06:51.952921+0000","last_undegraded":"2024-05-08T18:06:51.952921+0000","last_fullsized":"2024-05-08T18:06:51.952921+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:26.820238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858181+0000","last_change":"2024-05-08T18:04:51.831135+0000","last_active":"2024-05-08T18:05:30.858181+0000","last_peered":"2024-05-08T18:05:30.858181+0000","last_clean":"2024-05-08T18:05:30.858181+0000","last_became_active":"2024-05-08T18:04:51.830760+0000","last_became_peered":"2024-05-08T18:04:51.830760+0000","last_unstale":"2024-05-08T18:05:30.858181+0000","last_undegraded":"2024-05-08T18:05:30.858181+0000","last_fullsized":"2024-05-08T18:05:30.858181+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:15.559137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.865409+0000","last_change":"2024-05-08T18:04:53.839069+0000","last_active":"2024-05-08T18:05:30.865409+0000","last_peered":"2024-05-08T18:05:30.865409+0000","last_clean":"2024-05-08T18:05:30.865409+0000","last_became_active":"2024-05-08T18:04:53.838600+0000","last_became_peered":"2024-05-08T18:04:53.838600+0000","last_unstale":"2024-05-08T18:05:30.865409+0000","last_undegraded":"2024-05-08T18:05:30.865409+0000","last_fullsized":"2024-05-08T18:05:30.865409+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:17.566875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"27'2","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866690+0000","last_change":"2024-05-08T18:04:53.839793+0000","last_active":"2024-05-08T18:05:30.866690+0000","last_peered":"2024-05-08T18:05:30.866690+0000","last_clean":"2024-05-08T18:05:30.866690+0000","last_became_active":"2024-05-08T18:04:53.839500+0000","last_became_peered":"2024-05-08T18:04:53.839500+0000","last_unstale":"2024-05-08T18:05:30.866690+0000","last_undegraded":"2024-05-08T18:05:30.866690+0000","last_fullsized":"2024-05-08T18:05:30.866690+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:19.575012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"29'8","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.219834+0000","last_change":"2024-05-08T18:04:45.819806+0000","last_active":"2024-05-08T18:06:50.219834+0000","last_peered":"2024-05-08T18:06:50.219834+0000","last_clean":"2024-05-08T18:06:50.219834+0000","last_became_active":"2024-05-08T18:04:45.819508+0000","last_became_peered":"2024-05-08T18:04:45.819508+0000","last_unstale":"2024-05-08T18:06:50.219834+0000","last_undegraded":"2024-05-08T18:06:50.219834+0000","last_fullsized":"2024-05-08T18:06:50.219834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:32.833248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859163+0000","last_change":"2024-05-08T18:04:43.844864+0000","last_active":"2024-05-08T18:05:30.859163+0000","last_peered":"2024-05-08T18:05:30.859163+0000","last_clean":"2024-05-08T18:05:30.859163+0000","last_became_active":"2024-05-08T18:04:43.837929+0000","last_became_peered":"2024-05-08T18:04:43.837929+0000","last_unstale":"2024-05-08T18:05:30.859163+0000","last_undegraded":"2024-05-08T18:05:30.859163+0000","last_fullsized":"2024-05-08T18:05:30.859163+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:06.177343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861102+0000","last_change":"2024-05-08T18:04:47.818763+0000","last_active":"2024-05-08T18:05:30.861102+0000","last_peered":"2024-05-08T18:05:30.861102+0000","last_clean":"2024-05-08T18:05:30.861102+0000","last_became_active":"2024-05-08T18:04:47.818443+0000","last_became_peered":"2024-05-08T18:04:47.818443+0000","last_unstale":"2024-05-08T18:05:30.861102+0000","last_undegraded":"2024-05-08T18:05:30.861102+0000","last_fullsized":"2024-05-08T18:05:30.861102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:15:57.213130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"29'32","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.958612+0000","last_change":"2024-05-08T18:04:49.834233+0000","last_active":"2024-05-08T18:06:51.958612+0000","last_peered":"2024-05-08T18:06:51.958612+0000","last_clean":"2024-05-08T18:06:51.958612+0000","last_became_active":"2024-05-08T18:04:49.833599+0000","last_became_peered":"2024-05-08T18:04:49.833599+0000","last_unstale":"2024-05-08T18:06:51.958612+0000","last_undegraded":"2024-05-08T18:06:51.958612+0000","last_fullsized":"2024-05-08T18:06:51.958612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:54:34.743324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858258+0000","last_change":"2024-05-08T18:04:51.844827+0000","last_active":"2024-05-08T18:05:30.858258+0000","last_peered":"2024-05-08T18:05:30.858258+0000","last_clean":"2024-05-08T18:05:30.858258+0000","last_became_active":"2024-05-08T18:04:51.844526+0000","last_became_peered":"2024-05-08T18:04:51.844526+0000","last_unstale":"2024-05-08T18:05:30.858258+0000","last_undegraded":"2024-05-08T18:05:30.858258+0000","last_fullsized":"2024-05-08T18:05:30.858258+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:13:54.471619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"29'11","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.639339+0000","last_change":"2024-05-08T18:04:53.850911+0000","last_active":"2024-05-08T18:06:51.639339+0000","last_peered":"2024-05-08T18:06:51.639339+0000","last_clean":"2024-05-08T18:06:51.639339+0000","last_became_active":"2024-05-08T18:04:53.850593+0000","last_became_peered":"2024-05-08T18:04:53.850593+0000","last_unstale":"2024-05-08T18:06:51.639339+0000","last_undegraded":"2024-05-08T18:06:51.639339+0000","last_fullsized":"2024-05-08T18:06:51.639339+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:11:21.798553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":62,"num_write":31,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"29'8","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399973+0000","last_change":"2024-05-08T18:04:45.819974+0000","last_active":"2024-05-08T18:06:53.399973+0000","last_peered":"2024-05-08T18:06:53.399973+0000","last_clean":"2024-05-08T18:06:53.399973+0000","last_became_active":"2024-05-08T18:04:45.818979+0000","last_became_peered":"2024-05-08T18:04:45.818979+0000","last_unstale":"2024-05-08T18:06:53.399973+0000","last_undegraded":"2024-05-08T18:06:53.399973+0000","last_fullsized":"2024-05-08T18:06:53.399973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:30.458103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863085+0000","last_change":"2024-05-08T18:04:43.836993+0000","last_active":"2024-05-08T18:05:30.863085+0000","last_peered":"2024-05-08T18:05:30.863085+0000","last_clean":"2024-05-08T18:05:30.863085+0000","last_became_active":"2024-05-08T18:04:43.836614+0000","last_became_peered":"2024-05-08T18:04:43.836614+0000","last_unstale":"2024-05-08T18:05:30.863085+0000","last_undegraded":"2024-05-08T18:05:30.863085+0000","last_fullsized":"2024-05-08T18:05:30.863085+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:05:33.776915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858802+0000","last_change":"2024-05-08T18:04:47.814847+0000","last_active":"2024-05-08T18:05:30.858802+0000","last_peered":"2024-05-08T18:05:30.858802+0000","last_clean":"2024-05-08T18:05:30.858802+0000","last_became_active":"2024-05-08T18:04:47.814561+0000","last_became_peered":"2024-05-08T18:04:47.814561+0000","last_unstale":"2024-05-08T18:05:30.858802+0000","last_undegraded":"2024-05-08T18:05:30.858802+0000","last_fullsized":"2024-05-08T18:05:30.858802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:58.115834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"29'7","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.433454+0000","last_change":"2024-05-08T18:04:49.825623+0000","last_active":"2024-05-08T18:07:53.433454+0000","last_peered":"2024-05-08T18:07:53.433454+0000","last_clean":"2024-05-08T18:07:53.433454+0000","last_became_active":"2024-05-08T18:04:49.825347+0000","last_became_peered":"2024-05-08T18:04:49.825347+0000","last_unstale":"2024-05-08T18:07:53.433454+0000","last_undegraded":"2024-05-08T18:07:53.433454+0000","last_fullsized":"2024-05-08T18:07:53.433454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:37.454897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863344+0000","last_change":"2024-05-08T18:04:51.843388+0000","last_active":"2024-05-08T18:05:30.863344+0000","last_peered":"2024-05-08T18:05:30.863344+0000","last_clean":"2024-05-08T18:05:30.863344+0000","last_became_active":"2024-05-08T18:04:51.843002+0000","last_became_peered":"2024-05-08T18:04:51.843002+0000","last_unstale":"2024-05-08T18:05:30.863344+0000","last_undegraded":"2024-05-08T18:05:30.863344+0000","last_fullsized":"2024-05-08T18:05:30.863344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:13.211841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859114+0000","last_change":"2024-05-08T18:04:53.840844+0000","last_active":"2024-05-08T18:05:30.859114+0000","last_peered":"2024-05-08T18:05:30.859114+0000","last_clean":"2024-05-08T18:05:30.859114+0000","last_became_active":"2024-05-08T18:04:53.840437+0000","last_became_peered":"2024-05-08T18:04:53.840437+0000","last_unstale":"2024-05-08T18:05:30.859114+0000","last_undegraded":"2024-05-08T18:05:30.859114+0000","last_fullsized":"2024-05-08T18:05:30.859114+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:42:52.195643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"29'8","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.861059+0000","last_change":"2024-05-08T18:05:30.868442+0000","last_active":"2024-05-08T18:07:53.861059+0000","last_peered":"2024-05-08T18:07:53.861059+0000","last_clean":"2024-05-08T18:07:53.861059+0000","last_became_active":"2024-05-08T18:05:30.868156+0000","last_became_peered":"2024-05-08T18:05:30.868156+0000","last_unstale":"2024-05-08T18:07:53.861059+0000","last_undegraded":"2024-05-08T18:07:53.861059+0000","last_fullsized":"2024-05-08T18:07:53.861059+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:44.461215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867524+0000","last_change":"2024-05-08T18:04:43.835856+0000","last_active":"2024-05-08T18:05:30.867524+0000","last_peered":"2024-05-08T18:05:30.867524+0000","last_clean":"2024-05-08T18:05:30.867524+0000","last_became_active":"2024-05-08T18:04:43.835435+0000","last_became_peered":"2024-05-08T18:04:43.835435+0000","last_unstale":"2024-05-08T18:05:30.867524+0000","last_undegraded":"2024-05-08T18:05:30.867524+0000","last_fullsized":"2024-05-08T18:05:30.867524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:47.971014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860152+0000","last_change":"2024-05-08T18:04:47.824614+0000","last_active":"2024-05-08T18:05:30.860152+0000","last_peered":"2024-05-08T18:05:30.860152+0000","last_clean":"2024-05-08T18:05:30.860152+0000","last_became_active":"2024-05-08T18:04:47.824140+0000","last_became_peered":"2024-05-08T18:04:47.824140+0000","last_unstale":"2024-05-08T18:05:30.860152+0000","last_undegraded":"2024-05-08T18:05:30.860152+0000","last_fullsized":"2024-05-08T18:05:30.860152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:36:47.466900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"29'41","reported_seq":100,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.004393+0000","last_change":"2024-05-08T18:04:49.824946+0000","last_active":"2024-05-08T18:07:54.004393+0000","last_peered":"2024-05-08T18:07:54.004393+0000","last_clean":"2024-05-08T18:07:54.004393+0000","last_became_active":"2024-05-08T18:04:49.824600+0000","last_became_peered":"2024-05-08T18:04:49.824600+0000","last_unstale":"2024-05-08T18:07:54.004393+0000","last_undegraded":"2024-05-08T18:07:54.004393+0000","last_fullsized":"2024-05-08T18:07:54.004393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:11.205149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":64,"num_write":46,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860821+0000","last_change":"2024-05-08T18:04:51.833599+0000","last_active":"2024-05-08T18:05:30.860821+0000","last_peered":"2024-05-08T18:05:30.860821+0000","last_clean":"2024-05-08T18:05:30.860821+0000","last_became_active":"2024-05-08T18:04:51.829937+0000","last_became_peered":"2024-05-08T18:04:51.829937+0000","last_unstale":"2024-05-08T18:05:30.860821+0000","last_undegraded":"2024-05-08T18:05:30.860821+0000","last_fullsized":"2024-05-08T18:05:30.860821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:02:48.266056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858578+0000","last_change":"2024-05-08T18:04:53.842981+0000","last_active":"2024-05-08T18:05:30.858578+0000","last_peered":"2024-05-08T18:05:30.858578+0000","last_clean":"2024-05-08T18:05:30.858578+0000","last_became_active":"2024-05-08T18:04:53.842610+0000","last_became_peered":"2024-05-08T18:04:53.842610+0000","last_unstale":"2024-05-08T18:05:30.858578+0000","last_undegraded":"2024-05-08T18:05:30.858578+0000","last_fullsized":"2024-05-08T18:05:30.858578+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:16:15.240226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857482+0000","last_change":"2024-05-08T18:04:45.829838+0000","last_active":"2024-05-08T18:05:30.857482+0000","last_peered":"2024-05-08T18:05:30.857482+0000","last_clean":"2024-05-08T18:05:30.857482+0000","last_became_active":"2024-05-08T18:04:45.829458+0000","last_became_peered":"2024-05-08T18:04:45.829458+0000","last_unstale":"2024-05-08T18:05:30.857482+0000","last_undegraded":"2024-05-08T18:05:30.857482+0000","last_fullsized":"2024-05-08T18:05:30.857482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:15:55.206029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"29'4","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.352042+0000","last_change":"2024-05-08T18:04:43.837089+0000","last_active":"2024-05-08T18:06:53.352042+0000","last_peered":"2024-05-08T18:06:53.352042+0000","last_clean":"2024-05-08T18:06:53.352042+0000","last_became_active":"2024-05-08T18:04:43.836633+0000","last_became_peered":"2024-05-08T18:04:43.836633+0000","last_unstale":"2024-05-08T18:06:53.352042+0000","last_undegraded":"2024-05-08T18:06:53.352042+0000","last_fullsized":"2024-05-08T18:06:53.352042+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:55.493870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"20'1","reported_seq":218,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.049308+0000","last_change":"2024-05-08T18:04:47.820110+0000","last_active":"2024-05-08T18:06:53.049308+0000","last_peered":"2024-05-08T18:06:53.049308+0000","last_clean":"2024-05-08T18:06:53.049308+0000","last_became_active":"2024-05-08T18:04:47.819771+0000","last_became_peered":"2024-05-08T18:04:47.819771+0000","last_unstale":"2024-05-08T18:06:53.049308+0000","last_undegraded":"2024-05-08T18:06:53.049308+0000","last_fullsized":"2024-05-08T18:06:53.049308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:42:21.961312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"29'30","reported_seq":72,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.944676+0000","last_change":"2024-05-08T18:04:49.824323+0000","last_active":"2024-05-08T18:06:51.944676+0000","last_peered":"2024-05-08T18:06:51.944676+0000","last_clean":"2024-05-08T18:06:51.944676+0000","last_became_active":"2024-05-08T18:04:49.824022+0000","last_became_peered":"2024-05-08T18:04:49.824022+0000","last_unstale":"2024-05-08T18:06:51.944676+0000","last_undegraded":"2024-05-08T18:06:51.944676+0000","last_fullsized":"2024-05-08T18:06:51.944676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:41:41.249321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866329+0000","last_change":"2024-05-08T18:04:51.826713+0000","last_active":"2024-05-08T18:05:30.866329+0000","last_peered":"2024-05-08T18:05:30.866329+0000","last_clean":"2024-05-08T18:05:30.866329+0000","last_became_active":"2024-05-08T18:04:51.826424+0000","last_became_peered":"2024-05-08T18:04:51.826424+0000","last_unstale":"2024-05-08T18:05:30.866329+0000","last_undegraded":"2024-05-08T18:05:30.866329+0000","last_fullsized":"2024-05-08T18:05:30.866329+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:33:54.285599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"29'20","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995883+0000","last_change":"2024-05-08T18:05:30.873315+0000","last_active":"2024-05-08T18:07:53.995883+0000","last_peered":"2024-05-08T18:07:53.995883+0000","last_clean":"2024-05-08T18:07:53.995883+0000","last_became_active":"2024-05-08T18:05:30.868409+0000","last_became_peered":"2024-05-08T18:05:30.868409+0000","last_unstale":"2024-05-08T18:07:53.995883+0000","last_undegraded":"2024-05-08T18:07:53.995883+0000","last_fullsized":"2024-05-08T18:07:53.995883+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:07.623428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857341+0000","last_change":"2024-05-08T18:04:43.844983+0000","last_active":"2024-05-08T18:05:30.857341+0000","last_peered":"2024-05-08T18:05:30.857341+0000","last_clean":"2024-05-08T18:05:30.857341+0000","last_became_active":"2024-05-08T18:04:43.838957+0000","last_became_peered":"2024-05-08T18:04:43.838957+0000","last_unstale":"2024-05-08T18:05:30.857341+0000","last_undegraded":"2024-05-08T18:05:30.857341+0000","last_fullsized":"2024-05-08T18:05:30.857341+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:42:05.210775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859885+0000","last_change":"2024-05-08T18:04:47.820891+0000","last_active":"2024-05-08T18:05:30.859885+0000","last_peered":"2024-05-08T18:05:30.859885+0000","last_clean":"2024-05-08T18:05:30.859885+0000","last_became_active":"2024-05-08T18:04:47.820256+0000","last_became_peered":"2024-05-08T18:04:47.820256+0000","last_unstale":"2024-05-08T18:05:30.859885+0000","last_undegraded":"2024-05-08T18:05:30.859885+0000","last_fullsized":"2024-05-08T18:05:30.859885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:04:15.231714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"29'54","reported_seq":110,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.964783+0000","last_change":"2024-05-08T18:04:49.825705+0000","last_active":"2024-05-08T18:06:51.964783+0000","last_peered":"2024-05-08T18:06:51.964783+0000","last_clean":"2024-05-08T18:06:51.964783+0000","last_became_active":"2024-05-08T18:04:49.825366+0000","last_became_peered":"2024-05-08T18:04:49.825366+0000","last_unstale":"2024-05-08T18:06:51.964783+0000","last_undegraded":"2024-05-08T18:06:51.964783+0000","last_fullsized":"2024-05-08T18:06:51.964783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:27:27.454431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862900+0000","last_change":"2024-05-08T18:04:51.827568+0000","last_active":"2024-05-08T18:05:30.862900+0000","last_peered":"2024-05-08T18:05:30.862900+0000","last_clean":"2024-05-08T18:05:30.862900+0000","last_became_active":"2024-05-08T18:04:51.827127+0000","last_became_peered":"2024-05-08T18:04:51.827127+0000","last_unstale":"2024-05-08T18:05:30.862900+0000","last_undegraded":"2024-05-08T18:05:30.862900+0000","last_fullsized":"2024-05-08T18:05:30.862900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:27:29.461124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"29'14","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.333941+0000","last_change":"2024-05-08T18:04:53.844087+0000","last_active":"2024-05-08T18:06:53.333941+0000","last_peered":"2024-05-08T18:06:53.333941+0000","last_clean":"2024-05-08T18:06:53.333941+0000","last_became_active":"2024-05-08T18:04:53.843772+0000","last_became_peered":"2024-05-08T18:04:53.843772+0000","last_unstale":"2024-05-08T18:06:53.333941+0000","last_undegraded":"2024-05-08T18:06:53.333941+0000","last_fullsized":"2024-05-08T18:06:53.333941+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:26.083815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.738215+0000","last_change":"2024-05-08T18:04:45.811909+0000","last_active":"2024-05-08T18:06:48.738215+0000","last_peered":"2024-05-08T18:06:48.738215+0000","last_clean":"2024-05-08T18:06:48.738215+0000","last_became_active":"2024-05-08T18:04:45.811517+0000","last_became_peered":"2024-05-08T18:04:45.811517+0000","last_unstale":"2024-05-08T18:06:48.738215+0000","last_undegraded":"2024-05-08T18:06:48.738215+0000","last_fullsized":"2024-05-08T18:06:48.738215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:32.141179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860711+0000","last_change":"2024-05-08T18:04:43.836313+0000","last_active":"2024-05-08T18:05:30.860711+0000","last_peered":"2024-05-08T18:05:30.860711+0000","last_clean":"2024-05-08T18:05:30.860711+0000","last_became_active":"2024-05-08T18:04:43.834767+0000","last_became_peered":"2024-05-08T18:04:43.834767+0000","last_unstale":"2024-05-08T18:05:30.860711+0000","last_undegraded":"2024-05-08T18:05:30.860711+0000","last_fullsized":"2024-05-08T18:05:30.860711+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:22.454546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0,2],"acting":[1,3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867042+0000","last_change":"2024-05-08T18:04:47.821012+0000","last_active":"2024-05-08T18:05:30.867042+0000","last_peered":"2024-05-08T18:05:30.867042+0000","last_clean":"2024-05-08T18:05:30.867042+0000","last_became_active":"2024-05-08T18:04:47.820662+0000","last_became_peered":"2024-05-08T18:04:47.820662+0000","last_unstale":"2024-05-08T18:05:30.867042+0000","last_undegraded":"2024-05-08T18:05:30.867042+0000","last_fullsized":"2024-05-08T18:05:30.867042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:29:01.485039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"29'37","reported_seq":82,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.401221+0000","last_change":"2024-05-08T18:04:49.831871+0000","last_active":"2024-05-08T18:06:53.401221+0000","last_peered":"2024-05-08T18:06:53.401221+0000","last_clean":"2024-05-08T18:06:53.401221+0000","last_became_active":"2024-05-08T18:04:49.830476+0000","last_became_peered":"2024-05-08T18:04:49.830476+0000","last_unstale":"2024-05-08T18:06:53.401221+0000","last_undegraded":"2024-05-08T18:06:53.401221+0000","last_fullsized":"2024-05-08T18:06:53.401221+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:46.223733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862181+0000","last_change":"2024-05-08T18:04:51.845269+0000","last_active":"2024-05-08T18:05:30.862181+0000","last_peered":"2024-05-08T18:05:30.862181+0000","last_clean":"2024-05-08T18:05:30.862181+0000","last_became_active":"2024-05-08T18:04:51.844912+0000","last_became_peered":"2024-05-08T18:04:51.844912+0000","last_unstale":"2024-05-08T18:05:30.862181+0000","last_undegraded":"2024-05-08T18:05:30.862181+0000","last_fullsized":"2024-05-08T18:05:30.862181+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:17.567274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"29'2","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.037376+0000","last_change":"2024-05-08T18:04:53.847736+0000","last_active":"2024-05-08T18:07:54.037376+0000","last_peered":"2024-05-08T18:07:54.037376+0000","last_clean":"2024-05-08T18:07:54.037376+0000","last_became_active":"2024-05-08T18:04:53.847413+0000","last_became_peered":"2024-05-08T18:04:53.847413+0000","last_unstale":"2024-05-08T18:07:54.037376+0000","last_undegraded":"2024-05-08T18:07:54.037376+0000","last_fullsized":"2024-05-08T18:07:54.037376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:18.059582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"29'12","reported_seq":60,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.041935+0000","last_change":"2024-05-08T18:04:45.830619+0000","last_active":"2024-05-08T18:07:54.041935+0000","last_peered":"2024-05-08T18:07:54.041935+0000","last_clean":"2024-05-08T18:07:54.041935+0000","last_became_active":"2024-05-08T18:04:45.830329+0000","last_became_peered":"2024-05-08T18:04:45.830329+0000","last_unstale":"2024-05-08T18:07:54.041935+0000","last_undegraded":"2024-05-08T18:07:54.041935+0000","last_fullsized":"2024-05-08T18:07:54.041935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:13:12.748032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861213+0000","last_change":"2024-05-08T18:04:43.835662+0000","last_active":"2024-05-08T18:05:30.861213+0000","last_peered":"2024-05-08T18:05:30.861213+0000","last_clean":"2024-05-08T18:05:30.861213+0000","last_became_active":"2024-05-08T18:04:43.835227+0000","last_became_peered":"2024-05-08T18:04:43.835227+0000","last_unstale":"2024-05-08T18:05:30.861213+0000","last_undegraded":"2024-05-08T18:05:30.861213+0000","last_fullsized":"2024-05-08T18:05:30.861213+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:10.360847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3,2],"acting":[1,0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.871559+0000","last_change":"2024-05-08T18:05:30.870636+0000","last_active":"2024-05-08T18:05:30.871559+0000","last_peered":"2024-05-08T18:05:30.871559+0000","last_clean":"2024-05-08T18:05:30.871559+0000","last_became_active":"2024-05-08T18:05:30.870155+0000","last_became_peered":"2024-05-08T18:05:30.870155+0000","last_unstale":"2024-05-08T18:05:30.871559+0000","last_undegraded":"2024-05-08T18:05:30.871559+0000","last_fullsized":"2024-05-08T18:05:30.871559+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:56.207001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"29'32","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.962760+0000","last_change":"2024-05-08T18:04:49.832968+0000","last_active":"2024-05-08T18:06:51.962760+0000","last_peered":"2024-05-08T18:06:51.962760+0000","last_clean":"2024-05-08T18:06:51.962760+0000","last_became_active":"2024-05-08T18:04:49.832649+0000","last_became_peered":"2024-05-08T18:04:49.832649+0000","last_unstale":"2024-05-08T18:06:51.962760+0000","last_undegraded":"2024-05-08T18:06:51.962760+0000","last_fullsized":"2024-05-08T18:06:51.962760+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:52.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859450+0000","last_change":"2024-05-08T18:04:51.833712+0000","last_active":"2024-05-08T18:05:30.859450+0000","last_peered":"2024-05-08T18:05:30.859450+0000","last_clean":"2024-05-08T18:05:30.859450+0000","last_became_active":"2024-05-08T18:04:51.833421+0000","last_became_peered":"2024-05-08T18:04:51.833421+0000","last_unstale":"2024-05-08T18:05:30.859450+0000","last_undegraded":"2024-05-08T18:05:30.859450+0000","last_fullsized":"2024-05-08T18:05:30.859450+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:48.752288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"29'5","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.394775+0000","last_change":"2024-05-08T18:04:53.848762+0000","last_active":"2024-05-08T18:06:53.394775+0000","last_peered":"2024-05-08T18:06:53.394775+0000","last_clean":"2024-05-08T18:06:53.394775+0000","last_became_active":"2024-05-08T18:04:53.848430+0000","last_became_peered":"2024-05-08T18:04:53.848430+0000","last_unstale":"2024-05-08T18:06:53.394775+0000","last_undegraded":"2024-05-08T18:06:53.394775+0000","last_fullsized":"2024-05-08T18:06:53.394775+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:48.070547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"29'10","reported_seq":66,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.218980+0000","last_change":"2024-05-08T18:04:45.807312+0000","last_active":"2024-05-08T18:06:50.218980+0000","last_peered":"2024-05-08T18:06:50.218980+0000","last_clean":"2024-05-08T18:06:50.218980+0000","last_became_active":"2024-05-08T18:04:45.806808+0000","last_became_peered":"2024-05-08T18:04:45.806808+0000","last_unstale":"2024-05-08T18:06:50.218980+0000","last_undegraded":"2024-05-08T18:06:50.218980+0000","last_fullsized":"2024-05-08T18:06:50.218980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:27:20.114031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859441+0000","last_change":"2024-05-08T18:04:43.833094+0000","last_active":"2024-05-08T18:05:30.859441+0000","last_peered":"2024-05-08T18:05:30.859441+0000","last_clean":"2024-05-08T18:05:30.859441+0000","last_became_active":"2024-05-08T18:04:43.832147+0000","last_became_peered":"2024-05-08T18:04:43.832147+0000","last_unstale":"2024-05-08T18:05:30.859441+0000","last_undegraded":"2024-05-08T18:05:30.859441+0000","last_fullsized":"2024-05-08T18:05:30.859441+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:49.361708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862397+0000","last_change":"2024-05-08T18:04:47.815796+0000","last_active":"2024-05-08T18:05:30.862397+0000","last_peered":"2024-05-08T18:05:30.862397+0000","last_clean":"2024-05-08T18:05:30.862397+0000","last_became_active":"2024-05-08T18:04:47.815219+0000","last_became_peered":"2024-05-08T18:04:47.815219+0000","last_unstale":"2024-05-08T18:05:30.862397+0000","last_undegraded":"2024-05-08T18:05:30.862397+0000","last_fullsized":"2024-05-08T18:05:30.862397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:23:20.608088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"27'10","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861853+0000","last_change":"2024-05-08T18:04:49.833674+0000","last_active":"2024-05-08T18:05:30.861853+0000","last_peered":"2024-05-08T18:05:30.861853+0000","last_clean":"2024-05-08T18:05:30.861853+0000","last_became_active":"2024-05-08T18:04:49.830793+0000","last_became_peered":"2024-05-08T18:04:49.830793+0000","last_unstale":"2024-05-08T18:05:30.861853+0000","last_undegraded":"2024-05-08T18:05:30.861853+0000","last_fullsized":"2024-05-08T18:05:30.861853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:54:53.163574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"29'282","reported_seq":366,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903209+0000","last_change":"2024-05-08T18:04:51.834224+0000","last_active":"2024-05-08T18:07:52.903209+0000","last_peered":"2024-05-08T18:07:52.903209+0000","last_clean":"2024-05-08T18:07:52.903209+0000","last_became_active":"2024-05-08T18:04:51.833866+0000","last_became_peered":"2024-05-08T18:04:51.833866+0000","last_unstale":"2024-05-08T18:07:52.903209+0000","last_undegraded":"2024-05-08T18:07:52.903209+0000","last_fullsized":"2024-05-08T18:07:52.903209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:21:20.452363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.856865+0000","last_change":"2024-05-08T18:04:53.848804+0000","last_active":"2024-05-08T18:05:30.856865+0000","last_peered":"2024-05-08T18:05:30.856865+0000","last_clean":"2024-05-08T18:05:30.856865+0000","last_became_active":"2024-05-08T18:04:53.848321+0000","last_became_peered":"2024-05-08T18:04:53.848321+0000","last_unstale":"2024-05-08T18:05:30.856865+0000","last_undegraded":"2024-05-08T18:05:30.856865+0000","last_fullsized":"2024-05-08T18:05:30.856865+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:34.662852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"29'4","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399620+0000","last_change":"2024-05-08T18:04:45.814018+0000","last_active":"2024-05-08T18:06:53.399620+0000","last_peered":"2024-05-08T18:06:53.399620+0000","last_clean":"2024-05-08T18:06:53.399620+0000","last_became_active":"2024-05-08T18:04:45.813424+0000","last_became_peered":"2024-05-08T18:04:45.813424+0000","last_unstale":"2024-05-08T18:06:53.399620+0000","last_undegraded":"2024-05-08T18:06:53.399620+0000","last_fullsized":"2024-05-08T18:06:53.399620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:42:05.254786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866726+0000","last_change":"2024-05-08T18:04:43.841603+0000","last_active":"2024-05-08T18:05:30.866726+0000","last_peered":"2024-05-08T18:05:30.866726+0000","last_clean":"2024-05-08T18:05:30.866726+0000","last_became_active":"2024-05-08T18:04:43.840502+0000","last_became_peered":"2024-05-08T18:04:43.840502+0000","last_unstale":"2024-05-08T18:05:30.866726+0000","last_undegraded":"2024-05-08T18:05:30.866726+0000","last_fullsized":"2024-05-08T18:05:30.866726+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:40:40.407240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860986+0000","last_change":"2024-05-08T18:04:47.819165+0000","last_active":"2024-05-08T18:05:30.860986+0000","last_peered":"2024-05-08T18:05:30.860986+0000","last_clean":"2024-05-08T18:05:30.860986+0000","last_became_active":"2024-05-08T18:04:47.818839+0000","last_became_peered":"2024-05-08T18:04:47.818839+0000","last_unstale":"2024-05-08T18:05:30.860986+0000","last_undegraded":"2024-05-08T18:05:30.860986+0000","last_fullsized":"2024-05-08T18:05:30.860986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:04:15.231714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"29'37","reported_seq":82,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.936372+0000","last_change":"2024-05-08T18:04:49.824708+0000","last_active":"2024-05-08T18:06:51.936372+0000","last_peered":"2024-05-08T18:06:51.936372+0000","last_clean":"2024-05-08T18:06:51.936372+0000","last_became_active":"2024-05-08T18:04:49.824405+0000","last_became_peered":"2024-05-08T18:04:49.824405+0000","last_unstale":"2024-05-08T18:06:51.936372+0000","last_undegraded":"2024-05-08T18:06:51.936372+0000","last_fullsized":"2024-05-08T18:06:51.936372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:52.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858238+0000","last_change":"2024-05-08T18:04:51.845116+0000","last_active":"2024-05-08T18:05:30.858238+0000","last_peered":"2024-05-08T18:05:30.858238+0000","last_clean":"2024-05-08T18:05:30.858238+0000","last_became_active":"2024-05-08T18:04:51.844757+0000","last_became_peered":"2024-05-08T18:04:51.844757+0000","last_unstale":"2024-05-08T18:05:30.858238+0000","last_undegraded":"2024-05-08T18:05:30.858238+0000","last_fullsized":"2024-05-08T18:05:30.858238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:17.567274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866299+0000","last_change":"2024-05-08T18:04:53.841236+0000","last_active":"2024-05-08T18:05:30.866299+0000","last_peered":"2024-05-08T18:05:30.866299+0000","last_clean":"2024-05-08T18:05:30.866299+0000","last_became_active":"2024-05-08T18:04:53.840956+0000","last_became_peered":"2024-05-08T18:04:53.840956+0000","last_unstale":"2024-05-08T18:05:30.866299+0000","last_undegraded":"2024-05-08T18:05:30.866299+0000","last_fullsized":"2024-05-08T18:05:30.866299+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:58.125427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"29'3","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.866543+0000","last_change":"2024-05-08T18:04:45.807309+0000","last_active":"2024-05-08T18:07:53.866543+0000","last_peered":"2024-05-08T18:07:53.866543+0000","last_clean":"2024-05-08T18:07:53.866543+0000","last_became_active":"2024-05-08T18:04:45.806560+0000","last_became_peered":"2024-05-08T18:04:45.806560+0000","last_unstale":"2024-05-08T18:07:53.866543+0000","last_undegraded":"2024-05-08T18:07:53.866543+0000","last_fullsized":"2024-05-08T18:07:53.866543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:11:17.190072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861114+0000","last_change":"2024-05-08T18:04:43.845626+0000","last_active":"2024-05-08T18:05:30.861114+0000","last_peered":"2024-05-08T18:05:30.861114+0000","last_clean":"2024-05-08T18:05:30.861114+0000","last_became_active":"2024-05-08T18:04:43.845214+0000","last_became_peered":"2024-05-08T18:04:43.845214+0000","last_unstale":"2024-05-08T18:05:30.861114+0000","last_undegraded":"2024-05-08T18:05:30.861114+0000","last_fullsized":"2024-05-08T18:05:30.861114+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:00.592659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.865365+0000","last_change":"2024-05-08T18:04:47.823059+0000","last_active":"2024-05-08T18:05:30.865365+0000","last_peered":"2024-05-08T18:05:30.865365+0000","last_clean":"2024-05-08T18:05:30.865365+0000","last_became_active":"2024-05-08T18:04:47.822748+0000","last_became_peered":"2024-05-08T18:04:47.822748+0000","last_unstale":"2024-05-08T18:05:30.865365+0000","last_undegraded":"2024-05-08T18:05:30.865365+0000","last_fullsized":"2024-05-08T18:05:30.865365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:20.980114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"29'31","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.937310+0000","last_change":"2024-05-08T18:04:49.831735+0000","last_active":"2024-05-08T18:06:51.937310+0000","last_peered":"2024-05-08T18:06:51.937310+0000","last_clean":"2024-05-08T18:06:51.937310+0000","last_became_active":"2024-05-08T18:04:49.831440+0000","last_became_peered":"2024-05-08T18:04:49.831440+0000","last_unstale":"2024-05-08T18:06:51.937310+0000","last_undegraded":"2024-05-08T18:06:51.937310+0000","last_fullsized":"2024-05-08T18:06:51.937310+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:40.102877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857159+0000","last_change":"2024-05-08T18:04:51.844443+0000","last_active":"2024-05-08T18:05:30.857159+0000","last_peered":"2024-05-08T18:05:30.857159+0000","last_clean":"2024-05-08T18:05:30.857159+0000","last_became_active":"2024-05-08T18:04:51.844125+0000","last_became_peered":"2024-05-08T18:04:51.844125+0000","last_unstale":"2024-05-08T18:05:30.857159+0000","last_undegraded":"2024-05-08T18:05:30.857159+0000","last_fullsized":"2024-05-08T18:05:30.857159+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:22.017553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862839+0000","last_change":"2024-05-08T18:04:53.849962+0000","last_active":"2024-05-08T18:05:30.862839+0000","last_peered":"2024-05-08T18:05:30.862839+0000","last_clean":"2024-05-08T18:05:30.862839+0000","last_became_active":"2024-05-08T18:04:53.849514+0000","last_became_peered":"2024-05-08T18:04:53.849514+0000","last_unstale":"2024-05-08T18:05:30.862839+0000","last_undegraded":"2024-05-08T18:05:30.862839+0000","last_fullsized":"2024-05-08T18:05:30.862839+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:31.875101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"29'4","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399430+0000","last_change":"2024-05-08T18:04:45.822495+0000","last_active":"2024-05-08T18:06:53.399430+0000","last_peered":"2024-05-08T18:06:53.399430+0000","last_clean":"2024-05-08T18:06:53.399430+0000","last_became_active":"2024-05-08T18:04:45.822204+0000","last_became_peered":"2024-05-08T18:04:45.822204+0000","last_unstale":"2024-05-08T18:06:53.399430+0000","last_undegraded":"2024-05-08T18:06:53.399430+0000","last_fullsized":"2024-05-08T18:06:53.399430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:06.207449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.058832+0000","last_change":"2024-05-08T18:04:43.835654+0000","last_active":"2024-05-08T18:07:54.058832+0000","last_peered":"2024-05-08T18:07:54.058832+0000","last_clean":"2024-05-08T18:07:54.058832+0000","last_became_active":"2024-05-08T18:04:43.834678+0000","last_became_peered":"2024-05-08T18:04:43.834678+0000","last_unstale":"2024-05-08T18:07:54.058832+0000","last_undegraded":"2024-05-08T18:07:54.058832+0000","last_fullsized":"2024-05-08T18:07:54.058832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:23:16.295599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859176+0000","last_change":"2024-05-08T18:04:47.820376+0000","last_active":"2024-05-08T18:05:30.859176+0000","last_peered":"2024-05-08T18:05:30.859176+0000","last_clean":"2024-05-08T18:05:30.859176+0000","last_became_active":"2024-05-08T18:04:47.820036+0000","last_became_peered":"2024-05-08T18:04:47.820036+0000","last_unstale":"2024-05-08T18:05:30.859176+0000","last_undegraded":"2024-05-08T18:05:30.859176+0000","last_fullsized":"2024-05-08T18:05:30.859176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:10:33.041988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"29'28","reported_seq":68,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.960416+0000","last_change":"2024-05-08T18:04:49.832032+0000","last_active":"2024-05-08T18:06:51.960416+0000","last_peered":"2024-05-08T18:06:51.960416+0000","last_clean":"2024-05-08T18:06:51.960416+0000","last_became_active":"2024-05-08T18:04:49.830882+0000","last_became_peered":"2024-05-08T18:04:49.830882+0000","last_unstale":"2024-05-08T18:06:51.960416+0000","last_undegraded":"2024-05-08T18:06:51.960416+0000","last_fullsized":"2024-05-08T18:06:51.960416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:21.475550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":40,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857634+0000","last_change":"2024-05-08T18:04:51.844109+0000","last_active":"2024-05-08T18:05:30.857634+0000","last_peered":"2024-05-08T18:05:30.857634+0000","last_clean":"2024-05-08T18:05:30.857634+0000","last_became_active":"2024-05-08T18:04:51.842734+0000","last_became_peered":"2024-05-08T18:04:51.842734+0000","last_unstale":"2024-05-08T18:05:30.857634+0000","last_undegraded":"2024-05-08T18:05:30.857634+0000","last_fullsized":"2024-05-08T18:05:30.857634+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:47.953187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'10","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.495078+0000","last_change":"2024-05-08T18:04:53.845139+0000","last_active":"2024-05-08T18:07:53.495078+0000","last_peered":"2024-05-08T18:07:53.495078+0000","last_clean":"2024-05-08T18:07:53.495078+0000","last_became_active":"2024-05-08T18:04:53.844891+0000","last_became_peered":"2024-05-08T18:04:53.844891+0000","last_unstale":"2024-05-08T18:07:53.495078+0000","last_undegraded":"2024-05-08T18:07:53.495078+0000","last_fullsized":"2024-05-08T18:07:53.495078+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:32.203551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"29'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.860618+0000","last_change":"2024-05-08T18:04:45.809216+0000","last_active":"2024-05-08T18:07:53.860618+0000","last_peered":"2024-05-08T18:07:53.860618+0000","last_clean":"2024-05-08T18:07:53.860618+0000","last_became_active":"2024-05-08T18:04:45.808936+0000","last_became_peered":"2024-05-08T18:04:45.808936+0000","last_unstale":"2024-05-08T18:07:53.860618+0000","last_undegraded":"2024-05-08T18:07:53.860618+0000","last_fullsized":"2024-05-08T18:07:53.860618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:51.368967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866160+0000","last_change":"2024-05-08T18:04:43.837820+0000","last_active":"2024-05-08T18:05:30.866160+0000","last_peered":"2024-05-08T18:05:30.866160+0000","last_clean":"2024-05-08T18:05:30.866160+0000","last_became_active":"2024-05-08T18:04:43.837070+0000","last_became_peered":"2024-05-08T18:04:43.837070+0000","last_unstale":"2024-05-08T18:05:30.866160+0000","last_undegraded":"2024-05-08T18:05:30.866160+0000","last_fullsized":"2024-05-08T18:05:30.866160+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:50.043665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"29'2","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.565208+0000","last_change":"2024-05-08T18:04:47.824489+0000","last_active":"2024-05-08T18:06:50.565208+0000","last_peered":"2024-05-08T18:06:50.565208+0000","last_clean":"2024-05-08T18:06:50.565208+0000","last_became_active":"2024-05-08T18:04:47.824172+0000","last_became_peered":"2024-05-08T18:04:47.824172+0000","last_unstale":"2024-05-08T18:06:50.565208+0000","last_undegraded":"2024-05-08T18:06:50.565208+0000","last_fullsized":"2024-05-08T18:06:50.565208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:14.408614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"29'32","reported_seq":76,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.418071+0000","last_change":"2024-05-08T18:04:49.832422+0000","last_active":"2024-05-08T18:07:53.418071+0000","last_peered":"2024-05-08T18:07:53.418071+0000","last_clean":"2024-05-08T18:07:53.418071+0000","last_became_active":"2024-05-08T18:04:49.831803+0000","last_became_peered":"2024-05-08T18:04:49.831803+0000","last_unstale":"2024-05-08T18:07:53.418071+0000","last_undegraded":"2024-05-08T18:07:53.418071+0000","last_fullsized":"2024-05-08T18:07:53.418071+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:03:53.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858576+0000","last_change":"2024-05-08T18:04:51.846422+0000","last_active":"2024-05-08T18:05:30.858576+0000","last_peered":"2024-05-08T18:05:30.858576+0000","last_clean":"2024-05-08T18:05:30.858576+0000","last_became_active":"2024-05-08T18:04:51.846225+0000","last_became_peered":"2024-05-08T18:04:51.846225+0000","last_unstale":"2024-05-08T18:05:30.858576+0000","last_undegraded":"2024-05-08T18:05:30.858576+0000","last_fullsized":"2024-05-08T18:05:30.858576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:42:25.975522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"29'4","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.972855+0000","last_change":"2024-05-08T18:04:53.843030+0000","last_active":"2024-05-08T18:06:51.972855+0000","last_peered":"2024-05-08T18:06:51.972855+0000","last_clean":"2024-05-08T18:06:51.972855+0000","last_became_active":"2024-05-08T18:04:53.842429+0000","last_became_peered":"2024-05-08T18:04:53.842429+0000","last_unstale":"2024-05-08T18:06:51.972855+0000","last_undegraded":"2024-05-08T18:06:51.972855+0000","last_fullsized":"2024-05-08T18:06:51.972855+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:30.303540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"27'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866744+0000","last_change":"2024-05-08T18:04:45.818828+0000","last_active":"2024-05-08T18:05:30.866744+0000","last_peered":"2024-05-08T18:05:30.866744+0000","last_clean":"2024-05-08T18:05:30.866744+0000","last_became_active":"2024-05-08T18:04:45.818161+0000","last_became_peered":"2024-05-08T18:04:45.818161+0000","last_unstale":"2024-05-08T18:05:30.866744+0000","last_undegraded":"2024-05-08T18:05:30.866744+0000","last_fullsized":"2024-05-08T18:05:30.866744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:59.477939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859517+0000","last_change":"2024-05-08T18:04:43.843055+0000","last_active":"2024-05-08T18:05:30.859517+0000","last_peered":"2024-05-08T18:05:30.859517+0000","last_clean":"2024-05-08T18:05:30.859517+0000","last_became_active":"2024-05-08T18:04:43.842483+0000","last_became_peered":"2024-05-08T18:04:43.842483+0000","last_unstale":"2024-05-08T18:05:30.859517+0000","last_undegraded":"2024-05-08T18:05:30.859517+0000","last_fullsized":"2024-05-08T18:05:30.859517+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:11.225102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"29'36","reported_seq":106,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995781+0000","last_change":"2024-05-08T18:04:45.808186+0000","last_active":"2024-05-08T18:07:53.995781+0000","last_peered":"2024-05-08T18:07:53.995781+0000","last_clean":"2024-05-08T18:07:53.995781+0000","last_became_active":"2024-05-08T18:04:45.807847+0000","last_became_peered":"2024-05-08T18:04:45.807847+0000","last_unstale":"2024-05-08T18:07:53.995781+0000","last_undegraded":"2024-05-08T18:07:53.995781+0000","last_fullsized":"2024-05-08T18:07:53.995781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:00:26.846485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":44,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858145+0000","last_change":"2024-05-08T18:04:43.836530+0000","last_active":"2024-05-08T18:05:30.858145+0000","last_peered":"2024-05-08T18:05:30.858145+0000","last_clean":"2024-05-08T18:05:30.858145+0000","last_became_active":"2024-05-08T18:04:43.836045+0000","last_became_peered":"2024-05-08T18:04:43.836045+0000","last_unstale":"2024-05-08T18:05:30.858145+0000","last_undegraded":"2024-05-08T18:05:30.858145+0000","last_fullsized":"2024-05-08T18:05:30.858145+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:22.454546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3,2],"acting":[0,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"29'11","reported_seq":68,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.782514+0000","last_change":"2024-05-08T18:04:45.826393+0000","last_active":"2024-05-08T18:07:53.782514+0000","last_peered":"2024-05-08T18:07:53.782514+0000","last_clean":"2024-05-08T18:07:53.782514+0000","last_became_active":"2024-05-08T18:04:45.825506+0000","last_became_peered":"2024-05-08T18:04:45.825506+0000","last_unstale":"2024-05-08T18:07:53.782514+0000","last_undegraded":"2024-05-08T18:07:53.782514+0000","last_fullsized":"2024-05-08T18:07:53.782514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:23:18.600988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":46,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"29'5","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.990901+0000","last_change":"2024-05-08T18:04:43.832854+0000","last_active":"2024-05-08T18:07:53.990901+0000","last_peered":"2024-05-08T18:07:53.990901+0000","last_clean":"2024-05-08T18:07:53.990901+0000","last_became_active":"2024-05-08T18:04:43.832406+0000","last_became_peered":"2024-05-08T18:04:43.832406+0000","last_unstale":"2024-05-08T18:07:53.990901+0000","last_undegraded":"2024-05-08T18:07:53.990901+0000","last_fullsized":"2024-05-08T18:07:53.990901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:16.456364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":29,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"29'2","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.774175+0000","last_change":"2024-05-08T18:04:45.807187+0000","last_active":"2024-05-08T18:07:53.774175+0000","last_peered":"2024-05-08T18:07:53.774175+0000","last_clean":"2024-05-08T18:07:53.774175+0000","last_became_active":"2024-05-08T18:04:45.806646+0000","last_became_peered":"2024-05-08T18:04:45.806646+0000","last_unstale":"2024-05-08T18:07:53.774175+0000","last_undegraded":"2024-05-08T18:07:53.774175+0000","last_fullsized":"2024-05-08T18:07:53.774175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:30.458103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"29'5","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.374758+0000","last_change":"2024-05-08T18:04:43.836158+0000","last_active":"2024-05-08T18:06:53.374758+0000","last_peered":"2024-05-08T18:06:53.374758+0000","last_clean":"2024-05-08T18:06:53.374758+0000","last_became_active":"2024-05-08T18:04:43.835163+0000","last_became_peered":"2024-05-08T18:04:43.835163+0000","last_unstale":"2024-05-08T18:06:53.374758+0000","last_undegraded":"2024-05-08T18:06:53.374758+0000","last_fullsized":"2024-05-08T18:06:53.374758+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:55.493870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":13,"num_write_kb":4746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,1],"acting":[0,3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"29'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.861341+0000","last_change":"2024-05-08T18:04:45.816663+0000","last_active":"2024-05-08T18:07:53.861341+0000","last_peered":"2024-05-08T18:07:53.861341+0000","last_clean":"2024-05-08T18:07:53.861341+0000","last_became_active":"2024-05-08T18:04:45.816391+0000","last_became_peered":"2024-05-08T18:04:45.816391+0000","last_unstale":"2024-05-08T18:07:53.861341+0000","last_undegraded":"2024-05-08T18:07:53.861341+0000","last_fullsized":"2024-05-08T18:07:53.861341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:02.599918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'4","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861292+0000","last_change":"2024-05-08T18:04:43.836083+0000","last_active":"2024-05-08T18:05:30.861292+0000","last_peered":"2024-05-08T18:05:30.861292+0000","last_clean":"2024-05-08T18:05:30.861292+0000","last_became_active":"2024-05-08T18:04:43.835268+0000","last_became_peered":"2024-05-08T18:04:43.835268+0000","last_unstale":"2024-05-08T18:05:30.861292+0000","last_undegraded":"2024-05-08T18:05:30.861292+0000","last_fullsized":"2024-05-08T18:05:30.861292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:47.971014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"29'12","reported_seq":77,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.035458+0000","last_change":"2024-05-08T18:04:45.820787+0000","last_active":"2024-05-08T18:07:54.035458+0000","last_peered":"2024-05-08T18:07:54.035458+0000","last_clean":"2024-05-08T18:07:54.035458+0000","last_became_active":"2024-05-08T18:04:45.819699+0000","last_became_peered":"2024-05-08T18:04:45.819699+0000","last_unstale":"2024-05-08T18:07:54.035458+0000","last_undegraded":"2024-05-08T18:07:54.035458+0000","last_fullsized":"2024-05-08T18:07:54.035458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:18.539896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862510+0000","last_change":"2024-05-08T18:04:43.837155+0000","last_active":"2024-05-08T18:05:30.862510+0000","last_peered":"2024-05-08T18:05:30.862510+0000","last_clean":"2024-05-08T18:05:30.862510+0000","last_became_active":"2024-05-08T18:04:43.836735+0000","last_became_peered":"2024-05-08T18:04:43.836735+0000","last_unstale":"2024-05-08T18:05:30.862510+0000","last_undegraded":"2024-05-08T18:05:30.862510+0000","last_fullsized":"2024-05-08T18:05:30.862510+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:54.711833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"29'14","reported_seq":70,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.219716+0000","last_change":"2024-05-08T18:04:45.812327+0000","last_active":"2024-05-08T18:06:50.219716+0000","last_peered":"2024-05-08T18:06:50.219716+0000","last_clean":"2024-05-08T18:06:50.219716+0000","last_became_active":"2024-05-08T18:04:45.811953+0000","last_became_peered":"2024-05-08T18:04:45.811953+0000","last_unstale":"2024-05-08T18:06:50.219716+0000","last_undegraded":"2024-05-08T18:06:50.219716+0000","last_fullsized":"2024-05-08T18:06:50.219716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:11:17.190072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867588+0000","last_change":"2024-05-08T18:04:43.843298+0000","last_active":"2024-05-08T18:05:30.867588+0000","last_peered":"2024-05-08T18:05:30.867588+0000","last_clean":"2024-05-08T18:05:30.867588+0000","last_became_active":"2024-05-08T18:04:43.842179+0000","last_became_peered":"2024-05-08T18:04:43.842179+0000","last_unstale":"2024-05-08T18:05:30.867588+0000","last_undegraded":"2024-05-08T18:05:30.867588+0000","last_fullsized":"2024-05-08T18:05:30.867588+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:55.493870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,0],"acting":[2,3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"29'8","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063691+0000","last_change":"2024-05-08T18:04:45.812893+0000","last_active":"2024-05-08T18:07:54.063691+0000","last_peered":"2024-05-08T18:07:54.063691+0000","last_clean":"2024-05-08T18:07:54.063691+0000","last_became_active":"2024-05-08T18:04:45.812637+0000","last_became_peered":"2024-05-08T18:04:45.812637+0000","last_unstale":"2024-05-08T18:07:54.063691+0000","last_undegraded":"2024-05-08T18:07:54.063691+0000","last_fullsized":"2024-05-08T18:07:54.063691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:18.973014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"29'6","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.981626+0000","last_change":"2024-05-08T18:04:43.834951+0000","last_active":"2024-05-08T18:07:53.981626+0000","last_peered":"2024-05-08T18:07:53.981626+0000","last_clean":"2024-05-08T18:07:53.981626+0000","last_became_active":"2024-05-08T18:04:43.834491+0000","last_became_peered":"2024-05-08T18:04:43.834491+0000","last_unstale":"2024-05-08T18:07:53.981626+0000","last_undegraded":"2024-05-08T18:07:53.981626+0000","last_fullsized":"2024-05-08T18:07:53.981626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:24:46.108691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"29'8","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.041264+0000","last_change":"2024-05-08T18:04:45.815948+0000","last_active":"2024-05-08T18:07:54.041264+0000","last_peered":"2024-05-08T18:07:54.041264+0000","last_clean":"2024-05-08T18:07:54.041264+0000","last_became_active":"2024-05-08T18:04:45.815677+0000","last_became_peered":"2024-05-08T18:04:45.815677+0000","last_unstale":"2024-05-08T18:07:54.041264+0000","last_undegraded":"2024-05-08T18:07:54.041264+0000","last_fullsized":"2024-05-08T18:07:54.041264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:16.870297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857605+0000","last_change":"2024-05-08T18:04:43.846054+0000","last_active":"2024-05-08T18:05:30.857605+0000","last_peered":"2024-05-08T18:05:30.857605+0000","last_clean":"2024-05-08T18:05:30.857605+0000","last_became_active":"2024-05-08T18:04:43.843826+0000","last_became_peered":"2024-05-08T18:04:43.843826+0000","last_unstale":"2024-05-08T18:05:30.857605+0000","last_undegraded":"2024-05-08T18:05:30.857605+0000","last_fullsized":"2024-05-08T18:05:30.857605+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:08:10.712708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399417+0000","last_change":"2024-05-08T18:04:45.827535+0000","last_active":"2024-05-08T18:06:53.399417+0000","last_peered":"2024-05-08T18:06:53.399417+0000","last_clean":"2024-05-08T18:06:53.399417+0000","last_became_active":"2024-05-08T18:04:45.827252+0000","last_became_peered":"2024-05-08T18:04:45.827252+0000","last_unstale":"2024-05-08T18:06:53.399417+0000","last_undegraded":"2024-05-08T18:06:53.399417+0000","last_fullsized":"2024-05-08T18:06:53.399417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:07.190532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858630+0000","last_change":"2024-05-08T18:04:43.820995+0000","last_active":"2024-05-08T18:05:30.858630+0000","last_peered":"2024-05-08T18:05:30.858630+0000","last_clean":"2024-05-08T18:05:30.858630+0000","last_became_active":"2024-05-08T18:04:43.820360+0000","last_became_peered":"2024-05-08T18:04:43.820360+0000","last_unstale":"2024-05-08T18:05:30.858630+0000","last_undegraded":"2024-05-08T18:05:30.858630+0000","last_fullsized":"2024-05-08T18:05:30.858630+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:54.711833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860974+0000","last_change":"2024-05-08T18:04:45.823656+0000","last_active":"2024-05-08T18:05:30.860974+0000","last_peered":"2024-05-08T18:05:30.860974+0000","last_clean":"2024-05-08T18:05:30.860974+0000","last_became_active":"2024-05-08T18:04:45.823385+0000","last_became_peered":"2024-05-08T18:04:45.823385+0000","last_unstale":"2024-05-08T18:05:30.860974+0000","last_undegraded":"2024-05-08T18:05:30.860974+0000","last_fullsized":"2024-05-08T18:05:30.860974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:26.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'6","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.755149+0000","last_change":"2024-05-08T18:04:43.836532+0000","last_active":"2024-05-08T18:07:53.755149+0000","last_peered":"2024-05-08T18:07:53.755149+0000","last_clean":"2024-05-08T18:07:53.755149+0000","last_became_active":"2024-05-08T18:04:43.835838+0000","last_became_peered":"2024-05-08T18:04:43.835838+0000","last_unstale":"2024-05-08T18:07:53.755149+0000","last_undegraded":"2024-05-08T18:07:53.755149+0000","last_fullsized":"2024-05-08T18:07:53.755149+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:11.225102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8,"num_write":30,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"29'10","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995916+0000","last_change":"2024-05-08T18:04:45.810992+0000","last_active":"2024-05-08T18:07:53.995916+0000","last_peered":"2024-05-08T18:07:53.995916+0000","last_clean":"2024-05-08T18:07:53.995916+0000","last_became_active":"2024-05-08T18:04:45.810703+0000","last_became_peered":"2024-05-08T18:04:45.810703+0000","last_unstale":"2024-05-08T18:07:53.995916+0000","last_undegraded":"2024-05-08T18:07:53.995916+0000","last_fullsized":"2024-05-08T18:07:53.995916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:05.828431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"29'3","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744531+0000","last_change":"2024-05-08T18:04:43.836123+0000","last_active":"2024-05-08T18:06:58.744531+0000","last_peered":"2024-05-08T18:06:58.744531+0000","last_clean":"2024-05-08T18:06:58.744531+0000","last_became_active":"2024-05-08T18:04:43.835518+0000","last_became_peered":"2024-05-08T18:04:43.835518+0000","last_unstale":"2024-05-08T18:06:58.744531+0000","last_undegraded":"2024-05-08T18:06:58.744531+0000","last_fullsized":"2024-05-08T18:06:58.744531+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:06.338736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,3],"acting":[1,2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"29'4","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.219429+0000","last_change":"2024-05-08T18:04:45.821376+0000","last_active":"2024-05-08T18:06:50.219429+0000","last_peered":"2024-05-08T18:06:50.219429+0000","last_clean":"2024-05-08T18:06:50.219429+0000","last_became_active":"2024-05-08T18:04:45.820746+0000","last_became_peered":"2024-05-08T18:04:45.820746+0000","last_unstale":"2024-05-08T18:06:50.219429+0000","last_undegraded":"2024-05-08T18:06:50.219429+0000","last_fullsized":"2024-05-08T18:06:50.219429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:32.833248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859364+0000","last_change":"2024-05-08T18:04:43.847998+0000","last_active":"2024-05-08T18:05:30.859364+0000","last_peered":"2024-05-08T18:05:30.859364+0000","last_clean":"2024-05-08T18:05:30.859364+0000","last_became_active":"2024-05-08T18:04:43.844404+0000","last_became_peered":"2024-05-08T18:04:43.844404+0000","last_unstale":"2024-05-08T18:05:30.859364+0000","last_undegraded":"2024-05-08T18:05:30.859364+0000","last_fullsized":"2024-05-08T18:05:30.859364+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:28.450844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"29'4","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.400051+0000","last_change":"2024-05-08T18:04:45.809582+0000","last_active":"2024-05-08T18:06:53.400051+0000","last_peered":"2024-05-08T18:06:53.400051+0000","last_clean":"2024-05-08T18:06:53.400051+0000","last_became_active":"2024-05-08T18:04:45.809271+0000","last_became_peered":"2024-05-08T18:04:45.809271+0000","last_unstale":"2024-05-08T18:06:53.400051+0000","last_undegraded":"2024-05-08T18:06:53.400051+0000","last_fullsized":"2024-05-08T18:06:53.400051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:32.141179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861152+0000","last_change":"2024-05-08T18:04:43.844366+0000","last_active":"2024-05-08T18:05:30.861152+0000","last_peered":"2024-05-08T18:05:30.861152+0000","last_clean":"2024-05-08T18:05:30.861152+0000","last_became_active":"2024-05-08T18:04:43.838125+0000","last_became_peered":"2024-05-08T18:04:43.838125+0000","last_unstale":"2024-05-08T18:05:30.861152+0000","last_undegraded":"2024-05-08T18:05:30.861152+0000","last_fullsized":"2024-05-08T18:05:30.861152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:24.149010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"29'15","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070567+0000","last_change":"2024-05-08T18:04:45.828761+0000","last_active":"2024-05-08T18:07:54.070567+0000","last_peered":"2024-05-08T18:07:54.070567+0000","last_clean":"2024-05-08T18:07:54.070567+0000","last_became_active":"2024-05-08T18:04:45.828488+0000","last_became_peered":"2024-05-08T18:04:45.828488+0000","last_unstale":"2024-05-08T18:07:54.070567+0000","last_undegraded":"2024-05-08T18:07:54.070567+0000","last_fullsized":"2024-05-08T18:07:54.070567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:27:23.439814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867753+0000","last_change":"2024-05-08T18:04:43.843455+0000","last_active":"2024-05-08T18:05:30.867753+0000","last_peered":"2024-05-08T18:05:30.867753+0000","last_clean":"2024-05-08T18:05:30.867753+0000","last_became_active":"2024-05-08T18:04:43.842670+0000","last_became_peered":"2024-05-08T18:04:43.842670+0000","last_unstale":"2024-05-08T18:05:30.867753+0000","last_undegraded":"2024-05-08T18:05:30.867753+0000","last_fullsized":"2024-05-08T18:05:30.867753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:01:49.620924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"29'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.547055+0000","last_change":"2024-05-08T18:04:45.812074+0000","last_active":"2024-05-08T18:06:09.547055+0000","last_peered":"2024-05-08T18:06:09.547055+0000","last_clean":"2024-05-08T18:06:09.547055+0000","last_became_active":"2024-05-08T18:04:45.810562+0000","last_became_peered":"2024-05-08T18:04:45.810562+0000","last_unstale":"2024-05-08T18:06:09.547055+0000","last_undegraded":"2024-05-08T18:06:09.547055+0000","last_fullsized":"2024-05-08T18:06:09.547055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:58.784190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864727+0000","last_change":"2024-05-08T18:04:43.838746+0000","last_active":"2024-05-08T18:05:30.864727+0000","last_peered":"2024-05-08T18:05:30.864727+0000","last_clean":"2024-05-08T18:05:30.864727+0000","last_became_active":"2024-05-08T18:04:43.835943+0000","last_became_peered":"2024-05-08T18:04:43.835943+0000","last_unstale":"2024-05-08T18:05:30.864727+0000","last_undegraded":"2024-05-08T18:05:30.864727+0000","last_fullsized":"2024-05-08T18:05:30.864727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:27:18.106771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857870+0000","last_change":"2024-05-08T18:04:43.836378+0000","last_active":"2024-05-08T18:05:30.857870+0000","last_peered":"2024-05-08T18:05:30.857870+0000","last_clean":"2024-05-08T18:05:30.857870+0000","last_became_active":"2024-05-08T18:04:43.836007+0000","last_became_peered":"2024-05-08T18:04:43.836007+0000","last_unstale":"2024-05-08T18:05:30.857870+0000","last_undegraded":"2024-05-08T18:05:30.857870+0000","last_fullsized":"2024-05-08T18:05:30.857870+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:57:00.868492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"29'8","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.219707+0000","last_change":"2024-05-08T18:04:45.812333+0000","last_active":"2024-05-08T18:06:50.219707+0000","last_peered":"2024-05-08T18:06:50.219707+0000","last_clean":"2024-05-08T18:06:50.219707+0000","last_became_active":"2024-05-08T18:04:45.811648+0000","last_became_peered":"2024-05-08T18:04:45.811648+0000","last_unstale":"2024-05-08T18:06:50.219707+0000","last_undegraded":"2024-05-08T18:06:50.219707+0000","last_fullsized":"2024-05-08T18:06:50.219707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:18.539896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":31,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"29'5","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.740580+0000","last_change":"2024-05-08T18:04:43.836500+0000","last_active":"2024-05-08T18:07:03.740580+0000","last_peered":"2024-05-08T18:07:03.740580+0000","last_clean":"2024-05-08T18:07:03.740580+0000","last_became_active":"2024-05-08T18:04:43.835712+0000","last_became_peered":"2024-05-08T18:04:43.835712+0000","last_unstale":"2024-05-08T18:07:03.740580+0000","last_undegraded":"2024-05-08T18:07:03.740580+0000","last_fullsized":"2024-05-08T18:07:03.740580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:06.338736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8,"num_write":21,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"29'10","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.775008+0000","last_change":"2024-05-08T18:04:45.812258+0000","last_active":"2024-05-08T18:07:53.775008+0000","last_peered":"2024-05-08T18:07:53.775008+0000","last_clean":"2024-05-08T18:07:53.775008+0000","last_became_active":"2024-05-08T18:04:45.811921+0000","last_became_peered":"2024-05-08T18:04:45.811921+0000","last_unstale":"2024-05-08T18:07:53.775008+0000","last_undegraded":"2024-05-08T18:07:53.775008+0000","last_fullsized":"2024-05-08T18:07:53.775008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:00:26.846485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2256,"ondisk_log_size":2256,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":13385,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1402,"num_read_kb":1188,"num_write":531,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":10826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1275,"ondisk_log_size":1275,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":961,"num_read_kb":961,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":25809792,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":138,"num_read_kb":87,"num_write":309,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34422784,"data_stored":34422784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":256,"acting":256,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705707,"num_pgs":179,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48332,"kb_used_data":8836,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323508,"statfs":{"total":96636764160,"available":96587272192,"internally_reserved":0,"allocated":9048064,"data_stored":8743050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705708,"num_pgs":153,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48848,"kb_used_data":9352,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322992,"statfs":{"total":96636764160,"available":96586743808,"internally_reserved":0,"allocated":9576448,"data_stored":9329849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705708,"num_pgs":157,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48872,"kb_used_data":9376,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322968,"statfs":{"total":96636764160,"available":96586719232,"internally_reserved":0,"allocated":9601024,"data_stored":9332048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39980,"kb_used_data":8804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331860,"statfs":{"total":96636764160,"available":96595824640,"internally_reserved":0,"allocated":9015296,"data_stored":8741305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":8605696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":8605696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":8605696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":8605696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":4269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T18:08:01.894 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T18:08:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.030+0000 7f7bb326f640 1 Processor -- start 2024-05-08T18:08:02.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.030+0000 7f7bb326f640 1 -- start start 2024-05-08T18:08:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.031+0000 7f7bb326f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7bac09b020 0x7f7bac096390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.031+0000 7f7bb326f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bac09ed20 con 0x7f7bac0968d0 2024-05-08T18:08:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.031+0000 7f7bb326f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f7bac075d10 con 0x7f7bac09b020 2024-05-08T18:08:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.031+0000 7f7bb100b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7bac09b020 0x7f7bac096390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:02.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.031+0000 7f7bb080a640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7bac0968d0 0x7f7bac133fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:45182/0 (socket says 172.21.15.46:45182) 2024-05-08T18:08:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.031+0000 7f7bb080a640 1 -- 172.21.15.46:0/3979735891 learned_addr learned my addr 172.21.15.46:0/3979735891 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:08:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.031+0000 7f7bb100b640 1 -- 172.21.15.46:0/3979735891 >> v1:172.21.15.46:6789/0 conn(0x7f7bac0968d0 legacy=0x7f7bac133fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.031+0000 7f7bb100b640 1 -- 172.21.15.46:0/3979735891 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bac096d40 con 0x7f7bac09b020 2024-05-08T18:08:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.032+0000 7f7bb100b640 1 --2- 172.21.15.46:0/3979735891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bac09b020 0x7f7bac096390 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8009720 tx=0x7f7ba802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b9a9029128d6ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.032+0000 7f7ba3fff640 1 -- 172.21.15.46:0/3979735891 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ba8035020 con 0x7f7bac09b020 2024-05-08T18:08:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.032+0000 7f7ba3fff640 1 -- 172.21.15.46:0/3979735891 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba8031760 con 0x7f7bac09b020 2024-05-08T18:08:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.032+0000 7f7ba3fff640 1 -- 172.21.15.46:0/3979735891 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ba8031a60 con 0x7f7bac09b020 2024-05-08T18:08:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.032+0000 7f7bb326f640 1 -- 172.21.15.46:0/3979735891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bac09b020 msgr2=0x7f7bac096390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.032+0000 7f7bb326f640 1 --2- 172.21.15.46:0/3979735891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bac09b020 0x7f7bac096390 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8009720 tx=0x7f7ba802b8d0 comp rx=0 tx=0).stop 2024-05-08T18:08:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.032+0000 7f7bb326f640 1 -- 172.21.15.46:0/3979735891 shutdown_connections 2024-05-08T18:08:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.032+0000 7f7bb326f640 1 --2- 172.21.15.46:0/3979735891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bac09b020 0x7f7bac096390 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.032+0000 7f7bb326f640 1 -- 172.21.15.46:0/3979735891 >> 172.21.15.46:0/3979735891 conn(0x7f7bac09e440 msgr2=0x7f7bac09a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.033+0000 7f7bb326f640 1 -- 172.21.15.46:0/3979735891 shutdown_connections 2024-05-08T18:08:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.033+0000 7f7bb326f640 1 -- 172.21.15.46:0/3979735891 wait complete. 2024-05-08T18:08:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.033+0000 7f7bb326f640 1 Processor -- start 2024-05-08T18:08:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.033+0000 7f7bb326f640 1 -- start start 2024-05-08T18:08:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.034+0000 7f7bb326f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bac0968d0 0x7f7bac133c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.034+0000 7f7bb326f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac06d280 con 0x7f7bac0968d0 2024-05-08T18:08:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.034+0000 7f7bb100b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bac0968d0 0x7f7bac133c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.034+0000 7f7bb100b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bac0968d0 0x7f7bac133c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52056/0 (socket says 172.21.15.46:52056) 2024-05-08T18:08:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.034+0000 7f7bb100b640 1 -- 172.21.15.46:0/2368521499 learned_addr learned my addr 172.21.15.46:0/2368521499 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:08:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.034+0000 7f7bb100b640 1 -- 172.21.15.46:0/2368521499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bac126010 con 0x7f7bac0968d0 2024-05-08T18:08:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.034+0000 7f7bb100b640 1 --2- 172.21.15.46:0/2368521499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bac0968d0 0x7f7bac133c50 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7ba802b910 tx=0x7f7ba80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.034+0000 7f7ba1ffb640 1 -- 172.21.15.46:0/2368521499 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ba803bcd0 con 0x7f7bac0968d0 2024-05-08T18:08:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.034+0000 7f7ba1ffb640 1 -- 172.21.15.46:0/2368521499 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba80453f0 con 0x7f7bac0968d0 2024-05-08T18:08:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.034+0000 7f7bb326f640 1 -- 172.21.15.46:0/2368521499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bac125930 con 0x7f7bac0968d0 2024-05-08T18:08:02.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.034+0000 7f7ba1ffb640 1 -- 172.21.15.46:0/2368521499 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7ba8045710 con 0x7f7bac0968d0 2024-05-08T18:08:02.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.035+0000 7f7bb326f640 1 -- 172.21.15.46:0/2368521499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bac134190 con 0x7f7bac0968d0 2024-05-08T18:08:02.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.035+0000 7f7ba1ffb640 1 -- 172.21.15.46:0/2368521499 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f7ba804c060 con 0x7f7bac0968d0 2024-05-08T18:08:02.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.036+0000 7f7bb326f640 1 -- 172.21.15.46:0/2368521499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bac0640a0 con 0x7f7bac0968d0 2024-05-08T18:08:02.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.036+0000 7f7ba1ffb640 1 --2- 172.21.15.46:0/2368521499 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7b9003d290 0x7f7b9003f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:02.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.036+0000 7f7ba1ffb640 1 -- 172.21.15.46:0/2368521499 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f7ba8073f10 con 0x7f7bac0968d0 2024-05-08T18:08:02.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.036+0000 7f7bb080a640 1 --2- 172.21.15.46:0/2368521499 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7b9003d290 0x7f7b9003f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:02.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.037+0000 7f7bb080a640 1 --2- 172.21.15.46:0/2368521499 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7b9003d290 0x7f7b9003f750 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c007fd0 tx=0x7f7b9c007aa0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:02.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.040+0000 7f7ba1ffb640 1 -- 172.21.15.46:0/2368521499 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f7ba8040070 con 0x7f7bac0968d0 2024-05-08T18:08:02.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.230+0000 7f7bb326f640 1 -- 172.21.15.46:0/2368521499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7bac098270 con 0x7f7bac0968d0 2024-05-08T18:08:02.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.231+0000 7f7ba1ffb640 1 -- 172.21.15.46:0/2368521499 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+20946 (secure 0 0 0) 0x7f7ba80316b0 con 0x7f7bac0968d0 2024-05-08T18:08:02.230 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:08:02.230 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":29,"fsid":"d8088d1f-616e-46b1-995a-85e572267d38","created":"2024-05-08T18:04:25.152571+0000","modified":"2024-05-08T18:05:30.851587+0000","last_up_change":"2024-05-08T18:04:33.542468+0000","last_in_change":"2024-05-08T18:04:26.221033+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T18:04:33.679407+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T18:04:34.053953+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.buckets.data","create_time":"2024-05-08T18:04:41.787736+0000","flags":1,"flags_names":"hashpspool","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"client.0","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":12288,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.buckets.index","create_time":"2024-05-08T18:04:44.148479+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":".rgw.root","create_time":"2024-05-08T18:04:46.111194+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.log","create_time":"2024-05-08T18:04:47.889985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"default.rgw.control","create_time":"2024-05-08T18:04:49.839292+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"default.rgw.meta","create_time":"2024-05-08T18:04:51.907492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7304d61-4d9e-455b-a968-5641c682e642","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":337143975},{"type":"v1","addr":"172.21.15.46:6801","nonce":337143975}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":337143975},{"type":"v1","addr":"172.21.15.46:6803","nonce":337143975}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":337143975},{"type":"v1","addr":"172.21.15.46:6807","nonce":337143975}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":337143975},{"type":"v1","addr":"172.21.15.46:6805","nonce":337143975}]},"public_addr":"172.21.15.46:6801/337143975","cluster_addr":"172.21.15.46:6803/337143975","heartbeat_back_addr":"172.21.15.46:6807/337143975","heartbeat_front_addr":"172.21.15.46:6805/337143975","state":["exists","up"]},{"osd":1,"uuid":"9a62c75d-b151-4453-a538-14408e8f4b78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":2141923372},{"type":"v1","addr":"172.21.15.50:6809","nonce":2141923372}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":2141923372},{"type":"v1","addr":"172.21.15.50:6812","nonce":2141923372}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2141923372},{"type":"v1","addr":"172.21.15.50:6820","nonce":2141923372}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":2141923372},{"type":"v1","addr":"172.21.15.50:6816","nonce":2141923372}]},"public_addr":"172.21.15.50:6809/2141923372","cluster_addr":"172.21.15.50:6812/2141923372","heartbeat_back_addr":"172.21.15.50:6820/2141923372","heartbeat_front_addr":"172.21.15.50:6816/2141923372","state":["exists","up"]},{"osd":2,"uuid":"c2d4e8c1-79bb-4061-ade1-f0fc079c52f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6811","nonce":3039646617},{"type":"v1","addr":"172.21.15.50:6813","nonce":3039646617}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6815","nonce":3039646617},{"type":"v1","addr":"172.21.15.50:6817","nonce":3039646617}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":3039646617},{"type":"v1","addr":"172.21.15.50:6823","nonce":3039646617}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6819","nonce":3039646617},{"type":"v1","addr":"172.21.15.50:6821","nonce":3039646617}]},"public_addr":"172.21.15.50:6813/3039646617","cluster_addr":"172.21.15.50:6817/3039646617","heartbeat_back_addr":"172.21.15.50:6823/3039646617","heartbeat_front_addr":"172.21.15.50:6821/3039646617","state":["exists","up"]},{"osd":3,"uuid":"e5e292cd-9ed4-44c9-9e91-7ee3736272ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":3130492744},{"type":"v1","addr":"172.21.15.50:6801","nonce":3130492744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":3130492744},{"type":"v1","addr":"172.21.15.50:6803","nonce":3130492744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":3130492744},{"type":"v1","addr":"172.21.15.50:6807","nonce":3130492744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":3130492744},{"type":"v1","addr":"172.21.15.50:6805","nonce":3130492744}]},"public_addr":"172.21.15.50:6801/3130492744","cluster_addr":"172.21.15.50:6803/3130492744","heartbeat_back_addr":"172.21.15.50:6807/3130492744","heartbeat_front_addr":"172.21.15.50:6805/3130492744","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T18:04:31.282728+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T18:04:31.387361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T18:04:31.386518+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T18:04:31.319396+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.11","mappings":[{"from":3,"to":2}]},{"pgid":"4.16","mappings":[{"from":3,"to":2}]},{"pgid":"4.1f","mappings":[{"from":3,"to":2}]},{"pgid":"5.4","mappings":[{"from":0,"to":2}]},{"pgid":"5.7","mappings":[{"from":0,"to":2}]},{"pgid":"5.14","mappings":[{"from":0,"to":1}]},{"pgid":"5.1c","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"client.0":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"3","m":"1","name":"testprofile","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T18:08:02.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.233+0000 7f7bb326f640 1 -- 172.21.15.46:0/2368521499 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7b9003d290 msgr2=0x7f7b9003f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:02.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.233+0000 7f7bb326f640 1 --2- 172.21.15.46:0/2368521499 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7b9003d290 0x7f7b9003f750 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c007fd0 tx=0x7f7b9c007aa0 comp rx=0 tx=0).stop 2024-05-08T18:08:02.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.233+0000 7f7bb326f640 1 -- 172.21.15.46:0/2368521499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bac0968d0 msgr2=0x7f7bac133c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:02.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.233+0000 7f7bb326f640 1 --2- 172.21.15.46:0/2368521499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bac0968d0 0x7f7bac133c50 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f7ba802b910 tx=0x7f7ba80046c0 comp rx=0 tx=0).stop 2024-05-08T18:08:02.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.233+0000 7f7bb326f640 1 -- 172.21.15.46:0/2368521499 shutdown_connections 2024-05-08T18:08:02.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.233+0000 7f7bb326f640 1 --2- 172.21.15.46:0/2368521499 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f7b9003d290 0x7f7b9003f750 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:02.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.233+0000 7f7bb326f640 1 --2- 172.21.15.46:0/2368521499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bac0968d0 0x7f7bac133c50 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:02.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.233+0000 7f7bb326f640 1 -- 172.21.15.46:0/2368521499 >> 172.21.15.46:0/2368521499 conn(0x7f7bac09e440 msgr2=0x7f7bac072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:02.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.233+0000 7f7bb326f640 1 -- 172.21.15.46:0/2368521499 shutdown_connections 2024-05-08T18:08:02.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:02.233+0000 7f7bb326f640 1 -- 172.21.15.46:0/2368521499 wait complete. 2024-05-08T18:08:03.243 INFO:tasks.ceph:Scrubbing osd.0 2024-05-08T18:08:03.243 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-08T18:08:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.365+0000 7f521d72b640 1 Processor -- start 2024-05-08T18:08:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.365+0000 7f521d72b640 1 -- start start 2024-05-08T18:08:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.366+0000 7f521d72b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f521812fe70 0x7f5218130250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:03.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.366+0000 7f521d72b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5218071170 con 0x7f5218130790 2024-05-08T18:08:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.366+0000 7f521d72b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f521809f580 con 0x7f521812fe70 2024-05-08T18:08:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.366+0000 7f5216ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f521812fe70 0x7f5218130250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.366+0000 7f52167fc640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5218130790 0x7f521806f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51800/0 (socket says 172.21.15.46:51800) 2024-05-08T18:08:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.366+0000 7f52167fc640 1 -- 172.21.15.46:0/2253782151 learned_addr learned my addr 172.21.15.46:0/2253782151 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:08:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.366+0000 7f5216ffd640 1 -- 172.21.15.46:0/2253782151 >> v1:172.21.15.46:6789/0 conn(0x7f5218130790 legacy=0x7f521806f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:03.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.366+0000 7f5216ffd640 1 -- 172.21.15.46:0/2253782151 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521806fbf0 con 0x7f521812fe70 2024-05-08T18:08:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.367+0000 7f5216ffd640 1 --2- 172.21.15.46:0/2253782151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f521812fe70 0x7f5218130250 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5208009720 tx=0x7f520802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278d24e328ad7e7c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.367+0000 7f5215ffb640 1 -- 172.21.15.46:0/2253782151 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52080316d0 con 0x7f521812fe70 2024-05-08T18:08:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.367+0000 7f5215ffb640 1 -- 172.21.15.46:0/2253782151 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5208037070 con 0x7f521812fe70 2024-05-08T18:08:03.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.367+0000 7f5215ffb640 1 -- 172.21.15.46:0/2253782151 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52080319c0 con 0x7f521812fe70 2024-05-08T18:08:03.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.368+0000 7f521d72b640 1 -- 172.21.15.46:0/2253782151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f521812fe70 msgr2=0x7f5218130250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:03.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.368+0000 7f521d72b640 1 --2- 172.21.15.46:0/2253782151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f521812fe70 0x7f5218130250 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f5208009720 tx=0x7f520802b840 comp rx=0 tx=0).stop 2024-05-08T18:08:03.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.368+0000 7f521d72b640 1 -- 172.21.15.46:0/2253782151 shutdown_connections 2024-05-08T18:08:03.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.368+0000 7f521d72b640 1 --2- 172.21.15.46:0/2253782151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f521812fe70 0x7f5218130250 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.368+0000 7f521d72b640 1 -- 172.21.15.46:0/2253782151 >> 172.21.15.46:0/2253782151 conn(0x7f521809ea60 msgr2=0x7f521809ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:03.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.368+0000 7f521d72b640 1 -- 172.21.15.46:0/2253782151 shutdown_connections 2024-05-08T18:08:03.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.368+0000 7f521d72b640 1 -- 172.21.15.46:0/2253782151 wait complete. 2024-05-08T18:08:03.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.369+0000 7f521d72b640 1 Processor -- start 2024-05-08T18:08:03.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.369+0000 7f521d72b640 1 -- start start 2024-05-08T18:08:03.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.369+0000 7f521d72b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f521812fe70 0x7f521812dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:03.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.369+0000 7f521d72b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5218070300 con 0x7f521812fe70 2024-05-08T18:08:03.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.369+0000 7f5216ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f521812fe70 0x7f521812dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:03.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.369+0000 7f5216ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f521812fe70 0x7f521812dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43816/0 (socket says 172.21.15.46:43816) 2024-05-08T18:08:03.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.369+0000 7f5216ffd640 1 -- 172.21.15.46:0/1262195891 learned_addr learned my addr 172.21.15.46:0/1262195891 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:08:03.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.370+0000 7f5216ffd640 1 -- 172.21.15.46:0/1262195891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521812d7b0 con 0x7f521812fe70 2024-05-08T18:08:03.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.370+0000 7f5216ffd640 1 --2- 172.21.15.46:0/1262195891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f521812fe70 0x7f521812dac0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f520802b880 tx=0x7f52080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:03.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.370+0000 7f51ff7fe640 1 -- 172.21.15.46:0/1262195891 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5208032770 con 0x7f521812fe70 2024-05-08T18:08:03.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.370+0000 7f51ff7fe640 1 -- 172.21.15.46:0/1262195891 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5208032910 con 0x7f521812fe70 2024-05-08T18:08:03.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.370+0000 7f521d72b640 1 -- 172.21.15.46:0/1262195891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f521812cc00 con 0x7f521812fe70 2024-05-08T18:08:03.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.370+0000 7f51ff7fe640 1 -- 172.21.15.46:0/1262195891 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5208032c10 con 0x7f521812fe70 2024-05-08T18:08:03.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.370+0000 7f521d72b640 1 -- 172.21.15.46:0/1262195891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f521812f9f0 con 0x7f521812fe70 2024-05-08T18:08:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.371+0000 7f521d72b640 1 -- 172.21.15.46:0/1262195891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f52180640a0 con 0x7f521812fe70 2024-05-08T18:08:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.371+0000 7f51ff7fe640 1 -- 172.21.15.46:0/1262195891 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f520804c060 con 0x7f521812fe70 2024-05-08T18:08:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.371+0000 7f51ff7fe640 1 --2- 172.21.15.46:0/1262195891 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f520403d290 0x7f520403f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.371+0000 7f51ff7fe640 1 -- 172.21.15.46:0/1262195891 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f520802f050 con 0x7f521812fe70 2024-05-08T18:08:03.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.371+0000 7f51ff7fe640 1 --2- 172.21.15.46:0/1262195891 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x7f5204049680 0x7f520404bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:03.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.371+0000 7f51ff7fe640 1 -- 172.21.15.46:0/1262195891 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f520804cec0 con 0x7f5204049680 2024-05-08T18:08:03.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.371+0000 7f51ff7fe640 1 -- 172.21.15.46:0/1262195891 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f520807a050 con 0x7f521812fe70 2024-05-08T18:08:03.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.372+0000 7f52167fc640 1 --2- 172.21.15.46:0/1262195891 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f520403d290 0x7f520403f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:03.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.372+0000 7f52177fe640 1 --2- 172.21.15.46:0/1262195891 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x7f5204049680 0x7f520404bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:03.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.372+0000 7f52177fe640 1 --2- 172.21.15.46:0/1262195891 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x7f5204049680 0x7f520404bb20 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:03.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.372+0000 7f52167fc640 1 --2- 172.21.15.46:0/1262195891 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f520403d290 0x7f520403f750 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f520c007fd0 tx=0x7f520c007aa0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:03.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.373+0000 7f51ff7fe640 1 -- 172.21.15.46:0/1262195891 <== osd.0 v2:172.21.15.46:6800/337143975 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f520804cec0 con 0x7f5204049680 2024-05-08T18:08:03.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.386+0000 7f521d72b640 1 -- 172.21.15.46:0/1262195891 --> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f521806d2b0 con 0x7f5204049680 2024-05-08T18:08:03.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.386+0000 7f51ff7fe640 1 -- 172.21.15.46:0/1262195891 <== osd.0 v2:172.21.15.46:6800/337143975 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f521806d2b0 con 0x7f5204049680 2024-05-08T18:08:03.384 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T18:08:03.385 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T18:08:03.385 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T18:08:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.386+0000 7f521d72b640 1 -- 172.21.15.46:0/1262195891 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x7f5204049680 msgr2=0x7f520404bb20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.386+0000 7f521d72b640 1 --2- 172.21.15.46:0/1262195891 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x7f5204049680 0x7f520404bb20 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.386+0000 7f521d72b640 1 -- 172.21.15.46:0/1262195891 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f520403d290 msgr2=0x7f520403f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.386+0000 7f521d72b640 1 --2- 172.21.15.46:0/1262195891 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f520403d290 0x7f520403f750 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f520c007fd0 tx=0x7f520c007aa0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.386+0000 7f521d72b640 1 -- 172.21.15.46:0/1262195891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f521812fe70 msgr2=0x7f521812dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:03.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.387+0000 7f521d72b640 1 --2- 172.21.15.46:0/1262195891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f521812fe70 0x7f521812dac0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f520802b880 tx=0x7f52080046c0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.387+0000 7f521d72b640 1 -- 172.21.15.46:0/1262195891 shutdown_connections 2024-05-08T18:08:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.387+0000 7f521d72b640 1 --2- 172.21.15.46:0/1262195891 >> [v2:172.21.15.46:6800/337143975,v1:172.21.15.46:6801/337143975] conn(0x7f5204049680 0x7f520404bb20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.387+0000 7f521d72b640 1 --2- 172.21.15.46:0/1262195891 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f520403d290 0x7f520403f750 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.387+0000 7f521d72b640 1 --2- 172.21.15.46:0/1262195891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f521812fe70 0x7f521812dac0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.387+0000 7f521d72b640 1 -- 172.21.15.46:0/1262195891 >> 172.21.15.46:0/1262195891 conn(0x7f521809ea60 msgr2=0x7f52180729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.387+0000 7f521d72b640 1 -- 172.21.15.46:0/1262195891 shutdown_connections 2024-05-08T18:08:03.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.387+0000 7f521d72b640 1 -- 172.21.15.46:0/1262195891 wait complete. 2024-05-08T18:08:03.395 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-08T18:08:03.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.500+0000 7fad2fbc0640 1 Processor -- start 2024-05-08T18:08:03.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.501+0000 7fad2fbc0640 1 -- start start 2024-05-08T18:08:03.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.501+0000 7fad2fbc0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad28070650 0x7fad28070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:03.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.501+0000 7fad2fbc0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad28073480 con 0x7fad28072540 2024-05-08T18:08:03.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.501+0000 7fad2fbc0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fad28071a80 con 0x7fad28070650 2024-05-08T18:08:03.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.501+0000 7fad2d95c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad28070650 0x7fad28070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:03.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.501+0000 7fad2d15b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fad28072540 0x7fad2812f580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51812/0 (socket says 172.21.15.46:51812) 2024-05-08T18:08:03.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.501+0000 7fad2d15b640 1 -- 172.21.15.46:0/196425522 learned_addr learned my addr 172.21.15.46:0/196425522 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:08:03.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.501+0000 7fad2d95c640 1 -- 172.21.15.46:0/196425522 >> v1:172.21.15.46:6789/0 conn(0x7fad28072540 legacy=0x7fad2812f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:03.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.501+0000 7fad2d95c640 1 -- 172.21.15.46:0/196425522 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad28072c20 con 0x7fad28070650 2024-05-08T18:08:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.502+0000 7fad2d95c640 1 --2- 172.21.15.46:0/196425522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad28070650 0x7fad28070a30 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fad24009720 tx=0x7fad2402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b25fd9429c0630e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.502+0000 7fad2c95a640 1 -- 172.21.15.46:0/196425522 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad240316d0 con 0x7fad28070650 2024-05-08T18:08:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.502+0000 7fad2c95a640 1 -- 172.21.15.46:0/196425522 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad24037070 con 0x7fad28070650 2024-05-08T18:08:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.502+0000 7fad2c95a640 1 -- 172.21.15.46:0/196425522 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad240319c0 con 0x7fad28070650 2024-05-08T18:08:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.503+0000 7fad2fbc0640 1 -- 172.21.15.46:0/196425522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad28070650 msgr2=0x7fad28070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.503+0000 7fad2fbc0640 1 --2- 172.21.15.46:0/196425522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad28070650 0x7fad28070a30 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fad24009720 tx=0x7fad2402b840 comp rx=0 tx=0).stop 2024-05-08T18:08:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.503+0000 7fad2fbc0640 1 -- 172.21.15.46:0/196425522 shutdown_connections 2024-05-08T18:08:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.503+0000 7fad2fbc0640 1 --2- 172.21.15.46:0/196425522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad28070650 0x7fad28070a30 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.503+0000 7fad2fbc0640 1 -- 172.21.15.46:0/196425522 >> 172.21.15.46:0/196425522 conn(0x7fad280a1e60 msgr2=0x7fad280a2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.503+0000 7fad2fbc0640 1 -- 172.21.15.46:0/196425522 shutdown_connections 2024-05-08T18:08:03.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.503+0000 7fad2fbc0640 1 -- 172.21.15.46:0/196425522 wait complete. 2024-05-08T18:08:03.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.504+0000 7fad2fbc0640 1 Processor -- start 2024-05-08T18:08:03.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.504+0000 7fad2fbc0640 1 -- start start 2024-05-08T18:08:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.504+0000 7fad2fbc0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad28070650 0x7fad281c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.504+0000 7fad2fbc0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad2812fc90 con 0x7fad28070650 2024-05-08T18:08:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.504+0000 7fad2d95c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad28070650 0x7fad281c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.504+0000 7fad2d95c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad28070650 0x7fad281c4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43840/0 (socket says 172.21.15.46:43840) 2024-05-08T18:08:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.504+0000 7fad2d95c640 1 -- 172.21.15.46:0/2753255216 learned_addr learned my addr 172.21.15.46:0/2753255216 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:08:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.505+0000 7fad2d95c640 1 -- 172.21.15.46:0/2753255216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad2811e930 con 0x7fad28070650 2024-05-08T18:08:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.505+0000 7fad2d95c640 1 --2- 172.21.15.46:0/2753255216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad28070650 0x7fad281c4450 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fad24034040 tx=0x7fad240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.505+0000 7fad1e7fc640 1 -- 172.21.15.46:0/2753255216 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad24032770 con 0x7fad28070650 2024-05-08T18:08:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.505+0000 7fad1e7fc640 1 -- 172.21.15.46:0/2753255216 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad24032910 con 0x7fad28070650 2024-05-08T18:08:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.505+0000 7fad1e7fc640 1 -- 172.21.15.46:0/2753255216 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad24032c30 con 0x7fad28070650 2024-05-08T18:08:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.505+0000 7fad2fbc0640 1 -- 172.21.15.46:0/2753255216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad2811f100 con 0x7fad28070650 2024-05-08T18:08:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.505+0000 7fad2fbc0640 1 -- 172.21.15.46:0/2753255216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad281c4990 con 0x7fad28070650 2024-05-08T18:08:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.506+0000 7fad1e7fc640 1 -- 172.21.15.46:0/2753255216 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fad2404c060 con 0x7fad28070650 2024-05-08T18:08:03.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.506+0000 7fad2fbc0640 1 -- 172.21.15.46:0/2753255216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad28120080 con 0x7fad28070650 2024-05-08T18:08:03.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.506+0000 7fad1e7fc640 1 --2- 172.21.15.46:0/2753255216 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fad0c03d2b0 0x7fad0c03f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:03.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.506+0000 7fad1e7fc640 1 -- 172.21.15.46:0/2753255216 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fad2402f050 con 0x7fad28070650 2024-05-08T18:08:03.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.506+0000 7fad2d15b640 1 --2- 172.21.15.46:0/2753255216 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fad0c03d2b0 0x7fad0c03f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:03.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.507+0000 7fad2d15b640 1 --2- 172.21.15.46:0/2753255216 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fad0c03d2b0 0x7fad0c03f770 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fad18007fd0 tx=0x7fad18007aa0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:03.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.511+0000 7fad1e7fc640 1 -- 172.21.15.46:0/2753255216 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fad24040070 con 0x7fad28070650 2024-05-08T18:08:03.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.705+0000 7fad2fbc0640 1 -- 172.21.15.46:0/2753255216 --> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fad2812c5d0 con 0x7fad0c03d2b0 2024-05-08T18:08:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.706+0000 7fad1e7fc640 1 -- 172.21.15.46:0/2753255216 <== mgr.4098 v2:172.21.15.50:6824/2602185226 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fad2812c5d0 con 0x7fad0c03d2b0 2024-05-08T18:08:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 0 to deep-scrub 2024-05-08T18:08:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.709+0000 7fad2fbc0640 1 -- 172.21.15.46:0/2753255216 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fad0c03d2b0 msgr2=0x7fad0c03f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.709+0000 7fad2fbc0640 1 --2- 172.21.15.46:0/2753255216 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fad0c03d2b0 0x7fad0c03f770 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fad18007fd0 tx=0x7fad18007aa0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.709+0000 7fad2fbc0640 1 -- 172.21.15.46:0/2753255216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad28070650 msgr2=0x7fad281c4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.709+0000 7fad2fbc0640 1 --2- 172.21.15.46:0/2753255216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad28070650 0x7fad281c4450 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fad24034040 tx=0x7fad240046c0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.709+0000 7fad2fbc0640 1 -- 172.21.15.46:0/2753255216 shutdown_connections 2024-05-08T18:08:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.709+0000 7fad2fbc0640 1 --2- 172.21.15.46:0/2753255216 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fad0c03d2b0 0x7fad0c03f770 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.709+0000 7fad2fbc0640 1 --2- 172.21.15.46:0/2753255216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad28070650 0x7fad281c4450 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.709+0000 7fad2fbc0640 1 -- 172.21.15.46:0/2753255216 >> 172.21.15.46:0/2753255216 conn(0x7fad280a1e60 msgr2=0x7fad2812bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.709+0000 7fad2fbc0640 1 -- 172.21.15.46:0/2753255216 shutdown_connections 2024-05-08T18:08:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.709+0000 7fad2fbc0640 1 -- 172.21.15.46:0/2753255216 wait complete. 2024-05-08T18:08:03.718 INFO:tasks.ceph:Scrubbing osd.1 2024-05-08T18:08:03.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-08T18:08:03.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.826+0000 7fe6ba302640 1 Processor -- start 2024-05-08T18:08:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.826+0000 7fe6ba302640 1 -- start start 2024-05-08T18:08:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.826+0000 7fe6ba302640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe6b4128970 0x7fe6b4128d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.826+0000 7fe6ba302640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6b4072950 con 0x7fe6b4072fa0 2024-05-08T18:08:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.826+0000 7fe6ba302640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe6b4128530 con 0x7fe6b4128970 2024-05-08T18:08:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.826+0000 7fe6b3fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe6b4128970 0x7fe6b4128d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.826+0000 7fe6abfff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe6b4072fa0 0x7fe6b412b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51828/0 (socket says 172.21.15.46:51828) 2024-05-08T18:08:03.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.827+0000 7fe6abfff640 1 -- 172.21.15.46:0/1202567169 learned_addr learned my addr 172.21.15.46:0/1202567169 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:08:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.827+0000 7fe6b3fff640 1 -- 172.21.15.46:0/1202567169 >> v1:172.21.15.46:6789/0 conn(0x7fe6b4072fa0 legacy=0x7fe6b412b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.827+0000 7fe6b3fff640 1 -- 172.21.15.46:0/1202567169 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b4073f00 con 0x7fe6b4128970 2024-05-08T18:08:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.828+0000 7fe6b3fff640 1 --2- 172.21.15.46:0/1202567169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b4128970 0x7fe6b4128d50 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009720 tx=0x7fe6a402c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44171837803d030 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.828+0000 7fe6b37fe640 1 -- 172.21.15.46:0/1202567169 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6a4035070 con 0x7fe6b4128970 2024-05-08T18:08:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.828+0000 7fe6b37fe640 1 -- 172.21.15.46:0/1202567169 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a4031740 con 0x7fe6b4128970 2024-05-08T18:08:03.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.828+0000 7fe6b37fe640 1 -- 172.21.15.46:0/1202567169 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6a4031a40 con 0x7fe6b4128970 2024-05-08T18:08:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.828+0000 7fe6ba302640 1 -- 172.21.15.46:0/1202567169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b4128970 msgr2=0x7fe6b4128d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.828+0000 7fe6ba302640 1 --2- 172.21.15.46:0/1202567169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b4128970 0x7fe6b4128d50 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009720 tx=0x7fe6a402c950 comp rx=0 tx=0).stop 2024-05-08T18:08:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.829+0000 7fe6ba302640 1 -- 172.21.15.46:0/1202567169 shutdown_connections 2024-05-08T18:08:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.829+0000 7fe6ba302640 1 --2- 172.21.15.46:0/1202567169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b4128970 0x7fe6b4128d50 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.829+0000 7fe6ba302640 1 -- 172.21.15.46:0/1202567169 >> 172.21.15.46:0/1202567169 conn(0x7fe6b409ab60 msgr2=0x7fe6b406e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.829+0000 7fe6ba302640 1 -- 172.21.15.46:0/1202567169 shutdown_connections 2024-05-08T18:08:03.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.829+0000 7fe6ba302640 1 -- 172.21.15.46:0/1202567169 wait complete. 2024-05-08T18:08:03.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.829+0000 7fe6ba302640 1 Processor -- start 2024-05-08T18:08:03.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.829+0000 7fe6ba302640 1 -- start start 2024-05-08T18:08:03.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.830+0000 7fe6ba302640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b4072fa0 0x7fe6b41c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:03.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.830+0000 7fe6ba302640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b406fed0 con 0x7fe6b4072fa0 2024-05-08T18:08:03.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.830+0000 7fe6b3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b4072fa0 0x7fe6b41c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:03.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.830+0000 7fe6b3fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b4072fa0 0x7fe6b41c15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43858/0 (socket says 172.21.15.46:43858) 2024-05-08T18:08:03.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.830+0000 7fe6b3fff640 1 -- 172.21.15.46:0/1893555368 learned_addr learned my addr 172.21.15.46:0/1893555368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:08:03.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.830+0000 7fe6b3fff640 1 -- 172.21.15.46:0/1893555368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b41152a0 con 0x7fe6b4072fa0 2024-05-08T18:08:03.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.830+0000 7fe6b3fff640 1 --2- 172.21.15.46:0/1893555368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b4072fa0 0x7fe6b41c15e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009720 tx=0x7fe6a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:03.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.831+0000 7fe6b17fa640 1 -- 172.21.15.46:0/1893555368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6a404a070 con 0x7fe6b4072fa0 2024-05-08T18:08:03.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.831+0000 7fe6b17fa640 1 -- 172.21.15.46:0/1893555368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a4031de0 con 0x7fe6b4072fa0 2024-05-08T18:08:03.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.831+0000 7fe6b17fa640 1 -- 172.21.15.46:0/1893555368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe6a4035040 con 0x7fe6b4072fa0 2024-05-08T18:08:03.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.831+0000 7fe6ba302640 1 -- 172.21.15.46:0/1893555368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b4117220 con 0x7fe6b4072fa0 2024-05-08T18:08:03.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.831+0000 7fe6ba302640 1 -- 172.21.15.46:0/1893555368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b4115c20 con 0x7fe6b4072fa0 2024-05-08T18:08:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.831+0000 7fe6ba302640 1 -- 172.21.15.46:0/1893555368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe6b41174b0 con 0x7fe6b4072fa0 2024-05-08T18:08:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.832+0000 7fe6b17fa640 1 -- 172.21.15.46:0/1893555368 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fe6a404d070 con 0x7fe6b4072fa0 2024-05-08T18:08:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.832+0000 7fe6b17fa640 1 --2- 172.21.15.46:0/1893555368 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe69403d2e0 0x7fe69403f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.832+0000 7fe6b17fa640 1 -- 172.21.15.46:0/1893555368 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fe6a402e040 con 0x7fe6b4072fa0 2024-05-08T18:08:03.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.832+0000 7fe6b17fa640 1 --2- 172.21.15.46:0/1893555368 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x7fe6940497b0 0x7fe69404bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.832+0000 7fe6b17fa640 1 -- 172.21.15.46:0/1893555368 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6a404ae70 con 0x7fe6940497b0 2024-05-08T18:08:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.832+0000 7fe6b17fa640 1 -- 172.21.15.46:0/1893555368 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe6a4076470 con 0x7fe6b4072fa0 2024-05-08T18:08:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.832+0000 7fe6abfff640 1 --2- 172.21.15.46:0/1893555368 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe69403d2e0 0x7fe69403f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.832+0000 7fe6b889f640 1 --2- 172.21.15.46:0/1893555368 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x7fe6940497b0 0x7fe69404bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.833+0000 7fe6abfff640 1 --2- 172.21.15.46:0/1893555368 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe69403d2e0 0x7fe69403f7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe69c007fd0 tx=0x7fe69c007aa0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.833+0000 7fe6b889f640 1 --2- 172.21.15.46:0/1893555368 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x7fe6940497b0 0x7fe69404bc50 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:03.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.834+0000 7fe6b17fa640 1 -- 172.21.15.46:0/1893555368 <== osd.1 v2:172.21.15.50:6808/2141923372 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6a404ae70 con 0x7fe6940497b0 2024-05-08T18:08:03.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.849+0000 7fe6ba302640 1 -- 172.21.15.46:0/1893555368 --> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe6b4115f30 con 0x7fe6940497b0 2024-05-08T18:08:03.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.849+0000 7fe6b17fa640 1 -- 172.21.15.46:0/1893555368 <== osd.1 v2:172.21.15.50:6808/2141923372 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe6b4115f30 con 0x7fe6940497b0 2024-05-08T18:08:03.847 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T18:08:03.847 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T18:08:03.847 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T18:08:03.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.849+0000 7fe6ba302640 1 -- 172.21.15.46:0/1893555368 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x7fe6940497b0 msgr2=0x7fe69404bc50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:03.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.849+0000 7fe6ba302640 1 --2- 172.21.15.46:0/1893555368 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x7fe6940497b0 0x7fe69404bc50 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.849+0000 7fe6ba302640 1 -- 172.21.15.46:0/1893555368 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe69403d2e0 msgr2=0x7fe69403f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:03.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.849+0000 7fe6ba302640 1 --2- 172.21.15.46:0/1893555368 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe69403d2e0 0x7fe69403f7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe69c007fd0 tx=0x7fe69c007aa0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.849+0000 7fe6ba302640 1 -- 172.21.15.46:0/1893555368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b4072fa0 msgr2=0x7fe6b41c15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:03.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.849+0000 7fe6ba302640 1 --2- 172.21.15.46:0/1893555368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b4072fa0 0x7fe6b41c15e0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009720 tx=0x7fe6a40046c0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.850+0000 7fe6ba302640 1 -- 172.21.15.46:0/1893555368 shutdown_connections 2024-05-08T18:08:03.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.850+0000 7fe6ba302640 1 --2- 172.21.15.46:0/1893555368 >> [v2:172.21.15.50:6808/2141923372,v1:172.21.15.50:6809/2141923372] conn(0x7fe6940497b0 0x7fe69404bc50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.850+0000 7fe6ba302640 1 --2- 172.21.15.46:0/1893555368 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe69403d2e0 0x7fe69403f7a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.850+0000 7fe6ba302640 1 --2- 172.21.15.46:0/1893555368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6b4072fa0 0x7fe6b41c15e0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.850+0000 7fe6ba302640 1 -- 172.21.15.46:0/1893555368 >> 172.21.15.46:0/1893555368 conn(0x7fe6b409ab60 msgr2=0x7fe6b4129600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:03.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.850+0000 7fe6ba302640 1 -- 172.21.15.46:0/1893555368 shutdown_connections 2024-05-08T18:08:03.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.850+0000 7fe6ba302640 1 -- 172.21.15.46:0/1893555368 wait complete. 2024-05-08T18:08:03.858 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-08T18:08:03.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.963+0000 7fa003c2a640 1 Processor -- start 2024-05-08T18:08:03.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.964+0000 7fa003c2a640 1 -- start start 2024-05-08T18:08:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.964+0000 7fa003c2a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ffc06ff30 0x7f9ffc134d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.964+0000 7fa003c2a640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ffc097e80 con 0x7f9ffc12fdb0 2024-05-08T18:08:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.964+0000 7fa003c2a640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f9ffc071640 con 0x7f9ffc06ff30 2024-05-08T18:08:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.964+0000 7fa0011c5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9ffc06ff30 0x7f9ffc134d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.964+0000 7fa0019c6640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9ffc12fdb0 0x7f9ffc130190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51836/0 (socket says 172.21.15.46:51836) 2024-05-08T18:08:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.964+0000 7fa0019c6640 1 -- 172.21.15.46:0/2302993773 learned_addr learned my addr 172.21.15.46:0/2302993773 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:08:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.964+0000 7fa0009c4640 1 -- 172.21.15.46:0/2302993773 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2465908345 0 0) 0x7f9ffc097e80 con 0x7f9ffc12fdb0 2024-05-08T18:08:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.964+0000 7fa0009c4640 1 -- 172.21.15.46:0/2302993773 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fe8003520 con 0x7f9ffc12fdb0 2024-05-08T18:08:03.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.964+0000 7fa0011c5640 1 -- 172.21.15.46:0/2302993773 >> v1:172.21.15.46:6789/0 conn(0x7f9ffc12fdb0 legacy=0x7f9ffc130190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:03.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.964+0000 7fa0011c5640 1 -- 172.21.15.46:0/2302993773 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ffc069550 con 0x7f9ffc06ff30 2024-05-08T18:08:03.964 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.965+0000 7fa0011c5640 1 --2- 172.21.15.46:0/2302993773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc06ff30 0x7f9ffc134d80 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009720 tx=0x7f9fec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bdfaa9e42f72e08 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:03.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.965+0000 7fa0009c4640 1 -- 172.21.15.46:0/2302993773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fec0316d0 con 0x7f9ffc06ff30 2024-05-08T18:08:03.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.965+0000 7fa0009c4640 1 -- 172.21.15.46:0/2302993773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fec037070 con 0x7f9ffc06ff30 2024-05-08T18:08:03.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.965+0000 7fa0009c4640 1 -- 172.21.15.46:0/2302993773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fec0319c0 con 0x7f9ffc06ff30 2024-05-08T18:08:03.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.966+0000 7fa003c2a640 1 -- 172.21.15.46:0/2302993773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc06ff30 msgr2=0x7f9ffc134d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:03.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.966+0000 7fa003c2a640 1 --2- 172.21.15.46:0/2302993773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc06ff30 0x7f9ffc134d80 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009720 tx=0x7f9fec02b840 comp rx=0 tx=0).stop 2024-05-08T18:08:03.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.966+0000 7fa003c2a640 1 -- 172.21.15.46:0/2302993773 shutdown_connections 2024-05-08T18:08:03.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.966+0000 7fa003c2a640 1 --2- 172.21.15.46:0/2302993773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc06ff30 0x7f9ffc134d80 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:03.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.966+0000 7fa003c2a640 1 -- 172.21.15.46:0/2302993773 >> 172.21.15.46:0/2302993773 conn(0x7f9ffc09d9b0 msgr2=0x7f9ffc09ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:03.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.966+0000 7fa003c2a640 1 -- 172.21.15.46:0/2302993773 shutdown_connections 2024-05-08T18:08:03.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.966+0000 7fa003c2a640 1 -- 172.21.15.46:0/2302993773 wait complete. 2024-05-08T18:08:03.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.967+0000 7fa003c2a640 1 Processor -- start 2024-05-08T18:08:03.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.967+0000 7fa003c2a640 1 -- start start 2024-05-08T18:08:03.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.967+0000 7fa003c2a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc06ff30 0x7f9ffc0933a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:03.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.967+0000 7fa003c2a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ffc1308a0 con 0x7f9ffc06ff30 2024-05-08T18:08:03.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.967+0000 7fa0019c6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc06ff30 0x7f9ffc0933a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:03.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.967+0000 7fa0019c6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc06ff30 0x7f9ffc0933a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43878/0 (socket says 172.21.15.46:43878) 2024-05-08T18:08:03.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.967+0000 7fa0019c6640 1 -- 172.21.15.46:0/3278650004 learned_addr learned my addr 172.21.15.46:0/3278650004 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:08:03.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.968+0000 7fa0019c6640 1 -- 172.21.15.46:0/3278650004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ffc095f80 con 0x7f9ffc06ff30 2024-05-08T18:08:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.968+0000 7fa0019c6640 1 --2- 172.21.15.46:0/3278650004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc06ff30 0x7f9ffc0933a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9ff800c450 tx=0x7f9ff800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.968+0000 7f9ff27fc640 1 -- 172.21.15.46:0/3278650004 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ff8017070 con 0x7f9ffc06ff30 2024-05-08T18:08:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.968+0000 7f9ff27fc640 1 -- 172.21.15.46:0/3278650004 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ff8005cd0 con 0x7f9ffc06ff30 2024-05-08T18:08:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.968+0000 7f9ff27fc640 1 -- 172.21.15.46:0/3278650004 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ff80070d0 con 0x7f9ffc06ff30 2024-05-08T18:08:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.968+0000 7fa003c2a640 1 -- 172.21.15.46:0/3278650004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ffc096210 con 0x7f9ffc06ff30 2024-05-08T18:08:03.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.968+0000 7fa003c2a640 1 -- 172.21.15.46:0/3278650004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ffc093ac0 con 0x7f9ffc06ff30 2024-05-08T18:08:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.969+0000 7f9ff27fc640 1 -- 172.21.15.46:0/3278650004 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f9ff801a070 con 0x7f9ffc06ff30 2024-05-08T18:08:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.969+0000 7fa003c2a640 1 -- 172.21.15.46:0/3278650004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ffc099830 con 0x7f9ffc06ff30 2024-05-08T18:08:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.969+0000 7f9ff27fc640 1 --2- 172.21.15.46:0/3278650004 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9fe003d2b0 0x7f9fe003f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.969+0000 7f9ff27fc640 1 -- 172.21.15.46:0/3278650004 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f9ff8051130 con 0x7f9ffc06ff30 2024-05-08T18:08:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.969+0000 7fa0011c5640 1 --2- 172.21.15.46:0/3278650004 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9fe003d2b0 0x7f9fe003f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:03.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.970+0000 7fa0011c5640 1 --2- 172.21.15.46:0/3278650004 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9fe003d2b0 0x7f9fe003f770 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9fec036000 tx=0x7f9fec008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:03.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:03.974+0000 7f9ff27fc640 1 -- 172.21.15.46:0/3278650004 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f9ffc099830 con 0x7f9ffc06ff30 2024-05-08T18:08:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.165+0000 7fa003c2a640 1 -- 172.21.15.46:0/3278650004 --> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f9ffc134f40 con 0x7f9fe003d2b0 2024-05-08T18:08:04.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.166+0000 7f9ff27fc640 1 -- 172.21.15.46:0/3278650004 <== mgr.4098 v2:172.21.15.50:6824/2602185226 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f9fec037040 con 0x7f9fe003d2b0 2024-05-08T18:08:04.164 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 1 to deep-scrub 2024-05-08T18:08:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.168+0000 7fa003c2a640 1 -- 172.21.15.46:0/3278650004 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9fe003d2b0 msgr2=0x7f9fe003f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.168+0000 7fa003c2a640 1 --2- 172.21.15.46:0/3278650004 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9fe003d2b0 0x7f9fe003f770 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9fec036000 tx=0x7f9fec008030 comp rx=0 tx=0).stop 2024-05-08T18:08:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.168+0000 7fa003c2a640 1 -- 172.21.15.46:0/3278650004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc06ff30 msgr2=0x7f9ffc0933a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.168+0000 7fa003c2a640 1 --2- 172.21.15.46:0/3278650004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc06ff30 0x7f9ffc0933a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f9ff800c450 tx=0x7f9ff800c910 comp rx=0 tx=0).stop 2024-05-08T18:08:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.168+0000 7fa003c2a640 1 -- 172.21.15.46:0/3278650004 shutdown_connections 2024-05-08T18:08:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.168+0000 7fa003c2a640 1 --2- 172.21.15.46:0/3278650004 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f9fe003d2b0 0x7f9fe003f770 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.168+0000 7fa003c2a640 1 --2- 172.21.15.46:0/3278650004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ffc06ff30 0x7f9ffc0933a0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.168+0000 7fa003c2a640 1 -- 172.21.15.46:0/3278650004 >> 172.21.15.46:0/3278650004 conn(0x7f9ffc09d9b0 msgr2=0x7f9ffc071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.168+0000 7fa003c2a640 1 -- 172.21.15.46:0/3278650004 shutdown_connections 2024-05-08T18:08:04.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.168+0000 7fa003c2a640 1 -- 172.21.15.46:0/3278650004 wait complete. 2024-05-08T18:08:04.178 INFO:tasks.ceph:Scrubbing osd.2 2024-05-08T18:08:04.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-08T18:08:04.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.286+0000 7f897979f640 1 Processor -- start 2024-05-08T18:08:04.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.286+0000 7f897979f640 1 -- start start 2024-05-08T18:08:04.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.286+0000 7f897979f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f897412f200 0x7f897412f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:04.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.286+0000 7f897979f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897409ead0 con 0x7f897412fb20 2024-05-08T18:08:04.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.286+0000 7f897979f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f89740705c0 con 0x7f897412f200 2024-05-08T18:08:04.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.287+0000 7f8973fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f897412f200 0x7f897412f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:04.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.287+0000 7f896bfff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f897412fb20 0x7f897412e180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51838/0 (socket says 172.21.15.46:51838) 2024-05-08T18:08:04.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.287+0000 7f896bfff640 1 -- 172.21.15.46:0/2743971347 learned_addr learned my addr 172.21.15.46:0/2743971347 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:08:04.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.287+0000 7f8973fff640 1 -- 172.21.15.46:0/2743971347 >> v1:172.21.15.46:6789/0 conn(0x7f897412fb20 legacy=0x7f897412e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.287+0000 7f89737fe640 1 -- 172.21.15.46:0/2743971347 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3734968704 0 0) 0x7f897409ead0 con 0x7f897412fb20 2024-05-08T18:08:04.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.287+0000 7f8973fff640 1 -- 172.21.15.46:0/2743971347 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897412ff50 con 0x7f897412f200 2024-05-08T18:08:04.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.288+0000 7f8973fff640 1 --2- 172.21.15.46:0/2743971347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897412f200 0x7f897412f5e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f8964009720 tx=0x7f896402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9195b38db27a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:04.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.288+0000 7f89737fe640 1 -- 172.21.15.46:0/2743971347 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89640316d0 con 0x7f897412f200 2024-05-08T18:08:04.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.288+0000 7f89737fe640 1 -- 172.21.15.46:0/2743971347 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8964037070 con 0x7f897412f200 2024-05-08T18:08:04.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.288+0000 7f89737fe640 1 -- 172.21.15.46:0/2743971347 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89640319c0 con 0x7f897412f200 2024-05-08T18:08:04.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.288+0000 7f897979f640 1 -- 172.21.15.46:0/2743971347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897412f200 msgr2=0x7f897412f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.288+0000 7f897979f640 1 --2- 172.21.15.46:0/2743971347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897412f200 0x7f897412f5e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f8964009720 tx=0x7f896402b840 comp rx=0 tx=0).stop 2024-05-08T18:08:04.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.288+0000 7f897979f640 1 -- 172.21.15.46:0/2743971347 shutdown_connections 2024-05-08T18:08:04.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.288+0000 7f897979f640 1 --2- 172.21.15.46:0/2743971347 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897412f200 0x7f897412f5e0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.288+0000 7f897979f640 1 -- 172.21.15.46:0/2743971347 >> 172.21.15.46:0/2743971347 conn(0x7f897409d470 msgr2=0x7f8974073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:04.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.288+0000 7f897979f640 1 -- 172.21.15.46:0/2743971347 shutdown_connections 2024-05-08T18:08:04.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.288+0000 7f897979f640 1 -- 172.21.15.46:0/2743971347 wait complete. 2024-05-08T18:08:04.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.289+0000 7f897979f640 1 Processor -- start 2024-05-08T18:08:04.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.289+0000 7f897979f640 1 -- start start 2024-05-08T18:08:04.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.290+0000 7f897979f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897412f200 0x7f897411e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:04.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.290+0000 7f897979f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8974070fb0 con 0x7f897412f200 2024-05-08T18:08:04.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.290+0000 7f8973fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897412f200 0x7f897411e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:04.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.290+0000 7f8973fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897412f200 0x7f897411e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43894/0 (socket says 172.21.15.46:43894) 2024-05-08T18:08:04.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.290+0000 7f8973fff640 1 -- 172.21.15.46:0/2598975739 learned_addr learned my addr 172.21.15.46:0/2598975739 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:08:04.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.290+0000 7f8973fff640 1 -- 172.21.15.46:0/2598975739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f897412e430 con 0x7f897412f200 2024-05-08T18:08:04.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.290+0000 7f8973fff640 1 --2- 172.21.15.46:0/2598975739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897412f200 0x7f897411e5c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8964009840 tx=0x7f89640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:04.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.290+0000 7f89717fa640 1 -- 172.21.15.46:0/2598975739 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89640319b0 con 0x7f897412f200 2024-05-08T18:08:04.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.290+0000 7f89717fa640 1 -- 172.21.15.46:0/2598975739 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8964004030 con 0x7f897412f200 2024-05-08T18:08:04.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.290+0000 7f89717fa640 1 -- 172.21.15.46:0/2598975739 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8964004310 con 0x7f897412f200 2024-05-08T18:08:04.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.290+0000 7f897979f640 1 -- 172.21.15.46:0/2598975739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8974121130 con 0x7f897412f200 2024-05-08T18:08:04.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.291+0000 7f897979f640 1 -- 172.21.15.46:0/2598975739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f897411eb00 con 0x7f897412f200 2024-05-08T18:08:04.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.291+0000 7f897979f640 1 -- 172.21.15.46:0/2598975739 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8974121410 con 0x7f897412f200 2024-05-08T18:08:04.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.291+0000 7f89717fa640 1 -- 172.21.15.46:0/2598975739 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f8964003610 con 0x7f897412f200 2024-05-08T18:08:04.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.292+0000 7f89717fa640 1 --2- 172.21.15.46:0/2598975739 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f895403d400 0x7f895403f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:04.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.292+0000 7f89717fa640 1 -- 172.21.15.46:0/2598975739 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f896402f050 con 0x7f897412f200 2024-05-08T18:08:04.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.292+0000 7f89717fa640 1 --2- 172.21.15.46:0/2598975739 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f89540497f0 0x7f895404bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:04.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.292+0000 7f89717fa640 1 -- 172.21.15.46:0/2598975739 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f896402de50 con 0x7f89540497f0 2024-05-08T18:08:04.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.292+0000 7f89717fa640 1 -- 172.21.15.46:0/2598975739 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8964076290 con 0x7f897412f200 2024-05-08T18:08:04.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.292+0000 7f896bfff640 1 --2- 172.21.15.46:0/2598975739 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f895403d400 0x7f895403f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:04.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.292+0000 7f8978f9e640 1 --2- 172.21.15.46:0/2598975739 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f89540497f0 0x7f895404bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:04.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.292+0000 7f896bfff640 1 --2- 172.21.15.46:0/2598975739 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f895403d400 0x7f895403f8c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f895c000c00 tx=0x7f895c0078d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:04.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.292+0000 7f8978f9e640 1 --2- 172.21.15.46:0/2598975739 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f89540497f0 0x7f895404bc90 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:04.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.293+0000 7f89717fa640 1 -- 172.21.15.46:0/2598975739 <== osd.2 v2:172.21.15.50:6811/3039646617 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f896402de50 con 0x7f89540497f0 2024-05-08T18:08:04.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.306+0000 7f897979f640 1 -- 172.21.15.46:0/2598975739 --> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f89740640a0 con 0x7f89540497f0 2024-05-08T18:08:04.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.307+0000 7f89717fa640 1 -- 172.21.15.46:0/2598975739 <== osd.2 v2:172.21.15.50:6811/3039646617 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f89740640a0 con 0x7f89540497f0 2024-05-08T18:08:04.305 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T18:08:04.305 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T18:08:04.306 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T18:08:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.308+0000 7f897979f640 1 -- 172.21.15.46:0/2598975739 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f89540497f0 msgr2=0x7f895404bc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.308+0000 7f897979f640 1 --2- 172.21.15.46:0/2598975739 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f89540497f0 0x7f895404bc90 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.308+0000 7f897979f640 1 -- 172.21.15.46:0/2598975739 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f895403d400 msgr2=0x7f895403f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.308+0000 7f897979f640 1 --2- 172.21.15.46:0/2598975739 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f895403d400 0x7f895403f8c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f895c000c00 tx=0x7f895c0078d0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.308+0000 7f897979f640 1 -- 172.21.15.46:0/2598975739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897412f200 msgr2=0x7f897411e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.308+0000 7f897979f640 1 --2- 172.21.15.46:0/2598975739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897412f200 0x7f897411e5c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8964009840 tx=0x7f89640046c0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.308+0000 7f897979f640 1 -- 172.21.15.46:0/2598975739 shutdown_connections 2024-05-08T18:08:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.308+0000 7f897979f640 1 --2- 172.21.15.46:0/2598975739 >> [v2:172.21.15.50:6811/3039646617,v1:172.21.15.50:6813/3039646617] conn(0x7f89540497f0 0x7f895404bc90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.308+0000 7f897979f640 1 --2- 172.21.15.46:0/2598975739 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f895403d400 0x7f895403f8c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.308+0000 7f897979f640 1 --2- 172.21.15.46:0/2598975739 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897412f200 0x7f897411e5c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.308+0000 7f897979f640 1 -- 172.21.15.46:0/2598975739 >> 172.21.15.46:0/2598975739 conn(0x7f897409d470 msgr2=0x7f897409a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.308+0000 7f897979f640 1 -- 172.21.15.46:0/2598975739 shutdown_connections 2024-05-08T18:08:04.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.308+0000 7f897979f640 1 -- 172.21.15.46:0/2598975739 wait complete. 2024-05-08T18:08:04.316 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-08T18:08:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.421+0000 7f0da8fe8640 1 Processor -- start 2024-05-08T18:08:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.421+0000 7f0da8fe8640 1 -- start start 2024-05-08T18:08:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.422+0000 7f0da8fe8640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0da4130480 0x7f0da4132870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:04.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.422+0000 7f0da8fe8640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0da41144f0 con 0x7f0da4129ea0 2024-05-08T18:08:04.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.422+0000 7f0da8fe8640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f0da4129ca0 con 0x7f0da4130480 2024-05-08T18:08:04.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.422+0000 7f0da1d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0da4130480 0x7f0da4132870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:04.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.422+0000 7f0da259c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0da4129ea0 0x7f0da412a280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51852/0 (socket says 172.21.15.46:51852) 2024-05-08T18:08:04.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.422+0000 7f0da259c640 1 -- 172.21.15.46:0/3544031192 learned_addr learned my addr 172.21.15.46:0/3544031192 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:08:04.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.422+0000 7f0da1d9b640 1 -- 172.21.15.46:0/3544031192 >> v1:172.21.15.46:6789/0 conn(0x7f0da4129ea0 legacy=0x7f0da412a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.422+0000 7f0da159a640 1 -- 172.21.15.46:0/3544031192 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 448692086 0 0) 0x7f0da41144f0 con 0x7f0da4129ea0 2024-05-08T18:08:04.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.422+0000 7f0da1d9b640 1 -- 172.21.15.46:0/3544031192 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da409acc0 con 0x7f0da4130480 2024-05-08T18:08:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.423+0000 7f0da1d9b640 1 --2- 172.21.15.46:0/3544031192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0da4130480 0x7f0da4132870 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009720 tx=0x7f0d8c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f11690603e8b95 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.423+0000 7f0da159a640 1 -- 172.21.15.46:0/3544031192 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d8c0316d0 con 0x7f0da4130480 2024-05-08T18:08:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.423+0000 7f0da159a640 1 -- 172.21.15.46:0/3544031192 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d8c037070 con 0x7f0da4130480 2024-05-08T18:08:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.423+0000 7f0da159a640 1 -- 172.21.15.46:0/3544031192 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d8c0319c0 con 0x7f0da4130480 2024-05-08T18:08:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.423+0000 7f0da8fe8640 1 -- 172.21.15.46:0/3544031192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0da4130480 msgr2=0x7f0da4132870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.423+0000 7f0da8fe8640 1 --2- 172.21.15.46:0/3544031192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0da4130480 0x7f0da4132870 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009720 tx=0x7f0d8c02b840 comp rx=0 tx=0).stop 2024-05-08T18:08:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.423+0000 7f0da8fe8640 1 -- 172.21.15.46:0/3544031192 shutdown_connections 2024-05-08T18:08:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.423+0000 7f0da8fe8640 1 --2- 172.21.15.46:0/3544031192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0da4130480 0x7f0da4132870 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.423+0000 7f0da8fe8640 1 -- 172.21.15.46:0/3544031192 >> 172.21.15.46:0/3544031192 conn(0x7f0da409d7a0 msgr2=0x7f0da409dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.424+0000 7f0da8fe8640 1 -- 172.21.15.46:0/3544031192 shutdown_connections 2024-05-08T18:08:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.424+0000 7f0da8fe8640 1 -- 172.21.15.46:0/3544031192 wait complete. 2024-05-08T18:08:04.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.424+0000 7f0da8fe8640 1 Processor -- start 2024-05-08T18:08:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.424+0000 7f0da8fe8640 1 -- start start 2024-05-08T18:08:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.425+0000 7f0da8fe8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0da4129ea0 0x7f0da4071fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.425+0000 7f0da8fe8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0da4113c10 con 0x7f0da4129ea0 2024-05-08T18:08:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.425+0000 7f0da259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0da4129ea0 0x7f0da4071fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.425+0000 7f0da259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0da4129ea0 0x7f0da4071fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43910/0 (socket says 172.21.15.46:43910) 2024-05-08T18:08:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.425+0000 7f0da259c640 1 -- 172.21.15.46:0/2505631810 learned_addr learned my addr 172.21.15.46:0/2505631810 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:08:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.425+0000 7f0da259c640 1 -- 172.21.15.46:0/2505631810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da411a5e0 con 0x7f0da4129ea0 2024-05-08T18:08:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.425+0000 7f0da259c640 1 --2- 172.21.15.46:0/2505631810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0da4129ea0 0x7f0da4071fb0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00c3d0 tx=0x7f0d9c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.426+0000 7f0d9affd640 1 -- 172.21.15.46:0/2505631810 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d9c0074d0 con 0x7f0da4129ea0 2024-05-08T18:08:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.426+0000 7f0d9affd640 1 -- 172.21.15.46:0/2505631810 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d9c0047e0 con 0x7f0da4129ea0 2024-05-08T18:08:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.426+0000 7f0d9affd640 1 -- 172.21.15.46:0/2505631810 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d9c007670 con 0x7f0da4129ea0 2024-05-08T18:08:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.426+0000 7f0da8fe8640 1 -- 172.21.15.46:0/2505631810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da4119dd0 con 0x7f0da4129ea0 2024-05-08T18:08:04.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.426+0000 7f0da8fe8640 1 -- 172.21.15.46:0/2505631810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da40724f0 con 0x7f0da4129ea0 2024-05-08T18:08:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.427+0000 7f0d9affd640 1 -- 172.21.15.46:0/2505631810 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f0d9c011070 con 0x7f0da4129ea0 2024-05-08T18:08:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.427+0000 7f0da8fe8640 1 -- 172.21.15.46:0/2505631810 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da4130160 con 0x7f0da4129ea0 2024-05-08T18:08:04.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.427+0000 7f0d9affd640 1 --2- 172.21.15.46:0/2505631810 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0d8403d2b0 0x7f0d8403f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.427+0000 7f0d9affd640 1 -- 172.21.15.46:0/2505631810 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f0d9c051a00 con 0x7f0da4129ea0 2024-05-08T18:08:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.427+0000 7f0da1d9b640 1 --2- 172.21.15.46:0/2505631810 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0d8403d2b0 0x7f0d8403f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:04.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.428+0000 7f0da1d9b640 1 --2- 172.21.15.46:0/2505631810 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0d8403d2b0 0x7f0d8403f770 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c036000 tx=0x7f0d8c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:04.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.431+0000 7f0d9affd640 1 -- 172.21.15.46:0/2505631810 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f0da4130160 con 0x7f0da4129ea0 2024-05-08T18:08:04.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.625+0000 7f0da8fe8640 1 -- 172.21.15.46:0/2505631810 --> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f0da409e040 con 0x7f0d8403d2b0 2024-05-08T18:08:04.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.627+0000 7f0d9affd640 1 -- 172.21.15.46:0/2505631810 <== mgr.4098 v2:172.21.15.50:6824/2602185226 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0da409e040 con 0x7f0d8403d2b0 2024-05-08T18:08:04.624 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 2 to deep-scrub 2024-05-08T18:08:04.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.628+0000 7f0da8fe8640 1 -- 172.21.15.46:0/2505631810 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0d8403d2b0 msgr2=0x7f0d8403f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.628+0000 7f0da8fe8640 1 --2- 172.21.15.46:0/2505631810 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0d8403d2b0 0x7f0d8403f770 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c036000 tx=0x7f0d8c008030 comp rx=0 tx=0).stop 2024-05-08T18:08:04.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.628+0000 7f0da8fe8640 1 -- 172.21.15.46:0/2505631810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0da4129ea0 msgr2=0x7f0da4071fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.628+0000 7f0da8fe8640 1 --2- 172.21.15.46:0/2505631810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0da4129ea0 0x7f0da4071fb0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00c3d0 tx=0x7f0d9c00c890 comp rx=0 tx=0).stop 2024-05-08T18:08:04.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.628+0000 7f0da8fe8640 1 -- 172.21.15.46:0/2505631810 shutdown_connections 2024-05-08T18:08:04.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.628+0000 7f0da8fe8640 1 --2- 172.21.15.46:0/2505631810 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f0d8403d2b0 0x7f0d8403f770 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.628+0000 7f0da8fe8640 1 --2- 172.21.15.46:0/2505631810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0da4129ea0 0x7f0da4071fb0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.628+0000 7f0da8fe8640 1 -- 172.21.15.46:0/2505631810 >> 172.21.15.46:0/2505631810 conn(0x7f0da409d7a0 msgr2=0x7f0da40703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:04.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.628+0000 7f0da8fe8640 1 -- 172.21.15.46:0/2505631810 shutdown_connections 2024-05-08T18:08:04.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.628+0000 7f0da8fe8640 1 -- 172.21.15.46:0/2505631810 wait complete. 2024-05-08T18:08:04.637 INFO:tasks.ceph:Scrubbing osd.3 2024-05-08T18:08:04.637 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-08T18:08:04.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.745+0000 7f685683c640 1 Processor -- start 2024-05-08T18:08:04.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.746+0000 7f685683c640 1 -- start start 2024-05-08T18:08:04.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.746+0000 7f685683c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f685006fed0 0x7f685012c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:04.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.746+0000 7f685683c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68500737e0 con 0x7f685012c9c0 2024-05-08T18:08:04.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.746+0000 7f685683c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f6850073250 con 0x7f685006fed0 2024-05-08T18:08:04.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.746+0000 7f684ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f685006fed0 0x7f685012c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:04.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.746+0000 7f684f7fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f685012c9c0 0x7f6850071870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51864/0 (socket says 172.21.15.46:51864) 2024-05-08T18:08:04.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.746+0000 7f684f7fe640 1 -- 172.21.15.46:0/1279801673 learned_addr learned my addr 172.21.15.46:0/1279801673 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:08:04.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.746+0000 7f684ffff640 1 -- 172.21.15.46:0/1279801673 >> v1:172.21.15.46:6789/0 conn(0x7f685012c9c0 legacy=0x7f6850071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.746+0000 7f684ffff640 1 -- 172.21.15.46:0/1279801673 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685012c090 con 0x7f685006fed0 2024-05-08T18:08:04.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.747+0000 7f684ffff640 1 --2- 172.21.15.46:0/1279801673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685006fed0 0x7f685012c480 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6840009720 tx=0x7f684002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cbda6579af2e5a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:04.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.747+0000 7f684effd640 1 -- 172.21.15.46:0/1279801673 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68400316d0 con 0x7f685006fed0 2024-05-08T18:08:04.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.747+0000 7f684effd640 1 -- 172.21.15.46:0/1279801673 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6840037070 con 0x7f685006fed0 2024-05-08T18:08:04.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.747+0000 7f684effd640 1 -- 172.21.15.46:0/1279801673 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68400319c0 con 0x7f685006fed0 2024-05-08T18:08:04.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.748+0000 7f685683c640 1 -- 172.21.15.46:0/1279801673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685006fed0 msgr2=0x7f685012c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.748+0000 7f685683c640 1 --2- 172.21.15.46:0/1279801673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685006fed0 0x7f685012c480 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6840009720 tx=0x7f684002b840 comp rx=0 tx=0).stop 2024-05-08T18:08:04.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.748+0000 7f685683c640 1 -- 172.21.15.46:0/1279801673 shutdown_connections 2024-05-08T18:08:04.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.748+0000 7f685683c640 1 --2- 172.21.15.46:0/1279801673 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685006fed0 0x7f685012c480 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.748+0000 7f685683c640 1 -- 172.21.15.46:0/1279801673 >> 172.21.15.46:0/1279801673 conn(0x7f685009ab60 msgr2=0x7f685006e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:04.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.748+0000 7f685683c640 1 -- 172.21.15.46:0/1279801673 shutdown_connections 2024-05-08T18:08:04.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.748+0000 7f685683c640 1 -- 172.21.15.46:0/1279801673 wait complete. 2024-05-08T18:08:04.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.749+0000 7f685683c640 1 Processor -- start 2024-05-08T18:08:04.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.749+0000 7f685683c640 1 -- start start 2024-05-08T18:08:04.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.749+0000 7f685683c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685006fed0 0x7f68501c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:04.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.749+0000 7f685683c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6850071f80 con 0x7f685006fed0 2024-05-08T18:08:04.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.749+0000 7f684ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685006fed0 0x7f68501c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:04.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.750+0000 7f684ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685006fed0 0x7f68501c1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43926/0 (socket says 172.21.15.46:43926) 2024-05-08T18:08:04.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.750+0000 7f684ffff640 1 -- 172.21.15.46:0/346279623 learned_addr learned my addr 172.21.15.46:0/346279623 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:08:04.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.750+0000 7f684ffff640 1 -- 172.21.15.46:0/346279623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68501261f0 con 0x7f685006fed0 2024-05-08T18:08:04.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.750+0000 7f684ffff640 1 --2- 172.21.15.46:0/346279623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685006fed0 0x7f68501c1ec0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6840034040 tx=0x7f68400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:04.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.750+0000 7f684cff9640 1 -- 172.21.15.46:0/346279623 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6840032640 con 0x7f685006fed0 2024-05-08T18:08:04.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.750+0000 7f684cff9640 1 -- 172.21.15.46:0/346279623 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68400327e0 con 0x7f685006fed0 2024-05-08T18:08:04.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.750+0000 7f685683c640 1 -- 172.21.15.46:0/346279623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6850125a90 con 0x7f685006fed0 2024-05-08T18:08:04.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.750+0000 7f684cff9640 1 -- 172.21.15.46:0/346279623 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6840032ac0 con 0x7f685006fed0 2024-05-08T18:08:04.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.750+0000 7f685683c640 1 -- 172.21.15.46:0/346279623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68501173e0 con 0x7f685006fed0 2024-05-08T18:08:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.751+0000 7f685683c640 1 -- 172.21.15.46:0/346279623 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f68500640a0 con 0x7f685006fed0 2024-05-08T18:08:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.751+0000 7f684cff9640 1 -- 172.21.15.46:0/346279623 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f684004c060 con 0x7f685006fed0 2024-05-08T18:08:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.751+0000 7f684cff9640 1 --2- 172.21.15.46:0/346279623 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f683c03d2b0 0x7f683c03f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.751+0000 7f684cff9640 1 -- 172.21.15.46:0/346279623 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f684002f050 con 0x7f685006fed0 2024-05-08T18:08:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.751+0000 7f684cff9640 1 --2- 172.21.15.46:0/346279623 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x7f683c049720 0x7f683c04bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.751+0000 7f684cff9640 1 -- 172.21.15.46:0/346279623 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f684003ce50 con 0x7f683c049720 2024-05-08T18:08:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.751+0000 7f684cff9640 1 -- 172.21.15.46:0/346279623 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f684007a050 con 0x7f685006fed0 2024-05-08T18:08:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.752+0000 7f6854dd9640 1 --2- 172.21.15.46:0/346279623 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x7f683c049720 0x7f683c04bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.752+0000 7f684f7fe640 1 --2- 172.21.15.46:0/346279623 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f683c03d2b0 0x7f683c03f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.752+0000 7f6854dd9640 1 --2- 172.21.15.46:0/346279623 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x7f683c049720 0x7f683c04bbc0 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.752+0000 7f684f7fe640 1 --2- 172.21.15.46:0/346279623 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f683c03d2b0 0x7f683c03f770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6844007fd0 tx=0x7f6844007aa0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:04.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.753+0000 7f684cff9640 1 -- 172.21.15.46:0/346279623 <== osd.3 v2:172.21.15.50:6800/3130492744 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f684003ce50 con 0x7f683c049720 2024-05-08T18:08:04.764 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.766+0000 7f685683c640 1 -- 172.21.15.46:0/346279623 --> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6850117670 con 0x7f683c049720 2024-05-08T18:08:04.765 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.767+0000 7f684cff9640 1 -- 172.21.15.46:0/346279623 <== osd.3 v2:172.21.15.50:6800/3130492744 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6850117670 con 0x7f683c049720 2024-05-08T18:08:04.765 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-08T18:08:04.765 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T18:08:04.765 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-08T18:08:04.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.767+0000 7f685683c640 1 -- 172.21.15.46:0/346279623 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x7f683c049720 msgr2=0x7f683c04bbc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.767+0000 7f685683c640 1 --2- 172.21.15.46:0/346279623 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x7f683c049720 0x7f683c04bbc0 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.766 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.767+0000 7f685683c640 1 -- 172.21.15.46:0/346279623 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f683c03d2b0 msgr2=0x7f683c03f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.767+0000 7f685683c640 1 --2- 172.21.15.46:0/346279623 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f683c03d2b0 0x7f683c03f770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6844007fd0 tx=0x7f6844007aa0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.767+0000 7f685683c640 1 -- 172.21.15.46:0/346279623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685006fed0 msgr2=0x7f68501c1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.767+0000 7f685683c640 1 --2- 172.21.15.46:0/346279623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685006fed0 0x7f68501c1ec0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6840034040 tx=0x7f68400046c0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.767+0000 7f685683c640 1 -- 172.21.15.46:0/346279623 shutdown_connections 2024-05-08T18:08:04.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.767+0000 7f685683c640 1 --2- 172.21.15.46:0/346279623 >> [v2:172.21.15.50:6800/3130492744,v1:172.21.15.50:6801/3130492744] conn(0x7f683c049720 0x7f683c04bbc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.767+0000 7f685683c640 1 --2- 172.21.15.46:0/346279623 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f683c03d2b0 0x7f683c03f770 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.767+0000 7f685683c640 1 --2- 172.21.15.46:0/346279623 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f685006fed0 0x7f68501c1ec0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.767+0000 7f685683c640 1 -- 172.21.15.46:0/346279623 >> 172.21.15.46:0/346279623 conn(0x7f685009ab60 msgr2=0x7f685009b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:04.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.767+0000 7f685683c640 1 -- 172.21.15.46:0/346279623 shutdown_connections 2024-05-08T18:08:04.767 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.767+0000 7f685683c640 1 -- 172.21.15.46:0/346279623 wait complete. 2024-05-08T18:08:04.776 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-08T18:08:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.880+0000 7fb9f8895640 1 Processor -- start 2024-05-08T18:08:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.880+0000 7fb9f8895640 1 -- start start 2024-05-08T18:08:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.881+0000 7fb9f8895640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb9f412fed0 0x7fb9f4134910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:04.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.881+0000 7fb9f8895640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9f406f050 con 0x7fb9f412f3e0 2024-05-08T18:08:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.881+0000 7fb9f8895640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fb9f40707b0 con 0x7fb9f412fed0 2024-05-08T18:08:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.881+0000 7fb9f1d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb9f412fed0 0x7fb9f4134910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.881+0000 7fb9f259c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb9f412f3e0 0x7fb9f412f7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51872/0 (socket says 172.21.15.46:51872) 2024-05-08T18:08:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.881+0000 7fb9f259c640 1 -- 172.21.15.46:0/671679272 learned_addr learned my addr 172.21.15.46:0/671679272 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:08:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.881+0000 7fb9f1d9b640 1 -- 172.21.15.46:0/671679272 >> v1:172.21.15.46:6789/0 conn(0x7fb9f412f3e0 legacy=0x7fb9f412f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.881+0000 7fb9f1d9b640 1 -- 172.21.15.46:0/671679272 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f409b440 con 0x7fb9f412fed0 2024-05-08T18:08:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.882+0000 7fb9f1d9b640 1 --2- 172.21.15.46:0/671679272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f412fed0 0x7fb9f4134910 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009720 tx=0x7fb9dc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1364ccd62ff50d67 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.882+0000 7fb9f159a640 1 -- 172.21.15.46:0/671679272 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9dc0316d0 con 0x7fb9f412fed0 2024-05-08T18:08:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.882+0000 7fb9f159a640 1 -- 172.21.15.46:0/671679272 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9dc037070 con 0x7fb9f412fed0 2024-05-08T18:08:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.882+0000 7fb9f159a640 1 -- 172.21.15.46:0/671679272 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9dc0319c0 con 0x7fb9f412fed0 2024-05-08T18:08:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.883+0000 7fb9f8895640 1 -- 172.21.15.46:0/671679272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f412fed0 msgr2=0x7fb9f4134910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.883+0000 7fb9f8895640 1 --2- 172.21.15.46:0/671679272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f412fed0 0x7fb9f4134910 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc009720 tx=0x7fb9dc02b840 comp rx=0 tx=0).stop 2024-05-08T18:08:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.883+0000 7fb9f8895640 1 -- 172.21.15.46:0/671679272 shutdown_connections 2024-05-08T18:08:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.883+0000 7fb9f8895640 1 --2- 172.21.15.46:0/671679272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f412fed0 0x7fb9f4134910 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.883+0000 7fb9f8895640 1 -- 172.21.15.46:0/671679272 >> 172.21.15.46:0/671679272 conn(0x7fb9f409c4a0 msgr2=0x7fb9f409c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.883+0000 7fb9f8895640 1 -- 172.21.15.46:0/671679272 shutdown_connections 2024-05-08T18:08:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.883+0000 7fb9f8895640 1 -- 172.21.15.46:0/671679272 wait complete. 2024-05-08T18:08:04.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.884+0000 7fb9f8895640 1 Processor -- start 2024-05-08T18:08:04.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.884+0000 7fb9f8895640 1 -- start start 2024-05-08T18:08:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.884+0000 7fb9f8895640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f412f3e0 0x7fb9f411e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.884+0000 7fb9f8895640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4134e50 con 0x7fb9f412f3e0 2024-05-08T18:08:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.884+0000 7fb9f259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f412f3e0 0x7fb9f411e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.884+0000 7fb9f259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f412f3e0 0x7fb9f411e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43948/0 (socket says 172.21.15.46:43948) 2024-05-08T18:08:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.884+0000 7fb9f259c640 1 -- 172.21.15.46:0/376308156 learned_addr learned my addr 172.21.15.46:0/376308156 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:08:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.884+0000 7fb9f259c640 1 -- 172.21.15.46:0/376308156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f41345e0 con 0x7fb9f412f3e0 2024-05-08T18:08:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.885+0000 7fb9f259c640 1 --2- 172.21.15.46:0/376308156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f412f3e0 0x7fb9f411e170 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb9e400c480 tx=0x7fb9e400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.885+0000 7fb9eaffd640 1 -- 172.21.15.46:0/376308156 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9e4017070 con 0x7fb9f412f3e0 2024-05-08T18:08:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.885+0000 7fb9eaffd640 1 -- 172.21.15.46:0/376308156 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9e4005c10 con 0x7fb9f412f3e0 2024-05-08T18:08:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.885+0000 7fb9f8895640 1 -- 172.21.15.46:0/376308156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f4120ed0 con 0x7fb9f412f3e0 2024-05-08T18:08:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.885+0000 7fb9eaffd640 1 -- 172.21.15.46:0/376308156 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb9e4007070 con 0x7fb9f412f3e0 2024-05-08T18:08:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.885+0000 7fb9f8895640 1 -- 172.21.15.46:0/376308156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f411e6b0 con 0x7fb9f412f3e0 2024-05-08T18:08:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.886+0000 7fb9eaffd640 1 -- 172.21.15.46:0/376308156 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fb9e401a070 con 0x7fb9f412f3e0 2024-05-08T18:08:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.886+0000 7fb9f8895640 1 -- 172.21.15.46:0/376308156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9f40640a0 con 0x7fb9f412f3e0 2024-05-08T18:08:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.886+0000 7fb9eaffd640 1 --2- 172.21.15.46:0/376308156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb9d403d2b0 0x7fb9d403f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.886+0000 7fb9eaffd640 1 -- 172.21.15.46:0/376308156 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fb9e4051030 con 0x7fb9f412f3e0 2024-05-08T18:08:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.886+0000 7fb9f1d9b640 1 --2- 172.21.15.46:0/376308156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb9d403d2b0 0x7fb9d403f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:04.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.887+0000 7fb9f1d9b640 1 --2- 172.21.15.46:0/376308156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb9d403d2b0 0x7fb9d403f770 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc036000 tx=0x7fb9dc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:04.890+0000 7fb9eaffd640 1 -- 172.21.15.46:0/376308156 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fb9e401ac60 con 0x7fb9f412f3e0 2024-05-08T18:08:05.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.078+0000 7fb9f8895640 1 -- 172.21.15.46:0/376308156 --> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fb9f411f360 con 0x7fb9d403d2b0 2024-05-08T18:08:05.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.079+0000 7fb9eaffd640 1 -- 172.21.15.46:0/376308156 <== mgr.4098 v2:172.21.15.50:6824/2602185226 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb9f411f360 con 0x7fb9d403d2b0 2024-05-08T18:08:05.077 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 3 to deep-scrub 2024-05-08T18:08:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.081+0000 7fb9f8895640 1 -- 172.21.15.46:0/376308156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb9d403d2b0 msgr2=0x7fb9d403f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.081+0000 7fb9f8895640 1 --2- 172.21.15.46:0/376308156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb9d403d2b0 0x7fb9d403f770 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc036000 tx=0x7fb9dc008030 comp rx=0 tx=0).stop 2024-05-08T18:08:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.081+0000 7fb9f8895640 1 -- 172.21.15.46:0/376308156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f412f3e0 msgr2=0x7fb9f411e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.081+0000 7fb9f8895640 1 --2- 172.21.15.46:0/376308156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f412f3e0 0x7fb9f411e170 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb9e400c480 tx=0x7fb9e400c940 comp rx=0 tx=0).stop 2024-05-08T18:08:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.081+0000 7fb9f8895640 1 -- 172.21.15.46:0/376308156 shutdown_connections 2024-05-08T18:08:05.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.081+0000 7fb9f8895640 1 --2- 172.21.15.46:0/376308156 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fb9d403d2b0 0x7fb9d403f770 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:05.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.081+0000 7fb9f8895640 1 --2- 172.21.15.46:0/376308156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9f412f3e0 0x7fb9f411e170 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:05.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.081+0000 7fb9f8895640 1 -- 172.21.15.46:0/376308156 >> 172.21.15.46:0/376308156 conn(0x7fb9f409c4a0 msgr2=0x7fb9f4071520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:05.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.081+0000 7fb9f8895640 1 -- 172.21.15.46:0/376308156 shutdown_connections 2024-05-08T18:08:05.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.081+0000 7fb9f8895640 1 -- 172.21.15.46:0/376308156 wait complete. 2024-05-08T18:08:05.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T18:08:05.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.193+0000 7f99fc569640 1 Processor -- start 2024-05-08T18:08:05.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.193+0000 7f99fc569640 1 -- start start 2024-05-08T18:08:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.194+0000 7f99fc569640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f99f412c530 0x7f99f412c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.194+0000 7f99fc569640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f412c230 con 0x7f99f4071d40 2024-05-08T18:08:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.194+0000 7f99fc569640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f99f409ad20 con 0x7f99f412c530 2024-05-08T18:08:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.194+0000 7f99fa305640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f99f412c530 0x7f99f412c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.194+0000 7f99f9b04640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f99f4071d40 0x7f99f4072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:51882/0 (socket says 172.21.15.46:51882) 2024-05-08T18:08:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.194+0000 7f99f9b04640 1 -- 172.21.15.46:0/177713655 learned_addr learned my addr 172.21.15.46:0/177713655 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:08:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.194+0000 7f99fa305640 1 -- 172.21.15.46:0/177713655 >> v1:172.21.15.46:6789/0 conn(0x7f99f4071d40 legacy=0x7f99f4072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:05.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.194+0000 7f99fa305640 1 -- 172.21.15.46:0/177713655 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f4073510 con 0x7f99f412c530 2024-05-08T18:08:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.195+0000 7f99fa305640 1 --2- 172.21.15.46:0/177713655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99f412c530 0x7f99f412c910 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009720 tx=0x7f99f002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c86bc9071acf4c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:05.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.195+0000 7f99f9303640 1 -- 172.21.15.46:0/177713655 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99f00316d0 con 0x7f99f412c530 2024-05-08T18:08:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.195+0000 7f99f9303640 1 -- 172.21.15.46:0/177713655 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f0037070 con 0x7f99f412c530 2024-05-08T18:08:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.195+0000 7f99f9303640 1 -- 172.21.15.46:0/177713655 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99f00319c0 con 0x7f99f412c530 2024-05-08T18:08:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.196+0000 7f99fc569640 1 -- 172.21.15.46:0/177713655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99f412c530 msgr2=0x7f99f412c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.196+0000 7f99fc569640 1 --2- 172.21.15.46:0/177713655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99f412c530 0x7f99f412c910 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009720 tx=0x7f99f002b840 comp rx=0 tx=0).stop 2024-05-08T18:08:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.196+0000 7f99fc569640 1 -- 172.21.15.46:0/177713655 shutdown_connections 2024-05-08T18:08:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.196+0000 7f99fc569640 1 --2- 172.21.15.46:0/177713655 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99f412c530 0x7f99f412c910 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.196+0000 7f99fc569640 1 -- 172.21.15.46:0/177713655 >> 172.21.15.46:0/177713655 conn(0x7f99f409d7a0 msgr2=0x7f99f409dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.196+0000 7f99fc569640 1 -- 172.21.15.46:0/177713655 shutdown_connections 2024-05-08T18:08:05.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.196+0000 7f99fc569640 1 -- 172.21.15.46:0/177713655 wait complete. 2024-05-08T18:08:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.197+0000 7f99fc569640 1 Processor -- start 2024-05-08T18:08:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.197+0000 7f99fc569640 1 -- start start 2024-05-08T18:08:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.197+0000 7f99fc569640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99f4071d40 0x7f99f41c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.197+0000 7f99fc569640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f99f412cfd0 con 0x7f99f4071d40 2024-05-08T18:08:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.197+0000 7f99fa305640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99f4071d40 0x7f99f41c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.197+0000 7f99fa305640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99f4071d40 0x7f99f41c4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43962/0 (socket says 172.21.15.46:43962) 2024-05-08T18:08:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.197+0000 7f99fa305640 1 -- 172.21.15.46:0/2123171474 learned_addr learned my addr 172.21.15.46:0/2123171474 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:08:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.198+0000 7f99fa305640 1 -- 172.21.15.46:0/2123171474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f4071ad0 con 0x7f99f4071d40 2024-05-08T18:08:05.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.198+0000 7f99fa305640 1 --2- 172.21.15.46:0/2123171474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99f4071d40 0x7f99f41c4060 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f99f0034040 tx=0x7f99f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.198+0000 7f99eaffd640 1 -- 172.21.15.46:0/2123171474 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99f0032770 con 0x7f99f4071d40 2024-05-08T18:08:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.198+0000 7f99eaffd640 1 -- 172.21.15.46:0/2123171474 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f0032930 con 0x7f99f4071d40 2024-05-08T18:08:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.198+0000 7f99fc569640 1 -- 172.21.15.46:0/2123171474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f4070ea0 con 0x7f99f4071d40 2024-05-08T18:08:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.198+0000 7f99eaffd640 1 -- 172.21.15.46:0/2123171474 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99f0032c50 con 0x7f99f4071d40 2024-05-08T18:08:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.198+0000 7f99fc569640 1 -- 172.21.15.46:0/2123171474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f40715c0 con 0x7f99f4071d40 2024-05-08T18:08:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.199+0000 7f99eaffd640 1 -- 172.21.15.46:0/2123171474 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f99f004c060 con 0x7f99f4071d40 2024-05-08T18:08:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.199+0000 7f99fc569640 1 -- 172.21.15.46:0/2123171474 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f4117da0 con 0x7f99f4071d40 2024-05-08T18:08:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.199+0000 7f99eaffd640 1 --2- 172.21.15.46:0/2123171474 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f99d803d2b0 0x7f99d803f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:05.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.199+0000 7f99eaffd640 1 -- 172.21.15.46:0/2123171474 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f99f002f050 con 0x7f99f4071d40 2024-05-08T18:08:05.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.200+0000 7f99f9b04640 1 --2- 172.21.15.46:0/2123171474 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f99d803d2b0 0x7f99d803f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:05.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.200+0000 7f99f9b04640 1 --2- 172.21.15.46:0/2123171474 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f99d803d2b0 0x7f99d803f770 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f99e4007fd0 tx=0x7f99e4007aa0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:05.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.204+0000 7f99eaffd640 1 -- 172.21.15.46:0/2123171474 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f99f0040070 con 0x7f99f4071d40 2024-05-08T18:08:05.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.389+0000 7f99fc569640 1 -- 172.21.15.46:0/2123171474 --> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99f40640a0 con 0x7f99d803d2b0 2024-05-08T18:08:05.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.405+0000 7f99eaffd640 1 -- 172.21.15.46:0/2123171474 <== mgr.4098 v2:172.21.15.50:6824/2602185226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665903 (secure 0 0 0) 0x7f99f40640a0 con 0x7f99d803d2b0 2024-05-08T18:08:05.410 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:08:05.415 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T18:08:05.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.409+0000 7f99fc569640 1 -- 172.21.15.46:0/2123171474 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f99d803d2b0 msgr2=0x7f99d803f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:05.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.409+0000 7f99fc569640 1 --2- 172.21.15.46:0/2123171474 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f99d803d2b0 0x7f99d803f770 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f99e4007fd0 tx=0x7f99e4007aa0 comp rx=0 tx=0).stop 2024-05-08T18:08:05.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.409+0000 7f99fc569640 1 -- 172.21.15.46:0/2123171474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99f4071d40 msgr2=0x7f99f41c4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:05.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.409+0000 7f99fc569640 1 --2- 172.21.15.46:0/2123171474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99f4071d40 0x7f99f41c4060 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f99f0034040 tx=0x7f99f00046c0 comp rx=0 tx=0).stop 2024-05-08T18:08:05.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.410+0000 7f99fc569640 1 -- 172.21.15.46:0/2123171474 shutdown_connections 2024-05-08T18:08:05.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.410+0000 7f99fc569640 1 --2- 172.21.15.46:0/2123171474 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f99d803d2b0 0x7f99d803f770 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:05.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.410+0000 7f99fc569640 1 --2- 172.21.15.46:0/2123171474 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99f4071d40 0x7f99f41c4060 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:05.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.410+0000 7f99fc569640 1 -- 172.21.15.46:0/2123171474 >> 172.21.15.46:0/2123171474 conn(0x7f99f409d7a0 msgr2=0x7f99f406f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:05.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.410+0000 7f99fc569640 1 -- 172.21.15.46:0/2123171474 shutdown_connections 2024-05-08T18:08:05.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:05.410+0000 7f99fc569640 1 -- 172.21.15.46:0/2123171474 wait complete. 2024-05-08T18:08:05.419 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-05-08T18:08:03.728418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26416886,"num_objects":227,"num_object_clones":0,"num_object_copies":468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":227,"num_whiteouts":0,"num_read":4834,"num_read_kb":4416,"num_write":2046,"num_write_kb":60584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4431,"ondisk_log_size":4431,"up":658,"acting":658,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":658,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":186032,"kb_used_data":36368,"kb_used_omap":6,"kb_used_meta":149625,"kb_avail":377301328,"statfs":{"total":386547056640,"available":386356559872,"internally_reserved":0,"allocated":37240832,"data_stored":36146252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":153216790},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25815784,"num_objects":9,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":497,"num_read_kb":452,"num_write":384,"num_write_kb":29371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003321"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866950+0000","last_change":"2024-05-08T18:04:43.836900+0000","last_active":"2024-05-08T18:05:30.866950+0000","last_peered":"2024-05-08T18:05:30.866950+0000","last_clean":"2024-05-08T18:05:30.866950+0000","last_became_active":"2024-05-08T18:04:43.836496+0000","last_became_peered":"2024-05-08T18:04:43.836496+0000","last_unstale":"2024-05-08T18:05:30.866950+0000","last_undegraded":"2024-05-08T18:05:30.866950+0000","last_fullsized":"2024-05-08T18:05:30.866950+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:22.454546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'6","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.860763+0000","last_change":"2024-05-08T18:04:45.812340+0000","last_active":"2024-05-08T18:07:53.860763+0000","last_peered":"2024-05-08T18:07:53.860763+0000","last_clean":"2024-05-08T18:07:53.860763+0000","last_became_active":"2024-05-08T18:04:45.811368+0000","last_became_peered":"2024-05-08T18:04:45.811368+0000","last_unstale":"2024-05-08T18:07:53.860763+0000","last_undegraded":"2024-05-08T18:07:53.860763+0000","last_fullsized":"2024-05-08T18:07:53.860763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:36:45.459800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.206267+0000","last_change":"2024-05-08T18:04:43.835466+0000","last_active":"2024-05-08T18:06:50.206267+0000","last_peered":"2024-05-08T18:06:50.206267+0000","last_clean":"2024-05-08T18:06:50.206267+0000","last_became_active":"2024-05-08T18:04:43.834612+0000","last_became_peered":"2024-05-08T18:04:43.834612+0000","last_unstale":"2024-05-08T18:06:50.206267+0000","last_undegraded":"2024-05-08T18:06:50.206267+0000","last_fullsized":"2024-05-08T18:06:50.206267+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:46.422109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"29'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.546966+0000","last_change":"2024-05-08T18:04:45.828749+0000","last_active":"2024-05-08T18:06:09.546966+0000","last_peered":"2024-05-08T18:06:09.546966+0000","last_clean":"2024-05-08T18:06:09.546966+0000","last_became_active":"2024-05-08T18:04:45.827965+0000","last_became_peered":"2024-05-08T18:04:45.827965+0000","last_unstale":"2024-05-08T18:06:09.546966+0000","last_undegraded":"2024-05-08T18:06:09.546966+0000","last_fullsized":"2024-05-08T18:06:09.546966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:56:23.117814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861383+0000","last_change":"2024-05-08T18:04:43.843806+0000","last_active":"2024-05-08T18:05:30.861383+0000","last_peered":"2024-05-08T18:05:30.861383+0000","last_clean":"2024-05-08T18:05:30.861383+0000","last_became_active":"2024-05-08T18:04:43.837456+0000","last_became_peered":"2024-05-08T18:04:43.837456+0000","last_unstale":"2024-05-08T18:05:30.861383+0000","last_undegraded":"2024-05-08T18:05:30.861383+0000","last_fullsized":"2024-05-08T18:05:30.861383+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:00:24.839225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"29'12","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063738+0000","last_change":"2024-05-08T18:04:45.825315+0000","last_active":"2024-05-08T18:07:54.063738+0000","last_peered":"2024-05-08T18:07:54.063738+0000","last_clean":"2024-05-08T18:07:54.063738+0000","last_became_active":"2024-05-08T18:04:45.825022+0000","last_became_peered":"2024-05-08T18:04:45.825022+0000","last_unstale":"2024-05-08T18:07:54.063738+0000","last_undegraded":"2024-05-08T18:07:54.063738+0000","last_fullsized":"2024-05-08T18:07:54.063738+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:45:53.925354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863324+0000","last_change":"2024-05-08T18:04:43.842478+0000","last_active":"2024-05-08T18:05:30.863324+0000","last_peered":"2024-05-08T18:05:30.863324+0000","last_clean":"2024-05-08T18:05:30.863324+0000","last_became_active":"2024-05-08T18:04:43.842077+0000","last_became_peered":"2024-05-08T18:04:43.842077+0000","last_unstale":"2024-05-08T18:05:30.863324+0000","last_undegraded":"2024-05-08T18:05:30.863324+0000","last_fullsized":"2024-05-08T18:05:30.863324+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:06.338736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"29'16","reported_seq":70,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.035049+0000","last_change":"2024-05-08T18:04:45.824870+0000","last_active":"2024-05-08T18:07:54.035049+0000","last_peered":"2024-05-08T18:07:54.035049+0000","last_clean":"2024-05-08T18:07:54.035049+0000","last_became_active":"2024-05-08T18:04:45.824581+0000","last_became_peered":"2024-05-08T18:04:45.824581+0000","last_unstale":"2024-05-08T18:07:54.035049+0000","last_undegraded":"2024-05-08T18:07:54.035049+0000","last_fullsized":"2024-05-08T18:07:54.035049+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:48.934445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859690+0000","last_change":"2024-05-08T18:04:43.833461+0000","last_active":"2024-05-08T18:05:30.859690+0000","last_peered":"2024-05-08T18:05:30.859690+0000","last_clean":"2024-05-08T18:05:30.859690+0000","last_became_active":"2024-05-08T18:04:43.832722+0000","last_became_peered":"2024-05-08T18:04:43.832722+0000","last_unstale":"2024-05-08T18:05:30.859690+0000","last_undegraded":"2024-05-08T18:05:30.859690+0000","last_fullsized":"2024-05-08T18:05:30.859690+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:11:15.182813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"29'8","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.738600+0000","last_change":"2024-05-08T18:04:45.808678+0000","last_active":"2024-05-08T18:06:09.738600+0000","last_peered":"2024-05-08T18:06:09.738600+0000","last_clean":"2024-05-08T18:06:09.738600+0000","last_became_active":"2024-05-08T18:04:45.808383+0000","last_became_peered":"2024-05-08T18:04:45.808383+0000","last_unstale":"2024-05-08T18:06:09.738600+0000","last_undegraded":"2024-05-08T18:06:09.738600+0000","last_fullsized":"2024-05-08T18:06:09.738600+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:02.599918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862756+0000","last_change":"2024-05-08T18:04:43.841284+0000","last_active":"2024-05-08T18:05:30.862756+0000","last_peered":"2024-05-08T18:05:30.862756+0000","last_clean":"2024-05-08T18:05:30.862756+0000","last_became_active":"2024-05-08T18:04:43.840851+0000","last_became_peered":"2024-05-08T18:04:43.840851+0000","last_unstale":"2024-05-08T18:05:30.862756+0000","last_undegraded":"2024-05-08T18:05:30.862756+0000","last_fullsized":"2024-05-08T18:05:30.862756+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:10.360847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063767+0000","last_change":"2024-05-08T18:04:45.829481+0000","last_active":"2024-05-08T18:07:54.063767+0000","last_peered":"2024-05-08T18:07:54.063767+0000","last_clean":"2024-05-08T18:07:54.063767+0000","last_became_active":"2024-05-08T18:04:45.828334+0000","last_became_peered":"2024-05-08T18:04:45.828334+0000","last_unstale":"2024-05-08T18:07:54.063767+0000","last_undegraded":"2024-05-08T18:07:54.063767+0000","last_fullsized":"2024-05-08T18:07:54.063767+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:20:31.411889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.978496+0000","last_change":"2024-05-08T18:04:43.833826+0000","last_active":"2024-05-08T18:07:53.978496+0000","last_peered":"2024-05-08T18:07:53.978496+0000","last_clean":"2024-05-08T18:07:53.978496+0000","last_became_active":"2024-05-08T18:04:43.833437+0000","last_became_peered":"2024-05-08T18:04:43.833437+0000","last_unstale":"2024-05-08T18:07:53.978496+0000","last_undegraded":"2024-05-08T18:07:53.978496+0000","last_fullsized":"2024-05-08T18:07:53.978496+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:46.740116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.868015+0000","last_change":"2024-05-08T18:04:45.812158+0000","last_active":"2024-05-08T18:05:30.868015+0000","last_peered":"2024-05-08T18:05:30.868015+0000","last_clean":"2024-05-08T18:05:30.868015+0000","last_became_active":"2024-05-08T18:04:45.811884+0000","last_became_peered":"2024-05-08T18:04:45.811884+0000","last_unstale":"2024-05-08T18:05:30.868015+0000","last_undegraded":"2024-05-08T18:05:30.868015+0000","last_fullsized":"2024-05-08T18:05:30.868015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:30.458103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858546+0000","last_change":"2024-05-08T18:04:43.821005+0000","last_active":"2024-05-08T18:05:30.858546+0000","last_peered":"2024-05-08T18:05:30.858546+0000","last_clean":"2024-05-08T18:05:30.858546+0000","last_became_active":"2024-05-08T18:04:43.820312+0000","last_became_peered":"2024-05-08T18:04:43.820312+0000","last_unstale":"2024-05-08T18:05:30.858546+0000","last_undegraded":"2024-05-08T18:05:30.858546+0000","last_fullsized":"2024-05-08T18:05:30.858546+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:40:40.407240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"29'7","reported_seq":61,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070848+0000","last_change":"2024-05-08T18:04:45.823448+0000","last_active":"2024-05-08T18:07:54.070848+0000","last_peered":"2024-05-08T18:07:54.070848+0000","last_clean":"2024-05-08T18:07:54.070848+0000","last_became_active":"2024-05-08T18:04:45.822620+0000","last_became_peered":"2024-05-08T18:04:45.822620+0000","last_unstale":"2024-05-08T18:07:54.070848+0000","last_undegraded":"2024-05-08T18:07:54.070848+0000","last_fullsized":"2024-05-08T18:07:54.070848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:08.184602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"29'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995728+0000","last_change":"2024-05-08T18:04:45.826814+0000","last_active":"2024-05-08T18:07:53.995728+0000","last_peered":"2024-05-08T18:07:53.995728+0000","last_clean":"2024-05-08T18:07:53.995728+0000","last_became_active":"2024-05-08T18:04:45.826493+0000","last_became_peered":"2024-05-08T18:04:45.826493+0000","last_unstale":"2024-05-08T18:07:53.995728+0000","last_undegraded":"2024-05-08T18:07:53.995728+0000","last_fullsized":"2024-05-08T18:07:53.995728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:12.401514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866321+0000","last_change":"2024-05-08T18:04:43.838700+0000","last_active":"2024-05-08T18:05:30.866321+0000","last_peered":"2024-05-08T18:05:30.866321+0000","last_clean":"2024-05-08T18:05:30.866321+0000","last_became_active":"2024-05-08T18:04:43.838332+0000","last_became_peered":"2024-05-08T18:04:43.838332+0000","last_unstale":"2024-05-08T18:05:30.866321+0000","last_undegraded":"2024-05-08T18:05:30.866321+0000","last_fullsized":"2024-05-08T18:05:30.866321+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:01:49.620924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"29'4","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399796+0000","last_change":"2024-05-08T18:04:45.807732+0000","last_active":"2024-05-08T18:06:53.399796+0000","last_peered":"2024-05-08T18:06:53.399796+0000","last_clean":"2024-05-08T18:06:53.399796+0000","last_became_active":"2024-05-08T18:04:45.807438+0000","last_became_peered":"2024-05-08T18:04:45.807438+0000","last_unstale":"2024-05-08T18:06:53.399796+0000","last_undegraded":"2024-05-08T18:06:53.399796+0000","last_fullsized":"2024-05-08T18:06:53.399796+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:27:20.114031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867876+0000","last_change":"2024-05-08T18:04:43.843599+0000","last_active":"2024-05-08T18:05:30.867876+0000","last_peered":"2024-05-08T18:05:30.867876+0000","last_clean":"2024-05-08T18:05:30.867876+0000","last_became_active":"2024-05-08T18:04:43.842741+0000","last_became_peered":"2024-05-08T18:04:43.842741+0000","last_unstale":"2024-05-08T18:05:30.867876+0000","last_undegraded":"2024-05-08T18:05:30.867876+0000","last_fullsized":"2024-05-08T18:05:30.867876+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:54.711833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"29'11","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.002969+0000","last_change":"2024-05-08T18:04:45.811141+0000","last_active":"2024-05-08T18:07:54.002969+0000","last_peered":"2024-05-08T18:07:54.002969+0000","last_clean":"2024-05-08T18:07:54.002969+0000","last_became_active":"2024-05-08T18:04:45.810827+0000","last_became_peered":"2024-05-08T18:04:45.810827+0000","last_unstale":"2024-05-08T18:07:54.002969+0000","last_undegraded":"2024-05-08T18:07:54.002969+0000","last_fullsized":"2024-05-08T18:07:54.002969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:26.952570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"29'3","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746208+0000","last_change":"2024-05-08T18:04:43.838115+0000","last_active":"2024-05-08T18:07:08.746208+0000","last_peered":"2024-05-08T18:07:08.746208+0000","last_clean":"2024-05-08T18:07:08.746208+0000","last_became_active":"2024-05-08T18:04:43.837706+0000","last_became_peered":"2024-05-08T18:04:43.837706+0000","last_unstale":"2024-05-08T18:07:08.746208+0000","last_undegraded":"2024-05-08T18:07:08.746208+0000","last_fullsized":"2024-05-08T18:07:08.746208+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:26.918679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"29'8","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.738522+0000","last_change":"2024-05-08T18:04:45.822292+0000","last_active":"2024-05-08T18:06:09.738522+0000","last_peered":"2024-05-08T18:06:09.738522+0000","last_clean":"2024-05-08T18:06:09.738522+0000","last_became_active":"2024-05-08T18:04:45.822000+0000","last_became_peered":"2024-05-08T18:04:45.822000+0000","last_unstale":"2024-05-08T18:06:09.738522+0000","last_undegraded":"2024-05-08T18:06:09.738522+0000","last_fullsized":"2024-05-08T18:06:09.738522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:08.184602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866478+0000","last_change":"2024-05-08T18:04:43.841486+0000","last_active":"2024-05-08T18:05:30.866478+0000","last_peered":"2024-05-08T18:05:30.866478+0000","last_clean":"2024-05-08T18:05:30.866478+0000","last_became_active":"2024-05-08T18:04:43.841063+0000","last_became_peered":"2024-05-08T18:04:43.841063+0000","last_unstale":"2024-05-08T18:05:30.866478+0000","last_undegraded":"2024-05-08T18:05:30.866478+0000","last_fullsized":"2024-05-08T18:05:30.866478+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:05:33.776915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"29'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.116184+0000","last_change":"2024-05-08T18:04:45.824006+0000","last_active":"2024-05-08T18:07:54.116184+0000","last_peered":"2024-05-08T18:07:54.116184+0000","last_clean":"2024-05-08T18:07:54.116184+0000","last_became_active":"2024-05-08T18:04:45.823611+0000","last_became_peered":"2024-05-08T18:04:45.823611+0000","last_unstale":"2024-05-08T18:07:54.116184+0000","last_undegraded":"2024-05-08T18:07:54.116184+0000","last_fullsized":"2024-05-08T18:07:54.116184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:02.599918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860254+0000","last_change":"2024-05-08T18:04:43.835685+0000","last_active":"2024-05-08T18:05:30.860254+0000","last_peered":"2024-05-08T18:05:30.860254+0000","last_clean":"2024-05-08T18:05:30.860254+0000","last_became_active":"2024-05-08T18:04:43.834456+0000","last_became_peered":"2024-05-08T18:04:43.834456+0000","last_unstale":"2024-05-08T18:05:30.860254+0000","last_undegraded":"2024-05-08T18:05:30.860254+0000","last_fullsized":"2024-05-08T18:05:30.860254+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:50.043665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399432+0000","last_change":"2024-05-08T18:04:45.808763+0000","last_active":"2024-05-08T18:06:53.399432+0000","last_peered":"2024-05-08T18:06:53.399432+0000","last_clean":"2024-05-08T18:06:53.399432+0000","last_became_active":"2024-05-08T18:04:45.808416+0000","last_became_peered":"2024-05-08T18:04:45.808416+0000","last_unstale":"2024-05-08T18:06:53.399432+0000","last_undegraded":"2024-05-08T18:06:53.399432+0000","last_fullsized":"2024-05-08T18:06:53.399432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:35.780903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'4","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858633+0000","last_change":"2024-05-08T18:04:43.845458+0000","last_active":"2024-05-08T18:05:30.858633+0000","last_peered":"2024-05-08T18:05:30.858633+0000","last_clean":"2024-05-08T18:05:30.858633+0000","last_became_active":"2024-05-08T18:04:43.844702+0000","last_became_peered":"2024-05-08T18:04:43.844702+0000","last_unstale":"2024-05-08T18:05:30.858633+0000","last_undegraded":"2024-05-08T18:05:30.858633+0000","last_fullsized":"2024-05-08T18:05:30.858633+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:30:58.229253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"29'35","reported_seq":99,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.016701+0000","last_change":"2024-05-08T18:04:53.849775+0000","last_active":"2024-05-08T18:07:54.016701+0000","last_peered":"2024-05-08T18:07:54.016701+0000","last_clean":"2024-05-08T18:07:54.016701+0000","last_became_active":"2024-05-08T18:04:53.849475+0000","last_became_peered":"2024-05-08T18:04:53.849475+0000","last_unstale":"2024-05-08T18:07:54.016701+0000","last_undegraded":"2024-05-08T18:07:54.016701+0000","last_fullsized":"2024-05-08T18:07:54.016701+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:12.918578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"29'12","reported_seq":66,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995839+0000","last_change":"2024-05-08T18:04:45.824289+0000","last_active":"2024-05-08T18:07:53.995839+0000","last_peered":"2024-05-08T18:07:53.995839+0000","last_clean":"2024-05-08T18:07:53.995839+0000","last_became_active":"2024-05-08T18:04:45.824005+0000","last_became_peered":"2024-05-08T18:04:45.824005+0000","last_unstale":"2024-05-08T18:07:53.995839+0000","last_undegraded":"2024-05-08T18:07:53.995839+0000","last_fullsized":"2024-05-08T18:07:53.995839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:45.743430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859173+0000","last_change":"2024-05-08T18:04:43.834249+0000","last_active":"2024-05-08T18:05:30.859173+0000","last_peered":"2024-05-08T18:05:30.859173+0000","last_clean":"2024-05-08T18:05:30.859173+0000","last_became_active":"2024-05-08T18:04:43.833822+0000","last_became_peered":"2024-05-08T18:04:43.833822+0000","last_unstale":"2024-05-08T18:05:30.859173+0000","last_undegraded":"2024-05-08T18:05:30.859173+0000","last_fullsized":"2024-05-08T18:05:30.859173+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:00:37.867739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859597+0000","last_change":"2024-05-08T18:04:47.820083+0000","last_active":"2024-05-08T18:05:30.859597+0000","last_peered":"2024-05-08T18:05:30.859597+0000","last_clean":"2024-05-08T18:05:30.859597+0000","last_became_active":"2024-05-08T18:04:47.819682+0000","last_became_peered":"2024-05-08T18:04:47.819682+0000","last_unstale":"2024-05-08T18:05:30.859597+0000","last_undegraded":"2024-05-08T18:05:30.859597+0000","last_fullsized":"2024-05-08T18:05:30.859597+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:00:55.772655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"29'31","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.748532+0000","last_change":"2024-05-08T18:04:49.833877+0000","last_active":"2024-05-08T18:07:18.748532+0000","last_peered":"2024-05-08T18:07:18.748532+0000","last_clean":"2024-05-08T18:07:18.748532+0000","last_became_active":"2024-05-08T18:04:49.832148+0000","last_became_peered":"2024-05-08T18:04:49.832148+0000","last_unstale":"2024-05-08T18:07:18.748532+0000","last_undegraded":"2024-05-08T18:07:18.748532+0000","last_fullsized":"2024-05-08T18:07:18.748532+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:20:21.388280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5807,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":31,"num_read_kb":25,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866106+0000","last_change":"2024-05-08T18:04:51.845281+0000","last_active":"2024-05-08T18:05:30.866106+0000","last_peered":"2024-05-08T18:05:30.866106+0000","last_clean":"2024-05-08T18:05:30.866106+0000","last_became_active":"2024-05-08T18:04:51.844901+0000","last_became_peered":"2024-05-08T18:04:51.844901+0000","last_unstale":"2024-05-08T18:05:30.866106+0000","last_undegraded":"2024-05-08T18:05:30.866106+0000","last_fullsized":"2024-05-08T18:05:30.866106+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:46.002263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"29'11","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.552238+0000","last_change":"2024-05-08T18:04:53.849933+0000","last_active":"2024-05-08T18:06:09.552238+0000","last_peered":"2024-05-08T18:06:09.552238+0000","last_clean":"2024-05-08T18:06:09.552238+0000","last_became_active":"2024-05-08T18:04:53.849579+0000","last_became_peered":"2024-05-08T18:04:53.849579+0000","last_unstale":"2024-05-08T18:06:09.552238+0000","last_undegraded":"2024-05-08T18:06:09.552238+0000","last_fullsized":"2024-05-08T18:06:09.552238+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:42:45.987685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"29'6","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.064004+0000","last_change":"2024-05-08T18:04:45.826976+0000","last_active":"2024-05-08T18:07:54.064004+0000","last_peered":"2024-05-08T18:07:54.064004+0000","last_clean":"2024-05-08T18:07:54.064004+0000","last_became_active":"2024-05-08T18:04:45.825162+0000","last_became_peered":"2024-05-08T18:04:45.825162+0000","last_unstale":"2024-05-08T18:07:54.064004+0000","last_undegraded":"2024-05-08T18:07:54.064004+0000","last_fullsized":"2024-05-08T18:07:54.064004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:36.088260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862438+0000","last_change":"2024-05-08T18:04:43.827844+0000","last_active":"2024-05-08T18:05:30.862438+0000","last_peered":"2024-05-08T18:05:30.862438+0000","last_clean":"2024-05-08T18:05:30.862438+0000","last_became_active":"2024-05-08T18:04:43.827374+0000","last_became_peered":"2024-05-08T18:04:43.827374+0000","last_unstale":"2024-05-08T18:05:30.862438+0000","last_undegraded":"2024-05-08T18:05:30.862438+0000","last_fullsized":"2024-05-08T18:05:30.862438+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:46:26.918679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:33.732471+0000","last_change":"2024-05-08T18:05:30.868535+0000","last_active":"2024-05-08T18:05:33.732471+0000","last_peered":"2024-05-08T18:05:33.732471+0000","last_clean":"2024-05-08T18:05:33.732471+0000","last_became_active":"2024-05-08T18:05:30.868191+0000","last_became_peered":"2024-05-08T18:05:30.868191+0000","last_unstale":"2024-05-08T18:05:33.732471+0000","last_undegraded":"2024-05-08T18:05:33.732471+0000","last_fullsized":"2024-05-08T18:05:33.732471+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:19.468033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"27'6","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862185+0000","last_change":"2024-05-08T18:04:49.826617+0000","last_active":"2024-05-08T18:05:30.862185+0000","last_peered":"2024-05-08T18:05:30.862185+0000","last_clean":"2024-05-08T18:05:30.862185+0000","last_became_active":"2024-05-08T18:04:49.825265+0000","last_became_peered":"2024-05-08T18:04:49.825265+0000","last_unstale":"2024-05-08T18:05:30.862185+0000","last_undegraded":"2024-05-08T18:05:30.862185+0000","last_fullsized":"2024-05-08T18:05:30.862185+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:35.867914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862381+0000","last_change":"2024-05-08T18:04:51.834892+0000","last_active":"2024-05-08T18:05:30.862381+0000","last_peered":"2024-05-08T18:05:30.862381+0000","last_clean":"2024-05-08T18:05:30.862381+0000","last_became_active":"2024-05-08T18:04:51.833523+0000","last_became_peered":"2024-05-08T18:04:51.833523+0000","last_unstale":"2024-05-08T18:05:30.862381+0000","last_undegraded":"2024-05-08T18:05:30.862381+0000","last_fullsized":"2024-05-08T18:05:30.862381+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:01.176258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'2","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858040+0000","last_change":"2024-05-08T18:04:53.840760+0000","last_active":"2024-05-08T18:05:30.858040+0000","last_peered":"2024-05-08T18:05:30.858040+0000","last_clean":"2024-05-08T18:05:30.858040+0000","last_became_active":"2024-05-08T18:04:53.840395+0000","last_became_peered":"2024-05-08T18:04:53.840395+0000","last_unstale":"2024-05-08T18:05:30.858040+0000","last_undegraded":"2024-05-08T18:05:30.858040+0000","last_fullsized":"2024-05-08T18:05:30.858040+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:54.850231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:33.732213+0000","last_change":"2024-05-08T18:05:30.876470+0000","last_active":"2024-05-08T18:05:33.732213+0000","last_peered":"2024-05-08T18:05:33.732213+0000","last_clean":"2024-05-08T18:05:33.732213+0000","last_became_active":"2024-05-08T18:05:30.869162+0000","last_became_peered":"2024-05-08T18:05:30.869162+0000","last_unstale":"2024-05-08T18:05:33.732213+0000","last_undegraded":"2024-05-08T18:05:33.732213+0000","last_fullsized":"2024-05-08T18:05:33.732213+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:06:04.889530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":45,"num_write":24,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857600+0000","last_change":"2024-05-08T18:04:43.844847+0000","last_active":"2024-05-08T18:05:30.857600+0000","last_peered":"2024-05-08T18:05:30.857600+0000","last_clean":"2024-05-08T18:05:30.857600+0000","last_became_active":"2024-05-08T18:04:43.838666+0000","last_became_peered":"2024-05-08T18:04:43.838666+0000","last_unstale":"2024-05-08T18:05:30.857600+0000","last_undegraded":"2024-05-08T18:05:30.857600+0000","last_fullsized":"2024-05-08T18:05:30.857600+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:32.496387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861628+0000","last_change":"2024-05-08T18:04:47.819743+0000","last_active":"2024-05-08T18:05:30.861628+0000","last_peered":"2024-05-08T18:05:30.861628+0000","last_clean":"2024-05-08T18:05:30.861628+0000","last_became_active":"2024-05-08T18:04:47.819169+0000","last_became_peered":"2024-05-08T18:04:47.819169+0000","last_unstale":"2024-05-08T18:05:30.861628+0000","last_undegraded":"2024-05-08T18:05:30.861628+0000","last_fullsized":"2024-05-08T18:05:30.861628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:15:57.213130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"29'12","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.746941+0000","last_change":"2024-05-08T18:04:49.831839+0000","last_active":"2024-05-08T18:07:13.746941+0000","last_peered":"2024-05-08T18:07:13.746941+0000","last_clean":"2024-05-08T18:07:13.746941+0000","last_became_active":"2024-05-08T18:04:49.831500+0000","last_became_peered":"2024-05-08T18:04:49.831500+0000","last_unstale":"2024-05-08T18:07:13.746941+0000","last_undegraded":"2024-05-08T18:07:13.746941+0000","last_fullsized":"2024-05-08T18:07:13.746941+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:19:01.714463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"29'282","reported_seq":381,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903208+0000","last_change":"2024-05-08T18:04:51.849859+0000","last_active":"2024-05-08T18:07:52.903208+0000","last_peered":"2024-05-08T18:07:52.903208+0000","last_clean":"2024-05-08T18:07:52.903208+0000","last_became_active":"2024-05-08T18:04:51.849301+0000","last_became_peered":"2024-05-08T18:04:51.849301+0000","last_unstale":"2024-05-08T18:07:52.903208+0000","last_undegraded":"2024-05-08T18:07:52.903208+0000","last_fullsized":"2024-05-08T18:07:52.903208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:02:26.756919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862433+0000","last_change":"2024-05-08T18:04:53.840922+0000","last_active":"2024-05-08T18:05:30.862433+0000","last_peered":"2024-05-08T18:05:30.862433+0000","last_clean":"2024-05-08T18:05:30.862433+0000","last_became_active":"2024-05-08T18:04:53.840559+0000","last_became_peered":"2024-05-08T18:04:53.840559+0000","last_unstale":"2024-05-08T18:05:30.862433+0000","last_undegraded":"2024-05-08T18:05:30.862433+0000","last_fullsized":"2024-05-08T18:05:30.862433+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:24.814022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"29'6","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.035260+0000","last_change":"2024-05-08T18:04:45.812045+0000","last_active":"2024-05-08T18:07:54.035260+0000","last_peered":"2024-05-08T18:07:54.035260+0000","last_clean":"2024-05-08T18:07:54.035260+0000","last_became_active":"2024-05-08T18:04:45.811570+0000","last_became_peered":"2024-05-08T18:04:45.811570+0000","last_unstale":"2024-05-08T18:07:54.035260+0000","last_undegraded":"2024-05-08T18:07:54.035260+0000","last_fullsized":"2024-05-08T18:07:54.035260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:22:19.585204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.987908+0000","last_change":"2024-05-08T18:04:43.838356+0000","last_active":"2024-05-08T18:07:53.987908+0000","last_peered":"2024-05-08T18:07:53.987908+0000","last_clean":"2024-05-08T18:07:53.987908+0000","last_became_active":"2024-05-08T18:04:43.836410+0000","last_became_peered":"2024-05-08T18:04:43.836410+0000","last_unstale":"2024-05-08T18:07:53.987908+0000","last_undegraded":"2024-05-08T18:07:53.987908+0000","last_fullsized":"2024-05-08T18:07:53.987908+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:59:26.742382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860150+0000","last_change":"2024-05-08T18:04:47.819091+0000","last_active":"2024-05-08T18:05:30.860150+0000","last_peered":"2024-05-08T18:05:30.860150+0000","last_clean":"2024-05-08T18:05:30.860150+0000","last_became_active":"2024-05-08T18:04:47.818797+0000","last_became_peered":"2024-05-08T18:04:47.818797+0000","last_unstale":"2024-05-08T18:05:30.860150+0000","last_undegraded":"2024-05-08T18:05:30.860150+0000","last_fullsized":"2024-05-08T18:05:30.860150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:44.738079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"29'31","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.948644+0000","last_change":"2024-05-08T18:04:49.831750+0000","last_active":"2024-05-08T18:06:51.948644+0000","last_peered":"2024-05-08T18:06:51.948644+0000","last_clean":"2024-05-08T18:06:51.948644+0000","last_became_active":"2024-05-08T18:04:49.831379+0000","last_became_peered":"2024-05-08T18:04:49.831379+0000","last_unstale":"2024-05-08T18:06:51.948644+0000","last_undegraded":"2024-05-08T18:06:51.948644+0000","last_fullsized":"2024-05-08T18:06:51.948644+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:15:59.220647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"29'282","reported_seq":441,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.067044+0000","last_change":"2024-05-08T18:04:51.827294+0000","last_active":"2024-05-08T18:07:54.067044+0000","last_peered":"2024-05-08T18:07:54.067044+0000","last_clean":"2024-05-08T18:07:54.067044+0000","last_became_active":"2024-05-08T18:04:51.826926+0000","last_became_peered":"2024-05-08T18:04:51.826926+0000","last_unstale":"2024-05-08T18:07:54.067044+0000","last_undegraded":"2024-05-08T18:07:54.067044+0000","last_fullsized":"2024-05-08T18:07:54.067044+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:00:59.786865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'6","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858148+0000","last_change":"2024-05-08T18:04:45.827616+0000","last_active":"2024-05-08T18:05:30.858148+0000","last_peered":"2024-05-08T18:05:30.858148+0000","last_clean":"2024-05-08T18:05:30.858148+0000","last_became_active":"2024-05-08T18:04:45.827351+0000","last_became_peered":"2024-05-08T18:04:45.827351+0000","last_unstale":"2024-05-08T18:05:30.858148+0000","last_undegraded":"2024-05-08T18:05:30.858148+0000","last_fullsized":"2024-05-08T18:05:30.858148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:31.853297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866835+0000","last_change":"2024-05-08T18:04:43.839205+0000","last_active":"2024-05-08T18:05:30.866835+0000","last_peered":"2024-05-08T18:05:30.866835+0000","last_clean":"2024-05-08T18:05:30.866835+0000","last_became_active":"2024-05-08T18:04:43.838840+0000","last_became_peered":"2024-05-08T18:04:43.838840+0000","last_unstale":"2024-05-08T18:05:30.866835+0000","last_undegraded":"2024-05-08T18:05:30.866835+0000","last_fullsized":"2024-05-08T18:05:30.866835+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:50.043665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,0],"acting":[2,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861049+0000","last_change":"2024-05-08T18:04:47.825571+0000","last_active":"2024-05-08T18:05:30.861049+0000","last_peered":"2024-05-08T18:05:30.861049+0000","last_clean":"2024-05-08T18:05:30.861049+0000","last_became_active":"2024-05-08T18:04:47.825215+0000","last_became_peered":"2024-05-08T18:04:47.825215+0000","last_unstale":"2024-05-08T18:05:30.861049+0000","last_undegraded":"2024-05-08T18:05:30.861049+0000","last_fullsized":"2024-05-08T18:05:30.861049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:47.750530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"29'40","reported_seq":87,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.957612+0000","last_change":"2024-05-08T18:04:49.828257+0000","last_active":"2024-05-08T18:06:51.957612+0000","last_peered":"2024-05-08T18:06:51.957612+0000","last_clean":"2024-05-08T18:06:51.957612+0000","last_became_active":"2024-05-08T18:04:49.827954+0000","last_became_peered":"2024-05-08T18:04:49.827954+0000","last_unstale":"2024-05-08T18:06:51.957612+0000","last_undegraded":"2024-05-08T18:06:51.957612+0000","last_fullsized":"2024-05-08T18:06:51.957612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:39:43.565272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859378+0000","last_change":"2024-05-08T18:04:51.831489+0000","last_active":"2024-05-08T18:05:30.859378+0000","last_peered":"2024-05-08T18:05:30.859378+0000","last_clean":"2024-05-08T18:05:30.859378+0000","last_became_active":"2024-05-08T18:04:51.831149+0000","last_became_peered":"2024-05-08T18:04:51.831149+0000","last_unstale":"2024-05-08T18:05:30.859378+0000","last_undegraded":"2024-05-08T18:05:30.859378+0000","last_fullsized":"2024-05-08T18:05:30.859378+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:59.859356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"29'5","reported_seq":25,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.735436+0000","last_change":"2024-05-08T18:04:53.840960+0000","last_active":"2024-05-08T18:06:09.735436+0000","last_peered":"2024-05-08T18:06:09.735436+0000","last_clean":"2024-05-08T18:06:09.735436+0000","last_became_active":"2024-05-08T18:04:53.840652+0000","last_became_peered":"2024-05-08T18:04:53.840652+0000","last_unstale":"2024-05-08T18:06:09.735436+0000","last_undegraded":"2024-05-08T18:06:09.735436+0000","last_fullsized":"2024-05-08T18:06:09.735436+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:02:50.273794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"27'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861806+0000","last_change":"2024-05-08T18:04:45.804412+0000","last_active":"2024-05-08T18:05:30.861806+0000","last_peered":"2024-05-08T18:05:30.861806+0000","last_clean":"2024-05-08T18:05:30.861806+0000","last_became_active":"2024-05-08T18:04:45.804124+0000","last_became_peered":"2024-05-08T18:04:45.804124+0000","last_unstale":"2024-05-08T18:05:30.861806+0000","last_undegraded":"2024-05-08T18:05:30.861806+0000","last_fullsized":"2024-05-08T18:05:30.861806+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:08.184602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867328+0000","last_change":"2024-05-08T18:04:43.843177+0000","last_active":"2024-05-08T18:05:30.867328+0000","last_peered":"2024-05-08T18:05:30.867328+0000","last_clean":"2024-05-08T18:05:30.867328+0000","last_became_active":"2024-05-08T18:04:43.842126+0000","last_became_peered":"2024-05-08T18:04:43.842126+0000","last_unstale":"2024-05-08T18:05:30.867328+0000","last_undegraded":"2024-05-08T18:05:30.867328+0000","last_fullsized":"2024-05-08T18:05:30.867328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:41:01.395720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858650+0000","last_change":"2024-05-08T18:04:47.823007+0000","last_active":"2024-05-08T18:05:30.858650+0000","last_peered":"2024-05-08T18:05:30.858650+0000","last_clean":"2024-05-08T18:05:30.858650+0000","last_became_active":"2024-05-08T18:04:47.822688+0000","last_became_peered":"2024-05-08T18:04:47.822688+0000","last_unstale":"2024-05-08T18:05:30.858650+0000","last_undegraded":"2024-05-08T18:05:30.858650+0000","last_fullsized":"2024-05-08T18:05:30.858650+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:32.662502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"29'30","reported_seq":72,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.938361+0000","last_change":"2024-05-08T18:04:49.832010+0000","last_active":"2024-05-08T18:06:51.938361+0000","last_peered":"2024-05-08T18:06:51.938361+0000","last_clean":"2024-05-08T18:06:51.938361+0000","last_became_active":"2024-05-08T18:04:49.829892+0000","last_became_peered":"2024-05-08T18:04:49.829892+0000","last_unstale":"2024-05-08T18:06:51.938361+0000","last_undegraded":"2024-05-08T18:06:51.938361+0000","last_fullsized":"2024-05-08T18:06:51.938361+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:13:16.535751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858848+0000","last_change":"2024-05-08T18:04:51.833183+0000","last_active":"2024-05-08T18:05:30.858848+0000","last_peered":"2024-05-08T18:05:30.858848+0000","last_clean":"2024-05-08T18:05:30.858848+0000","last_became_active":"2024-05-08T18:04:51.832810+0000","last_became_peered":"2024-05-08T18:04:51.832810+0000","last_unstale":"2024-05-08T18:05:30.858848+0000","last_undegraded":"2024-05-08T18:05:30.858848+0000","last_fullsized":"2024-05-08T18:05:30.858848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:14:02.130044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"29'3","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:03.972536+0000","last_change":"2024-05-08T18:04:53.841201+0000","last_active":"2024-05-08T18:06:03.972536+0000","last_peered":"2024-05-08T18:06:03.972536+0000","last_clean":"2024-05-08T18:06:03.972536+0000","last_became_active":"2024-05-08T18:04:53.840894+0000","last_became_peered":"2024-05-08T18:04:53.840894+0000","last_unstale":"2024-05-08T18:06:03.972536+0000","last_undegraded":"2024-05-08T18:06:03.972536+0000","last_fullsized":"2024-05-08T18:06:03.972536+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:22:10.520507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"29'9","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.867093+0000","last_change":"2024-05-08T18:04:45.817096+0000","last_active":"2024-05-08T18:07:53.867093+0000","last_peered":"2024-05-08T18:07:53.867093+0000","last_clean":"2024-05-08T18:07:53.867093+0000","last_became_active":"2024-05-08T18:04:45.816817+0000","last_became_peered":"2024-05-08T18:04:45.816817+0000","last_unstale":"2024-05-08T18:07:53.867093+0000","last_undegraded":"2024-05-08T18:07:53.867093+0000","last_fullsized":"2024-05-08T18:07:53.867093+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:08:48.747375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"29'3","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.984852+0000","last_change":"2024-05-08T18:04:43.844652+0000","last_active":"2024-05-08T18:07:53.984852+0000","last_peered":"2024-05-08T18:07:53.984852+0000","last_clean":"2024-05-08T18:07:53.984852+0000","last_became_active":"2024-05-08T18:04:43.838781+0000","last_became_peered":"2024-05-08T18:04:43.838781+0000","last_unstale":"2024-05-08T18:07:53.984852+0000","last_undegraded":"2024-05-08T18:07:53.984852+0000","last_fullsized":"2024-05-08T18:07:53.984852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:22:17.577945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":668544,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":11,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858935+0000","last_change":"2024-05-08T18:04:47.819192+0000","last_active":"2024-05-08T18:05:30.858935+0000","last_peered":"2024-05-08T18:05:30.858935+0000","last_clean":"2024-05-08T18:05:30.858935+0000","last_became_active":"2024-05-08T18:04:47.818871+0000","last_became_peered":"2024-05-08T18:04:47.818871+0000","last_unstale":"2024-05-08T18:05:30.858935+0000","last_undegraded":"2024-05-08T18:05:30.858935+0000","last_fullsized":"2024-05-08T18:05:30.858935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:20.855010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"29'36","reported_seq":81,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.959548+0000","last_change":"2024-05-08T18:04:49.829583+0000","last_active":"2024-05-08T18:06:51.959548+0000","last_peered":"2024-05-08T18:06:51.959548+0000","last_clean":"2024-05-08T18:06:51.959548+0000","last_became_active":"2024-05-08T18:04:49.829270+0000","last_became_peered":"2024-05-08T18:04:49.829270+0000","last_unstale":"2024-05-08T18:06:51.959548+0000","last_undegraded":"2024-05-08T18:06:51.959548+0000","last_fullsized":"2024-05-08T18:06:51.959548+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:11:17.784122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"29'282","reported_seq":348,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.181903+0000","last_change":"2024-05-08T18:04:51.845064+0000","last_active":"2024-05-08T18:07:54.181903+0000","last_peered":"2024-05-08T18:07:54.181903+0000","last_clean":"2024-05-08T18:07:54.181903+0000","last_became_active":"2024-05-08T18:04:51.844529+0000","last_became_peered":"2024-05-08T18:04:51.844529+0000","last_unstale":"2024-05-08T18:07:54.181903+0000","last_undegraded":"2024-05-08T18:07:54.181903+0000","last_fullsized":"2024-05-08T18:07:54.181903+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:46.002263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"29'2","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.862719+0000","last_change":"2024-05-08T18:04:53.841062+0000","last_active":"2024-05-08T18:07:53.862719+0000","last_peered":"2024-05-08T18:07:53.862719+0000","last_clean":"2024-05-08T18:07:53.862719+0000","last_became_active":"2024-05-08T18:04:53.840767+0000","last_became_peered":"2024-05-08T18:04:53.840767+0000","last_unstale":"2024-05-08T18:07:53.862719+0000","last_undegraded":"2024-05-08T18:07:53.862719+0000","last_fullsized":"2024-05-08T18:07:53.862719+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:54:38.757755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"29'6","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995942+0000","last_change":"2024-05-08T18:04:45.814276+0000","last_active":"2024-05-08T18:07:53.995942+0000","last_peered":"2024-05-08T18:07:53.995942+0000","last_clean":"2024-05-08T18:07:53.995942+0000","last_became_active":"2024-05-08T18:04:45.813735+0000","last_became_peered":"2024-05-08T18:04:45.813735+0000","last_unstale":"2024-05-08T18:07:53.995942+0000","last_undegraded":"2024-05-08T18:07:53.995942+0000","last_fullsized":"2024-05-08T18:07:53.995942+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:18.463624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"27'4","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859087+0000","last_change":"2024-05-08T18:04:43.848140+0000","last_active":"2024-05-08T18:05:30.859087+0000","last_peered":"2024-05-08T18:05:30.859087+0000","last_clean":"2024-05-08T18:05:30.859087+0000","last_became_active":"2024-05-08T18:04:43.839047+0000","last_became_peered":"2024-05-08T18:04:43.839047+0000","last_unstale":"2024-05-08T18:05:30.859087+0000","last_undegraded":"2024-05-08T18:05:30.859087+0000","last_fullsized":"2024-05-08T18:05:30.859087+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:24.945310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858315+0000","last_change":"2024-05-08T18:04:47.819319+0000","last_active":"2024-05-08T18:05:30.858315+0000","last_peered":"2024-05-08T18:05:30.858315+0000","last_clean":"2024-05-08T18:05:30.858315+0000","last_became_active":"2024-05-08T18:04:47.818972+0000","last_became_peered":"2024-05-08T18:04:47.818972+0000","last_unstale":"2024-05-08T18:05:30.858315+0000","last_undegraded":"2024-05-08T18:05:30.858315+0000","last_fullsized":"2024-05-08T18:05:30.858315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:09:51.843752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"29'54","reported_seq":110,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.961509+0000","last_change":"2024-05-08T18:04:49.832104+0000","last_active":"2024-05-08T18:06:51.961509+0000","last_peered":"2024-05-08T18:06:51.961509+0000","last_clean":"2024-05-08T18:06:51.961509+0000","last_became_active":"2024-05-08T18:04:49.830424+0000","last_became_peered":"2024-05-08T18:04:49.830424+0000","last_unstale":"2024-05-08T18:06:51.961509+0000","last_undegraded":"2024-05-08T18:06:51.961509+0000","last_fullsized":"2024-05-08T18:06:51.961509+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:50.646336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858392+0000","last_change":"2024-05-08T18:04:51.830631+0000","last_active":"2024-05-08T18:05:30.858392+0000","last_peered":"2024-05-08T18:05:30.858392+0000","last_clean":"2024-05-08T18:05:30.858392+0000","last_became_active":"2024-05-08T18:04:51.830383+0000","last_became_peered":"2024-05-08T18:04:51.830383+0000","last_unstale":"2024-05-08T18:05:30.858392+0000","last_undegraded":"2024-05-08T18:05:30.858392+0000","last_fullsized":"2024-05-08T18:05:30.858392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:17:01.176258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"29'20","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.158011+0000","last_change":"2024-05-08T18:04:53.848105+0000","last_active":"2024-05-08T18:07:54.158011+0000","last_peered":"2024-05-08T18:07:54.158011+0000","last_clean":"2024-05-08T18:07:54.158011+0000","last_became_active":"2024-05-08T18:04:53.847842+0000","last_became_peered":"2024-05-08T18:04:53.847842+0000","last_unstale":"2024-05-08T18:07:54.158011+0000","last_undegraded":"2024-05-08T18:07:54.158011+0000","last_fullsized":"2024-05-08T18:07:54.158011+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:03:57.205665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":48,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"29'6","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.064006+0000","last_change":"2024-05-08T18:04:45.821729+0000","last_active":"2024-05-08T18:07:54.064006+0000","last_peered":"2024-05-08T18:07:54.064006+0000","last_clean":"2024-05-08T18:07:54.064006+0000","last_became_active":"2024-05-08T18:04:45.821412+0000","last_became_peered":"2024-05-08T18:04:45.821412+0000","last_unstale":"2024-05-08T18:07:54.064006+0000","last_undegraded":"2024-05-08T18:07:54.064006+0000","last_fullsized":"2024-05-08T18:07:54.064006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:26.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859201+0000","last_change":"2024-05-08T18:04:43.835922+0000","last_active":"2024-05-08T18:05:30.859201+0000","last_peered":"2024-05-08T18:05:30.859201+0000","last_clean":"2024-05-08T18:05:30.859201+0000","last_became_active":"2024-05-08T18:04:43.834723+0000","last_became_peered":"2024-05-08T18:04:43.834723+0000","last_unstale":"2024-05-08T18:05:30.859201+0000","last_undegraded":"2024-05-08T18:05:30.859201+0000","last_fullsized":"2024-05-08T18:05:30.859201+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:19.700373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864185+0000","last_change":"2024-05-08T18:04:47.823171+0000","last_active":"2024-05-08T18:05:30.864185+0000","last_peered":"2024-05-08T18:05:30.864185+0000","last_clean":"2024-05-08T18:05:30.864185+0000","last_became_active":"2024-05-08T18:04:47.822865+0000","last_became_peered":"2024-05-08T18:04:47.822865+0000","last_unstale":"2024-05-08T18:05:30.864185+0000","last_undegraded":"2024-05-08T18:05:30.864185+0000","last_fullsized":"2024-05-08T18:05:30.864185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:22:57.464984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"29'33","reported_seq":76,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.942615+0000","last_change":"2024-05-08T18:04:49.824918+0000","last_active":"2024-05-08T18:06:51.942615+0000","last_peered":"2024-05-08T18:06:51.942615+0000","last_clean":"2024-05-08T18:06:51.942615+0000","last_became_active":"2024-05-08T18:04:49.824610+0000","last_became_peered":"2024-05-08T18:04:49.824610+0000","last_unstale":"2024-05-08T18:06:51.942615+0000","last_undegraded":"2024-05-08T18:06:51.942615+0000","last_fullsized":"2024-05-08T18:06:51.942615+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:56:27.132432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864191+0000","last_change":"2024-05-08T18:04:51.849355+0000","last_active":"2024-05-08T18:05:30.864191+0000","last_peered":"2024-05-08T18:05:30.864191+0000","last_clean":"2024-05-08T18:05:30.864191+0000","last_became_active":"2024-05-08T18:04:51.848793+0000","last_became_peered":"2024-05-08T18:04:51.848793+0000","last_unstale":"2024-05-08T18:05:30.864191+0000","last_undegraded":"2024-05-08T18:05:30.864191+0000","last_fullsized":"2024-05-08T18:05:30.864191+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:40:15.186526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"29'5","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744070+0000","last_change":"2024-05-08T18:04:53.849242+0000","last_active":"2024-05-08T18:06:53.744070+0000","last_peered":"2024-05-08T18:06:53.744070+0000","last_clean":"2024-05-08T18:06:53.744070+0000","last_became_active":"2024-05-08T18:04:53.848945+0000","last_became_peered":"2024-05-08T18:04:53.848945+0000","last_unstale":"2024-05-08T18:06:53.744070+0000","last_undegraded":"2024-05-08T18:06:53.744070+0000","last_fullsized":"2024-05-08T18:06:53.744070+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:18:00.234436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"29'10","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.772802+0000","last_change":"2024-05-08T18:04:45.829691+0000","last_active":"2024-05-08T18:07:53.772802+0000","last_peered":"2024-05-08T18:07:53.772802+0000","last_clean":"2024-05-08T18:07:53.772802+0000","last_became_active":"2024-05-08T18:04:45.829120+0000","last_became_peered":"2024-05-08T18:04:45.829120+0000","last_unstale":"2024-05-08T18:07:53.772802+0000","last_undegraded":"2024-05-08T18:07:53.772802+0000","last_fullsized":"2024-05-08T18:07:53.772802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:29.119368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":57,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859093+0000","last_change":"2024-05-08T18:04:43.835329+0000","last_active":"2024-05-08T18:05:30.859093+0000","last_peered":"2024-05-08T18:05:30.859093+0000","last_clean":"2024-05-08T18:05:30.859093+0000","last_became_active":"2024-05-08T18:04:43.834536+0000","last_became_peered":"2024-05-08T18:04:43.834536+0000","last_unstale":"2024-05-08T18:05:30.859093+0000","last_undegraded":"2024-05-08T18:05:30.859093+0000","last_fullsized":"2024-05-08T18:05:30.859093+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:56.567490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"29'4","reported_seq":226,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.038721+0000","last_change":"2024-05-08T18:04:47.819789+0000","last_active":"2024-05-08T18:06:53.038721+0000","last_peered":"2024-05-08T18:06:53.038721+0000","last_clean":"2024-05-08T18:06:53.038721+0000","last_became_active":"2024-05-08T18:04:47.819085+0000","last_became_peered":"2024-05-08T18:04:47.819085+0000","last_unstale":"2024-05-08T18:06:53.038721+0000","last_undegraded":"2024-05-08T18:06:53.038721+0000","last_fullsized":"2024-05-08T18:06:53.038721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:24.812721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":298,"num_read_kb":298,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"29'36","reported_seq":81,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.967858+0000","last_change":"2024-05-08T18:04:49.825997+0000","last_active":"2024-05-08T18:06:51.967858+0000","last_peered":"2024-05-08T18:06:51.967858+0000","last_clean":"2024-05-08T18:06:51.967858+0000","last_became_active":"2024-05-08T18:04:49.825613+0000","last_became_peered":"2024-05-08T18:04:49.825613+0000","last_unstale":"2024-05-08T18:06:51.967858+0000","last_undegraded":"2024-05-08T18:06:51.967858+0000","last_fullsized":"2024-05-08T18:06:51.967858+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:49.758047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"29'282","reported_seq":431,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.162304+0000","last_change":"2024-05-08T18:04:51.831317+0000","last_active":"2024-05-08T18:07:54.162304+0000","last_peered":"2024-05-08T18:07:54.162304+0000","last_clean":"2024-05-08T18:07:54.162304+0000","last_became_active":"2024-05-08T18:04:51.830924+0000","last_became_peered":"2024-05-08T18:04:51.830924+0000","last_unstale":"2024-05-08T18:07:54.162304+0000","last_undegraded":"2024-05-08T18:07:54.162304+0000","last_fullsized":"2024-05-08T18:07:54.162304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:23.880081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"29'14","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.739659+0000","last_change":"2024-05-08T18:04:53.841349+0000","last_active":"2024-05-08T18:06:58.739659+0000","last_peered":"2024-05-08T18:06:58.739659+0000","last_clean":"2024-05-08T18:06:58.739659+0000","last_became_active":"2024-05-08T18:04:53.840708+0000","last_became_peered":"2024-05-08T18:04:53.840708+0000","last_unstale":"2024-05-08T18:06:58.739659+0000","last_undegraded":"2024-05-08T18:06:58.739659+0000","last_fullsized":"2024-05-08T18:06:58.739659+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:57:33.051338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063870+0000","last_change":"2024-05-08T18:04:45.812281+0000","last_active":"2024-05-08T18:07:54.063870+0000","last_peered":"2024-05-08T18:07:54.063870+0000","last_clean":"2024-05-08T18:07:54.063870+0000","last_became_active":"2024-05-08T18:04:45.811926+0000","last_became_peered":"2024-05-08T18:04:45.811926+0000","last_unstale":"2024-05-08T18:07:54.063870+0000","last_undegraded":"2024-05-08T18:07:54.063870+0000","last_fullsized":"2024-05-08T18:07:54.063870+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:26.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"29'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.728795+0000","last_change":"2024-05-08T18:04:43.844418+0000","last_active":"2024-05-08T18:06:09.728795+0000","last_peered":"2024-05-08T18:06:09.728795+0000","last_clean":"2024-05-08T18:06:09.728795+0000","last_became_active":"2024-05-08T18:04:43.843934+0000","last_became_peered":"2024-05-08T18:04:43.843934+0000","last_unstale":"2024-05-08T18:06:09.728795+0000","last_undegraded":"2024-05-08T18:06:09.728795+0000","last_fullsized":"2024-05-08T18:06:09.728795+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:30.825989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"29'3","reported_seq":413,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.049002+0000","last_change":"2024-05-08T18:04:47.818222+0000","last_active":"2024-05-08T18:06:53.049002+0000","last_peered":"2024-05-08T18:06:53.049002+0000","last_clean":"2024-05-08T18:06:53.049002+0000","last_became_active":"2024-05-08T18:04:47.817851+0000","last_became_peered":"2024-05-08T18:04:47.817851+0000","last_unstale":"2024-05-08T18:06:53.049002+0000","last_undegraded":"2024-05-08T18:06:53.049002+0000","last_fullsized":"2024-05-08T18:06:53.049002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:45:55.932454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":381,"num_read_kb":381,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"27'10","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863960+0000","last_change":"2024-05-08T18:04:49.832191+0000","last_active":"2024-05-08T18:05:30.863960+0000","last_peered":"2024-05-08T18:05:30.863960+0000","last_clean":"2024-05-08T18:05:30.863960+0000","last_became_active":"2024-05-08T18:04:49.830714+0000","last_became_peered":"2024-05-08T18:04:49.830714+0000","last_unstale":"2024-05-08T18:05:30.863960+0000","last_undegraded":"2024-05-08T18:05:30.863960+0000","last_fullsized":"2024-05-08T18:05:30.863960+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:14:01.011620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"29'282","reported_seq":468,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.120234+0000","last_change":"2024-05-08T18:04:51.832589+0000","last_active":"2024-05-08T18:07:54.120234+0000","last_peered":"2024-05-08T18:07:54.120234+0000","last_clean":"2024-05-08T18:07:54.120234+0000","last_became_active":"2024-05-08T18:04:51.831508+0000","last_became_peered":"2024-05-08T18:04:51.831508+0000","last_unstale":"2024-05-08T18:07:54.120234+0000","last_undegraded":"2024-05-08T18:07:54.120234+0000","last_fullsized":"2024-05-08T18:07:54.120234+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:15.559137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'4","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.179451+0000","last_change":"2024-05-08T18:04:53.840524+0000","last_active":"2024-05-08T18:06:53.179451+0000","last_peered":"2024-05-08T18:06:53.179451+0000","last_clean":"2024-05-08T18:06:53.179451+0000","last_became_active":"2024-05-08T18:04:53.840215+0000","last_became_peered":"2024-05-08T18:04:53.840215+0000","last_unstale":"2024-05-08T18:06:53.179451+0000","last_undegraded":"2024-05-08T18:06:53.179451+0000","last_fullsized":"2024-05-08T18:06:53.179451+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:39.550898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'2","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866626+0000","last_change":"2024-05-08T18:04:45.819074+0000","last_active":"2024-05-08T18:05:30.866626+0000","last_peered":"2024-05-08T18:05:30.866626+0000","last_clean":"2024-05-08T18:05:30.866626+0000","last_became_active":"2024-05-08T18:04:45.818533+0000","last_became_peered":"2024-05-08T18:04:45.818533+0000","last_unstale":"2024-05-08T18:05:30.866626+0000","last_undegraded":"2024-05-08T18:05:30.866626+0000","last_fullsized":"2024-05-08T18:05:30.866626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:21:34.419882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"29'1","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744105+0000","last_change":"2024-05-08T18:04:43.848496+0000","last_active":"2024-05-08T18:06:53.744105+0000","last_peered":"2024-05-08T18:06:53.744105+0000","last_clean":"2024-05-08T18:06:53.744105+0000","last_became_active":"2024-05-08T18:04:43.846125+0000","last_became_peered":"2024-05-08T18:04:43.846125+0000","last_unstale":"2024-05-08T18:06:53.744105+0000","last_undegraded":"2024-05-08T18:06:53.744105+0000","last_fullsized":"2024-05-08T18:06:53.744105+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:52:32.125079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861388+0000","last_change":"2024-05-08T18:04:47.822964+0000","last_active":"2024-05-08T18:05:30.861388+0000","last_peered":"2024-05-08T18:05:30.861388+0000","last_clean":"2024-05-08T18:05:30.861388+0000","last_became_active":"2024-05-08T18:04:47.822242+0000","last_became_peered":"2024-05-08T18:04:47.822242+0000","last_unstale":"2024-05-08T18:05:30.861388+0000","last_undegraded":"2024-05-08T18:05:30.861388+0000","last_fullsized":"2024-05-08T18:05:30.861388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:27:55.845147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"29'38","reported_seq":84,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.954827+0000","last_change":"2024-05-08T18:04:49.826706+0000","last_active":"2024-05-08T18:06:51.954827+0000","last_peered":"2024-05-08T18:06:51.954827+0000","last_clean":"2024-05-08T18:06:51.954827+0000","last_became_active":"2024-05-08T18:04:49.825314+0000","last_became_peered":"2024-05-08T18:04:49.825314+0000","last_unstale":"2024-05-08T18:06:51.954827+0000","last_undegraded":"2024-05-08T18:06:51.954827+0000","last_fullsized":"2024-05-08T18:06:51.954827+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:21:18.445670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862981+0000","last_change":"2024-05-08T18:04:51.833093+0000","last_active":"2024-05-08T18:05:30.862981+0000","last_peered":"2024-05-08T18:05:30.862981+0000","last_clean":"2024-05-08T18:05:30.862981+0000","last_became_active":"2024-05-08T18:04:51.832808+0000","last_became_peered":"2024-05-08T18:04:51.832808+0000","last_unstale":"2024-05-08T18:05:30.862981+0000","last_undegraded":"2024-05-08T18:05:30.862981+0000","last_fullsized":"2024-05-08T18:05:30.862981+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:54.955755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"29'8","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.096584+0000","last_change":"2024-05-08T18:04:53.837583+0000","last_active":"2024-05-08T18:07:54.096584+0000","last_peered":"2024-05-08T18:07:54.096584+0000","last_clean":"2024-05-08T18:07:54.096584+0000","last_became_active":"2024-05-08T18:04:53.837292+0000","last_became_peered":"2024-05-08T18:04:53.837292+0000","last_unstale":"2024-05-08T18:07:54.096584+0000","last_undegraded":"2024-05-08T18:07:54.096584+0000","last_fullsized":"2024-05-08T18:07:54.096584+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:15.652477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'4","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.035838+0000","last_change":"2024-05-08T18:04:45.829568+0000","last_active":"2024-05-08T18:07:54.035838+0000","last_peered":"2024-05-08T18:07:54.035838+0000","last_clean":"2024-05-08T18:07:54.035838+0000","last_became_active":"2024-05-08T18:04:45.828674+0000","last_became_peered":"2024-05-08T18:04:45.828674+0000","last_unstale":"2024-05-08T18:07:54.035838+0000","last_undegraded":"2024-05-08T18:07:54.035838+0000","last_fullsized":"2024-05-08T18:07:54.035838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:33.440280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":137,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859322+0000","last_change":"2024-05-08T18:04:35.552002+0000","last_active":"2024-05-08T18:05:30.859322+0000","last_peered":"2024-05-08T18:05:30.859322+0000","last_clean":"2024-05-08T18:05:30.859322+0000","last_became_active":"2024-05-08T18:04:35.551437+0000","last_became_peered":"2024-05-08T18:04:35.551437+0000","last_unstale":"2024-05-08T18:05:30.859322+0000","last_undegraded":"2024-05-08T18:05:30.859322+0000","last_fullsized":"2024-05-08T18:05:30.859322+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:58.749794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859862+0000","last_change":"2024-05-08T18:04:35.552341+0000","last_active":"2024-05-08T18:05:30.859862+0000","last_peered":"2024-05-08T18:05:30.859862+0000","last_clean":"2024-05-08T18:05:30.859862+0000","last_became_active":"2024-05-08T18:04:35.552017+0000","last_became_peered":"2024-05-08T18:04:35.552017+0000","last_unstale":"2024-05-08T18:05:30.859862+0000","last_undegraded":"2024-05-08T18:05:30.859862+0000","last_fullsized":"2024-05-08T18:05:30.859862+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:17:30.242269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858503+0000","last_change":"2024-05-08T18:04:43.837405+0000","last_active":"2024-05-08T18:05:30.858503+0000","last_peered":"2024-05-08T18:05:30.858503+0000","last_clean":"2024-05-08T18:05:30.858503+0000","last_became_active":"2024-05-08T18:04:43.837019+0000","last_became_peered":"2024-05-08T18:04:43.837019+0000","last_unstale":"2024-05-08T18:05:30.858503+0000","last_undegraded":"2024-05-08T18:05:30.858503+0000","last_fullsized":"2024-05-08T18:05:30.858503+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:42:03.247527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.871817+0000","last_change":"2024-05-08T18:05:30.871131+0000","last_active":"2024-05-08T18:05:30.871817+0000","last_peered":"2024-05-08T18:05:30.871817+0000","last_clean":"2024-05-08T18:05:30.871817+0000","last_became_active":"2024-05-08T18:05:30.870647+0000","last_became_peered":"2024-05-08T18:05:30.870647+0000","last_unstale":"2024-05-08T18:05:30.871817+0000","last_undegraded":"2024-05-08T18:05:30.871817+0000","last_fullsized":"2024-05-08T18:05:30.871817+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:48.638819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"29'77","reported_seq":146,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.965816+0000","last_change":"2024-05-08T18:04:49.832374+0000","last_active":"2024-05-08T18:06:51.965816+0000","last_peered":"2024-05-08T18:06:51.965816+0000","last_clean":"2024-05-08T18:06:51.965816+0000","last_became_active":"2024-05-08T18:04:49.830636+0000","last_became_peered":"2024-05-08T18:04:49.830636+0000","last_unstale":"2024-05-08T18:06:51.965816+0000","last_undegraded":"2024-05-08T18:06:51.965816+0000","last_fullsized":"2024-05-08T18:06:51.965816+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:23:12.330689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859612+0000","last_change":"2024-05-08T18:04:51.845249+0000","last_active":"2024-05-08T18:05:30.859612+0000","last_peered":"2024-05-08T18:05:30.859612+0000","last_clean":"2024-05-08T18:05:30.859612+0000","last_became_active":"2024-05-08T18:04:51.844737+0000","last_became_peered":"2024-05-08T18:04:51.844737+0000","last_unstale":"2024-05-08T18:05:30.859612+0000","last_undegraded":"2024-05-08T18:05:30.859612+0000","last_fullsized":"2024-05-08T18:05:30.859612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:08.922575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"29'85","reported_seq":166,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.117787+0000","last_change":"2024-05-08T18:04:53.839334+0000","last_active":"2024-05-08T18:07:54.117787+0000","last_peered":"2024-05-08T18:07:54.117787+0000","last_clean":"2024-05-08T18:07:54.117787+0000","last_became_active":"2024-05-08T18:04:53.838957+0000","last_became_peered":"2024-05-08T18:04:53.838957+0000","last_unstale":"2024-05-08T18:07:54.117787+0000","last_undegraded":"2024-05-08T18:07:54.117787+0000","last_fullsized":"2024-05-08T18:07:54.117787+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:25:57.828734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":216,"num_read_kb":214,"num_write":167,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"29'9","reported_seq":65,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070594+0000","last_change":"2024-05-08T18:04:45.826087+0000","last_active":"2024-05-08T18:07:54.070594+0000","last_peered":"2024-05-08T18:07:54.070594+0000","last_clean":"2024-05-08T18:07:54.070594+0000","last_became_active":"2024-05-08T18:04:45.825843+0000","last_became_peered":"2024-05-08T18:04:45.825843+0000","last_unstale":"2024-05-08T18:07:54.070594+0000","last_undegraded":"2024-05-08T18:07:54.070594+0000","last_fullsized":"2024-05-08T18:07:54.070594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:21:14.431053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857318+0000","last_change":"2024-05-08T18:04:35.552553+0000","last_active":"2024-05-08T18:05:30.857318+0000","last_peered":"2024-05-08T18:05:30.857318+0000","last_clean":"2024-05-08T18:05:30.857318+0000","last_became_active":"2024-05-08T18:04:35.552214+0000","last_became_peered":"2024-05-08T18:04:35.552214+0000","last_unstale":"2024-05-08T18:05:30.857318+0000","last_undegraded":"2024-05-08T18:05:30.857318+0000","last_fullsized":"2024-05-08T18:05:30.857318+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:12:16.476925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858237+0000","last_change":"2024-05-08T18:04:43.821058+0000","last_active":"2024-05-08T18:05:30.858237+0000","last_peered":"2024-05-08T18:05:30.858237+0000","last_clean":"2024-05-08T18:05:30.858237+0000","last_became_active":"2024-05-08T18:04:43.820369+0000","last_became_peered":"2024-05-08T18:04:43.820369+0000","last_unstale":"2024-05-08T18:05:30.858237+0000","last_undegraded":"2024-05-08T18:05:30.858237+0000","last_fullsized":"2024-05-08T18:05:30.858237+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:01:49.620924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.871018+0000","last_change":"2024-05-08T18:05:30.870035+0000","last_active":"2024-05-08T18:05:30.871018+0000","last_peered":"2024-05-08T18:05:30.871018+0000","last_clean":"2024-05-08T18:05:30.871018+0000","last_became_active":"2024-05-08T18:05:30.869578+0000","last_became_peered":"2024-05-08T18:05:30.869578+0000","last_unstale":"2024-05-08T18:05:30.871018+0000","last_undegraded":"2024-05-08T18:05:30.871018+0000","last_fullsized":"2024-05-08T18:05:30.871018+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:51:48.737463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"29'273","reported_seq":476,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903545+0000","last_change":"2024-05-08T18:04:49.833379+0000","last_active":"2024-05-08T18:07:52.903545+0000","last_peered":"2024-05-08T18:07:52.903545+0000","last_clean":"2024-05-08T18:07:52.903545+0000","last_became_active":"2024-05-08T18:04:49.833053+0000","last_became_peered":"2024-05-08T18:04:49.833053+0000","last_unstale":"2024-05-08T18:07:52.903545+0000","last_undegraded":"2024-05-08T18:07:52.903545+0000","last_fullsized":"2024-05-08T18:07:52.903545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:41:21.873389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":357,"num_read_kb":342,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859300+0000","last_change":"2024-05-08T18:04:51.849383+0000","last_active":"2024-05-08T18:05:30.859300+0000","last_peered":"2024-05-08T18:05:30.859300+0000","last_clean":"2024-05-08T18:05:30.859300+0000","last_became_active":"2024-05-08T18:04:51.848066+0000","last_became_peered":"2024-05-08T18:04:51.848066+0000","last_unstale":"2024-05-08T18:05:30.859300+0000","last_undegraded":"2024-05-08T18:05:30.859300+0000","last_fullsized":"2024-05-08T18:05:30.859300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:21.450184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"29'3","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.550250+0000","last_change":"2024-05-08T18:04:53.843187+0000","last_active":"2024-05-08T18:06:09.550250+0000","last_peered":"2024-05-08T18:06:09.550250+0000","last_clean":"2024-05-08T18:06:09.550250+0000","last_became_active":"2024-05-08T18:04:53.842899+0000","last_became_peered":"2024-05-08T18:04:53.842899+0000","last_unstale":"2024-05-08T18:06:09.550250+0000","last_undegraded":"2024-05-08T18:06:09.550250+0000","last_fullsized":"2024-05-08T18:06:09.550250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:05:40.879119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"29'8","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063713+0000","last_change":"2024-05-08T18:04:45.812417+0000","last_active":"2024-05-08T18:07:54.063713+0000","last_peered":"2024-05-08T18:07:54.063713+0000","last_clean":"2024-05-08T18:07:54.063713+0000","last_became_active":"2024-05-08T18:04:45.812091+0000","last_became_peered":"2024-05-08T18:04:45.812091+0000","last_unstale":"2024-05-08T18:07:54.063713+0000","last_undegraded":"2024-05-08T18:07:54.063713+0000","last_fullsized":"2024-05-08T18:07:54.063713+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:26.952570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863622+0000","last_change":"2024-05-08T18:04:35.551648+0000","last_active":"2024-05-08T18:05:30.863622+0000","last_peered":"2024-05-08T18:05:30.863622+0000","last_clean":"2024-05-08T18:05:30.863622+0000","last_became_active":"2024-05-08T18:04:35.551313+0000","last_became_peered":"2024-05-08T18:04:35.551313+0000","last_unstale":"2024-05-08T18:05:30.863622+0000","last_undegraded":"2024-05-08T18:05:30.863622+0000","last_fullsized":"2024-05-08T18:05:30.863622+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:34:32.994346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858040+0000","last_change":"2024-05-08T18:04:43.821180+0000","last_active":"2024-05-08T18:05:30.858040+0000","last_peered":"2024-05-08T18:05:30.858040+0000","last_clean":"2024-05-08T18:05:30.858040+0000","last_became_active":"2024-05-08T18:04:43.820335+0000","last_became_peered":"2024-05-08T18:04:43.820335+0000","last_unstale":"2024-05-08T18:05:30.858040+0000","last_undegraded":"2024-05-08T18:05:30.858040+0000","last_fullsized":"2024-05-08T18:05:30.858040+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:31.909506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"20'1","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866981+0000","last_change":"2024-05-08T18:04:47.824034+0000","last_active":"2024-05-08T18:05:30.866981+0000","last_peered":"2024-05-08T18:05:30.866981+0000","last_clean":"2024-05-08T18:05:30.866981+0000","last_became_active":"2024-05-08T18:04:47.823631+0000","last_became_peered":"2024-05-08T18:04:47.823631+0000","last_unstale":"2024-05-08T18:05:30.866981+0000","last_undegraded":"2024-05-08T18:05:30.866981+0000","last_fullsized":"2024-05-08T18:05:30.866981+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:43:00.791290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"29'17","reported_seq":85,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903379+0000","last_change":"2024-05-08T18:04:49.828228+0000","last_active":"2024-05-08T18:07:52.903379+0000","last_peered":"2024-05-08T18:07:52.903379+0000","last_clean":"2024-05-08T18:07:52.903379+0000","last_became_active":"2024-05-08T18:04:49.827735+0000","last_became_peered":"2024-05-08T18:04:49.827735+0000","last_unstale":"2024-05-08T18:07:52.903379+0000","last_undegraded":"2024-05-08T18:07:52.903379+0000","last_fullsized":"2024-05-08T18:07:52.903379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:04:17.239231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861148+0000","last_change":"2024-05-08T18:04:51.844276+0000","last_active":"2024-05-08T18:05:30.861148+0000","last_peered":"2024-05-08T18:05:30.861148+0000","last_clean":"2024-05-08T18:05:30.861148+0000","last_became_active":"2024-05-08T18:04:51.843940+0000","last_became_peered":"2024-05-08T18:04:51.843940+0000","last_unstale":"2024-05-08T18:05:30.861148+0000","last_undegraded":"2024-05-08T18:05:30.861148+0000","last_fullsized":"2024-05-08T18:05:30.861148+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:03:10.170093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"29'4","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.791011+0000","last_change":"2024-05-08T18:04:53.848676+0000","last_active":"2024-05-08T18:07:53.791011+0000","last_peered":"2024-05-08T18:07:53.791011+0000","last_clean":"2024-05-08T18:07:53.791011+0000","last_became_active":"2024-05-08T18:04:53.848255+0000","last_became_peered":"2024-05-08T18:04:53.848255+0000","last_unstale":"2024-05-08T18:07:53.791011+0000","last_undegraded":"2024-05-08T18:07:53.791011+0000","last_fullsized":"2024-05-08T18:07:53.791011+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:34:38.766674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"29'10","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063735+0000","last_change":"2024-05-08T18:04:45.807470+0000","last_active":"2024-05-08T18:07:54.063735+0000","last_peered":"2024-05-08T18:07:54.063735+0000","last_clean":"2024-05-08T18:07:54.063735+0000","last_became_active":"2024-05-08T18:04:45.807085+0000","last_became_peered":"2024-05-08T18:04:45.807085+0000","last_unstale":"2024-05-08T18:07:54.063735+0000","last_undegraded":"2024-05-08T18:07:54.063735+0000","last_fullsized":"2024-05-08T18:07:54.063735+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:05.828431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857751+0000","last_change":"2024-05-08T18:04:35.551456+0000","last_active":"2024-05-08T18:05:30.857751+0000","last_peered":"2024-05-08T18:05:30.857751+0000","last_clean":"2024-05-08T18:05:30.857751+0000","last_became_active":"2024-05-08T18:04:35.551120+0000","last_became_peered":"2024-05-08T18:04:35.551120+0000","last_unstale":"2024-05-08T18:05:30.857751+0000","last_undegraded":"2024-05-08T18:05:30.857751+0000","last_fullsized":"2024-05-08T18:05:30.857751+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:34:14.181756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864168+0000","last_change":"2024-05-08T18:04:43.840439+0000","last_active":"2024-05-08T18:05:30.864168+0000","last_peered":"2024-05-08T18:05:30.864168+0000","last_clean":"2024-05-08T18:05:30.864168+0000","last_became_active":"2024-05-08T18:04:43.840036+0000","last_became_peered":"2024-05-08T18:04:43.840036+0000","last_unstale":"2024-05-08T18:05:30.864168+0000","last_undegraded":"2024-05-08T18:05:30.864168+0000","last_fullsized":"2024-05-08T18:05:30.864168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:31.909506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0,3],"acting":[2,1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859629+0000","last_change":"2024-05-08T18:04:47.823507+0000","last_active":"2024-05-08T18:05:30.859629+0000","last_peered":"2024-05-08T18:05:30.859629+0000","last_clean":"2024-05-08T18:05:30.859629+0000","last_became_active":"2024-05-08T18:04:47.823093+0000","last_became_peered":"2024-05-08T18:04:47.823093+0000","last_unstale":"2024-05-08T18:05:30.859629+0000","last_undegraded":"2024-05-08T18:05:30.859629+0000","last_fullsized":"2024-05-08T18:05:30.859629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:18:59.706946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"29'13","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.412397+0000","last_change":"2024-05-08T18:04:49.832657+0000","last_active":"2024-05-08T18:07:53.412397+0000","last_peered":"2024-05-08T18:07:53.412397+0000","last_clean":"2024-05-08T18:07:53.412397+0000","last_became_active":"2024-05-08T18:04:49.830586+0000","last_became_peered":"2024-05-08T18:04:49.830586+0000","last_unstale":"2024-05-08T18:07:53.412397+0000","last_undegraded":"2024-05-08T18:07:53.412397+0000","last_fullsized":"2024-05-08T18:07:53.412397+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:01:11.727041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864596+0000","last_change":"2024-05-08T18:04:51.849539+0000","last_active":"2024-05-08T18:05:30.864596+0000","last_peered":"2024-05-08T18:05:30.864596+0000","last_clean":"2024-05-08T18:05:30.864596+0000","last_became_active":"2024-05-08T18:04:51.849137+0000","last_became_peered":"2024-05-08T18:04:51.849137+0000","last_unstale":"2024-05-08T18:05:30.864596+0000","last_undegraded":"2024-05-08T18:05:30.864596+0000","last_fullsized":"2024-05-08T18:05:30.864596+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:05.655002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'4","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.159086+0000","last_change":"2024-05-08T18:04:53.851345+0000","last_active":"2024-05-08T18:06:53.159086+0000","last_peered":"2024-05-08T18:06:53.159086+0000","last_clean":"2024-05-08T18:06:53.159086+0000","last_became_active":"2024-05-08T18:04:53.851008+0000","last_became_peered":"2024-05-08T18:04:53.851008+0000","last_unstale":"2024-05-08T18:06:53.159086+0000","last_undegraded":"2024-05-08T18:06:53.159086+0000","last_fullsized":"2024-05-08T18:06:53.159086+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:43:02.228949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.865362+0000","last_change":"2024-05-08T18:04:45.820976+0000","last_active":"2024-05-08T18:05:30.865362+0000","last_peered":"2024-05-08T18:05:30.865362+0000","last_clean":"2024-05-08T18:05:30.865362+0000","last_became_active":"2024-05-08T18:04:45.819326+0000","last_became_peered":"2024-05-08T18:04:45.819326+0000","last_unstale":"2024-05-08T18:05:30.865362+0000","last_undegraded":"2024-05-08T18:05:30.865362+0000","last_fullsized":"2024-05-08T18:05:30.865362+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:22:19.585204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862814+0000","last_change":"2024-05-08T18:04:43.840661+0000","last_active":"2024-05-08T18:05:30.862814+0000","last_peered":"2024-05-08T18:05:30.862814+0000","last_clean":"2024-05-08T18:05:30.862814+0000","last_became_active":"2024-05-08T18:04:43.840236+0000","last_became_peered":"2024-05-08T18:04:43.840236+0000","last_unstale":"2024-05-08T18:05:30.862814+0000","last_undegraded":"2024-05-08T18:05:30.862814+0000","last_fullsized":"2024-05-08T18:05:30.862814+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:40:40.407240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859953+0000","last_change":"2024-05-08T18:04:35.552512+0000","last_active":"2024-05-08T18:05:30.859953+0000","last_peered":"2024-05-08T18:05:30.859953+0000","last_clean":"2024-05-08T18:05:30.859953+0000","last_became_active":"2024-05-08T18:04:35.552173+0000","last_became_peered":"2024-05-08T18:04:35.552173+0000","last_unstale":"2024-05-08T18:05:30.859953+0000","last_undegraded":"2024-05-08T18:05:30.859953+0000","last_fullsized":"2024-05-08T18:05:30.859953+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:46:49.443150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"29'1","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.738959+0000","last_change":"2024-05-08T18:04:47.820423+0000","last_active":"2024-05-08T18:06:53.738959+0000","last_peered":"2024-05-08T18:06:53.738959+0000","last_clean":"2024-05-08T18:06:53.738959+0000","last_became_active":"2024-05-08T18:04:47.819910+0000","last_became_peered":"2024-05-08T18:04:47.819910+0000","last_unstale":"2024-05-08T18:06:53.738959+0000","last_undegraded":"2024-05-08T18:06:53.738959+0000","last_fullsized":"2024-05-08T18:06:53.738959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:41:39.241804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"29'61","reported_seq":120,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.738898+0000","last_change":"2024-05-08T18:04:49.831684+0000","last_active":"2024-05-08T18:06:53.738898+0000","last_peered":"2024-05-08T18:06:53.738898+0000","last_clean":"2024-05-08T18:06:53.738898+0000","last_became_active":"2024-05-08T18:04:49.831360+0000","last_became_peered":"2024-05-08T18:04:49.831360+0000","last_unstale":"2024-05-08T18:06:53.738898+0000","last_undegraded":"2024-05-08T18:06:53.738898+0000","last_fullsized":"2024-05-08T18:06:53.738898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:33:52.278907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":59,"num_read_kb":47,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859470+0000","last_change":"2024-05-08T18:04:51.836834+0000","last_active":"2024-05-08T18:05:30.859470+0000","last_peered":"2024-05-08T18:05:30.859470+0000","last_clean":"2024-05-08T18:05:30.859470+0000","last_became_active":"2024-05-08T18:04:51.836441+0000","last_became_peered":"2024-05-08T18:04:51.836441+0000","last_unstale":"2024-05-08T18:05:30.859470+0000","last_undegraded":"2024-05-08T18:05:30.859470+0000","last_fullsized":"2024-05-08T18:05:30.859470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:42.109569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"29'9","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.337708+0000","last_change":"2024-05-08T18:04:53.838192+0000","last_active":"2024-05-08T18:06:53.337708+0000","last_peered":"2024-05-08T18:06:53.337708+0000","last_clean":"2024-05-08T18:06:53.337708+0000","last_became_active":"2024-05-08T18:04:53.837920+0000","last_became_peered":"2024-05-08T18:04:53.837920+0000","last_unstale":"2024-05-08T18:06:53.337708+0000","last_undegraded":"2024-05-08T18:06:53.337708+0000","last_fullsized":"2024-05-08T18:06:53.337708+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:53.793862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"29'15","reported_seq":77,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.042063+0000","last_change":"2024-05-08T18:04:45.821636+0000","last_active":"2024-05-08T18:07:54.042063+0000","last_peered":"2024-05-08T18:07:54.042063+0000","last_clean":"2024-05-08T18:07:54.042063+0000","last_became_active":"2024-05-08T18:04:45.821107+0000","last_became_peered":"2024-05-08T18:04:45.821107+0000","last_unstale":"2024-05-08T18:07:54.042063+0000","last_undegraded":"2024-05-08T18:07:54.042063+0000","last_fullsized":"2024-05-08T18:07:54.042063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:22:55.457884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861385+0000","last_change":"2024-05-08T18:04:43.828309+0000","last_active":"2024-05-08T18:05:30.861385+0000","last_peered":"2024-05-08T18:05:30.861385+0000","last_clean":"2024-05-08T18:05:30.861385+0000","last_became_active":"2024-05-08T18:04:43.827920+0000","last_became_peered":"2024-05-08T18:04:43.827920+0000","last_unstale":"2024-05-08T18:05:30.861385+0000","last_undegraded":"2024-05-08T18:05:30.861385+0000","last_fullsized":"2024-05-08T18:05:30.861385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:41:01.395720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860131+0000","last_change":"2024-05-08T18:04:35.551253+0000","last_active":"2024-05-08T18:05:30.860131+0000","last_peered":"2024-05-08T18:05:30.860131+0000","last_clean":"2024-05-08T18:05:30.860131+0000","last_became_active":"2024-05-08T18:04:35.550912+0000","last_became_peered":"2024-05-08T18:04:35.550912+0000","last_unstale":"2024-05-08T18:05:30.860131+0000","last_undegraded":"2024-05-08T18:05:30.860131+0000","last_fullsized":"2024-05-08T18:05:30.860131+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:36:49.303072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857486+0000","last_change":"2024-05-08T18:04:47.814015+0000","last_active":"2024-05-08T18:05:30.857486+0000","last_peered":"2024-05-08T18:05:30.857486+0000","last_clean":"2024-05-08T18:05:30.857486+0000","last_became_active":"2024-05-08T18:04:47.813717+0000","last_became_peered":"2024-05-08T18:04:47.813717+0000","last_unstale":"2024-05-08T18:05:30.857486+0000","last_undegraded":"2024-05-08T18:05:30.857486+0000","last_fullsized":"2024-05-08T18:05:30.857486+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:52:41.988054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"27'14","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861597+0000","last_change":"2024-05-08T18:04:49.835869+0000","last_active":"2024-05-08T18:05:30.861597+0000","last_peered":"2024-05-08T18:05:30.861597+0000","last_clean":"2024-05-08T18:05:30.861597+0000","last_became_active":"2024-05-08T18:04:49.835451+0000","last_became_peered":"2024-05-08T18:04:49.835451+0000","last_unstale":"2024-05-08T18:05:30.861597+0000","last_undegraded":"2024-05-08T18:05:30.861597+0000","last_fullsized":"2024-05-08T18:05:30.861597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:33.133985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"29'282","reported_seq":369,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903220+0000","last_change":"2024-05-08T18:04:51.832417+0000","last_active":"2024-05-08T18:07:52.903220+0000","last_peered":"2024-05-08T18:07:52.903220+0000","last_clean":"2024-05-08T18:07:52.903220+0000","last_became_active":"2024-05-08T18:04:51.832109+0000","last_became_peered":"2024-05-08T18:04:51.832109+0000","last_unstale":"2024-05-08T18:07:52.903220+0000","last_undegraded":"2024-05-08T18:07:52.903220+0000","last_fullsized":"2024-05-08T18:07:52.903220+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:37.543160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860146+0000","last_change":"2024-05-08T18:04:53.844572+0000","last_active":"2024-05-08T18:05:30.860146+0000","last_peered":"2024-05-08T18:05:30.860146+0000","last_clean":"2024-05-08T18:05:30.860146+0000","last_became_active":"2024-05-08T18:04:53.842998+0000","last_became_peered":"2024-05-08T18:04:53.842998+0000","last_unstale":"2024-05-08T18:05:30.860146+0000","last_undegraded":"2024-05-08T18:05:30.860146+0000","last_fullsized":"2024-05-08T18:05:30.860146+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:25:50.643601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'4","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.775877+0000","last_change":"2024-05-08T18:04:45.830059+0000","last_active":"2024-05-08T18:07:53.775877+0000","last_peered":"2024-05-08T18:07:53.775877+0000","last_clean":"2024-05-08T18:07:53.775877+0000","last_became_active":"2024-05-08T18:04:45.829257+0000","last_became_peered":"2024-05-08T18:04:45.829257+0000","last_unstale":"2024-05-08T18:07:53.775877+0000","last_undegraded":"2024-05-08T18:07:53.775877+0000","last_fullsized":"2024-05-08T18:07:53.775877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:10:31.034888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"29'3","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.536792+0000","last_change":"2024-05-08T18:04:43.835453+0000","last_active":"2024-05-08T18:06:09.536792+0000","last_peered":"2024-05-08T18:06:09.536792+0000","last_clean":"2024-05-08T18:06:09.536792+0000","last_became_active":"2024-05-08T18:04:43.834610+0000","last_became_peered":"2024-05-08T18:04:43.834610+0000","last_unstale":"2024-05-08T18:06:09.536792+0000","last_undegraded":"2024-05-08T18:06:09.536792+0000","last_fullsized":"2024-05-08T18:06:09.536792+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:03.821172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859946+0000","last_change":"2024-05-08T18:04:35.551595+0000","last_active":"2024-05-08T18:05:30.859946+0000","last_peered":"2024-05-08T18:05:30.859946+0000","last_clean":"2024-05-08T18:05:30.859946+0000","last_became_active":"2024-05-08T18:04:35.551366+0000","last_became_peered":"2024-05-08T18:04:35.551366+0000","last_unstale":"2024-05-08T18:05:30.859946+0000","last_undegraded":"2024-05-08T18:05:30.859946+0000","last_fullsized":"2024-05-08T18:05:30.859946+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:05:00.951608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857461+0000","last_change":"2024-05-08T18:04:47.822209+0000","last_active":"2024-05-08T18:05:30.857461+0000","last_peered":"2024-05-08T18:05:30.857461+0000","last_clean":"2024-05-08T18:05:30.857461+0000","last_became_active":"2024-05-08T18:04:47.819866+0000","last_became_peered":"2024-05-08T18:04:47.819866+0000","last_unstale":"2024-05-08T18:05:30.857461+0000","last_undegraded":"2024-05-08T18:05:30.857461+0000","last_fullsized":"2024-05-08T18:05:30.857461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:28.233298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"27'10","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858934+0000","last_change":"2024-05-08T18:04:49.832338+0000","last_active":"2024-05-08T18:05:30.858934+0000","last_peered":"2024-05-08T18:05:30.858934+0000","last_clean":"2024-05-08T18:05:30.858934+0000","last_became_active":"2024-05-08T18:04:49.830452+0000","last_became_peered":"2024-05-08T18:04:49.830452+0000","last_unstale":"2024-05-08T18:05:30.858934+0000","last_undegraded":"2024-05-08T18:05:30.858934+0000","last_fullsized":"2024-05-08T18:05:30.858934+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:24:49.779432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"29'282","reported_seq":399,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.172497+0000","last_change":"2024-05-08T18:04:51.831530+0000","last_active":"2024-05-08T18:07:54.172497+0000","last_peered":"2024-05-08T18:07:54.172497+0000","last_clean":"2024-05-08T18:07:54.172497+0000","last_became_active":"2024-05-08T18:04:51.831155+0000","last_became_peered":"2024-05-08T18:04:51.831155+0000","last_unstale":"2024-05-08T18:07:54.172497+0000","last_undegraded":"2024-05-08T18:07:54.172497+0000","last_fullsized":"2024-05-08T18:07:54.172497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:58:32.247507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'5","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.023861+0000","last_change":"2024-05-08T18:04:53.849164+0000","last_active":"2024-05-08T18:07:54.023861+0000","last_peered":"2024-05-08T18:07:54.023861+0000","last_clean":"2024-05-08T18:07:54.023861+0000","last_became_active":"2024-05-08T18:04:53.848862+0000","last_became_peered":"2024-05-08T18:04:53.848862+0000","last_unstale":"2024-05-08T18:07:54.023861+0000","last_undegraded":"2024-05-08T18:07:54.023861+0000","last_fullsized":"2024-05-08T18:07:54.023861+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:33:19.797257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"29'6","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.867036+0000","last_change":"2024-05-08T18:04:45.823169+0000","last_active":"2024-05-08T18:07:53.867036+0000","last_peered":"2024-05-08T18:07:53.867036+0000","last_clean":"2024-05-08T18:07:53.867036+0000","last_became_active":"2024-05-08T18:04:45.822824+0000","last_became_peered":"2024-05-08T18:04:45.822824+0000","last_unstale":"2024-05-08T18:07:53.867036+0000","last_undegraded":"2024-05-08T18:07:53.867036+0000","last_fullsized":"2024-05-08T18:07:53.867036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:34.503647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862506+0000","last_change":"2024-05-08T18:04:43.839633+0000","last_active":"2024-05-08T18:05:30.862506+0000","last_peered":"2024-05-08T18:05:30.862506+0000","last_clean":"2024-05-08T18:05:30.862506+0000","last_became_active":"2024-05-08T18:04:43.839240+0000","last_became_peered":"2024-05-08T18:04:43.839240+0000","last_unstale":"2024-05-08T18:05:30.862506+0000","last_undegraded":"2024-05-08T18:05:30.862506+0000","last_fullsized":"2024-05-08T18:05:30.862506+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:03:31.909506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864842+0000","last_change":"2024-05-08T18:04:35.551765+0000","last_active":"2024-05-08T18:05:30.864842+0000","last_peered":"2024-05-08T18:05:30.864842+0000","last_clean":"2024-05-08T18:05:30.864842+0000","last_became_active":"2024-05-08T18:04:35.551412+0000","last_became_peered":"2024-05-08T18:04:35.551412+0000","last_unstale":"2024-05-08T18:05:30.864842+0000","last_undegraded":"2024-05-08T18:05:30.864842+0000","last_fullsized":"2024-05-08T18:05:30.864842+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:37:38.776558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857179+0000","last_change":"2024-05-08T18:04:47.816250+0000","last_active":"2024-05-08T18:05:30.857179+0000","last_peered":"2024-05-08T18:05:30.857179+0000","last_clean":"2024-05-08T18:05:30.857179+0000","last_became_active":"2024-05-08T18:04:47.815947+0000","last_became_peered":"2024-05-08T18:04:47.815947+0000","last_unstale":"2024-05-08T18:05:30.857179+0000","last_undegraded":"2024-05-08T18:05:30.857179+0000","last_fullsized":"2024-05-08T18:05:30.857179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:16:57.162048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"29'82","reported_seq":154,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.952921+0000","last_change":"2024-05-08T18:04:49.834985+0000","last_active":"2024-05-08T18:06:51.952921+0000","last_peered":"2024-05-08T18:06:51.952921+0000","last_clean":"2024-05-08T18:06:51.952921+0000","last_became_active":"2024-05-08T18:04:49.834586+0000","last_became_peered":"2024-05-08T18:04:49.834586+0000","last_unstale":"2024-05-08T18:06:51.952921+0000","last_undegraded":"2024-05-08T18:06:51.952921+0000","last_fullsized":"2024-05-08T18:06:51.952921+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:26.820238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":75,"num_read_kb":61,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858181+0000","last_change":"2024-05-08T18:04:51.831135+0000","last_active":"2024-05-08T18:05:30.858181+0000","last_peered":"2024-05-08T18:05:30.858181+0000","last_clean":"2024-05-08T18:05:30.858181+0000","last_became_active":"2024-05-08T18:04:51.830760+0000","last_became_peered":"2024-05-08T18:04:51.830760+0000","last_unstale":"2024-05-08T18:05:30.858181+0000","last_undegraded":"2024-05-08T18:05:30.858181+0000","last_fullsized":"2024-05-08T18:05:30.858181+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:15.559137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.865409+0000","last_change":"2024-05-08T18:04:53.839069+0000","last_active":"2024-05-08T18:05:30.865409+0000","last_peered":"2024-05-08T18:05:30.865409+0000","last_clean":"2024-05-08T18:05:30.865409+0000","last_became_active":"2024-05-08T18:04:53.838600+0000","last_became_peered":"2024-05-08T18:04:53.838600+0000","last_unstale":"2024-05-08T18:05:30.865409+0000","last_undegraded":"2024-05-08T18:05:30.865409+0000","last_fullsized":"2024-05-08T18:05:30.865409+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:17.566875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"27'2","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866690+0000","last_change":"2024-05-08T18:04:53.839793+0000","last_active":"2024-05-08T18:05:30.866690+0000","last_peered":"2024-05-08T18:05:30.866690+0000","last_clean":"2024-05-08T18:05:30.866690+0000","last_became_active":"2024-05-08T18:04:53.839500+0000","last_became_peered":"2024-05-08T18:04:53.839500+0000","last_unstale":"2024-05-08T18:05:30.866690+0000","last_undegraded":"2024-05-08T18:05:30.866690+0000","last_fullsized":"2024-05-08T18:05:30.866690+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:19.575012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"29'8","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.219834+0000","last_change":"2024-05-08T18:04:45.819806+0000","last_active":"2024-05-08T18:06:50.219834+0000","last_peered":"2024-05-08T18:06:50.219834+0000","last_clean":"2024-05-08T18:06:50.219834+0000","last_became_active":"2024-05-08T18:04:45.819508+0000","last_became_peered":"2024-05-08T18:04:45.819508+0000","last_unstale":"2024-05-08T18:06:50.219834+0000","last_undegraded":"2024-05-08T18:06:50.219834+0000","last_fullsized":"2024-05-08T18:06:50.219834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:32.833248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859163+0000","last_change":"2024-05-08T18:04:43.844864+0000","last_active":"2024-05-08T18:05:30.859163+0000","last_peered":"2024-05-08T18:05:30.859163+0000","last_clean":"2024-05-08T18:05:30.859163+0000","last_became_active":"2024-05-08T18:04:43.837929+0000","last_became_peered":"2024-05-08T18:04:43.837929+0000","last_unstale":"2024-05-08T18:05:30.859163+0000","last_undegraded":"2024-05-08T18:05:30.859163+0000","last_fullsized":"2024-05-08T18:05:30.859163+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:12:06.177343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861102+0000","last_change":"2024-05-08T18:04:47.818763+0000","last_active":"2024-05-08T18:05:30.861102+0000","last_peered":"2024-05-08T18:05:30.861102+0000","last_clean":"2024-05-08T18:05:30.861102+0000","last_became_active":"2024-05-08T18:04:47.818443+0000","last_became_peered":"2024-05-08T18:04:47.818443+0000","last_unstale":"2024-05-08T18:05:30.861102+0000","last_undegraded":"2024-05-08T18:05:30.861102+0000","last_fullsized":"2024-05-08T18:05:30.861102+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:15:57.213130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"29'32","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.958612+0000","last_change":"2024-05-08T18:04:49.834233+0000","last_active":"2024-05-08T18:06:51.958612+0000","last_peered":"2024-05-08T18:06:51.958612+0000","last_clean":"2024-05-08T18:06:51.958612+0000","last_became_active":"2024-05-08T18:04:49.833599+0000","last_became_peered":"2024-05-08T18:04:49.833599+0000","last_unstale":"2024-05-08T18:06:51.958612+0000","last_undegraded":"2024-05-08T18:06:51.958612+0000","last_fullsized":"2024-05-08T18:06:51.958612+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:54:34.743324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858258+0000","last_change":"2024-05-08T18:04:51.844827+0000","last_active":"2024-05-08T18:05:30.858258+0000","last_peered":"2024-05-08T18:05:30.858258+0000","last_clean":"2024-05-08T18:05:30.858258+0000","last_became_active":"2024-05-08T18:04:51.844526+0000","last_became_peered":"2024-05-08T18:04:51.844526+0000","last_unstale":"2024-05-08T18:05:30.858258+0000","last_undegraded":"2024-05-08T18:05:30.858258+0000","last_fullsized":"2024-05-08T18:05:30.858258+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:13:54.471619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"29'11","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.639339+0000","last_change":"2024-05-08T18:04:53.850911+0000","last_active":"2024-05-08T18:06:51.639339+0000","last_peered":"2024-05-08T18:06:51.639339+0000","last_clean":"2024-05-08T18:06:51.639339+0000","last_became_active":"2024-05-08T18:04:53.850593+0000","last_became_peered":"2024-05-08T18:04:53.850593+0000","last_unstale":"2024-05-08T18:06:51.639339+0000","last_undegraded":"2024-05-08T18:06:51.639339+0000","last_fullsized":"2024-05-08T18:06:51.639339+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:11:21.798553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":62,"num_write":31,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"29'8","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399973+0000","last_change":"2024-05-08T18:04:45.819974+0000","last_active":"2024-05-08T18:06:53.399973+0000","last_peered":"2024-05-08T18:06:53.399973+0000","last_clean":"2024-05-08T18:06:53.399973+0000","last_became_active":"2024-05-08T18:04:45.818979+0000","last_became_peered":"2024-05-08T18:04:45.818979+0000","last_unstale":"2024-05-08T18:06:53.399973+0000","last_undegraded":"2024-05-08T18:06:53.399973+0000","last_fullsized":"2024-05-08T18:06:53.399973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:30.458103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863085+0000","last_change":"2024-05-08T18:04:43.836993+0000","last_active":"2024-05-08T18:05:30.863085+0000","last_peered":"2024-05-08T18:05:30.863085+0000","last_clean":"2024-05-08T18:05:30.863085+0000","last_became_active":"2024-05-08T18:04:43.836614+0000","last_became_peered":"2024-05-08T18:04:43.836614+0000","last_unstale":"2024-05-08T18:05:30.863085+0000","last_undegraded":"2024-05-08T18:05:30.863085+0000","last_fullsized":"2024-05-08T18:05:30.863085+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:05:33.776915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858802+0000","last_change":"2024-05-08T18:04:47.814847+0000","last_active":"2024-05-08T18:05:30.858802+0000","last_peered":"2024-05-08T18:05:30.858802+0000","last_clean":"2024-05-08T18:05:30.858802+0000","last_became_active":"2024-05-08T18:04:47.814561+0000","last_became_peered":"2024-05-08T18:04:47.814561+0000","last_unstale":"2024-05-08T18:05:30.858802+0000","last_undegraded":"2024-05-08T18:05:30.858802+0000","last_fullsized":"2024-05-08T18:05:30.858802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:13:58.115834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"29'7","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.433454+0000","last_change":"2024-05-08T18:04:49.825623+0000","last_active":"2024-05-08T18:07:53.433454+0000","last_peered":"2024-05-08T18:07:53.433454+0000","last_clean":"2024-05-08T18:07:53.433454+0000","last_became_active":"2024-05-08T18:04:49.825347+0000","last_became_peered":"2024-05-08T18:04:49.825347+0000","last_unstale":"2024-05-08T18:07:53.433454+0000","last_undegraded":"2024-05-08T18:07:53.433454+0000","last_fullsized":"2024-05-08T18:07:53.433454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:37.454897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.863344+0000","last_change":"2024-05-08T18:04:51.843388+0000","last_active":"2024-05-08T18:05:30.863344+0000","last_peered":"2024-05-08T18:05:30.863344+0000","last_clean":"2024-05-08T18:05:30.863344+0000","last_became_active":"2024-05-08T18:04:51.843002+0000","last_became_peered":"2024-05-08T18:04:51.843002+0000","last_unstale":"2024-05-08T18:05:30.863344+0000","last_undegraded":"2024-05-08T18:05:30.863344+0000","last_fullsized":"2024-05-08T18:05:30.863344+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:13.211841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859114+0000","last_change":"2024-05-08T18:04:53.840844+0000","last_active":"2024-05-08T18:05:30.859114+0000","last_peered":"2024-05-08T18:05:30.859114+0000","last_clean":"2024-05-08T18:05:30.859114+0000","last_became_active":"2024-05-08T18:04:53.840437+0000","last_became_peered":"2024-05-08T18:04:53.840437+0000","last_unstale":"2024-05-08T18:05:30.859114+0000","last_undegraded":"2024-05-08T18:05:30.859114+0000","last_fullsized":"2024-05-08T18:05:30.859114+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:42:52.195643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"29'8","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.861059+0000","last_change":"2024-05-08T18:05:30.868442+0000","last_active":"2024-05-08T18:07:53.861059+0000","last_peered":"2024-05-08T18:07:53.861059+0000","last_clean":"2024-05-08T18:07:53.861059+0000","last_became_active":"2024-05-08T18:05:30.868156+0000","last_became_peered":"2024-05-08T18:05:30.868156+0000","last_unstale":"2024-05-08T18:07:53.861059+0000","last_undegraded":"2024-05-08T18:07:53.861059+0000","last_fullsized":"2024-05-08T18:07:53.861059+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:44.461215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867524+0000","last_change":"2024-05-08T18:04:43.835856+0000","last_active":"2024-05-08T18:05:30.867524+0000","last_peered":"2024-05-08T18:05:30.867524+0000","last_clean":"2024-05-08T18:05:30.867524+0000","last_became_active":"2024-05-08T18:04:43.835435+0000","last_became_peered":"2024-05-08T18:04:43.835435+0000","last_unstale":"2024-05-08T18:05:30.867524+0000","last_undegraded":"2024-05-08T18:05:30.867524+0000","last_fullsized":"2024-05-08T18:05:30.867524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:47.971014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860152+0000","last_change":"2024-05-08T18:04:47.824614+0000","last_active":"2024-05-08T18:05:30.860152+0000","last_peered":"2024-05-08T18:05:30.860152+0000","last_clean":"2024-05-08T18:05:30.860152+0000","last_became_active":"2024-05-08T18:04:47.824140+0000","last_became_peered":"2024-05-08T18:04:47.824140+0000","last_unstale":"2024-05-08T18:05:30.860152+0000","last_undegraded":"2024-05-08T18:05:30.860152+0000","last_fullsized":"2024-05-08T18:05:30.860152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:36:47.466900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"29'41","reported_seq":100,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.004393+0000","last_change":"2024-05-08T18:04:49.824946+0000","last_active":"2024-05-08T18:07:54.004393+0000","last_peered":"2024-05-08T18:07:54.004393+0000","last_clean":"2024-05-08T18:07:54.004393+0000","last_became_active":"2024-05-08T18:04:49.824600+0000","last_became_peered":"2024-05-08T18:04:49.824600+0000","last_unstale":"2024-05-08T18:07:54.004393+0000","last_undegraded":"2024-05-08T18:07:54.004393+0000","last_fullsized":"2024-05-08T18:07:54.004393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:11.205149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":64,"num_write":46,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860821+0000","last_change":"2024-05-08T18:04:51.833599+0000","last_active":"2024-05-08T18:05:30.860821+0000","last_peered":"2024-05-08T18:05:30.860821+0000","last_clean":"2024-05-08T18:05:30.860821+0000","last_became_active":"2024-05-08T18:04:51.829937+0000","last_became_peered":"2024-05-08T18:04:51.829937+0000","last_unstale":"2024-05-08T18:05:30.860821+0000","last_undegraded":"2024-05-08T18:05:30.860821+0000","last_fullsized":"2024-05-08T18:05:30.860821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:02:48.266056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858578+0000","last_change":"2024-05-08T18:04:53.842981+0000","last_active":"2024-05-08T18:05:30.858578+0000","last_peered":"2024-05-08T18:05:30.858578+0000","last_clean":"2024-05-08T18:05:30.858578+0000","last_became_active":"2024-05-08T18:04:53.842610+0000","last_became_peered":"2024-05-08T18:04:53.842610+0000","last_unstale":"2024-05-08T18:05:30.858578+0000","last_undegraded":"2024-05-08T18:05:30.858578+0000","last_fullsized":"2024-05-08T18:05:30.858578+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:16:15.240226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857482+0000","last_change":"2024-05-08T18:04:45.829838+0000","last_active":"2024-05-08T18:05:30.857482+0000","last_peered":"2024-05-08T18:05:30.857482+0000","last_clean":"2024-05-08T18:05:30.857482+0000","last_became_active":"2024-05-08T18:04:45.829458+0000","last_became_peered":"2024-05-08T18:04:45.829458+0000","last_unstale":"2024-05-08T18:05:30.857482+0000","last_undegraded":"2024-05-08T18:05:30.857482+0000","last_fullsized":"2024-05-08T18:05:30.857482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:15:55.206029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"29'4","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.352042+0000","last_change":"2024-05-08T18:04:43.837089+0000","last_active":"2024-05-08T18:06:53.352042+0000","last_peered":"2024-05-08T18:06:53.352042+0000","last_clean":"2024-05-08T18:06:53.352042+0000","last_became_active":"2024-05-08T18:04:43.836633+0000","last_became_peered":"2024-05-08T18:04:43.836633+0000","last_unstale":"2024-05-08T18:06:53.352042+0000","last_undegraded":"2024-05-08T18:06:53.352042+0000","last_fullsized":"2024-05-08T18:06:53.352042+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:55.493870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"20'1","reported_seq":218,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.049308+0000","last_change":"2024-05-08T18:04:47.820110+0000","last_active":"2024-05-08T18:06:53.049308+0000","last_peered":"2024-05-08T18:06:53.049308+0000","last_clean":"2024-05-08T18:06:53.049308+0000","last_became_active":"2024-05-08T18:04:47.819771+0000","last_became_peered":"2024-05-08T18:04:47.819771+0000","last_unstale":"2024-05-08T18:06:53.049308+0000","last_undegraded":"2024-05-08T18:06:53.049308+0000","last_fullsized":"2024-05-08T18:06:53.049308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:42:21.961312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"29'30","reported_seq":72,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.944676+0000","last_change":"2024-05-08T18:04:49.824323+0000","last_active":"2024-05-08T18:06:51.944676+0000","last_peered":"2024-05-08T18:06:51.944676+0000","last_clean":"2024-05-08T18:06:51.944676+0000","last_became_active":"2024-05-08T18:04:49.824022+0000","last_became_peered":"2024-05-08T18:04:49.824022+0000","last_unstale":"2024-05-08T18:06:51.944676+0000","last_undegraded":"2024-05-08T18:06:51.944676+0000","last_fullsized":"2024-05-08T18:06:51.944676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:41:41.249321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866329+0000","last_change":"2024-05-08T18:04:51.826713+0000","last_active":"2024-05-08T18:05:30.866329+0000","last_peered":"2024-05-08T18:05:30.866329+0000","last_clean":"2024-05-08T18:05:30.866329+0000","last_became_active":"2024-05-08T18:04:51.826424+0000","last_became_peered":"2024-05-08T18:04:51.826424+0000","last_unstale":"2024-05-08T18:05:30.866329+0000","last_undegraded":"2024-05-08T18:05:30.866329+0000","last_fullsized":"2024-05-08T18:05:30.866329+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:33:54.285599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"29'20","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995883+0000","last_change":"2024-05-08T18:05:30.873315+0000","last_active":"2024-05-08T18:07:53.995883+0000","last_peered":"2024-05-08T18:07:53.995883+0000","last_clean":"2024-05-08T18:07:53.995883+0000","last_became_active":"2024-05-08T18:05:30.868409+0000","last_became_peered":"2024-05-08T18:05:30.868409+0000","last_unstale":"2024-05-08T18:07:53.995883+0000","last_undegraded":"2024-05-08T18:07:53.995883+0000","last_fullsized":"2024-05-08T18:07:53.995883+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:32:07.623428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857341+0000","last_change":"2024-05-08T18:04:43.844983+0000","last_active":"2024-05-08T18:05:30.857341+0000","last_peered":"2024-05-08T18:05:30.857341+0000","last_clean":"2024-05-08T18:05:30.857341+0000","last_became_active":"2024-05-08T18:04:43.838957+0000","last_became_peered":"2024-05-08T18:04:43.838957+0000","last_unstale":"2024-05-08T18:05:30.857341+0000","last_undegraded":"2024-05-08T18:05:30.857341+0000","last_fullsized":"2024-05-08T18:05:30.857341+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:42:05.210775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859885+0000","last_change":"2024-05-08T18:04:47.820891+0000","last_active":"2024-05-08T18:05:30.859885+0000","last_peered":"2024-05-08T18:05:30.859885+0000","last_clean":"2024-05-08T18:05:30.859885+0000","last_became_active":"2024-05-08T18:04:47.820256+0000","last_became_peered":"2024-05-08T18:04:47.820256+0000","last_unstale":"2024-05-08T18:05:30.859885+0000","last_undegraded":"2024-05-08T18:05:30.859885+0000","last_fullsized":"2024-05-08T18:05:30.859885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:04:15.231714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"29'54","reported_seq":110,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.964783+0000","last_change":"2024-05-08T18:04:49.825705+0000","last_active":"2024-05-08T18:06:51.964783+0000","last_peered":"2024-05-08T18:06:51.964783+0000","last_clean":"2024-05-08T18:06:51.964783+0000","last_became_active":"2024-05-08T18:04:49.825366+0000","last_became_peered":"2024-05-08T18:04:49.825366+0000","last_unstale":"2024-05-08T18:06:51.964783+0000","last_undegraded":"2024-05-08T18:06:51.964783+0000","last_fullsized":"2024-05-08T18:06:51.964783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:27:27.454431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":40,"num_write":18,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862900+0000","last_change":"2024-05-08T18:04:51.827568+0000","last_active":"2024-05-08T18:05:30.862900+0000","last_peered":"2024-05-08T18:05:30.862900+0000","last_clean":"2024-05-08T18:05:30.862900+0000","last_became_active":"2024-05-08T18:04:51.827127+0000","last_became_peered":"2024-05-08T18:04:51.827127+0000","last_unstale":"2024-05-08T18:05:30.862900+0000","last_undegraded":"2024-05-08T18:05:30.862900+0000","last_fullsized":"2024-05-08T18:05:30.862900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:27:29.461124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"29'14","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.333941+0000","last_change":"2024-05-08T18:04:53.844087+0000","last_active":"2024-05-08T18:06:53.333941+0000","last_peered":"2024-05-08T18:06:53.333941+0000","last_clean":"2024-05-08T18:06:53.333941+0000","last_became_active":"2024-05-08T18:04:53.843772+0000","last_became_peered":"2024-05-08T18:04:53.843772+0000","last_unstale":"2024-05-08T18:06:53.333941+0000","last_undegraded":"2024-05-08T18:06:53.333941+0000","last_fullsized":"2024-05-08T18:06:53.333941+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:26.083815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.738215+0000","last_change":"2024-05-08T18:04:45.811909+0000","last_active":"2024-05-08T18:06:48.738215+0000","last_peered":"2024-05-08T18:06:48.738215+0000","last_clean":"2024-05-08T18:06:48.738215+0000","last_became_active":"2024-05-08T18:04:45.811517+0000","last_became_peered":"2024-05-08T18:04:45.811517+0000","last_unstale":"2024-05-08T18:06:48.738215+0000","last_undegraded":"2024-05-08T18:06:48.738215+0000","last_fullsized":"2024-05-08T18:06:48.738215+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:32.141179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860711+0000","last_change":"2024-05-08T18:04:43.836313+0000","last_active":"2024-05-08T18:05:30.860711+0000","last_peered":"2024-05-08T18:05:30.860711+0000","last_clean":"2024-05-08T18:05:30.860711+0000","last_became_active":"2024-05-08T18:04:43.834767+0000","last_became_peered":"2024-05-08T18:04:43.834767+0000","last_unstale":"2024-05-08T18:05:30.860711+0000","last_undegraded":"2024-05-08T18:05:30.860711+0000","last_fullsized":"2024-05-08T18:05:30.860711+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:22.454546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0,2],"acting":[1,3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867042+0000","last_change":"2024-05-08T18:04:47.821012+0000","last_active":"2024-05-08T18:05:30.867042+0000","last_peered":"2024-05-08T18:05:30.867042+0000","last_clean":"2024-05-08T18:05:30.867042+0000","last_became_active":"2024-05-08T18:04:47.820662+0000","last_became_peered":"2024-05-08T18:04:47.820662+0000","last_unstale":"2024-05-08T18:05:30.867042+0000","last_undegraded":"2024-05-08T18:05:30.867042+0000","last_fullsized":"2024-05-08T18:05:30.867042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:29:01.485039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"29'37","reported_seq":82,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.401221+0000","last_change":"2024-05-08T18:04:49.831871+0000","last_active":"2024-05-08T18:06:53.401221+0000","last_peered":"2024-05-08T18:06:53.401221+0000","last_clean":"2024-05-08T18:06:53.401221+0000","last_became_active":"2024-05-08T18:04:49.830476+0000","last_became_peered":"2024-05-08T18:04:49.830476+0000","last_unstale":"2024-05-08T18:06:53.401221+0000","last_undegraded":"2024-05-08T18:06:53.401221+0000","last_fullsized":"2024-05-08T18:06:53.401221+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:46.223733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862181+0000","last_change":"2024-05-08T18:04:51.845269+0000","last_active":"2024-05-08T18:05:30.862181+0000","last_peered":"2024-05-08T18:05:30.862181+0000","last_clean":"2024-05-08T18:05:30.862181+0000","last_became_active":"2024-05-08T18:04:51.844912+0000","last_became_peered":"2024-05-08T18:04:51.844912+0000","last_unstale":"2024-05-08T18:05:30.862181+0000","last_undegraded":"2024-05-08T18:05:30.862181+0000","last_fullsized":"2024-05-08T18:05:30.862181+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:17.567274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"29'2","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.037376+0000","last_change":"2024-05-08T18:04:53.847736+0000","last_active":"2024-05-08T18:07:54.037376+0000","last_peered":"2024-05-08T18:07:54.037376+0000","last_clean":"2024-05-08T18:07:54.037376+0000","last_became_active":"2024-05-08T18:04:53.847413+0000","last_became_peered":"2024-05-08T18:04:53.847413+0000","last_unstale":"2024-05-08T18:07:54.037376+0000","last_undegraded":"2024-05-08T18:07:54.037376+0000","last_fullsized":"2024-05-08T18:07:54.037376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:10:18.059582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"29'12","reported_seq":60,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.041935+0000","last_change":"2024-05-08T18:04:45.830619+0000","last_active":"2024-05-08T18:07:54.041935+0000","last_peered":"2024-05-08T18:07:54.041935+0000","last_clean":"2024-05-08T18:07:54.041935+0000","last_became_active":"2024-05-08T18:04:45.830329+0000","last_became_peered":"2024-05-08T18:04:45.830329+0000","last_unstale":"2024-05-08T18:07:54.041935+0000","last_undegraded":"2024-05-08T18:07:54.041935+0000","last_fullsized":"2024-05-08T18:07:54.041935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:13:12.748032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861213+0000","last_change":"2024-05-08T18:04:43.835662+0000","last_active":"2024-05-08T18:05:30.861213+0000","last_peered":"2024-05-08T18:05:30.861213+0000","last_clean":"2024-05-08T18:05:30.861213+0000","last_became_active":"2024-05-08T18:04:43.835227+0000","last_became_peered":"2024-05-08T18:04:43.835227+0000","last_unstale":"2024-05-08T18:05:30.861213+0000","last_undegraded":"2024-05-08T18:05:30.861213+0000","last_fullsized":"2024-05-08T18:05:30.861213+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:10:10.360847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3,2],"acting":[1,0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.871559+0000","last_change":"2024-05-08T18:05:30.870636+0000","last_active":"2024-05-08T18:05:30.871559+0000","last_peered":"2024-05-08T18:05:30.871559+0000","last_clean":"2024-05-08T18:05:30.871559+0000","last_became_active":"2024-05-08T18:05:30.870155+0000","last_became_peered":"2024-05-08T18:05:30.870155+0000","last_unstale":"2024-05-08T18:05:30.871559+0000","last_undegraded":"2024-05-08T18:05:30.871559+0000","last_fullsized":"2024-05-08T18:05:30.871559+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:56.207001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"29'32","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.962760+0000","last_change":"2024-05-08T18:04:49.832968+0000","last_active":"2024-05-08T18:06:51.962760+0000","last_peered":"2024-05-08T18:06:51.962760+0000","last_clean":"2024-05-08T18:06:51.962760+0000","last_became_active":"2024-05-08T18:04:49.832649+0000","last_became_peered":"2024-05-08T18:04:49.832649+0000","last_unstale":"2024-05-08T18:06:51.962760+0000","last_undegraded":"2024-05-08T18:06:51.962760+0000","last_fullsized":"2024-05-08T18:06:51.962760+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:52.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859450+0000","last_change":"2024-05-08T18:04:51.833712+0000","last_active":"2024-05-08T18:05:30.859450+0000","last_peered":"2024-05-08T18:05:30.859450+0000","last_clean":"2024-05-08T18:05:30.859450+0000","last_became_active":"2024-05-08T18:04:51.833421+0000","last_became_peered":"2024-05-08T18:04:51.833421+0000","last_unstale":"2024-05-08T18:05:30.859450+0000","last_undegraded":"2024-05-08T18:05:30.859450+0000","last_fullsized":"2024-05-08T18:05:30.859450+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:48.752288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"29'5","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.394775+0000","last_change":"2024-05-08T18:04:53.848762+0000","last_active":"2024-05-08T18:06:53.394775+0000","last_peered":"2024-05-08T18:06:53.394775+0000","last_clean":"2024-05-08T18:06:53.394775+0000","last_became_active":"2024-05-08T18:04:53.848430+0000","last_became_peered":"2024-05-08T18:04:53.848430+0000","last_unstale":"2024-05-08T18:06:53.394775+0000","last_undegraded":"2024-05-08T18:06:53.394775+0000","last_fullsized":"2024-05-08T18:06:53.394775+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:48.070547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"29'10","reported_seq":66,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.218980+0000","last_change":"2024-05-08T18:04:45.807312+0000","last_active":"2024-05-08T18:06:50.218980+0000","last_peered":"2024-05-08T18:06:50.218980+0000","last_clean":"2024-05-08T18:06:50.218980+0000","last_became_active":"2024-05-08T18:04:45.806808+0000","last_became_peered":"2024-05-08T18:04:45.806808+0000","last_unstale":"2024-05-08T18:06:50.218980+0000","last_undegraded":"2024-05-08T18:06:50.218980+0000","last_fullsized":"2024-05-08T18:06:50.218980+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:27:20.114031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859441+0000","last_change":"2024-05-08T18:04:43.833094+0000","last_active":"2024-05-08T18:05:30.859441+0000","last_peered":"2024-05-08T18:05:30.859441+0000","last_clean":"2024-05-08T18:05:30.859441+0000","last_became_active":"2024-05-08T18:04:43.832147+0000","last_became_peered":"2024-05-08T18:04:43.832147+0000","last_unstale":"2024-05-08T18:05:30.859441+0000","last_undegraded":"2024-05-08T18:05:30.859441+0000","last_fullsized":"2024-05-08T18:05:30.859441+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:49.361708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862397+0000","last_change":"2024-05-08T18:04:47.815796+0000","last_active":"2024-05-08T18:05:30.862397+0000","last_peered":"2024-05-08T18:05:30.862397+0000","last_clean":"2024-05-08T18:05:30.862397+0000","last_became_active":"2024-05-08T18:04:47.815219+0000","last_became_peered":"2024-05-08T18:04:47.815219+0000","last_unstale":"2024-05-08T18:05:30.862397+0000","last_undegraded":"2024-05-08T18:05:30.862397+0000","last_fullsized":"2024-05-08T18:05:30.862397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:23:20.608088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"27'10","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861853+0000","last_change":"2024-05-08T18:04:49.833674+0000","last_active":"2024-05-08T18:05:30.861853+0000","last_peered":"2024-05-08T18:05:30.861853+0000","last_clean":"2024-05-08T18:05:30.861853+0000","last_became_active":"2024-05-08T18:04:49.830793+0000","last_became_peered":"2024-05-08T18:04:49.830793+0000","last_unstale":"2024-05-08T18:05:30.861853+0000","last_undegraded":"2024-05-08T18:05:30.861853+0000","last_fullsized":"2024-05-08T18:05:30.861853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:54:53.163574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"29'282","reported_seq":366,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:52.903209+0000","last_change":"2024-05-08T18:04:51.834224+0000","last_active":"2024-05-08T18:07:52.903209+0000","last_peered":"2024-05-08T18:07:52.903209+0000","last_clean":"2024-05-08T18:07:52.903209+0000","last_became_active":"2024-05-08T18:04:51.833866+0000","last_became_peered":"2024-05-08T18:04:51.833866+0000","last_unstale":"2024-05-08T18:07:52.903209+0000","last_undegraded":"2024-05-08T18:07:52.903209+0000","last_fullsized":"2024-05-08T18:07:52.903209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:21:20.452363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.856865+0000","last_change":"2024-05-08T18:04:53.848804+0000","last_active":"2024-05-08T18:05:30.856865+0000","last_peered":"2024-05-08T18:05:30.856865+0000","last_clean":"2024-05-08T18:05:30.856865+0000","last_became_active":"2024-05-08T18:04:53.848321+0000","last_became_peered":"2024-05-08T18:04:53.848321+0000","last_unstale":"2024-05-08T18:05:30.856865+0000","last_undegraded":"2024-05-08T18:05:30.856865+0000","last_fullsized":"2024-05-08T18:05:30.856865+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:09:34.662852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"29'4","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399620+0000","last_change":"2024-05-08T18:04:45.814018+0000","last_active":"2024-05-08T18:06:53.399620+0000","last_peered":"2024-05-08T18:06:53.399620+0000","last_clean":"2024-05-08T18:06:53.399620+0000","last_became_active":"2024-05-08T18:04:45.813424+0000","last_became_peered":"2024-05-08T18:04:45.813424+0000","last_unstale":"2024-05-08T18:06:53.399620+0000","last_undegraded":"2024-05-08T18:06:53.399620+0000","last_fullsized":"2024-05-08T18:06:53.399620+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:42:05.254786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866726+0000","last_change":"2024-05-08T18:04:43.841603+0000","last_active":"2024-05-08T18:05:30.866726+0000","last_peered":"2024-05-08T18:05:30.866726+0000","last_clean":"2024-05-08T18:05:30.866726+0000","last_became_active":"2024-05-08T18:04:43.840502+0000","last_became_peered":"2024-05-08T18:04:43.840502+0000","last_unstale":"2024-05-08T18:05:30.866726+0000","last_undegraded":"2024-05-08T18:05:30.866726+0000","last_fullsized":"2024-05-08T18:05:30.866726+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:40:40.407240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860986+0000","last_change":"2024-05-08T18:04:47.819165+0000","last_active":"2024-05-08T18:05:30.860986+0000","last_peered":"2024-05-08T18:05:30.860986+0000","last_clean":"2024-05-08T18:05:30.860986+0000","last_became_active":"2024-05-08T18:04:47.818839+0000","last_became_peered":"2024-05-08T18:04:47.818839+0000","last_unstale":"2024-05-08T18:05:30.860986+0000","last_undegraded":"2024-05-08T18:05:30.860986+0000","last_fullsized":"2024-05-08T18:05:30.860986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:04:15.231714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"29'37","reported_seq":82,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.936372+0000","last_change":"2024-05-08T18:04:49.824708+0000","last_active":"2024-05-08T18:06:51.936372+0000","last_peered":"2024-05-08T18:06:51.936372+0000","last_clean":"2024-05-08T18:06:51.936372+0000","last_became_active":"2024-05-08T18:04:49.824405+0000","last_became_peered":"2024-05-08T18:04:49.824405+0000","last_unstale":"2024-05-08T18:06:51.936372+0000","last_undegraded":"2024-05-08T18:06:51.936372+0000","last_fullsized":"2024-05-08T18:06:51.936372+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:05:52.949062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858238+0000","last_change":"2024-05-08T18:04:51.845116+0000","last_active":"2024-05-08T18:05:30.858238+0000","last_peered":"2024-05-08T18:05:30.858238+0000","last_clean":"2024-05-08T18:05:30.858238+0000","last_became_active":"2024-05-08T18:04:51.844757+0000","last_became_peered":"2024-05-08T18:04:51.844757+0000","last_unstale":"2024-05-08T18:05:30.858238+0000","last_undegraded":"2024-05-08T18:05:30.858238+0000","last_fullsized":"2024-05-08T18:05:30.858238+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:17.567274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866299+0000","last_change":"2024-05-08T18:04:53.841236+0000","last_active":"2024-05-08T18:05:30.866299+0000","last_peered":"2024-05-08T18:05:30.866299+0000","last_clean":"2024-05-08T18:05:30.866299+0000","last_became_active":"2024-05-08T18:04:53.840956+0000","last_became_peered":"2024-05-08T18:04:53.840956+0000","last_unstale":"2024-05-08T18:05:30.866299+0000","last_undegraded":"2024-05-08T18:05:30.866299+0000","last_fullsized":"2024-05-08T18:05:30.866299+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:24:58.125427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"29'3","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.866543+0000","last_change":"2024-05-08T18:04:45.807309+0000","last_active":"2024-05-08T18:07:53.866543+0000","last_peered":"2024-05-08T18:07:53.866543+0000","last_clean":"2024-05-08T18:07:53.866543+0000","last_became_active":"2024-05-08T18:04:45.806560+0000","last_became_peered":"2024-05-08T18:04:45.806560+0000","last_unstale":"2024-05-08T18:07:53.866543+0000","last_undegraded":"2024-05-08T18:07:53.866543+0000","last_fullsized":"2024-05-08T18:07:53.866543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:11:17.190072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861114+0000","last_change":"2024-05-08T18:04:43.845626+0000","last_active":"2024-05-08T18:05:30.861114+0000","last_peered":"2024-05-08T18:05:30.861114+0000","last_clean":"2024-05-08T18:05:30.861114+0000","last_became_active":"2024-05-08T18:04:43.845214+0000","last_became_peered":"2024-05-08T18:04:43.845214+0000","last_unstale":"2024-05-08T18:05:30.861114+0000","last_undegraded":"2024-05-08T18:05:30.861114+0000","last_fullsized":"2024-05-08T18:05:30.861114+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:00.592659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.865365+0000","last_change":"2024-05-08T18:04:47.823059+0000","last_active":"2024-05-08T18:05:30.865365+0000","last_peered":"2024-05-08T18:05:30.865365+0000","last_clean":"2024-05-08T18:05:30.865365+0000","last_became_active":"2024-05-08T18:04:47.822748+0000","last_became_peered":"2024-05-08T18:04:47.822748+0000","last_unstale":"2024-05-08T18:05:30.865365+0000","last_undegraded":"2024-05-08T18:05:30.865365+0000","last_fullsized":"2024-05-08T18:05:30.865365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:20.980114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"29'31","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.937310+0000","last_change":"2024-05-08T18:04:49.831735+0000","last_active":"2024-05-08T18:06:51.937310+0000","last_peered":"2024-05-08T18:06:51.937310+0000","last_clean":"2024-05-08T18:06:51.937310+0000","last_became_active":"2024-05-08T18:04:49.831440+0000","last_became_peered":"2024-05-08T18:04:49.831440+0000","last_unstale":"2024-05-08T18:06:51.937310+0000","last_undegraded":"2024-05-08T18:06:51.937310+0000","last_fullsized":"2024-05-08T18:06:51.937310+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:05:40.102877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857159+0000","last_change":"2024-05-08T18:04:51.844443+0000","last_active":"2024-05-08T18:05:30.857159+0000","last_peered":"2024-05-08T18:05:30.857159+0000","last_clean":"2024-05-08T18:05:30.857159+0000","last_became_active":"2024-05-08T18:04:51.844125+0000","last_became_peered":"2024-05-08T18:04:51.844125+0000","last_unstale":"2024-05-08T18:05:30.857159+0000","last_undegraded":"2024-05-08T18:05:30.857159+0000","last_fullsized":"2024-05-08T18:05:30.857159+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:02:22.017553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":17,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862839+0000","last_change":"2024-05-08T18:04:53.849962+0000","last_active":"2024-05-08T18:05:30.862839+0000","last_peered":"2024-05-08T18:05:30.862839+0000","last_clean":"2024-05-08T18:05:30.862839+0000","last_became_active":"2024-05-08T18:04:53.849514+0000","last_became_peered":"2024-05-08T18:04:53.849514+0000","last_unstale":"2024-05-08T18:05:30.862839+0000","last_undegraded":"2024-05-08T18:05:30.862839+0000","last_fullsized":"2024-05-08T18:05:30.862839+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:26:31.875101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"29'4","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399430+0000","last_change":"2024-05-08T18:04:45.822495+0000","last_active":"2024-05-08T18:06:53.399430+0000","last_peered":"2024-05-08T18:06:53.399430+0000","last_clean":"2024-05-08T18:06:53.399430+0000","last_became_active":"2024-05-08T18:04:45.822204+0000","last_became_peered":"2024-05-08T18:04:45.822204+0000","last_unstale":"2024-05-08T18:06:53.399430+0000","last_undegraded":"2024-05-08T18:06:53.399430+0000","last_fullsized":"2024-05-08T18:06:53.399430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:06.207449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.058832+0000","last_change":"2024-05-08T18:04:43.835654+0000","last_active":"2024-05-08T18:07:54.058832+0000","last_peered":"2024-05-08T18:07:54.058832+0000","last_clean":"2024-05-08T18:07:54.058832+0000","last_became_active":"2024-05-08T18:04:43.834678+0000","last_became_peered":"2024-05-08T18:04:43.834678+0000","last_unstale":"2024-05-08T18:07:54.058832+0000","last_undegraded":"2024-05-08T18:07:54.058832+0000","last_fullsized":"2024-05-08T18:07:54.058832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:23:16.295599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859176+0000","last_change":"2024-05-08T18:04:47.820376+0000","last_active":"2024-05-08T18:05:30.859176+0000","last_peered":"2024-05-08T18:05:30.859176+0000","last_clean":"2024-05-08T18:05:30.859176+0000","last_became_active":"2024-05-08T18:04:47.820036+0000","last_became_peered":"2024-05-08T18:04:47.820036+0000","last_unstale":"2024-05-08T18:05:30.859176+0000","last_undegraded":"2024-05-08T18:05:30.859176+0000","last_fullsized":"2024-05-08T18:05:30.859176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:10:33.041988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"29'28","reported_seq":68,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.960416+0000","last_change":"2024-05-08T18:04:49.832032+0000","last_active":"2024-05-08T18:06:51.960416+0000","last_peered":"2024-05-08T18:06:51.960416+0000","last_clean":"2024-05-08T18:06:51.960416+0000","last_became_active":"2024-05-08T18:04:49.830882+0000","last_became_peered":"2024-05-08T18:04:49.830882+0000","last_unstale":"2024-05-08T18:06:51.960416+0000","last_undegraded":"2024-05-08T18:06:51.960416+0000","last_fullsized":"2024-05-08T18:06:51.960416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:16:21.475550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":40,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857634+0000","last_change":"2024-05-08T18:04:51.844109+0000","last_active":"2024-05-08T18:05:30.857634+0000","last_peered":"2024-05-08T18:05:30.857634+0000","last_clean":"2024-05-08T18:05:30.857634+0000","last_became_active":"2024-05-08T18:04:51.842734+0000","last_became_peered":"2024-05-08T18:04:51.842734+0000","last_unstale":"2024-05-08T18:05:30.857634+0000","last_undegraded":"2024-05-08T18:05:30.857634+0000","last_fullsized":"2024-05-08T18:05:30.857634+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:32:47.953187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'10","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.495078+0000","last_change":"2024-05-08T18:04:53.845139+0000","last_active":"2024-05-08T18:07:53.495078+0000","last_peered":"2024-05-08T18:07:53.495078+0000","last_clean":"2024-05-08T18:07:53.495078+0000","last_became_active":"2024-05-08T18:04:53.844891+0000","last_became_peered":"2024-05-08T18:04:53.844891+0000","last_unstale":"2024-05-08T18:07:53.495078+0000","last_undegraded":"2024-05-08T18:07:53.495078+0000","last_fullsized":"2024-05-08T18:07:53.495078+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:37:32.203551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"29'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.860618+0000","last_change":"2024-05-08T18:04:45.809216+0000","last_active":"2024-05-08T18:07:53.860618+0000","last_peered":"2024-05-08T18:07:53.860618+0000","last_clean":"2024-05-08T18:07:53.860618+0000","last_became_active":"2024-05-08T18:04:45.808936+0000","last_became_peered":"2024-05-08T18:04:45.808936+0000","last_unstale":"2024-05-08T18:07:53.860618+0000","last_undegraded":"2024-05-08T18:07:53.860618+0000","last_fullsized":"2024-05-08T18:07:53.860618+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:05:51.368967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866160+0000","last_change":"2024-05-08T18:04:43.837820+0000","last_active":"2024-05-08T18:05:30.866160+0000","last_peered":"2024-05-08T18:05:30.866160+0000","last_clean":"2024-05-08T18:05:30.866160+0000","last_became_active":"2024-05-08T18:04:43.837070+0000","last_became_peered":"2024-05-08T18:04:43.837070+0000","last_unstale":"2024-05-08T18:05:30.866160+0000","last_undegraded":"2024-05-08T18:05:30.866160+0000","last_fullsized":"2024-05-08T18:05:30.866160+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:42:50.043665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"29'2","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.565208+0000","last_change":"2024-05-08T18:04:47.824489+0000","last_active":"2024-05-08T18:06:50.565208+0000","last_peered":"2024-05-08T18:06:50.565208+0000","last_clean":"2024-05-08T18:06:50.565208+0000","last_became_active":"2024-05-08T18:04:47.824172+0000","last_became_peered":"2024-05-08T18:04:47.824172+0000","last_unstale":"2024-05-08T18:06:50.565208+0000","last_undegraded":"2024-05-08T18:06:50.565208+0000","last_fullsized":"2024-05-08T18:06:50.565208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:34:14.408614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"29'32","reported_seq":76,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.418071+0000","last_change":"2024-05-08T18:04:49.832422+0000","last_active":"2024-05-08T18:07:53.418071+0000","last_peered":"2024-05-08T18:07:53.418071+0000","last_clean":"2024-05-08T18:07:53.418071+0000","last_became_active":"2024-05-08T18:04:49.831803+0000","last_became_peered":"2024-05-08T18:04:49.831803+0000","last_unstale":"2024-05-08T18:07:53.418071+0000","last_undegraded":"2024-05-08T18:07:53.418071+0000","last_fullsized":"2024-05-08T18:07:53.418071+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:03:53.191234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858576+0000","last_change":"2024-05-08T18:04:51.846422+0000","last_active":"2024-05-08T18:05:30.858576+0000","last_peered":"2024-05-08T18:05:30.858576+0000","last_clean":"2024-05-08T18:05:30.858576+0000","last_became_active":"2024-05-08T18:04:51.846225+0000","last_became_peered":"2024-05-08T18:04:51.846225+0000","last_unstale":"2024-05-08T18:05:30.858576+0000","last_undegraded":"2024-05-08T18:05:30.858576+0000","last_fullsized":"2024-05-08T18:05:30.858576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:42:25.975522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"29'4","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:51.972855+0000","last_change":"2024-05-08T18:04:53.843030+0000","last_active":"2024-05-08T18:06:51.972855+0000","last_peered":"2024-05-08T18:06:51.972855+0000","last_clean":"2024-05-08T18:06:51.972855+0000","last_became_active":"2024-05-08T18:04:53.842429+0000","last_became_peered":"2024-05-08T18:04:53.842429+0000","last_unstale":"2024-05-08T18:06:51.972855+0000","last_undegraded":"2024-05-08T18:06:51.972855+0000","last_fullsized":"2024-05-08T18:06:51.972855+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:19:30.303540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"27'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.866744+0000","last_change":"2024-05-08T18:04:45.818828+0000","last_active":"2024-05-08T18:05:30.866744+0000","last_peered":"2024-05-08T18:05:30.866744+0000","last_clean":"2024-05-08T18:05:30.866744+0000","last_became_active":"2024-05-08T18:04:45.818161+0000","last_became_peered":"2024-05-08T18:04:45.818161+0000","last_unstale":"2024-05-08T18:05:30.866744+0000","last_undegraded":"2024-05-08T18:05:30.866744+0000","last_fullsized":"2024-05-08T18:05:30.866744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:28:59.477939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859517+0000","last_change":"2024-05-08T18:04:43.843055+0000","last_active":"2024-05-08T18:05:30.859517+0000","last_peered":"2024-05-08T18:05:30.859517+0000","last_clean":"2024-05-08T18:05:30.859517+0000","last_became_active":"2024-05-08T18:04:43.842483+0000","last_became_peered":"2024-05-08T18:04:43.842483+0000","last_unstale":"2024-05-08T18:05:30.859517+0000","last_undegraded":"2024-05-08T18:05:30.859517+0000","last_fullsized":"2024-05-08T18:05:30.859517+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:11.225102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"29'36","reported_seq":106,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995781+0000","last_change":"2024-05-08T18:04:45.808186+0000","last_active":"2024-05-08T18:07:53.995781+0000","last_peered":"2024-05-08T18:07:53.995781+0000","last_clean":"2024-05-08T18:07:53.995781+0000","last_became_active":"2024-05-08T18:04:45.807847+0000","last_became_peered":"2024-05-08T18:04:45.807847+0000","last_unstale":"2024-05-08T18:07:53.995781+0000","last_undegraded":"2024-05-08T18:07:53.995781+0000","last_fullsized":"2024-05-08T18:07:53.995781+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:00:26.846485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":44,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858145+0000","last_change":"2024-05-08T18:04:43.836530+0000","last_active":"2024-05-08T18:05:30.858145+0000","last_peered":"2024-05-08T18:05:30.858145+0000","last_clean":"2024-05-08T18:05:30.858145+0000","last_became_active":"2024-05-08T18:04:43.836045+0000","last_became_peered":"2024-05-08T18:04:43.836045+0000","last_unstale":"2024-05-08T18:05:30.858145+0000","last_undegraded":"2024-05-08T18:05:30.858145+0000","last_fullsized":"2024-05-08T18:05:30.858145+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:31:22.454546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3,2],"acting":[0,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"29'11","reported_seq":68,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.782514+0000","last_change":"2024-05-08T18:04:45.826393+0000","last_active":"2024-05-08T18:07:53.782514+0000","last_peered":"2024-05-08T18:07:53.782514+0000","last_clean":"2024-05-08T18:07:53.782514+0000","last_became_active":"2024-05-08T18:04:45.825506+0000","last_became_peered":"2024-05-08T18:04:45.825506+0000","last_unstale":"2024-05-08T18:07:53.782514+0000","last_undegraded":"2024-05-08T18:07:53.782514+0000","last_fullsized":"2024-05-08T18:07:53.782514+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:23:18.600988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":46,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"29'5","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.990901+0000","last_change":"2024-05-08T18:04:43.832854+0000","last_active":"2024-05-08T18:07:53.990901+0000","last_peered":"2024-05-08T18:07:53.990901+0000","last_clean":"2024-05-08T18:07:53.990901+0000","last_became_active":"2024-05-08T18:04:43.832406+0000","last_became_peered":"2024-05-08T18:04:43.832406+0000","last_unstale":"2024-05-08T18:07:53.990901+0000","last_undegraded":"2024-05-08T18:07:53.990901+0000","last_fullsized":"2024-05-08T18:07:53.990901+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:16.456364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":29,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"29'2","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.774175+0000","last_change":"2024-05-08T18:04:45.807187+0000","last_active":"2024-05-08T18:07:53.774175+0000","last_peered":"2024-05-08T18:07:53.774175+0000","last_clean":"2024-05-08T18:07:53.774175+0000","last_became_active":"2024-05-08T18:04:45.806646+0000","last_became_peered":"2024-05-08T18:04:45.806646+0000","last_unstale":"2024-05-08T18:07:53.774175+0000","last_undegraded":"2024-05-08T18:07:53.774175+0000","last_fullsized":"2024-05-08T18:07:53.774175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:30.458103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"29'5","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.374758+0000","last_change":"2024-05-08T18:04:43.836158+0000","last_active":"2024-05-08T18:06:53.374758+0000","last_peered":"2024-05-08T18:06:53.374758+0000","last_clean":"2024-05-08T18:06:53.374758+0000","last_became_active":"2024-05-08T18:04:43.835163+0000","last_became_peered":"2024-05-08T18:04:43.835163+0000","last_unstale":"2024-05-08T18:06:53.374758+0000","last_undegraded":"2024-05-08T18:06:53.374758+0000","last_fullsized":"2024-05-08T18:06:53.374758+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:55.493870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":13,"num_write_kb":4746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,1],"acting":[0,3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"29'2","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.861341+0000","last_change":"2024-05-08T18:04:45.816663+0000","last_active":"2024-05-08T18:07:53.861341+0000","last_peered":"2024-05-08T18:07:53.861341+0000","last_clean":"2024-05-08T18:07:53.861341+0000","last_became_active":"2024-05-08T18:04:45.816391+0000","last_became_peered":"2024-05-08T18:04:45.816391+0000","last_unstale":"2024-05-08T18:07:53.861341+0000","last_undegraded":"2024-05-08T18:07:53.861341+0000","last_fullsized":"2024-05-08T18:07:53.861341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:02.599918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'4","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861292+0000","last_change":"2024-05-08T18:04:43.836083+0000","last_active":"2024-05-08T18:05:30.861292+0000","last_peered":"2024-05-08T18:05:30.861292+0000","last_clean":"2024-05-08T18:05:30.861292+0000","last_became_active":"2024-05-08T18:04:43.835268+0000","last_became_peered":"2024-05-08T18:04:43.835268+0000","last_unstale":"2024-05-08T18:05:30.861292+0000","last_undegraded":"2024-05-08T18:05:30.861292+0000","last_fullsized":"2024-05-08T18:05:30.861292+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:47.971014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"29'12","reported_seq":77,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.035458+0000","last_change":"2024-05-08T18:04:45.820787+0000","last_active":"2024-05-08T18:07:54.035458+0000","last_peered":"2024-05-08T18:07:54.035458+0000","last_clean":"2024-05-08T18:07:54.035458+0000","last_became_active":"2024-05-08T18:04:45.819699+0000","last_became_peered":"2024-05-08T18:04:45.819699+0000","last_unstale":"2024-05-08T18:07:54.035458+0000","last_undegraded":"2024-05-08T18:07:54.035458+0000","last_fullsized":"2024-05-08T18:07:54.035458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:18.539896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.862510+0000","last_change":"2024-05-08T18:04:43.837155+0000","last_active":"2024-05-08T18:05:30.862510+0000","last_peered":"2024-05-08T18:05:30.862510+0000","last_clean":"2024-05-08T18:05:30.862510+0000","last_became_active":"2024-05-08T18:04:43.836735+0000","last_became_peered":"2024-05-08T18:04:43.836735+0000","last_unstale":"2024-05-08T18:05:30.862510+0000","last_undegraded":"2024-05-08T18:05:30.862510+0000","last_fullsized":"2024-05-08T18:05:30.862510+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:54.711833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"29'14","reported_seq":70,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.219716+0000","last_change":"2024-05-08T18:04:45.812327+0000","last_active":"2024-05-08T18:06:50.219716+0000","last_peered":"2024-05-08T18:06:50.219716+0000","last_clean":"2024-05-08T18:06:50.219716+0000","last_became_active":"2024-05-08T18:04:45.811953+0000","last_became_peered":"2024-05-08T18:04:45.811953+0000","last_unstale":"2024-05-08T18:06:50.219716+0000","last_undegraded":"2024-05-08T18:06:50.219716+0000","last_fullsized":"2024-05-08T18:06:50.219716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:11:17.190072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867588+0000","last_change":"2024-05-08T18:04:43.843298+0000","last_active":"2024-05-08T18:05:30.867588+0000","last_peered":"2024-05-08T18:05:30.867588+0000","last_clean":"2024-05-08T18:05:30.867588+0000","last_became_active":"2024-05-08T18:04:43.842179+0000","last_became_peered":"2024-05-08T18:04:43.842179+0000","last_unstale":"2024-05-08T18:05:30.867588+0000","last_undegraded":"2024-05-08T18:05:30.867588+0000","last_fullsized":"2024-05-08T18:05:30.867588+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:12:55.493870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,0],"acting":[2,3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"29'8","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.063691+0000","last_change":"2024-05-08T18:04:45.812893+0000","last_active":"2024-05-08T18:07:54.063691+0000","last_peered":"2024-05-08T18:07:54.063691+0000","last_clean":"2024-05-08T18:07:54.063691+0000","last_became_active":"2024-05-08T18:04:45.812637+0000","last_became_peered":"2024-05-08T18:04:45.812637+0000","last_unstale":"2024-05-08T18:07:54.063691+0000","last_undegraded":"2024-05-08T18:07:54.063691+0000","last_fullsized":"2024-05-08T18:07:54.063691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:55:18.973014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"29'6","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.981626+0000","last_change":"2024-05-08T18:04:43.834951+0000","last_active":"2024-05-08T18:07:53.981626+0000","last_peered":"2024-05-08T18:07:53.981626+0000","last_clean":"2024-05-08T18:07:53.981626+0000","last_became_active":"2024-05-08T18:04:43.834491+0000","last_became_peered":"2024-05-08T18:04:43.834491+0000","last_unstale":"2024-05-08T18:07:53.981626+0000","last_undegraded":"2024-05-08T18:07:53.981626+0000","last_fullsized":"2024-05-08T18:07:53.981626+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:24:46.108691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"29'8","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.041264+0000","last_change":"2024-05-08T18:04:45.815948+0000","last_active":"2024-05-08T18:07:54.041264+0000","last_peered":"2024-05-08T18:07:54.041264+0000","last_clean":"2024-05-08T18:07:54.041264+0000","last_became_active":"2024-05-08T18:04:45.815677+0000","last_became_peered":"2024-05-08T18:04:45.815677+0000","last_unstale":"2024-05-08T18:07:54.041264+0000","last_undegraded":"2024-05-08T18:07:54.041264+0000","last_fullsized":"2024-05-08T18:07:54.041264+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:35:16.870297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857605+0000","last_change":"2024-05-08T18:04:43.846054+0000","last_active":"2024-05-08T18:05:30.857605+0000","last_peered":"2024-05-08T18:05:30.857605+0000","last_clean":"2024-05-08T18:05:30.857605+0000","last_became_active":"2024-05-08T18:04:43.843826+0000","last_became_peered":"2024-05-08T18:04:43.843826+0000","last_unstale":"2024-05-08T18:05:30.857605+0000","last_undegraded":"2024-05-08T18:05:30.857605+0000","last_fullsized":"2024-05-08T18:05:30.857605+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:08:10.712708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.399417+0000","last_change":"2024-05-08T18:04:45.827535+0000","last_active":"2024-05-08T18:06:53.399417+0000","last_peered":"2024-05-08T18:06:53.399417+0000","last_clean":"2024-05-08T18:06:53.399417+0000","last_became_active":"2024-05-08T18:04:45.827252+0000","last_became_peered":"2024-05-08T18:04:45.827252+0000","last_unstale":"2024-05-08T18:06:53.399417+0000","last_undegraded":"2024-05-08T18:06:53.399417+0000","last_fullsized":"2024-05-08T18:06:53.399417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:07.190532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.858630+0000","last_change":"2024-05-08T18:04:43.820995+0000","last_active":"2024-05-08T18:05:30.858630+0000","last_peered":"2024-05-08T18:05:30.858630+0000","last_clean":"2024-05-08T18:05:30.858630+0000","last_became_active":"2024-05-08T18:04:43.820360+0000","last_became_peered":"2024-05-08T18:04:43.820360+0000","last_unstale":"2024-05-08T18:05:30.858630+0000","last_undegraded":"2024-05-08T18:05:30.858630+0000","last_fullsized":"2024-05-08T18:05:30.858630+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:54.711833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.860974+0000","last_change":"2024-05-08T18:04:45.823656+0000","last_active":"2024-05-08T18:05:30.860974+0000","last_peered":"2024-05-08T18:05:30.860974+0000","last_clean":"2024-05-08T18:05:30.860974+0000","last_became_active":"2024-05-08T18:04:45.823385+0000","last_became_peered":"2024-05-08T18:04:45.823385+0000","last_unstale":"2024-05-08T18:05:30.860974+0000","last_undegraded":"2024-05-08T18:05:30.860974+0000","last_fullsized":"2024-05-08T18:05:30.860974+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:26.156270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'6","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.755149+0000","last_change":"2024-05-08T18:04:43.836532+0000","last_active":"2024-05-08T18:07:53.755149+0000","last_peered":"2024-05-08T18:07:53.755149+0000","last_clean":"2024-05-08T18:07:53.755149+0000","last_became_active":"2024-05-08T18:04:43.835838+0000","last_became_peered":"2024-05-08T18:04:43.835838+0000","last_unstale":"2024-05-08T18:07:53.755149+0000","last_undegraded":"2024-05-08T18:07:53.755149+0000","last_fullsized":"2024-05-08T18:07:53.755149+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:07:11.225102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8,"num_write":30,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"29'10","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.995916+0000","last_change":"2024-05-08T18:04:45.810992+0000","last_active":"2024-05-08T18:07:53.995916+0000","last_peered":"2024-05-08T18:07:53.995916+0000","last_clean":"2024-05-08T18:07:53.995916+0000","last_became_active":"2024-05-08T18:04:45.810703+0000","last_became_peered":"2024-05-08T18:04:45.810703+0000","last_unstale":"2024-05-08T18:07:53.995916+0000","last_undegraded":"2024-05-08T18:07:53.995916+0000","last_fullsized":"2024-05-08T18:07:53.995916+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:05.828431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"29'3","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744531+0000","last_change":"2024-05-08T18:04:43.836123+0000","last_active":"2024-05-08T18:06:58.744531+0000","last_peered":"2024-05-08T18:06:58.744531+0000","last_clean":"2024-05-08T18:06:58.744531+0000","last_became_active":"2024-05-08T18:04:43.835518+0000","last_became_peered":"2024-05-08T18:04:43.835518+0000","last_unstale":"2024-05-08T18:06:58.744531+0000","last_undegraded":"2024-05-08T18:06:58.744531+0000","last_fullsized":"2024-05-08T18:06:58.744531+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:06.338736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,3],"acting":[1,2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"29'4","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.219429+0000","last_change":"2024-05-08T18:04:45.821376+0000","last_active":"2024-05-08T18:06:50.219429+0000","last_peered":"2024-05-08T18:06:50.219429+0000","last_clean":"2024-05-08T18:06:50.219429+0000","last_became_active":"2024-05-08T18:04:45.820746+0000","last_became_peered":"2024-05-08T18:04:45.820746+0000","last_unstale":"2024-05-08T18:06:50.219429+0000","last_undegraded":"2024-05-08T18:06:50.219429+0000","last_fullsized":"2024-05-08T18:06:50.219429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:47:32.833248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.859364+0000","last_change":"2024-05-08T18:04:43.847998+0000","last_active":"2024-05-08T18:05:30.859364+0000","last_peered":"2024-05-08T18:05:30.859364+0000","last_clean":"2024-05-08T18:05:30.859364+0000","last_became_active":"2024-05-08T18:04:43.844404+0000","last_became_peered":"2024-05-08T18:04:43.844404+0000","last_unstale":"2024-05-08T18:05:30.859364+0000","last_undegraded":"2024-05-08T18:05:30.859364+0000","last_fullsized":"2024-05-08T18:05:30.859364+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:28.450844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"29'4","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.400051+0000","last_change":"2024-05-08T18:04:45.809582+0000","last_active":"2024-05-08T18:06:53.400051+0000","last_peered":"2024-05-08T18:06:53.400051+0000","last_clean":"2024-05-08T18:06:53.400051+0000","last_became_active":"2024-05-08T18:04:45.809271+0000","last_became_peered":"2024-05-08T18:04:45.809271+0000","last_unstale":"2024-05-08T18:06:53.400051+0000","last_undegraded":"2024-05-08T18:06:53.400051+0000","last_fullsized":"2024-05-08T18:06:53.400051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:32.141179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.861152+0000","last_change":"2024-05-08T18:04:43.844366+0000","last_active":"2024-05-08T18:05:30.861152+0000","last_peered":"2024-05-08T18:05:30.861152+0000","last_clean":"2024-05-08T18:05:30.861152+0000","last_became_active":"2024-05-08T18:04:43.838125+0000","last_became_peered":"2024-05-08T18:04:43.838125+0000","last_unstale":"2024-05-08T18:05:30.861152+0000","last_undegraded":"2024-05-08T18:05:30.861152+0000","last_fullsized":"2024-05-08T18:05:30.861152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:28:24.149010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"29'15","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070567+0000","last_change":"2024-05-08T18:04:45.828761+0000","last_active":"2024-05-08T18:07:54.070567+0000","last_peered":"2024-05-08T18:07:54.070567+0000","last_clean":"2024-05-08T18:07:54.070567+0000","last_became_active":"2024-05-08T18:04:45.828488+0000","last_became_peered":"2024-05-08T18:04:45.828488+0000","last_unstale":"2024-05-08T18:07:54.070567+0000","last_undegraded":"2024-05-08T18:07:54.070567+0000","last_fullsized":"2024-05-08T18:07:54.070567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:27:23.439814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.867753+0000","last_change":"2024-05-08T18:04:43.843455+0000","last_active":"2024-05-08T18:05:30.867753+0000","last_peered":"2024-05-08T18:05:30.867753+0000","last_clean":"2024-05-08T18:05:30.867753+0000","last_became_active":"2024-05-08T18:04:43.842670+0000","last_became_peered":"2024-05-08T18:04:43.842670+0000","last_unstale":"2024-05-08T18:05:30.867753+0000","last_undegraded":"2024-05-08T18:05:30.867753+0000","last_fullsized":"2024-05-08T18:05:30.867753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:01:49.620924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"29'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:09.547055+0000","last_change":"2024-05-08T18:04:45.812074+0000","last_active":"2024-05-08T18:06:09.547055+0000","last_peered":"2024-05-08T18:06:09.547055+0000","last_clean":"2024-05-08T18:06:09.547055+0000","last_became_active":"2024-05-08T18:04:45.810562+0000","last_became_peered":"2024-05-08T18:04:45.810562+0000","last_unstale":"2024-05-08T18:06:09.547055+0000","last_undegraded":"2024-05-08T18:06:09.547055+0000","last_fullsized":"2024-05-08T18:06:09.547055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:42:58.784190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.864727+0000","last_change":"2024-05-08T18:04:43.838746+0000","last_active":"2024-05-08T18:05:30.864727+0000","last_peered":"2024-05-08T18:05:30.864727+0000","last_clean":"2024-05-08T18:05:30.864727+0000","last_became_active":"2024-05-08T18:04:43.835943+0000","last_became_peered":"2024-05-08T18:04:43.835943+0000","last_unstale":"2024-05-08T18:05:30.864727+0000","last_undegraded":"2024-05-08T18:05:30.864727+0000","last_fullsized":"2024-05-08T18:05:30.864727+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:27:18.106771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:05:30.857870+0000","last_change":"2024-05-08T18:04:43.836378+0000","last_active":"2024-05-08T18:05:30.857870+0000","last_peered":"2024-05-08T18:05:30.857870+0000","last_clean":"2024-05-08T18:05:30.857870+0000","last_became_active":"2024-05-08T18:04:43.836007+0000","last_became_peered":"2024-05-08T18:04:43.836007+0000","last_unstale":"2024-05-08T18:05:30.857870+0000","last_undegraded":"2024-05-08T18:05:30.857870+0000","last_fullsized":"2024-05-08T18:05:30.857870+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:57:00.868492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"29'8","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.219707+0000","last_change":"2024-05-08T18:04:45.812333+0000","last_active":"2024-05-08T18:06:50.219707+0000","last_peered":"2024-05-08T18:06:50.219707+0000","last_clean":"2024-05-08T18:06:50.219707+0000","last_became_active":"2024-05-08T18:04:45.811648+0000","last_became_peered":"2024-05-08T18:04:45.811648+0000","last_unstale":"2024-05-08T18:06:50.219707+0000","last_undegraded":"2024-05-08T18:06:50.219707+0000","last_fullsized":"2024-05-08T18:06:50.219707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:03:18.539896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":31,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"29'5","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.740580+0000","last_change":"2024-05-08T18:04:43.836500+0000","last_active":"2024-05-08T18:07:03.740580+0000","last_peered":"2024-05-08T18:07:03.740580+0000","last_clean":"2024-05-08T18:07:03.740580+0000","last_became_active":"2024-05-08T18:04:43.835712+0000","last_became_peered":"2024-05-08T18:04:43.835712+0000","last_unstale":"2024-05-08T18:07:03.740580+0000","last_undegraded":"2024-05-08T18:07:03.740580+0000","last_fullsized":"2024-05-08T18:07:03.740580+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:06.338736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4190208,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":12,"num_read_kb":8,"num_write":21,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"29'10","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.775008+0000","last_change":"2024-05-08T18:04:45.812258+0000","last_active":"2024-05-08T18:07:53.775008+0000","last_peered":"2024-05-08T18:07:53.775008+0000","last_clean":"2024-05-08T18:07:53.775008+0000","last_became_active":"2024-05-08T18:04:45.811921+0000","last_became_peered":"2024-05-08T18:04:45.811921+0000","last_unstale":"2024-05-08T18:07:53.775008+0000","last_undegraded":"2024-05-08T18:07:53.775008+0000","last_fullsized":"2024-05-08T18:07:53.775008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:00:26.846485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2256,"ondisk_log_size":2256,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":13385,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1402,"num_read_kb":1188,"num_write":531,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":10826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1275,"ondisk_log_size":1275,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":961,"num_read_kb":961,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":25809792,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":138,"num_read_kb":87,"num_write":309,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":34422784,"data_stored":34422784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":256,"acting":256,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705707,"num_pgs":179,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48332,"kb_used_data":8836,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323508,"statfs":{"total":96636764160,"available":96587272192,"internally_reserved":0,"allocated":9048064,"data_stored":8743050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705708,"num_pgs":153,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48848,"kb_used_data":9352,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322992,"statfs":{"total":96636764160,"available":96586743808,"internally_reserved":0,"allocated":9576448,"data_stored":9329849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705708,"num_pgs":157,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48872,"kb_used_data":9376,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94322968,"statfs":{"total":96636764160,"available":96586719232,"internally_reserved":0,"allocated":9601024,"data_stored":9332048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39980,"kb_used_data":8804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331860,"statfs":{"total":96636764160,"available":96595824640,"internally_reserved":0,"allocated":9015296,"data_stored":8741305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":8605696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":8605696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":8605696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8605696,"data_stored":8605696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":4269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T18:08:05.430 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.430 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.430 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.430 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.430 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.430 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.430 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.430 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.430 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.431 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.431 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.431 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.431 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.431 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.431 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.431 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.431 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.431 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.431 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.432 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.432 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.432 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.432 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.432 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.432 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.432 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.432 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.432 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.432 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.433 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.433 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.433 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.433 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.433 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.433 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.433 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.433 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.433 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.433 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.434 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.434 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.434 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.434 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.434 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.434 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.434 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.434 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.434 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.435 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.435 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.435 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.435 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.435 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.435 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.435 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.435 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.435 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.435 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.436 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.436 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.436 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.436 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.436 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.436 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.436 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.436 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.436 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.436 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.437 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.437 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.437 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.437 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.437 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.437 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.437 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.437 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.437 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.437 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.438 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.438 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.438 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.438 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.438 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.438 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.438 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.438 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.438 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.439 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.439 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.439 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.439 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.439 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.439 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.439 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.439 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.439 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.439 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.440 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.440 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.440 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.440 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.440 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.440 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.440 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.440 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.440 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.440 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.441 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.441 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.441 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.441 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.441 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.441 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.441 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.441 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.441 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.441 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.442 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.442 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.442 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.442 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.442 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.442 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.442 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.442 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.442 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.443 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.443 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.443 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.443 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.443 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.443 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.443 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.443 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.443 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.443 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.444 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.444 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.444 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.444 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.444 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.444 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.444 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.444 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.444 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.444 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.445 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.445 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.445 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.445 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.445 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.445 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.445 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.445 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.445 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.446 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.446 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.446 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.446 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.446 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.446 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.446 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.446 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.446 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.447 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.447 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.447 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.447 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.447 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.447 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.447 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.447 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.447 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.448 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.448 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.448 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.448 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.448 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.448 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.448 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.448 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.448 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.449 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.449 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.449 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.449 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.449 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.449 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.449 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.449 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.449 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.449 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.450 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.450 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.450 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.450 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.450 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.450 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.450 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.450 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.450 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.450 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.451 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.451 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.451 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.451 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.451 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.451 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.451 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.451 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.451 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.451 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.452 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.452 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.452 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.452 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.452 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.452 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.452 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.452 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.452 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.452 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.453 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.453 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.453 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.453 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.453 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.453 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.453 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.453 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.453 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.454 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.454 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.454 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.454 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.454 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.454 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.454 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.454 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.454 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.454 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.455 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.455 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.455 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.455 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.455 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.455 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.455 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.455 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.455 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.455 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.456 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.456 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.456 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.456 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.456 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.456 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.456 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.456 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.456 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.457 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.457 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.457 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.457 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.457 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:05.457 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T18:08:25.458 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T18:08:25.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.575+0000 7f199c266640 1 Processor -- start 2024-05-08T18:08:25.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.575+0000 7f199c266640 1 -- start start 2024-05-08T18:08:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.576+0000 7f199c266640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1994070650 0x7f1994070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.576+0000 7f199c266640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1994073480 con 0x7f1994072540 2024-05-08T18:08:25.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.576+0000 7f199c266640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f1994071a80 con 0x7f1994070650 2024-05-08T18:08:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.576+0000 7f199a002640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1994070650 0x7f1994070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.576+0000 7f1999801640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f1994072540 0x7f199412f580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:50290/0 (socket says 172.21.15.46:50290) 2024-05-08T18:08:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.576+0000 7f1999801640 1 -- 172.21.15.46:0/3478322373 learned_addr learned my addr 172.21.15.46:0/3478322373 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:08:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.577+0000 7f199a002640 1 -- 172.21.15.46:0/3478322373 >> v1:172.21.15.46:6789/0 conn(0x7f1994072540 legacy=0x7f199412f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:25.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.577+0000 7f199a002640 1 -- 172.21.15.46:0/3478322373 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1994072c20 con 0x7f1994070650 2024-05-08T18:08:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.578+0000 7f199a002640 1 --2- 172.21.15.46:0/3478322373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994070650 0x7f1994070a30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1990009720 tx=0x7f199002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5baa3bb89bc6531a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.578+0000 7f1999000640 1 -- 172.21.15.46:0/3478322373 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19900316d0 con 0x7f1994070650 2024-05-08T18:08:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.578+0000 7f1999000640 1 -- 172.21.15.46:0/3478322373 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1990037070 con 0x7f1994070650 2024-05-08T18:08:25.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.578+0000 7f1999000640 1 -- 172.21.15.46:0/3478322373 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19900319c0 con 0x7f1994070650 2024-05-08T18:08:25.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.578+0000 7f199c266640 1 -- 172.21.15.46:0/3478322373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994070650 msgr2=0x7f1994070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:25.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.578+0000 7f199c266640 1 --2- 172.21.15.46:0/3478322373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994070650 0x7f1994070a30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1990009720 tx=0x7f199002b840 comp rx=0 tx=0).stop 2024-05-08T18:08:25.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.578+0000 7f199c266640 1 -- 172.21.15.46:0/3478322373 shutdown_connections 2024-05-08T18:08:25.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.578+0000 7f199c266640 1 --2- 172.21.15.46:0/3478322373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994070650 0x7f1994070a30 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:25.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.579+0000 7f199c266640 1 -- 172.21.15.46:0/3478322373 >> 172.21.15.46:0/3478322373 conn(0x7f19940a1e60 msgr2=0x7f19940a2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:25.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.579+0000 7f199c266640 1 -- 172.21.15.46:0/3478322373 shutdown_connections 2024-05-08T18:08:25.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.579+0000 7f199c266640 1 -- 172.21.15.46:0/3478322373 wait complete. 2024-05-08T18:08:25.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.579+0000 7f199c266640 1 Processor -- start 2024-05-08T18:08:25.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.580+0000 7f199c266640 1 -- start start 2024-05-08T18:08:25.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.580+0000 7f199c266640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994070650 0x7f19941c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:25.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.580+0000 7f199c266640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f199412fc90 con 0x7f1994070650 2024-05-08T18:08:25.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.580+0000 7f199a002640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994070650 0x7f19941c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:25.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.580+0000 7f199a002640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994070650 0x7f19941c4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60724/0 (socket says 172.21.15.46:60724) 2024-05-08T18:08:25.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.580+0000 7f199a002640 1 -- 172.21.15.46:0/128033596 learned_addr learned my addr 172.21.15.46:0/128033596 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:08:25.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.580+0000 7f199a002640 1 -- 172.21.15.46:0/128033596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199412c5d0 con 0x7f1994070650 2024-05-08T18:08:25.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.580+0000 7f199a002640 1 --2- 172.21.15.46:0/128033596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994070650 0x7f19941c4450 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1990034040 tx=0x7f19900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:25.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.581+0000 7f198a7fc640 1 -- 172.21.15.46:0/128033596 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1990032770 con 0x7f1994070650 2024-05-08T18:08:25.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.581+0000 7f198a7fc640 1 -- 172.21.15.46:0/128033596 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1990032910 con 0x7f1994070650 2024-05-08T18:08:25.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.581+0000 7f199c266640 1 -- 172.21.15.46:0/128033596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199411e9b0 con 0x7f1994070650 2024-05-08T18:08:25.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.581+0000 7f198a7fc640 1 -- 172.21.15.46:0/128033596 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1990032c30 con 0x7f1994070650 2024-05-08T18:08:25.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.581+0000 7f199c266640 1 -- 172.21.15.46:0/128033596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199411f640 con 0x7f1994070650 2024-05-08T18:08:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.582+0000 7f198a7fc640 1 -- 172.21.15.46:0/128033596 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f199004c060 con 0x7f1994070650 2024-05-08T18:08:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.582+0000 7f199c266640 1 -- 172.21.15.46:0/128033596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1994120150 con 0x7f1994070650 2024-05-08T18:08:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.582+0000 7f198a7fc640 1 --2- 172.21.15.46:0/128033596 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f197803d2b0 0x7f197803f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.582+0000 7f198a7fc640 1 -- 172.21.15.46:0/128033596 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f199002f050 con 0x7f1994070650 2024-05-08T18:08:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.582+0000 7f1999801640 1 --2- 172.21.15.46:0/128033596 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f197803d2b0 0x7f197803f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:25.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.583+0000 7f1999801640 1 --2- 172.21.15.46:0/128033596 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f197803d2b0 0x7f197803f770 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1984007fd0 tx=0x7f1984007aa0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:25.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.587+0000 7f198a7fc640 1 -- 172.21.15.46:0/128033596 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f199412c5d0 con 0x7f1994070650 2024-05-08T18:08:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.773+0000 7f199c266640 1 -- 172.21.15.46:0/128033596 --> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f19940640a0 con 0x7f197803d2b0 2024-05-08T18:08:25.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.789+0000 7f198a7fc640 1 -- 172.21.15.46:0/128033596 <== mgr.4098 v2:172.21.15.50:6824/2602185226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+658988 (secure 0 0 0) 0x7f19940640a0 con 0x7f197803d2b0 2024-05-08T18:08:25.796 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:08:25.801 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T18:08:25.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.794+0000 7f199c266640 1 -- 172.21.15.46:0/128033596 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f197803d2b0 msgr2=0x7f197803f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:25.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.794+0000 7f199c266640 1 --2- 172.21.15.46:0/128033596 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f197803d2b0 0x7f197803f770 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1984007fd0 tx=0x7f1984007aa0 comp rx=0 tx=0).stop 2024-05-08T18:08:25.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.794+0000 7f199c266640 1 -- 172.21.15.46:0/128033596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994070650 msgr2=0x7f19941c4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:25.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.794+0000 7f199c266640 1 --2- 172.21.15.46:0/128033596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994070650 0x7f19941c4450 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1990034040 tx=0x7f19900046c0 comp rx=0 tx=0).stop 2024-05-08T18:08:25.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.794+0000 7f199c266640 1 -- 172.21.15.46:0/128033596 shutdown_connections 2024-05-08T18:08:25.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.794+0000 7f199c266640 1 --2- 172.21.15.46:0/128033596 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f197803d2b0 0x7f197803f770 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:25.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.794+0000 7f199c266640 1 --2- 172.21.15.46:0/128033596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1994070650 0x7f19941c4450 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:25.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.794+0000 7f199c266640 1 -- 172.21.15.46:0/128033596 >> 172.21.15.46:0/128033596 conn(0x7f19940a1e60 msgr2=0x7f199412bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:25.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.794+0000 7f199c266640 1 -- 172.21.15.46:0/128033596 shutdown_connections 2024-05-08T18:08:25.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:25.794+0000 7f199c266640 1 -- 172.21.15.46:0/128033596 wait complete. 2024-05-08T18:08:25.805 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-05-08T18:08:25.735710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605383,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":5050,"num_read_kb":4636,"num_write":2119,"num_write_kb":60585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1336,"num_omap_keys":6,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4683,"ondisk_log_size":4683,"up":658,"acting":658,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":658,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":156568,"kb_used_data":2744,"kb_used_omap":6,"kb_used_meta":153785,"kb_avail":377330792,"statfs":{"total":386547056640,"available":386386731008,"internally_reserved":0,"allocated":2809856,"data_stored":1720046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":157476630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1336,"num_omap_keys":6,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003868"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.742743+0000","last_change":"2024-05-08T18:04:43.836900+0000","last_active":"2024-05-08T18:06:48.742743+0000","last_peered":"2024-05-08T18:06:48.742743+0000","last_clean":"2024-05-08T18:06:48.742743+0000","last_became_active":"2024-05-08T18:04:43.836496+0000","last_became_peered":"2024-05-08T18:04:43.836496+0000","last_unstale":"2024-05-08T18:06:48.742743+0000","last_undegraded":"2024-05-08T18:06:48.742743+0000","last_fullsized":"2024-05-08T18:06:48.742743+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'6","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.866653+0000","last_change":"2024-05-08T18:04:45.812340+0000","last_active":"2024-05-08T18:07:53.866653+0000","last_peered":"2024-05-08T18:07:53.866653+0000","last_clean":"2024-05-08T18:07:53.866653+0000","last_became_active":"2024-05-08T18:04:45.811368+0000","last_became_peered":"2024-05-08T18:04:45.811368+0000","last_unstale":"2024-05-08T18:07:53.866653+0000","last_undegraded":"2024-05-08T18:07:53.866653+0000","last_fullsized":"2024-05-08T18:07:53.866653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"29'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.209476+0000","last_change":"2024-05-08T18:04:43.835466+0000","last_active":"2024-05-08T18:06:50.209476+0000","last_peered":"2024-05-08T18:06:50.209476+0000","last_clean":"2024-05-08T18:06:50.209476+0000","last_became_active":"2024-05-08T18:04:43.834612+0000","last_became_peered":"2024-05-08T18:04:43.834612+0000","last_unstale":"2024-05-08T18:06:50.209476+0000","last_undegraded":"2024-05-08T18:06:50.209476+0000","last_fullsized":"2024-05-08T18:06:50.209476+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.748401+0000","last_change":"2024-05-08T18:04:45.828749+0000","last_active":"2024-05-08T18:07:18.748401+0000","last_peered":"2024-05-08T18:07:18.748401+0000","last_clean":"2024-05-08T18:07:18.748401+0000","last_became_active":"2024-05-08T18:04:45.827965+0000","last_became_peered":"2024-05-08T18:04:45.827965+0000","last_unstale":"2024-05-08T18:07:18.748401+0000","last_undegraded":"2024-05-08T18:07:18.748401+0000","last_fullsized":"2024-05-08T18:07:18.748401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.743553+0000","last_change":"2024-05-08T18:04:43.843806+0000","last_active":"2024-05-08T18:06:48.743553+0000","last_peered":"2024-05-08T18:06:48.743553+0000","last_clean":"2024-05-08T18:06:48.743553+0000","last_became_active":"2024-05-08T18:04:43.837456+0000","last_became_peered":"2024-05-08T18:04:43.837456+0000","last_unstale":"2024-05-08T18:06:48.743553+0000","last_undegraded":"2024-05-08T18:06:48.743553+0000","last_fullsized":"2024-05-08T18:06:48.743553+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"29'12","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070668+0000","last_change":"2024-05-08T18:04:45.825315+0000","last_active":"2024-05-08T18:07:54.070668+0000","last_peered":"2024-05-08T18:07:54.070668+0000","last_clean":"2024-05-08T18:07:54.070668+0000","last_became_active":"2024-05-08T18:04:45.825022+0000","last_became_peered":"2024-05-08T18:04:45.825022+0000","last_unstale":"2024-05-08T18:07:54.070668+0000","last_undegraded":"2024-05-08T18:07:54.070668+0000","last_fullsized":"2024-05-08T18:07:54.070668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:43.741978+0000","last_change":"2024-05-08T18:04:43.842478+0000","last_active":"2024-05-08T18:06:43.741978+0000","last_peered":"2024-05-08T18:06:43.741978+0000","last_clean":"2024-05-08T18:06:43.741978+0000","last_became_active":"2024-05-08T18:04:43.842077+0000","last_became_peered":"2024-05-08T18:04:43.842077+0000","last_unstale":"2024-05-08T18:06:43.741978+0000","last_undegraded":"2024-05-08T18:06:43.741978+0000","last_fullsized":"2024-05-08T18:06:43.741978+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"29'16","reported_seq":71,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.041602+0000","last_change":"2024-05-08T18:04:45.824870+0000","last_active":"2024-05-08T18:07:54.041602+0000","last_peered":"2024-05-08T18:07:54.041602+0000","last_clean":"2024-05-08T18:07:54.041602+0000","last_became_active":"2024-05-08T18:04:45.824581+0000","last_became_peered":"2024-05-08T18:04:45.824581+0000","last_unstale":"2024-05-08T18:07:54.041602+0000","last_undegraded":"2024-05-08T18:07:54.041602+0000","last_fullsized":"2024-05-08T18:07:54.041602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747778+0000","last_change":"2024-05-08T18:04:43.833461+0000","last_active":"2024-05-08T18:07:13.747778+0000","last_peered":"2024-05-08T18:07:13.747778+0000","last_clean":"2024-05-08T18:07:13.747778+0000","last_became_active":"2024-05-08T18:04:43.832722+0000","last_became_peered":"2024-05-08T18:04:43.832722+0000","last_unstale":"2024-05-08T18:07:13.747778+0000","last_undegraded":"2024-05-08T18:07:13.747778+0000","last_fullsized":"2024-05-08T18:07:13.747778+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"29'8","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.743014+0000","last_change":"2024-05-08T18:04:45.808678+0000","last_active":"2024-05-08T18:06:48.743014+0000","last_peered":"2024-05-08T18:06:48.743014+0000","last_clean":"2024-05-08T18:06:48.743014+0000","last_became_active":"2024-05-08T18:04:45.808383+0000","last_became_peered":"2024-05-08T18:04:45.808383+0000","last_unstale":"2024-05-08T18:06:48.743014+0000","last_undegraded":"2024-05-08T18:06:48.743014+0000","last_fullsized":"2024-05-08T18:06:48.743014+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.746888+0000","last_change":"2024-05-08T18:04:43.841284+0000","last_active":"2024-05-08T18:07:13.746888+0000","last_peered":"2024-05-08T18:07:13.746888+0000","last_clean":"2024-05-08T18:07:13.746888+0000","last_became_active":"2024-05-08T18:04:43.840851+0000","last_became_peered":"2024-05-08T18:04:43.840851+0000","last_unstale":"2024-05-08T18:07:13.746888+0000","last_undegraded":"2024-05-08T18:07:13.746888+0000","last_fullsized":"2024-05-08T18:07:13.746888+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070775+0000","last_change":"2024-05-08T18:04:45.829481+0000","last_active":"2024-05-08T18:07:54.070775+0000","last_peered":"2024-05-08T18:07:54.070775+0000","last_clean":"2024-05-08T18:07:54.070775+0000","last_became_active":"2024-05-08T18:04:45.828334+0000","last_became_peered":"2024-05-08T18:04:45.828334+0000","last_unstale":"2024-05-08T18:07:54.070775+0000","last_undegraded":"2024-05-08T18:07:54.070775+0000","last_fullsized":"2024-05-08T18:07:54.070775+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"29'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.981231+0000","last_change":"2024-05-08T18:04:43.833826+0000","last_active":"2024-05-08T18:07:53.981231+0000","last_peered":"2024-05-08T18:07:53.981231+0000","last_clean":"2024-05-08T18:07:53.981231+0000","last_became_active":"2024-05-08T18:04:43.833437+0000","last_became_peered":"2024-05-08T18:04:43.833437+0000","last_unstale":"2024-05-08T18:07:53.981231+0000","last_undegraded":"2024-05-08T18:07:53.981231+0000","last_fullsized":"2024-05-08T18:07:53.981231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.743533+0000","last_change":"2024-05-08T18:04:45.812158+0000","last_active":"2024-05-08T18:06:53.743533+0000","last_peered":"2024-05-08T18:06:53.743533+0000","last_clean":"2024-05-08T18:06:53.743533+0000","last_became_active":"2024-05-08T18:04:45.811884+0000","last_became_peered":"2024-05-08T18:04:45.811884+0000","last_unstale":"2024-05-08T18:06:53.743533+0000","last_undegraded":"2024-05-08T18:06:53.743533+0000","last_fullsized":"2024-05-08T18:06:53.743533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.741950+0000","last_change":"2024-05-08T18:04:43.821005+0000","last_active":"2024-05-08T18:07:13.741950+0000","last_peered":"2024-05-08T18:07:13.741950+0000","last_clean":"2024-05-08T18:07:13.741950+0000","last_became_active":"2024-05-08T18:04:43.820312+0000","last_became_peered":"2024-05-08T18:04:43.820312+0000","last_unstale":"2024-05-08T18:07:13.741950+0000","last_undegraded":"2024-05-08T18:07:13.741950+0000","last_fullsized":"2024-05-08T18:07:13.741950+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"29'7","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.115771+0000","last_change":"2024-05-08T18:04:45.823448+0000","last_active":"2024-05-08T18:07:54.115771+0000","last_peered":"2024-05-08T18:07:54.115771+0000","last_clean":"2024-05-08T18:07:54.115771+0000","last_became_active":"2024-05-08T18:04:45.822620+0000","last_became_peered":"2024-05-08T18:04:45.822620+0000","last_unstale":"2024-05-08T18:07:54.115771+0000","last_undegraded":"2024-05-08T18:07:54.115771+0000","last_fullsized":"2024-05-08T18:07:54.115771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"29'4","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.002897+0000","last_change":"2024-05-08T18:04:45.826814+0000","last_active":"2024-05-08T18:07:54.002897+0000","last_peered":"2024-05-08T18:07:54.002897+0000","last_clean":"2024-05-08T18:07:54.002897+0000","last_became_active":"2024-05-08T18:04:45.826493+0000","last_became_peered":"2024-05-08T18:04:45.826493+0000","last_unstale":"2024-05-08T18:07:54.002897+0000","last_undegraded":"2024-05-08T18:07:54.002897+0000","last_fullsized":"2024-05-08T18:07:54.002897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746269+0000","last_change":"2024-05-08T18:04:43.838700+0000","last_active":"2024-05-08T18:07:08.746269+0000","last_peered":"2024-05-08T18:07:08.746269+0000","last_clean":"2024-05-08T18:07:08.746269+0000","last_became_active":"2024-05-08T18:04:43.838332+0000","last_became_peered":"2024-05-08T18:04:43.838332+0000","last_unstale":"2024-05-08T18:07:08.746269+0000","last_undegraded":"2024-05-08T18:07:08.746269+0000","last_fullsized":"2024-05-08T18:07:08.746269+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"29'4","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746077+0000","last_change":"2024-05-08T18:04:45.807732+0000","last_active":"2024-05-08T18:07:08.746077+0000","last_peered":"2024-05-08T18:07:08.746077+0000","last_clean":"2024-05-08T18:07:08.746077+0000","last_became_active":"2024-05-08T18:04:45.807438+0000","last_became_peered":"2024-05-08T18:04:45.807438+0000","last_unstale":"2024-05-08T18:07:08.746077+0000","last_undegraded":"2024-05-08T18:07:08.746077+0000","last_fullsized":"2024-05-08T18:07:08.746077+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746183+0000","last_change":"2024-05-08T18:04:43.843599+0000","last_active":"2024-05-08T18:07:08.746183+0000","last_peered":"2024-05-08T18:07:08.746183+0000","last_clean":"2024-05-08T18:07:08.746183+0000","last_became_active":"2024-05-08T18:04:43.842741+0000","last_became_peered":"2024-05-08T18:04:43.842741+0000","last_unstale":"2024-05-08T18:07:08.746183+0000","last_undegraded":"2024-05-08T18:07:08.746183+0000","last_fullsized":"2024-05-08T18:07:08.746183+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"29'11","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.116188+0000","last_change":"2024-05-08T18:04:45.811141+0000","last_active":"2024-05-08T18:07:54.116188+0000","last_peered":"2024-05-08T18:07:54.116188+0000","last_clean":"2024-05-08T18:07:54.116188+0000","last_became_active":"2024-05-08T18:04:45.810827+0000","last_became_peered":"2024-05-08T18:04:45.810827+0000","last_unstale":"2024-05-08T18:07:54.116188+0000","last_undegraded":"2024-05-08T18:07:54.116188+0000","last_fullsized":"2024-05-08T18:07:54.116188+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"29'4","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.670348+0000","last_change":"2024-05-08T18:04:43.838115+0000","last_active":"2024-05-08T18:08:00.670348+0000","last_peered":"2024-05-08T18:08:00.670348+0000","last_clean":"2024-05-08T18:08:00.670348+0000","last_became_active":"2024-05-08T18:04:43.837706+0000","last_became_peered":"2024-05-08T18:04:43.837706+0000","last_unstale":"2024-05-08T18:08:00.670348+0000","last_undegraded":"2024-05-08T18:08:00.670348+0000","last_fullsized":"2024-05-08T18:08:00.670348+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"29'8","reported_seq":60,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.742019+0000","last_change":"2024-05-08T18:04:45.822292+0000","last_active":"2024-05-08T18:07:13.742019+0000","last_peered":"2024-05-08T18:07:13.742019+0000","last_clean":"2024-05-08T18:07:13.742019+0000","last_became_active":"2024-05-08T18:04:45.822000+0000","last_became_peered":"2024-05-08T18:04:45.822000+0000","last_unstale":"2024-05-08T18:07:13.742019+0000","last_undegraded":"2024-05-08T18:07:13.742019+0000","last_fullsized":"2024-05-08T18:07:13.742019+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746175+0000","last_change":"2024-05-08T18:04:43.841486+0000","last_active":"2024-05-08T18:07:08.746175+0000","last_peered":"2024-05-08T18:07:08.746175+0000","last_clean":"2024-05-08T18:07:08.746175+0000","last_became_active":"2024-05-08T18:04:43.841063+0000","last_became_peered":"2024-05-08T18:04:43.841063+0000","last_unstale":"2024-05-08T18:07:08.746175+0000","last_undegraded":"2024-05-08T18:07:08.746175+0000","last_fullsized":"2024-05-08T18:07:08.746175+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"29'2","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.116529+0000","last_change":"2024-05-08T18:04:45.824006+0000","last_active":"2024-05-08T18:07:54.116529+0000","last_peered":"2024-05-08T18:07:54.116529+0000","last_clean":"2024-05-08T18:07:54.116529+0000","last_became_active":"2024-05-08T18:04:45.823611+0000","last_became_peered":"2024-05-08T18:04:45.823611+0000","last_unstale":"2024-05-08T18:07:54.116529+0000","last_undegraded":"2024-05-08T18:07:54.116529+0000","last_fullsized":"2024-05-08T18:07:54.116529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.741273+0000","last_change":"2024-05-08T18:04:43.835685+0000","last_active":"2024-05-08T18:07:08.741273+0000","last_peered":"2024-05-08T18:07:08.741273+0000","last_clean":"2024-05-08T18:07:08.741273+0000","last_became_active":"2024-05-08T18:04:43.834456+0000","last_became_peered":"2024-05-08T18:04:43.834456+0000","last_unstale":"2024-05-08T18:07:08.741273+0000","last_undegraded":"2024-05-08T18:07:08.741273+0000","last_fullsized":"2024-05-08T18:07:08.741273+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"29'6","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747008+0000","last_change":"2024-05-08T18:04:45.808763+0000","last_active":"2024-05-08T18:07:13.747008+0000","last_peered":"2024-05-08T18:07:13.747008+0000","last_clean":"2024-05-08T18:07:13.747008+0000","last_became_active":"2024-05-08T18:04:45.808416+0000","last_became_peered":"2024-05-08T18:04:45.808416+0000","last_unstale":"2024-05-08T18:07:13.747008+0000","last_undegraded":"2024-05-08T18:07:13.747008+0000","last_fullsized":"2024-05-08T18:07:13.747008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'4","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745942+0000","last_change":"2024-05-08T18:04:43.845458+0000","last_active":"2024-05-08T18:07:03.745942+0000","last_peered":"2024-05-08T18:07:03.745942+0000","last_clean":"2024-05-08T18:07:03.745942+0000","last_became_active":"2024-05-08T18:04:43.844702+0000","last_became_peered":"2024-05-08T18:04:43.844702+0000","last_unstale":"2024-05-08T18:07:03.745942+0000","last_undegraded":"2024-05-08T18:07:03.745942+0000","last_fullsized":"2024-05-08T18:07:03.745942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"29'35","reported_seq":100,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.080132+0000","last_change":"2024-05-08T18:04:53.849775+0000","last_active":"2024-05-08T18:07:54.080132+0000","last_peered":"2024-05-08T18:07:54.080132+0000","last_clean":"2024-05-08T18:07:54.080132+0000","last_became_active":"2024-05-08T18:04:53.849475+0000","last_became_peered":"2024-05-08T18:04:53.849475+0000","last_unstale":"2024-05-08T18:07:54.080132+0000","last_undegraded":"2024-05-08T18:07:54.080132+0000","last_fullsized":"2024-05-08T18:07:54.080132+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"29'12","reported_seq":67,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.002987+0000","last_change":"2024-05-08T18:04:45.824289+0000","last_active":"2024-05-08T18:07:54.002987+0000","last_peered":"2024-05-08T18:07:54.002987+0000","last_clean":"2024-05-08T18:07:54.002987+0000","last_became_active":"2024-05-08T18:04:45.824005+0000","last_became_peered":"2024-05-08T18:04:45.824005+0000","last_unstale":"2024-05-08T18:07:54.002987+0000","last_undegraded":"2024-05-08T18:07:54.002987+0000","last_fullsized":"2024-05-08T18:07:54.002987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745982+0000","last_change":"2024-05-08T18:04:43.834249+0000","last_active":"2024-05-08T18:07:03.745982+0000","last_peered":"2024-05-08T18:07:03.745982+0000","last_clean":"2024-05-08T18:07:03.745982+0000","last_became_active":"2024-05-08T18:04:43.833822+0000","last_became_peered":"2024-05-08T18:04:43.833822+0000","last_unstale":"2024-05-08T18:07:03.745982+0000","last_undegraded":"2024-05-08T18:07:03.745982+0000","last_fullsized":"2024-05-08T18:07:03.745982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.924513+0000","last_change":"2024-05-08T18:08:09.924538+0000","last_active":"2024-05-08T18:08:09.924513+0000","last_peered":"2024-05-08T18:08:09.924513+0000","last_clean":"2024-05-08T18:08:09.924513+0000","last_became_active":"2024-05-08T18:04:47.819682+0000","last_became_peered":"2024-05-08T18:04:47.819682+0000","last_unstale":"2024-05-08T18:08:09.924513+0000","last_undegraded":"2024-05-08T18:08:09.924513+0000","last_fullsized":"2024-05-08T18:08:09.924513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:09.924406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:09.924406+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.924406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:58.088967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"29'35","reported_seq":82,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.674088+0000","last_change":"2024-05-08T18:04:49.833877+0000","last_active":"2024-05-08T18:08:00.674088+0000","last_peered":"2024-05-08T18:08:00.674088+0000","last_clean":"2024-05-08T18:08:00.674088+0000","last_became_active":"2024-05-08T18:04:49.832148+0000","last_became_peered":"2024-05-08T18:04:49.832148+0000","last_unstale":"2024-05-08T18:08:00.674088+0000","last_undegraded":"2024-05-08T18:08:00.674088+0000","last_fullsized":"2024-05-08T18:08:00.674088+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":40,"num_read_kb":42,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.510810+0000","last_change":"2024-05-08T18:08:21.510876+0000","last_active":"2024-05-08T18:08:21.510810+0000","last_peered":"2024-05-08T18:08:21.510810+0000","last_clean":"2024-05-08T18:08:21.510810+0000","last_became_active":"2024-05-08T18:04:51.844901+0000","last_became_peered":"2024-05-08T18:04:51.844901+0000","last_unstale":"2024-05-08T18:08:21.510810+0000","last_undegraded":"2024-05-08T18:08:21.510810+0000","last_fullsized":"2024-05-08T18:08:21.510810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:21.510687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:21.510687+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.510687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:55:15.292759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"29'11","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.747622+0000","last_change":"2024-05-08T18:04:53.849933+0000","last_active":"2024-05-08T18:07:18.747622+0000","last_peered":"2024-05-08T18:07:18.747622+0000","last_clean":"2024-05-08T18:07:18.747622+0000","last_became_active":"2024-05-08T18:04:53.849579+0000","last_became_peered":"2024-05-08T18:04:53.849579+0000","last_unstale":"2024-05-08T18:07:18.747622+0000","last_undegraded":"2024-05-08T18:07:18.747622+0000","last_fullsized":"2024-05-08T18:07:18.747622+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"29'6","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070924+0000","last_change":"2024-05-08T18:04:45.826976+0000","last_active":"2024-05-08T18:07:54.070924+0000","last_peered":"2024-05-08T18:07:54.070924+0000","last_clean":"2024-05-08T18:07:54.070924+0000","last_became_active":"2024-05-08T18:04:45.825162+0000","last_became_peered":"2024-05-08T18:04:45.825162+0000","last_unstale":"2024-05-08T18:07:54.070924+0000","last_undegraded":"2024-05-08T18:07:54.070924+0000","last_fullsized":"2024-05-08T18:07:54.070924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745158+0000","last_change":"2024-05-08T18:04:43.827844+0000","last_active":"2024-05-08T18:07:03.745158+0000","last_peered":"2024-05-08T18:07:03.745158+0000","last_clean":"2024-05-08T18:07:03.745158+0000","last_became_active":"2024-05-08T18:04:43.827374+0000","last_became_peered":"2024-05-08T18:04:43.827374+0000","last_unstale":"2024-05-08T18:07:03.745158+0000","last_undegraded":"2024-05-08T18:07:03.745158+0000","last_fullsized":"2024-05-08T18:07:03.745158+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.752839+0000","last_change":"2024-05-08T18:05:30.868535+0000","last_active":"2024-05-08T18:07:53.752839+0000","last_peered":"2024-05-08T18:07:53.752839+0000","last_clean":"2024-05-08T18:07:53.752839+0000","last_became_active":"2024-05-08T18:05:30.868191+0000","last_became_peered":"2024-05-08T18:05:30.868191+0000","last_unstale":"2024-05-08T18:07:53.752839+0000","last_undegraded":"2024-05-08T18:07:53.752839+0000","last_fullsized":"2024-05-08T18:07:53.752839+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"29'9","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.529952+0000","last_change":"2024-05-08T18:08:10.529988+0000","last_active":"2024-05-08T18:08:10.529952+0000","last_peered":"2024-05-08T18:08:10.529952+0000","last_clean":"2024-05-08T18:08:10.529952+0000","last_became_active":"2024-05-08T18:04:49.825265+0000","last_became_peered":"2024-05-08T18:04:49.825265+0000","last_unstale":"2024-05-08T18:08:10.529952+0000","last_undegraded":"2024-05-08T18:08:10.529952+0000","last_fullsized":"2024-05-08T18:08:10.529952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'9","last_scrub_stamp":"2024-05-08T18:08:10.529858+0000","last_deep_scrub":"29'9","last_deep_scrub_stamp":"2024-05-08T18:08:10.529858+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.529858+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:00.215888+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.491557+0000","last_change":"2024-05-08T18:08:13.491631+0000","last_active":"2024-05-08T18:08:13.491557+0000","last_peered":"2024-05-08T18:08:13.491557+0000","last_clean":"2024-05-08T18:08:13.491557+0000","last_became_active":"2024-05-08T18:04:51.833523+0000","last_became_peered":"2024-05-08T18:04:51.833523+0000","last_unstale":"2024-05-08T18:08:13.491557+0000","last_undegraded":"2024-05-08T18:08:13.491557+0000","last_fullsized":"2024-05-08T18:08:13.491557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:13.491458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:13.491458+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.491458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:07.228339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'2","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:23.743329+0000","last_change":"2024-05-08T18:04:53.840760+0000","last_active":"2024-05-08T18:07:23.743329+0000","last_peered":"2024-05-08T18:07:23.743329+0000","last_clean":"2024-05-08T18:07:23.743329+0000","last_became_active":"2024-05-08T18:04:53.840395+0000","last_became_peered":"2024-05-08T18:04:53.840395+0000","last_unstale":"2024-05-08T18:07:23.743329+0000","last_undegraded":"2024-05-08T18:07:23.743329+0000","last_fullsized":"2024-05-08T18:07:23.743329+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:58.753460+0000","last_change":"2024-05-08T18:05:30.876470+0000","last_active":"2024-05-08T18:07:58.753460+0000","last_peered":"2024-05-08T18:07:58.753460+0000","last_clean":"2024-05-08T18:07:58.753460+0000","last_became_active":"2024-05-08T18:05:30.869162+0000","last_became_peered":"2024-05-08T18:05:30.869162+0000","last_unstale":"2024-05-08T18:07:58.753460+0000","last_undegraded":"2024-05-08T18:07:58.753460+0000","last_fullsized":"2024-05-08T18:07:58.753460+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":45,"num_write":24,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745862+0000","last_change":"2024-05-08T18:04:43.844847+0000","last_active":"2024-05-08T18:07:03.745862+0000","last_peered":"2024-05-08T18:07:03.745862+0000","last_clean":"2024-05-08T18:07:03.745862+0000","last_became_active":"2024-05-08T18:04:43.838666+0000","last_became_peered":"2024-05-08T18:04:43.838666+0000","last_unstale":"2024-05-08T18:07:03.745862+0000","last_undegraded":"2024-05-08T18:07:03.745862+0000","last_fullsized":"2024-05-08T18:07:03.745862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.742090+0000","last_change":"2024-05-08T18:04:47.819743+0000","last_active":"2024-05-08T18:07:13.742090+0000","last_peered":"2024-05-08T18:07:13.742090+0000","last_clean":"2024-05-08T18:07:13.742090+0000","last_became_active":"2024-05-08T18:04:47.819169+0000","last_became_peered":"2024-05-08T18:04:47.819169+0000","last_unstale":"2024-05-08T18:07:13.742090+0000","last_undegraded":"2024-05-08T18:07:13.742090+0000","last_fullsized":"2024-05-08T18:07:13.742090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"29'17","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.168244+0000","last_change":"2024-05-08T18:08:11.168281+0000","last_active":"2024-05-08T18:08:11.168244+0000","last_peered":"2024-05-08T18:08:11.168244+0000","last_clean":"2024-05-08T18:08:11.168244+0000","last_became_active":"2024-05-08T18:04:49.831500+0000","last_became_peered":"2024-05-08T18:04:49.831500+0000","last_unstale":"2024-05-08T18:08:11.168244+0000","last_undegraded":"2024-05-08T18:08:11.168244+0000","last_fullsized":"2024-05-08T18:08:11.168244+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'17","last_scrub_stamp":"2024-05-08T18:08:11.168144+0000","last_deep_scrub":"29'17","last_deep_scrub_stamp":"2024-05-08T18:08:11.168144+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.168144+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:03.071434+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"29'285","reported_seq":385,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.774392+0000","last_change":"2024-05-08T18:04:51.849859+0000","last_active":"2024-05-08T18:08:00.774392+0000","last_peered":"2024-05-08T18:08:00.774392+0000","last_clean":"2024-05-08T18:08:00.774392+0000","last_became_active":"2024-05-08T18:04:51.849301+0000","last_became_peered":"2024-05-08T18:04:51.849301+0000","last_unstale":"2024-05-08T18:08:00.774392+0000","last_undegraded":"2024-05-08T18:08:00.774392+0000","last_fullsized":"2024-05-08T18:08:00.774392+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:23.748336+0000","last_change":"2024-05-08T18:04:53.840922+0000","last_active":"2024-05-08T18:07:23.748336+0000","last_peered":"2024-05-08T18:07:23.748336+0000","last_clean":"2024-05-08T18:07:23.748336+0000","last_became_active":"2024-05-08T18:04:53.840559+0000","last_became_peered":"2024-05-08T18:04:53.840559+0000","last_unstale":"2024-05-08T18:07:23.748336+0000","last_undegraded":"2024-05-08T18:07:23.748336+0000","last_fullsized":"2024-05-08T18:07:23.748336+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"29'6","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.041576+0000","last_change":"2024-05-08T18:04:45.812045+0000","last_active":"2024-05-08T18:07:54.041576+0000","last_peered":"2024-05-08T18:07:54.041576+0000","last_clean":"2024-05-08T18:07:54.041576+0000","last_became_active":"2024-05-08T18:04:45.811570+0000","last_became_peered":"2024-05-08T18:04:45.811570+0000","last_unstale":"2024-05-08T18:07:54.041576+0000","last_undegraded":"2024-05-08T18:07:54.041576+0000","last_fullsized":"2024-05-08T18:07:54.041576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"29'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.990442+0000","last_change":"2024-05-08T18:04:43.838356+0000","last_active":"2024-05-08T18:07:53.990442+0000","last_peered":"2024-05-08T18:07:53.990442+0000","last_clean":"2024-05-08T18:07:53.990442+0000","last_became_active":"2024-05-08T18:04:43.836410+0000","last_became_peered":"2024-05-08T18:04:43.836410+0000","last_unstale":"2024-05-08T18:07:53.990442+0000","last_undegraded":"2024-05-08T18:07:53.990442+0000","last_fullsized":"2024-05-08T18:07:53.990442+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.849059+0000","last_change":"2024-05-08T18:08:11.849101+0000","last_active":"2024-05-08T18:08:11.849059+0000","last_peered":"2024-05-08T18:08:11.849059+0000","last_clean":"2024-05-08T18:08:11.849059+0000","last_became_active":"2024-05-08T18:04:47.818797+0000","last_became_peered":"2024-05-08T18:04:47.818797+0000","last_unstale":"2024-05-08T18:08:11.849059+0000","last_undegraded":"2024-05-08T18:08:11.849059+0000","last_fullsized":"2024-05-08T18:08:11.849059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:11.848881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:11.848881+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.848881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:19.374213+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"29'37","reported_seq":92,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.217059+0000","last_change":"2024-05-08T18:08:12.217092+0000","last_active":"2024-05-08T18:08:12.217059+0000","last_peered":"2024-05-08T18:08:12.217059+0000","last_clean":"2024-05-08T18:08:12.217059+0000","last_became_active":"2024-05-08T18:04:49.831379+0000","last_became_peered":"2024-05-08T18:04:49.831379+0000","last_unstale":"2024-05-08T18:08:12.217059+0000","last_undegraded":"2024-05-08T18:08:12.217059+0000","last_fullsized":"2024-05-08T18:08:12.217059+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'37","last_scrub_stamp":"2024-05-08T18:08:12.216965+0000","last_deep_scrub":"29'37","last_deep_scrub_stamp":"2024-05-08T18:08:12.216965+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.216965+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:49:34.759465+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"29'285","reported_seq":453,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.529055+0000","last_change":"2024-05-08T18:08:11.529087+0000","last_active":"2024-05-08T18:08:11.529055+0000","last_peered":"2024-05-08T18:08:11.529055+0000","last_clean":"2024-05-08T18:08:11.529055+0000","last_became_active":"2024-05-08T18:04:51.826926+0000","last_became_peered":"2024-05-08T18:04:51.826926+0000","last_unstale":"2024-05-08T18:08:11.529055+0000","last_undegraded":"2024-05-08T18:08:11.529055+0000","last_fullsized":"2024-05-08T18:08:11.529055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:11.528959+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:11.528959+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.528959+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:44.650679+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'6","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744861+0000","last_change":"2024-05-08T18:04:45.827616+0000","last_active":"2024-05-08T18:06:58.744861+0000","last_peered":"2024-05-08T18:06:58.744861+0000","last_clean":"2024-05-08T18:06:58.744861+0000","last_became_active":"2024-05-08T18:04:45.827351+0000","last_became_peered":"2024-05-08T18:04:45.827351+0000","last_unstale":"2024-05-08T18:06:58.744861+0000","last_undegraded":"2024-05-08T18:06:58.744861+0000","last_fullsized":"2024-05-08T18:06:58.744861+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744399+0000","last_change":"2024-05-08T18:04:43.839205+0000","last_active":"2024-05-08T18:06:58.744399+0000","last_peered":"2024-05-08T18:06:58.744399+0000","last_clean":"2024-05-08T18:06:58.744399+0000","last_became_active":"2024-05-08T18:04:43.838840+0000","last_became_peered":"2024-05-08T18:04:43.838840+0000","last_unstale":"2024-05-08T18:06:58.744399+0000","last_undegraded":"2024-05-08T18:06:58.744399+0000","last_fullsized":"2024-05-08T18:06:58.744399+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,0],"acting":[2,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.468389+0000","last_change":"2024-05-08T18:08:14.468441+0000","last_active":"2024-05-08T18:08:14.468389+0000","last_peered":"2024-05-08T18:08:14.468389+0000","last_clean":"2024-05-08T18:08:14.468389+0000","last_became_active":"2024-05-08T18:04:47.825215+0000","last_became_peered":"2024-05-08T18:04:47.825215+0000","last_unstale":"2024-05-08T18:08:14.468389+0000","last_undegraded":"2024-05-08T18:08:14.468389+0000","last_fullsized":"2024-05-08T18:08:14.468389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:14.468285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:14.468285+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.468285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:38.045371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"29'51","reported_seq":111,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.277773+0000","last_change":"2024-05-08T18:08:14.277853+0000","last_active":"2024-05-08T18:08:14.277773+0000","last_peered":"2024-05-08T18:08:14.277773+0000","last_clean":"2024-05-08T18:08:14.277773+0000","last_became_active":"2024-05-08T18:04:49.827954+0000","last_became_peered":"2024-05-08T18:04:49.827954+0000","last_unstale":"2024-05-08T18:08:14.277773+0000","last_undegraded":"2024-05-08T18:08:14.277773+0000","last_fullsized":"2024-05-08T18:08:14.277773+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'51","last_scrub_stamp":"2024-05-08T18:08:14.277647+0000","last_deep_scrub":"29'51","last_deep_scrub_stamp":"2024-05-08T18:08:14.277647+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.277647+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:23:20.826582+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.436717+0000","last_change":"2024-05-08T18:08:07.436758+0000","last_active":"2024-05-08T18:08:07.436717+0000","last_peered":"2024-05-08T18:08:07.436717+0000","last_clean":"2024-05-08T18:08:07.436717+0000","last_became_active":"2024-05-08T18:04:51.831149+0000","last_became_peered":"2024-05-08T18:04:51.831149+0000","last_unstale":"2024-05-08T18:08:07.436717+0000","last_undegraded":"2024-05-08T18:08:07.436717+0000","last_fullsized":"2024-05-08T18:08:07.436717+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:07.436627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:07.436627+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.436627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:56:38.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"29'5","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744266+0000","last_change":"2024-05-08T18:04:53.840960+0000","last_active":"2024-05-08T18:06:58.744266+0000","last_peered":"2024-05-08T18:06:58.744266+0000","last_clean":"2024-05-08T18:06:58.744266+0000","last_became_active":"2024-05-08T18:04:53.840652+0000","last_became_peered":"2024-05-08T18:04:53.840652+0000","last_unstale":"2024-05-08T18:06:58.744266+0000","last_undegraded":"2024-05-08T18:06:58.744266+0000","last_fullsized":"2024-05-08T18:06:58.744266+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"27'4","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744453+0000","last_change":"2024-05-08T18:04:45.804412+0000","last_active":"2024-05-08T18:06:58.744453+0000","last_peered":"2024-05-08T18:06:58.744453+0000","last_clean":"2024-05-08T18:06:58.744453+0000","last_became_active":"2024-05-08T18:04:45.804124+0000","last_became_peered":"2024-05-08T18:04:45.804124+0000","last_unstale":"2024-05-08T18:06:58.744453+0000","last_undegraded":"2024-05-08T18:06:58.744453+0000","last_fullsized":"2024-05-08T18:06:58.744453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744348+0000","last_change":"2024-05-08T18:04:43.843177+0000","last_active":"2024-05-08T18:06:58.744348+0000","last_peered":"2024-05-08T18:06:58.744348+0000","last_clean":"2024-05-08T18:06:58.744348+0000","last_became_active":"2024-05-08T18:04:43.842126+0000","last_became_peered":"2024-05-08T18:04:43.842126+0000","last_unstale":"2024-05-08T18:06:58.744348+0000","last_undegraded":"2024-05-08T18:06:58.744348+0000","last_fullsized":"2024-05-08T18:06:58.744348+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.583926+0000","last_change":"2024-05-08T18:08:22.583963+0000","last_active":"2024-05-08T18:08:22.583926+0000","last_peered":"2024-05-08T18:08:22.583926+0000","last_clean":"2024-05-08T18:08:22.583926+0000","last_became_active":"2024-05-08T18:04:47.822688+0000","last_became_peered":"2024-05-08T18:04:47.822688+0000","last_unstale":"2024-05-08T18:08:22.583926+0000","last_undegraded":"2024-05-08T18:08:22.583926+0000","last_fullsized":"2024-05-08T18:08:22.583926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.583822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.583822+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.583822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:22.094540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"29'36","reported_seq":91,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.839677+0000","last_change":"2024-05-08T18:08:16.839747+0000","last_active":"2024-05-08T18:08:16.839677+0000","last_peered":"2024-05-08T18:08:16.839677+0000","last_clean":"2024-05-08T18:08:16.839677+0000","last_became_active":"2024-05-08T18:04:49.829892+0000","last_became_peered":"2024-05-08T18:04:49.829892+0000","last_unstale":"2024-05-08T18:08:16.839677+0000","last_undegraded":"2024-05-08T18:08:16.839677+0000","last_fullsized":"2024-05-08T18:08:16.839677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:08:16.839580+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:08:16.839580+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.839580+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:53.387986+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:04.455783+0000","last_change":"2024-05-08T18:08:04.455804+0000","last_active":"2024-05-08T18:08:04.455783+0000","last_peered":"2024-05-08T18:08:04.455783+0000","last_clean":"2024-05-08T18:08:04.455783+0000","last_became_active":"2024-05-08T18:04:51.832810+0000","last_became_peered":"2024-05-08T18:04:51.832810+0000","last_unstale":"2024-05-08T18:08:04.455783+0000","last_undegraded":"2024-05-08T18:08:04.455783+0000","last_fullsized":"2024-05-08T18:08:04.455783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:04.455699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:04.455699+0000","last_clean_scrub_stamp":"2024-05-08T18:08:04.455699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:21.455723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"29'3","reported_seq":24,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744578+0000","last_change":"2024-05-08T18:04:53.841201+0000","last_active":"2024-05-08T18:06:58.744578+0000","last_peered":"2024-05-08T18:06:58.744578+0000","last_clean":"2024-05-08T18:06:58.744578+0000","last_became_active":"2024-05-08T18:04:53.840894+0000","last_became_peered":"2024-05-08T18:04:53.840894+0000","last_unstale":"2024-05-08T18:06:58.744578+0000","last_undegraded":"2024-05-08T18:06:58.744578+0000","last_fullsized":"2024-05-08T18:06:58.744578+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"29'9","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.116462+0000","last_change":"2024-05-08T18:04:45.817096+0000","last_active":"2024-05-08T18:07:54.116462+0000","last_peered":"2024-05-08T18:07:54.116462+0000","last_clean":"2024-05-08T18:07:54.116462+0000","last_became_active":"2024-05-08T18:04:45.816817+0000","last_became_peered":"2024-05-08T18:04:45.816817+0000","last_unstale":"2024-05-08T18:07:54.116462+0000","last_undegraded":"2024-05-08T18:07:54.116462+0000","last_fullsized":"2024-05-08T18:07:54.116462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"29'4","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.668918+0000","last_change":"2024-05-08T18:04:43.844652+0000","last_active":"2024-05-08T18:08:00.668918+0000","last_peered":"2024-05-08T18:08:00.668918+0000","last_clean":"2024-05-08T18:08:00.668918+0000","last_became_active":"2024-05-08T18:04:43.838781+0000","last_became_peered":"2024-05-08T18:04:43.838781+0000","last_unstale":"2024-05-08T18:08:00.668918+0000","last_undegraded":"2024-05-08T18:08:00.668918+0000","last_fullsized":"2024-05-08T18:08:00.668918+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.740623+0000","last_change":"2024-05-08T18:04:47.819192+0000","last_active":"2024-05-08T18:07:03.740623+0000","last_peered":"2024-05-08T18:07:03.740623+0000","last_clean":"2024-05-08T18:07:03.740623+0000","last_became_active":"2024-05-08T18:04:47.818871+0000","last_became_peered":"2024-05-08T18:04:47.818871+0000","last_unstale":"2024-05-08T18:07:03.740623+0000","last_undegraded":"2024-05-08T18:07:03.740623+0000","last_fullsized":"2024-05-08T18:07:03.740623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"29'45","reported_seq":103,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.892863+0000","last_change":"2024-05-08T18:08:08.892899+0000","last_active":"2024-05-08T18:08:08.892863+0000","last_peered":"2024-05-08T18:08:08.892863+0000","last_clean":"2024-05-08T18:08:08.892863+0000","last_became_active":"2024-05-08T18:04:49.829270+0000","last_became_peered":"2024-05-08T18:04:49.829270+0000","last_unstale":"2024-05-08T18:08:08.892863+0000","last_undegraded":"2024-05-08T18:08:08.892863+0000","last_fullsized":"2024-05-08T18:08:08.892863+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'45","last_scrub_stamp":"2024-05-08T18:08:08.892763+0000","last_deep_scrub":"29'45","last_deep_scrub_stamp":"2024-05-08T18:08:08.892763+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.892763+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:11.460610+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"29'285","reported_seq":360,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.575979+0000","last_change":"2024-05-08T18:08:13.576007+0000","last_active":"2024-05-08T18:08:13.575979+0000","last_peered":"2024-05-08T18:08:13.575979+0000","last_clean":"2024-05-08T18:08:13.575979+0000","last_became_active":"2024-05-08T18:04:51.844529+0000","last_became_peered":"2024-05-08T18:04:51.844529+0000","last_unstale":"2024-05-08T18:08:13.575979+0000","last_undegraded":"2024-05-08T18:08:13.575979+0000","last_fullsized":"2024-05-08T18:08:13.575979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:13.575886+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:13.575886+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.575886+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:46.724169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"29'2","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.868126+0000","last_change":"2024-05-08T18:04:53.841062+0000","last_active":"2024-05-08T18:07:53.868126+0000","last_peered":"2024-05-08T18:07:53.868126+0000","last_clean":"2024-05-08T18:07:53.868126+0000","last_became_active":"2024-05-08T18:04:53.840767+0000","last_became_peered":"2024-05-08T18:04:53.840767+0000","last_unstale":"2024-05-08T18:07:53.868126+0000","last_undegraded":"2024-05-08T18:07:53.868126+0000","last_fullsized":"2024-05-08T18:07:53.868126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"29'6","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.002486+0000","last_change":"2024-05-08T18:04:45.814276+0000","last_active":"2024-05-08T18:07:54.002486+0000","last_peered":"2024-05-08T18:07:54.002486+0000","last_clean":"2024-05-08T18:07:54.002486+0000","last_became_active":"2024-05-08T18:04:45.813735+0000","last_became_peered":"2024-05-08T18:04:45.813735+0000","last_unstale":"2024-05-08T18:07:54.002486+0000","last_undegraded":"2024-05-08T18:07:54.002486+0000","last_fullsized":"2024-05-08T18:07:54.002486+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"27'4","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.745008+0000","last_change":"2024-05-08T18:04:43.848140+0000","last_active":"2024-05-08T18:06:58.745008+0000","last_peered":"2024-05-08T18:06:58.745008+0000","last_clean":"2024-05-08T18:06:58.745008+0000","last_became_active":"2024-05-08T18:04:43.839047+0000","last_became_peered":"2024-05-08T18:04:43.839047+0000","last_unstale":"2024-05-08T18:06:58.745008+0000","last_undegraded":"2024-05-08T18:06:58.745008+0000","last_fullsized":"2024-05-08T18:06:58.745008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.739730+0000","last_change":"2024-05-08T18:04:47.819319+0000","last_active":"2024-05-08T18:06:58.739730+0000","last_peered":"2024-05-08T18:06:58.739730+0000","last_clean":"2024-05-08T18:06:58.739730+0000","last_became_active":"2024-05-08T18:04:47.818972+0000","last_became_peered":"2024-05-08T18:04:47.818972+0000","last_unstale":"2024-05-08T18:06:58.739730+0000","last_undegraded":"2024-05-08T18:06:58.739730+0000","last_fullsized":"2024-05-08T18:06:58.739730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"29'63","reported_seq":136,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.861664+0000","last_change":"2024-05-08T18:08:12.861715+0000","last_active":"2024-05-08T18:08:12.861664+0000","last_peered":"2024-05-08T18:08:12.861664+0000","last_clean":"2024-05-08T18:08:12.861664+0000","last_became_active":"2024-05-08T18:04:49.830424+0000","last_became_peered":"2024-05-08T18:04:49.830424+0000","last_unstale":"2024-05-08T18:08:12.861664+0000","last_undegraded":"2024-05-08T18:08:12.861664+0000","last_fullsized":"2024-05-08T18:08:12.861664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'63","last_scrub_stamp":"2024-05-08T18:08:12.861577+0000","last_deep_scrub":"29'63","last_deep_scrub_stamp":"2024-05-08T18:08:12.861577+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.861577+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:02.260578+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.479280+0000","last_change":"2024-05-08T18:08:10.479304+0000","last_active":"2024-05-08T18:08:10.479280+0000","last_peered":"2024-05-08T18:08:10.479280+0000","last_clean":"2024-05-08T18:08:10.479280+0000","last_became_active":"2024-05-08T18:04:51.830383+0000","last_became_peered":"2024-05-08T18:04:51.830383+0000","last_unstale":"2024-05-08T18:08:10.479280+0000","last_undegraded":"2024-05-08T18:08:10.479280+0000","last_fullsized":"2024-05-08T18:08:10.479280+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:10.479187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:10.479187+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.479187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:04.216069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"29'20","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.161729+0000","last_change":"2024-05-08T18:04:53.848105+0000","last_active":"2024-05-08T18:07:54.161729+0000","last_peered":"2024-05-08T18:07:54.161729+0000","last_clean":"2024-05-08T18:07:54.161729+0000","last_became_active":"2024-05-08T18:04:53.847842+0000","last_became_peered":"2024-05-08T18:04:53.847842+0000","last_unstale":"2024-05-08T18:07:54.161729+0000","last_undegraded":"2024-05-08T18:07:54.161729+0000","last_fullsized":"2024-05-08T18:07:54.161729+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":48,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.511443+0000","last_change":"2024-05-08T18:08:23.511472+0000","last_active":"2024-05-08T18:08:23.511443+0000","last_peered":"2024-05-08T18:08:23.511443+0000","last_clean":"2024-05-08T18:08:23.511443+0000","last_became_active":"2024-05-08T18:04:45.821412+0000","last_became_peered":"2024-05-08T18:04:45.821412+0000","last_unstale":"2024-05-08T18:08:23.511443+0000","last_undegraded":"2024-05-08T18:08:23.511443+0000","last_fullsized":"2024-05-08T18:08:23.511443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:23.511346+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:23.511346+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.511346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:04.193921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.739776+0000","last_change":"2024-05-08T18:04:43.835922+0000","last_active":"2024-05-08T18:06:58.739776+0000","last_peered":"2024-05-08T18:06:58.739776+0000","last_clean":"2024-05-08T18:06:58.739776+0000","last_became_active":"2024-05-08T18:04:43.834723+0000","last_became_peered":"2024-05-08T18:04:43.834723+0000","last_unstale":"2024-05-08T18:06:58.739776+0000","last_undegraded":"2024-05-08T18:06:58.739776+0000","last_fullsized":"2024-05-08T18:06:58.739776+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.295215+0000","last_change":"2024-05-08T18:08:22.295265+0000","last_active":"2024-05-08T18:08:22.295215+0000","last_peered":"2024-05-08T18:08:22.295215+0000","last_clean":"2024-05-08T18:08:22.295215+0000","last_became_active":"2024-05-08T18:04:47.822865+0000","last_became_peered":"2024-05-08T18:04:47.822865+0000","last_unstale":"2024-05-08T18:08:22.295215+0000","last_undegraded":"2024-05-08T18:08:22.295215+0000","last_fullsized":"2024-05-08T18:08:22.295215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.295113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.295113+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.295113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:31:07.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"29'40","reported_seq":96,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.481554+0000","last_change":"2024-05-08T18:08:11.481605+0000","last_active":"2024-05-08T18:08:11.481554+0000","last_peered":"2024-05-08T18:08:11.481554+0000","last_clean":"2024-05-08T18:08:11.481554+0000","last_became_active":"2024-05-08T18:04:49.824610+0000","last_became_peered":"2024-05-08T18:04:49.824610+0000","last_unstale":"2024-05-08T18:08:11.481554+0000","last_undegraded":"2024-05-08T18:08:11.481554+0000","last_fullsized":"2024-05-08T18:08:11.481554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'40","last_scrub_stamp":"2024-05-08T18:08:11.481461+0000","last_deep_scrub":"29'40","last_deep_scrub_stamp":"2024-05-08T18:08:11.481461+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.481461+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:15.729540+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.747008+0000","last_change":"2024-05-08T18:04:51.849355+0000","last_active":"2024-05-08T18:07:08.747008+0000","last_peered":"2024-05-08T18:07:08.747008+0000","last_clean":"2024-05-08T18:07:08.747008+0000","last_became_active":"2024-05-08T18:04:51.848793+0000","last_became_peered":"2024-05-08T18:04:51.848793+0000","last_unstale":"2024-05-08T18:07:08.747008+0000","last_undegraded":"2024-05-08T18:07:08.747008+0000","last_fullsized":"2024-05-08T18:07:08.747008+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"29'5","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.119708+0000","last_change":"2024-05-08T18:04:53.849242+0000","last_active":"2024-05-08T18:07:54.119708+0000","last_peered":"2024-05-08T18:07:54.119708+0000","last_clean":"2024-05-08T18:07:54.119708+0000","last_became_active":"2024-05-08T18:04:53.848945+0000","last_became_peered":"2024-05-08T18:04:53.848945+0000","last_unstale":"2024-05-08T18:07:54.119708+0000","last_undegraded":"2024-05-08T18:07:54.119708+0000","last_fullsized":"2024-05-08T18:07:54.119708+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"29'10","reported_seq":76,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.782369+0000","last_change":"2024-05-08T18:04:45.829691+0000","last_active":"2024-05-08T18:07:53.782369+0000","last_peered":"2024-05-08T18:07:53.782369+0000","last_clean":"2024-05-08T18:07:53.782369+0000","last_became_active":"2024-05-08T18:04:45.829120+0000","last_became_peered":"2024-05-08T18:04:45.829120+0000","last_unstale":"2024-05-08T18:07:53.782369+0000","last_undegraded":"2024-05-08T18:07:53.782369+0000","last_fullsized":"2024-05-08T18:07:53.782369+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":57,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744173+0000","last_change":"2024-05-08T18:04:43.835329+0000","last_active":"2024-05-08T18:06:53.744173+0000","last_peered":"2024-05-08T18:06:53.744173+0000","last_clean":"2024-05-08T18:06:53.744173+0000","last_became_active":"2024-05-08T18:04:43.834536+0000","last_became_peered":"2024-05-08T18:04:43.834536+0000","last_unstale":"2024-05-08T18:06:53.744173+0000","last_undegraded":"2024-05-08T18:06:53.744173+0000","last_fullsized":"2024-05-08T18:06:53.744173+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"29'4","reported_seq":229,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.594171+0000","last_change":"2024-05-08T18:04:47.819789+0000","last_active":"2024-05-08T18:08:00.594171+0000","last_peered":"2024-05-08T18:08:00.594171+0000","last_clean":"2024-05-08T18:08:00.594171+0000","last_became_active":"2024-05-08T18:04:47.819085+0000","last_became_peered":"2024-05-08T18:04:47.819085+0000","last_unstale":"2024-05-08T18:08:00.594171+0000","last_undegraded":"2024-05-08T18:08:00.594171+0000","last_fullsized":"2024-05-08T18:08:00.594171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":301,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"29'45","reported_seq":103,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.280826+0000","last_change":"2024-05-08T18:08:16.280897+0000","last_active":"2024-05-08T18:08:16.280826+0000","last_peered":"2024-05-08T18:08:16.280826+0000","last_clean":"2024-05-08T18:08:16.280826+0000","last_became_active":"2024-05-08T18:04:49.825613+0000","last_became_peered":"2024-05-08T18:04:49.825613+0000","last_unstale":"2024-05-08T18:08:16.280826+0000","last_undegraded":"2024-05-08T18:08:16.280826+0000","last_fullsized":"2024-05-08T18:08:16.280826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'45","last_scrub_stamp":"2024-05-08T18:08:16.280719+0000","last_deep_scrub":"29'45","last_deep_scrub_stamp":"2024-05-08T18:08:16.280719+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.280719+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:29:19.130403+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"29'285","reported_seq":443,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.137858+0000","last_change":"2024-05-08T18:08:06.137883+0000","last_active":"2024-05-08T18:08:06.137858+0000","last_peered":"2024-05-08T18:08:06.137858+0000","last_clean":"2024-05-08T18:08:06.137858+0000","last_became_active":"2024-05-08T18:04:51.830924+0000","last_became_peered":"2024-05-08T18:04:51.830924+0000","last_unstale":"2024-05-08T18:08:06.137858+0000","last_undegraded":"2024-05-08T18:08:06.137858+0000","last_fullsized":"2024-05-08T18:08:06.137858+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:06.137767+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:06.137767+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.137767+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:43.622614+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"29'14","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.121335+0000","last_change":"2024-05-08T18:04:53.841349+0000","last_active":"2024-05-08T18:07:54.121335+0000","last_peered":"2024-05-08T18:07:54.121335+0000","last_clean":"2024-05-08T18:07:54.121335+0000","last_became_active":"2024-05-08T18:04:53.840708+0000","last_became_peered":"2024-05-08T18:04:53.840708+0000","last_unstale":"2024-05-08T18:07:54.121335+0000","last_undegraded":"2024-05-08T18:07:54.121335+0000","last_fullsized":"2024-05-08T18:07:54.121335+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"29'6","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.071043+0000","last_change":"2024-05-08T18:04:45.812281+0000","last_active":"2024-05-08T18:07:54.071043+0000","last_peered":"2024-05-08T18:07:54.071043+0000","last_clean":"2024-05-08T18:07:54.071043+0000","last_became_active":"2024-05-08T18:04:45.811926+0000","last_became_peered":"2024-05-08T18:04:45.811926+0000","last_unstale":"2024-05-08T18:07:54.071043+0000","last_undegraded":"2024-05-08T18:07:54.071043+0000","last_fullsized":"2024-05-08T18:07:54.071043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"29'2","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744143+0000","last_change":"2024-05-08T18:04:43.844418+0000","last_active":"2024-05-08T18:06:53.744143+0000","last_peered":"2024-05-08T18:06:53.744143+0000","last_clean":"2024-05-08T18:06:53.744143+0000","last_became_active":"2024-05-08T18:04:43.843934+0000","last_became_peered":"2024-05-08T18:04:43.843934+0000","last_unstale":"2024-05-08T18:06:53.744143+0000","last_undegraded":"2024-05-08T18:06:53.744143+0000","last_fullsized":"2024-05-08T18:06:53.744143+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"29'5","reported_seq":428,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.550034+0000","last_change":"2024-05-08T18:08:20.550071+0000","last_active":"2024-05-08T18:08:20.550034+0000","last_peered":"2024-05-08T18:08:20.550034+0000","last_clean":"2024-05-08T18:08:20.550034+0000","last_became_active":"2024-05-08T18:04:47.817851+0000","last_became_peered":"2024-05-08T18:04:47.817851+0000","last_unstale":"2024-05-08T18:08:20.550034+0000","last_undegraded":"2024-05-08T18:08:20.550034+0000","last_fullsized":"2024-05-08T18:08:20.550034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'5","last_scrub_stamp":"2024-05-08T18:08:20.549943+0000","last_deep_scrub":"29'5","last_deep_scrub_stamp":"2024-05-08T18:08:20.549943+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.549943+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:31.792211+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"29'15","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.800413+0000","last_change":"2024-05-08T18:08:18.800470+0000","last_active":"2024-05-08T18:08:18.800413+0000","last_peered":"2024-05-08T18:08:18.800413+0000","last_clean":"2024-05-08T18:08:18.800413+0000","last_became_active":"2024-05-08T18:04:49.830714+0000","last_became_peered":"2024-05-08T18:04:49.830714+0000","last_unstale":"2024-05-08T18:08:18.800413+0000","last_undegraded":"2024-05-08T18:08:18.800413+0000","last_fullsized":"2024-05-08T18:08:18.800413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:18.800324+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:18.800324+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.800324+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:41.821593+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"29'285","reported_seq":480,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.542263+0000","last_change":"2024-05-08T18:08:05.542295+0000","last_active":"2024-05-08T18:08:05.542263+0000","last_peered":"2024-05-08T18:08:05.542263+0000","last_clean":"2024-05-08T18:08:05.542263+0000","last_became_active":"2024-05-08T18:04:51.831508+0000","last_became_peered":"2024-05-08T18:04:51.831508+0000","last_unstale":"2024-05-08T18:08:05.542263+0000","last_undegraded":"2024-05-08T18:08:05.542263+0000","last_fullsized":"2024-05-08T18:08:05.542263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:05.542169+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:05.542169+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.542169+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:49:28.084669+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'4","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744222+0000","last_change":"2024-05-08T18:04:53.840524+0000","last_active":"2024-05-08T18:06:58.744222+0000","last_peered":"2024-05-08T18:06:58.744222+0000","last_clean":"2024-05-08T18:06:58.744222+0000","last_became_active":"2024-05-08T18:04:53.840215+0000","last_became_peered":"2024-05-08T18:04:53.840215+0000","last_unstale":"2024-05-08T18:06:58.744222+0000","last_undegraded":"2024-05-08T18:06:58.744222+0000","last_fullsized":"2024-05-08T18:06:58.744222+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'2","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745463+0000","last_change":"2024-05-08T18:04:45.819074+0000","last_active":"2024-05-08T18:07:03.745463+0000","last_peered":"2024-05-08T18:07:03.745463+0000","last_clean":"2024-05-08T18:07:03.745463+0000","last_became_active":"2024-05-08T18:04:45.818533+0000","last_became_peered":"2024-05-08T18:04:45.818533+0000","last_unstale":"2024-05-08T18:07:03.745463+0000","last_undegraded":"2024-05-08T18:07:03.745463+0000","last_fullsized":"2024-05-08T18:07:03.745463+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.669968+0000","last_change":"2024-05-08T18:04:43.848496+0000","last_active":"2024-05-08T18:08:00.669968+0000","last_peered":"2024-05-08T18:08:00.669968+0000","last_clean":"2024-05-08T18:08:00.669968+0000","last_became_active":"2024-05-08T18:04:43.846125+0000","last_became_peered":"2024-05-08T18:04:43.846125+0000","last_unstale":"2024-05-08T18:08:00.669968+0000","last_undegraded":"2024-05-08T18:08:00.669968+0000","last_fullsized":"2024-05-08T18:08:00.669968+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.895797+0000","last_change":"2024-05-08T18:08:05.895821+0000","last_active":"2024-05-08T18:08:05.895797+0000","last_peered":"2024-05-08T18:08:05.895797+0000","last_clean":"2024-05-08T18:08:05.895797+0000","last_became_active":"2024-05-08T18:04:47.822242+0000","last_became_peered":"2024-05-08T18:04:47.822242+0000","last_unstale":"2024-05-08T18:08:05.895797+0000","last_undegraded":"2024-05-08T18:08:05.895797+0000","last_fullsized":"2024-05-08T18:08:05.895797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:05.895706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:05.895706+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.895706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:00:55.969119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"29'48","reported_seq":107,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.545377+0000","last_change":"2024-05-08T18:08:08.545419+0000","last_active":"2024-05-08T18:08:08.545377+0000","last_peered":"2024-05-08T18:08:08.545377+0000","last_clean":"2024-05-08T18:08:08.545377+0000","last_became_active":"2024-05-08T18:04:49.825314+0000","last_became_peered":"2024-05-08T18:04:49.825314+0000","last_unstale":"2024-05-08T18:08:08.545377+0000","last_undegraded":"2024-05-08T18:08:08.545377+0000","last_fullsized":"2024-05-08T18:08:08.545377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'48","last_scrub_stamp":"2024-05-08T18:08:08.545269+0000","last_deep_scrub":"29'48","last_deep_scrub_stamp":"2024-05-08T18:08:08.545269+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.545269+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:19.079352+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.136687+0000","last_change":"2024-05-08T18:08:05.136723+0000","last_active":"2024-05-08T18:08:05.136687+0000","last_peered":"2024-05-08T18:08:05.136687+0000","last_clean":"2024-05-08T18:08:05.136687+0000","last_became_active":"2024-05-08T18:04:51.832808+0000","last_became_peered":"2024-05-08T18:04:51.832808+0000","last_unstale":"2024-05-08T18:08:05.136687+0000","last_undegraded":"2024-05-08T18:08:05.136687+0000","last_fullsized":"2024-05-08T18:08:05.136687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:05.136592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:05.136592+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.136592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:30.377482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"29'8","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.160000+0000","last_change":"2024-05-08T18:04:53.837583+0000","last_active":"2024-05-08T18:07:54.160000+0000","last_peered":"2024-05-08T18:07:54.160000+0000","last_clean":"2024-05-08T18:07:54.160000+0000","last_became_active":"2024-05-08T18:04:53.837292+0000","last_became_peered":"2024-05-08T18:04:53.837292+0000","last_unstale":"2024-05-08T18:07:54.160000+0000","last_undegraded":"2024-05-08T18:07:54.160000+0000","last_fullsized":"2024-05-08T18:07:54.160000+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'4","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.041724+0000","last_change":"2024-05-08T18:04:45.829568+0000","last_active":"2024-05-08T18:07:54.041724+0000","last_peered":"2024-05-08T18:07:54.041724+0000","last_clean":"2024-05-08T18:07:54.041724+0000","last_became_active":"2024-05-08T18:04:45.828674+0000","last_became_peered":"2024-05-08T18:04:45.828674+0000","last_unstale":"2024-05-08T18:07:54.041724+0000","last_undegraded":"2024-05-08T18:07:54.041724+0000","last_fullsized":"2024-05-08T18:07:54.041724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":138,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:38.741527+0000","last_change":"2024-05-08T18:04:35.552002+0000","last_active":"2024-05-08T18:06:38.741527+0000","last_peered":"2024-05-08T18:06:38.741527+0000","last_clean":"2024-05-08T18:06:38.741527+0000","last_became_active":"2024-05-08T18:04:35.551437+0000","last_became_peered":"2024-05-08T18:04:35.551437+0000","last_unstale":"2024-05-08T18:06:38.741527+0000","last_undegraded":"2024-05-08T18:06:38.741527+0000","last_fullsized":"2024-05-08T18:06:38.741527+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:43.742741+0000","last_change":"2024-05-08T18:04:35.552341+0000","last_active":"2024-05-08T18:06:43.742741+0000","last_peered":"2024-05-08T18:06:43.742741+0000","last_clean":"2024-05-08T18:06:43.742741+0000","last_became_active":"2024-05-08T18:04:35.552017+0000","last_became_peered":"2024-05-08T18:04:35.552017+0000","last_unstale":"2024-05-08T18:06:43.742741+0000","last_undegraded":"2024-05-08T18:06:43.742741+0000","last_fullsized":"2024-05-08T18:06:43.742741+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.743488+0000","last_change":"2024-05-08T18:04:43.837405+0000","last_active":"2024-05-08T18:06:48.743488+0000","last_peered":"2024-05-08T18:06:48.743488+0000","last_clean":"2024-05-08T18:06:48.743488+0000","last_became_active":"2024-05-08T18:04:43.837019+0000","last_became_peered":"2024-05-08T18:04:43.837019+0000","last_unstale":"2024-05-08T18:06:48.743488+0000","last_undegraded":"2024-05-08T18:06:48.743488+0000","last_fullsized":"2024-05-08T18:06:48.743488+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.743818+0000","last_change":"2024-05-08T18:05:30.871131+0000","last_active":"2024-05-08T18:06:53.743818+0000","last_peered":"2024-05-08T18:06:53.743818+0000","last_clean":"2024-05-08T18:06:53.743818+0000","last_became_active":"2024-05-08T18:05:30.870647+0000","last_became_peered":"2024-05-08T18:05:30.870647+0000","last_unstale":"2024-05-08T18:06:53.743818+0000","last_undegraded":"2024-05-08T18:06:53.743818+0000","last_fullsized":"2024-05-08T18:06:53.743818+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"29'86","reported_seq":168,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.712287+0000","last_change":"2024-05-08T18:04:49.832374+0000","last_active":"2024-05-08T18:08:00.712287+0000","last_peered":"2024-05-08T18:08:00.712287+0000","last_clean":"2024-05-08T18:08:00.712287+0000","last_became_active":"2024-05-08T18:04:49.830636+0000","last_became_peered":"2024-05-08T18:04:49.830636+0000","last_unstale":"2024-05-08T18:08:00.712287+0000","last_undegraded":"2024-05-08T18:08:00.712287+0000","last_fullsized":"2024-05-08T18:08:00.712287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.600189+0000","last_change":"2024-05-08T18:08:21.600230+0000","last_active":"2024-05-08T18:08:21.600189+0000","last_peered":"2024-05-08T18:08:21.600189+0000","last_clean":"2024-05-08T18:08:21.600189+0000","last_became_active":"2024-05-08T18:04:51.844737+0000","last_became_peered":"2024-05-08T18:04:51.844737+0000","last_unstale":"2024-05-08T18:08:21.600189+0000","last_undegraded":"2024-05-08T18:08:21.600189+0000","last_fullsized":"2024-05-08T18:08:21.600189+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:21.600069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:21.600069+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.600069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:48.902927+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"29'85","reported_seq":167,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.171802+0000","last_change":"2024-05-08T18:04:53.839334+0000","last_active":"2024-05-08T18:07:54.171802+0000","last_peered":"2024-05-08T18:07:54.171802+0000","last_clean":"2024-05-08T18:07:54.171802+0000","last_became_active":"2024-05-08T18:04:53.838957+0000","last_became_peered":"2024-05-08T18:04:53.838957+0000","last_unstale":"2024-05-08T18:07:54.171802+0000","last_undegraded":"2024-05-08T18:07:54.171802+0000","last_fullsized":"2024-05-08T18:07:54.171802+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":216,"num_read_kb":214,"num_write":167,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"29'9","reported_seq":66,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.115990+0000","last_change":"2024-05-08T18:04:45.826087+0000","last_active":"2024-05-08T18:07:54.115990+0000","last_peered":"2024-05-08T18:07:54.115990+0000","last_clean":"2024-05-08T18:07:54.115990+0000","last_became_active":"2024-05-08T18:04:45.825843+0000","last_became_peered":"2024-05-08T18:04:45.825843+0000","last_unstale":"2024-05-08T18:07:54.115990+0000","last_undegraded":"2024-05-08T18:07:54.115990+0000","last_fullsized":"2024-05-08T18:07:54.115990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:38.741917+0000","last_change":"2024-05-08T18:04:35.552553+0000","last_active":"2024-05-08T18:06:38.741917+0000","last_peered":"2024-05-08T18:06:38.741917+0000","last_clean":"2024-05-08T18:06:38.741917+0000","last_became_active":"2024-05-08T18:04:35.552214+0000","last_became_peered":"2024-05-08T18:04:35.552214+0000","last_unstale":"2024-05-08T18:06:38.741917+0000","last_undegraded":"2024-05-08T18:06:38.741917+0000","last_fullsized":"2024-05-08T18:06:38.741917+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.738105+0000","last_change":"2024-05-08T18:04:43.821058+0000","last_active":"2024-05-08T18:06:48.738105+0000","last_peered":"2024-05-08T18:06:48.738105+0000","last_clean":"2024-05-08T18:06:48.738105+0000","last_became_active":"2024-05-08T18:04:43.820369+0000","last_became_peered":"2024-05-08T18:04:43.820369+0000","last_unstale":"2024-05-08T18:06:48.738105+0000","last_undegraded":"2024-05-08T18:06:48.738105+0000","last_fullsized":"2024-05-08T18:06:48.738105+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.745186+0000","last_change":"2024-05-08T18:05:30.870035+0000","last_active":"2024-05-08T18:06:58.745186+0000","last_peered":"2024-05-08T18:06:58.745186+0000","last_clean":"2024-05-08T18:06:58.745186+0000","last_became_active":"2024-05-08T18:05:30.869578+0000","last_became_peered":"2024-05-08T18:05:30.869578+0000","last_unstale":"2024-05-08T18:06:58.745186+0000","last_undegraded":"2024-05-08T18:06:58.745186+0000","last_fullsized":"2024-05-08T18:06:58.745186+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"29'290","reported_seq":515,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.566274+0000","last_change":"2024-05-08T18:08:07.566330+0000","last_active":"2024-05-08T18:08:07.566274+0000","last_peered":"2024-05-08T18:08:07.566274+0000","last_clean":"2024-05-08T18:08:07.566274+0000","last_became_active":"2024-05-08T18:04:49.833053+0000","last_became_peered":"2024-05-08T18:04:49.833053+0000","last_unstale":"2024-05-08T18:08:07.566274+0000","last_undegraded":"2024-05-08T18:08:07.566274+0000","last_fullsized":"2024-05-08T18:08:07.566274+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'290","last_scrub_stamp":"2024-05-08T18:08:07.566185+0000","last_deep_scrub":"29'290","last_deep_scrub_stamp":"2024-05-08T18:08:07.566185+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.566185+0000","objects_scrubbed":10,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:24.566210+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":378,"num_read_kb":363,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.745057+0000","last_change":"2024-05-08T18:04:51.849383+0000","last_active":"2024-05-08T18:06:58.745057+0000","last_peered":"2024-05-08T18:06:58.745057+0000","last_clean":"2024-05-08T18:06:58.745057+0000","last_became_active":"2024-05-08T18:04:51.848066+0000","last_became_peered":"2024-05-08T18:04:51.848066+0000","last_unstale":"2024-05-08T18:06:58.745057+0000","last_undegraded":"2024-05-08T18:06:58.745057+0000","last_fullsized":"2024-05-08T18:06:58.745057+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"29'3","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745324+0000","last_change":"2024-05-08T18:04:53.843187+0000","last_active":"2024-05-08T18:07:03.745324+0000","last_peered":"2024-05-08T18:07:03.745324+0000","last_clean":"2024-05-08T18:07:03.745324+0000","last_became_active":"2024-05-08T18:04:53.842899+0000","last_became_peered":"2024-05-08T18:04:53.842899+0000","last_unstale":"2024-05-08T18:07:03.745324+0000","last_undegraded":"2024-05-08T18:07:03.745324+0000","last_fullsized":"2024-05-08T18:07:03.745324+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"29'8","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070059+0000","last_change":"2024-05-08T18:04:45.812417+0000","last_active":"2024-05-08T18:07:54.070059+0000","last_peered":"2024-05-08T18:07:54.070059+0000","last_clean":"2024-05-08T18:07:54.070059+0000","last_became_active":"2024-05-08T18:04:45.812091+0000","last_became_peered":"2024-05-08T18:04:45.812091+0000","last_unstale":"2024-05-08T18:07:54.070059+0000","last_undegraded":"2024-05-08T18:07:54.070059+0000","last_fullsized":"2024-05-08T18:07:54.070059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:38.741097+0000","last_change":"2024-05-08T18:04:35.551648+0000","last_active":"2024-05-08T18:06:38.741097+0000","last_peered":"2024-05-08T18:06:38.741097+0000","last_clean":"2024-05-08T18:06:38.741097+0000","last_became_active":"2024-05-08T18:04:35.551313+0000","last_became_peered":"2024-05-08T18:04:35.551313+0000","last_unstale":"2024-05-08T18:06:38.741097+0000","last_undegraded":"2024-05-08T18:06:38.741097+0000","last_fullsized":"2024-05-08T18:06:38.741097+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:43.737190+0000","last_change":"2024-05-08T18:04:43.821180+0000","last_active":"2024-05-08T18:06:43.737190+0000","last_peered":"2024-05-08T18:06:43.737190+0000","last_clean":"2024-05-08T18:06:43.737190+0000","last_became_active":"2024-05-08T18:04:43.820335+0000","last_became_peered":"2024-05-08T18:04:43.820335+0000","last_unstale":"2024-05-08T18:06:43.737190+0000","last_undegraded":"2024-05-08T18:06:43.737190+0000","last_fullsized":"2024-05-08T18:06:43.737190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"29'2","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.343008+0000","last_change":"2024-05-08T18:08:20.343043+0000","last_active":"2024-05-08T18:08:20.343008+0000","last_peered":"2024-05-08T18:08:20.343008+0000","last_clean":"2024-05-08T18:08:20.343008+0000","last_became_active":"2024-05-08T18:04:47.823631+0000","last_became_peered":"2024-05-08T18:04:47.823631+0000","last_unstale":"2024-05-08T18:08:20.343008+0000","last_undegraded":"2024-05-08T18:08:20.343008+0000","last_fullsized":"2024-05-08T18:08:20.343008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:20.342916+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:20.342916+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.342916+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:26:02.075842+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"29'24","reported_seq":101,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.233358+0000","last_change":"2024-05-08T18:08:15.233395+0000","last_active":"2024-05-08T18:08:15.233358+0000","last_peered":"2024-05-08T18:08:15.233358+0000","last_clean":"2024-05-08T18:08:15.233358+0000","last_became_active":"2024-05-08T18:04:49.827735+0000","last_became_peered":"2024-05-08T18:04:49.827735+0000","last_unstale":"2024-05-08T18:08:15.233358+0000","last_undegraded":"2024-05-08T18:08:15.233358+0000","last_fullsized":"2024-05-08T18:08:15.233358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'24","last_scrub_stamp":"2024-05-08T18:08:15.233271+0000","last_deep_scrub":"29'24","last_deep_scrub_stamp":"2024-05-08T18:08:15.233271+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.233271+0000","objects_scrubbed":8,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:25.767355+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.442651+0000","last_change":"2024-05-08T18:08:09.442677+0000","last_active":"2024-05-08T18:08:09.442651+0000","last_peered":"2024-05-08T18:08:09.442651+0000","last_clean":"2024-05-08T18:08:09.442651+0000","last_became_active":"2024-05-08T18:04:51.843940+0000","last_became_peered":"2024-05-08T18:04:51.843940+0000","last_unstale":"2024-05-08T18:08:09.442651+0000","last_undegraded":"2024-05-08T18:08:09.442651+0000","last_fullsized":"2024-05-08T18:08:09.442651+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:09.442556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:09.442556+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.442556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:48:00.984508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"29'4","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.000331+0000","last_change":"2024-05-08T18:04:53.848676+0000","last_active":"2024-05-08T18:07:54.000331+0000","last_peered":"2024-05-08T18:07:54.000331+0000","last_clean":"2024-05-08T18:07:54.000331+0000","last_became_active":"2024-05-08T18:04:53.848255+0000","last_became_peered":"2024-05-08T18:04:53.848255+0000","last_unstale":"2024-05-08T18:07:54.000331+0000","last_undegraded":"2024-05-08T18:07:54.000331+0000","last_fullsized":"2024-05-08T18:07:54.000331+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"29'10","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070249+0000","last_change":"2024-05-08T18:04:45.807470+0000","last_active":"2024-05-08T18:07:54.070249+0000","last_peered":"2024-05-08T18:07:54.070249+0000","last_clean":"2024-05-08T18:07:54.070249+0000","last_became_active":"2024-05-08T18:04:45.807085+0000","last_became_peered":"2024-05-08T18:04:45.807085+0000","last_unstale":"2024-05-08T18:07:54.070249+0000","last_undegraded":"2024-05-08T18:07:54.070249+0000","last_fullsized":"2024-05-08T18:07:54.070249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:43.742675+0000","last_change":"2024-05-08T18:04:35.551456+0000","last_active":"2024-05-08T18:06:43.742675+0000","last_peered":"2024-05-08T18:06:43.742675+0000","last_clean":"2024-05-08T18:06:43.742675+0000","last_became_active":"2024-05-08T18:04:35.551120+0000","last_became_peered":"2024-05-08T18:04:35.551120+0000","last_unstale":"2024-05-08T18:06:43.742675+0000","last_undegraded":"2024-05-08T18:06:43.742675+0000","last_fullsized":"2024-05-08T18:06:43.742675+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.742661+0000","last_change":"2024-05-08T18:04:43.840439+0000","last_active":"2024-05-08T18:06:48.742661+0000","last_peered":"2024-05-08T18:06:48.742661+0000","last_clean":"2024-05-08T18:06:48.742661+0000","last_became_active":"2024-05-08T18:04:43.840036+0000","last_became_peered":"2024-05-08T18:04:43.840036+0000","last_unstale":"2024-05-08T18:06:48.742661+0000","last_undegraded":"2024-05-08T18:06:48.742661+0000","last_fullsized":"2024-05-08T18:06:48.742661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0,3],"acting":[2,1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.607224+0000","last_change":"2024-05-08T18:08:20.607246+0000","last_active":"2024-05-08T18:08:20.607224+0000","last_peered":"2024-05-08T18:08:20.607224+0000","last_clean":"2024-05-08T18:08:20.607224+0000","last_became_active":"2024-05-08T18:04:47.823093+0000","last_became_peered":"2024-05-08T18:04:47.823093+0000","last_unstale":"2024-05-08T18:08:20.607224+0000","last_undegraded":"2024-05-08T18:08:20.607224+0000","last_fullsized":"2024-05-08T18:08:20.607224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:20.607136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:20.607136+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.607136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:01.289711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"29'13","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.428946+0000","last_change":"2024-05-08T18:04:49.832657+0000","last_active":"2024-05-08T18:07:53.428946+0000","last_peered":"2024-05-08T18:07:53.428946+0000","last_clean":"2024-05-08T18:07:53.428946+0000","last_became_active":"2024-05-08T18:04:49.830586+0000","last_became_peered":"2024-05-08T18:04:49.830586+0000","last_unstale":"2024-05-08T18:07:53.428946+0000","last_undegraded":"2024-05-08T18:07:53.428946+0000","last_fullsized":"2024-05-08T18:07:53.428946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.746127+0000","last_change":"2024-05-08T18:04:51.849539+0000","last_active":"2024-05-08T18:07:03.746127+0000","last_peered":"2024-05-08T18:07:03.746127+0000","last_clean":"2024-05-08T18:07:03.746127+0000","last_became_active":"2024-05-08T18:04:51.849137+0000","last_became_peered":"2024-05-08T18:04:51.849137+0000","last_unstale":"2024-05-08T18:07:03.746127+0000","last_undegraded":"2024-05-08T18:07:03.746127+0000","last_fullsized":"2024-05-08T18:07:03.746127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'4","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745230+0000","last_change":"2024-05-08T18:04:53.851345+0000","last_active":"2024-05-08T18:07:03.745230+0000","last_peered":"2024-05-08T18:07:03.745230+0000","last_clean":"2024-05-08T18:07:03.745230+0000","last_became_active":"2024-05-08T18:04:53.851008+0000","last_became_peered":"2024-05-08T18:04:53.851008+0000","last_unstale":"2024-05-08T18:07:03.745230+0000","last_undegraded":"2024-05-08T18:07:03.745230+0000","last_fullsized":"2024-05-08T18:07:03.745230+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'2","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.743751+0000","last_change":"2024-05-08T18:04:45.820976+0000","last_active":"2024-05-08T18:06:53.743751+0000","last_peered":"2024-05-08T18:06:53.743751+0000","last_clean":"2024-05-08T18:06:53.743751+0000","last_became_active":"2024-05-08T18:04:45.819326+0000","last_became_peered":"2024-05-08T18:04:45.819326+0000","last_unstale":"2024-05-08T18:06:53.743751+0000","last_undegraded":"2024-05-08T18:06:53.743751+0000","last_fullsized":"2024-05-08T18:06:53.743751+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.742690+0000","last_change":"2024-05-08T18:04:43.840661+0000","last_active":"2024-05-08T18:06:48.742690+0000","last_peered":"2024-05-08T18:06:48.742690+0000","last_clean":"2024-05-08T18:06:48.742690+0000","last_became_active":"2024-05-08T18:04:43.840236+0000","last_became_peered":"2024-05-08T18:04:43.840236+0000","last_unstale":"2024-05-08T18:06:48.742690+0000","last_undegraded":"2024-05-08T18:06:48.742690+0000","last_fullsized":"2024-05-08T18:06:48.742690+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:43.737265+0000","last_change":"2024-05-08T18:04:35.552512+0000","last_active":"2024-05-08T18:06:43.737265+0000","last_peered":"2024-05-08T18:06:43.737265+0000","last_clean":"2024-05-08T18:06:43.737265+0000","last_became_active":"2024-05-08T18:04:35.552173+0000","last_became_peered":"2024-05-08T18:04:35.552173+0000","last_unstale":"2024-05-08T18:06:43.737265+0000","last_undegraded":"2024-05-08T18:06:43.737265+0000","last_fullsized":"2024-05-08T18:06:43.737265+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"29'1","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.606233+0000","last_change":"2024-05-08T18:04:47.820423+0000","last_active":"2024-05-08T18:08:00.606233+0000","last_peered":"2024-05-08T18:08:00.606233+0000","last_clean":"2024-05-08T18:08:00.606233+0000","last_became_active":"2024-05-08T18:04:47.819910+0000","last_became_peered":"2024-05-08T18:04:47.819910+0000","last_unstale":"2024-05-08T18:08:00.606233+0000","last_undegraded":"2024-05-08T18:08:00.606233+0000","last_fullsized":"2024-05-08T18:08:00.606233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"29'73","reported_seq":149,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.598709+0000","last_change":"2024-05-08T18:08:15.598754+0000","last_active":"2024-05-08T18:08:15.598709+0000","last_peered":"2024-05-08T18:08:15.598709+0000","last_clean":"2024-05-08T18:08:15.598709+0000","last_became_active":"2024-05-08T18:04:49.831360+0000","last_became_peered":"2024-05-08T18:04:49.831360+0000","last_unstale":"2024-05-08T18:08:15.598709+0000","last_undegraded":"2024-05-08T18:08:15.598709+0000","last_fullsized":"2024-05-08T18:08:15.598709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'73","last_scrub_stamp":"2024-05-08T18:08:15.598616+0000","last_deep_scrub":"29'73","last_deep_scrub_stamp":"2024-05-08T18:08:15.598616+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.598616+0000","objects_scrubbed":9,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:23:01.195330+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":336,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.250177+0000","last_change":"2024-05-08T18:08:13.250201+0000","last_active":"2024-05-08T18:08:13.250177+0000","last_peered":"2024-05-08T18:08:13.250177+0000","last_clean":"2024-05-08T18:08:13.250177+0000","last_became_active":"2024-05-08T18:04:51.836441+0000","last_became_peered":"2024-05-08T18:04:51.836441+0000","last_unstale":"2024-05-08T18:08:13.250177+0000","last_undegraded":"2024-05-08T18:08:13.250177+0000","last_fullsized":"2024-05-08T18:08:13.250177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:13.250078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:13.250078+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.250078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:13.374535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"29'9","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.741320+0000","last_change":"2024-05-08T18:04:53.838192+0000","last_active":"2024-05-08T18:07:08.741320+0000","last_peered":"2024-05-08T18:07:08.741320+0000","last_clean":"2024-05-08T18:07:08.741320+0000","last_became_active":"2024-05-08T18:04:53.837920+0000","last_became_peered":"2024-05-08T18:04:53.837920+0000","last_unstale":"2024-05-08T18:07:08.741320+0000","last_undegraded":"2024-05-08T18:07:08.741320+0000","last_fullsized":"2024-05-08T18:07:08.741320+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"29'15","reported_seq":78,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.116334+0000","last_change":"2024-05-08T18:04:45.821636+0000","last_active":"2024-05-08T18:07:54.116334+0000","last_peered":"2024-05-08T18:07:54.116334+0000","last_clean":"2024-05-08T18:07:54.116334+0000","last_became_active":"2024-05-08T18:04:45.821107+0000","last_became_peered":"2024-05-08T18:04:45.821107+0000","last_unstale":"2024-05-08T18:07:54.116334+0000","last_undegraded":"2024-05-08T18:07:54.116334+0000","last_fullsized":"2024-05-08T18:07:54.116334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.738166+0000","last_change":"2024-05-08T18:04:43.828309+0000","last_active":"2024-05-08T18:06:48.738166+0000","last_peered":"2024-05-08T18:06:48.738166+0000","last_clean":"2024-05-08T18:06:48.738166+0000","last_became_active":"2024-05-08T18:04:43.827920+0000","last_became_peered":"2024-05-08T18:04:43.827920+0000","last_unstale":"2024-05-08T18:06:48.738166+0000","last_undegraded":"2024-05-08T18:06:48.738166+0000","last_fullsized":"2024-05-08T18:06:48.738166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:43.742777+0000","last_change":"2024-05-08T18:04:35.551253+0000","last_active":"2024-05-08T18:06:43.742777+0000","last_peered":"2024-05-08T18:06:43.742777+0000","last_clean":"2024-05-08T18:06:43.742777+0000","last_became_active":"2024-05-08T18:04:35.550912+0000","last_became_peered":"2024-05-08T18:04:35.550912+0000","last_unstale":"2024-05-08T18:06:43.742777+0000","last_undegraded":"2024-05-08T18:06:43.742777+0000","last_fullsized":"2024-05-08T18:06:43.742777+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.811050+0000","last_change":"2024-05-08T18:08:15.811076+0000","last_active":"2024-05-08T18:08:15.811050+0000","last_peered":"2024-05-08T18:08:15.811050+0000","last_clean":"2024-05-08T18:08:15.811050+0000","last_became_active":"2024-05-08T18:04:47.813717+0000","last_became_peered":"2024-05-08T18:04:47.813717+0000","last_unstale":"2024-05-08T18:08:15.811050+0000","last_undegraded":"2024-05-08T18:08:15.811050+0000","last_fullsized":"2024-05-08T18:08:15.811050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:15.810959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:15.810959+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.810959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:35.598153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"29'21","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.551930+0000","last_change":"2024-05-08T18:08:06.551971+0000","last_active":"2024-05-08T18:08:06.551930+0000","last_peered":"2024-05-08T18:08:06.551930+0000","last_clean":"2024-05-08T18:08:06.551930+0000","last_became_active":"2024-05-08T18:04:49.835451+0000","last_became_peered":"2024-05-08T18:04:49.835451+0000","last_unstale":"2024-05-08T18:08:06.551930+0000","last_undegraded":"2024-05-08T18:08:06.551930+0000","last_fullsized":"2024-05-08T18:08:06.551930+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'21","last_scrub_stamp":"2024-05-08T18:08:06.551844+0000","last_deep_scrub":"29'21","last_deep_scrub_stamp":"2024-05-08T18:08:06.551844+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.551844+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:05.040309+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"29'285","reported_seq":381,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:04.536879+0000","last_change":"2024-05-08T18:08:04.536905+0000","last_active":"2024-05-08T18:08:04.536879+0000","last_peered":"2024-05-08T18:08:04.536879+0000","last_clean":"2024-05-08T18:08:04.536879+0000","last_became_active":"2024-05-08T18:04:51.832109+0000","last_became_peered":"2024-05-08T18:04:51.832109+0000","last_unstale":"2024-05-08T18:08:04.536879+0000","last_undegraded":"2024-05-08T18:08:04.536879+0000","last_fullsized":"2024-05-08T18:08:04.536879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:04.536786+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:04.536786+0000","last_clean_scrub_stamp":"2024-05-08T18:08:04.536786+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:42.101932+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746998+0000","last_change":"2024-05-08T18:04:53.844572+0000","last_active":"2024-05-08T18:07:08.746998+0000","last_peered":"2024-05-08T18:07:08.746998+0000","last_clean":"2024-05-08T18:07:08.746998+0000","last_became_active":"2024-05-08T18:04:53.842998+0000","last_became_peered":"2024-05-08T18:04:53.842998+0000","last_unstale":"2024-05-08T18:07:08.746998+0000","last_undegraded":"2024-05-08T18:07:08.746998+0000","last_fullsized":"2024-05-08T18:07:08.746998+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'4","reported_seq":72,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.838487+0000","last_change":"2024-05-08T18:08:19.838524+0000","last_active":"2024-05-08T18:08:19.838487+0000","last_peered":"2024-05-08T18:08:19.838487+0000","last_clean":"2024-05-08T18:08:19.838487+0000","last_became_active":"2024-05-08T18:04:45.829257+0000","last_became_peered":"2024-05-08T18:04:45.829257+0000","last_unstale":"2024-05-08T18:08:19.838487+0000","last_undegraded":"2024-05-08T18:08:19.838487+0000","last_fullsized":"2024-05-08T18:08:19.838487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:19.838313+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:19.838313+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.838313+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:43.545756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"29'3","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744271+0000","last_change":"2024-05-08T18:04:43.835453+0000","last_active":"2024-05-08T18:06:53.744271+0000","last_peered":"2024-05-08T18:06:53.744271+0000","last_clean":"2024-05-08T18:06:53.744271+0000","last_became_active":"2024-05-08T18:04:43.834610+0000","last_became_peered":"2024-05-08T18:04:43.834610+0000","last_unstale":"2024-05-08T18:06:53.744271+0000","last_undegraded":"2024-05-08T18:06:53.744271+0000","last_fullsized":"2024-05-08T18:06:53.744271+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:43.737303+0000","last_change":"2024-05-08T18:04:35.551595+0000","last_active":"2024-05-08T18:06:43.737303+0000","last_peered":"2024-05-08T18:06:43.737303+0000","last_clean":"2024-05-08T18:06:43.737303+0000","last_became_active":"2024-05-08T18:04:35.551366+0000","last_became_peered":"2024-05-08T18:04:35.551366+0000","last_unstale":"2024-05-08T18:06:43.737303+0000","last_undegraded":"2024-05-08T18:06:43.737303+0000","last_fullsized":"2024-05-08T18:06:43.737303+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.923238+0000","last_change":"2024-05-08T18:08:06.923265+0000","last_active":"2024-05-08T18:08:06.923238+0000","last_peered":"2024-05-08T18:08:06.923238+0000","last_clean":"2024-05-08T18:08:06.923238+0000","last_became_active":"2024-05-08T18:04:47.819866+0000","last_became_peered":"2024-05-08T18:04:47.819866+0000","last_unstale":"2024-05-08T18:08:06.923238+0000","last_undegraded":"2024-05-08T18:08:06.923238+0000","last_fullsized":"2024-05-08T18:08:06.923238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:06.923149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:06.923149+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.923149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:30.572799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"27'10","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744950+0000","last_change":"2024-05-08T18:04:49.832338+0000","last_active":"2024-05-08T18:06:58.744950+0000","last_peered":"2024-05-08T18:06:58.744950+0000","last_clean":"2024-05-08T18:06:58.744950+0000","last_became_active":"2024-05-08T18:04:49.830452+0000","last_became_peered":"2024-05-08T18:04:49.830452+0000","last_unstale":"2024-05-08T18:06:58.744950+0000","last_undegraded":"2024-05-08T18:06:58.744950+0000","last_fullsized":"2024-05-08T18:06:58.744950+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"29'285","reported_seq":411,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.471758+0000","last_change":"2024-05-08T18:08:06.471786+0000","last_active":"2024-05-08T18:08:06.471758+0000","last_peered":"2024-05-08T18:08:06.471758+0000","last_clean":"2024-05-08T18:08:06.471758+0000","last_became_active":"2024-05-08T18:04:51.831155+0000","last_became_peered":"2024-05-08T18:04:51.831155+0000","last_unstale":"2024-05-08T18:08:06.471758+0000","last_undegraded":"2024-05-08T18:08:06.471758+0000","last_fullsized":"2024-05-08T18:08:06.471758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:06.471664+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:06.471664+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.471664+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:29:09.321348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'5","reported_seq":24,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.068065+0000","last_change":"2024-05-08T18:04:53.849164+0000","last_active":"2024-05-08T18:07:54.068065+0000","last_peered":"2024-05-08T18:07:54.068065+0000","last_clean":"2024-05-08T18:07:54.068065+0000","last_became_active":"2024-05-08T18:04:53.848862+0000","last_became_peered":"2024-05-08T18:04:53.848862+0000","last_unstale":"2024-05-08T18:07:54.068065+0000","last_undegraded":"2024-05-08T18:07:54.068065+0000","last_fullsized":"2024-05-08T18:07:54.068065+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"29'6","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.867124+0000","last_change":"2024-05-08T18:04:45.823169+0000","last_active":"2024-05-08T18:07:53.867124+0000","last_peered":"2024-05-08T18:07:53.867124+0000","last_clean":"2024-05-08T18:07:53.867124+0000","last_became_active":"2024-05-08T18:04:45.822824+0000","last_became_peered":"2024-05-08T18:04:45.822824+0000","last_unstale":"2024-05-08T18:07:53.867124+0000","last_undegraded":"2024-05-08T18:07:53.867124+0000","last_fullsized":"2024-05-08T18:07:53.867124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.743851+0000","last_change":"2024-05-08T18:04:43.839633+0000","last_active":"2024-05-08T18:06:53.743851+0000","last_peered":"2024-05-08T18:06:53.743851+0000","last_clean":"2024-05-08T18:06:53.743851+0000","last_became_active":"2024-05-08T18:04:43.839240+0000","last_became_peered":"2024-05-08T18:04:43.839240+0000","last_unstale":"2024-05-08T18:06:53.743851+0000","last_undegraded":"2024-05-08T18:06:53.743851+0000","last_fullsized":"2024-05-08T18:06:53.743851+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.743591+0000","last_change":"2024-05-08T18:04:35.551765+0000","last_active":"2024-05-08T18:06:48.743591+0000","last_peered":"2024-05-08T18:06:48.743591+0000","last_clean":"2024-05-08T18:06:48.743591+0000","last_became_active":"2024-05-08T18:04:35.551412+0000","last_became_peered":"2024-05-08T18:04:35.551412+0000","last_unstale":"2024-05-08T18:06:48.743591+0000","last_undegraded":"2024-05-08T18:06:48.743591+0000","last_fullsized":"2024-05-08T18:06:48.743591+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.843811+0000","last_change":"2024-05-08T18:08:13.843881+0000","last_active":"2024-05-08T18:08:13.843811+0000","last_peered":"2024-05-08T18:08:13.843811+0000","last_clean":"2024-05-08T18:08:13.843811+0000","last_became_active":"2024-05-08T18:04:47.815947+0000","last_became_peered":"2024-05-08T18:04:47.815947+0000","last_unstale":"2024-05-08T18:08:13.843811+0000","last_undegraded":"2024-05-08T18:08:13.843811+0000","last_fullsized":"2024-05-08T18:08:13.843811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:13.843666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:13.843666+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.843666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:56.027223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"29'96","reported_seq":189,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.152986+0000","last_change":"2024-05-08T18:08:10.153072+0000","last_active":"2024-05-08T18:08:10.152986+0000","last_peered":"2024-05-08T18:08:10.152986+0000","last_clean":"2024-05-08T18:08:10.152986+0000","last_became_active":"2024-05-08T18:04:49.834586+0000","last_became_peered":"2024-05-08T18:04:49.834586+0000","last_unstale":"2024-05-08T18:08:10.152986+0000","last_undegraded":"2024-05-08T18:08:10.152986+0000","last_fullsized":"2024-05-08T18:08:10.152986+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'96","last_scrub_stamp":"2024-05-08T18:08:10.152888+0000","last_deep_scrub":"29'96","last_deep_scrub_stamp":"2024-05-08T18:08:10.152888+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.152888+0000","objects_scrubbed":8,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:16.511455+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.455879+0000","last_change":"2024-05-08T18:08:08.455904+0000","last_active":"2024-05-08T18:08:08.455879+0000","last_peered":"2024-05-08T18:08:08.455879+0000","last_clean":"2024-05-08T18:08:08.455879+0000","last_became_active":"2024-05-08T18:04:51.830760+0000","last_became_peered":"2024-05-08T18:04:51.830760+0000","last_unstale":"2024-05-08T18:08:08.455879+0000","last_undegraded":"2024-05-08T18:08:08.455879+0000","last_fullsized":"2024-05-08T18:08:08.455879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:08.455773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:08.455773+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.455773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:12.703852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.342682+0000","last_change":"2024-05-08T18:08:21.342717+0000","last_active":"2024-05-08T18:08:21.342682+0000","last_peered":"2024-05-08T18:08:21.342682+0000","last_clean":"2024-05-08T18:08:21.342682+0000","last_became_active":"2024-05-08T18:04:53.838600+0000","last_became_peered":"2024-05-08T18:04:53.838600+0000","last_unstale":"2024-05-08T18:08:21.342682+0000","last_undegraded":"2024-05-08T18:08:21.342682+0000","last_fullsized":"2024-05-08T18:08:21.342682+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:21.342590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:21.342590+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.342590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:44.919676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"27'2","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744383+0000","last_change":"2024-05-08T18:04:53.839793+0000","last_active":"2024-05-08T18:06:58.744383+0000","last_peered":"2024-05-08T18:06:58.744383+0000","last_clean":"2024-05-08T18:06:58.744383+0000","last_became_active":"2024-05-08T18:04:53.839500+0000","last_became_peered":"2024-05-08T18:04:53.839500+0000","last_unstale":"2024-05-08T18:06:58.744383+0000","last_undegraded":"2024-05-08T18:06:58.744383+0000","last_fullsized":"2024-05-08T18:06:58.744383+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"29'8","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746030+0000","last_change":"2024-05-08T18:04:45.819806+0000","last_active":"2024-05-08T18:07:08.746030+0000","last_peered":"2024-05-08T18:07:08.746030+0000","last_clean":"2024-05-08T18:07:08.746030+0000","last_became_active":"2024-05-08T18:04:45.819508+0000","last_became_peered":"2024-05-08T18:04:45.819508+0000","last_unstale":"2024-05-08T18:07:08.746030+0000","last_undegraded":"2024-05-08T18:07:08.746030+0000","last_fullsized":"2024-05-08T18:07:08.746030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.746021+0000","last_change":"2024-05-08T18:04:43.844864+0000","last_active":"2024-05-08T18:07:03.746021+0000","last_peered":"2024-05-08T18:07:03.746021+0000","last_clean":"2024-05-08T18:07:03.746021+0000","last_became_active":"2024-05-08T18:04:43.837929+0000","last_became_peered":"2024-05-08T18:04:43.837929+0000","last_unstale":"2024-05-08T18:07:03.746021+0000","last_undegraded":"2024-05-08T18:07:03.746021+0000","last_fullsized":"2024-05-08T18:07:03.746021+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.504440+0000","last_change":"2024-05-08T18:08:19.504467+0000","last_active":"2024-05-08T18:08:19.504440+0000","last_peered":"2024-05-08T18:08:19.504440+0000","last_clean":"2024-05-08T18:08:19.504440+0000","last_became_active":"2024-05-08T18:04:47.818443+0000","last_became_peered":"2024-05-08T18:04:47.818443+0000","last_unstale":"2024-05-08T18:08:19.504440+0000","last_undegraded":"2024-05-08T18:08:19.504440+0000","last_fullsized":"2024-05-08T18:08:19.504440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:19.504350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:19.504350+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.504350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:24.197683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"29'35","reported_seq":83,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.690456+0000","last_change":"2024-05-08T18:04:49.834233+0000","last_active":"2024-05-08T18:08:00.690456+0000","last_peered":"2024-05-08T18:08:00.690456+0000","last_clean":"2024-05-08T18:08:00.690456+0000","last_became_active":"2024-05-08T18:04:49.833599+0000","last_became_peered":"2024-05-08T18:04:49.833599+0000","last_unstale":"2024-05-08T18:08:00.690456+0000","last_undegraded":"2024-05-08T18:08:00.690456+0000","last_fullsized":"2024-05-08T18:08:00.690456+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747581+0000","last_change":"2024-05-08T18:04:51.844827+0000","last_active":"2024-05-08T18:07:13.747581+0000","last_peered":"2024-05-08T18:07:13.747581+0000","last_clean":"2024-05-08T18:07:13.747581+0000","last_became_active":"2024-05-08T18:04:51.844526+0000","last_became_peered":"2024-05-08T18:04:51.844526+0000","last_unstale":"2024-05-08T18:07:13.747581+0000","last_undegraded":"2024-05-08T18:07:13.747581+0000","last_fullsized":"2024-05-08T18:07:13.747581+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"29'11","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.739019+0000","last_change":"2024-05-08T18:04:53.850911+0000","last_active":"2024-05-08T18:06:53.739019+0000","last_peered":"2024-05-08T18:06:53.739019+0000","last_clean":"2024-05-08T18:06:53.739019+0000","last_became_active":"2024-05-08T18:04:53.850593+0000","last_became_peered":"2024-05-08T18:04:53.850593+0000","last_unstale":"2024-05-08T18:06:53.739019+0000","last_undegraded":"2024-05-08T18:06:53.739019+0000","last_fullsized":"2024-05-08T18:06:53.739019+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":62,"num_write":31,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"29'8","reported_seq":74,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746124+0000","last_change":"2024-05-08T18:04:45.819974+0000","last_active":"2024-05-08T18:07:08.746124+0000","last_peered":"2024-05-08T18:07:08.746124+0000","last_clean":"2024-05-08T18:07:08.746124+0000","last_became_active":"2024-05-08T18:04:45.818979+0000","last_became_peered":"2024-05-08T18:04:45.818979+0000","last_unstale":"2024-05-08T18:07:08.746124+0000","last_undegraded":"2024-05-08T18:07:08.746124+0000","last_fullsized":"2024-05-08T18:07:08.746124+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746094+0000","last_change":"2024-05-08T18:04:43.836993+0000","last_active":"2024-05-08T18:07:08.746094+0000","last_peered":"2024-05-08T18:07:08.746094+0000","last_clean":"2024-05-08T18:07:08.746094+0000","last_became_active":"2024-05-08T18:04:43.836614+0000","last_became_peered":"2024-05-08T18:04:43.836614+0000","last_unstale":"2024-05-08T18:07:08.746094+0000","last_undegraded":"2024-05-08T18:07:08.746094+0000","last_fullsized":"2024-05-08T18:07:08.746094+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.821717+0000","last_change":"2024-05-08T18:08:14.821740+0000","last_active":"2024-05-08T18:08:14.821717+0000","last_peered":"2024-05-08T18:08:14.821717+0000","last_clean":"2024-05-08T18:08:14.821717+0000","last_became_active":"2024-05-08T18:04:47.814561+0000","last_became_peered":"2024-05-08T18:04:47.814561+0000","last_unstale":"2024-05-08T18:08:14.821717+0000","last_undegraded":"2024-05-08T18:08:14.821717+0000","last_fullsized":"2024-05-08T18:08:14.821717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:14.821626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:14.821626+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.821626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:24.805865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"29'10","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.572538+0000","last_change":"2024-05-08T18:08:18.572575+0000","last_active":"2024-05-08T18:08:18.572538+0000","last_peered":"2024-05-08T18:08:18.572538+0000","last_clean":"2024-05-08T18:08:18.572538+0000","last_became_active":"2024-05-08T18:04:49.825347+0000","last_became_peered":"2024-05-08T18:04:49.825347+0000","last_unstale":"2024-05-08T18:08:18.572538+0000","last_undegraded":"2024-05-08T18:08:18.572538+0000","last_fullsized":"2024-05-08T18:08:18.572538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:18.572458+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:18.572458+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.572458+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:21.293654+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.319006+0000","last_change":"2024-05-08T18:08:17.319068+0000","last_active":"2024-05-08T18:08:17.319006+0000","last_peered":"2024-05-08T18:08:17.319006+0000","last_clean":"2024-05-08T18:08:17.319006+0000","last_became_active":"2024-05-08T18:04:51.843002+0000","last_became_peered":"2024-05-08T18:04:51.843002+0000","last_unstale":"2024-05-08T18:08:17.319006+0000","last_undegraded":"2024-05-08T18:08:17.319006+0000","last_fullsized":"2024-05-08T18:08:17.319006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:17.318901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:17.318901+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.318901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:07.004932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:23.743399+0000","last_change":"2024-05-08T18:04:53.840844+0000","last_active":"2024-05-08T18:07:23.743399+0000","last_peered":"2024-05-08T18:07:23.743399+0000","last_clean":"2024-05-08T18:07:23.743399+0000","last_became_active":"2024-05-08T18:04:53.840437+0000","last_became_peered":"2024-05-08T18:04:53.840437+0000","last_unstale":"2024-05-08T18:07:23.743399+0000","last_undegraded":"2024-05-08T18:07:23.743399+0000","last_fullsized":"2024-05-08T18:07:23.743399+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"29'8","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.867017+0000","last_change":"2024-05-08T18:05:30.868442+0000","last_active":"2024-05-08T18:07:53.867017+0000","last_peered":"2024-05-08T18:07:53.867017+0000","last_clean":"2024-05-08T18:07:53.867017+0000","last_became_active":"2024-05-08T18:05:30.868156+0000","last_became_peered":"2024-05-08T18:05:30.868156+0000","last_unstale":"2024-05-08T18:07:53.867017+0000","last_undegraded":"2024-05-08T18:07:53.867017+0000","last_fullsized":"2024-05-08T18:07:53.867017+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746159+0000","last_change":"2024-05-08T18:04:43.835856+0000","last_active":"2024-05-08T18:07:08.746159+0000","last_peered":"2024-05-08T18:07:08.746159+0000","last_clean":"2024-05-08T18:07:08.746159+0000","last_became_active":"2024-05-08T18:04:43.835435+0000","last_became_peered":"2024-05-08T18:04:43.835435+0000","last_unstale":"2024-05-08T18:07:08.746159+0000","last_undegraded":"2024-05-08T18:07:08.746159+0000","last_fullsized":"2024-05-08T18:07:08.746159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.385584+0000","last_change":"2024-05-08T18:08:19.385611+0000","last_active":"2024-05-08T18:08:19.385584+0000","last_peered":"2024-05-08T18:08:19.385584+0000","last_clean":"2024-05-08T18:08:19.385584+0000","last_became_active":"2024-05-08T18:04:47.824140+0000","last_became_peered":"2024-05-08T18:04:47.824140+0000","last_unstale":"2024-05-08T18:08:19.385584+0000","last_undegraded":"2024-05-08T18:08:19.385584+0000","last_fullsized":"2024-05-08T18:08:19.385584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:19.385463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:19.385463+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.385463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:15.297455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"29'48","reported_seq":120,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.582586+0000","last_change":"2024-05-08T18:08:19.582659+0000","last_active":"2024-05-08T18:08:19.582586+0000","last_peered":"2024-05-08T18:08:19.582586+0000","last_clean":"2024-05-08T18:08:19.582586+0000","last_became_active":"2024-05-08T18:04:49.824600+0000","last_became_peered":"2024-05-08T18:04:49.824600+0000","last_unstale":"2024-05-08T18:08:19.582586+0000","last_undegraded":"2024-05-08T18:08:19.582586+0000","last_fullsized":"2024-05-08T18:08:19.582586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'48","last_scrub_stamp":"2024-05-08T18:08:19.582484+0000","last_deep_scrub":"29'48","last_deep_scrub_stamp":"2024-05-08T18:08:19.582484+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.582484+0000","objects_scrubbed":6,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:25.159113+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":706,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.521105+0000","last_change":"2024-05-08T18:08:09.521130+0000","last_active":"2024-05-08T18:08:09.521105+0000","last_peered":"2024-05-08T18:08:09.521105+0000","last_clean":"2024-05-08T18:08:09.521105+0000","last_became_active":"2024-05-08T18:04:51.829937+0000","last_became_peered":"2024-05-08T18:04:51.829937+0000","last_unstale":"2024-05-08T18:08:09.521105+0000","last_undegraded":"2024-05-08T18:08:09.521105+0000","last_fullsized":"2024-05-08T18:08:09.521105+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:09.521013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:09.521013+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.521013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:29:12.370696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:23.749205+0000","last_change":"2024-05-08T18:04:53.842981+0000","last_active":"2024-05-08T18:07:23.749205+0000","last_peered":"2024-05-08T18:07:23.749205+0000","last_clean":"2024-05-08T18:07:23.749205+0000","last_became_active":"2024-05-08T18:04:53.842610+0000","last_became_peered":"2024-05-08T18:04:53.842610+0000","last_unstale":"2024-05-08T18:07:23.749205+0000","last_undegraded":"2024-05-08T18:07:23.749205+0000","last_fullsized":"2024-05-08T18:07:23.749205+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.866944+0000","last_change":"2024-05-08T18:08:22.866966+0000","last_active":"2024-05-08T18:08:22.866944+0000","last_peered":"2024-05-08T18:08:22.866944+0000","last_clean":"2024-05-08T18:08:22.866944+0000","last_became_active":"2024-05-08T18:04:45.829458+0000","last_became_peered":"2024-05-08T18:04:45.829458+0000","last_unstale":"2024-05-08T18:08:22.866944+0000","last_undegraded":"2024-05-08T18:08:22.866944+0000","last_fullsized":"2024-05-08T18:08:22.866944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.866853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.866853+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.866853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:05.403374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"29'4","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746252+0000","last_change":"2024-05-08T18:04:43.837089+0000","last_active":"2024-05-08T18:07:08.746252+0000","last_peered":"2024-05-08T18:07:08.746252+0000","last_clean":"2024-05-08T18:07:08.746252+0000","last_became_active":"2024-05-08T18:04:43.836633+0000","last_became_peered":"2024-05-08T18:04:43.836633+0000","last_unstale":"2024-05-08T18:07:08.746252+0000","last_undegraded":"2024-05-08T18:07:08.746252+0000","last_fullsized":"2024-05-08T18:07:08.746252+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"29'2","reported_seq":230,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.909561+0000","last_change":"2024-05-08T18:08:07.909590+0000","last_active":"2024-05-08T18:08:07.909561+0000","last_peered":"2024-05-08T18:08:07.909561+0000","last_clean":"2024-05-08T18:08:07.909561+0000","last_became_active":"2024-05-08T18:04:47.819771+0000","last_became_peered":"2024-05-08T18:04:47.819771+0000","last_unstale":"2024-05-08T18:08:07.909561+0000","last_undegraded":"2024-05-08T18:08:07.909561+0000","last_fullsized":"2024-05-08T18:08:07.909561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:07.909472+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:07.909472+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.909472+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:04.798235+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"29'36","reported_seq":91,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.344301+0000","last_change":"2024-05-08T18:08:18.344360+0000","last_active":"2024-05-08T18:08:18.344301+0000","last_peered":"2024-05-08T18:08:18.344301+0000","last_clean":"2024-05-08T18:08:18.344301+0000","last_became_active":"2024-05-08T18:04:49.824022+0000","last_became_peered":"2024-05-08T18:04:49.824022+0000","last_unstale":"2024-05-08T18:08:18.344301+0000","last_undegraded":"2024-05-08T18:08:18.344301+0000","last_fullsized":"2024-05-08T18:08:18.344301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:08:18.344202+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:08:18.344202+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.344202+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:36:47.464601+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.198536+0000","last_change":"2024-05-08T18:08:08.198575+0000","last_active":"2024-05-08T18:08:08.198536+0000","last_peered":"2024-05-08T18:08:08.198536+0000","last_clean":"2024-05-08T18:08:08.198536+0000","last_became_active":"2024-05-08T18:04:51.826424+0000","last_became_peered":"2024-05-08T18:04:51.826424+0000","last_unstale":"2024-05-08T18:08:08.198536+0000","last_undegraded":"2024-05-08T18:08:08.198536+0000","last_fullsized":"2024-05-08T18:08:08.198536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:08.198405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:08.198405+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.198405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:15.614149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"29'20","reported_seq":37,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.002699+0000","last_change":"2024-05-08T18:05:30.873315+0000","last_active":"2024-05-08T18:07:54.002699+0000","last_peered":"2024-05-08T18:07:54.002699+0000","last_clean":"2024-05-08T18:07:54.002699+0000","last_became_active":"2024-05-08T18:05:30.868409+0000","last_became_peered":"2024-05-08T18:05:30.868409+0000","last_unstale":"2024-05-08T18:07:54.002699+0000","last_undegraded":"2024-05-08T18:07:54.002699+0000","last_fullsized":"2024-05-08T18:07:54.002699+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746752+0000","last_change":"2024-05-08T18:04:43.844983+0000","last_active":"2024-05-08T18:07:08.746752+0000","last_peered":"2024-05-08T18:07:08.746752+0000","last_clean":"2024-05-08T18:07:08.746752+0000","last_became_active":"2024-05-08T18:04:43.838957+0000","last_became_peered":"2024-05-08T18:04:43.838957+0000","last_unstale":"2024-05-08T18:07:08.746752+0000","last_undegraded":"2024-05-08T18:07:08.746752+0000","last_fullsized":"2024-05-08T18:07:08.746752+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.606098+0000","last_change":"2024-05-08T18:08:23.606122+0000","last_active":"2024-05-08T18:08:23.606098+0000","last_peered":"2024-05-08T18:08:23.606098+0000","last_clean":"2024-05-08T18:08:23.606098+0000","last_became_active":"2024-05-08T18:04:47.820256+0000","last_became_peered":"2024-05-08T18:04:47.820256+0000","last_unstale":"2024-05-08T18:08:23.606098+0000","last_undegraded":"2024-05-08T18:08:23.606098+0000","last_fullsized":"2024-05-08T18:08:23.606098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:23.606002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:23.606002+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.606002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:04.849646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"29'63","reported_seq":136,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.582374+0000","last_change":"2024-05-08T18:08:16.582414+0000","last_active":"2024-05-08T18:08:16.582374+0000","last_peered":"2024-05-08T18:08:16.582374+0000","last_clean":"2024-05-08T18:08:16.582374+0000","last_became_active":"2024-05-08T18:04:49.825366+0000","last_became_peered":"2024-05-08T18:04:49.825366+0000","last_unstale":"2024-05-08T18:08:16.582374+0000","last_undegraded":"2024-05-08T18:08:16.582374+0000","last_fullsized":"2024-05-08T18:08:16.582374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'63","last_scrub_stamp":"2024-05-08T18:08:16.582295+0000","last_deep_scrub":"29'63","last_deep_scrub_stamp":"2024-05-08T18:08:16.582295+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.582295+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:21.275627+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.157727+0000","last_change":"2024-05-08T18:08:07.157775+0000","last_active":"2024-05-08T18:08:07.157727+0000","last_peered":"2024-05-08T18:08:07.157727+0000","last_clean":"2024-05-08T18:08:07.157727+0000","last_became_active":"2024-05-08T18:04:51.827127+0000","last_became_peered":"2024-05-08T18:04:51.827127+0000","last_unstale":"2024-05-08T18:08:07.157727+0000","last_undegraded":"2024-05-08T18:08:07.157727+0000","last_fullsized":"2024-05-08T18:08:07.157727+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:07.157606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:07.157606+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.157606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:37:53.105588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"29'14","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.741986+0000","last_change":"2024-05-08T18:04:53.844087+0000","last_active":"2024-05-08T18:07:13.741986+0000","last_peered":"2024-05-08T18:07:13.741986+0000","last_clean":"2024-05-08T18:07:13.741986+0000","last_became_active":"2024-05-08T18:04:53.843772+0000","last_became_peered":"2024-05-08T18:04:53.843772+0000","last_unstale":"2024-05-08T18:07:13.741986+0000","last_undegraded":"2024-05-08T18:07:13.741986+0000","last_fullsized":"2024-05-08T18:07:13.741986+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.115358+0000","last_change":"2024-05-08T18:04:45.811909+0000","last_active":"2024-05-08T18:07:54.115358+0000","last_peered":"2024-05-08T18:07:54.115358+0000","last_clean":"2024-05-08T18:07:54.115358+0000","last_became_active":"2024-05-08T18:04:45.811517+0000","last_became_peered":"2024-05-08T18:04:45.811517+0000","last_unstale":"2024-05-08T18:07:54.115358+0000","last_undegraded":"2024-05-08T18:07:54.115358+0000","last_fullsized":"2024-05-08T18:07:54.115358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746042+0000","last_change":"2024-05-08T18:04:43.836313+0000","last_active":"2024-05-08T18:07:08.746042+0000","last_peered":"2024-05-08T18:07:08.746042+0000","last_clean":"2024-05-08T18:07:08.746042+0000","last_became_active":"2024-05-08T18:04:43.834767+0000","last_became_peered":"2024-05-08T18:04:43.834767+0000","last_unstale":"2024-05-08T18:07:08.746042+0000","last_undegraded":"2024-05-08T18:07:08.746042+0000","last_fullsized":"2024-05-08T18:07:08.746042+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0,2],"acting":[1,3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.743557+0000","last_change":"2024-05-08T18:04:47.821012+0000","last_active":"2024-05-08T18:06:53.743557+0000","last_peered":"2024-05-08T18:06:53.743557+0000","last_clean":"2024-05-08T18:06:53.743557+0000","last_became_active":"2024-05-08T18:04:47.820662+0000","last_became_peered":"2024-05-08T18:04:47.820662+0000","last_unstale":"2024-05-08T18:06:53.743557+0000","last_undegraded":"2024-05-08T18:06:53.743557+0000","last_fullsized":"2024-05-08T18:06:53.743557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"29'46","reported_seq":104,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.901855+0000","last_change":"2024-05-08T18:08:10.901892+0000","last_active":"2024-05-08T18:08:10.901855+0000","last_peered":"2024-05-08T18:08:10.901855+0000","last_clean":"2024-05-08T18:08:10.901855+0000","last_became_active":"2024-05-08T18:04:49.830476+0000","last_became_peered":"2024-05-08T18:04:49.830476+0000","last_unstale":"2024-05-08T18:08:10.901855+0000","last_undegraded":"2024-05-08T18:08:10.901855+0000","last_fullsized":"2024-05-08T18:08:10.901855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'46","last_scrub_stamp":"2024-05-08T18:08:10.901767+0000","last_deep_scrub":"29'46","last_deep_scrub_stamp":"2024-05-08T18:08:10.901767+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.901767+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:07.184811+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.490196+0000","last_change":"2024-05-08T18:08:17.490220+0000","last_active":"2024-05-08T18:08:17.490196+0000","last_peered":"2024-05-08T18:08:17.490196+0000","last_clean":"2024-05-08T18:08:17.490196+0000","last_became_active":"2024-05-08T18:04:51.844912+0000","last_became_peered":"2024-05-08T18:04:51.844912+0000","last_unstale":"2024-05-08T18:08:17.490196+0000","last_undegraded":"2024-05-08T18:08:17.490196+0000","last_fullsized":"2024-05-08T18:08:17.490196+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:17.490110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:17.490110+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.490110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:06:17.806415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"29'2","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.043126+0000","last_change":"2024-05-08T18:04:53.847736+0000","last_active":"2024-05-08T18:07:54.043126+0000","last_peered":"2024-05-08T18:07:54.043126+0000","last_clean":"2024-05-08T18:07:54.043126+0000","last_became_active":"2024-05-08T18:04:53.847413+0000","last_became_peered":"2024-05-08T18:04:53.847413+0000","last_unstale":"2024-05-08T18:07:54.043126+0000","last_undegraded":"2024-05-08T18:07:54.043126+0000","last_fullsized":"2024-05-08T18:07:54.043126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"29'12","reported_seq":69,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.835658+0000","last_change":"2024-05-08T18:08:17.835711+0000","last_active":"2024-05-08T18:08:17.835658+0000","last_peered":"2024-05-08T18:08:17.835658+0000","last_clean":"2024-05-08T18:08:17.835658+0000","last_became_active":"2024-05-08T18:04:45.830329+0000","last_became_peered":"2024-05-08T18:04:45.830329+0000","last_unstale":"2024-05-08T18:08:17.835658+0000","last_undegraded":"2024-05-08T18:08:17.835658+0000","last_fullsized":"2024-05-08T18:08:17.835658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:17.835540+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:17.835540+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.835540+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:36.036099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.746942+0000","last_change":"2024-05-08T18:04:43.835662+0000","last_active":"2024-05-08T18:07:13.746942+0000","last_peered":"2024-05-08T18:07:13.746942+0000","last_clean":"2024-05-08T18:07:13.746942+0000","last_became_active":"2024-05-08T18:04:43.835227+0000","last_became_peered":"2024-05-08T18:04:43.835227+0000","last_unstale":"2024-05-08T18:07:13.746942+0000","last_undegraded":"2024-05-08T18:07:13.746942+0000","last_fullsized":"2024-05-08T18:07:13.746942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3,2],"acting":[1,0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.747677+0000","last_change":"2024-05-08T18:05:30.870636+0000","last_active":"2024-05-08T18:07:18.747677+0000","last_peered":"2024-05-08T18:07:18.747677+0000","last_clean":"2024-05-08T18:07:18.747677+0000","last_became_active":"2024-05-08T18:05:30.870155+0000","last_became_peered":"2024-05-08T18:05:30.870155+0000","last_unstale":"2024-05-08T18:07:18.747677+0000","last_undegraded":"2024-05-08T18:07:18.747677+0000","last_fullsized":"2024-05-08T18:07:18.747677+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"29'39","reported_seq":95,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.536504+0000","last_change":"2024-05-08T18:08:12.536541+0000","last_active":"2024-05-08T18:08:12.536504+0000","last_peered":"2024-05-08T18:08:12.536504+0000","last_clean":"2024-05-08T18:08:12.536504+0000","last_became_active":"2024-05-08T18:04:49.832649+0000","last_became_peered":"2024-05-08T18:04:49.832649+0000","last_unstale":"2024-05-08T18:08:12.536504+0000","last_undegraded":"2024-05-08T18:08:12.536504+0000","last_fullsized":"2024-05-08T18:08:12.536504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'39","last_scrub_stamp":"2024-05-08T18:08:12.536412+0000","last_deep_scrub":"29'39","last_deep_scrub_stamp":"2024-05-08T18:08:12.536412+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.536412+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:31.365352+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.444656+0000","last_change":"2024-05-08T18:08:05.444675+0000","last_active":"2024-05-08T18:08:05.444656+0000","last_peered":"2024-05-08T18:08:05.444656+0000","last_clean":"2024-05-08T18:08:05.444656+0000","last_became_active":"2024-05-08T18:04:51.833421+0000","last_became_peered":"2024-05-08T18:04:51.833421+0000","last_unstale":"2024-05-08T18:08:05.444656+0000","last_undegraded":"2024-05-08T18:08:05.444656+0000","last_fullsized":"2024-05-08T18:08:05.444656+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:05.444585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:05.444585+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.444585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:15.978668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"29'5","reported_seq":24,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747669+0000","last_change":"2024-05-08T18:04:53.848762+0000","last_active":"2024-05-08T18:07:13.747669+0000","last_peered":"2024-05-08T18:07:13.747669+0000","last_clean":"2024-05-08T18:07:13.747669+0000","last_became_active":"2024-05-08T18:04:53.848430+0000","last_became_peered":"2024-05-08T18:04:53.848430+0000","last_unstale":"2024-05-08T18:07:13.747669+0000","last_undegraded":"2024-05-08T18:07:13.747669+0000","last_fullsized":"2024-05-08T18:07:13.747669+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"29'10","reported_seq":67,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.742706+0000","last_change":"2024-05-08T18:04:45.807312+0000","last_active":"2024-05-08T18:07:18.742706+0000","last_peered":"2024-05-08T18:07:18.742706+0000","last_clean":"2024-05-08T18:07:18.742706+0000","last_became_active":"2024-05-08T18:04:45.806808+0000","last_became_peered":"2024-05-08T18:04:45.806808+0000","last_unstale":"2024-05-08T18:07:18.742706+0000","last_undegraded":"2024-05-08T18:07:18.742706+0000","last_fullsized":"2024-05-08T18:07:18.742706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747763+0000","last_change":"2024-05-08T18:04:43.833094+0000","last_active":"2024-05-08T18:07:13.747763+0000","last_peered":"2024-05-08T18:07:13.747763+0000","last_clean":"2024-05-08T18:07:13.747763+0000","last_became_active":"2024-05-08T18:04:43.832147+0000","last_became_peered":"2024-05-08T18:04:43.832147+0000","last_unstale":"2024-05-08T18:07:13.747763+0000","last_undegraded":"2024-05-08T18:07:13.747763+0000","last_fullsized":"2024-05-08T18:07:13.747763+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.517382+0000","last_change":"2024-05-08T18:08:22.517404+0000","last_active":"2024-05-08T18:08:22.517382+0000","last_peered":"2024-05-08T18:08:22.517382+0000","last_clean":"2024-05-08T18:08:22.517382+0000","last_became_active":"2024-05-08T18:04:47.815219+0000","last_became_peered":"2024-05-08T18:04:47.815219+0000","last_unstale":"2024-05-08T18:08:22.517382+0000","last_undegraded":"2024-05-08T18:08:22.517382+0000","last_fullsized":"2024-05-08T18:08:22.517382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.517292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.517292+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.517292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:28.093920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"27'10","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744250+0000","last_change":"2024-05-08T18:04:49.833674+0000","last_active":"2024-05-08T18:06:53.744250+0000","last_peered":"2024-05-08T18:06:53.744250+0000","last_clean":"2024-05-08T18:06:53.744250+0000","last_became_active":"2024-05-08T18:04:49.830793+0000","last_became_peered":"2024-05-08T18:04:49.830793+0000","last_unstale":"2024-05-08T18:06:53.744250+0000","last_undegraded":"2024-05-08T18:06:53.744250+0000","last_fullsized":"2024-05-08T18:06:53.744250+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"29'285","reported_seq":378,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.160228+0000","last_change":"2024-05-08T18:08:09.160257+0000","last_active":"2024-05-08T18:08:09.160228+0000","last_peered":"2024-05-08T18:08:09.160228+0000","last_clean":"2024-05-08T18:08:09.160228+0000","last_became_active":"2024-05-08T18:04:51.833866+0000","last_became_peered":"2024-05-08T18:04:51.833866+0000","last_unstale":"2024-05-08T18:08:09.160228+0000","last_undegraded":"2024-05-08T18:08:09.160228+0000","last_fullsized":"2024-05-08T18:08:09.160228+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:09.160141+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:09.160141+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.160141+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:29:48.216550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747522+0000","last_change":"2024-05-08T18:04:53.848804+0000","last_active":"2024-05-08T18:07:13.747522+0000","last_peered":"2024-05-08T18:07:13.747522+0000","last_clean":"2024-05-08T18:07:13.747522+0000","last_became_active":"2024-05-08T18:04:53.848321+0000","last_became_peered":"2024-05-08T18:04:53.848321+0000","last_unstale":"2024-05-08T18:07:13.747522+0000","last_undegraded":"2024-05-08T18:07:13.747522+0000","last_fullsized":"2024-05-08T18:07:13.747522+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"29'4","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.747624+0000","last_change":"2024-05-08T18:04:45.814018+0000","last_active":"2024-05-08T18:07:18.747624+0000","last_peered":"2024-05-08T18:07:18.747624+0000","last_clean":"2024-05-08T18:07:18.747624+0000","last_became_active":"2024-05-08T18:04:45.813424+0000","last_became_peered":"2024-05-08T18:04:45.813424+0000","last_unstale":"2024-05-08T18:07:18.747624+0000","last_undegraded":"2024-05-08T18:07:18.747624+0000","last_fullsized":"2024-05-08T18:07:18.747624+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.746977+0000","last_change":"2024-05-08T18:04:43.841603+0000","last_active":"2024-05-08T18:07:13.746977+0000","last_peered":"2024-05-08T18:07:13.746977+0000","last_clean":"2024-05-08T18:07:13.746977+0000","last_became_active":"2024-05-08T18:04:43.840502+0000","last_became_peered":"2024-05-08T18:04:43.840502+0000","last_unstale":"2024-05-08T18:07:13.746977+0000","last_undegraded":"2024-05-08T18:07:13.746977+0000","last_fullsized":"2024-05-08T18:07:13.746977+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.519639+0000","last_change":"2024-05-08T18:08:18.519666+0000","last_active":"2024-05-08T18:08:18.519639+0000","last_peered":"2024-05-08T18:08:18.519639+0000","last_clean":"2024-05-08T18:08:18.519639+0000","last_became_active":"2024-05-08T18:04:47.818839+0000","last_became_peered":"2024-05-08T18:04:47.818839+0000","last_unstale":"2024-05-08T18:08:18.519639+0000","last_undegraded":"2024-05-08T18:08:18.519639+0000","last_fullsized":"2024-05-08T18:08:18.519639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:18.519542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:18.519542+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.519542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:14:21.272156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"29'46","reported_seq":104,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.495547+0000","last_change":"2024-05-08T18:08:12.495592+0000","last_active":"2024-05-08T18:08:12.495547+0000","last_peered":"2024-05-08T18:08:12.495547+0000","last_clean":"2024-05-08T18:08:12.495547+0000","last_became_active":"2024-05-08T18:04:49.824405+0000","last_became_peered":"2024-05-08T18:04:49.824405+0000","last_unstale":"2024-05-08T18:08:12.495547+0000","last_undegraded":"2024-05-08T18:08:12.495547+0000","last_fullsized":"2024-05-08T18:08:12.495547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'46","last_scrub_stamp":"2024-05-08T18:08:12.495454+0000","last_deep_scrub":"29'46","last_deep_scrub_stamp":"2024-05-08T18:08:12.495454+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.495454+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:48:04.037407+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.562545+0000","last_change":"2024-05-08T18:08:17.562569+0000","last_active":"2024-05-08T18:08:17.562545+0000","last_peered":"2024-05-08T18:08:17.562545+0000","last_clean":"2024-05-08T18:08:17.562545+0000","last_became_active":"2024-05-08T18:04:51.844757+0000","last_became_peered":"2024-05-08T18:04:51.844757+0000","last_unstale":"2024-05-08T18:08:17.562545+0000","last_undegraded":"2024-05-08T18:08:17.562545+0000","last_fullsized":"2024-05-08T18:08:17.562545+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:17.562441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:17.562441+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.562441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:50:03.644980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.747785+0000","last_change":"2024-05-08T18:04:53.841236+0000","last_active":"2024-05-08T18:07:18.747785+0000","last_peered":"2024-05-08T18:07:18.747785+0000","last_clean":"2024-05-08T18:07:18.747785+0000","last_became_active":"2024-05-08T18:04:53.840956+0000","last_became_peered":"2024-05-08T18:04:53.840956+0000","last_unstale":"2024-05-08T18:07:18.747785+0000","last_undegraded":"2024-05-08T18:07:18.747785+0000","last_fullsized":"2024-05-08T18:07:18.747785+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"29'3","reported_seq":37,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.115262+0000","last_change":"2024-05-08T18:04:45.807309+0000","last_active":"2024-05-08T18:07:54.115262+0000","last_peered":"2024-05-08T18:07:54.115262+0000","last_clean":"2024-05-08T18:07:54.115262+0000","last_became_active":"2024-05-08T18:04:45.806560+0000","last_became_peered":"2024-05-08T18:04:45.806560+0000","last_unstale":"2024-05-08T18:07:54.115262+0000","last_undegraded":"2024-05-08T18:07:54.115262+0000","last_fullsized":"2024-05-08T18:07:54.115262+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747834+0000","last_change":"2024-05-08T18:04:43.845626+0000","last_active":"2024-05-08T18:07:13.747834+0000","last_peered":"2024-05-08T18:07:13.747834+0000","last_clean":"2024-05-08T18:07:13.747834+0000","last_became_active":"2024-05-08T18:04:43.845214+0000","last_became_peered":"2024-05-08T18:04:43.845214+0000","last_unstale":"2024-05-08T18:07:13.747834+0000","last_undegraded":"2024-05-08T18:07:13.747834+0000","last_fullsized":"2024-05-08T18:07:13.747834+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.286599+0000","last_change":"2024-05-08T18:08:23.286625+0000","last_active":"2024-05-08T18:08:23.286599+0000","last_peered":"2024-05-08T18:08:23.286599+0000","last_clean":"2024-05-08T18:08:23.286599+0000","last_became_active":"2024-05-08T18:04:47.822748+0000","last_became_peered":"2024-05-08T18:04:47.822748+0000","last_unstale":"2024-05-08T18:08:23.286599+0000","last_undegraded":"2024-05-08T18:08:23.286599+0000","last_fullsized":"2024-05-08T18:08:23.286599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:23.286509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:23.286509+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.286509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:17:35.464618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"29'37","reported_seq":92,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.576260+0000","last_change":"2024-05-08T18:08:14.576300+0000","last_active":"2024-05-08T18:08:14.576260+0000","last_peered":"2024-05-08T18:08:14.576260+0000","last_clean":"2024-05-08T18:08:14.576260+0000","last_became_active":"2024-05-08T18:04:49.831440+0000","last_became_peered":"2024-05-08T18:04:49.831440+0000","last_unstale":"2024-05-08T18:08:14.576260+0000","last_undegraded":"2024-05-08T18:08:14.576260+0000","last_fullsized":"2024-05-08T18:08:14.576260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'37","last_scrub_stamp":"2024-05-08T18:08:14.576150+0000","last_deep_scrub":"29'37","last_deep_scrub_stamp":"2024-05-08T18:08:14.576150+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.576150+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:06:14.892455+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.748366+0000","last_change":"2024-05-08T18:04:51.844443+0000","last_active":"2024-05-08T18:07:18.748366+0000","last_peered":"2024-05-08T18:07:18.748366+0000","last_clean":"2024-05-08T18:07:18.748366+0000","last_became_active":"2024-05-08T18:04:51.844125+0000","last_became_peered":"2024-05-08T18:04:51.844125+0000","last_unstale":"2024-05-08T18:07:18.748366+0000","last_undegraded":"2024-05-08T18:07:18.748366+0000","last_fullsized":"2024-05-08T18:07:18.748366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.748549+0000","last_change":"2024-05-08T18:04:53.849962+0000","last_active":"2024-05-08T18:07:18.748549+0000","last_peered":"2024-05-08T18:07:18.748549+0000","last_clean":"2024-05-08T18:07:18.748549+0000","last_became_active":"2024-05-08T18:04:53.849514+0000","last_became_peered":"2024-05-08T18:04:53.849514+0000","last_unstale":"2024-05-08T18:07:18.748549+0000","last_undegraded":"2024-05-08T18:07:18.748549+0000","last_fullsized":"2024-05-08T18:07:18.748549+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"29'4","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747631+0000","last_change":"2024-05-08T18:04:45.822495+0000","last_active":"2024-05-08T18:07:13.747631+0000","last_peered":"2024-05-08T18:07:13.747631+0000","last_clean":"2024-05-08T18:07:13.747631+0000","last_became_active":"2024-05-08T18:04:45.822204+0000","last_became_peered":"2024-05-08T18:04:45.822204+0000","last_unstale":"2024-05-08T18:07:13.747631+0000","last_undegraded":"2024-05-08T18:07:13.747631+0000","last_fullsized":"2024-05-08T18:07:13.747631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"29'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.061394+0000","last_change":"2024-05-08T18:04:43.835654+0000","last_active":"2024-05-08T18:07:54.061394+0000","last_peered":"2024-05-08T18:07:54.061394+0000","last_clean":"2024-05-08T18:07:54.061394+0000","last_became_active":"2024-05-08T18:04:43.834678+0000","last_became_peered":"2024-05-08T18:04:43.834678+0000","last_unstale":"2024-05-08T18:07:54.061394+0000","last_undegraded":"2024-05-08T18:07:54.061394+0000","last_fullsized":"2024-05-08T18:07:54.061394+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.501159+0000","last_change":"2024-05-08T18:08:16.501182+0000","last_active":"2024-05-08T18:08:16.501159+0000","last_peered":"2024-05-08T18:08:16.501159+0000","last_clean":"2024-05-08T18:08:16.501159+0000","last_became_active":"2024-05-08T18:04:47.820036+0000","last_became_peered":"2024-05-08T18:04:47.820036+0000","last_unstale":"2024-05-08T18:08:16.501159+0000","last_undegraded":"2024-05-08T18:08:16.501159+0000","last_fullsized":"2024-05-08T18:08:16.501159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:16.501065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:16.501065+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.501065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:49.649348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"29'32","reported_seq":85,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.857112+0000","last_change":"2024-05-08T18:08:20.857184+0000","last_active":"2024-05-08T18:08:20.857112+0000","last_peered":"2024-05-08T18:08:20.857112+0000","last_clean":"2024-05-08T18:08:20.857112+0000","last_became_active":"2024-05-08T18:04:49.830882+0000","last_became_peered":"2024-05-08T18:04:49.830882+0000","last_unstale":"2024-05-08T18:08:20.857112+0000","last_undegraded":"2024-05-08T18:08:20.857112+0000","last_fullsized":"2024-05-08T18:08:20.857112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'32","last_scrub_stamp":"2024-05-08T18:08:20.857010+0000","last_deep_scrub":"29'32","last_deep_scrub_stamp":"2024-05-08T18:08:20.857010+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.857010+0000","objects_scrubbed":2,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:06.682042+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":49,"num_write":14,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:23.749138+0000","last_change":"2024-05-08T18:04:51.844109+0000","last_active":"2024-05-08T18:07:23.749138+0000","last_peered":"2024-05-08T18:07:23.749138+0000","last_clean":"2024-05-08T18:07:23.749138+0000","last_became_active":"2024-05-08T18:04:51.842734+0000","last_became_peered":"2024-05-08T18:04:51.842734+0000","last_unstale":"2024-05-08T18:07:23.749138+0000","last_undegraded":"2024-05-08T18:07:23.749138+0000","last_fullsized":"2024-05-08T18:07:23.749138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'10","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.779738+0000","last_change":"2024-05-08T18:04:53.845139+0000","last_active":"2024-05-08T18:07:53.779738+0000","last_peered":"2024-05-08T18:07:53.779738+0000","last_clean":"2024-05-08T18:07:53.779738+0000","last_became_active":"2024-05-08T18:04:53.844891+0000","last_became_peered":"2024-05-08T18:04:53.844891+0000","last_unstale":"2024-05-08T18:07:53.779738+0000","last_undegraded":"2024-05-08T18:07:53.779738+0000","last_fullsized":"2024-05-08T18:07:53.779738+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"29'2","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.866484+0000","last_change":"2024-05-08T18:04:45.809216+0000","last_active":"2024-05-08T18:07:53.866484+0000","last_peered":"2024-05-08T18:07:53.866484+0000","last_clean":"2024-05-08T18:07:53.866484+0000","last_became_active":"2024-05-08T18:04:45.808936+0000","last_became_peered":"2024-05-08T18:04:45.808936+0000","last_unstale":"2024-05-08T18:07:53.866484+0000","last_undegraded":"2024-05-08T18:07:53.866484+0000","last_fullsized":"2024-05-08T18:07:53.866484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.743083+0000","last_change":"2024-05-08T18:04:43.837820+0000","last_active":"2024-05-08T18:06:48.743083+0000","last_peered":"2024-05-08T18:06:48.743083+0000","last_clean":"2024-05-08T18:06:48.743083+0000","last_became_active":"2024-05-08T18:04:43.837070+0000","last_became_peered":"2024-05-08T18:04:43.837070+0000","last_unstale":"2024-05-08T18:06:48.743083+0000","last_undegraded":"2024-05-08T18:06:48.743083+0000","last_fullsized":"2024-05-08T18:06:48.743083+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"29'3","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.481725+0000","last_change":"2024-05-08T18:08:15.481759+0000","last_active":"2024-05-08T18:08:15.481725+0000","last_peered":"2024-05-08T18:08:15.481725+0000","last_clean":"2024-05-08T18:08:15.481725+0000","last_became_active":"2024-05-08T18:04:47.824172+0000","last_became_peered":"2024-05-08T18:04:47.824172+0000","last_unstale":"2024-05-08T18:08:15.481725+0000","last_undegraded":"2024-05-08T18:08:15.481725+0000","last_fullsized":"2024-05-08T18:08:15.481725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3","last_scrub_stamp":"2024-05-08T18:08:15.481627+0000","last_deep_scrub":"29'3","last_deep_scrub_stamp":"2024-05-08T18:08:15.481627+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.481627+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:31:00.387286+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"29'35","reported_seq":84,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.705408+0000","last_change":"2024-05-08T18:04:49.832422+0000","last_active":"2024-05-08T18:08:00.705408+0000","last_peered":"2024-05-08T18:08:00.705408+0000","last_clean":"2024-05-08T18:08:00.705408+0000","last_became_active":"2024-05-08T18:04:49.831803+0000","last_became_peered":"2024-05-08T18:04:49.831803+0000","last_unstale":"2024-05-08T18:08:00.705408+0000","last_undegraded":"2024-05-08T18:08:00.705408+0000","last_fullsized":"2024-05-08T18:08:00.705408+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:48.804261+0000","last_clean_scrub_stamp":"2024-05-08T18:04:48.804261+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:23.743357+0000","last_change":"2024-05-08T18:04:51.846422+0000","last_active":"2024-05-08T18:07:23.743357+0000","last_peered":"2024-05-08T18:07:23.743357+0000","last_clean":"2024-05-08T18:07:23.743357+0000","last_became_active":"2024-05-08T18:04:51.846225+0000","last_became_peered":"2024-05-08T18:04:51.846225+0000","last_unstale":"2024-05-08T18:07:23.743357+0000","last_undegraded":"2024-05-08T18:07:23.743357+0000","last_fullsized":"2024-05-08T18:07:23.743357+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:50.810954+0000","last_clean_scrub_stamp":"2024-05-08T18:04:50.810954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"29'4","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747795+0000","last_change":"2024-05-08T18:04:53.843030+0000","last_active":"2024-05-08T18:07:13.747795+0000","last_peered":"2024-05-08T18:07:13.747795+0000","last_clean":"2024-05-08T18:07:13.747795+0000","last_became_active":"2024-05-08T18:04:53.842429+0000","last_became_peered":"2024-05-08T18:04:53.842429+0000","last_unstale":"2024-05-08T18:07:13.747795+0000","last_undegraded":"2024-05-08T18:07:13.747795+0000","last_fullsized":"2024-05-08T18:07:13.747795+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"27'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744615+0000","last_change":"2024-05-08T18:04:45.818828+0000","last_active":"2024-05-08T18:06:58.744615+0000","last_peered":"2024-05-08T18:06:58.744615+0000","last_clean":"2024-05-08T18:06:58.744615+0000","last_became_active":"2024-05-08T18:04:45.818161+0000","last_became_peered":"2024-05-08T18:04:45.818161+0000","last_unstale":"2024-05-08T18:06:58.744615+0000","last_undegraded":"2024-05-08T18:06:58.744615+0000","last_fullsized":"2024-05-08T18:06:58.744615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.742602+0000","last_change":"2024-05-08T18:04:43.843055+0000","last_active":"2024-05-08T18:06:48.742602+0000","last_peered":"2024-05-08T18:06:48.742602+0000","last_clean":"2024-05-08T18:06:48.742602+0000","last_became_active":"2024-05-08T18:04:43.842483+0000","last_became_peered":"2024-05-08T18:04:43.842483+0000","last_unstale":"2024-05-08T18:06:48.742602+0000","last_undegraded":"2024-05-08T18:06:48.742602+0000","last_fullsized":"2024-05-08T18:06:48.742602+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"29'36","reported_seq":107,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.002529+0000","last_change":"2024-05-08T18:04:45.808186+0000","last_active":"2024-05-08T18:07:54.002529+0000","last_peered":"2024-05-08T18:07:54.002529+0000","last_clean":"2024-05-08T18:07:54.002529+0000","last_became_active":"2024-05-08T18:04:45.807847+0000","last_became_peered":"2024-05-08T18:04:45.807847+0000","last_unstale":"2024-05-08T18:07:54.002529+0000","last_undegraded":"2024-05-08T18:07:54.002529+0000","last_fullsized":"2024-05-08T18:07:54.002529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":44,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.738086+0000","last_change":"2024-05-08T18:04:43.836530+0000","last_active":"2024-05-08T18:06:48.738086+0000","last_peered":"2024-05-08T18:06:48.738086+0000","last_clean":"2024-05-08T18:06:48.738086+0000","last_became_active":"2024-05-08T18:04:43.836045+0000","last_became_peered":"2024-05-08T18:04:43.836045+0000","last_unstale":"2024-05-08T18:06:48.738086+0000","last_undegraded":"2024-05-08T18:06:48.738086+0000","last_fullsized":"2024-05-08T18:06:48.738086+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3,2],"acting":[0,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"29'11","reported_seq":69,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.116104+0000","last_change":"2024-05-08T18:04:45.826393+0000","last_active":"2024-05-08T18:07:54.116104+0000","last_peered":"2024-05-08T18:07:54.116104+0000","last_clean":"2024-05-08T18:07:54.116104+0000","last_became_active":"2024-05-08T18:04:45.825506+0000","last_became_peered":"2024-05-08T18:04:45.825506+0000","last_unstale":"2024-05-08T18:07:54.116104+0000","last_undegraded":"2024-05-08T18:07:54.116104+0000","last_fullsized":"2024-05-08T18:07:54.116104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":46,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"29'6","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.668310+0000","last_change":"2024-05-08T18:04:43.832854+0000","last_active":"2024-05-08T18:08:00.668310+0000","last_peered":"2024-05-08T18:08:00.668310+0000","last_clean":"2024-05-08T18:08:00.668310+0000","last_became_active":"2024-05-08T18:04:43.832406+0000","last_became_peered":"2024-05-08T18:04:43.832406+0000","last_unstale":"2024-05-08T18:08:00.668310+0000","last_undegraded":"2024-05-08T18:08:00.668310+0000","last_fullsized":"2024-05-08T18:08:00.668310+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":8,"num_write":30,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"29'2","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.781886+0000","last_change":"2024-05-08T18:04:45.807187+0000","last_active":"2024-05-08T18:07:53.781886+0000","last_peered":"2024-05-08T18:07:53.781886+0000","last_clean":"2024-05-08T18:07:53.781886+0000","last_became_active":"2024-05-08T18:04:45.806646+0000","last_became_peered":"2024-05-08T18:04:45.806646+0000","last_unstale":"2024-05-08T18:07:53.781886+0000","last_undegraded":"2024-05-08T18:07:53.781886+0000","last_fullsized":"2024-05-08T18:07:53.781886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"29'6","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.669859+0000","last_change":"2024-05-08T18:04:43.836158+0000","last_active":"2024-05-08T18:08:00.669859+0000","last_peered":"2024-05-08T18:08:00.669859+0000","last_clean":"2024-05-08T18:08:00.669859+0000","last_became_active":"2024-05-08T18:04:43.835163+0000","last_became_peered":"2024-05-08T18:04:43.835163+0000","last_unstale":"2024-05-08T18:08:00.669859+0000","last_undegraded":"2024-05-08T18:08:00.669859+0000","last_fullsized":"2024-05-08T18:08:00.669859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":4746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,1],"acting":[0,3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"29'2","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.867052+0000","last_change":"2024-05-08T18:04:45.816663+0000","last_active":"2024-05-08T18:07:53.867052+0000","last_peered":"2024-05-08T18:07:53.867052+0000","last_clean":"2024-05-08T18:07:53.867052+0000","last_became_active":"2024-05-08T18:04:45.816391+0000","last_became_peered":"2024-05-08T18:04:45.816391+0000","last_unstale":"2024-05-08T18:07:53.867052+0000","last_undegraded":"2024-05-08T18:07:53.867052+0000","last_fullsized":"2024-05-08T18:07:53.867052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'4","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.738989+0000","last_change":"2024-05-08T18:04:43.836083+0000","last_active":"2024-05-08T18:06:53.738989+0000","last_peered":"2024-05-08T18:06:53.738989+0000","last_clean":"2024-05-08T18:06:53.738989+0000","last_became_active":"2024-05-08T18:04:43.835268+0000","last_became_peered":"2024-05-08T18:04:43.835268+0000","last_unstale":"2024-05-08T18:06:53.738989+0000","last_undegraded":"2024-05-08T18:06:53.738989+0000","last_fullsized":"2024-05-08T18:06:53.738989+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"29'12","reported_seq":78,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.041846+0000","last_change":"2024-05-08T18:04:45.820787+0000","last_active":"2024-05-08T18:07:54.041846+0000","last_peered":"2024-05-08T18:07:54.041846+0000","last_clean":"2024-05-08T18:07:54.041846+0000","last_became_active":"2024-05-08T18:04:45.819699+0000","last_became_peered":"2024-05-08T18:04:45.819699+0000","last_unstale":"2024-05-08T18:07:54.041846+0000","last_undegraded":"2024-05-08T18:07:54.041846+0000","last_fullsized":"2024-05-08T18:07:54.041846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.743834+0000","last_change":"2024-05-08T18:04:43.837155+0000","last_active":"2024-05-08T18:06:53.743834+0000","last_peered":"2024-05-08T18:06:53.743834+0000","last_clean":"2024-05-08T18:06:53.743834+0000","last_became_active":"2024-05-08T18:04:43.836735+0000","last_became_peered":"2024-05-08T18:04:43.836735+0000","last_unstale":"2024-05-08T18:06:53.743834+0000","last_undegraded":"2024-05-08T18:06:53.743834+0000","last_fullsized":"2024-05-08T18:06:53.743834+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"29'14","reported_seq":71,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.738934+0000","last_change":"2024-05-08T18:04:45.812327+0000","last_active":"2024-05-08T18:06:53.738934+0000","last_peered":"2024-05-08T18:06:53.738934+0000","last_clean":"2024-05-08T18:06:53.738934+0000","last_became_active":"2024-05-08T18:04:45.811953+0000","last_became_peered":"2024-05-08T18:04:45.811953+0000","last_unstale":"2024-05-08T18:06:53.738934+0000","last_undegraded":"2024-05-08T18:06:53.738934+0000","last_fullsized":"2024-05-08T18:06:53.738934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.743509+0000","last_change":"2024-05-08T18:04:43.843298+0000","last_active":"2024-05-08T18:06:53.743509+0000","last_peered":"2024-05-08T18:06:53.743509+0000","last_clean":"2024-05-08T18:06:53.743509+0000","last_became_active":"2024-05-08T18:04:43.842179+0000","last_became_peered":"2024-05-08T18:04:43.842179+0000","last_unstale":"2024-05-08T18:06:53.743509+0000","last_undegraded":"2024-05-08T18:06:53.743509+0000","last_fullsized":"2024-05-08T18:06:53.743509+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,0],"acting":[2,3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"29'8","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070501+0000","last_change":"2024-05-08T18:04:45.812893+0000","last_active":"2024-05-08T18:07:54.070501+0000","last_peered":"2024-05-08T18:07:54.070501+0000","last_clean":"2024-05-08T18:07:54.070501+0000","last_became_active":"2024-05-08T18:04:45.812637+0000","last_became_peered":"2024-05-08T18:04:45.812637+0000","last_unstale":"2024-05-08T18:07:54.070501+0000","last_undegraded":"2024-05-08T18:07:54.070501+0000","last_fullsized":"2024-05-08T18:07:54.070501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"29'6","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.984349+0000","last_change":"2024-05-08T18:04:43.834951+0000","last_active":"2024-05-08T18:07:53.984349+0000","last_peered":"2024-05-08T18:07:53.984349+0000","last_clean":"2024-05-08T18:07:53.984349+0000","last_became_active":"2024-05-08T18:04:43.834491+0000","last_became_peered":"2024-05-08T18:04:43.834491+0000","last_unstale":"2024-05-08T18:07:53.984349+0000","last_undegraded":"2024-05-08T18:07:53.984349+0000","last_fullsized":"2024-05-08T18:07:53.984349+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"29'8","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.041422+0000","last_change":"2024-05-08T18:04:45.815948+0000","last_active":"2024-05-08T18:07:54.041422+0000","last_peered":"2024-05-08T18:07:54.041422+0000","last_clean":"2024-05-08T18:07:54.041422+0000","last_became_active":"2024-05-08T18:04:45.815677+0000","last_became_peered":"2024-05-08T18:04:45.815677+0000","last_unstale":"2024-05-08T18:07:54.041422+0000","last_undegraded":"2024-05-08T18:07:54.041422+0000","last_fullsized":"2024-05-08T18:07:54.041422+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744094+0000","last_change":"2024-05-08T18:04:43.846054+0000","last_active":"2024-05-08T18:06:53.744094+0000","last_peered":"2024-05-08T18:06:53.744094+0000","last_clean":"2024-05-08T18:06:53.744094+0000","last_became_active":"2024-05-08T18:04:43.843826+0000","last_became_peered":"2024-05-08T18:04:43.843826+0000","last_unstale":"2024-05-08T18:06:53.744094+0000","last_undegraded":"2024-05-08T18:06:53.744094+0000","last_fullsized":"2024-05-08T18:06:53.744094+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"29'6","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746982+0000","last_change":"2024-05-08T18:04:45.827535+0000","last_active":"2024-05-08T18:07:08.746982+0000","last_peered":"2024-05-08T18:07:08.746982+0000","last_clean":"2024-05-08T18:07:08.746982+0000","last_became_active":"2024-05-08T18:04:45.827252+0000","last_became_peered":"2024-05-08T18:04:45.827252+0000","last_unstale":"2024-05-08T18:07:08.746982+0000","last_undegraded":"2024-05-08T18:07:08.746982+0000","last_fullsized":"2024-05-08T18:07:08.746982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.739685+0000","last_change":"2024-05-08T18:04:43.820995+0000","last_active":"2024-05-08T18:06:58.739685+0000","last_peered":"2024-05-08T18:06:58.739685+0000","last_clean":"2024-05-08T18:06:58.739685+0000","last_became_active":"2024-05-08T18:04:43.820360+0000","last_became_peered":"2024-05-08T18:04:43.820360+0000","last_unstale":"2024-05-08T18:06:58.739685+0000","last_undegraded":"2024-05-08T18:06:58.739685+0000","last_fullsized":"2024-05-08T18:06:58.739685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.740670+0000","last_change":"2024-05-08T18:04:45.823656+0000","last_active":"2024-05-08T18:07:03.740670+0000","last_peered":"2024-05-08T18:07:03.740670+0000","last_clean":"2024-05-08T18:07:03.740670+0000","last_became_active":"2024-05-08T18:04:45.823385+0000","last_became_peered":"2024-05-08T18:04:45.823385+0000","last_unstale":"2024-05-08T18:07:03.740670+0000","last_undegraded":"2024-05-08T18:07:03.740670+0000","last_fullsized":"2024-05-08T18:07:03.740670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'6","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.757844+0000","last_change":"2024-05-08T18:04:43.836532+0000","last_active":"2024-05-08T18:07:53.757844+0000","last_peered":"2024-05-08T18:07:53.757844+0000","last_clean":"2024-05-08T18:07:53.757844+0000","last_became_active":"2024-05-08T18:04:43.835838+0000","last_became_peered":"2024-05-08T18:04:43.835838+0000","last_unstale":"2024-05-08T18:07:53.757844+0000","last_undegraded":"2024-05-08T18:07:53.757844+0000","last_fullsized":"2024-05-08T18:07:53.757844+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8,"num_write":30,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"29'10","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.002652+0000","last_change":"2024-05-08T18:04:45.810992+0000","last_active":"2024-05-08T18:07:54.002652+0000","last_peered":"2024-05-08T18:07:54.002652+0000","last_clean":"2024-05-08T18:07:54.002652+0000","last_became_active":"2024-05-08T18:04:45.810703+0000","last_became_peered":"2024-05-08T18:04:45.810703+0000","last_unstale":"2024-05-08T18:07:54.002652+0000","last_undegraded":"2024-05-08T18:07:54.002652+0000","last_fullsized":"2024-05-08T18:07:54.002652+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"29'4","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.668657+0000","last_change":"2024-05-08T18:04:43.836123+0000","last_active":"2024-05-08T18:08:00.668657+0000","last_peered":"2024-05-08T18:08:00.668657+0000","last_clean":"2024-05-08T18:08:00.668657+0000","last_became_active":"2024-05-08T18:04:43.835518+0000","last_became_peered":"2024-05-08T18:04:43.835518+0000","last_unstale":"2024-05-08T18:08:00.668657+0000","last_undegraded":"2024-05-08T18:08:00.668657+0000","last_fullsized":"2024-05-08T18:08:00.668657+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,3],"acting":[1,2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"29'4","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745381+0000","last_change":"2024-05-08T18:04:45.821376+0000","last_active":"2024-05-08T18:07:03.745381+0000","last_peered":"2024-05-08T18:07:03.745381+0000","last_clean":"2024-05-08T18:07:03.745381+0000","last_became_active":"2024-05-08T18:04:45.820746+0000","last_became_peered":"2024-05-08T18:04:45.820746+0000","last_unstale":"2024-05-08T18:07:03.745381+0000","last_undegraded":"2024-05-08T18:07:03.745381+0000","last_fullsized":"2024-05-08T18:07:03.745381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.745035+0000","last_change":"2024-05-08T18:04:43.847998+0000","last_active":"2024-05-08T18:06:58.745035+0000","last_peered":"2024-05-08T18:06:58.745035+0000","last_clean":"2024-05-08T18:06:58.745035+0000","last_became_active":"2024-05-08T18:04:43.844404+0000","last_became_peered":"2024-05-08T18:04:43.844404+0000","last_unstale":"2024-05-08T18:06:58.745035+0000","last_undegraded":"2024-05-08T18:06:58.745035+0000","last_fullsized":"2024-05-08T18:06:58.745035+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"29'4","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745363+0000","last_change":"2024-05-08T18:04:45.809582+0000","last_active":"2024-05-08T18:07:03.745363+0000","last_peered":"2024-05-08T18:07:03.745363+0000","last_clean":"2024-05-08T18:07:03.745363+0000","last_became_active":"2024-05-08T18:04:45.809271+0000","last_became_peered":"2024-05-08T18:04:45.809271+0000","last_unstale":"2024-05-08T18:07:03.745363+0000","last_undegraded":"2024-05-08T18:07:03.745363+0000","last_fullsized":"2024-05-08T18:07:03.745363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.745103+0000","last_change":"2024-05-08T18:04:43.844366+0000","last_active":"2024-05-08T18:06:58.745103+0000","last_peered":"2024-05-08T18:06:58.745103+0000","last_clean":"2024-05-08T18:06:58.745103+0000","last_became_active":"2024-05-08T18:04:43.838125+0000","last_became_peered":"2024-05-08T18:04:43.838125+0000","last_unstale":"2024-05-08T18:06:58.745103+0000","last_undegraded":"2024-05-08T18:06:58.745103+0000","last_fullsized":"2024-05-08T18:06:58.745103+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"29'15","reported_seq":84,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.868964+0000","last_change":"2024-05-08T18:08:21.868994+0000","last_active":"2024-05-08T18:08:21.868964+0000","last_peered":"2024-05-08T18:08:21.868964+0000","last_clean":"2024-05-08T18:08:21.868964+0000","last_became_active":"2024-05-08T18:04:45.828488+0000","last_became_peered":"2024-05-08T18:04:45.828488+0000","last_unstale":"2024-05-08T18:08:21.868964+0000","last_undegraded":"2024-05-08T18:08:21.868964+0000","last_fullsized":"2024-05-08T18:08:21.868964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:21.868870+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:21.868870+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.868870+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:18:57.376906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745250+0000","last_change":"2024-05-08T18:04:43.843455+0000","last_active":"2024-05-08T18:07:03.745250+0000","last_peered":"2024-05-08T18:07:03.745250+0000","last_clean":"2024-05-08T18:07:03.745250+0000","last_became_active":"2024-05-08T18:04:43.842670+0000","last_became_peered":"2024-05-08T18:04:43.842670+0000","last_unstale":"2024-05-08T18:07:03.745250+0000","last_undegraded":"2024-05-08T18:07:03.745250+0000","last_fullsized":"2024-05-08T18:07:03.745250+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744378+0000","last_change":"2024-05-08T18:04:45.812074+0000","last_active":"2024-05-08T18:06:58.744378+0000","last_peered":"2024-05-08T18:06:58.744378+0000","last_clean":"2024-05-08T18:06:58.744378+0000","last_became_active":"2024-05-08T18:04:45.810562+0000","last_became_peered":"2024-05-08T18:04:45.810562+0000","last_unstale":"2024-05-08T18:06:58.744378+0000","last_undegraded":"2024-05-08T18:06:58.744378+0000","last_fullsized":"2024-05-08T18:06:58.744378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.746137+0000","last_change":"2024-05-08T18:04:43.838746+0000","last_active":"2024-05-08T18:07:03.746137+0000","last_peered":"2024-05-08T18:07:03.746137+0000","last_clean":"2024-05-08T18:07:03.746137+0000","last_became_active":"2024-05-08T18:04:43.835943+0000","last_became_peered":"2024-05-08T18:04:43.835943+0000","last_unstale":"2024-05-08T18:07:03.746137+0000","last_undegraded":"2024-05-08T18:07:03.746137+0000","last_fullsized":"2024-05-08T18:07:03.746137+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745822+0000","last_change":"2024-05-08T18:04:43.836378+0000","last_active":"2024-05-08T18:07:03.745822+0000","last_peered":"2024-05-08T18:07:03.745822+0000","last_clean":"2024-05-08T18:07:03.745822+0000","last_became_active":"2024-05-08T18:04:43.836007+0000","last_became_peered":"2024-05-08T18:04:43.836007+0000","last_unstale":"2024-05-08T18:07:03.745822+0000","last_undegraded":"2024-05-08T18:07:03.745822+0000","last_fullsized":"2024-05-08T18:07:03.745822+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"29'8","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.742097+0000","last_change":"2024-05-08T18:04:45.812333+0000","last_active":"2024-05-08T18:07:13.742097+0000","last_peered":"2024-05-08T18:07:13.742097+0000","last_clean":"2024-05-08T18:07:13.742097+0000","last_became_active":"2024-05-08T18:04:45.811648+0000","last_became_peered":"2024-05-08T18:04:45.811648+0000","last_unstale":"2024-05-08T18:07:13.742097+0000","last_undegraded":"2024-05-08T18:07:13.742097+0000","last_fullsized":"2024-05-08T18:07:13.742097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":31,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"29'6","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.668927+0000","last_change":"2024-05-08T18:04:43.836500+0000","last_active":"2024-05-08T18:08:00.668927+0000","last_peered":"2024-05-08T18:08:00.668927+0000","last_clean":"2024-05-08T18:08:00.668927+0000","last_became_active":"2024-05-08T18:04:43.835712+0000","last_became_peered":"2024-05-08T18:04:43.835712+0000","last_unstale":"2024-05-08T18:08:00.668927+0000","last_undegraded":"2024-05-08T18:08:00.668927+0000","last_fullsized":"2024-05-08T18:08:00.668927+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":22,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"29'10","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.782661+0000","last_change":"2024-05-08T18:04:45.812258+0000","last_active":"2024-05-08T18:07:53.782661+0000","last_peered":"2024-05-08T18:07:53.782661+0000","last_clean":"2024-05-08T18:07:53.782661+0000","last_became_active":"2024-05-08T18:04:45.811921+0000","last_became_peered":"2024-05-08T18:04:45.811921+0000","last_unstale":"2024-05-08T18:07:53.782661+0000","last_undegraded":"2024-05-08T18:07:53.782661+0000","last_fullsized":"2024-05-08T18:07:53.782661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2280,"ondisk_log_size":2280,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1600,"num_read_kb":1397,"num_write":597,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1336,"num_omap_keys":6,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1491,"ondisk_log_size":1491,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":145,"num_read_kb":87,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":256,"acting":256,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705712,"num_pgs":179,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44084,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327756,"statfs":{"total":96636764160,"available":96591622144,"internally_reserved":0,"allocated":438272,"data_stored":135643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705713,"num_pgs":153,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40444,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331396,"statfs":{"total":96636764160,"available":96595349504,"internally_reserved":0,"allocated":970752,"data_stored":724153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705713,"num_pgs":157,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40464,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331376,"statfs":{"total":96636764160,"available":96595329024,"internally_reserved":0,"allocated":991232,"data_stored":724641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705713,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":409600,"data_stored":135609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T18:08:25.850 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.850 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.850 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.851 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.851 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.851 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.851 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.851 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.851 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.851 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.851 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.851 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.851 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.852 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.852 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.852 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.852 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.852 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.852 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.852 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.852 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.853 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.853 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.853 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.853 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.853 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.853 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.853 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.853 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.853 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.854 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.854 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.854 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.854 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.854 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.854 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.854 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.854 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.854 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.855 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.855 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.855 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.855 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.855 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.855 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.855 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.855 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.855 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.855 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.856 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.856 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.856 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.856 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.856 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.856 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.856 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.856 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.856 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.856 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.857 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.857 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.857 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.857 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.857 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.857 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.857 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.857 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.857 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.858 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.858 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.858 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.858 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.858 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.858 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.858 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.858 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.858 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.858 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.859 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.859 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.859 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.859 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.859 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.859 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.859 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.859 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.859 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.859 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.860 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.860 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.860 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.860 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.860 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.860 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.860 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.860 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.860 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.860 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.860 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.861 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.861 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.861 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.861 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.861 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.861 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.861 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.861 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.861 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.862 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.862 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.862 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.862 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.862 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.862 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.862 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.863 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.863 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.863 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.863 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.863 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.863 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.863 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.863 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.863 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.864 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.864 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.864 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.864 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.864 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.864 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.864 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.864 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.864 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.865 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.865 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.865 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.865 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.865 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.865 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.865 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.865 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.865 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.865 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.866 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.866 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.866 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.866 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.866 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.866 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.866 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.867 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-05-08T18:04:48.804261+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=48, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.867 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-05-08T18:04:50.810954+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=50, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.867 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.867 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.867 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.867 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.868 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.868 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.868 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.868 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.868 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.868 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.868 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.868 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.868 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.868 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.868 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.869 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.869 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.869 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.869 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.869 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.869 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.869 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.869 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.869 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.869 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.869 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.870 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.870 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.870 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.870 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.870 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.870 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.870 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.870 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.870 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.871 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:25.871 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T18:08:45.872 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T18:08:45.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.990+0000 7f66a65dd640 1 Processor -- start 2024-05-08T18:08:45.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.990+0000 7f66a65dd640 1 -- start start 2024-05-08T18:08:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.990+0000 7f66a65dd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f66a012f420 0x7f66a0131810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.990+0000 7f66a65dd640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66a012c230 con 0x7f66a012c530 2024-05-08T18:08:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.990+0000 7f66a65dd640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f66a009ad20 con 0x7f66a012f420 2024-05-08T18:08:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.990+0000 7f669ffff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f66a012c530 0x7f66a012c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:35870/0 (socket says 172.21.15.46:35870) 2024-05-08T18:08:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.990+0000 7f669ffff640 1 -- 172.21.15.46:0/477807168 learned_addr learned my addr 172.21.15.46:0/477807168 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:08:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.990+0000 7f669f7fe640 1 --2- 172.21.15.46:0/477807168 >> v2:172.21.15.46:3300/0 conn(0x7f66a012f420 0x7f66a0131810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.991+0000 7f669effd640 1 -- 172.21.15.46:0/477807168 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1674664750 0 0) 0x7f66a012c230 con 0x7f66a012c530 2024-05-08T18:08:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.991+0000 7f669effd640 1 -- 172.21.15.46:0/477807168 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6688003520 con 0x7f66a012c530 2024-05-08T18:08:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.991+0000 7f669f7fe640 1 -- 172.21.15.46:0/477807168 >> v1:172.21.15.46:6789/0 conn(0x7f66a012c530 legacy=0x7f66a012c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.991+0000 7f669f7fe640 1 -- 172.21.15.46:0/477807168 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a0073510 con 0x7f66a012f420 2024-05-08T18:08:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.992+0000 7f669f7fe640 1 --2- 172.21.15.46:0/477807168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a012f420 0x7f66a0131810 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6694009720 tx=0x7f669402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49f8e389be9706b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.992+0000 7f669effd640 1 -- 172.21.15.46:0/477807168 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66940316d0 con 0x7f66a012f420 2024-05-08T18:08:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.992+0000 7f669effd640 1 -- 172.21.15.46:0/477807168 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6694037070 con 0x7f66a012f420 2024-05-08T18:08:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.992+0000 7f669effd640 1 -- 172.21.15.46:0/477807168 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66940319c0 con 0x7f66a012f420 2024-05-08T18:08:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.992+0000 7f66a65dd640 1 -- 172.21.15.46:0/477807168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a012f420 msgr2=0x7f66a0131810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:45.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.992+0000 7f66a65dd640 1 --2- 172.21.15.46:0/477807168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a012f420 0x7f66a0131810 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6694009720 tx=0x7f669402b840 comp rx=0 tx=0).stop 2024-05-08T18:08:45.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.992+0000 7f66a65dd640 1 -- 172.21.15.46:0/477807168 shutdown_connections 2024-05-08T18:08:45.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.992+0000 7f66a65dd640 1 --2- 172.21.15.46:0/477807168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a012f420 0x7f66a0131810 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:45.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.992+0000 7f66a65dd640 1 -- 172.21.15.46:0/477807168 >> 172.21.15.46:0/477807168 conn(0x7f66a009d7a0 msgr2=0x7f66a009dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:45.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.992+0000 7f66a65dd640 1 -- 172.21.15.46:0/477807168 shutdown_connections 2024-05-08T18:08:45.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.992+0000 7f66a65dd640 1 -- 172.21.15.46:0/477807168 wait complete. 2024-05-08T18:08:45.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.993+0000 7f66a65dd640 1 Processor -- start 2024-05-08T18:08:45.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.993+0000 7f66a65dd640 1 -- start start 2024-05-08T18:08:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.994+0000 7f66a65dd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a012c530 0x7f66a01c4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.994+0000 7f66a65dd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f66a0131ed0 con 0x7f66a012c530 2024-05-08T18:08:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.994+0000 7f669ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a012c530 0x7f66a01c4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.994+0000 7f669ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a012c530 0x7f66a01c4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51834/0 (socket says 172.21.15.46:51834) 2024-05-08T18:08:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.994+0000 7f669ffff640 1 -- 172.21.15.46:0/790555406 learned_addr learned my addr 172.21.15.46:0/790555406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:08:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.994+0000 7f669ffff640 1 -- 172.21.15.46:0/790555406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a012a3e0 con 0x7f66a012c530 2024-05-08T18:08:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.994+0000 7f669ffff640 1 --2- 172.21.15.46:0/790555406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a012c530 0x7f66a01c4280 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f669000c480 tx=0x7f669000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.994+0000 7f669cff9640 1 -- 172.21.15.46:0/790555406 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6690017070 con 0x7f66a012c530 2024-05-08T18:08:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.994+0000 7f669cff9640 1 -- 172.21.15.46:0/790555406 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6690005cd0 con 0x7f66a012c530 2024-05-08T18:08:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.995+0000 7f66a65dd640 1 -- 172.21.15.46:0/790555406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a012b4f0 con 0x7f66a012c530 2024-05-08T18:08:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.995+0000 7f669cff9640 1 -- 172.21.15.46:0/790555406 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6690007200 con 0x7f66a012c530 2024-05-08T18:08:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.995+0000 7f66a65dd640 1 -- 172.21.15.46:0/790555406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a01c49a0 con 0x7f66a012c530 2024-05-08T18:08:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.995+0000 7f669cff9640 1 -- 172.21.15.46:0/790555406 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f669001a070 con 0x7f66a012c530 2024-05-08T18:08:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.995+0000 7f66a65dd640 1 -- 172.21.15.46:0/790555406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66a00640a0 con 0x7f66a012c530 2024-05-08T18:08:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.996+0000 7f669cff9640 1 --2- 172.21.15.46:0/790555406 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f668c03d2b0 0x7f668c03f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:08:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.996+0000 7f669cff9640 1 -- 172.21.15.46:0/790555406 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f6690051160 con 0x7f66a012c530 2024-05-08T18:08:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.996+0000 7f669f7fe640 1 --2- 172.21.15.46:0/790555406 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f668c03d2b0 0x7f668c03f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:08:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:45.997+0000 7f669f7fe640 1 --2- 172.21.15.46:0/790555406 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f668c03d2b0 0x7f668c03f770 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6694036000 tx=0x7f6694008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:08:45.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:46.000+0000 7f669cff9640 1 -- 172.21.15.46:0/790555406 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f6690010c00 con 0x7f66a012c530 2024-05-08T18:08:46.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:46.190+0000 7f66a65dd640 1 -- 172.21.15.46:0/790555406 --> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66a0131820 con 0x7f668c03d2b0 2024-05-08T18:08:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:46.207+0000 7f669cff9640 1 -- 172.21.15.46:0/790555406 <== mgr.4098 v2:172.21.15.50:6824/2602185226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+662112 (secure 0 0 0) 0x7f66a0131820 con 0x7f668c03d2b0 2024-05-08T18:08:46.213 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:08:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T18:08:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:46.212+0000 7f66a65dd640 1 -- 172.21.15.46:0/790555406 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f668c03d2b0 msgr2=0x7f668c03f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:46.212+0000 7f66a65dd640 1 --2- 172.21.15.46:0/790555406 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f668c03d2b0 0x7f668c03f770 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6694036000 tx=0x7f6694008030 comp rx=0 tx=0).stop 2024-05-08T18:08:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:46.212+0000 7f66a65dd640 1 -- 172.21.15.46:0/790555406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a012c530 msgr2=0x7f66a01c4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:08:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:46.212+0000 7f66a65dd640 1 --2- 172.21.15.46:0/790555406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a012c530 0x7f66a01c4280 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f669000c480 tx=0x7f669000c940 comp rx=0 tx=0).stop 2024-05-08T18:08:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:46.212+0000 7f66a65dd640 1 -- 172.21.15.46:0/790555406 shutdown_connections 2024-05-08T18:08:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:46.212+0000 7f66a65dd640 1 --2- 172.21.15.46:0/790555406 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f668c03d2b0 0x7f668c03f770 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:46.212+0000 7f66a65dd640 1 --2- 172.21.15.46:0/790555406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f66a012c530 0x7f66a01c4280 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:08:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:46.212+0000 7f66a65dd640 1 -- 172.21.15.46:0/790555406 >> 172.21.15.46:0/790555406 conn(0x7f66a009d7a0 msgr2=0x7f66a0130fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:08:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:46.212+0000 7f66a65dd640 1 -- 172.21.15.46:0/790555406 shutdown_connections 2024-05-08T18:08:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:08:46.212+0000 7f66a65dd640 1 -- 172.21.15.46:0/790555406 wait complete. 2024-05-08T18:08:46.222 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-05-08T18:08:45.742909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605383,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":5050,"num_read_kb":4636,"num_write":2119,"num_write_kb":60585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3169,"num_omap_keys":15,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4732,"ondisk_log_size":4732,"up":658,"acting":658,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":658,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":144088,"kb_used_data":2744,"kb_used_omap":6,"kb_used_meta":141305,"kb_avail":377343272,"statfs":{"total":386547056640,"available":386399510528,"internally_reserved":0,"allocated":2809856,"data_stored":1720046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":144697110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1439,"num_omap_keys":7,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004039"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.742743+0000","last_change":"2024-05-08T18:04:43.836900+0000","last_active":"2024-05-08T18:06:48.742743+0000","last_peered":"2024-05-08T18:06:48.742743+0000","last_clean":"2024-05-08T18:06:48.742743+0000","last_became_active":"2024-05-08T18:04:43.836496+0000","last_became_peered":"2024-05-08T18:04:43.836496+0000","last_unstale":"2024-05-08T18:06:48.742743+0000","last_undegraded":"2024-05-08T18:06:48.742743+0000","last_fullsized":"2024-05-08T18:06:48.742743+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'6","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.866653+0000","last_change":"2024-05-08T18:04:45.812340+0000","last_active":"2024-05-08T18:07:53.866653+0000","last_peered":"2024-05-08T18:07:53.866653+0000","last_clean":"2024-05-08T18:07:53.866653+0000","last_became_active":"2024-05-08T18:04:45.811368+0000","last_became_peered":"2024-05-08T18:04:45.811368+0000","last_unstale":"2024-05-08T18:07:53.866653+0000","last_undegraded":"2024-05-08T18:07:53.866653+0000","last_fullsized":"2024-05-08T18:07:53.866653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"29'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:50.209476+0000","last_change":"2024-05-08T18:04:43.835466+0000","last_active":"2024-05-08T18:06:50.209476+0000","last_peered":"2024-05-08T18:06:50.209476+0000","last_clean":"2024-05-08T18:06:50.209476+0000","last_became_active":"2024-05-08T18:04:43.834612+0000","last_became_peered":"2024-05-08T18:04:43.834612+0000","last_unstale":"2024-05-08T18:06:50.209476+0000","last_undegraded":"2024-05-08T18:06:50.209476+0000","last_fullsized":"2024-05-08T18:06:50.209476+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"29'2","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:40.037447+0000","last_change":"2024-05-08T18:08:40.037474+0000","last_active":"2024-05-08T18:08:40.037447+0000","last_peered":"2024-05-08T18:08:40.037447+0000","last_clean":"2024-05-08T18:08:40.037447+0000","last_became_active":"2024-05-08T18:04:45.827965+0000","last_became_peered":"2024-05-08T18:04:45.827965+0000","last_unstale":"2024-05-08T18:08:40.037447+0000","last_undegraded":"2024-05-08T18:08:40.037447+0000","last_fullsized":"2024-05-08T18:08:40.037447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:40.037352+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:40.037352+0000","last_clean_scrub_stamp":"2024-05-08T18:08:40.037352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:22:58.859282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.743553+0000","last_change":"2024-05-08T18:04:43.843806+0000","last_active":"2024-05-08T18:06:48.743553+0000","last_peered":"2024-05-08T18:06:48.743553+0000","last_clean":"2024-05-08T18:06:48.743553+0000","last_became_active":"2024-05-08T18:04:43.837456+0000","last_became_peered":"2024-05-08T18:04:43.837456+0000","last_unstale":"2024-05-08T18:06:48.743553+0000","last_undegraded":"2024-05-08T18:06:48.743553+0000","last_fullsized":"2024-05-08T18:06:48.743553+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"29'12","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070668+0000","last_change":"2024-05-08T18:04:45.825315+0000","last_active":"2024-05-08T18:07:54.070668+0000","last_peered":"2024-05-08T18:07:54.070668+0000","last_clean":"2024-05-08T18:07:54.070668+0000","last_became_active":"2024-05-08T18:04:45.825022+0000","last_became_peered":"2024-05-08T18:04:45.825022+0000","last_unstale":"2024-05-08T18:07:54.070668+0000","last_undegraded":"2024-05-08T18:07:54.070668+0000","last_fullsized":"2024-05-08T18:07:54.070668+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:40.176922+0000","last_change":"2024-05-08T18:08:40.176922+0000","last_active":"2024-05-08T18:08:40.176922+0000","last_peered":"2024-05-08T18:08:40.176922+0000","last_clean":"2024-05-08T18:08:40.176922+0000","last_became_active":"2024-05-08T18:04:43.842077+0000","last_became_peered":"2024-05-08T18:04:43.842077+0000","last_unstale":"2024-05-08T18:08:40.176922+0000","last_undegraded":"2024-05-08T18:08:40.176922+0000","last_fullsized":"2024-05-08T18:08:40.176922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:40.176797+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:40.176797+0000","last_clean_scrub_stamp":"2024-05-08T18:08:40.176797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:14.062529+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"29'16","reported_seq":71,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.041602+0000","last_change":"2024-05-08T18:04:45.824870+0000","last_active":"2024-05-08T18:07:54.041602+0000","last_peered":"2024-05-08T18:07:54.041602+0000","last_clean":"2024-05-08T18:07:54.041602+0000","last_became_active":"2024-05-08T18:04:45.824581+0000","last_became_peered":"2024-05-08T18:04:45.824581+0000","last_unstale":"2024-05-08T18:07:54.041602+0000","last_undegraded":"2024-05-08T18:07:54.041602+0000","last_fullsized":"2024-05-08T18:07:54.041602+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747778+0000","last_change":"2024-05-08T18:04:43.833461+0000","last_active":"2024-05-08T18:07:13.747778+0000","last_peered":"2024-05-08T18:07:13.747778+0000","last_clean":"2024-05-08T18:07:13.747778+0000","last_became_active":"2024-05-08T18:04:43.832722+0000","last_became_peered":"2024-05-08T18:04:43.832722+0000","last_unstale":"2024-05-08T18:07:13.747778+0000","last_undegraded":"2024-05-08T18:07:13.747778+0000","last_fullsized":"2024-05-08T18:07:13.747778+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"29'8","reported_seq":61,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:35.526773+0000","last_change":"2024-05-08T18:08:35.526800+0000","last_active":"2024-05-08T18:08:35.526773+0000","last_peered":"2024-05-08T18:08:35.526773+0000","last_clean":"2024-05-08T18:08:35.526773+0000","last_became_active":"2024-05-08T18:04:45.808383+0000","last_became_peered":"2024-05-08T18:04:45.808383+0000","last_unstale":"2024-05-08T18:08:35.526773+0000","last_undegraded":"2024-05-08T18:08:35.526773+0000","last_fullsized":"2024-05-08T18:08:35.526773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:35.526676+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:35.526676+0000","last_clean_scrub_stamp":"2024-05-08T18:08:35.526676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:24.925676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:42.091190+0000","last_change":"2024-05-08T18:08:42.091220+0000","last_active":"2024-05-08T18:08:42.091190+0000","last_peered":"2024-05-08T18:08:42.091190+0000","last_clean":"2024-05-08T18:08:42.091190+0000","last_became_active":"2024-05-08T18:04:43.840851+0000","last_became_peered":"2024-05-08T18:04:43.840851+0000","last_unstale":"2024-05-08T18:08:42.091190+0000","last_undegraded":"2024-05-08T18:08:42.091190+0000","last_fullsized":"2024-05-08T18:08:42.091190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:42.091091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:42.091091+0000","last_clean_scrub_stamp":"2024-05-08T18:08:42.091091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:24.274648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:39.004535+0000","last_change":"2024-05-08T18:08:39.004566+0000","last_active":"2024-05-08T18:08:39.004535+0000","last_peered":"2024-05-08T18:08:39.004535+0000","last_clean":"2024-05-08T18:08:39.004535+0000","last_became_active":"2024-05-08T18:04:45.828334+0000","last_became_peered":"2024-05-08T18:04:45.828334+0000","last_unstale":"2024-05-08T18:08:39.004535+0000","last_undegraded":"2024-05-08T18:08:39.004535+0000","last_fullsized":"2024-05-08T18:08:39.004535+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:39.004442+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:39.004442+0000","last_clean_scrub_stamp":"2024-05-08T18:08:39.004442+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:31:47.634365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"29'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.981231+0000","last_change":"2024-05-08T18:04:43.833826+0000","last_active":"2024-05-08T18:07:53.981231+0000","last_peered":"2024-05-08T18:07:53.981231+0000","last_clean":"2024-05-08T18:07:53.981231+0000","last_became_active":"2024-05-08T18:04:43.833437+0000","last_became_peered":"2024-05-08T18:04:43.833437+0000","last_unstale":"2024-05-08T18:07:53.981231+0000","last_undegraded":"2024-05-08T18:07:53.981231+0000","last_fullsized":"2024-05-08T18:07:53.981231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:29.307559+0000","last_change":"2024-05-08T18:08:29.307581+0000","last_active":"2024-05-08T18:08:29.307559+0000","last_peered":"2024-05-08T18:08:29.307559+0000","last_clean":"2024-05-08T18:08:29.307559+0000","last_became_active":"2024-05-08T18:04:45.811884+0000","last_became_peered":"2024-05-08T18:04:45.811884+0000","last_unstale":"2024-05-08T18:08:29.307559+0000","last_undegraded":"2024-05-08T18:08:29.307559+0000","last_fullsized":"2024-05-08T18:08:29.307559+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:29.307470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:29.307470+0000","last_clean_scrub_stamp":"2024-05-08T18:08:29.307470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:16:25.939226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.741950+0000","last_change":"2024-05-08T18:04:43.821005+0000","last_active":"2024-05-08T18:07:13.741950+0000","last_peered":"2024-05-08T18:07:13.741950+0000","last_clean":"2024-05-08T18:07:13.741950+0000","last_became_active":"2024-05-08T18:04:43.820312+0000","last_became_peered":"2024-05-08T18:04:43.820312+0000","last_unstale":"2024-05-08T18:07:13.741950+0000","last_undegraded":"2024-05-08T18:07:13.741950+0000","last_fullsized":"2024-05-08T18:07:13.741950+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"29'7","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.115771+0000","last_change":"2024-05-08T18:04:45.823448+0000","last_active":"2024-05-08T18:07:54.115771+0000","last_peered":"2024-05-08T18:07:54.115771+0000","last_clean":"2024-05-08T18:07:54.115771+0000","last_became_active":"2024-05-08T18:04:45.822620+0000","last_became_peered":"2024-05-08T18:04:45.822620+0000","last_unstale":"2024-05-08T18:07:54.115771+0000","last_undegraded":"2024-05-08T18:07:54.115771+0000","last_fullsized":"2024-05-08T18:07:54.115771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"29'4","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.002897+0000","last_change":"2024-05-08T18:04:45.826814+0000","last_active":"2024-05-08T18:07:54.002897+0000","last_peered":"2024-05-08T18:07:54.002897+0000","last_clean":"2024-05-08T18:07:54.002897+0000","last_became_active":"2024-05-08T18:04:45.826493+0000","last_became_peered":"2024-05-08T18:04:45.826493+0000","last_unstale":"2024-05-08T18:07:54.002897+0000","last_undegraded":"2024-05-08T18:07:54.002897+0000","last_fullsized":"2024-05-08T18:07:54.002897+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746269+0000","last_change":"2024-05-08T18:04:43.838700+0000","last_active":"2024-05-08T18:07:08.746269+0000","last_peered":"2024-05-08T18:07:08.746269+0000","last_clean":"2024-05-08T18:07:08.746269+0000","last_became_active":"2024-05-08T18:04:43.838332+0000","last_became_peered":"2024-05-08T18:04:43.838332+0000","last_unstale":"2024-05-08T18:07:08.746269+0000","last_undegraded":"2024-05-08T18:07:08.746269+0000","last_fullsized":"2024-05-08T18:07:08.746269+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"29'4","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:36.520147+0000","last_change":"2024-05-08T18:08:36.520179+0000","last_active":"2024-05-08T18:08:36.520147+0000","last_peered":"2024-05-08T18:08:36.520147+0000","last_clean":"2024-05-08T18:08:36.520147+0000","last_became_active":"2024-05-08T18:04:45.807438+0000","last_became_peered":"2024-05-08T18:04:45.807438+0000","last_unstale":"2024-05-08T18:08:36.520147+0000","last_undegraded":"2024-05-08T18:08:36.520147+0000","last_fullsized":"2024-05-08T18:08:36.520147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:36.520045+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:36.520045+0000","last_clean_scrub_stamp":"2024-05-08T18:08:36.520045+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:18.703602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:30.298724+0000","last_change":"2024-05-08T18:08:30.298757+0000","last_active":"2024-05-08T18:08:30.298724+0000","last_peered":"2024-05-08T18:08:30.298724+0000","last_clean":"2024-05-08T18:08:30.298724+0000","last_became_active":"2024-05-08T18:04:43.842741+0000","last_became_peered":"2024-05-08T18:04:43.842741+0000","last_unstale":"2024-05-08T18:08:30.298724+0000","last_undegraded":"2024-05-08T18:08:30.298724+0000","last_fullsized":"2024-05-08T18:08:30.298724+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:30.298603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:30.298603+0000","last_clean_scrub_stamp":"2024-05-08T18:08:30.298603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:30:25.561243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"29'12","reported_seq":67,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:43.533148+0000","last_change":"2024-05-08T18:08:43.533178+0000","last_active":"2024-05-08T18:08:43.533148+0000","last_peered":"2024-05-08T18:08:43.533148+0000","last_clean":"2024-05-08T18:08:43.533148+0000","last_became_active":"2024-05-08T18:04:45.810827+0000","last_became_peered":"2024-05-08T18:04:45.810827+0000","last_unstale":"2024-05-08T18:08:43.533148+0000","last_undegraded":"2024-05-08T18:08:43.533148+0000","last_fullsized":"2024-05-08T18:08:43.533148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:43.533034+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:43.533034+0000","last_clean_scrub_stamp":"2024-05-08T18:08:43.533034+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:04.129239+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"29'4","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.670348+0000","last_change":"2024-05-08T18:04:43.838115+0000","last_active":"2024-05-08T18:08:00.670348+0000","last_peered":"2024-05-08T18:08:00.670348+0000","last_clean":"2024-05-08T18:08:00.670348+0000","last_became_active":"2024-05-08T18:04:43.837706+0000","last_became_peered":"2024-05-08T18:04:43.837706+0000","last_unstale":"2024-05-08T18:08:00.670348+0000","last_undegraded":"2024-05-08T18:08:00.670348+0000","last_fullsized":"2024-05-08T18:08:00.670348+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"29'8","reported_seq":68,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:32.455142+0000","last_change":"2024-05-08T18:08:32.455190+0000","last_active":"2024-05-08T18:08:32.455142+0000","last_peered":"2024-05-08T18:08:32.455142+0000","last_clean":"2024-05-08T18:08:32.455142+0000","last_became_active":"2024-05-08T18:04:45.822000+0000","last_became_peered":"2024-05-08T18:04:45.822000+0000","last_unstale":"2024-05-08T18:08:32.455142+0000","last_undegraded":"2024-05-08T18:08:32.455142+0000","last_fullsized":"2024-05-08T18:08:32.455142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:32.455035+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:32.455035+0000","last_clean_scrub_stamp":"2024-05-08T18:08:32.455035+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:21.854035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:40.545746+0000","last_change":"2024-05-08T18:08:40.545772+0000","last_active":"2024-05-08T18:08:40.545746+0000","last_peered":"2024-05-08T18:08:40.545746+0000","last_clean":"2024-05-08T18:08:40.545746+0000","last_became_active":"2024-05-08T18:04:43.841063+0000","last_became_peered":"2024-05-08T18:04:43.841063+0000","last_unstale":"2024-05-08T18:08:40.545746+0000","last_undegraded":"2024-05-08T18:08:40.545746+0000","last_fullsized":"2024-05-08T18:08:40.545746+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:40.545644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:40.545644+0000","last_clean_scrub_stamp":"2024-05-08T18:08:40.545644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:55:34.045722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"29'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:26.277429+0000","last_change":"2024-05-08T18:08:26.277484+0000","last_active":"2024-05-08T18:08:26.277429+0000","last_peered":"2024-05-08T18:08:26.277429+0000","last_clean":"2024-05-08T18:08:26.277429+0000","last_became_active":"2024-05-08T18:04:45.823611+0000","last_became_peered":"2024-05-08T18:04:45.823611+0000","last_unstale":"2024-05-08T18:08:26.277429+0000","last_undegraded":"2024-05-08T18:08:26.277429+0000","last_fullsized":"2024-05-08T18:08:26.277429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:26.277327+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:26.277327+0000","last_clean_scrub_stamp":"2024-05-08T18:08:26.277327+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:13.871401+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.741273+0000","last_change":"2024-05-08T18:04:43.835685+0000","last_active":"2024-05-08T18:07:08.741273+0000","last_peered":"2024-05-08T18:07:08.741273+0000","last_clean":"2024-05-08T18:07:08.741273+0000","last_became_active":"2024-05-08T18:04:43.834456+0000","last_became_peered":"2024-05-08T18:04:43.834456+0000","last_unstale":"2024-05-08T18:07:08.741273+0000","last_undegraded":"2024-05-08T18:07:08.741273+0000","last_fullsized":"2024-05-08T18:07:08.741273+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"29'6","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:37.506169+0000","last_change":"2024-05-08T18:08:37.506219+0000","last_active":"2024-05-08T18:08:37.506169+0000","last_peered":"2024-05-08T18:08:37.506169+0000","last_clean":"2024-05-08T18:08:37.506169+0000","last_became_active":"2024-05-08T18:04:45.808416+0000","last_became_peered":"2024-05-08T18:04:45.808416+0000","last_unstale":"2024-05-08T18:08:37.506169+0000","last_undegraded":"2024-05-08T18:08:37.506169+0000","last_fullsized":"2024-05-08T18:08:37.506169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:37.506056+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:37.506056+0000","last_clean_scrub_stamp":"2024-05-08T18:08:37.506056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:14.952253+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'4","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745942+0000","last_change":"2024-05-08T18:04:43.845458+0000","last_active":"2024-05-08T18:07:03.745942+0000","last_peered":"2024-05-08T18:07:03.745942+0000","last_clean":"2024-05-08T18:07:03.745942+0000","last_became_active":"2024-05-08T18:04:43.844702+0000","last_became_peered":"2024-05-08T18:04:43.844702+0000","last_unstale":"2024-05-08T18:07:03.745942+0000","last_undegraded":"2024-05-08T18:07:03.745942+0000","last_fullsized":"2024-05-08T18:07:03.745942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"29'35","reported_seq":100,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.080132+0000","last_change":"2024-05-08T18:04:53.849775+0000","last_active":"2024-05-08T18:07:54.080132+0000","last_peered":"2024-05-08T18:07:54.080132+0000","last_clean":"2024-05-08T18:07:54.080132+0000","last_became_active":"2024-05-08T18:04:53.849475+0000","last_became_peered":"2024-05-08T18:04:53.849475+0000","last_unstale":"2024-05-08T18:07:54.080132+0000","last_undegraded":"2024-05-08T18:07:54.080132+0000","last_fullsized":"2024-05-08T18:07:54.080132+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"29'12","reported_seq":67,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.002987+0000","last_change":"2024-05-08T18:04:45.824289+0000","last_active":"2024-05-08T18:07:54.002987+0000","last_peered":"2024-05-08T18:07:54.002987+0000","last_clean":"2024-05-08T18:07:54.002987+0000","last_became_active":"2024-05-08T18:04:45.824005+0000","last_became_peered":"2024-05-08T18:04:45.824005+0000","last_unstale":"2024-05-08T18:07:54.002987+0000","last_undegraded":"2024-05-08T18:07:54.002987+0000","last_fullsized":"2024-05-08T18:07:54.002987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745982+0000","last_change":"2024-05-08T18:04:43.834249+0000","last_active":"2024-05-08T18:07:03.745982+0000","last_peered":"2024-05-08T18:07:03.745982+0000","last_clean":"2024-05-08T18:07:03.745982+0000","last_became_active":"2024-05-08T18:04:43.833822+0000","last_became_peered":"2024-05-08T18:04:43.833822+0000","last_unstale":"2024-05-08T18:07:03.745982+0000","last_undegraded":"2024-05-08T18:07:03.745982+0000","last_fullsized":"2024-05-08T18:07:03.745982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.924513+0000","last_change":"2024-05-08T18:08:09.924538+0000","last_active":"2024-05-08T18:08:09.924513+0000","last_peered":"2024-05-08T18:08:09.924513+0000","last_clean":"2024-05-08T18:08:09.924513+0000","last_became_active":"2024-05-08T18:04:47.819682+0000","last_became_peered":"2024-05-08T18:04:47.819682+0000","last_unstale":"2024-05-08T18:08:09.924513+0000","last_undegraded":"2024-05-08T18:08:09.924513+0000","last_fullsized":"2024-05-08T18:08:09.924513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:09.924406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:09.924406+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.924406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:58.088967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"29'39","reported_seq":94,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:31.864262+0000","last_change":"2024-05-08T18:08:31.864298+0000","last_active":"2024-05-08T18:08:31.864262+0000","last_peered":"2024-05-08T18:08:31.864262+0000","last_clean":"2024-05-08T18:08:31.864262+0000","last_became_active":"2024-05-08T18:04:49.832148+0000","last_became_peered":"2024-05-08T18:04:49.832148+0000","last_unstale":"2024-05-08T18:08:31.864262+0000","last_undegraded":"2024-05-08T18:08:31.864262+0000","last_fullsized":"2024-05-08T18:08:31.864262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'39","last_scrub_stamp":"2024-05-08T18:08:31.864172+0000","last_deep_scrub":"29'39","last_deep_scrub_stamp":"2024-05-08T18:08:31.864172+0000","last_clean_scrub_stamp":"2024-05-08T18:08:31.864172+0000","objects_scrubbed":4,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:47:21.937257+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":40,"num_read_kb":42,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.510810+0000","last_change":"2024-05-08T18:08:21.510876+0000","last_active":"2024-05-08T18:08:21.510810+0000","last_peered":"2024-05-08T18:08:21.510810+0000","last_clean":"2024-05-08T18:08:21.510810+0000","last_became_active":"2024-05-08T18:04:51.844901+0000","last_became_peered":"2024-05-08T18:04:51.844901+0000","last_unstale":"2024-05-08T18:08:21.510810+0000","last_undegraded":"2024-05-08T18:08:21.510810+0000","last_fullsized":"2024-05-08T18:08:21.510810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:21.510687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:21.510687+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.510687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:55:15.292759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"29'11","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.747622+0000","last_change":"2024-05-08T18:04:53.849933+0000","last_active":"2024-05-08T18:07:18.747622+0000","last_peered":"2024-05-08T18:07:18.747622+0000","last_clean":"2024-05-08T18:07:18.747622+0000","last_became_active":"2024-05-08T18:04:53.849579+0000","last_became_peered":"2024-05-08T18:04:53.849579+0000","last_unstale":"2024-05-08T18:07:18.747622+0000","last_undegraded":"2024-05-08T18:07:18.747622+0000","last_fullsized":"2024-05-08T18:07:18.747622+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"29'6","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070924+0000","last_change":"2024-05-08T18:04:45.826976+0000","last_active":"2024-05-08T18:07:54.070924+0000","last_peered":"2024-05-08T18:07:54.070924+0000","last_clean":"2024-05-08T18:07:54.070924+0000","last_became_active":"2024-05-08T18:04:45.825162+0000","last_became_peered":"2024-05-08T18:04:45.825162+0000","last_unstale":"2024-05-08T18:07:54.070924+0000","last_undegraded":"2024-05-08T18:07:54.070924+0000","last_fullsized":"2024-05-08T18:07:54.070924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745158+0000","last_change":"2024-05-08T18:04:43.827844+0000","last_active":"2024-05-08T18:07:03.745158+0000","last_peered":"2024-05-08T18:07:03.745158+0000","last_clean":"2024-05-08T18:07:03.745158+0000","last_became_active":"2024-05-08T18:04:43.827374+0000","last_became_peered":"2024-05-08T18:04:43.827374+0000","last_unstale":"2024-05-08T18:07:03.745158+0000","last_undegraded":"2024-05-08T18:07:03.745158+0000","last_fullsized":"2024-05-08T18:07:03.745158+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.752839+0000","last_change":"2024-05-08T18:05:30.868535+0000","last_active":"2024-05-08T18:07:53.752839+0000","last_peered":"2024-05-08T18:07:53.752839+0000","last_clean":"2024-05-08T18:07:53.752839+0000","last_became_active":"2024-05-08T18:05:30.868191+0000","last_became_peered":"2024-05-08T18:05:30.868191+0000","last_unstale":"2024-05-08T18:07:53.752839+0000","last_undegraded":"2024-05-08T18:07:53.752839+0000","last_fullsized":"2024-05-08T18:07:53.752839+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"29'9","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.529952+0000","last_change":"2024-05-08T18:08:10.529988+0000","last_active":"2024-05-08T18:08:10.529952+0000","last_peered":"2024-05-08T18:08:10.529952+0000","last_clean":"2024-05-08T18:08:10.529952+0000","last_became_active":"2024-05-08T18:04:49.825265+0000","last_became_peered":"2024-05-08T18:04:49.825265+0000","last_unstale":"2024-05-08T18:08:10.529952+0000","last_undegraded":"2024-05-08T18:08:10.529952+0000","last_fullsized":"2024-05-08T18:08:10.529952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'9","last_scrub_stamp":"2024-05-08T18:08:10.529858+0000","last_deep_scrub":"29'9","last_deep_scrub_stamp":"2024-05-08T18:08:10.529858+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.529858+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:00.215888+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.491557+0000","last_change":"2024-05-08T18:08:13.491631+0000","last_active":"2024-05-08T18:08:13.491557+0000","last_peered":"2024-05-08T18:08:13.491557+0000","last_clean":"2024-05-08T18:08:13.491557+0000","last_became_active":"2024-05-08T18:04:51.833523+0000","last_became_peered":"2024-05-08T18:04:51.833523+0000","last_unstale":"2024-05-08T18:08:13.491557+0000","last_undegraded":"2024-05-08T18:08:13.491557+0000","last_fullsized":"2024-05-08T18:08:13.491557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:13.491458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:13.491458+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.491458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:07.228339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'2","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:33.477085+0000","last_change":"2024-05-08T18:08:33.477157+0000","last_active":"2024-05-08T18:08:33.477085+0000","last_peered":"2024-05-08T18:08:33.477085+0000","last_clean":"2024-05-08T18:08:33.477085+0000","last_became_active":"2024-05-08T18:04:53.840395+0000","last_became_peered":"2024-05-08T18:04:53.840395+0000","last_unstale":"2024-05-08T18:08:33.477085+0000","last_undegraded":"2024-05-08T18:08:33.477085+0000","last_fullsized":"2024-05-08T18:08:33.477085+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:33.476981+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:33.476981+0000","last_clean_scrub_stamp":"2024-05-08T18:08:33.476981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:15.660538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:58.753460+0000","last_change":"2024-05-08T18:05:30.876470+0000","last_active":"2024-05-08T18:07:58.753460+0000","last_peered":"2024-05-08T18:07:58.753460+0000","last_clean":"2024-05-08T18:07:58.753460+0000","last_became_active":"2024-05-08T18:05:30.869162+0000","last_became_peered":"2024-05-08T18:05:30.869162+0000","last_unstale":"2024-05-08T18:07:58.753460+0000","last_undegraded":"2024-05-08T18:07:58.753460+0000","last_fullsized":"2024-05-08T18:07:58.753460+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":45,"num_write":24,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745862+0000","last_change":"2024-05-08T18:04:43.844847+0000","last_active":"2024-05-08T18:07:03.745862+0000","last_peered":"2024-05-08T18:07:03.745862+0000","last_clean":"2024-05-08T18:07:03.745862+0000","last_became_active":"2024-05-08T18:04:43.838666+0000","last_became_peered":"2024-05-08T18:04:43.838666+0000","last_unstale":"2024-05-08T18:07:03.745862+0000","last_undegraded":"2024-05-08T18:07:03.745862+0000","last_fullsized":"2024-05-08T18:07:03.745862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:27.491659+0000","last_change":"2024-05-08T18:08:27.491684+0000","last_active":"2024-05-08T18:08:27.491659+0000","last_peered":"2024-05-08T18:08:27.491659+0000","last_clean":"2024-05-08T18:08:27.491659+0000","last_became_active":"2024-05-08T18:04:47.819169+0000","last_became_peered":"2024-05-08T18:04:47.819169+0000","last_unstale":"2024-05-08T18:08:27.491659+0000","last_undegraded":"2024-05-08T18:08:27.491659+0000","last_fullsized":"2024-05-08T18:08:27.491659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:27.491574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:27.491574+0000","last_clean_scrub_stamp":"2024-05-08T18:08:27.491574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:24.380337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"29'17","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.168244+0000","last_change":"2024-05-08T18:08:11.168281+0000","last_active":"2024-05-08T18:08:11.168244+0000","last_peered":"2024-05-08T18:08:11.168244+0000","last_clean":"2024-05-08T18:08:11.168244+0000","last_became_active":"2024-05-08T18:04:49.831500+0000","last_became_peered":"2024-05-08T18:04:49.831500+0000","last_unstale":"2024-05-08T18:08:11.168244+0000","last_undegraded":"2024-05-08T18:08:11.168244+0000","last_fullsized":"2024-05-08T18:08:11.168244+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'17","last_scrub_stamp":"2024-05-08T18:08:11.168144+0000","last_deep_scrub":"29'17","last_deep_scrub_stamp":"2024-05-08T18:08:11.168144+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.168144+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:03.071434+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"29'286","reported_seq":394,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:41.051149+0000","last_change":"2024-05-08T18:08:41.051182+0000","last_active":"2024-05-08T18:08:41.051149+0000","last_peered":"2024-05-08T18:08:41.051149+0000","last_clean":"2024-05-08T18:08:41.051149+0000","last_became_active":"2024-05-08T18:04:51.849301+0000","last_became_peered":"2024-05-08T18:04:51.849301+0000","last_unstale":"2024-05-08T18:08:41.051149+0000","last_undegraded":"2024-05-08T18:08:41.051149+0000","last_fullsized":"2024-05-08T18:08:41.051149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'286","last_scrub_stamp":"2024-05-08T18:08:41.051045+0000","last_deep_scrub":"29'286","last_deep_scrub_stamp":"2024-05-08T18:08:41.051045+0000","last_clean_scrub_stamp":"2024-05-08T18:08:41.051045+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:35:15.383617+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:23.748336+0000","last_change":"2024-05-08T18:04:53.840922+0000","last_active":"2024-05-08T18:07:23.748336+0000","last_peered":"2024-05-08T18:07:23.748336+0000","last_clean":"2024-05-08T18:07:23.748336+0000","last_became_active":"2024-05-08T18:04:53.840559+0000","last_became_peered":"2024-05-08T18:04:53.840559+0000","last_unstale":"2024-05-08T18:07:23.748336+0000","last_undegraded":"2024-05-08T18:07:23.748336+0000","last_fullsized":"2024-05-08T18:07:23.748336+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"29'6","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.041576+0000","last_change":"2024-05-08T18:04:45.812045+0000","last_active":"2024-05-08T18:07:54.041576+0000","last_peered":"2024-05-08T18:07:54.041576+0000","last_clean":"2024-05-08T18:07:54.041576+0000","last_became_active":"2024-05-08T18:04:45.811570+0000","last_became_peered":"2024-05-08T18:04:45.811570+0000","last_unstale":"2024-05-08T18:07:54.041576+0000","last_undegraded":"2024-05-08T18:07:54.041576+0000","last_fullsized":"2024-05-08T18:07:54.041576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"29'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.990442+0000","last_change":"2024-05-08T18:04:43.838356+0000","last_active":"2024-05-08T18:07:53.990442+0000","last_peered":"2024-05-08T18:07:53.990442+0000","last_clean":"2024-05-08T18:07:53.990442+0000","last_became_active":"2024-05-08T18:04:43.836410+0000","last_became_peered":"2024-05-08T18:04:43.836410+0000","last_unstale":"2024-05-08T18:07:53.990442+0000","last_undegraded":"2024-05-08T18:07:53.990442+0000","last_fullsized":"2024-05-08T18:07:53.990442+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.849059+0000","last_change":"2024-05-08T18:08:11.849101+0000","last_active":"2024-05-08T18:08:11.849059+0000","last_peered":"2024-05-08T18:08:11.849059+0000","last_clean":"2024-05-08T18:08:11.849059+0000","last_became_active":"2024-05-08T18:04:47.818797+0000","last_became_peered":"2024-05-08T18:04:47.818797+0000","last_unstale":"2024-05-08T18:08:11.849059+0000","last_undegraded":"2024-05-08T18:08:11.849059+0000","last_fullsized":"2024-05-08T18:08:11.849059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:11.848881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:11.848881+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.848881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:19.374213+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"29'37","reported_seq":92,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.217059+0000","last_change":"2024-05-08T18:08:12.217092+0000","last_active":"2024-05-08T18:08:12.217059+0000","last_peered":"2024-05-08T18:08:12.217059+0000","last_clean":"2024-05-08T18:08:12.217059+0000","last_became_active":"2024-05-08T18:04:49.831379+0000","last_became_peered":"2024-05-08T18:04:49.831379+0000","last_unstale":"2024-05-08T18:08:12.217059+0000","last_undegraded":"2024-05-08T18:08:12.217059+0000","last_fullsized":"2024-05-08T18:08:12.217059+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'37","last_scrub_stamp":"2024-05-08T18:08:12.216965+0000","last_deep_scrub":"29'37","last_deep_scrub_stamp":"2024-05-08T18:08:12.216965+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.216965+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:49:34.759465+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"29'285","reported_seq":453,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.529055+0000","last_change":"2024-05-08T18:08:11.529087+0000","last_active":"2024-05-08T18:08:11.529055+0000","last_peered":"2024-05-08T18:08:11.529055+0000","last_clean":"2024-05-08T18:08:11.529055+0000","last_became_active":"2024-05-08T18:04:51.826926+0000","last_became_peered":"2024-05-08T18:04:51.826926+0000","last_unstale":"2024-05-08T18:08:11.529055+0000","last_undegraded":"2024-05-08T18:08:11.529055+0000","last_fullsized":"2024-05-08T18:08:11.529055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:11.528959+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:11.528959+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.528959+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:44.650679+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'6","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:42.053526+0000","last_change":"2024-05-08T18:08:42.053554+0000","last_active":"2024-05-08T18:08:42.053526+0000","last_peered":"2024-05-08T18:08:42.053526+0000","last_clean":"2024-05-08T18:08:42.053526+0000","last_became_active":"2024-05-08T18:04:45.827351+0000","last_became_peered":"2024-05-08T18:04:45.827351+0000","last_unstale":"2024-05-08T18:08:42.053526+0000","last_undegraded":"2024-05-08T18:08:42.053526+0000","last_fullsized":"2024-05-08T18:08:42.053526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-05-08T18:08:42.053430+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-05-08T18:08:42.053430+0000","last_clean_scrub_stamp":"2024-05-08T18:08:42.053430+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:29.385518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744399+0000","last_change":"2024-05-08T18:04:43.839205+0000","last_active":"2024-05-08T18:06:58.744399+0000","last_peered":"2024-05-08T18:06:58.744399+0000","last_clean":"2024-05-08T18:06:58.744399+0000","last_became_active":"2024-05-08T18:04:43.838840+0000","last_became_peered":"2024-05-08T18:04:43.838840+0000","last_unstale":"2024-05-08T18:06:58.744399+0000","last_undegraded":"2024-05-08T18:06:58.744399+0000","last_fullsized":"2024-05-08T18:06:58.744399+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,0],"acting":[2,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.468389+0000","last_change":"2024-05-08T18:08:14.468441+0000","last_active":"2024-05-08T18:08:14.468389+0000","last_peered":"2024-05-08T18:08:14.468389+0000","last_clean":"2024-05-08T18:08:14.468389+0000","last_became_active":"2024-05-08T18:04:47.825215+0000","last_became_peered":"2024-05-08T18:04:47.825215+0000","last_unstale":"2024-05-08T18:08:14.468389+0000","last_undegraded":"2024-05-08T18:08:14.468389+0000","last_fullsized":"2024-05-08T18:08:14.468389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:14.468285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:14.468285+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.468285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:38.045371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"29'51","reported_seq":111,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.277773+0000","last_change":"2024-05-08T18:08:14.277853+0000","last_active":"2024-05-08T18:08:14.277773+0000","last_peered":"2024-05-08T18:08:14.277773+0000","last_clean":"2024-05-08T18:08:14.277773+0000","last_became_active":"2024-05-08T18:04:49.827954+0000","last_became_peered":"2024-05-08T18:04:49.827954+0000","last_unstale":"2024-05-08T18:08:14.277773+0000","last_undegraded":"2024-05-08T18:08:14.277773+0000","last_fullsized":"2024-05-08T18:08:14.277773+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'51","last_scrub_stamp":"2024-05-08T18:08:14.277647+0000","last_deep_scrub":"29'51","last_deep_scrub_stamp":"2024-05-08T18:08:14.277647+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.277647+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:23:20.826582+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.436717+0000","last_change":"2024-05-08T18:08:07.436758+0000","last_active":"2024-05-08T18:08:07.436717+0000","last_peered":"2024-05-08T18:08:07.436717+0000","last_clean":"2024-05-08T18:08:07.436717+0000","last_became_active":"2024-05-08T18:04:51.831149+0000","last_became_peered":"2024-05-08T18:04:51.831149+0000","last_unstale":"2024-05-08T18:08:07.436717+0000","last_undegraded":"2024-05-08T18:08:07.436717+0000","last_fullsized":"2024-05-08T18:08:07.436717+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:07.436627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:07.436627+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.436627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:56:38.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"29'5","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744266+0000","last_change":"2024-05-08T18:04:53.840960+0000","last_active":"2024-05-08T18:06:58.744266+0000","last_peered":"2024-05-08T18:06:58.744266+0000","last_clean":"2024-05-08T18:06:58.744266+0000","last_became_active":"2024-05-08T18:04:53.840652+0000","last_became_peered":"2024-05-08T18:04:53.840652+0000","last_unstale":"2024-05-08T18:06:58.744266+0000","last_undegraded":"2024-05-08T18:06:58.744266+0000","last_fullsized":"2024-05-08T18:06:58.744266+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"27'4","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:38.499682+0000","last_change":"2024-05-08T18:08:38.499721+0000","last_active":"2024-05-08T18:08:38.499682+0000","last_peered":"2024-05-08T18:08:38.499682+0000","last_clean":"2024-05-08T18:08:38.499682+0000","last_became_active":"2024-05-08T18:04:45.804124+0000","last_became_peered":"2024-05-08T18:04:45.804124+0000","last_unstale":"2024-05-08T18:08:38.499682+0000","last_undegraded":"2024-05-08T18:08:38.499682+0000","last_fullsized":"2024-05-08T18:08:38.499682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-08T18:08:38.499539+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-08T18:08:38.499539+0000","last_clean_scrub_stamp":"2024-05-08T18:08:38.499539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:58.286733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:36.190485+0000","last_change":"2024-05-08T18:08:36.190511+0000","last_active":"2024-05-08T18:08:36.190485+0000","last_peered":"2024-05-08T18:08:36.190485+0000","last_clean":"2024-05-08T18:08:36.190485+0000","last_became_active":"2024-05-08T18:04:43.842126+0000","last_became_peered":"2024-05-08T18:04:43.842126+0000","last_unstale":"2024-05-08T18:08:36.190485+0000","last_undegraded":"2024-05-08T18:08:36.190485+0000","last_fullsized":"2024-05-08T18:08:36.190485+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:36.190391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:36.190391+0000","last_clean_scrub_stamp":"2024-05-08T18:08:36.190391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:10:38.441913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.583926+0000","last_change":"2024-05-08T18:08:22.583963+0000","last_active":"2024-05-08T18:08:22.583926+0000","last_peered":"2024-05-08T18:08:22.583926+0000","last_clean":"2024-05-08T18:08:22.583926+0000","last_became_active":"2024-05-08T18:04:47.822688+0000","last_became_peered":"2024-05-08T18:04:47.822688+0000","last_unstale":"2024-05-08T18:08:22.583926+0000","last_undegraded":"2024-05-08T18:08:22.583926+0000","last_fullsized":"2024-05-08T18:08:22.583926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.583822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.583822+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.583822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:22.094540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"29'36","reported_seq":91,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.839677+0000","last_change":"2024-05-08T18:08:16.839747+0000","last_active":"2024-05-08T18:08:16.839677+0000","last_peered":"2024-05-08T18:08:16.839677+0000","last_clean":"2024-05-08T18:08:16.839677+0000","last_became_active":"2024-05-08T18:04:49.829892+0000","last_became_peered":"2024-05-08T18:04:49.829892+0000","last_unstale":"2024-05-08T18:08:16.839677+0000","last_undegraded":"2024-05-08T18:08:16.839677+0000","last_fullsized":"2024-05-08T18:08:16.839677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:08:16.839580+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:08:16.839580+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.839580+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:53.387986+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:04.455783+0000","last_change":"2024-05-08T18:08:04.455804+0000","last_active":"2024-05-08T18:08:04.455783+0000","last_peered":"2024-05-08T18:08:04.455783+0000","last_clean":"2024-05-08T18:08:04.455783+0000","last_became_active":"2024-05-08T18:04:51.832810+0000","last_became_peered":"2024-05-08T18:04:51.832810+0000","last_unstale":"2024-05-08T18:08:04.455783+0000","last_undegraded":"2024-05-08T18:08:04.455783+0000","last_fullsized":"2024-05-08T18:08:04.455783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:04.455699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:04.455699+0000","last_clean_scrub_stamp":"2024-05-08T18:08:04.455699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:21.455723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"29'4","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:29.520453+0000","last_change":"2024-05-08T18:08:29.520481+0000","last_active":"2024-05-08T18:08:29.520453+0000","last_peered":"2024-05-08T18:08:29.520453+0000","last_clean":"2024-05-08T18:08:29.520453+0000","last_became_active":"2024-05-08T18:04:53.840894+0000","last_became_peered":"2024-05-08T18:04:53.840894+0000","last_unstale":"2024-05-08T18:08:29.520453+0000","last_undegraded":"2024-05-08T18:08:29.520453+0000","last_fullsized":"2024-05-08T18:08:29.520453+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:29.520364+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:29.520364+0000","last_clean_scrub_stamp":"2024-05-08T18:08:29.520364+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:53.170014+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"29'10","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:37.218745+0000","last_change":"2024-05-08T18:08:37.218777+0000","last_active":"2024-05-08T18:08:37.218745+0000","last_peered":"2024-05-08T18:08:37.218745+0000","last_clean":"2024-05-08T18:08:37.218745+0000","last_became_active":"2024-05-08T18:04:45.816817+0000","last_became_peered":"2024-05-08T18:04:45.816817+0000","last_unstale":"2024-05-08T18:08:37.218745+0000","last_undegraded":"2024-05-08T18:08:37.218745+0000","last_fullsized":"2024-05-08T18:08:37.218745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:37.218630+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:37.218630+0000","last_clean_scrub_stamp":"2024-05-08T18:08:37.218630+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:39.786476+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"29'4","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.668918+0000","last_change":"2024-05-08T18:04:43.844652+0000","last_active":"2024-05-08T18:08:00.668918+0000","last_peered":"2024-05-08T18:08:00.668918+0000","last_clean":"2024-05-08T18:08:00.668918+0000","last_became_active":"2024-05-08T18:04:43.838781+0000","last_became_peered":"2024-05-08T18:04:43.838781+0000","last_unstale":"2024-05-08T18:08:00.668918+0000","last_undegraded":"2024-05-08T18:08:00.668918+0000","last_fullsized":"2024-05-08T18:08:00.668918+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:30.495270+0000","last_change":"2024-05-08T18:08:30.495329+0000","last_active":"2024-05-08T18:08:30.495270+0000","last_peered":"2024-05-08T18:08:30.495270+0000","last_clean":"2024-05-08T18:08:30.495270+0000","last_became_active":"2024-05-08T18:04:47.818871+0000","last_became_peered":"2024-05-08T18:04:47.818871+0000","last_unstale":"2024-05-08T18:08:30.495270+0000","last_undegraded":"2024-05-08T18:08:30.495270+0000","last_fullsized":"2024-05-08T18:08:30.495270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:30.495168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:30.495168+0000","last_clean_scrub_stamp":"2024-05-08T18:08:30.495168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:26.778212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"29'45","reported_seq":103,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.892863+0000","last_change":"2024-05-08T18:08:08.892899+0000","last_active":"2024-05-08T18:08:08.892863+0000","last_peered":"2024-05-08T18:08:08.892863+0000","last_clean":"2024-05-08T18:08:08.892863+0000","last_became_active":"2024-05-08T18:04:49.829270+0000","last_became_peered":"2024-05-08T18:04:49.829270+0000","last_unstale":"2024-05-08T18:08:08.892863+0000","last_undegraded":"2024-05-08T18:08:08.892863+0000","last_fullsized":"2024-05-08T18:08:08.892863+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'45","last_scrub_stamp":"2024-05-08T18:08:08.892763+0000","last_deep_scrub":"29'45","last_deep_scrub_stamp":"2024-05-08T18:08:08.892763+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.892763+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:11.460610+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"29'285","reported_seq":360,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.575979+0000","last_change":"2024-05-08T18:08:13.576007+0000","last_active":"2024-05-08T18:08:13.575979+0000","last_peered":"2024-05-08T18:08:13.575979+0000","last_clean":"2024-05-08T18:08:13.575979+0000","last_became_active":"2024-05-08T18:04:51.844529+0000","last_became_peered":"2024-05-08T18:04:51.844529+0000","last_unstale":"2024-05-08T18:08:13.575979+0000","last_undegraded":"2024-05-08T18:08:13.575979+0000","last_fullsized":"2024-05-08T18:08:13.575979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:13.575886+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:13.575886+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.575886+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:46.724169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"29'2","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.868126+0000","last_change":"2024-05-08T18:04:53.841062+0000","last_active":"2024-05-08T18:07:53.868126+0000","last_peered":"2024-05-08T18:07:53.868126+0000","last_clean":"2024-05-08T18:07:53.868126+0000","last_became_active":"2024-05-08T18:04:53.840767+0000","last_became_peered":"2024-05-08T18:04:53.840767+0000","last_unstale":"2024-05-08T18:07:53.868126+0000","last_undegraded":"2024-05-08T18:07:53.868126+0000","last_fullsized":"2024-05-08T18:07:53.868126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"29'6","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:35.195828+0000","last_change":"2024-05-08T18:08:35.195855+0000","last_active":"2024-05-08T18:08:35.195828+0000","last_peered":"2024-05-08T18:08:35.195828+0000","last_clean":"2024-05-08T18:08:35.195828+0000","last_became_active":"2024-05-08T18:04:45.813735+0000","last_became_peered":"2024-05-08T18:04:45.813735+0000","last_unstale":"2024-05-08T18:08:35.195828+0000","last_undegraded":"2024-05-08T18:08:35.195828+0000","last_fullsized":"2024-05-08T18:08:35.195828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:35.195741+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:35.195741+0000","last_clean_scrub_stamp":"2024-05-08T18:08:35.195741+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:44:58.936221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"27'4","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.745008+0000","last_change":"2024-05-08T18:04:43.848140+0000","last_active":"2024-05-08T18:06:58.745008+0000","last_peered":"2024-05-08T18:06:58.745008+0000","last_clean":"2024-05-08T18:06:58.745008+0000","last_became_active":"2024-05-08T18:04:43.839047+0000","last_became_peered":"2024-05-08T18:04:43.839047+0000","last_unstale":"2024-05-08T18:06:58.745008+0000","last_undegraded":"2024-05-08T18:06:58.745008+0000","last_fullsized":"2024-05-08T18:06:58.745008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:28.472742+0000","last_change":"2024-05-08T18:08:28.472766+0000","last_active":"2024-05-08T18:08:28.472742+0000","last_peered":"2024-05-08T18:08:28.472742+0000","last_clean":"2024-05-08T18:08:28.472742+0000","last_became_active":"2024-05-08T18:04:47.818972+0000","last_became_peered":"2024-05-08T18:04:47.818972+0000","last_unstale":"2024-05-08T18:08:28.472742+0000","last_undegraded":"2024-05-08T18:08:28.472742+0000","last_fullsized":"2024-05-08T18:08:28.472742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:28.472654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:28.472654+0000","last_clean_scrub_stamp":"2024-05-08T18:08:28.472654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:31.040500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"29'63","reported_seq":136,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.861664+0000","last_change":"2024-05-08T18:08:12.861715+0000","last_active":"2024-05-08T18:08:12.861664+0000","last_peered":"2024-05-08T18:08:12.861664+0000","last_clean":"2024-05-08T18:08:12.861664+0000","last_became_active":"2024-05-08T18:04:49.830424+0000","last_became_peered":"2024-05-08T18:04:49.830424+0000","last_unstale":"2024-05-08T18:08:12.861664+0000","last_undegraded":"2024-05-08T18:08:12.861664+0000","last_fullsized":"2024-05-08T18:08:12.861664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'63","last_scrub_stamp":"2024-05-08T18:08:12.861577+0000","last_deep_scrub":"29'63","last_deep_scrub_stamp":"2024-05-08T18:08:12.861577+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.861577+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:02.260578+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.479280+0000","last_change":"2024-05-08T18:08:10.479304+0000","last_active":"2024-05-08T18:08:10.479280+0000","last_peered":"2024-05-08T18:08:10.479280+0000","last_clean":"2024-05-08T18:08:10.479280+0000","last_became_active":"2024-05-08T18:04:51.830383+0000","last_became_peered":"2024-05-08T18:04:51.830383+0000","last_unstale":"2024-05-08T18:08:10.479280+0000","last_undegraded":"2024-05-08T18:08:10.479280+0000","last_fullsized":"2024-05-08T18:08:10.479280+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:10.479187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:10.479187+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.479187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:04.216069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"29'20","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.161729+0000","last_change":"2024-05-08T18:04:53.848105+0000","last_active":"2024-05-08T18:07:54.161729+0000","last_peered":"2024-05-08T18:07:54.161729+0000","last_clean":"2024-05-08T18:07:54.161729+0000","last_became_active":"2024-05-08T18:04:53.847842+0000","last_became_peered":"2024-05-08T18:04:53.847842+0000","last_unstale":"2024-05-08T18:07:54.161729+0000","last_undegraded":"2024-05-08T18:07:54.161729+0000","last_fullsized":"2024-05-08T18:07:54.161729+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":48,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.511443+0000","last_change":"2024-05-08T18:08:23.511472+0000","last_active":"2024-05-08T18:08:23.511443+0000","last_peered":"2024-05-08T18:08:23.511443+0000","last_clean":"2024-05-08T18:08:23.511443+0000","last_became_active":"2024-05-08T18:04:45.821412+0000","last_became_peered":"2024-05-08T18:04:45.821412+0000","last_unstale":"2024-05-08T18:08:23.511443+0000","last_undegraded":"2024-05-08T18:08:23.511443+0000","last_fullsized":"2024-05-08T18:08:23.511443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:23.511346+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:23.511346+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.511346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:04.193921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.739776+0000","last_change":"2024-05-08T18:04:43.835922+0000","last_active":"2024-05-08T18:06:58.739776+0000","last_peered":"2024-05-08T18:06:58.739776+0000","last_clean":"2024-05-08T18:06:58.739776+0000","last_became_active":"2024-05-08T18:04:43.834723+0000","last_became_peered":"2024-05-08T18:04:43.834723+0000","last_unstale":"2024-05-08T18:06:58.739776+0000","last_undegraded":"2024-05-08T18:06:58.739776+0000","last_fullsized":"2024-05-08T18:06:58.739776+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.295215+0000","last_change":"2024-05-08T18:08:22.295265+0000","last_active":"2024-05-08T18:08:22.295215+0000","last_peered":"2024-05-08T18:08:22.295215+0000","last_clean":"2024-05-08T18:08:22.295215+0000","last_became_active":"2024-05-08T18:04:47.822865+0000","last_became_peered":"2024-05-08T18:04:47.822865+0000","last_unstale":"2024-05-08T18:08:22.295215+0000","last_undegraded":"2024-05-08T18:08:22.295215+0000","last_fullsized":"2024-05-08T18:08:22.295215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.295113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.295113+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.295113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:31:07.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"29'40","reported_seq":96,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.481554+0000","last_change":"2024-05-08T18:08:11.481605+0000","last_active":"2024-05-08T18:08:11.481554+0000","last_peered":"2024-05-08T18:08:11.481554+0000","last_clean":"2024-05-08T18:08:11.481554+0000","last_became_active":"2024-05-08T18:04:49.824610+0000","last_became_peered":"2024-05-08T18:04:49.824610+0000","last_unstale":"2024-05-08T18:08:11.481554+0000","last_undegraded":"2024-05-08T18:08:11.481554+0000","last_fullsized":"2024-05-08T18:08:11.481554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'40","last_scrub_stamp":"2024-05-08T18:08:11.481461+0000","last_deep_scrub":"29'40","last_deep_scrub_stamp":"2024-05-08T18:08:11.481461+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.481461+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:15.729540+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:28.829222+0000","last_change":"2024-05-08T18:08:28.829264+0000","last_active":"2024-05-08T18:08:28.829222+0000","last_peered":"2024-05-08T18:08:28.829222+0000","last_clean":"2024-05-08T18:08:28.829222+0000","last_became_active":"2024-05-08T18:04:51.848793+0000","last_became_peered":"2024-05-08T18:04:51.848793+0000","last_unstale":"2024-05-08T18:08:28.829222+0000","last_undegraded":"2024-05-08T18:08:28.829222+0000","last_fullsized":"2024-05-08T18:08:28.829222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:28.829111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:28.829111+0000","last_clean_scrub_stamp":"2024-05-08T18:08:28.829111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:35.508902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"29'5","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.119708+0000","last_change":"2024-05-08T18:04:53.849242+0000","last_active":"2024-05-08T18:07:54.119708+0000","last_peered":"2024-05-08T18:07:54.119708+0000","last_clean":"2024-05-08T18:07:54.119708+0000","last_became_active":"2024-05-08T18:04:53.848945+0000","last_became_peered":"2024-05-08T18:04:53.848945+0000","last_unstale":"2024-05-08T18:07:54.119708+0000","last_undegraded":"2024-05-08T18:07:54.119708+0000","last_fullsized":"2024-05-08T18:07:54.119708+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"29'10","reported_seq":84,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.836191+0000","last_change":"2024-05-08T18:08:23.836220+0000","last_active":"2024-05-08T18:08:23.836191+0000","last_peered":"2024-05-08T18:08:23.836191+0000","last_clean":"2024-05-08T18:08:23.836191+0000","last_became_active":"2024-05-08T18:04:45.829120+0000","last_became_peered":"2024-05-08T18:04:45.829120+0000","last_unstale":"2024-05-08T18:08:23.836191+0000","last_undegraded":"2024-05-08T18:08:23.836191+0000","last_fullsized":"2024-05-08T18:08:23.836191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:23.836088+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:23.836088+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.836088+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:15.906999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":57,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744173+0000","last_change":"2024-05-08T18:04:43.835329+0000","last_active":"2024-05-08T18:06:53.744173+0000","last_peered":"2024-05-08T18:06:53.744173+0000","last_clean":"2024-05-08T18:06:53.744173+0000","last_became_active":"2024-05-08T18:04:43.834536+0000","last_became_peered":"2024-05-08T18:04:43.834536+0000","last_unstale":"2024-05-08T18:06:53.744173+0000","last_undegraded":"2024-05-08T18:06:53.744173+0000","last_fullsized":"2024-05-08T18:06:53.744173+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"29'5","reported_seq":238,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:26.539732+0000","last_change":"2024-05-08T18:08:26.539766+0000","last_active":"2024-05-08T18:08:26.539732+0000","last_peered":"2024-05-08T18:08:26.539732+0000","last_clean":"2024-05-08T18:08:26.539732+0000","last_became_active":"2024-05-08T18:04:47.819085+0000","last_became_peered":"2024-05-08T18:04:47.819085+0000","last_unstale":"2024-05-08T18:08:26.539732+0000","last_undegraded":"2024-05-08T18:08:26.539732+0000","last_fullsized":"2024-05-08T18:08:26.539732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'5","last_scrub_stamp":"2024-05-08T18:08:26.539641+0000","last_deep_scrub":"29'5","last_deep_scrub_stamp":"2024-05-08T18:08:26.539641+0000","last_clean_scrub_stamp":"2024-05-08T18:08:26.539641+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:50.189291+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":301,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"29'45","reported_seq":103,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.280826+0000","last_change":"2024-05-08T18:08:16.280897+0000","last_active":"2024-05-08T18:08:16.280826+0000","last_peered":"2024-05-08T18:08:16.280826+0000","last_clean":"2024-05-08T18:08:16.280826+0000","last_became_active":"2024-05-08T18:04:49.825613+0000","last_became_peered":"2024-05-08T18:04:49.825613+0000","last_unstale":"2024-05-08T18:08:16.280826+0000","last_undegraded":"2024-05-08T18:08:16.280826+0000","last_fullsized":"2024-05-08T18:08:16.280826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'45","last_scrub_stamp":"2024-05-08T18:08:16.280719+0000","last_deep_scrub":"29'45","last_deep_scrub_stamp":"2024-05-08T18:08:16.280719+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.280719+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:29:19.130403+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"29'285","reported_seq":443,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.137858+0000","last_change":"2024-05-08T18:08:06.137883+0000","last_active":"2024-05-08T18:08:06.137858+0000","last_peered":"2024-05-08T18:08:06.137858+0000","last_clean":"2024-05-08T18:08:06.137858+0000","last_became_active":"2024-05-08T18:04:51.830924+0000","last_became_peered":"2024-05-08T18:04:51.830924+0000","last_unstale":"2024-05-08T18:08:06.137858+0000","last_undegraded":"2024-05-08T18:08:06.137858+0000","last_fullsized":"2024-05-08T18:08:06.137858+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:06.137767+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:06.137767+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.137767+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:43.622614+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"29'14","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.121335+0000","last_change":"2024-05-08T18:04:53.841349+0000","last_active":"2024-05-08T18:07:54.121335+0000","last_peered":"2024-05-08T18:07:54.121335+0000","last_clean":"2024-05-08T18:07:54.121335+0000","last_became_active":"2024-05-08T18:04:53.840708+0000","last_became_peered":"2024-05-08T18:04:53.840708+0000","last_unstale":"2024-05-08T18:07:54.121335+0000","last_undegraded":"2024-05-08T18:07:54.121335+0000","last_fullsized":"2024-05-08T18:07:54.121335+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"29'6","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:31.301310+0000","last_change":"2024-05-08T18:08:31.301337+0000","last_active":"2024-05-08T18:08:31.301310+0000","last_peered":"2024-05-08T18:08:31.301310+0000","last_clean":"2024-05-08T18:08:31.301310+0000","last_became_active":"2024-05-08T18:04:45.811926+0000","last_became_peered":"2024-05-08T18:04:45.811926+0000","last_unstale":"2024-05-08T18:08:31.301310+0000","last_undegraded":"2024-05-08T18:08:31.301310+0000","last_fullsized":"2024-05-08T18:08:31.301310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:31.301216+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:31.301216+0000","last_clean_scrub_stamp":"2024-05-08T18:08:31.301216+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:32:41.104191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"29'2","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744143+0000","last_change":"2024-05-08T18:04:43.844418+0000","last_active":"2024-05-08T18:06:53.744143+0000","last_peered":"2024-05-08T18:06:53.744143+0000","last_clean":"2024-05-08T18:06:53.744143+0000","last_became_active":"2024-05-08T18:04:43.843934+0000","last_became_peered":"2024-05-08T18:04:43.843934+0000","last_unstale":"2024-05-08T18:06:53.744143+0000","last_undegraded":"2024-05-08T18:06:53.744143+0000","last_fullsized":"2024-05-08T18:06:53.744143+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"29'5","reported_seq":428,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.550034+0000","last_change":"2024-05-08T18:08:20.550071+0000","last_active":"2024-05-08T18:08:20.550034+0000","last_peered":"2024-05-08T18:08:20.550034+0000","last_clean":"2024-05-08T18:08:20.550034+0000","last_became_active":"2024-05-08T18:04:47.817851+0000","last_became_peered":"2024-05-08T18:04:47.817851+0000","last_unstale":"2024-05-08T18:08:20.550034+0000","last_undegraded":"2024-05-08T18:08:20.550034+0000","last_fullsized":"2024-05-08T18:08:20.550034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'5","last_scrub_stamp":"2024-05-08T18:08:20.549943+0000","last_deep_scrub":"29'5","last_deep_scrub_stamp":"2024-05-08T18:08:20.549943+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.549943+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:31.792211+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"29'15","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.800413+0000","last_change":"2024-05-08T18:08:18.800470+0000","last_active":"2024-05-08T18:08:18.800413+0000","last_peered":"2024-05-08T18:08:18.800413+0000","last_clean":"2024-05-08T18:08:18.800413+0000","last_became_active":"2024-05-08T18:04:49.830714+0000","last_became_peered":"2024-05-08T18:04:49.830714+0000","last_unstale":"2024-05-08T18:08:18.800413+0000","last_undegraded":"2024-05-08T18:08:18.800413+0000","last_fullsized":"2024-05-08T18:08:18.800413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:18.800324+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:18.800324+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.800324+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:41.821593+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"29'285","reported_seq":480,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.542263+0000","last_change":"2024-05-08T18:08:05.542295+0000","last_active":"2024-05-08T18:08:05.542263+0000","last_peered":"2024-05-08T18:08:05.542263+0000","last_clean":"2024-05-08T18:08:05.542263+0000","last_became_active":"2024-05-08T18:04:51.831508+0000","last_became_peered":"2024-05-08T18:04:51.831508+0000","last_unstale":"2024-05-08T18:08:05.542263+0000","last_undegraded":"2024-05-08T18:08:05.542263+0000","last_fullsized":"2024-05-08T18:08:05.542263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:05.542169+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:05.542169+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.542169+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:49:28.084669+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'4","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744222+0000","last_change":"2024-05-08T18:04:53.840524+0000","last_active":"2024-05-08T18:06:58.744222+0000","last_peered":"2024-05-08T18:06:58.744222+0000","last_clean":"2024-05-08T18:06:58.744222+0000","last_became_active":"2024-05-08T18:04:53.840215+0000","last_became_peered":"2024-05-08T18:04:53.840215+0000","last_unstale":"2024-05-08T18:06:58.744222+0000","last_undegraded":"2024-05-08T18:06:58.744222+0000","last_fullsized":"2024-05-08T18:06:58.744222+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'2","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:30.526060+0000","last_change":"2024-05-08T18:08:30.526084+0000","last_active":"2024-05-08T18:08:30.526060+0000","last_peered":"2024-05-08T18:08:30.526060+0000","last_clean":"2024-05-08T18:08:30.526060+0000","last_became_active":"2024-05-08T18:04:45.818533+0000","last_became_peered":"2024-05-08T18:04:45.818533+0000","last_unstale":"2024-05-08T18:08:30.526060+0000","last_undegraded":"2024-05-08T18:08:30.526060+0000","last_fullsized":"2024-05-08T18:08:30.526060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:30.525970+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:30.525970+0000","last_clean_scrub_stamp":"2024-05-08T18:08:30.525970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:27.414733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.669968+0000","last_change":"2024-05-08T18:04:43.848496+0000","last_active":"2024-05-08T18:08:00.669968+0000","last_peered":"2024-05-08T18:08:00.669968+0000","last_clean":"2024-05-08T18:08:00.669968+0000","last_became_active":"2024-05-08T18:04:43.846125+0000","last_became_peered":"2024-05-08T18:04:43.846125+0000","last_unstale":"2024-05-08T18:08:00.669968+0000","last_undegraded":"2024-05-08T18:08:00.669968+0000","last_fullsized":"2024-05-08T18:08:00.669968+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.895797+0000","last_change":"2024-05-08T18:08:05.895821+0000","last_active":"2024-05-08T18:08:05.895797+0000","last_peered":"2024-05-08T18:08:05.895797+0000","last_clean":"2024-05-08T18:08:05.895797+0000","last_became_active":"2024-05-08T18:04:47.822242+0000","last_became_peered":"2024-05-08T18:04:47.822242+0000","last_unstale":"2024-05-08T18:08:05.895797+0000","last_undegraded":"2024-05-08T18:08:05.895797+0000","last_fullsized":"2024-05-08T18:08:05.895797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:05.895706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:05.895706+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.895706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:00:55.969119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"29'48","reported_seq":107,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.545377+0000","last_change":"2024-05-08T18:08:08.545419+0000","last_active":"2024-05-08T18:08:08.545377+0000","last_peered":"2024-05-08T18:08:08.545377+0000","last_clean":"2024-05-08T18:08:08.545377+0000","last_became_active":"2024-05-08T18:04:49.825314+0000","last_became_peered":"2024-05-08T18:04:49.825314+0000","last_unstale":"2024-05-08T18:08:08.545377+0000","last_undegraded":"2024-05-08T18:08:08.545377+0000","last_fullsized":"2024-05-08T18:08:08.545377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'48","last_scrub_stamp":"2024-05-08T18:08:08.545269+0000","last_deep_scrub":"29'48","last_deep_scrub_stamp":"2024-05-08T18:08:08.545269+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.545269+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:19.079352+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.136687+0000","last_change":"2024-05-08T18:08:05.136723+0000","last_active":"2024-05-08T18:08:05.136687+0000","last_peered":"2024-05-08T18:08:05.136687+0000","last_clean":"2024-05-08T18:08:05.136687+0000","last_became_active":"2024-05-08T18:04:51.832808+0000","last_became_peered":"2024-05-08T18:04:51.832808+0000","last_unstale":"2024-05-08T18:08:05.136687+0000","last_undegraded":"2024-05-08T18:08:05.136687+0000","last_fullsized":"2024-05-08T18:08:05.136687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:05.136592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:05.136592+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.136592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:30.377482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"29'9","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:25.505987+0000","last_change":"2024-05-08T18:08:25.506016+0000","last_active":"2024-05-08T18:08:25.505987+0000","last_peered":"2024-05-08T18:08:25.505987+0000","last_clean":"2024-05-08T18:08:25.505987+0000","last_became_active":"2024-05-08T18:04:53.837292+0000","last_became_peered":"2024-05-08T18:04:53.837292+0000","last_unstale":"2024-05-08T18:08:25.505987+0000","last_undegraded":"2024-05-08T18:08:25.505987+0000","last_fullsized":"2024-05-08T18:08:25.505987+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'9","last_scrub_stamp":"2024-05-08T18:08:25.505900+0000","last_deep_scrub":"29'9","last_deep_scrub_stamp":"2024-05-08T18:08:25.505900+0000","last_clean_scrub_stamp":"2024-05-08T18:08:25.505900+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:25.016619+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'4","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:25.876154+0000","last_change":"2024-05-08T18:08:25.876183+0000","last_active":"2024-05-08T18:08:25.876154+0000","last_peered":"2024-05-08T18:08:25.876154+0000","last_clean":"2024-05-08T18:08:25.876154+0000","last_became_active":"2024-05-08T18:04:45.828674+0000","last_became_peered":"2024-05-08T18:04:45.828674+0000","last_unstale":"2024-05-08T18:08:25.876154+0000","last_undegraded":"2024-05-08T18:08:25.876154+0000","last_fullsized":"2024-05-08T18:08:25.876154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:25.876052+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:25.876052+0000","last_clean_scrub_stamp":"2024-05-08T18:08:25.876052+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:30:56.782177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'72","reported_seq":139,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:04.167292+0000","last_change":"2024-05-08T18:04:35.552002+0000","last_active":"2024-05-08T18:08:04.167292+0000","last_peered":"2024-05-08T18:08:04.167292+0000","last_clean":"2024-05-08T18:08:04.167292+0000","last_became_active":"2024-05-08T18:04:35.551437+0000","last_became_peered":"2024-05-08T18:04:35.551437+0000","last_unstale":"2024-05-08T18:08:04.167292+0000","last_undegraded":"2024-05-08T18:08:04.167292+0000","last_fullsized":"2024-05-08T18:08:04.167292+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:43.742741+0000","last_change":"2024-05-08T18:04:35.552341+0000","last_active":"2024-05-08T18:06:43.742741+0000","last_peered":"2024-05-08T18:06:43.742741+0000","last_clean":"2024-05-08T18:06:43.742741+0000","last_became_active":"2024-05-08T18:04:35.552017+0000","last_became_peered":"2024-05-08T18:04:35.552017+0000","last_unstale":"2024-05-08T18:06:43.742741+0000","last_undegraded":"2024-05-08T18:06:43.742741+0000","last_fullsized":"2024-05-08T18:06:43.742741+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.743488+0000","last_change":"2024-05-08T18:04:43.837405+0000","last_active":"2024-05-08T18:06:48.743488+0000","last_peered":"2024-05-08T18:06:48.743488+0000","last_clean":"2024-05-08T18:06:48.743488+0000","last_became_active":"2024-05-08T18:04:43.837019+0000","last_became_peered":"2024-05-08T18:04:43.837019+0000","last_unstale":"2024-05-08T18:06:48.743488+0000","last_undegraded":"2024-05-08T18:06:48.743488+0000","last_fullsized":"2024-05-08T18:06:48.743488+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.743818+0000","last_change":"2024-05-08T18:05:30.871131+0000","last_active":"2024-05-08T18:06:53.743818+0000","last_peered":"2024-05-08T18:06:53.743818+0000","last_clean":"2024-05-08T18:06:53.743818+0000","last_became_active":"2024-05-08T18:05:30.870647+0000","last_became_peered":"2024-05-08T18:05:30.870647+0000","last_unstale":"2024-05-08T18:06:53.743818+0000","last_undegraded":"2024-05-08T18:06:53.743818+0000","last_fullsized":"2024-05-08T18:06:53.743818+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"29'91","reported_seq":181,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:34.946616+0000","last_change":"2024-05-08T18:08:34.946658+0000","last_active":"2024-05-08T18:08:34.946616+0000","last_peered":"2024-05-08T18:08:34.946616+0000","last_clean":"2024-05-08T18:08:34.946616+0000","last_became_active":"2024-05-08T18:04:49.830636+0000","last_became_peered":"2024-05-08T18:04:49.830636+0000","last_unstale":"2024-05-08T18:08:34.946616+0000","last_undegraded":"2024-05-08T18:08:34.946616+0000","last_fullsized":"2024-05-08T18:08:34.946616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'91","last_scrub_stamp":"2024-05-08T18:08:34.946526+0000","last_deep_scrub":"29'91","last_deep_scrub_stamp":"2024-05-08T18:08:34.946526+0000","last_clean_scrub_stamp":"2024-05-08T18:08:34.946526+0000","objects_scrubbed":5,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:49.081723+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.600189+0000","last_change":"2024-05-08T18:08:21.600230+0000","last_active":"2024-05-08T18:08:21.600189+0000","last_peered":"2024-05-08T18:08:21.600189+0000","last_clean":"2024-05-08T18:08:21.600189+0000","last_became_active":"2024-05-08T18:04:51.844737+0000","last_became_peered":"2024-05-08T18:04:51.844737+0000","last_unstale":"2024-05-08T18:08:21.600189+0000","last_undegraded":"2024-05-08T18:08:21.600189+0000","last_fullsized":"2024-05-08T18:08:21.600189+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:21.600069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:21.600069+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.600069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:48.902927+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"29'86","reported_seq":176,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:25.255787+0000","last_change":"2024-05-08T18:08:25.255820+0000","last_active":"2024-05-08T18:08:25.255787+0000","last_peered":"2024-05-08T18:08:25.255787+0000","last_clean":"2024-05-08T18:08:25.255787+0000","last_became_active":"2024-05-08T18:04:53.838957+0000","last_became_peered":"2024-05-08T18:04:53.838957+0000","last_unstale":"2024-05-08T18:08:25.255787+0000","last_undegraded":"2024-05-08T18:08:25.255787+0000","last_fullsized":"2024-05-08T18:08:25.255787+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'86","last_scrub_stamp":"2024-05-08T18:08:25.255680+0000","last_deep_scrub":"29'86","last_deep_scrub_stamp":"2024-05-08T18:08:25.255680+0000","last_clean_scrub_stamp":"2024-05-08T18:08:25.255680+0000","objects_scrubbed":1,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:14:28.008294+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":216,"num_read_kb":214,"num_write":167,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"29'9","reported_seq":66,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.115990+0000","last_change":"2024-05-08T18:04:45.826087+0000","last_active":"2024-05-08T18:07:54.115990+0000","last_peered":"2024-05-08T18:07:54.115990+0000","last_clean":"2024-05-08T18:07:54.115990+0000","last_became_active":"2024-05-08T18:04:45.825843+0000","last_became_peered":"2024-05-08T18:04:45.825843+0000","last_unstale":"2024-05-08T18:07:54.115990+0000","last_undegraded":"2024-05-08T18:07:54.115990+0000","last_fullsized":"2024-05-08T18:07:54.115990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.080708+0000","last_change":"2024-05-08T18:04:35.552553+0000","last_active":"2024-05-08T18:08:05.080708+0000","last_peered":"2024-05-08T18:08:05.080708+0000","last_clean":"2024-05-08T18:08:05.080708+0000","last_became_active":"2024-05-08T18:04:35.552214+0000","last_became_peered":"2024-05-08T18:04:35.552214+0000","last_unstale":"2024-05-08T18:08:05.080708+0000","last_undegraded":"2024-05-08T18:08:05.080708+0000","last_fullsized":"2024-05-08T18:08:05.080708+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.738105+0000","last_change":"2024-05-08T18:04:43.821058+0000","last_active":"2024-05-08T18:06:48.738105+0000","last_peered":"2024-05-08T18:06:48.738105+0000","last_clean":"2024-05-08T18:06:48.738105+0000","last_became_active":"2024-05-08T18:04:43.820369+0000","last_became_peered":"2024-05-08T18:04:43.820369+0000","last_unstale":"2024-05-08T18:06:48.738105+0000","last_undegraded":"2024-05-08T18:06:48.738105+0000","last_fullsized":"2024-05-08T18:06:48.738105+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.745186+0000","last_change":"2024-05-08T18:05:30.870035+0000","last_active":"2024-05-08T18:06:58.745186+0000","last_peered":"2024-05-08T18:06:58.745186+0000","last_clean":"2024-05-08T18:06:58.745186+0000","last_became_active":"2024-05-08T18:05:30.869578+0000","last_became_peered":"2024-05-08T18:05:30.869578+0000","last_unstale":"2024-05-08T18:06:58.745186+0000","last_undegraded":"2024-05-08T18:06:58.745186+0000","last_fullsized":"2024-05-08T18:06:58.745186+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"29'290","reported_seq":515,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.566274+0000","last_change":"2024-05-08T18:08:07.566330+0000","last_active":"2024-05-08T18:08:07.566274+0000","last_peered":"2024-05-08T18:08:07.566274+0000","last_clean":"2024-05-08T18:08:07.566274+0000","last_became_active":"2024-05-08T18:04:49.833053+0000","last_became_peered":"2024-05-08T18:04:49.833053+0000","last_unstale":"2024-05-08T18:08:07.566274+0000","last_undegraded":"2024-05-08T18:08:07.566274+0000","last_fullsized":"2024-05-08T18:08:07.566274+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'290","last_scrub_stamp":"2024-05-08T18:08:07.566185+0000","last_deep_scrub":"29'290","last_deep_scrub_stamp":"2024-05-08T18:08:07.566185+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.566185+0000","objects_scrubbed":10,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:24.566210+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":378,"num_read_kb":363,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:32.883501+0000","last_change":"2024-05-08T18:08:32.883526+0000","last_active":"2024-05-08T18:08:32.883501+0000","last_peered":"2024-05-08T18:08:32.883501+0000","last_clean":"2024-05-08T18:08:32.883501+0000","last_became_active":"2024-05-08T18:04:51.848066+0000","last_became_peered":"2024-05-08T18:04:51.848066+0000","last_unstale":"2024-05-08T18:08:32.883501+0000","last_undegraded":"2024-05-08T18:08:32.883501+0000","last_fullsized":"2024-05-08T18:08:32.883501+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:32.883401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:32.883401+0000","last_clean_scrub_stamp":"2024-05-08T18:08:32.883401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:44.412793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"29'3","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745324+0000","last_change":"2024-05-08T18:04:53.843187+0000","last_active":"2024-05-08T18:07:03.745324+0000","last_peered":"2024-05-08T18:07:03.745324+0000","last_clean":"2024-05-08T18:07:03.745324+0000","last_became_active":"2024-05-08T18:04:53.842899+0000","last_became_peered":"2024-05-08T18:04:53.842899+0000","last_unstale":"2024-05-08T18:07:03.745324+0000","last_undegraded":"2024-05-08T18:07:03.745324+0000","last_fullsized":"2024-05-08T18:07:03.745324+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"29'8","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070059+0000","last_change":"2024-05-08T18:04:45.812417+0000","last_active":"2024-05-08T18:07:54.070059+0000","last_peered":"2024-05-08T18:07:54.070059+0000","last_clean":"2024-05-08T18:07:54.070059+0000","last_became_active":"2024-05-08T18:04:45.812091+0000","last_became_peered":"2024-05-08T18:04:45.812091+0000","last_unstale":"2024-05-08T18:07:54.070059+0000","last_undegraded":"2024-05-08T18:07:54.070059+0000","last_fullsized":"2024-05-08T18:07:54.070059+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:43.042058+0000","last_change":"2024-05-08T18:08:43.042058+0000","last_active":"2024-05-08T18:08:43.042058+0000","last_peered":"2024-05-08T18:08:43.042058+0000","last_clean":"2024-05-08T18:08:43.042058+0000","last_became_active":"2024-05-08T18:04:35.551313+0000","last_became_peered":"2024-05-08T18:04:35.551313+0000","last_unstale":"2024-05-08T18:08:43.042058+0000","last_undegraded":"2024-05-08T18:08:43.042058+0000","last_fullsized":"2024-05-08T18:08:43.042058+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:43.041942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:43.041942+0000","last_clean_scrub_stamp":"2024-05-08T18:08:43.041942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:53.026181+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:03.707332+0000","last_change":"2024-05-08T18:04:43.821180+0000","last_active":"2024-05-08T18:08:03.707332+0000","last_peered":"2024-05-08T18:08:03.707332+0000","last_clean":"2024-05-08T18:08:03.707332+0000","last_became_active":"2024-05-08T18:04:43.820335+0000","last_became_peered":"2024-05-08T18:04:43.820335+0000","last_unstale":"2024-05-08T18:08:03.707332+0000","last_undegraded":"2024-05-08T18:08:03.707332+0000","last_fullsized":"2024-05-08T18:08:03.707332+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"29'2","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.343008+0000","last_change":"2024-05-08T18:08:20.343043+0000","last_active":"2024-05-08T18:08:20.343008+0000","last_peered":"2024-05-08T18:08:20.343008+0000","last_clean":"2024-05-08T18:08:20.343008+0000","last_became_active":"2024-05-08T18:04:47.823631+0000","last_became_peered":"2024-05-08T18:04:47.823631+0000","last_unstale":"2024-05-08T18:08:20.343008+0000","last_undegraded":"2024-05-08T18:08:20.343008+0000","last_fullsized":"2024-05-08T18:08:20.343008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:20.342916+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:20.342916+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.342916+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:26:02.075842+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"29'24","reported_seq":101,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.233358+0000","last_change":"2024-05-08T18:08:15.233395+0000","last_active":"2024-05-08T18:08:15.233358+0000","last_peered":"2024-05-08T18:08:15.233358+0000","last_clean":"2024-05-08T18:08:15.233358+0000","last_became_active":"2024-05-08T18:04:49.827735+0000","last_became_peered":"2024-05-08T18:04:49.827735+0000","last_unstale":"2024-05-08T18:08:15.233358+0000","last_undegraded":"2024-05-08T18:08:15.233358+0000","last_fullsized":"2024-05-08T18:08:15.233358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'24","last_scrub_stamp":"2024-05-08T18:08:15.233271+0000","last_deep_scrub":"29'24","last_deep_scrub_stamp":"2024-05-08T18:08:15.233271+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.233271+0000","objects_scrubbed":8,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:25.767355+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.442651+0000","last_change":"2024-05-08T18:08:09.442677+0000","last_active":"2024-05-08T18:08:09.442651+0000","last_peered":"2024-05-08T18:08:09.442651+0000","last_clean":"2024-05-08T18:08:09.442651+0000","last_became_active":"2024-05-08T18:04:51.843940+0000","last_became_peered":"2024-05-08T18:04:51.843940+0000","last_unstale":"2024-05-08T18:08:09.442651+0000","last_undegraded":"2024-05-08T18:08:09.442651+0000","last_fullsized":"2024-05-08T18:08:09.442651+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:09.442556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:09.442556+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.442556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:48:00.984508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"29'4","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.000331+0000","last_change":"2024-05-08T18:04:53.848676+0000","last_active":"2024-05-08T18:07:54.000331+0000","last_peered":"2024-05-08T18:07:54.000331+0000","last_clean":"2024-05-08T18:07:54.000331+0000","last_became_active":"2024-05-08T18:04:53.848255+0000","last_became_peered":"2024-05-08T18:04:53.848255+0000","last_unstale":"2024-05-08T18:07:54.000331+0000","last_undegraded":"2024-05-08T18:07:54.000331+0000","last_fullsized":"2024-05-08T18:07:54.000331+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"29'10","reported_seq":61,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:39.382343+0000","last_change":"2024-05-08T18:08:39.382370+0000","last_active":"2024-05-08T18:08:39.382343+0000","last_peered":"2024-05-08T18:08:39.382343+0000","last_clean":"2024-05-08T18:08:39.382343+0000","last_became_active":"2024-05-08T18:04:45.807085+0000","last_became_peered":"2024-05-08T18:04:45.807085+0000","last_unstale":"2024-05-08T18:08:39.382343+0000","last_undegraded":"2024-05-08T18:08:39.382343+0000","last_fullsized":"2024-05-08T18:08:39.382343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:39.382233+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:39.382233+0000","last_clean_scrub_stamp":"2024-05-08T18:08:39.382233+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:03.089676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:43.742675+0000","last_change":"2024-05-08T18:04:35.551456+0000","last_active":"2024-05-08T18:06:43.742675+0000","last_peered":"2024-05-08T18:06:43.742675+0000","last_clean":"2024-05-08T18:06:43.742675+0000","last_became_active":"2024-05-08T18:04:35.551120+0000","last_became_peered":"2024-05-08T18:04:35.551120+0000","last_unstale":"2024-05-08T18:06:43.742675+0000","last_undegraded":"2024-05-08T18:06:43.742675+0000","last_fullsized":"2024-05-08T18:06:43.742675+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.742661+0000","last_change":"2024-05-08T18:04:43.840439+0000","last_active":"2024-05-08T18:06:48.742661+0000","last_peered":"2024-05-08T18:06:48.742661+0000","last_clean":"2024-05-08T18:06:48.742661+0000","last_became_active":"2024-05-08T18:04:43.840036+0000","last_became_peered":"2024-05-08T18:04:43.840036+0000","last_unstale":"2024-05-08T18:06:48.742661+0000","last_undegraded":"2024-05-08T18:06:48.742661+0000","last_fullsized":"2024-05-08T18:06:48.742661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0,3],"acting":[2,1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.607224+0000","last_change":"2024-05-08T18:08:20.607246+0000","last_active":"2024-05-08T18:08:20.607224+0000","last_peered":"2024-05-08T18:08:20.607224+0000","last_clean":"2024-05-08T18:08:20.607224+0000","last_became_active":"2024-05-08T18:04:47.823093+0000","last_became_peered":"2024-05-08T18:04:47.823093+0000","last_unstale":"2024-05-08T18:08:20.607224+0000","last_undegraded":"2024-05-08T18:08:20.607224+0000","last_fullsized":"2024-05-08T18:08:20.607224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:20.607136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:20.607136+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.607136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:01.289711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"29'20","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:24.842547+0000","last_change":"2024-05-08T18:08:24.842630+0000","last_active":"2024-05-08T18:08:24.842547+0000","last_peered":"2024-05-08T18:08:24.842547+0000","last_clean":"2024-05-08T18:08:24.842547+0000","last_became_active":"2024-05-08T18:04:49.830586+0000","last_became_peered":"2024-05-08T18:04:49.830586+0000","last_unstale":"2024-05-08T18:08:24.842547+0000","last_undegraded":"2024-05-08T18:08:24.842547+0000","last_fullsized":"2024-05-08T18:08:24.842547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'20","last_scrub_stamp":"2024-05-08T18:08:24.842450+0000","last_deep_scrub":"29'20","last_deep_scrub_stamp":"2024-05-08T18:08:24.842450+0000","last_clean_scrub_stamp":"2024-05-08T18:08:24.842450+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:35.285600+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":394,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:37.005677+0000","last_change":"2024-05-08T18:08:37.005716+0000","last_active":"2024-05-08T18:08:37.005677+0000","last_peered":"2024-05-08T18:08:37.005677+0000","last_clean":"2024-05-08T18:08:37.005677+0000","last_became_active":"2024-05-08T18:04:51.849137+0000","last_became_peered":"2024-05-08T18:04:51.849137+0000","last_unstale":"2024-05-08T18:08:37.005677+0000","last_undegraded":"2024-05-08T18:08:37.005677+0000","last_fullsized":"2024-05-08T18:08:37.005677+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:37.005585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:37.005585+0000","last_clean_scrub_stamp":"2024-05-08T18:08:37.005585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:35:48.297068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'4","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745230+0000","last_change":"2024-05-08T18:04:53.851345+0000","last_active":"2024-05-08T18:07:03.745230+0000","last_peered":"2024-05-08T18:07:03.745230+0000","last_clean":"2024-05-08T18:07:03.745230+0000","last_became_active":"2024-05-08T18:04:53.851008+0000","last_became_peered":"2024-05-08T18:04:53.851008+0000","last_unstale":"2024-05-08T18:07:03.745230+0000","last_undegraded":"2024-05-08T18:07:03.745230+0000","last_fullsized":"2024-05-08T18:07:03.745230+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'2","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:27.488684+0000","last_change":"2024-05-08T18:08:27.488724+0000","last_active":"2024-05-08T18:08:27.488684+0000","last_peered":"2024-05-08T18:08:27.488684+0000","last_clean":"2024-05-08T18:08:27.488684+0000","last_became_active":"2024-05-08T18:04:45.819326+0000","last_became_peered":"2024-05-08T18:04:45.819326+0000","last_unstale":"2024-05-08T18:08:27.488684+0000","last_undegraded":"2024-05-08T18:08:27.488684+0000","last_fullsized":"2024-05-08T18:08:27.488684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:27.488574+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:27.488574+0000","last_clean_scrub_stamp":"2024-05-08T18:08:27.488574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:06.323416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.742690+0000","last_change":"2024-05-08T18:04:43.840661+0000","last_active":"2024-05-08T18:06:48.742690+0000","last_peered":"2024-05-08T18:06:48.742690+0000","last_clean":"2024-05-08T18:06:48.742690+0000","last_became_active":"2024-05-08T18:04:43.840236+0000","last_became_peered":"2024-05-08T18:04:43.840236+0000","last_unstale":"2024-05-08T18:06:48.742690+0000","last_undegraded":"2024-05-08T18:06:48.742690+0000","last_fullsized":"2024-05-08T18:06:48.742690+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:43.737265+0000","last_change":"2024-05-08T18:04:35.552512+0000","last_active":"2024-05-08T18:06:43.737265+0000","last_peered":"2024-05-08T18:06:43.737265+0000","last_clean":"2024-05-08T18:06:43.737265+0000","last_became_active":"2024-05-08T18:04:35.552173+0000","last_became_peered":"2024-05-08T18:04:35.552173+0000","last_unstale":"2024-05-08T18:06:43.737265+0000","last_undegraded":"2024-05-08T18:06:43.737265+0000","last_fullsized":"2024-05-08T18:06:43.737265+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"29'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:24.599637+0000","last_change":"2024-05-08T18:08:24.599665+0000","last_active":"2024-05-08T18:08:24.599637+0000","last_peered":"2024-05-08T18:08:24.599637+0000","last_clean":"2024-05-08T18:08:24.599637+0000","last_became_active":"2024-05-08T18:04:47.819910+0000","last_became_peered":"2024-05-08T18:04:47.819910+0000","last_unstale":"2024-05-08T18:08:24.599637+0000","last_undegraded":"2024-05-08T18:08:24.599637+0000","last_fullsized":"2024-05-08T18:08:24.599637+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:24.599549+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:24.599549+0000","last_clean_scrub_stamp":"2024-05-08T18:08:24.599549+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:03.434391+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"29'73","reported_seq":149,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.598709+0000","last_change":"2024-05-08T18:08:15.598754+0000","last_active":"2024-05-08T18:08:15.598709+0000","last_peered":"2024-05-08T18:08:15.598709+0000","last_clean":"2024-05-08T18:08:15.598709+0000","last_became_active":"2024-05-08T18:04:49.831360+0000","last_became_peered":"2024-05-08T18:04:49.831360+0000","last_unstale":"2024-05-08T18:08:15.598709+0000","last_undegraded":"2024-05-08T18:08:15.598709+0000","last_fullsized":"2024-05-08T18:08:15.598709+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'73","last_scrub_stamp":"2024-05-08T18:08:15.598616+0000","last_deep_scrub":"29'73","last_deep_scrub_stamp":"2024-05-08T18:08:15.598616+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.598616+0000","objects_scrubbed":9,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:23:01.195330+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":336,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.250177+0000","last_change":"2024-05-08T18:08:13.250201+0000","last_active":"2024-05-08T18:08:13.250177+0000","last_peered":"2024-05-08T18:08:13.250177+0000","last_clean":"2024-05-08T18:08:13.250177+0000","last_became_active":"2024-05-08T18:04:51.836441+0000","last_became_peered":"2024-05-08T18:04:51.836441+0000","last_unstale":"2024-05-08T18:08:13.250177+0000","last_undegraded":"2024-05-08T18:08:13.250177+0000","last_fullsized":"2024-05-08T18:08:13.250177+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:13.250078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:13.250078+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.250078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:13.374535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"29'9","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:29.474467+0000","last_change":"2024-05-08T18:08:29.474491+0000","last_active":"2024-05-08T18:08:29.474467+0000","last_peered":"2024-05-08T18:08:29.474467+0000","last_clean":"2024-05-08T18:08:29.474467+0000","last_became_active":"2024-05-08T18:04:53.837920+0000","last_became_peered":"2024-05-08T18:04:53.837920+0000","last_unstale":"2024-05-08T18:08:29.474467+0000","last_undegraded":"2024-05-08T18:08:29.474467+0000","last_fullsized":"2024-05-08T18:08:29.474467+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'9","last_scrub_stamp":"2024-05-08T18:08:29.474349+0000","last_deep_scrub":"29'9","last_deep_scrub_stamp":"2024-05-08T18:08:29.474349+0000","last_clean_scrub_stamp":"2024-05-08T18:08:29.474349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:17.638910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"29'16","reported_seq":87,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:24.469971+0000","last_change":"2024-05-08T18:08:24.470006+0000","last_active":"2024-05-08T18:08:24.469971+0000","last_peered":"2024-05-08T18:08:24.469971+0000","last_clean":"2024-05-08T18:08:24.469971+0000","last_became_active":"2024-05-08T18:04:45.821107+0000","last_became_peered":"2024-05-08T18:04:45.821107+0000","last_unstale":"2024-05-08T18:08:24.469971+0000","last_undegraded":"2024-05-08T18:08:24.469971+0000","last_fullsized":"2024-05-08T18:08:24.469971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'16","last_scrub_stamp":"2024-05-08T18:08:24.469872+0000","last_deep_scrub":"29'16","last_deep_scrub_stamp":"2024-05-08T18:08:24.469872+0000","last_clean_scrub_stamp":"2024-05-08T18:08:24.469872+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:51.772730+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.738166+0000","last_change":"2024-05-08T18:04:43.828309+0000","last_active":"2024-05-08T18:06:48.738166+0000","last_peered":"2024-05-08T18:06:48.738166+0000","last_clean":"2024-05-08T18:06:48.738166+0000","last_became_active":"2024-05-08T18:04:43.827920+0000","last_became_peered":"2024-05-08T18:04:43.827920+0000","last_unstale":"2024-05-08T18:06:48.738166+0000","last_undegraded":"2024-05-08T18:06:48.738166+0000","last_fullsized":"2024-05-08T18:06:48.738166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:43.742777+0000","last_change":"2024-05-08T18:04:35.551253+0000","last_active":"2024-05-08T18:06:43.742777+0000","last_peered":"2024-05-08T18:06:43.742777+0000","last_clean":"2024-05-08T18:06:43.742777+0000","last_became_active":"2024-05-08T18:04:35.550912+0000","last_became_peered":"2024-05-08T18:04:35.550912+0000","last_unstale":"2024-05-08T18:06:43.742777+0000","last_undegraded":"2024-05-08T18:06:43.742777+0000","last_fullsized":"2024-05-08T18:06:43.742777+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.811050+0000","last_change":"2024-05-08T18:08:15.811076+0000","last_active":"2024-05-08T18:08:15.811050+0000","last_peered":"2024-05-08T18:08:15.811050+0000","last_clean":"2024-05-08T18:08:15.811050+0000","last_became_active":"2024-05-08T18:04:47.813717+0000","last_became_peered":"2024-05-08T18:04:47.813717+0000","last_unstale":"2024-05-08T18:08:15.811050+0000","last_undegraded":"2024-05-08T18:08:15.811050+0000","last_fullsized":"2024-05-08T18:08:15.811050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:15.810959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:15.810959+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.810959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:35.598153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"29'21","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.551930+0000","last_change":"2024-05-08T18:08:06.551971+0000","last_active":"2024-05-08T18:08:06.551930+0000","last_peered":"2024-05-08T18:08:06.551930+0000","last_clean":"2024-05-08T18:08:06.551930+0000","last_became_active":"2024-05-08T18:04:49.835451+0000","last_became_peered":"2024-05-08T18:04:49.835451+0000","last_unstale":"2024-05-08T18:08:06.551930+0000","last_undegraded":"2024-05-08T18:08:06.551930+0000","last_fullsized":"2024-05-08T18:08:06.551930+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'21","last_scrub_stamp":"2024-05-08T18:08:06.551844+0000","last_deep_scrub":"29'21","last_deep_scrub_stamp":"2024-05-08T18:08:06.551844+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.551844+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:05.040309+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"29'285","reported_seq":381,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:04.536879+0000","last_change":"2024-05-08T18:08:04.536905+0000","last_active":"2024-05-08T18:08:04.536879+0000","last_peered":"2024-05-08T18:08:04.536879+0000","last_clean":"2024-05-08T18:08:04.536879+0000","last_became_active":"2024-05-08T18:04:51.832109+0000","last_became_peered":"2024-05-08T18:04:51.832109+0000","last_unstale":"2024-05-08T18:08:04.536879+0000","last_undegraded":"2024-05-08T18:08:04.536879+0000","last_fullsized":"2024-05-08T18:08:04.536879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:04.536786+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:04.536786+0000","last_clean_scrub_stamp":"2024-05-08T18:08:04.536786+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:42.101932+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746998+0000","last_change":"2024-05-08T18:04:53.844572+0000","last_active":"2024-05-08T18:07:08.746998+0000","last_peered":"2024-05-08T18:07:08.746998+0000","last_clean":"2024-05-08T18:07:08.746998+0000","last_became_active":"2024-05-08T18:04:53.842998+0000","last_became_peered":"2024-05-08T18:04:53.842998+0000","last_unstale":"2024-05-08T18:07:08.746998+0000","last_undegraded":"2024-05-08T18:07:08.746998+0000","last_fullsized":"2024-05-08T18:07:08.746998+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'4","reported_seq":72,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.838487+0000","last_change":"2024-05-08T18:08:19.838524+0000","last_active":"2024-05-08T18:08:19.838487+0000","last_peered":"2024-05-08T18:08:19.838487+0000","last_clean":"2024-05-08T18:08:19.838487+0000","last_became_active":"2024-05-08T18:04:45.829257+0000","last_became_peered":"2024-05-08T18:04:45.829257+0000","last_unstale":"2024-05-08T18:08:19.838487+0000","last_undegraded":"2024-05-08T18:08:19.838487+0000","last_fullsized":"2024-05-08T18:08:19.838487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:19.838313+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:19.838313+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.838313+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:43.545756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"29'3","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744271+0000","last_change":"2024-05-08T18:04:43.835453+0000","last_active":"2024-05-08T18:06:53.744271+0000","last_peered":"2024-05-08T18:06:53.744271+0000","last_clean":"2024-05-08T18:06:53.744271+0000","last_became_active":"2024-05-08T18:04:43.834610+0000","last_became_peered":"2024-05-08T18:04:43.834610+0000","last_unstale":"2024-05-08T18:06:53.744271+0000","last_undegraded":"2024-05-08T18:06:53.744271+0000","last_fullsized":"2024-05-08T18:06:53.744271+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:43.737303+0000","last_change":"2024-05-08T18:04:35.551595+0000","last_active":"2024-05-08T18:06:43.737303+0000","last_peered":"2024-05-08T18:06:43.737303+0000","last_clean":"2024-05-08T18:06:43.737303+0000","last_became_active":"2024-05-08T18:04:35.551366+0000","last_became_peered":"2024-05-08T18:04:35.551366+0000","last_unstale":"2024-05-08T18:06:43.737303+0000","last_undegraded":"2024-05-08T18:06:43.737303+0000","last_fullsized":"2024-05-08T18:06:43.737303+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.923238+0000","last_change":"2024-05-08T18:08:06.923265+0000","last_active":"2024-05-08T18:08:06.923238+0000","last_peered":"2024-05-08T18:08:06.923238+0000","last_clean":"2024-05-08T18:08:06.923238+0000","last_became_active":"2024-05-08T18:04:47.819866+0000","last_became_peered":"2024-05-08T18:04:47.819866+0000","last_unstale":"2024-05-08T18:08:06.923238+0000","last_undegraded":"2024-05-08T18:08:06.923238+0000","last_fullsized":"2024-05-08T18:08:06.923238+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:06.923149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:06.923149+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.923149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:30.572799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"29'15","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:35.962354+0000","last_change":"2024-05-08T18:08:35.962385+0000","last_active":"2024-05-08T18:08:35.962354+0000","last_peered":"2024-05-08T18:08:35.962354+0000","last_clean":"2024-05-08T18:08:35.962354+0000","last_became_active":"2024-05-08T18:04:49.830452+0000","last_became_peered":"2024-05-08T18:04:49.830452+0000","last_unstale":"2024-05-08T18:08:35.962354+0000","last_undegraded":"2024-05-08T18:08:35.962354+0000","last_fullsized":"2024-05-08T18:08:35.962354+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:35.962267+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:35.962267+0000","last_clean_scrub_stamp":"2024-05-08T18:08:35.962267+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:45.951593+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"29'285","reported_seq":411,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.471758+0000","last_change":"2024-05-08T18:08:06.471786+0000","last_active":"2024-05-08T18:08:06.471758+0000","last_peered":"2024-05-08T18:08:06.471758+0000","last_clean":"2024-05-08T18:08:06.471758+0000","last_became_active":"2024-05-08T18:04:51.831155+0000","last_became_peered":"2024-05-08T18:04:51.831155+0000","last_unstale":"2024-05-08T18:08:06.471758+0000","last_undegraded":"2024-05-08T18:08:06.471758+0000","last_fullsized":"2024-05-08T18:08:06.471758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:06.471664+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:06.471664+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.471664+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:29:09.321348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'5","reported_seq":24,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.068065+0000","last_change":"2024-05-08T18:04:53.849164+0000","last_active":"2024-05-08T18:07:54.068065+0000","last_peered":"2024-05-08T18:07:54.068065+0000","last_clean":"2024-05-08T18:07:54.068065+0000","last_became_active":"2024-05-08T18:04:53.848862+0000","last_became_peered":"2024-05-08T18:04:53.848862+0000","last_unstale":"2024-05-08T18:07:54.068065+0000","last_undegraded":"2024-05-08T18:07:54.068065+0000","last_fullsized":"2024-05-08T18:07:54.068065+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"29'6","reported_seq":60,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:41.133432+0000","last_change":"2024-05-08T18:08:41.133464+0000","last_active":"2024-05-08T18:08:41.133432+0000","last_peered":"2024-05-08T18:08:41.133432+0000","last_clean":"2024-05-08T18:08:41.133432+0000","last_became_active":"2024-05-08T18:04:45.822824+0000","last_became_peered":"2024-05-08T18:04:45.822824+0000","last_unstale":"2024-05-08T18:08:41.133432+0000","last_undegraded":"2024-05-08T18:08:41.133432+0000","last_fullsized":"2024-05-08T18:08:41.133432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:41.133339+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:41.133339+0000","last_clean_scrub_stamp":"2024-05-08T18:08:41.133339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:17.740267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.743851+0000","last_change":"2024-05-08T18:04:43.839633+0000","last_active":"2024-05-08T18:06:53.743851+0000","last_peered":"2024-05-08T18:06:53.743851+0000","last_clean":"2024-05-08T18:06:53.743851+0000","last_became_active":"2024-05-08T18:04:43.839240+0000","last_became_peered":"2024-05-08T18:04:43.839240+0000","last_unstale":"2024-05-08T18:06:53.743851+0000","last_undegraded":"2024-05-08T18:06:53.743851+0000","last_fullsized":"2024-05-08T18:06:53.743851+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.743591+0000","last_change":"2024-05-08T18:04:35.551765+0000","last_active":"2024-05-08T18:06:48.743591+0000","last_peered":"2024-05-08T18:06:48.743591+0000","last_clean":"2024-05-08T18:06:48.743591+0000","last_became_active":"2024-05-08T18:04:35.551412+0000","last_became_peered":"2024-05-08T18:04:35.551412+0000","last_unstale":"2024-05-08T18:06:48.743591+0000","last_undegraded":"2024-05-08T18:06:48.743591+0000","last_fullsized":"2024-05-08T18:06:48.743591+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.843811+0000","last_change":"2024-05-08T18:08:13.843881+0000","last_active":"2024-05-08T18:08:13.843811+0000","last_peered":"2024-05-08T18:08:13.843811+0000","last_clean":"2024-05-08T18:08:13.843811+0000","last_became_active":"2024-05-08T18:04:47.815947+0000","last_became_peered":"2024-05-08T18:04:47.815947+0000","last_unstale":"2024-05-08T18:08:13.843811+0000","last_undegraded":"2024-05-08T18:08:13.843811+0000","last_fullsized":"2024-05-08T18:08:13.843811+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:13.843666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:13.843666+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.843666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:56.027223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"29'96","reported_seq":189,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.152986+0000","last_change":"2024-05-08T18:08:10.153072+0000","last_active":"2024-05-08T18:08:10.152986+0000","last_peered":"2024-05-08T18:08:10.152986+0000","last_clean":"2024-05-08T18:08:10.152986+0000","last_became_active":"2024-05-08T18:04:49.834586+0000","last_became_peered":"2024-05-08T18:04:49.834586+0000","last_unstale":"2024-05-08T18:08:10.152986+0000","last_undegraded":"2024-05-08T18:08:10.152986+0000","last_fullsized":"2024-05-08T18:08:10.152986+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'96","last_scrub_stamp":"2024-05-08T18:08:10.152888+0000","last_deep_scrub":"29'96","last_deep_scrub_stamp":"2024-05-08T18:08:10.152888+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.152888+0000","objects_scrubbed":8,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:16.511455+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.455879+0000","last_change":"2024-05-08T18:08:08.455904+0000","last_active":"2024-05-08T18:08:08.455879+0000","last_peered":"2024-05-08T18:08:08.455879+0000","last_clean":"2024-05-08T18:08:08.455879+0000","last_became_active":"2024-05-08T18:04:51.830760+0000","last_became_peered":"2024-05-08T18:04:51.830760+0000","last_unstale":"2024-05-08T18:08:08.455879+0000","last_undegraded":"2024-05-08T18:08:08.455879+0000","last_fullsized":"2024-05-08T18:08:08.455879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:08.455773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:08.455773+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.455773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:12.703852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.342682+0000","last_change":"2024-05-08T18:08:21.342717+0000","last_active":"2024-05-08T18:08:21.342682+0000","last_peered":"2024-05-08T18:08:21.342682+0000","last_clean":"2024-05-08T18:08:21.342682+0000","last_became_active":"2024-05-08T18:04:53.838600+0000","last_became_peered":"2024-05-08T18:04:53.838600+0000","last_unstale":"2024-05-08T18:08:21.342682+0000","last_undegraded":"2024-05-08T18:08:21.342682+0000","last_fullsized":"2024-05-08T18:08:21.342682+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:21.342590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:21.342590+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.342590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:44.919676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"27'2","reported_seq":21,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744383+0000","last_change":"2024-05-08T18:04:53.839793+0000","last_active":"2024-05-08T18:06:58.744383+0000","last_peered":"2024-05-08T18:06:58.744383+0000","last_clean":"2024-05-08T18:06:58.744383+0000","last_became_active":"2024-05-08T18:04:53.839500+0000","last_became_peered":"2024-05-08T18:04:53.839500+0000","last_unstale":"2024-05-08T18:06:58.744383+0000","last_undegraded":"2024-05-08T18:06:58.744383+0000","last_fullsized":"2024-05-08T18:06:58.744383+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"29'8","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:39.209848+0000","last_change":"2024-05-08T18:08:39.209919+0000","last_active":"2024-05-08T18:08:39.209848+0000","last_peered":"2024-05-08T18:08:39.209848+0000","last_clean":"2024-05-08T18:08:39.209848+0000","last_became_active":"2024-05-08T18:04:45.819508+0000","last_became_peered":"2024-05-08T18:04:45.819508+0000","last_unstale":"2024-05-08T18:08:39.209848+0000","last_undegraded":"2024-05-08T18:08:39.209848+0000","last_fullsized":"2024-05-08T18:08:39.209848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:39.209750+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:39.209750+0000","last_clean_scrub_stamp":"2024-05-08T18:08:39.209750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:48.744139+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.746021+0000","last_change":"2024-05-08T18:04:43.844864+0000","last_active":"2024-05-08T18:07:03.746021+0000","last_peered":"2024-05-08T18:07:03.746021+0000","last_clean":"2024-05-08T18:07:03.746021+0000","last_became_active":"2024-05-08T18:04:43.837929+0000","last_became_peered":"2024-05-08T18:04:43.837929+0000","last_unstale":"2024-05-08T18:07:03.746021+0000","last_undegraded":"2024-05-08T18:07:03.746021+0000","last_fullsized":"2024-05-08T18:07:03.746021+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.504440+0000","last_change":"2024-05-08T18:08:19.504467+0000","last_active":"2024-05-08T18:08:19.504440+0000","last_peered":"2024-05-08T18:08:19.504440+0000","last_clean":"2024-05-08T18:08:19.504440+0000","last_became_active":"2024-05-08T18:04:47.818443+0000","last_became_peered":"2024-05-08T18:04:47.818443+0000","last_unstale":"2024-05-08T18:08:19.504440+0000","last_undegraded":"2024-05-08T18:08:19.504440+0000","last_fullsized":"2024-05-08T18:08:19.504440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:19.504350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:19.504350+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.504350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:24.197683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"29'40","reported_seq":96,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:30.848309+0000","last_change":"2024-05-08T18:08:30.848380+0000","last_active":"2024-05-08T18:08:30.848309+0000","last_peered":"2024-05-08T18:08:30.848309+0000","last_clean":"2024-05-08T18:08:30.848309+0000","last_became_active":"2024-05-08T18:04:49.833599+0000","last_became_peered":"2024-05-08T18:04:49.833599+0000","last_unstale":"2024-05-08T18:08:30.848309+0000","last_undegraded":"2024-05-08T18:08:30.848309+0000","last_fullsized":"2024-05-08T18:08:30.848309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'40","last_scrub_stamp":"2024-05-08T18:08:30.848207+0000","last_deep_scrub":"29'40","last_deep_scrub_stamp":"2024-05-08T18:08:30.848207+0000","last_clean_scrub_stamp":"2024-05-08T18:08:30.848207+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:31:32.685411+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:26.837175+0000","last_change":"2024-05-08T18:08:26.837221+0000","last_active":"2024-05-08T18:08:26.837175+0000","last_peered":"2024-05-08T18:08:26.837175+0000","last_clean":"2024-05-08T18:08:26.837175+0000","last_became_active":"2024-05-08T18:04:51.844526+0000","last_became_peered":"2024-05-08T18:04:51.844526+0000","last_unstale":"2024-05-08T18:08:26.837175+0000","last_undegraded":"2024-05-08T18:08:26.837175+0000","last_fullsized":"2024-05-08T18:08:26.837175+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:26.837073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:26.837073+0000","last_clean_scrub_stamp":"2024-05-08T18:08:26.837073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:03.923893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"29'11","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.739019+0000","last_change":"2024-05-08T18:04:53.850911+0000","last_active":"2024-05-08T18:06:53.739019+0000","last_peered":"2024-05-08T18:06:53.739019+0000","last_clean":"2024-05-08T18:06:53.739019+0000","last_became_active":"2024-05-08T18:04:53.850593+0000","last_became_peered":"2024-05-08T18:04:53.850593+0000","last_unstale":"2024-05-08T18:06:53.739019+0000","last_undegraded":"2024-05-08T18:06:53.739019+0000","last_fullsized":"2024-05-08T18:06:53.739019+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":62,"num_write":31,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"29'8","reported_seq":82,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:31.489072+0000","last_change":"2024-05-08T18:08:31.489126+0000","last_active":"2024-05-08T18:08:31.489072+0000","last_peered":"2024-05-08T18:08:31.489072+0000","last_clean":"2024-05-08T18:08:31.489072+0000","last_became_active":"2024-05-08T18:04:45.818979+0000","last_became_peered":"2024-05-08T18:04:45.818979+0000","last_unstale":"2024-05-08T18:08:31.489072+0000","last_undegraded":"2024-05-08T18:08:31.489072+0000","last_fullsized":"2024-05-08T18:08:31.489072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:31.488942+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:31.488942+0000","last_clean_scrub_stamp":"2024-05-08T18:08:31.488942+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:34.056789+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746094+0000","last_change":"2024-05-08T18:04:43.836993+0000","last_active":"2024-05-08T18:07:08.746094+0000","last_peered":"2024-05-08T18:07:08.746094+0000","last_clean":"2024-05-08T18:07:08.746094+0000","last_became_active":"2024-05-08T18:04:43.836614+0000","last_became_peered":"2024-05-08T18:04:43.836614+0000","last_unstale":"2024-05-08T18:07:08.746094+0000","last_undegraded":"2024-05-08T18:07:08.746094+0000","last_fullsized":"2024-05-08T18:07:08.746094+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.821717+0000","last_change":"2024-05-08T18:08:14.821740+0000","last_active":"2024-05-08T18:08:14.821717+0000","last_peered":"2024-05-08T18:08:14.821717+0000","last_clean":"2024-05-08T18:08:14.821717+0000","last_became_active":"2024-05-08T18:04:47.814561+0000","last_became_peered":"2024-05-08T18:04:47.814561+0000","last_unstale":"2024-05-08T18:08:14.821717+0000","last_undegraded":"2024-05-08T18:08:14.821717+0000","last_fullsized":"2024-05-08T18:08:14.821717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:14.821626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:14.821626+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.821626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:24.805865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"29'10","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.572538+0000","last_change":"2024-05-08T18:08:18.572575+0000","last_active":"2024-05-08T18:08:18.572538+0000","last_peered":"2024-05-08T18:08:18.572538+0000","last_clean":"2024-05-08T18:08:18.572538+0000","last_became_active":"2024-05-08T18:04:49.825347+0000","last_became_peered":"2024-05-08T18:04:49.825347+0000","last_unstale":"2024-05-08T18:08:18.572538+0000","last_undegraded":"2024-05-08T18:08:18.572538+0000","last_fullsized":"2024-05-08T18:08:18.572538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:18.572458+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:18.572458+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.572458+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:21.293654+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.319006+0000","last_change":"2024-05-08T18:08:17.319068+0000","last_active":"2024-05-08T18:08:17.319006+0000","last_peered":"2024-05-08T18:08:17.319006+0000","last_clean":"2024-05-08T18:08:17.319006+0000","last_became_active":"2024-05-08T18:04:51.843002+0000","last_became_peered":"2024-05-08T18:04:51.843002+0000","last_unstale":"2024-05-08T18:08:17.319006+0000","last_undegraded":"2024-05-08T18:08:17.319006+0000","last_fullsized":"2024-05-08T18:08:17.319006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:17.318901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:17.318901+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.318901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:07.004932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:23.743399+0000","last_change":"2024-05-08T18:04:53.840844+0000","last_active":"2024-05-08T18:07:23.743399+0000","last_peered":"2024-05-08T18:07:23.743399+0000","last_clean":"2024-05-08T18:07:23.743399+0000","last_became_active":"2024-05-08T18:04:53.840437+0000","last_became_peered":"2024-05-08T18:04:53.840437+0000","last_unstale":"2024-05-08T18:07:23.743399+0000","last_undegraded":"2024-05-08T18:07:23.743399+0000","last_fullsized":"2024-05-08T18:07:23.743399+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"29'8","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.867017+0000","last_change":"2024-05-08T18:05:30.868442+0000","last_active":"2024-05-08T18:07:53.867017+0000","last_peered":"2024-05-08T18:07:53.867017+0000","last_clean":"2024-05-08T18:07:53.867017+0000","last_became_active":"2024-05-08T18:05:30.868156+0000","last_became_peered":"2024-05-08T18:05:30.868156+0000","last_unstale":"2024-05-08T18:07:53.867017+0000","last_undegraded":"2024-05-08T18:07:53.867017+0000","last_fullsized":"2024-05-08T18:07:53.867017+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746159+0000","last_change":"2024-05-08T18:04:43.835856+0000","last_active":"2024-05-08T18:07:08.746159+0000","last_peered":"2024-05-08T18:07:08.746159+0000","last_clean":"2024-05-08T18:07:08.746159+0000","last_became_active":"2024-05-08T18:04:43.835435+0000","last_became_peered":"2024-05-08T18:04:43.835435+0000","last_unstale":"2024-05-08T18:07:08.746159+0000","last_undegraded":"2024-05-08T18:07:08.746159+0000","last_fullsized":"2024-05-08T18:07:08.746159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.385584+0000","last_change":"2024-05-08T18:08:19.385611+0000","last_active":"2024-05-08T18:08:19.385584+0000","last_peered":"2024-05-08T18:08:19.385584+0000","last_clean":"2024-05-08T18:08:19.385584+0000","last_became_active":"2024-05-08T18:04:47.824140+0000","last_became_peered":"2024-05-08T18:04:47.824140+0000","last_unstale":"2024-05-08T18:08:19.385584+0000","last_undegraded":"2024-05-08T18:08:19.385584+0000","last_fullsized":"2024-05-08T18:08:19.385584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:19.385463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:19.385463+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.385463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:15.297455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"29'48","reported_seq":120,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.582586+0000","last_change":"2024-05-08T18:08:19.582659+0000","last_active":"2024-05-08T18:08:19.582586+0000","last_peered":"2024-05-08T18:08:19.582586+0000","last_clean":"2024-05-08T18:08:19.582586+0000","last_became_active":"2024-05-08T18:04:49.824600+0000","last_became_peered":"2024-05-08T18:04:49.824600+0000","last_unstale":"2024-05-08T18:08:19.582586+0000","last_undegraded":"2024-05-08T18:08:19.582586+0000","last_fullsized":"2024-05-08T18:08:19.582586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'48","last_scrub_stamp":"2024-05-08T18:08:19.582484+0000","last_deep_scrub":"29'48","last_deep_scrub_stamp":"2024-05-08T18:08:19.582484+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.582484+0000","objects_scrubbed":6,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:25.159113+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":706,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.521105+0000","last_change":"2024-05-08T18:08:09.521130+0000","last_active":"2024-05-08T18:08:09.521105+0000","last_peered":"2024-05-08T18:08:09.521105+0000","last_clean":"2024-05-08T18:08:09.521105+0000","last_became_active":"2024-05-08T18:04:51.829937+0000","last_became_peered":"2024-05-08T18:04:51.829937+0000","last_unstale":"2024-05-08T18:08:09.521105+0000","last_undegraded":"2024-05-08T18:08:09.521105+0000","last_fullsized":"2024-05-08T18:08:09.521105+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:09.521013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:09.521013+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.521013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:29:12.370696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:23.749205+0000","last_change":"2024-05-08T18:04:53.842981+0000","last_active":"2024-05-08T18:07:23.749205+0000","last_peered":"2024-05-08T18:07:23.749205+0000","last_clean":"2024-05-08T18:07:23.749205+0000","last_became_active":"2024-05-08T18:04:53.842610+0000","last_became_peered":"2024-05-08T18:04:53.842610+0000","last_unstale":"2024-05-08T18:07:23.749205+0000","last_undegraded":"2024-05-08T18:07:23.749205+0000","last_fullsized":"2024-05-08T18:07:23.749205+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.866944+0000","last_change":"2024-05-08T18:08:22.866966+0000","last_active":"2024-05-08T18:08:22.866944+0000","last_peered":"2024-05-08T18:08:22.866944+0000","last_clean":"2024-05-08T18:08:22.866944+0000","last_became_active":"2024-05-08T18:04:45.829458+0000","last_became_peered":"2024-05-08T18:04:45.829458+0000","last_unstale":"2024-05-08T18:08:22.866944+0000","last_undegraded":"2024-05-08T18:08:22.866944+0000","last_fullsized":"2024-05-08T18:08:22.866944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.866853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.866853+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.866853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:05.403374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"29'4","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746252+0000","last_change":"2024-05-08T18:04:43.837089+0000","last_active":"2024-05-08T18:07:08.746252+0000","last_peered":"2024-05-08T18:07:08.746252+0000","last_clean":"2024-05-08T18:07:08.746252+0000","last_became_active":"2024-05-08T18:04:43.836633+0000","last_became_peered":"2024-05-08T18:04:43.836633+0000","last_unstale":"2024-05-08T18:07:08.746252+0000","last_undegraded":"2024-05-08T18:07:08.746252+0000","last_fullsized":"2024-05-08T18:07:08.746252+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"29'2","reported_seq":230,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.909561+0000","last_change":"2024-05-08T18:08:07.909590+0000","last_active":"2024-05-08T18:08:07.909561+0000","last_peered":"2024-05-08T18:08:07.909561+0000","last_clean":"2024-05-08T18:08:07.909561+0000","last_became_active":"2024-05-08T18:04:47.819771+0000","last_became_peered":"2024-05-08T18:04:47.819771+0000","last_unstale":"2024-05-08T18:08:07.909561+0000","last_undegraded":"2024-05-08T18:08:07.909561+0000","last_fullsized":"2024-05-08T18:08:07.909561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:07.909472+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:07.909472+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.909472+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:04.798235+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"29'36","reported_seq":91,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.344301+0000","last_change":"2024-05-08T18:08:18.344360+0000","last_active":"2024-05-08T18:08:18.344301+0000","last_peered":"2024-05-08T18:08:18.344301+0000","last_clean":"2024-05-08T18:08:18.344301+0000","last_became_active":"2024-05-08T18:04:49.824022+0000","last_became_peered":"2024-05-08T18:04:49.824022+0000","last_unstale":"2024-05-08T18:08:18.344301+0000","last_undegraded":"2024-05-08T18:08:18.344301+0000","last_fullsized":"2024-05-08T18:08:18.344301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:08:18.344202+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:08:18.344202+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.344202+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:36:47.464601+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.198536+0000","last_change":"2024-05-08T18:08:08.198575+0000","last_active":"2024-05-08T18:08:08.198536+0000","last_peered":"2024-05-08T18:08:08.198536+0000","last_clean":"2024-05-08T18:08:08.198536+0000","last_became_active":"2024-05-08T18:04:51.826424+0000","last_became_peered":"2024-05-08T18:04:51.826424+0000","last_unstale":"2024-05-08T18:08:08.198536+0000","last_undegraded":"2024-05-08T18:08:08.198536+0000","last_fullsized":"2024-05-08T18:08:08.198536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:08.198405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:08.198405+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.198405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:15.614149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"29'20","reported_seq":37,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.002699+0000","last_change":"2024-05-08T18:05:30.873315+0000","last_active":"2024-05-08T18:07:54.002699+0000","last_peered":"2024-05-08T18:07:54.002699+0000","last_clean":"2024-05-08T18:07:54.002699+0000","last_became_active":"2024-05-08T18:05:30.868409+0000","last_became_peered":"2024-05-08T18:05:30.868409+0000","last_unstale":"2024-05-08T18:07:54.002699+0000","last_undegraded":"2024-05-08T18:07:54.002699+0000","last_fullsized":"2024-05-08T18:07:54.002699+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746752+0000","last_change":"2024-05-08T18:04:43.844983+0000","last_active":"2024-05-08T18:07:08.746752+0000","last_peered":"2024-05-08T18:07:08.746752+0000","last_clean":"2024-05-08T18:07:08.746752+0000","last_became_active":"2024-05-08T18:04:43.838957+0000","last_became_peered":"2024-05-08T18:04:43.838957+0000","last_unstale":"2024-05-08T18:07:08.746752+0000","last_undegraded":"2024-05-08T18:07:08.746752+0000","last_fullsized":"2024-05-08T18:07:08.746752+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.606098+0000","last_change":"2024-05-08T18:08:23.606122+0000","last_active":"2024-05-08T18:08:23.606098+0000","last_peered":"2024-05-08T18:08:23.606098+0000","last_clean":"2024-05-08T18:08:23.606098+0000","last_became_active":"2024-05-08T18:04:47.820256+0000","last_became_peered":"2024-05-08T18:04:47.820256+0000","last_unstale":"2024-05-08T18:08:23.606098+0000","last_undegraded":"2024-05-08T18:08:23.606098+0000","last_fullsized":"2024-05-08T18:08:23.606098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:23.606002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:23.606002+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.606002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:04.849646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"29'63","reported_seq":136,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.582374+0000","last_change":"2024-05-08T18:08:16.582414+0000","last_active":"2024-05-08T18:08:16.582374+0000","last_peered":"2024-05-08T18:08:16.582374+0000","last_clean":"2024-05-08T18:08:16.582374+0000","last_became_active":"2024-05-08T18:04:49.825366+0000","last_became_peered":"2024-05-08T18:04:49.825366+0000","last_unstale":"2024-05-08T18:08:16.582374+0000","last_undegraded":"2024-05-08T18:08:16.582374+0000","last_fullsized":"2024-05-08T18:08:16.582374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'63","last_scrub_stamp":"2024-05-08T18:08:16.582295+0000","last_deep_scrub":"29'63","last_deep_scrub_stamp":"2024-05-08T18:08:16.582295+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.582295+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:21.275627+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.157727+0000","last_change":"2024-05-08T18:08:07.157775+0000","last_active":"2024-05-08T18:08:07.157727+0000","last_peered":"2024-05-08T18:08:07.157727+0000","last_clean":"2024-05-08T18:08:07.157727+0000","last_became_active":"2024-05-08T18:04:51.827127+0000","last_became_peered":"2024-05-08T18:04:51.827127+0000","last_unstale":"2024-05-08T18:08:07.157727+0000","last_undegraded":"2024-05-08T18:08:07.157727+0000","last_fullsized":"2024-05-08T18:08:07.157727+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:07.157606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:07.157606+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.157606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:37:53.105588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"29'14","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.741986+0000","last_change":"2024-05-08T18:04:53.844087+0000","last_active":"2024-05-08T18:07:13.741986+0000","last_peered":"2024-05-08T18:07:13.741986+0000","last_clean":"2024-05-08T18:07:13.741986+0000","last_became_active":"2024-05-08T18:04:53.843772+0000","last_became_peered":"2024-05-08T18:04:53.843772+0000","last_unstale":"2024-05-08T18:07:13.741986+0000","last_undegraded":"2024-05-08T18:07:13.741986+0000","last_fullsized":"2024-05-08T18:07:13.741986+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"29'2","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:43.350968+0000","last_change":"2024-05-08T18:08:43.351032+0000","last_active":"2024-05-08T18:08:43.350968+0000","last_peered":"2024-05-08T18:08:43.350968+0000","last_clean":"2024-05-08T18:08:43.350968+0000","last_became_active":"2024-05-08T18:04:45.811517+0000","last_became_peered":"2024-05-08T18:04:45.811517+0000","last_unstale":"2024-05-08T18:08:43.350968+0000","last_undegraded":"2024-05-08T18:08:43.350968+0000","last_fullsized":"2024-05-08T18:08:43.350968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:43.350844+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:43.350844+0000","last_clean_scrub_stamp":"2024-05-08T18:08:43.350844+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:35.421755+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746042+0000","last_change":"2024-05-08T18:04:43.836313+0000","last_active":"2024-05-08T18:07:08.746042+0000","last_peered":"2024-05-08T18:07:08.746042+0000","last_clean":"2024-05-08T18:07:08.746042+0000","last_became_active":"2024-05-08T18:04:43.834767+0000","last_became_peered":"2024-05-08T18:04:43.834767+0000","last_unstale":"2024-05-08T18:07:08.746042+0000","last_undegraded":"2024-05-08T18:07:08.746042+0000","last_fullsized":"2024-05-08T18:07:08.746042+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0,2],"acting":[1,3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:24.284995+0000","last_change":"2024-05-08T18:08:24.285019+0000","last_active":"2024-05-08T18:08:24.284995+0000","last_peered":"2024-05-08T18:08:24.284995+0000","last_clean":"2024-05-08T18:08:24.284995+0000","last_became_active":"2024-05-08T18:04:47.820662+0000","last_became_peered":"2024-05-08T18:04:47.820662+0000","last_unstale":"2024-05-08T18:08:24.284995+0000","last_undegraded":"2024-05-08T18:08:24.284995+0000","last_fullsized":"2024-05-08T18:08:24.284995+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:24.284899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:24.284899+0000","last_clean_scrub_stamp":"2024-05-08T18:08:24.284899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:02:49.472387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"29'46","reported_seq":104,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.901855+0000","last_change":"2024-05-08T18:08:10.901892+0000","last_active":"2024-05-08T18:08:10.901855+0000","last_peered":"2024-05-08T18:08:10.901855+0000","last_clean":"2024-05-08T18:08:10.901855+0000","last_became_active":"2024-05-08T18:04:49.830476+0000","last_became_peered":"2024-05-08T18:04:49.830476+0000","last_unstale":"2024-05-08T18:08:10.901855+0000","last_undegraded":"2024-05-08T18:08:10.901855+0000","last_fullsized":"2024-05-08T18:08:10.901855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'46","last_scrub_stamp":"2024-05-08T18:08:10.901767+0000","last_deep_scrub":"29'46","last_deep_scrub_stamp":"2024-05-08T18:08:10.901767+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.901767+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:07.184811+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.490196+0000","last_change":"2024-05-08T18:08:17.490220+0000","last_active":"2024-05-08T18:08:17.490196+0000","last_peered":"2024-05-08T18:08:17.490196+0000","last_clean":"2024-05-08T18:08:17.490196+0000","last_became_active":"2024-05-08T18:04:51.844912+0000","last_became_peered":"2024-05-08T18:04:51.844912+0000","last_unstale":"2024-05-08T18:08:17.490196+0000","last_undegraded":"2024-05-08T18:08:17.490196+0000","last_fullsized":"2024-05-08T18:08:17.490196+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:17.490110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:17.490110+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.490110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:06:17.806415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"29'2","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.043126+0000","last_change":"2024-05-08T18:04:53.847736+0000","last_active":"2024-05-08T18:07:54.043126+0000","last_peered":"2024-05-08T18:07:54.043126+0000","last_clean":"2024-05-08T18:07:54.043126+0000","last_became_active":"2024-05-08T18:04:53.847413+0000","last_became_peered":"2024-05-08T18:04:53.847413+0000","last_unstale":"2024-05-08T18:07:54.043126+0000","last_undegraded":"2024-05-08T18:07:54.043126+0000","last_fullsized":"2024-05-08T18:07:54.043126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"29'12","reported_seq":69,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.835658+0000","last_change":"2024-05-08T18:08:17.835711+0000","last_active":"2024-05-08T18:08:17.835658+0000","last_peered":"2024-05-08T18:08:17.835658+0000","last_clean":"2024-05-08T18:08:17.835658+0000","last_became_active":"2024-05-08T18:04:45.830329+0000","last_became_peered":"2024-05-08T18:04:45.830329+0000","last_unstale":"2024-05-08T18:08:17.835658+0000","last_undegraded":"2024-05-08T18:08:17.835658+0000","last_fullsized":"2024-05-08T18:08:17.835658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:17.835540+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:17.835540+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.835540+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:36.036099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.746942+0000","last_change":"2024-05-08T18:04:43.835662+0000","last_active":"2024-05-08T18:07:13.746942+0000","last_peered":"2024-05-08T18:07:13.746942+0000","last_clean":"2024-05-08T18:07:13.746942+0000","last_became_active":"2024-05-08T18:04:43.835227+0000","last_became_peered":"2024-05-08T18:04:43.835227+0000","last_unstale":"2024-05-08T18:07:13.746942+0000","last_undegraded":"2024-05-08T18:07:13.746942+0000","last_fullsized":"2024-05-08T18:07:13.746942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3,2],"acting":[1,0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.747677+0000","last_change":"2024-05-08T18:05:30.870636+0000","last_active":"2024-05-08T18:07:18.747677+0000","last_peered":"2024-05-08T18:07:18.747677+0000","last_clean":"2024-05-08T18:07:18.747677+0000","last_became_active":"2024-05-08T18:05:30.870155+0000","last_became_peered":"2024-05-08T18:05:30.870155+0000","last_unstale":"2024-05-08T18:07:18.747677+0000","last_undegraded":"2024-05-08T18:07:18.747677+0000","last_fullsized":"2024-05-08T18:07:18.747677+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"29'39","reported_seq":95,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.536504+0000","last_change":"2024-05-08T18:08:12.536541+0000","last_active":"2024-05-08T18:08:12.536504+0000","last_peered":"2024-05-08T18:08:12.536504+0000","last_clean":"2024-05-08T18:08:12.536504+0000","last_became_active":"2024-05-08T18:04:49.832649+0000","last_became_peered":"2024-05-08T18:04:49.832649+0000","last_unstale":"2024-05-08T18:08:12.536504+0000","last_undegraded":"2024-05-08T18:08:12.536504+0000","last_fullsized":"2024-05-08T18:08:12.536504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'39","last_scrub_stamp":"2024-05-08T18:08:12.536412+0000","last_deep_scrub":"29'39","last_deep_scrub_stamp":"2024-05-08T18:08:12.536412+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.536412+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:31.365352+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.444656+0000","last_change":"2024-05-08T18:08:05.444675+0000","last_active":"2024-05-08T18:08:05.444656+0000","last_peered":"2024-05-08T18:08:05.444656+0000","last_clean":"2024-05-08T18:08:05.444656+0000","last_became_active":"2024-05-08T18:04:51.833421+0000","last_became_peered":"2024-05-08T18:04:51.833421+0000","last_unstale":"2024-05-08T18:08:05.444656+0000","last_undegraded":"2024-05-08T18:08:05.444656+0000","last_fullsized":"2024-05-08T18:08:05.444656+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:05.444585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:05.444585+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.444585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:15.978668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"29'5","reported_seq":24,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747669+0000","last_change":"2024-05-08T18:04:53.848762+0000","last_active":"2024-05-08T18:07:13.747669+0000","last_peered":"2024-05-08T18:07:13.747669+0000","last_clean":"2024-05-08T18:07:13.747669+0000","last_became_active":"2024-05-08T18:04:53.848430+0000","last_became_peered":"2024-05-08T18:04:53.848430+0000","last_unstale":"2024-05-08T18:07:13.747669+0000","last_undegraded":"2024-05-08T18:07:13.747669+0000","last_fullsized":"2024-05-08T18:07:13.747669+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"29'10","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:37.437418+0000","last_change":"2024-05-08T18:08:37.437444+0000","last_active":"2024-05-08T18:08:37.437418+0000","last_peered":"2024-05-08T18:08:37.437418+0000","last_clean":"2024-05-08T18:08:37.437418+0000","last_became_active":"2024-05-08T18:04:45.806808+0000","last_became_peered":"2024-05-08T18:04:45.806808+0000","last_unstale":"2024-05-08T18:08:37.437418+0000","last_undegraded":"2024-05-08T18:08:37.437418+0000","last_fullsized":"2024-05-08T18:08:37.437418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:37.437332+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:37.437332+0000","last_clean_scrub_stamp":"2024-05-08T18:08:37.437332+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:55.637891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747763+0000","last_change":"2024-05-08T18:04:43.833094+0000","last_active":"2024-05-08T18:07:13.747763+0000","last_peered":"2024-05-08T18:07:13.747763+0000","last_clean":"2024-05-08T18:07:13.747763+0000","last_became_active":"2024-05-08T18:04:43.832147+0000","last_became_peered":"2024-05-08T18:04:43.832147+0000","last_unstale":"2024-05-08T18:07:13.747763+0000","last_undegraded":"2024-05-08T18:07:13.747763+0000","last_fullsized":"2024-05-08T18:07:13.747763+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.517382+0000","last_change":"2024-05-08T18:08:22.517404+0000","last_active":"2024-05-08T18:08:22.517382+0000","last_peered":"2024-05-08T18:08:22.517382+0000","last_clean":"2024-05-08T18:08:22.517382+0000","last_became_active":"2024-05-08T18:04:47.815219+0000","last_became_peered":"2024-05-08T18:04:47.815219+0000","last_unstale":"2024-05-08T18:08:22.517382+0000","last_undegraded":"2024-05-08T18:08:22.517382+0000","last_fullsized":"2024-05-08T18:08:22.517382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.517292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.517292+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.517292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:28.093920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"29'15","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:38.037851+0000","last_change":"2024-05-08T18:08:38.037896+0000","last_active":"2024-05-08T18:08:38.037851+0000","last_peered":"2024-05-08T18:08:38.037851+0000","last_clean":"2024-05-08T18:08:38.037851+0000","last_became_active":"2024-05-08T18:04:49.830793+0000","last_became_peered":"2024-05-08T18:04:49.830793+0000","last_unstale":"2024-05-08T18:08:38.037851+0000","last_undegraded":"2024-05-08T18:08:38.037851+0000","last_fullsized":"2024-05-08T18:08:38.037851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:38.037761+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:38.037761+0000","last_clean_scrub_stamp":"2024-05-08T18:08:38.037761+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:37:36.379302+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"29'285","reported_seq":378,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.160228+0000","last_change":"2024-05-08T18:08:09.160257+0000","last_active":"2024-05-08T18:08:09.160228+0000","last_peered":"2024-05-08T18:08:09.160228+0000","last_clean":"2024-05-08T18:08:09.160228+0000","last_became_active":"2024-05-08T18:04:51.833866+0000","last_became_peered":"2024-05-08T18:04:51.833866+0000","last_unstale":"2024-05-08T18:08:09.160228+0000","last_undegraded":"2024-05-08T18:08:09.160228+0000","last_fullsized":"2024-05-08T18:08:09.160228+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:09.160141+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:09.160141+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.160141+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:29:48.216550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747522+0000","last_change":"2024-05-08T18:04:53.848804+0000","last_active":"2024-05-08T18:07:13.747522+0000","last_peered":"2024-05-08T18:07:13.747522+0000","last_clean":"2024-05-08T18:07:13.747522+0000","last_became_active":"2024-05-08T18:04:53.848321+0000","last_became_peered":"2024-05-08T18:04:53.848321+0000","last_unstale":"2024-05-08T18:07:13.747522+0000","last_undegraded":"2024-05-08T18:07:13.747522+0000","last_fullsized":"2024-05-08T18:07:13.747522+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"29'4","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:33.249650+0000","last_change":"2024-05-08T18:08:33.249675+0000","last_active":"2024-05-08T18:08:33.249650+0000","last_peered":"2024-05-08T18:08:33.249650+0000","last_clean":"2024-05-08T18:08:33.249650+0000","last_became_active":"2024-05-08T18:04:45.813424+0000","last_became_peered":"2024-05-08T18:04:45.813424+0000","last_unstale":"2024-05-08T18:08:33.249650+0000","last_undegraded":"2024-05-08T18:08:33.249650+0000","last_fullsized":"2024-05-08T18:08:33.249650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:33.249561+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:33.249561+0000","last_clean_scrub_stamp":"2024-05-08T18:08:33.249561+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:14.493205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:42.532785+0000","last_change":"2024-05-08T18:08:42.532811+0000","last_active":"2024-05-08T18:08:42.532785+0000","last_peered":"2024-05-08T18:08:42.532785+0000","last_clean":"2024-05-08T18:08:42.532785+0000","last_became_active":"2024-05-08T18:04:43.840502+0000","last_became_peered":"2024-05-08T18:04:43.840502+0000","last_unstale":"2024-05-08T18:08:42.532785+0000","last_undegraded":"2024-05-08T18:08:42.532785+0000","last_fullsized":"2024-05-08T18:08:42.532785+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:42.532676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:42.532676+0000","last_clean_scrub_stamp":"2024-05-08T18:08:42.532676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:06.240119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.519639+0000","last_change":"2024-05-08T18:08:18.519666+0000","last_active":"2024-05-08T18:08:18.519639+0000","last_peered":"2024-05-08T18:08:18.519639+0000","last_clean":"2024-05-08T18:08:18.519639+0000","last_became_active":"2024-05-08T18:04:47.818839+0000","last_became_peered":"2024-05-08T18:04:47.818839+0000","last_unstale":"2024-05-08T18:08:18.519639+0000","last_undegraded":"2024-05-08T18:08:18.519639+0000","last_fullsized":"2024-05-08T18:08:18.519639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:18.519542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:18.519542+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.519542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:14:21.272156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"29'46","reported_seq":104,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.495547+0000","last_change":"2024-05-08T18:08:12.495592+0000","last_active":"2024-05-08T18:08:12.495547+0000","last_peered":"2024-05-08T18:08:12.495547+0000","last_clean":"2024-05-08T18:08:12.495547+0000","last_became_active":"2024-05-08T18:04:49.824405+0000","last_became_peered":"2024-05-08T18:04:49.824405+0000","last_unstale":"2024-05-08T18:08:12.495547+0000","last_undegraded":"2024-05-08T18:08:12.495547+0000","last_fullsized":"2024-05-08T18:08:12.495547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'46","last_scrub_stamp":"2024-05-08T18:08:12.495454+0000","last_deep_scrub":"29'46","last_deep_scrub_stamp":"2024-05-08T18:08:12.495454+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.495454+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:48:04.037407+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.562545+0000","last_change":"2024-05-08T18:08:17.562569+0000","last_active":"2024-05-08T18:08:17.562545+0000","last_peered":"2024-05-08T18:08:17.562545+0000","last_clean":"2024-05-08T18:08:17.562545+0000","last_became_active":"2024-05-08T18:04:51.844757+0000","last_became_peered":"2024-05-08T18:04:51.844757+0000","last_unstale":"2024-05-08T18:08:17.562545+0000","last_undegraded":"2024-05-08T18:08:17.562545+0000","last_fullsized":"2024-05-08T18:08:17.562545+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:17.562441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:17.562441+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.562441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:50:03.644980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:33.515849+0000","last_change":"2024-05-08T18:08:33.515879+0000","last_active":"2024-05-08T18:08:33.515849+0000","last_peered":"2024-05-08T18:08:33.515849+0000","last_clean":"2024-05-08T18:08:33.515849+0000","last_became_active":"2024-05-08T18:04:53.840956+0000","last_became_peered":"2024-05-08T18:04:53.840956+0000","last_unstale":"2024-05-08T18:08:33.515849+0000","last_undegraded":"2024-05-08T18:08:33.515849+0000","last_fullsized":"2024-05-08T18:08:33.515849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:33.515757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:33.515757+0000","last_clean_scrub_stamp":"2024-05-08T18:08:33.515757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:29.798801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"29'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:28.283892+0000","last_change":"2024-05-08T18:08:28.283933+0000","last_active":"2024-05-08T18:08:28.283892+0000","last_peered":"2024-05-08T18:08:28.283892+0000","last_clean":"2024-05-08T18:08:28.283892+0000","last_became_active":"2024-05-08T18:04:45.806560+0000","last_became_peered":"2024-05-08T18:04:45.806560+0000","last_unstale":"2024-05-08T18:08:28.283892+0000","last_undegraded":"2024-05-08T18:08:28.283892+0000","last_fullsized":"2024-05-08T18:08:28.283892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:28.283774+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:28.283774+0000","last_clean_scrub_stamp":"2024-05-08T18:08:28.283774+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:55:22.065846+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747834+0000","last_change":"2024-05-08T18:04:43.845626+0000","last_active":"2024-05-08T18:07:13.747834+0000","last_peered":"2024-05-08T18:07:13.747834+0000","last_clean":"2024-05-08T18:07:13.747834+0000","last_became_active":"2024-05-08T18:04:43.845214+0000","last_became_peered":"2024-05-08T18:04:43.845214+0000","last_unstale":"2024-05-08T18:07:13.747834+0000","last_undegraded":"2024-05-08T18:07:13.747834+0000","last_fullsized":"2024-05-08T18:07:13.747834+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.286599+0000","last_change":"2024-05-08T18:08:23.286625+0000","last_active":"2024-05-08T18:08:23.286599+0000","last_peered":"2024-05-08T18:08:23.286599+0000","last_clean":"2024-05-08T18:08:23.286599+0000","last_became_active":"2024-05-08T18:04:47.822748+0000","last_became_peered":"2024-05-08T18:04:47.822748+0000","last_unstale":"2024-05-08T18:08:23.286599+0000","last_undegraded":"2024-05-08T18:08:23.286599+0000","last_fullsized":"2024-05-08T18:08:23.286599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:23.286509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:23.286509+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.286509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:17:35.464618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"29'37","reported_seq":92,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.576260+0000","last_change":"2024-05-08T18:08:14.576300+0000","last_active":"2024-05-08T18:08:14.576260+0000","last_peered":"2024-05-08T18:08:14.576260+0000","last_clean":"2024-05-08T18:08:14.576260+0000","last_became_active":"2024-05-08T18:04:49.831440+0000","last_became_peered":"2024-05-08T18:04:49.831440+0000","last_unstale":"2024-05-08T18:08:14.576260+0000","last_undegraded":"2024-05-08T18:08:14.576260+0000","last_fullsized":"2024-05-08T18:08:14.576260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'37","last_scrub_stamp":"2024-05-08T18:08:14.576150+0000","last_deep_scrub":"29'37","last_deep_scrub_stamp":"2024-05-08T18:08:14.576150+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.576150+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:06:14.892455+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:27.822646+0000","last_change":"2024-05-08T18:08:27.822675+0000","last_active":"2024-05-08T18:08:27.822646+0000","last_peered":"2024-05-08T18:08:27.822646+0000","last_clean":"2024-05-08T18:08:27.822646+0000","last_became_active":"2024-05-08T18:04:51.844125+0000","last_became_peered":"2024-05-08T18:04:51.844125+0000","last_unstale":"2024-05-08T18:08:27.822646+0000","last_undegraded":"2024-05-08T18:08:27.822646+0000","last_fullsized":"2024-05-08T18:08:27.822646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:27.822555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:27.822555+0000","last_clean_scrub_stamp":"2024-05-08T18:08:27.822555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:47:50.940092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.748549+0000","last_change":"2024-05-08T18:04:53.849962+0000","last_active":"2024-05-08T18:07:18.748549+0000","last_peered":"2024-05-08T18:07:18.748549+0000","last_clean":"2024-05-08T18:07:18.748549+0000","last_became_active":"2024-05-08T18:04:53.849514+0000","last_became_peered":"2024-05-08T18:04:53.849514+0000","last_unstale":"2024-05-08T18:07:18.748549+0000","last_undegraded":"2024-05-08T18:07:18.748549+0000","last_fullsized":"2024-05-08T18:07:18.748549+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"29'4","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747631+0000","last_change":"2024-05-08T18:04:45.822495+0000","last_active":"2024-05-08T18:07:13.747631+0000","last_peered":"2024-05-08T18:07:13.747631+0000","last_clean":"2024-05-08T18:07:13.747631+0000","last_became_active":"2024-05-08T18:04:45.822204+0000","last_became_peered":"2024-05-08T18:04:45.822204+0000","last_unstale":"2024-05-08T18:07:13.747631+0000","last_undegraded":"2024-05-08T18:07:13.747631+0000","last_fullsized":"2024-05-08T18:07:13.747631+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"29'2","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:03.707416+0000","last_change":"2024-05-08T18:04:43.835654+0000","last_active":"2024-05-08T18:08:03.707416+0000","last_peered":"2024-05-08T18:08:03.707416+0000","last_clean":"2024-05-08T18:08:03.707416+0000","last_became_active":"2024-05-08T18:04:43.834678+0000","last_became_peered":"2024-05-08T18:04:43.834678+0000","last_unstale":"2024-05-08T18:08:03.707416+0000","last_undegraded":"2024-05-08T18:08:03.707416+0000","last_fullsized":"2024-05-08T18:08:03.707416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.501159+0000","last_change":"2024-05-08T18:08:16.501182+0000","last_active":"2024-05-08T18:08:16.501159+0000","last_peered":"2024-05-08T18:08:16.501159+0000","last_clean":"2024-05-08T18:08:16.501159+0000","last_became_active":"2024-05-08T18:04:47.820036+0000","last_became_peered":"2024-05-08T18:04:47.820036+0000","last_unstale":"2024-05-08T18:08:16.501159+0000","last_undegraded":"2024-05-08T18:08:16.501159+0000","last_fullsized":"2024-05-08T18:08:16.501159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:16.501065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:16.501065+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.501065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:49.649348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"29'32","reported_seq":85,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.857112+0000","last_change":"2024-05-08T18:08:20.857184+0000","last_active":"2024-05-08T18:08:20.857112+0000","last_peered":"2024-05-08T18:08:20.857112+0000","last_clean":"2024-05-08T18:08:20.857112+0000","last_became_active":"2024-05-08T18:04:49.830882+0000","last_became_peered":"2024-05-08T18:04:49.830882+0000","last_unstale":"2024-05-08T18:08:20.857112+0000","last_undegraded":"2024-05-08T18:08:20.857112+0000","last_fullsized":"2024-05-08T18:08:20.857112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'32","last_scrub_stamp":"2024-05-08T18:08:20.857010+0000","last_deep_scrub":"29'32","last_deep_scrub_stamp":"2024-05-08T18:08:20.857010+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.857010+0000","objects_scrubbed":2,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:06.682042+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":49,"num_write":14,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:29.815248+0000","last_change":"2024-05-08T18:08:29.815276+0000","last_active":"2024-05-08T18:08:29.815248+0000","last_peered":"2024-05-08T18:08:29.815248+0000","last_clean":"2024-05-08T18:08:29.815248+0000","last_became_active":"2024-05-08T18:04:51.842734+0000","last_became_peered":"2024-05-08T18:04:51.842734+0000","last_unstale":"2024-05-08T18:08:29.815248+0000","last_undegraded":"2024-05-08T18:08:29.815248+0000","last_fullsized":"2024-05-08T18:08:29.815248+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:29.815133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:29.815133+0000","last_clean_scrub_stamp":"2024-05-08T18:08:29.815133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:51:13.451798+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'10","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.779738+0000","last_change":"2024-05-08T18:04:53.845139+0000","last_active":"2024-05-08T18:07:53.779738+0000","last_peered":"2024-05-08T18:07:53.779738+0000","last_clean":"2024-05-08T18:07:53.779738+0000","last_became_active":"2024-05-08T18:04:53.844891+0000","last_became_peered":"2024-05-08T18:04:53.844891+0000","last_unstale":"2024-05-08T18:07:53.779738+0000","last_undegraded":"2024-05-08T18:07:53.779738+0000","last_fullsized":"2024-05-08T18:07:53.779738+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"29'2","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:27.291722+0000","last_change":"2024-05-08T18:08:27.291750+0000","last_active":"2024-05-08T18:08:27.291722+0000","last_peered":"2024-05-08T18:08:27.291722+0000","last_clean":"2024-05-08T18:08:27.291722+0000","last_became_active":"2024-05-08T18:04:45.808936+0000","last_became_peered":"2024-05-08T18:04:45.808936+0000","last_unstale":"2024-05-08T18:08:27.291722+0000","last_undegraded":"2024-05-08T18:08:27.291722+0000","last_fullsized":"2024-05-08T18:08:27.291722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:27.291616+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:27.291616+0000","last_clean_scrub_stamp":"2024-05-08T18:08:27.291616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:50:13.374155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:48.743083+0000","last_change":"2024-05-08T18:04:43.837820+0000","last_active":"2024-05-08T18:06:48.743083+0000","last_peered":"2024-05-08T18:06:48.743083+0000","last_clean":"2024-05-08T18:06:48.743083+0000","last_became_active":"2024-05-08T18:04:43.837070+0000","last_became_peered":"2024-05-08T18:04:43.837070+0000","last_unstale":"2024-05-08T18:06:48.743083+0000","last_undegraded":"2024-05-08T18:06:48.743083+0000","last_fullsized":"2024-05-08T18:06:48.743083+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"29'3","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.481725+0000","last_change":"2024-05-08T18:08:15.481759+0000","last_active":"2024-05-08T18:08:15.481725+0000","last_peered":"2024-05-08T18:08:15.481725+0000","last_clean":"2024-05-08T18:08:15.481725+0000","last_became_active":"2024-05-08T18:04:47.824172+0000","last_became_peered":"2024-05-08T18:04:47.824172+0000","last_unstale":"2024-05-08T18:08:15.481725+0000","last_undegraded":"2024-05-08T18:08:15.481725+0000","last_fullsized":"2024-05-08T18:08:15.481725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3","last_scrub_stamp":"2024-05-08T18:08:15.481627+0000","last_deep_scrub":"29'3","last_deep_scrub_stamp":"2024-05-08T18:08:15.481627+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.481627+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:31:00.387286+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"29'40","reported_seq":97,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:33.898989+0000","last_change":"2024-05-08T18:08:33.899046+0000","last_active":"2024-05-08T18:08:33.898989+0000","last_peered":"2024-05-08T18:08:33.898989+0000","last_clean":"2024-05-08T18:08:33.898989+0000","last_became_active":"2024-05-08T18:04:49.831803+0000","last_became_peered":"2024-05-08T18:04:49.831803+0000","last_unstale":"2024-05-08T18:08:33.898989+0000","last_undegraded":"2024-05-08T18:08:33.898989+0000","last_fullsized":"2024-05-08T18:08:33.898989+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'40","last_scrub_stamp":"2024-05-08T18:08:33.898897+0000","last_deep_scrub":"29'40","last_deep_scrub_stamp":"2024-05-08T18:08:33.898897+0000","last_clean_scrub_stamp":"2024-05-08T18:08:33.898897+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:05:09.280810+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1270,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:25.577300+0000","last_change":"2024-05-08T18:08:25.577348+0000","last_active":"2024-05-08T18:08:25.577300+0000","last_peered":"2024-05-08T18:08:25.577300+0000","last_clean":"2024-05-08T18:08:25.577300+0000","last_became_active":"2024-05-08T18:04:51.846225+0000","last_became_peered":"2024-05-08T18:04:51.846225+0000","last_unstale":"2024-05-08T18:08:25.577300+0000","last_undegraded":"2024-05-08T18:08:25.577300+0000","last_fullsized":"2024-05-08T18:08:25.577300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:25.577162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:25.577162+0000","last_clean_scrub_stamp":"2024-05-08T18:08:25.577162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:15.650574+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"29'4","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747795+0000","last_change":"2024-05-08T18:04:53.843030+0000","last_active":"2024-05-08T18:07:13.747795+0000","last_peered":"2024-05-08T18:07:13.747795+0000","last_clean":"2024-05-08T18:07:13.747795+0000","last_became_active":"2024-05-08T18:04:53.842429+0000","last_became_peered":"2024-05-08T18:04:53.842429+0000","last_unstale":"2024-05-08T18:07:13.747795+0000","last_undegraded":"2024-05-08T18:07:13.747795+0000","last_fullsized":"2024-05-08T18:07:13.747795+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"27'2","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:32.506118+0000","last_change":"2024-05-08T18:08:32.506149+0000","last_active":"2024-05-08T18:08:32.506118+0000","last_peered":"2024-05-08T18:08:32.506118+0000","last_clean":"2024-05-08T18:08:32.506118+0000","last_became_active":"2024-05-08T18:04:45.818161+0000","last_became_peered":"2024-05-08T18:04:45.818161+0000","last_unstale":"2024-05-08T18:08:32.506118+0000","last_undegraded":"2024-05-08T18:08:32.506118+0000","last_fullsized":"2024-05-08T18:08:32.506118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:32.506023+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:32.506023+0000","last_clean_scrub_stamp":"2024-05-08T18:08:32.506023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:20.670584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:38.182425+0000","last_change":"2024-05-08T18:08:38.182450+0000","last_active":"2024-05-08T18:08:38.182425+0000","last_peered":"2024-05-08T18:08:38.182425+0000","last_clean":"2024-05-08T18:08:38.182425+0000","last_became_active":"2024-05-08T18:04:43.842483+0000","last_became_peered":"2024-05-08T18:04:43.842483+0000","last_unstale":"2024-05-08T18:08:38.182425+0000","last_undegraded":"2024-05-08T18:08:38.182425+0000","last_fullsized":"2024-05-08T18:08:38.182425+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:38.182336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:38.182336+0000","last_clean_scrub_stamp":"2024-05-08T18:08:38.182336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:54:29.772068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"29'36","reported_seq":115,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:41.377102+0000","last_change":"2024-05-08T18:08:41.377160+0000","last_active":"2024-05-08T18:08:41.377102+0000","last_peered":"2024-05-08T18:08:41.377102+0000","last_clean":"2024-05-08T18:08:41.377102+0000","last_became_active":"2024-05-08T18:04:45.807847+0000","last_became_peered":"2024-05-08T18:04:45.807847+0000","last_unstale":"2024-05-08T18:08:41.377102+0000","last_undegraded":"2024-05-08T18:08:41.377102+0000","last_fullsized":"2024-05-08T18:08:41.377102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:08:41.376990+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:08:41.376990+0000","last_clean_scrub_stamp":"2024-05-08T18:08:41.376990+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:19:16.885027+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":44,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:36.472617+0000","last_change":"2024-05-08T18:08:36.472685+0000","last_active":"2024-05-08T18:08:36.472617+0000","last_peered":"2024-05-08T18:08:36.472617+0000","last_clean":"2024-05-08T18:08:36.472617+0000","last_became_active":"2024-05-08T18:04:43.836045+0000","last_became_peered":"2024-05-08T18:04:43.836045+0000","last_unstale":"2024-05-08T18:08:36.472617+0000","last_undegraded":"2024-05-08T18:08:36.472617+0000","last_fullsized":"2024-05-08T18:08:36.472617+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:36.472507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:36.472507+0000","last_clean_scrub_stamp":"2024-05-08T18:08:36.472507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:13.020912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3,2],"acting":[0,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"29'11","reported_seq":69,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.116104+0000","last_change":"2024-05-08T18:04:45.826393+0000","last_active":"2024-05-08T18:07:54.116104+0000","last_peered":"2024-05-08T18:07:54.116104+0000","last_clean":"2024-05-08T18:07:54.116104+0000","last_became_active":"2024-05-08T18:04:45.825506+0000","last_became_peered":"2024-05-08T18:04:45.825506+0000","last_unstale":"2024-05-08T18:07:54.116104+0000","last_undegraded":"2024-05-08T18:07:54.116104+0000","last_fullsized":"2024-05-08T18:07:54.116104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":46,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"29'6","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.668310+0000","last_change":"2024-05-08T18:04:43.832854+0000","last_active":"2024-05-08T18:08:00.668310+0000","last_peered":"2024-05-08T18:08:00.668310+0000","last_clean":"2024-05-08T18:08:00.668310+0000","last_became_active":"2024-05-08T18:04:43.832406+0000","last_became_peered":"2024-05-08T18:04:43.832406+0000","last_unstale":"2024-05-08T18:08:00.668310+0000","last_undegraded":"2024-05-08T18:08:00.668310+0000","last_fullsized":"2024-05-08T18:08:00.668310+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":8,"num_write":30,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"29'2","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:35.470944+0000","last_change":"2024-05-08T18:08:35.470976+0000","last_active":"2024-05-08T18:08:35.470944+0000","last_peered":"2024-05-08T18:08:35.470944+0000","last_clean":"2024-05-08T18:08:35.470944+0000","last_became_active":"2024-05-08T18:04:45.806646+0000","last_became_peered":"2024-05-08T18:04:45.806646+0000","last_unstale":"2024-05-08T18:08:35.470944+0000","last_undegraded":"2024-05-08T18:08:35.470944+0000","last_fullsized":"2024-05-08T18:08:35.470944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:35.470848+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:35.470848+0000","last_clean_scrub_stamp":"2024-05-08T18:08:35.470848+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:55.258041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"29'6","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:40.377246+0000","last_change":"2024-05-08T18:08:40.377297+0000","last_active":"2024-05-08T18:08:40.377246+0000","last_peered":"2024-05-08T18:08:40.377246+0000","last_clean":"2024-05-08T18:08:40.377246+0000","last_became_active":"2024-05-08T18:04:43.835163+0000","last_became_peered":"2024-05-08T18:04:43.835163+0000","last_unstale":"2024-05-08T18:08:40.377246+0000","last_undegraded":"2024-05-08T18:08:40.377246+0000","last_fullsized":"2024-05-08T18:08:40.377246+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:40.377108+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:40.377108+0000","last_clean_scrub_stamp":"2024-05-08T18:08:40.377108+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:26.202140+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":4746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,1],"acting":[0,3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"29'2","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:34.514209+0000","last_change":"2024-05-08T18:08:34.514234+0000","last_active":"2024-05-08T18:08:34.514209+0000","last_peered":"2024-05-08T18:08:34.514209+0000","last_clean":"2024-05-08T18:08:34.514209+0000","last_became_active":"2024-05-08T18:04:45.816391+0000","last_became_peered":"2024-05-08T18:04:45.816391+0000","last_unstale":"2024-05-08T18:08:34.514209+0000","last_undegraded":"2024-05-08T18:08:34.514209+0000","last_fullsized":"2024-05-08T18:08:34.514209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:34.514122+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:34.514122+0000","last_clean_scrub_stamp":"2024-05-08T18:08:34.514122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:11.960319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'4","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:42.343504+0000","last_change":"2024-05-08T18:08:42.343533+0000","last_active":"2024-05-08T18:08:42.343504+0000","last_peered":"2024-05-08T18:08:42.343504+0000","last_clean":"2024-05-08T18:08:42.343504+0000","last_became_active":"2024-05-08T18:04:43.835268+0000","last_became_peered":"2024-05-08T18:04:43.835268+0000","last_unstale":"2024-05-08T18:08:42.343504+0000","last_undegraded":"2024-05-08T18:08:42.343504+0000","last_fullsized":"2024-05-08T18:08:42.343504+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-08T18:08:42.343414+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-08T18:08:42.343414+0000","last_clean_scrub_stamp":"2024-05-08T18:08:42.343414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:24.879935+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"29'12","reported_seq":86,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:28.537320+0000","last_change":"2024-05-08T18:08:28.537351+0000","last_active":"2024-05-08T18:08:28.537320+0000","last_peered":"2024-05-08T18:08:28.537320+0000","last_clean":"2024-05-08T18:08:28.537320+0000","last_became_active":"2024-05-08T18:04:45.819699+0000","last_became_peered":"2024-05-08T18:04:45.819699+0000","last_unstale":"2024-05-08T18:08:28.537320+0000","last_undegraded":"2024-05-08T18:08:28.537320+0000","last_fullsized":"2024-05-08T18:08:28.537320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:28.537227+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:28.537227+0000","last_clean_scrub_stamp":"2024-05-08T18:08:28.537227+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:18.610639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.743834+0000","last_change":"2024-05-08T18:04:43.837155+0000","last_active":"2024-05-08T18:06:53.743834+0000","last_peered":"2024-05-08T18:06:53.743834+0000","last_clean":"2024-05-08T18:06:53.743834+0000","last_became_active":"2024-05-08T18:04:43.836735+0000","last_became_peered":"2024-05-08T18:04:43.836735+0000","last_unstale":"2024-05-08T18:06:53.743834+0000","last_undegraded":"2024-05-08T18:06:53.743834+0000","last_fullsized":"2024-05-08T18:06:53.743834+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"29'14","reported_seq":71,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.738934+0000","last_change":"2024-05-08T18:04:45.812327+0000","last_active":"2024-05-08T18:06:53.738934+0000","last_peered":"2024-05-08T18:06:53.738934+0000","last_clean":"2024-05-08T18:06:53.738934+0000","last_became_active":"2024-05-08T18:04:45.811953+0000","last_became_peered":"2024-05-08T18:04:45.811953+0000","last_unstale":"2024-05-08T18:06:53.738934+0000","last_undegraded":"2024-05-08T18:06:53.738934+0000","last_fullsized":"2024-05-08T18:06:53.738934+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:34.213459+0000","last_change":"2024-05-08T18:08:34.213484+0000","last_active":"2024-05-08T18:08:34.213459+0000","last_peered":"2024-05-08T18:08:34.213459+0000","last_clean":"2024-05-08T18:08:34.213459+0000","last_became_active":"2024-05-08T18:04:43.842179+0000","last_became_peered":"2024-05-08T18:04:43.842179+0000","last_unstale":"2024-05-08T18:08:34.213459+0000","last_undegraded":"2024-05-08T18:08:34.213459+0000","last_fullsized":"2024-05-08T18:08:34.213459+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:34.213370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:34.213370+0000","last_clean_scrub_stamp":"2024-05-08T18:08:34.213370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:13.048212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,0],"acting":[2,3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"29'8","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.070501+0000","last_change":"2024-05-08T18:04:45.812893+0000","last_active":"2024-05-08T18:07:54.070501+0000","last_peered":"2024-05-08T18:07:54.070501+0000","last_clean":"2024-05-08T18:07:54.070501+0000","last_became_active":"2024-05-08T18:04:45.812637+0000","last_became_peered":"2024-05-08T18:04:45.812637+0000","last_unstale":"2024-05-08T18:07:54.070501+0000","last_undegraded":"2024-05-08T18:07:54.070501+0000","last_fullsized":"2024-05-08T18:07:54.070501+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"29'6","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.984349+0000","last_change":"2024-05-08T18:04:43.834951+0000","last_active":"2024-05-08T18:07:53.984349+0000","last_peered":"2024-05-08T18:07:53.984349+0000","last_clean":"2024-05-08T18:07:53.984349+0000","last_became_active":"2024-05-08T18:04:43.834491+0000","last_became_peered":"2024-05-08T18:04:43.834491+0000","last_unstale":"2024-05-08T18:07:53.984349+0000","last_undegraded":"2024-05-08T18:07:53.984349+0000","last_fullsized":"2024-05-08T18:07:53.984349+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"29'8","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:34.558842+0000","last_change":"2024-05-08T18:08:34.558889+0000","last_active":"2024-05-08T18:08:34.558842+0000","last_peered":"2024-05-08T18:08:34.558842+0000","last_clean":"2024-05-08T18:08:34.558842+0000","last_became_active":"2024-05-08T18:04:45.815677+0000","last_became_peered":"2024-05-08T18:04:45.815677+0000","last_unstale":"2024-05-08T18:08:34.558842+0000","last_undegraded":"2024-05-08T18:08:34.558842+0000","last_fullsized":"2024-05-08T18:08:34.558842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:34.558754+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:34.558754+0000","last_clean_scrub_stamp":"2024-05-08T18:08:34.558754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:30.175682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:53.744094+0000","last_change":"2024-05-08T18:04:43.846054+0000","last_active":"2024-05-08T18:06:53.744094+0000","last_peered":"2024-05-08T18:06:53.744094+0000","last_clean":"2024-05-08T18:06:53.744094+0000","last_became_active":"2024-05-08T18:04:43.843826+0000","last_became_peered":"2024-05-08T18:04:43.843826+0000","last_unstale":"2024-05-08T18:06:53.744094+0000","last_undegraded":"2024-05-08T18:06:53.744094+0000","last_fullsized":"2024-05-08T18:06:53.744094+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"29'6","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:43.071536+0000","last_change":"2024-05-08T18:08:43.071567+0000","last_active":"2024-05-08T18:08:43.071536+0000","last_peered":"2024-05-08T18:08:43.071536+0000","last_clean":"2024-05-08T18:08:43.071536+0000","last_became_active":"2024-05-08T18:04:45.827252+0000","last_became_peered":"2024-05-08T18:04:45.827252+0000","last_unstale":"2024-05-08T18:08:43.071536+0000","last_undegraded":"2024-05-08T18:08:43.071536+0000","last_fullsized":"2024-05-08T18:08:43.071536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:43.071451+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:43.071451+0000","last_clean_scrub_stamp":"2024-05-08T18:08:43.071451+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:41:02.800248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.739685+0000","last_change":"2024-05-08T18:04:43.820995+0000","last_active":"2024-05-08T18:06:58.739685+0000","last_peered":"2024-05-08T18:06:58.739685+0000","last_clean":"2024-05-08T18:06:58.739685+0000","last_became_active":"2024-05-08T18:04:43.820360+0000","last_became_peered":"2024-05-08T18:04:43.820360+0000","last_unstale":"2024-05-08T18:06:58.739685+0000","last_undegraded":"2024-05-08T18:06:58.739685+0000","last_fullsized":"2024-05-08T18:06:58.739685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:31.445820+0000","last_change":"2024-05-08T18:08:31.445845+0000","last_active":"2024-05-08T18:08:31.445820+0000","last_peered":"2024-05-08T18:08:31.445820+0000","last_clean":"2024-05-08T18:08:31.445820+0000","last_became_active":"2024-05-08T18:04:45.823385+0000","last_became_peered":"2024-05-08T18:04:45.823385+0000","last_unstale":"2024-05-08T18:08:31.445820+0000","last_undegraded":"2024-05-08T18:08:31.445820+0000","last_fullsized":"2024-05-08T18:08:31.445820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:31.445729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:31.445729+0000","last_clean_scrub_stamp":"2024-05-08T18:08:31.445729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:27.062658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'6","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.757844+0000","last_change":"2024-05-08T18:04:43.836532+0000","last_active":"2024-05-08T18:07:53.757844+0000","last_peered":"2024-05-08T18:07:53.757844+0000","last_clean":"2024-05-08T18:07:53.757844+0000","last_became_active":"2024-05-08T18:04:43.835838+0000","last_became_peered":"2024-05-08T18:04:43.835838+0000","last_unstale":"2024-05-08T18:07:53.757844+0000","last_undegraded":"2024-05-08T18:07:53.757844+0000","last_fullsized":"2024-05-08T18:07:53.757844+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8,"num_write":30,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"29'10","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:41.515456+0000","last_change":"2024-05-08T18:08:41.515484+0000","last_active":"2024-05-08T18:08:41.515456+0000","last_peered":"2024-05-08T18:08:41.515456+0000","last_clean":"2024-05-08T18:08:41.515456+0000","last_became_active":"2024-05-08T18:04:45.810703+0000","last_became_peered":"2024-05-08T18:04:45.810703+0000","last_unstale":"2024-05-08T18:08:41.515456+0000","last_undegraded":"2024-05-08T18:08:41.515456+0000","last_fullsized":"2024-05-08T18:08:41.515456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:41.515361+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:41.515361+0000","last_clean_scrub_stamp":"2024-05-08T18:08:41.515361+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:54:04.536630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"29'4","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:00.668657+0000","last_change":"2024-05-08T18:04:43.836123+0000","last_active":"2024-05-08T18:08:00.668657+0000","last_peered":"2024-05-08T18:08:00.668657+0000","last_clean":"2024-05-08T18:08:00.668657+0000","last_became_active":"2024-05-08T18:04:43.835518+0000","last_became_peered":"2024-05-08T18:04:43.835518+0000","last_unstale":"2024-05-08T18:08:00.668657+0000","last_undegraded":"2024-05-08T18:08:00.668657+0000","last_fullsized":"2024-05-08T18:08:00.668657+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,3],"acting":[1,2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"29'4","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:26.522634+0000","last_change":"2024-05-08T18:08:26.522702+0000","last_active":"2024-05-08T18:08:26.522634+0000","last_peered":"2024-05-08T18:08:26.522634+0000","last_clean":"2024-05-08T18:08:26.522634+0000","last_became_active":"2024-05-08T18:04:45.820746+0000","last_became_peered":"2024-05-08T18:04:45.820746+0000","last_unstale":"2024-05-08T18:08:26.522634+0000","last_undegraded":"2024-05-08T18:08:26.522634+0000","last_fullsized":"2024-05-08T18:08:26.522634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:26.522525+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:26.522525+0000","last_clean_scrub_stamp":"2024-05-08T18:08:26.522525+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:07.766169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.745035+0000","last_change":"2024-05-08T18:04:43.847998+0000","last_active":"2024-05-08T18:06:58.745035+0000","last_peered":"2024-05-08T18:06:58.745035+0000","last_clean":"2024-05-08T18:06:58.745035+0000","last_became_active":"2024-05-08T18:04:43.844404+0000","last_became_peered":"2024-05-08T18:04:43.844404+0000","last_unstale":"2024-05-08T18:06:58.745035+0000","last_undegraded":"2024-05-08T18:06:58.745035+0000","last_fullsized":"2024-05-08T18:06:58.745035+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"29'4","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:39.501832+0000","last_change":"2024-05-08T18:08:39.501858+0000","last_active":"2024-05-08T18:08:39.501832+0000","last_peered":"2024-05-08T18:08:39.501832+0000","last_clean":"2024-05-08T18:08:39.501832+0000","last_became_active":"2024-05-08T18:04:45.809271+0000","last_became_peered":"2024-05-08T18:04:45.809271+0000","last_unstale":"2024-05-08T18:08:39.501832+0000","last_undegraded":"2024-05-08T18:08:39.501832+0000","last_fullsized":"2024-05-08T18:08:39.501832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:39.501726+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:39.501726+0000","last_clean_scrub_stamp":"2024-05-08T18:08:39.501726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:16.050132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.745103+0000","last_change":"2024-05-08T18:04:43.844366+0000","last_active":"2024-05-08T18:06:58.745103+0000","last_peered":"2024-05-08T18:06:58.745103+0000","last_clean":"2024-05-08T18:06:58.745103+0000","last_became_active":"2024-05-08T18:04:43.838125+0000","last_became_peered":"2024-05-08T18:04:43.838125+0000","last_unstale":"2024-05-08T18:06:58.745103+0000","last_undegraded":"2024-05-08T18:06:58.745103+0000","last_fullsized":"2024-05-08T18:06:58.745103+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"29'15","reported_seq":84,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.868964+0000","last_change":"2024-05-08T18:08:21.868994+0000","last_active":"2024-05-08T18:08:21.868964+0000","last_peered":"2024-05-08T18:08:21.868964+0000","last_clean":"2024-05-08T18:08:21.868964+0000","last_became_active":"2024-05-08T18:04:45.828488+0000","last_became_peered":"2024-05-08T18:04:45.828488+0000","last_unstale":"2024-05-08T18:08:21.868964+0000","last_undegraded":"2024-05-08T18:08:21.868964+0000","last_fullsized":"2024-05-08T18:08:21.868964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:21.868870+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:21.868870+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.868870+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:18:57.376906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:32.274959+0000","last_change":"2024-05-08T18:08:32.274984+0000","last_active":"2024-05-08T18:08:32.274959+0000","last_peered":"2024-05-08T18:08:32.274959+0000","last_clean":"2024-05-08T18:08:32.274959+0000","last_became_active":"2024-05-08T18:04:43.842670+0000","last_became_peered":"2024-05-08T18:04:43.842670+0000","last_unstale":"2024-05-08T18:08:32.274959+0000","last_undegraded":"2024-05-08T18:08:32.274959+0000","last_fullsized":"2024-05-08T18:08:32.274959+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:32.274865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:32.274865+0000","last_clean_scrub_stamp":"2024-05-08T18:08:32.274865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:55.489696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.744378+0000","last_change":"2024-05-08T18:04:45.812074+0000","last_active":"2024-05-08T18:06:58.744378+0000","last_peered":"2024-05-08T18:06:58.744378+0000","last_clean":"2024-05-08T18:06:58.744378+0000","last_became_active":"2024-05-08T18:04:45.810562+0000","last_became_peered":"2024-05-08T18:04:45.810562+0000","last_unstale":"2024-05-08T18:06:58.744378+0000","last_undegraded":"2024-05-08T18:06:58.744378+0000","last_fullsized":"2024-05-08T18:06:58.744378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.746137+0000","last_change":"2024-05-08T18:04:43.838746+0000","last_active":"2024-05-08T18:07:03.746137+0000","last_peered":"2024-05-08T18:07:03.746137+0000","last_clean":"2024-05-08T18:07:03.746137+0000","last_became_active":"2024-05-08T18:04:43.835943+0000","last_became_peered":"2024-05-08T18:04:43.835943+0000","last_unstale":"2024-05-08T18:07:03.746137+0000","last_undegraded":"2024-05-08T18:07:03.746137+0000","last_fullsized":"2024-05-08T18:07:03.746137+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745822+0000","last_change":"2024-05-08T18:04:43.836378+0000","last_active":"2024-05-08T18:07:03.745822+0000","last_peered":"2024-05-08T18:07:03.745822+0000","last_clean":"2024-05-08T18:07:03.745822+0000","last_became_active":"2024-05-08T18:04:43.836007+0000","last_became_peered":"2024-05-08T18:04:43.836007+0000","last_unstale":"2024-05-08T18:07:03.745822+0000","last_undegraded":"2024-05-08T18:07:03.745822+0000","last_fullsized":"2024-05-08T18:07:03.745822+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"29'8","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.742097+0000","last_change":"2024-05-08T18:04:45.812333+0000","last_active":"2024-05-08T18:07:13.742097+0000","last_peered":"2024-05-08T18:07:13.742097+0000","last_clean":"2024-05-08T18:07:13.742097+0000","last_became_active":"2024-05-08T18:04:45.811648+0000","last_became_peered":"2024-05-08T18:04:45.811648+0000","last_unstale":"2024-05-08T18:07:13.742097+0000","last_undegraded":"2024-05-08T18:07:13.742097+0000","last_fullsized":"2024-05-08T18:07:13.742097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":31,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"29'6","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:38.419488+0000","last_change":"2024-05-08T18:08:38.419521+0000","last_active":"2024-05-08T18:08:38.419488+0000","last_peered":"2024-05-08T18:08:38.419488+0000","last_clean":"2024-05-08T18:08:38.419488+0000","last_became_active":"2024-05-08T18:04:43.835712+0000","last_became_peered":"2024-05-08T18:04:43.835712+0000","last_unstale":"2024-05-08T18:08:38.419488+0000","last_undegraded":"2024-05-08T18:08:38.419488+0000","last_fullsized":"2024-05-08T18:08:38.419488+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:38.419396+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:38.419396+0000","last_clean_scrub_stamp":"2024-05-08T18:08:38.419396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:54:01.440665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":22,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"29'10","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.782661+0000","last_change":"2024-05-08T18:04:45.812258+0000","last_active":"2024-05-08T18:07:53.782661+0000","last_peered":"2024-05-08T18:07:53.782661+0000","last_clean":"2024-05-08T18:07:53.782661+0000","last_became_active":"2024-05-08T18:04:45.811921+0000","last_became_peered":"2024-05-08T18:04:45.811921+0000","last_unstale":"2024-05-08T18:07:53.782661+0000","last_undegraded":"2024-05-08T18:07:53.782661+0000","last_fullsized":"2024-05-08T18:07:53.782661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2281,"ondisk_log_size":2281,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1600,"num_read_kb":1397,"num_write":597,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3000,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1527,"ondisk_log_size":1527,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":480,"ondisk_log_size":480,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":145,"num_read_kb":87,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":256,"acting":256,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705716,"num_pgs":179,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":438272,"data_stored":135643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705717,"num_pgs":153,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44604,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327236,"statfs":{"total":96636764160,"available":96591089664,"internally_reserved":0,"allocated":970752,"data_stored":724153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705717,"num_pgs":157,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40464,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331376,"statfs":{"total":96636764160,"available":96595329024,"internally_reserved":0,"allocated":991232,"data_stored":724641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705717,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":409600,"data_stored":135609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T18:08:46.232 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.232 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.232 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.232 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.232 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.233 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.233 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.233 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.233 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.233 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.233 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.233 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.234 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.234 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.234 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.234 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.234 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.234 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.234 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.234 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.235 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.235 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.235 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.235 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.235 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.235 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.235 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.235 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.236 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.236 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.236 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.236 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.236 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.236 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.236 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.236 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.237 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.237 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.237 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.237 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.237 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.237 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.237 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.238 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.238 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.238 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.238 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.238 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.238 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.238 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.239 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.239 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.239 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.239 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.239 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.239 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.239 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.239 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.240 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.240 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.240 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.240 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.240 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.240 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.240 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.240 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.241 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.241 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.241 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.241 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.241 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.241 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.241 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.241 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.242 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.242 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.242 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.242 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.242 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.242 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.242 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.243 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.243 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.243 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.243 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.243 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.243 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.243 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.243 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.244 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.244 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.244 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.244 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.244 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.244 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.244 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.245 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.245 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.245 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.245 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.245 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.245 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.245 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.246 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.246 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.246 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.246 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.246 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:08:46.246 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T18:09:06.247 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T18:09:06.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.364+0000 7fe12f6cf640 1 Processor -- start 2024-05-08T18:09:06.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.364+0000 7fe12f6cf640 1 -- start start 2024-05-08T18:09:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.365+0000 7fe12f6cf640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe128070650 0x7fe128070a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:09:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.365+0000 7fe12f6cf640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe128073480 con 0x7fe128072540 2024-05-08T18:09:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.365+0000 7fe12f6cf640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fe128071a80 con 0x7fe128070650 2024-05-08T18:09:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.365+0000 7fe12d46b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe128070650 0x7fe128070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:09:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.365+0000 7fe12cc6a640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe128072540 0x7fe12812f580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59226/0 (socket says 172.21.15.46:59226) 2024-05-08T18:09:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.365+0000 7fe12cc6a640 1 -- 172.21.15.46:0/616982431 learned_addr learned my addr 172.21.15.46:0/616982431 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:09:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.365+0000 7fe12d46b640 1 -- 172.21.15.46:0/616982431 >> v1:172.21.15.46:6789/0 conn(0x7fe128072540 legacy=0x7fe12812f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:09:06.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.365+0000 7fe12d46b640 1 -- 172.21.15.46:0/616982431 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe128072c20 con 0x7fe128070650 2024-05-08T18:09:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.366+0000 7fe12d46b640 1 --2- 172.21.15.46:0/616982431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128070650 0x7fe128070a30 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe124009720 tx=0x7fe12402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c12eff70644f7d05 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:09:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.366+0000 7fe11ffff640 1 -- 172.21.15.46:0/616982431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1240316d0 con 0x7fe128070650 2024-05-08T18:09:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.366+0000 7fe11ffff640 1 -- 172.21.15.46:0/616982431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe124037070 con 0x7fe128070650 2024-05-08T18:09:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.366+0000 7fe11ffff640 1 -- 172.21.15.46:0/616982431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1240319c0 con 0x7fe128070650 2024-05-08T18:09:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.367+0000 7fe12f6cf640 1 -- 172.21.15.46:0/616982431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128070650 msgr2=0x7fe128070a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:09:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.367+0000 7fe12f6cf640 1 --2- 172.21.15.46:0/616982431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128070650 0x7fe128070a30 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe124009720 tx=0x7fe12402b840 comp rx=0 tx=0).stop 2024-05-08T18:09:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.367+0000 7fe12f6cf640 1 -- 172.21.15.46:0/616982431 shutdown_connections 2024-05-08T18:09:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.367+0000 7fe12f6cf640 1 --2- 172.21.15.46:0/616982431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128070650 0x7fe128070a30 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:09:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.367+0000 7fe12f6cf640 1 -- 172.21.15.46:0/616982431 >> 172.21.15.46:0/616982431 conn(0x7fe1280a1e60 msgr2=0x7fe1280a2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:09:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.367+0000 7fe12f6cf640 1 -- 172.21.15.46:0/616982431 shutdown_connections 2024-05-08T18:09:06.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.367+0000 7fe12f6cf640 1 -- 172.21.15.46:0/616982431 wait complete. 2024-05-08T18:09:06.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.368+0000 7fe12f6cf640 1 Processor -- start 2024-05-08T18:09:06.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.368+0000 7fe12f6cf640 1 -- start start 2024-05-08T18:09:06.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.368+0000 7fe12f6cf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128070650 0x7fe1281c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:09:06.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.368+0000 7fe12f6cf640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe12812fc90 con 0x7fe128070650 2024-05-08T18:09:06.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.368+0000 7fe12d46b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128070650 0x7fe1281c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:09:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.368+0000 7fe12d46b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128070650 0x7fe1281c4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41698/0 (socket says 172.21.15.46:41698) 2024-05-08T18:09:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.368+0000 7fe12d46b640 1 -- 172.21.15.46:0/2261771045 learned_addr learned my addr 172.21.15.46:0/2261771045 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:09:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.369+0000 7fe12d46b640 1 -- 172.21.15.46:0/2261771045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12812c5d0 con 0x7fe128070650 2024-05-08T18:09:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.369+0000 7fe12d46b640 1 --2- 172.21.15.46:0/2261771045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128070650 0x7fe1281c4450 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe124034040 tx=0x7fe1240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:09:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.369+0000 7fe11dffb640 1 -- 172.21.15.46:0/2261771045 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe124032770 con 0x7fe128070650 2024-05-08T18:09:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.369+0000 7fe11dffb640 1 -- 172.21.15.46:0/2261771045 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe124032910 con 0x7fe128070650 2024-05-08T18:09:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.369+0000 7fe11dffb640 1 -- 172.21.15.46:0/2261771045 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe124032c30 con 0x7fe128070650 2024-05-08T18:09:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.369+0000 7fe12f6cf640 1 -- 172.21.15.46:0/2261771045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12811e9b0 con 0x7fe128070650 2024-05-08T18:09:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.369+0000 7fe12f6cf640 1 -- 172.21.15.46:0/2261771045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe12811f640 con 0x7fe128070650 2024-05-08T18:09:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.370+0000 7fe11dffb640 1 -- 172.21.15.46:0/2261771045 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fe12404c060 con 0x7fe128070650 2024-05-08T18:09:06.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.370+0000 7fe12f6cf640 1 -- 172.21.15.46:0/2261771045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe128120150 con 0x7fe128070650 2024-05-08T18:09:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.371+0000 7fe11dffb640 1 --2- 172.21.15.46:0/2261771045 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe10c03d2b0 0x7fe10c03f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:09:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.371+0000 7fe11dffb640 1 -- 172.21.15.46:0/2261771045 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fe12402f050 con 0x7fe128070650 2024-05-08T18:09:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.371+0000 7fe12cc6a640 1 --2- 172.21.15.46:0/2261771045 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe10c03d2b0 0x7fe10c03f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:09:06.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.372+0000 7fe12cc6a640 1 --2- 172.21.15.46:0/2261771045 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe10c03d2b0 0x7fe10c03f770 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe118007fd0 tx=0x7fe118007aa0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:09:06.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.376+0000 7fe11dffb640 1 -- 172.21.15.46:0/2261771045 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fe12812c5d0 con 0x7fe128070650 2024-05-08T18:09:06.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.562+0000 7fe12f6cf640 1 -- 172.21.15.46:0/2261771045 --> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe1280640a0 con 0x7fe10c03d2b0 2024-05-08T18:09:06.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.578+0000 7fe11dffb640 1 -- 172.21.15.46:0/2261771045 <== mgr.4098 v2:172.21.15.50:6824/2602185226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665015 (secure 0 0 0) 0x7fe1280640a0 con 0x7fe10c03d2b0 2024-05-08T18:09:06.583 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:09:06.589 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T18:09:06.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.582+0000 7fe12f6cf640 1 -- 172.21.15.46:0/2261771045 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe10c03d2b0 msgr2=0x7fe10c03f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:09:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.583+0000 7fe12f6cf640 1 --2- 172.21.15.46:0/2261771045 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe10c03d2b0 0x7fe10c03f770 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe118007fd0 tx=0x7fe118007aa0 comp rx=0 tx=0).stop 2024-05-08T18:09:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.583+0000 7fe12f6cf640 1 -- 172.21.15.46:0/2261771045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128070650 msgr2=0x7fe1281c4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:09:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.583+0000 7fe12f6cf640 1 --2- 172.21.15.46:0/2261771045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128070650 0x7fe1281c4450 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe124034040 tx=0x7fe1240046c0 comp rx=0 tx=0).stop 2024-05-08T18:09:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.583+0000 7fe12f6cf640 1 -- 172.21.15.46:0/2261771045 shutdown_connections 2024-05-08T18:09:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.583+0000 7fe12f6cf640 1 --2- 172.21.15.46:0/2261771045 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fe10c03d2b0 0x7fe10c03f770 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:09:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.583+0000 7fe12f6cf640 1 --2- 172.21.15.46:0/2261771045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe128070650 0x7fe1281c4450 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:09:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.583+0000 7fe12f6cf640 1 -- 172.21.15.46:0/2261771045 >> 172.21.15.46:0/2261771045 conn(0x7fe1280a1e60 msgr2=0x7fe12812bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:09:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.583+0000 7fe12f6cf640 1 -- 172.21.15.46:0/2261771045 shutdown_connections 2024-05-08T18:09:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:06.583+0000 7fe12f6cf640 1 -- 172.21.15.46:0/2261771045 wait complete. 2024-05-08T18:09:06.592 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-05-08T18:09:05.749677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605383,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":5050,"num_read_kb":4636,"num_write":2119,"num_write_kb":60585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3349,"num_omap_keys":16,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4739,"ondisk_log_size":4739,"up":658,"acting":658,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":658,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":144088,"kb_used_data":2744,"kb_used_omap":6,"kb_used_meta":141305,"kb_avail":377343272,"statfs":{"total":386547056640,"available":386399510528,"internally_reserved":0,"allocated":2809856,"data_stored":1720046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":144697110},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":180,"num_omap_keys":1,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003637"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:46.010735+0000","last_change":"2024-05-08T18:08:46.010735+0000","last_active":"2024-05-08T18:08:46.010735+0000","last_peered":"2024-05-08T18:08:46.010735+0000","last_clean":"2024-05-08T18:08:46.010735+0000","last_became_active":"2024-05-08T18:04:43.836496+0000","last_became_peered":"2024-05-08T18:04:43.836496+0000","last_unstale":"2024-05-08T18:08:46.010735+0000","last_undegraded":"2024-05-08T18:08:46.010735+0000","last_fullsized":"2024-05-08T18:08:46.010735+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:46.010599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:46.010599+0000","last_clean_scrub_stamp":"2024-05-08T18:08:46.010599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:07.142721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'6","reported_seq":60,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:49.481085+0000","last_change":"2024-05-08T18:08:49.481113+0000","last_active":"2024-05-08T18:08:49.481085+0000","last_peered":"2024-05-08T18:08:49.481085+0000","last_clean":"2024-05-08T18:08:49.481085+0000","last_became_active":"2024-05-08T18:04:45.811368+0000","last_became_peered":"2024-05-08T18:04:45.811368+0000","last_unstale":"2024-05-08T18:08:49.481085+0000","last_undegraded":"2024-05-08T18:08:49.481085+0000","last_fullsized":"2024-05-08T18:08:49.481085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:49.480997+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:49.480997+0000","last_clean_scrub_stamp":"2024-05-08T18:08:49.480997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:21:18.714592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"29'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:50.347475+0000","last_change":"2024-05-08T18:08:50.347475+0000","last_active":"2024-05-08T18:08:50.347475+0000","last_peered":"2024-05-08T18:08:50.347475+0000","last_clean":"2024-05-08T18:08:50.347475+0000","last_became_active":"2024-05-08T18:04:43.834612+0000","last_became_peered":"2024-05-08T18:04:43.834612+0000","last_unstale":"2024-05-08T18:08:50.347475+0000","last_undegraded":"2024-05-08T18:08:50.347475+0000","last_fullsized":"2024-05-08T18:08:50.347475+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:50.347354+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:50.347354+0000","last_clean_scrub_stamp":"2024-05-08T18:08:50.347354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:07:56.846928+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"29'2","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:40.037447+0000","last_change":"2024-05-08T18:08:40.037474+0000","last_active":"2024-05-08T18:08:40.037447+0000","last_peered":"2024-05-08T18:08:40.037447+0000","last_clean":"2024-05-08T18:08:40.037447+0000","last_became_active":"2024-05-08T18:04:45.827965+0000","last_became_peered":"2024-05-08T18:04:45.827965+0000","last_unstale":"2024-05-08T18:08:40.037447+0000","last_undegraded":"2024-05-08T18:08:40.037447+0000","last_fullsized":"2024-05-08T18:08:40.037447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:40.037352+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:40.037352+0000","last_clean_scrub_stamp":"2024-05-08T18:08:40.037352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:22:58.859282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:00.008391+0000","last_change":"2024-05-08T18:09:00.008419+0000","last_active":"2024-05-08T18:09:00.008391+0000","last_peered":"2024-05-08T18:09:00.008391+0000","last_clean":"2024-05-08T18:09:00.008391+0000","last_became_active":"2024-05-08T18:04:43.837456+0000","last_became_peered":"2024-05-08T18:04:43.837456+0000","last_unstale":"2024-05-08T18:09:00.008391+0000","last_undegraded":"2024-05-08T18:09:00.008391+0000","last_fullsized":"2024-05-08T18:09:00.008391+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:00.008279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:00.008279+0000","last_clean_scrub_stamp":"2024-05-08T18:09:00.008279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:38.600725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"29'12","reported_seq":66,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:50.085604+0000","last_change":"2024-05-08T18:08:50.085643+0000","last_active":"2024-05-08T18:08:50.085604+0000","last_peered":"2024-05-08T18:08:50.085604+0000","last_clean":"2024-05-08T18:08:50.085604+0000","last_became_active":"2024-05-08T18:04:45.825022+0000","last_became_peered":"2024-05-08T18:04:45.825022+0000","last_unstale":"2024-05-08T18:08:50.085604+0000","last_undegraded":"2024-05-08T18:08:50.085604+0000","last_fullsized":"2024-05-08T18:08:50.085604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:50.085506+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:50.085506+0000","last_clean_scrub_stamp":"2024-05-08T18:08:50.085506+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:14:46.319120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:40.176922+0000","last_change":"2024-05-08T18:08:40.176922+0000","last_active":"2024-05-08T18:08:40.176922+0000","last_peered":"2024-05-08T18:08:40.176922+0000","last_clean":"2024-05-08T18:08:40.176922+0000","last_became_active":"2024-05-08T18:04:43.842077+0000","last_became_peered":"2024-05-08T18:04:43.842077+0000","last_unstale":"2024-05-08T18:08:40.176922+0000","last_undegraded":"2024-05-08T18:08:40.176922+0000","last_fullsized":"2024-05-08T18:08:40.176922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:40.176797+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:40.176797+0000","last_clean_scrub_stamp":"2024-05-08T18:08:40.176797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:14.062529+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"29'16","reported_seq":79,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:51.114482+0000","last_change":"2024-05-08T18:08:51.114508+0000","last_active":"2024-05-08T18:08:51.114482+0000","last_peered":"2024-05-08T18:08:51.114482+0000","last_clean":"2024-05-08T18:08:51.114482+0000","last_became_active":"2024-05-08T18:04:45.824581+0000","last_became_peered":"2024-05-08T18:04:45.824581+0000","last_unstale":"2024-05-08T18:08:51.114482+0000","last_undegraded":"2024-05-08T18:08:51.114482+0000","last_fullsized":"2024-05-08T18:08:51.114482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'16","last_scrub_stamp":"2024-05-08T18:08:51.114374+0000","last_deep_scrub":"29'16","last_deep_scrub_stamp":"2024-05-08T18:08:51.114374+0000","last_clean_scrub_stamp":"2024-05-08T18:08:51.114374+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:34.341808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747778+0000","last_change":"2024-05-08T18:04:43.833461+0000","last_active":"2024-05-08T18:07:13.747778+0000","last_peered":"2024-05-08T18:07:13.747778+0000","last_clean":"2024-05-08T18:07:13.747778+0000","last_became_active":"2024-05-08T18:04:43.832722+0000","last_became_peered":"2024-05-08T18:04:43.832722+0000","last_unstale":"2024-05-08T18:07:13.747778+0000","last_undegraded":"2024-05-08T18:07:13.747778+0000","last_fullsized":"2024-05-08T18:07:13.747778+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"29'8","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:35.526800+0000","last_change":"2024-05-08T18:08:35.526800+0000","last_active":"2024-05-08T18:08:35.526800+0000","last_peered":"2024-05-08T18:08:35.526800+0000","last_clean":"2024-05-08T18:08:35.526800+0000","last_became_active":"2024-05-08T18:04:45.808383+0000","last_became_peered":"2024-05-08T18:04:45.808383+0000","last_unstale":"2024-05-08T18:08:35.526800+0000","last_undegraded":"2024-05-08T18:08:35.526800+0000","last_fullsized":"2024-05-08T18:08:35.526800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:35.526676+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:35.526676+0000","last_clean_scrub_stamp":"2024-05-08T18:08:35.526676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:24.925676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:42.091190+0000","last_change":"2024-05-08T18:08:42.091220+0000","last_active":"2024-05-08T18:08:42.091190+0000","last_peered":"2024-05-08T18:08:42.091190+0000","last_clean":"2024-05-08T18:08:42.091190+0000","last_became_active":"2024-05-08T18:04:43.840851+0000","last_became_peered":"2024-05-08T18:04:43.840851+0000","last_unstale":"2024-05-08T18:08:42.091190+0000","last_undegraded":"2024-05-08T18:08:42.091190+0000","last_fullsized":"2024-05-08T18:08:42.091190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:42.091091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:42.091091+0000","last_clean_scrub_stamp":"2024-05-08T18:08:42.091091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:24.274648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:39.004566+0000","last_change":"2024-05-08T18:08:39.004566+0000","last_active":"2024-05-08T18:08:39.004566+0000","last_peered":"2024-05-08T18:08:39.004566+0000","last_clean":"2024-05-08T18:08:39.004566+0000","last_became_active":"2024-05-08T18:04:45.828334+0000","last_became_peered":"2024-05-08T18:04:45.828334+0000","last_unstale":"2024-05-08T18:08:39.004566+0000","last_undegraded":"2024-05-08T18:08:39.004566+0000","last_fullsized":"2024-05-08T18:08:39.004566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:39.004442+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:39.004442+0000","last_clean_scrub_stamp":"2024-05-08T18:08:39.004442+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:31:47.634365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"29'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.981231+0000","last_change":"2024-05-08T18:04:43.833826+0000","last_active":"2024-05-08T18:07:53.981231+0000","last_peered":"2024-05-08T18:07:53.981231+0000","last_clean":"2024-05-08T18:07:53.981231+0000","last_became_active":"2024-05-08T18:04:43.833437+0000","last_became_peered":"2024-05-08T18:04:43.833437+0000","last_unstale":"2024-05-08T18:07:53.981231+0000","last_undegraded":"2024-05-08T18:07:53.981231+0000","last_fullsized":"2024-05-08T18:07:53.981231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:29.307581+0000","last_change":"2024-05-08T18:08:29.307581+0000","last_active":"2024-05-08T18:08:29.307581+0000","last_peered":"2024-05-08T18:08:29.307581+0000","last_clean":"2024-05-08T18:08:29.307581+0000","last_became_active":"2024-05-08T18:04:45.811884+0000","last_became_peered":"2024-05-08T18:04:45.811884+0000","last_unstale":"2024-05-08T18:08:29.307581+0000","last_undegraded":"2024-05-08T18:08:29.307581+0000","last_fullsized":"2024-05-08T18:08:29.307581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:29.307470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:29.307470+0000","last_clean_scrub_stamp":"2024-05-08T18:08:29.307470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:16:25.939226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:53.384389+0000","last_change":"2024-05-08T18:08:53.384464+0000","last_active":"2024-05-08T18:08:53.384389+0000","last_peered":"2024-05-08T18:08:53.384389+0000","last_clean":"2024-05-08T18:08:53.384389+0000","last_became_active":"2024-05-08T18:04:43.820312+0000","last_became_peered":"2024-05-08T18:04:43.820312+0000","last_unstale":"2024-05-08T18:08:53.384389+0000","last_undegraded":"2024-05-08T18:08:53.384389+0000","last_fullsized":"2024-05-08T18:08:53.384389+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:53.384283+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:53.384283+0000","last_clean_scrub_stamp":"2024-05-08T18:08:53.384283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:05:28.766196+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"29'8","reported_seq":72,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:50.932035+0000","last_change":"2024-05-08T18:08:50.932035+0000","last_active":"2024-05-08T18:08:50.932035+0000","last_peered":"2024-05-08T18:08:50.932035+0000","last_clean":"2024-05-08T18:08:50.932035+0000","last_became_active":"2024-05-08T18:04:45.822620+0000","last_became_peered":"2024-05-08T18:04:45.822620+0000","last_unstale":"2024-05-08T18:08:50.932035+0000","last_undegraded":"2024-05-08T18:08:50.932035+0000","last_fullsized":"2024-05-08T18:08:50.932035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:50.931921+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:50.931921+0000","last_clean_scrub_stamp":"2024-05-08T18:08:50.931921+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:50:39.324779+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"29'4","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:46.060404+0000","last_change":"2024-05-08T18:08:46.060430+0000","last_active":"2024-05-08T18:08:46.060404+0000","last_peered":"2024-05-08T18:08:46.060404+0000","last_clean":"2024-05-08T18:08:46.060404+0000","last_became_active":"2024-05-08T18:04:45.826493+0000","last_became_peered":"2024-05-08T18:04:45.826493+0000","last_unstale":"2024-05-08T18:08:46.060404+0000","last_undegraded":"2024-05-08T18:08:46.060404+0000","last_fullsized":"2024-05-08T18:08:46.060404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:46.060304+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:46.060304+0000","last_clean_scrub_stamp":"2024-05-08T18:08:46.060304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:28:42.176968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:46.481335+0000","last_change":"2024-05-08T18:08:46.481360+0000","last_active":"2024-05-08T18:08:46.481335+0000","last_peered":"2024-05-08T18:08:46.481335+0000","last_clean":"2024-05-08T18:08:46.481335+0000","last_became_active":"2024-05-08T18:04:43.838332+0000","last_became_peered":"2024-05-08T18:04:43.838332+0000","last_unstale":"2024-05-08T18:08:46.481335+0000","last_undegraded":"2024-05-08T18:08:46.481335+0000","last_fullsized":"2024-05-08T18:08:46.481335+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:46.481227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:46.481227+0000","last_clean_scrub_stamp":"2024-05-08T18:08:46.481227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:44:09.862039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"29'4","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:36.520147+0000","last_change":"2024-05-08T18:08:36.520179+0000","last_active":"2024-05-08T18:08:36.520147+0000","last_peered":"2024-05-08T18:08:36.520147+0000","last_clean":"2024-05-08T18:08:36.520147+0000","last_became_active":"2024-05-08T18:04:45.807438+0000","last_became_peered":"2024-05-08T18:04:45.807438+0000","last_unstale":"2024-05-08T18:08:36.520147+0000","last_undegraded":"2024-05-08T18:08:36.520147+0000","last_fullsized":"2024-05-08T18:08:36.520147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:36.520045+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:36.520045+0000","last_clean_scrub_stamp":"2024-05-08T18:08:36.520045+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:18.703602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:30.298724+0000","last_change":"2024-05-08T18:08:30.298757+0000","last_active":"2024-05-08T18:08:30.298724+0000","last_peered":"2024-05-08T18:08:30.298724+0000","last_clean":"2024-05-08T18:08:30.298724+0000","last_became_active":"2024-05-08T18:04:43.842741+0000","last_became_peered":"2024-05-08T18:04:43.842741+0000","last_unstale":"2024-05-08T18:08:30.298724+0000","last_undegraded":"2024-05-08T18:08:30.298724+0000","last_fullsized":"2024-05-08T18:08:30.298724+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:30.298603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:30.298603+0000","last_clean_scrub_stamp":"2024-05-08T18:08:30.298603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:30:25.561243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"29'12","reported_seq":67,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:43.533148+0000","last_change":"2024-05-08T18:08:43.533178+0000","last_active":"2024-05-08T18:08:43.533148+0000","last_peered":"2024-05-08T18:08:43.533148+0000","last_clean":"2024-05-08T18:08:43.533148+0000","last_became_active":"2024-05-08T18:04:45.810827+0000","last_became_peered":"2024-05-08T18:04:45.810827+0000","last_unstale":"2024-05-08T18:08:43.533148+0000","last_undegraded":"2024-05-08T18:08:43.533148+0000","last_fullsized":"2024-05-08T18:08:43.533148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:43.533034+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:43.533034+0000","last_clean_scrub_stamp":"2024-05-08T18:08:43.533034+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:04.129239+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"29'4","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:50.465263+0000","last_change":"2024-05-08T18:08:50.465291+0000","last_active":"2024-05-08T18:08:50.465263+0000","last_peered":"2024-05-08T18:08:50.465263+0000","last_clean":"2024-05-08T18:08:50.465263+0000","last_became_active":"2024-05-08T18:04:43.837706+0000","last_became_peered":"2024-05-08T18:04:43.837706+0000","last_unstale":"2024-05-08T18:08:50.465263+0000","last_undegraded":"2024-05-08T18:08:50.465263+0000","last_fullsized":"2024-05-08T18:08:50.465263+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:50.465171+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:50.465171+0000","last_clean_scrub_stamp":"2024-05-08T18:08:50.465171+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:00:00.657178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"29'8","reported_seq":68,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:32.455142+0000","last_change":"2024-05-08T18:08:32.455190+0000","last_active":"2024-05-08T18:08:32.455142+0000","last_peered":"2024-05-08T18:08:32.455142+0000","last_clean":"2024-05-08T18:08:32.455142+0000","last_became_active":"2024-05-08T18:04:45.822000+0000","last_became_peered":"2024-05-08T18:04:45.822000+0000","last_unstale":"2024-05-08T18:08:32.455142+0000","last_undegraded":"2024-05-08T18:08:32.455142+0000","last_fullsized":"2024-05-08T18:08:32.455142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:32.455035+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:32.455035+0000","last_clean_scrub_stamp":"2024-05-08T18:08:32.455035+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:21.854035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:40.545746+0000","last_change":"2024-05-08T18:08:40.545772+0000","last_active":"2024-05-08T18:08:40.545746+0000","last_peered":"2024-05-08T18:08:40.545746+0000","last_clean":"2024-05-08T18:08:40.545746+0000","last_became_active":"2024-05-08T18:04:43.841063+0000","last_became_peered":"2024-05-08T18:04:43.841063+0000","last_unstale":"2024-05-08T18:08:40.545746+0000","last_undegraded":"2024-05-08T18:08:40.545746+0000","last_fullsized":"2024-05-08T18:08:40.545746+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:40.545644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:40.545644+0000","last_clean_scrub_stamp":"2024-05-08T18:08:40.545644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:55:34.045722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"29'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:26.277429+0000","last_change":"2024-05-08T18:08:26.277484+0000","last_active":"2024-05-08T18:08:26.277429+0000","last_peered":"2024-05-08T18:08:26.277429+0000","last_clean":"2024-05-08T18:08:26.277429+0000","last_became_active":"2024-05-08T18:04:45.823611+0000","last_became_peered":"2024-05-08T18:04:45.823611+0000","last_unstale":"2024-05-08T18:08:26.277429+0000","last_undegraded":"2024-05-08T18:08:26.277429+0000","last_fullsized":"2024-05-08T18:08:26.277429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:26.277327+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:26.277327+0000","last_clean_scrub_stamp":"2024-05-08T18:08:26.277327+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:13.871401+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:46.372006+0000","last_change":"2024-05-08T18:08:46.372029+0000","last_active":"2024-05-08T18:08:46.372006+0000","last_peered":"2024-05-08T18:08:46.372006+0000","last_clean":"2024-05-08T18:08:46.372006+0000","last_became_active":"2024-05-08T18:04:43.834456+0000","last_became_peered":"2024-05-08T18:04:43.834456+0000","last_unstale":"2024-05-08T18:08:46.372006+0000","last_undegraded":"2024-05-08T18:08:46.372006+0000","last_fullsized":"2024-05-08T18:08:46.372006+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:46.371925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:46.371925+0000","last_clean_scrub_stamp":"2024-05-08T18:08:46.371925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:23.458745+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"29'6","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:37.506169+0000","last_change":"2024-05-08T18:08:37.506219+0000","last_active":"2024-05-08T18:08:37.506169+0000","last_peered":"2024-05-08T18:08:37.506169+0000","last_clean":"2024-05-08T18:08:37.506169+0000","last_became_active":"2024-05-08T18:04:45.808416+0000","last_became_peered":"2024-05-08T18:04:45.808416+0000","last_unstale":"2024-05-08T18:08:37.506169+0000","last_undegraded":"2024-05-08T18:08:37.506169+0000","last_fullsized":"2024-05-08T18:08:37.506169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:37.506056+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:37.506056+0000","last_clean_scrub_stamp":"2024-05-08T18:08:37.506056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:14.952253+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'4","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745942+0000","last_change":"2024-05-08T18:04:43.845458+0000","last_active":"2024-05-08T18:07:03.745942+0000","last_peered":"2024-05-08T18:07:03.745942+0000","last_clean":"2024-05-08T18:07:03.745942+0000","last_became_active":"2024-05-08T18:04:43.844702+0000","last_became_peered":"2024-05-08T18:04:43.844702+0000","last_unstale":"2024-05-08T18:07:03.745942+0000","last_undegraded":"2024-05-08T18:07:03.745942+0000","last_fullsized":"2024-05-08T18:07:03.745942+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"29'36","reported_seq":109,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:02.553501+0000","last_change":"2024-05-08T18:09:02.553531+0000","last_active":"2024-05-08T18:09:02.553501+0000","last_peered":"2024-05-08T18:09:02.553501+0000","last_clean":"2024-05-08T18:09:02.553501+0000","last_became_active":"2024-05-08T18:04:53.849475+0000","last_became_peered":"2024-05-08T18:04:53.849475+0000","last_unstale":"2024-05-08T18:09:02.553501+0000","last_undegraded":"2024-05-08T18:09:02.553501+0000","last_fullsized":"2024-05-08T18:09:02.553501+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:09:02.553411+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:09:02.553411+0000","last_clean_scrub_stamp":"2024-05-08T18:09:02.553411+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:25:37.317356+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"29'12","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:52.160816+0000","last_change":"2024-05-08T18:08:52.160843+0000","last_active":"2024-05-08T18:08:52.160816+0000","last_peered":"2024-05-08T18:08:52.160816+0000","last_clean":"2024-05-08T18:08:52.160816+0000","last_became_active":"2024-05-08T18:04:45.824005+0000","last_became_peered":"2024-05-08T18:04:45.824005+0000","last_unstale":"2024-05-08T18:08:52.160816+0000","last_undegraded":"2024-05-08T18:08:52.160816+0000","last_fullsized":"2024-05-08T18:08:52.160816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:52.160704+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:52.160704+0000","last_clean_scrub_stamp":"2024-05-08T18:08:52.160704+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:13.667834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745982+0000","last_change":"2024-05-08T18:04:43.834249+0000","last_active":"2024-05-08T18:07:03.745982+0000","last_peered":"2024-05-08T18:07:03.745982+0000","last_clean":"2024-05-08T18:07:03.745982+0000","last_became_active":"2024-05-08T18:04:43.833822+0000","last_became_peered":"2024-05-08T18:04:43.833822+0000","last_unstale":"2024-05-08T18:07:03.745982+0000","last_undegraded":"2024-05-08T18:07:03.745982+0000","last_fullsized":"2024-05-08T18:07:03.745982+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.924513+0000","last_change":"2024-05-08T18:08:09.924538+0000","last_active":"2024-05-08T18:08:09.924513+0000","last_peered":"2024-05-08T18:08:09.924513+0000","last_clean":"2024-05-08T18:08:09.924513+0000","last_became_active":"2024-05-08T18:04:47.819682+0000","last_became_peered":"2024-05-08T18:04:47.819682+0000","last_unstale":"2024-05-08T18:08:09.924513+0000","last_undegraded":"2024-05-08T18:08:09.924513+0000","last_fullsized":"2024-05-08T18:08:09.924513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:09.924406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:09.924406+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.924406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:58.088967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"29'39","reported_seq":94,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:31.864262+0000","last_change":"2024-05-08T18:08:31.864298+0000","last_active":"2024-05-08T18:08:31.864262+0000","last_peered":"2024-05-08T18:08:31.864262+0000","last_clean":"2024-05-08T18:08:31.864262+0000","last_became_active":"2024-05-08T18:04:49.832148+0000","last_became_peered":"2024-05-08T18:04:49.832148+0000","last_unstale":"2024-05-08T18:08:31.864262+0000","last_undegraded":"2024-05-08T18:08:31.864262+0000","last_fullsized":"2024-05-08T18:08:31.864262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'39","last_scrub_stamp":"2024-05-08T18:08:31.864172+0000","last_deep_scrub":"29'39","last_deep_scrub_stamp":"2024-05-08T18:08:31.864172+0000","last_clean_scrub_stamp":"2024-05-08T18:08:31.864172+0000","objects_scrubbed":4,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:47:21.937257+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":40,"num_read_kb":42,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.510810+0000","last_change":"2024-05-08T18:08:21.510876+0000","last_active":"2024-05-08T18:08:21.510810+0000","last_peered":"2024-05-08T18:08:21.510810+0000","last_clean":"2024-05-08T18:08:21.510810+0000","last_became_active":"2024-05-08T18:04:51.844901+0000","last_became_peered":"2024-05-08T18:04:51.844901+0000","last_unstale":"2024-05-08T18:08:21.510810+0000","last_undegraded":"2024-05-08T18:08:21.510810+0000","last_fullsized":"2024-05-08T18:08:21.510810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:21.510687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:21.510687+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.510687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:55:15.292759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"29'11","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:03.522110+0000","last_change":"2024-05-08T18:09:03.522138+0000","last_active":"2024-05-08T18:09:03.522110+0000","last_peered":"2024-05-08T18:09:03.522110+0000","last_clean":"2024-05-08T18:09:03.522110+0000","last_became_active":"2024-05-08T18:04:53.849579+0000","last_became_peered":"2024-05-08T18:04:53.849579+0000","last_unstale":"2024-05-08T18:09:03.522110+0000","last_undegraded":"2024-05-08T18:09:03.522110+0000","last_fullsized":"2024-05-08T18:09:03.522110+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'11","last_scrub_stamp":"2024-05-08T18:09:03.522007+0000","last_deep_scrub":"29'11","last_deep_scrub_stamp":"2024-05-08T18:09:03.522007+0000","last_clean_scrub_stamp":"2024-05-08T18:09:03.522007+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:01.403489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:44.084234+0000","last_change":"2024-05-08T18:08:44.084259+0000","last_active":"2024-05-08T18:08:44.084234+0000","last_peered":"2024-05-08T18:08:44.084234+0000","last_clean":"2024-05-08T18:08:44.084234+0000","last_became_active":"2024-05-08T18:04:45.825162+0000","last_became_peered":"2024-05-08T18:04:45.825162+0000","last_unstale":"2024-05-08T18:08:44.084234+0000","last_undegraded":"2024-05-08T18:08:44.084234+0000","last_fullsized":"2024-05-08T18:08:44.084234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:44.084137+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:44.084137+0000","last_clean_scrub_stamp":"2024-05-08T18:08:44.084137+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:00:15.074875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:44.010669+0000","last_change":"2024-05-08T18:08:44.010708+0000","last_active":"2024-05-08T18:08:44.010669+0000","last_peered":"2024-05-08T18:08:44.010669+0000","last_clean":"2024-05-08T18:08:44.010669+0000","last_became_active":"2024-05-08T18:04:43.827374+0000","last_became_peered":"2024-05-08T18:04:43.827374+0000","last_unstale":"2024-05-08T18:08:44.010669+0000","last_undegraded":"2024-05-08T18:08:44.010669+0000","last_fullsized":"2024-05-08T18:08:44.010669+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:44.010568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:44.010568+0000","last_clean_scrub_stamp":"2024-05-08T18:08:44.010568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:59:57.344162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.752839+0000","last_change":"2024-05-08T18:05:30.868535+0000","last_active":"2024-05-08T18:07:53.752839+0000","last_peered":"2024-05-08T18:07:53.752839+0000","last_clean":"2024-05-08T18:07:53.752839+0000","last_became_active":"2024-05-08T18:05:30.868191+0000","last_became_peered":"2024-05-08T18:05:30.868191+0000","last_unstale":"2024-05-08T18:07:53.752839+0000","last_undegraded":"2024-05-08T18:07:53.752839+0000","last_fullsized":"2024-05-08T18:07:53.752839+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"29'9","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.529952+0000","last_change":"2024-05-08T18:08:10.529988+0000","last_active":"2024-05-08T18:08:10.529952+0000","last_peered":"2024-05-08T18:08:10.529952+0000","last_clean":"2024-05-08T18:08:10.529952+0000","last_became_active":"2024-05-08T18:04:49.825265+0000","last_became_peered":"2024-05-08T18:04:49.825265+0000","last_unstale":"2024-05-08T18:08:10.529952+0000","last_undegraded":"2024-05-08T18:08:10.529952+0000","last_fullsized":"2024-05-08T18:08:10.529952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'9","last_scrub_stamp":"2024-05-08T18:08:10.529858+0000","last_deep_scrub":"29'9","last_deep_scrub_stamp":"2024-05-08T18:08:10.529858+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.529858+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:00.215888+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.491557+0000","last_change":"2024-05-08T18:08:13.491631+0000","last_active":"2024-05-08T18:08:13.491557+0000","last_peered":"2024-05-08T18:08:13.491557+0000","last_clean":"2024-05-08T18:08:13.491557+0000","last_became_active":"2024-05-08T18:04:51.833523+0000","last_became_peered":"2024-05-08T18:04:51.833523+0000","last_unstale":"2024-05-08T18:08:13.491557+0000","last_undegraded":"2024-05-08T18:08:13.491557+0000","last_fullsized":"2024-05-08T18:08:13.491557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:13.491458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:13.491458+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.491458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:07.228339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'2","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:33.477085+0000","last_change":"2024-05-08T18:08:33.477157+0000","last_active":"2024-05-08T18:08:33.477085+0000","last_peered":"2024-05-08T18:08:33.477085+0000","last_clean":"2024-05-08T18:08:33.477085+0000","last_became_active":"2024-05-08T18:04:53.840395+0000","last_became_peered":"2024-05-08T18:04:53.840395+0000","last_unstale":"2024-05-08T18:08:33.477085+0000","last_undegraded":"2024-05-08T18:08:33.477085+0000","last_fullsized":"2024-05-08T18:08:33.477085+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:33.476981+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:33.476981+0000","last_clean_scrub_stamp":"2024-05-08T18:08:33.476981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:15.660538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:58.075763+0000","last_change":"2024-05-08T18:08:58.075787+0000","last_active":"2024-05-08T18:08:58.075763+0000","last_peered":"2024-05-08T18:08:58.075763+0000","last_clean":"2024-05-08T18:08:58.075763+0000","last_became_active":"2024-05-08T18:05:30.869162+0000","last_became_peered":"2024-05-08T18:05:30.869162+0000","last_unstale":"2024-05-08T18:08:58.075763+0000","last_undegraded":"2024-05-08T18:08:58.075763+0000","last_fullsized":"2024-05-08T18:08:58.075763+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-05-08T18:08:58.075662+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-05-08T18:08:58.075662+0000","last_clean_scrub_stamp":"2024-05-08T18:08:58.075662+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:36:28.054883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":45,"num_write":24,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745862+0000","last_change":"2024-05-08T18:04:43.844847+0000","last_active":"2024-05-08T18:07:03.745862+0000","last_peered":"2024-05-08T18:07:03.745862+0000","last_clean":"2024-05-08T18:07:03.745862+0000","last_became_active":"2024-05-08T18:04:43.838666+0000","last_became_peered":"2024-05-08T18:04:43.838666+0000","last_unstale":"2024-05-08T18:07:03.745862+0000","last_undegraded":"2024-05-08T18:07:03.745862+0000","last_fullsized":"2024-05-08T18:07:03.745862+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:27.491659+0000","last_change":"2024-05-08T18:08:27.491684+0000","last_active":"2024-05-08T18:08:27.491659+0000","last_peered":"2024-05-08T18:08:27.491659+0000","last_clean":"2024-05-08T18:08:27.491659+0000","last_became_active":"2024-05-08T18:04:47.819169+0000","last_became_peered":"2024-05-08T18:04:47.819169+0000","last_unstale":"2024-05-08T18:08:27.491659+0000","last_undegraded":"2024-05-08T18:08:27.491659+0000","last_fullsized":"2024-05-08T18:08:27.491659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:27.491574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:27.491574+0000","last_clean_scrub_stamp":"2024-05-08T18:08:27.491574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:24.380337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"29'17","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.168244+0000","last_change":"2024-05-08T18:08:11.168281+0000","last_active":"2024-05-08T18:08:11.168244+0000","last_peered":"2024-05-08T18:08:11.168244+0000","last_clean":"2024-05-08T18:08:11.168244+0000","last_became_active":"2024-05-08T18:04:49.831500+0000","last_became_peered":"2024-05-08T18:04:49.831500+0000","last_unstale":"2024-05-08T18:08:11.168244+0000","last_undegraded":"2024-05-08T18:08:11.168244+0000","last_fullsized":"2024-05-08T18:08:11.168244+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'17","last_scrub_stamp":"2024-05-08T18:08:11.168144+0000","last_deep_scrub":"29'17","last_deep_scrub_stamp":"2024-05-08T18:08:11.168144+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.168144+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:03.071434+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"29'286","reported_seq":394,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:41.051149+0000","last_change":"2024-05-08T18:08:41.051182+0000","last_active":"2024-05-08T18:08:41.051149+0000","last_peered":"2024-05-08T18:08:41.051149+0000","last_clean":"2024-05-08T18:08:41.051149+0000","last_became_active":"2024-05-08T18:04:51.849301+0000","last_became_peered":"2024-05-08T18:04:51.849301+0000","last_unstale":"2024-05-08T18:08:41.051149+0000","last_undegraded":"2024-05-08T18:08:41.051149+0000","last_fullsized":"2024-05-08T18:08:41.051149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'286","last_scrub_stamp":"2024-05-08T18:08:41.051045+0000","last_deep_scrub":"29'286","last_deep_scrub_stamp":"2024-05-08T18:08:41.051045+0000","last_clean_scrub_stamp":"2024-05-08T18:08:41.051045+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:35:15.383617+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:53.985162+0000","last_change":"2024-05-08T18:08:53.985191+0000","last_active":"2024-05-08T18:08:53.985162+0000","last_peered":"2024-05-08T18:08:53.985162+0000","last_clean":"2024-05-08T18:08:53.985162+0000","last_became_active":"2024-05-08T18:04:53.840559+0000","last_became_peered":"2024-05-08T18:04:53.840559+0000","last_unstale":"2024-05-08T18:08:53.985162+0000","last_undegraded":"2024-05-08T18:08:53.985162+0000","last_fullsized":"2024-05-08T18:08:53.985162+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:53.985043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:53.985043+0000","last_clean_scrub_stamp":"2024-05-08T18:08:53.985043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:06:41.672862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"29'6","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:45.404482+0000","last_change":"2024-05-08T18:08:45.404510+0000","last_active":"2024-05-08T18:08:45.404482+0000","last_peered":"2024-05-08T18:08:45.404482+0000","last_clean":"2024-05-08T18:08:45.404482+0000","last_became_active":"2024-05-08T18:04:45.811570+0000","last_became_peered":"2024-05-08T18:04:45.811570+0000","last_unstale":"2024-05-08T18:08:45.404482+0000","last_undegraded":"2024-05-08T18:08:45.404482+0000","last_fullsized":"2024-05-08T18:08:45.404482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:45.404379+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:45.404379+0000","last_clean_scrub_stamp":"2024-05-08T18:08:45.404379+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:31:16.310504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"29'2","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:48.487816+0000","last_change":"2024-05-08T18:08:48.487844+0000","last_active":"2024-05-08T18:08:48.487816+0000","last_peered":"2024-05-08T18:08:48.487816+0000","last_clean":"2024-05-08T18:08:48.487816+0000","last_became_active":"2024-05-08T18:04:43.836410+0000","last_became_peered":"2024-05-08T18:04:43.836410+0000","last_unstale":"2024-05-08T18:08:48.487816+0000","last_undegraded":"2024-05-08T18:08:48.487816+0000","last_fullsized":"2024-05-08T18:08:48.487816+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:48.487704+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:48.487704+0000","last_clean_scrub_stamp":"2024-05-08T18:08:48.487704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:10:35.537762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.849059+0000","last_change":"2024-05-08T18:08:11.849101+0000","last_active":"2024-05-08T18:08:11.849059+0000","last_peered":"2024-05-08T18:08:11.849059+0000","last_clean":"2024-05-08T18:08:11.849059+0000","last_became_active":"2024-05-08T18:04:47.818797+0000","last_became_peered":"2024-05-08T18:04:47.818797+0000","last_unstale":"2024-05-08T18:08:11.849059+0000","last_undegraded":"2024-05-08T18:08:11.849059+0000","last_fullsized":"2024-05-08T18:08:11.849059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:11.848881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:11.848881+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.848881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:19.374213+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"29'37","reported_seq":92,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.217059+0000","last_change":"2024-05-08T18:08:12.217092+0000","last_active":"2024-05-08T18:08:12.217059+0000","last_peered":"2024-05-08T18:08:12.217059+0000","last_clean":"2024-05-08T18:08:12.217059+0000","last_became_active":"2024-05-08T18:04:49.831379+0000","last_became_peered":"2024-05-08T18:04:49.831379+0000","last_unstale":"2024-05-08T18:08:12.217059+0000","last_undegraded":"2024-05-08T18:08:12.217059+0000","last_fullsized":"2024-05-08T18:08:12.217059+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'37","last_scrub_stamp":"2024-05-08T18:08:12.216965+0000","last_deep_scrub":"29'37","last_deep_scrub_stamp":"2024-05-08T18:08:12.216965+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.216965+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:49:34.759465+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"29'285","reported_seq":453,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.529055+0000","last_change":"2024-05-08T18:08:11.529087+0000","last_active":"2024-05-08T18:08:11.529055+0000","last_peered":"2024-05-08T18:08:11.529055+0000","last_clean":"2024-05-08T18:08:11.529055+0000","last_became_active":"2024-05-08T18:04:51.826926+0000","last_became_peered":"2024-05-08T18:04:51.826926+0000","last_unstale":"2024-05-08T18:08:11.529055+0000","last_undegraded":"2024-05-08T18:08:11.529055+0000","last_fullsized":"2024-05-08T18:08:11.529055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:11.528959+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:11.528959+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.528959+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:44.650679+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'6","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:42.053526+0000","last_change":"2024-05-08T18:08:42.053554+0000","last_active":"2024-05-08T18:08:42.053526+0000","last_peered":"2024-05-08T18:08:42.053526+0000","last_clean":"2024-05-08T18:08:42.053526+0000","last_became_active":"2024-05-08T18:04:45.827351+0000","last_became_peered":"2024-05-08T18:04:45.827351+0000","last_unstale":"2024-05-08T18:08:42.053526+0000","last_undegraded":"2024-05-08T18:08:42.053526+0000","last_fullsized":"2024-05-08T18:08:42.053526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-05-08T18:08:42.053430+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-05-08T18:08:42.053430+0000","last_clean_scrub_stamp":"2024-05-08T18:08:42.053430+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:29.385518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:47.984239+0000","last_change":"2024-05-08T18:08:47.984304+0000","last_active":"2024-05-08T18:08:47.984239+0000","last_peered":"2024-05-08T18:08:47.984239+0000","last_clean":"2024-05-08T18:08:47.984239+0000","last_became_active":"2024-05-08T18:04:43.838840+0000","last_became_peered":"2024-05-08T18:04:43.838840+0000","last_unstale":"2024-05-08T18:08:47.984239+0000","last_undegraded":"2024-05-08T18:08:47.984239+0000","last_fullsized":"2024-05-08T18:08:47.984239+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:47.984130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:47.984130+0000","last_clean_scrub_stamp":"2024-05-08T18:08:47.984130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:31.557621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,0],"acting":[2,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.468389+0000","last_change":"2024-05-08T18:08:14.468441+0000","last_active":"2024-05-08T18:08:14.468389+0000","last_peered":"2024-05-08T18:08:14.468389+0000","last_clean":"2024-05-08T18:08:14.468389+0000","last_became_active":"2024-05-08T18:04:47.825215+0000","last_became_peered":"2024-05-08T18:04:47.825215+0000","last_unstale":"2024-05-08T18:08:14.468389+0000","last_undegraded":"2024-05-08T18:08:14.468389+0000","last_fullsized":"2024-05-08T18:08:14.468389+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:14.468285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:14.468285+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.468285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:38.045371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"29'51","reported_seq":111,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.277773+0000","last_change":"2024-05-08T18:08:14.277853+0000","last_active":"2024-05-08T18:08:14.277773+0000","last_peered":"2024-05-08T18:08:14.277773+0000","last_clean":"2024-05-08T18:08:14.277773+0000","last_became_active":"2024-05-08T18:04:49.827954+0000","last_became_peered":"2024-05-08T18:04:49.827954+0000","last_unstale":"2024-05-08T18:08:14.277773+0000","last_undegraded":"2024-05-08T18:08:14.277773+0000","last_fullsized":"2024-05-08T18:08:14.277773+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'51","last_scrub_stamp":"2024-05-08T18:08:14.277647+0000","last_deep_scrub":"29'51","last_deep_scrub_stamp":"2024-05-08T18:08:14.277647+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.277647+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:23:20.826582+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.436717+0000","last_change":"2024-05-08T18:08:07.436758+0000","last_active":"2024-05-08T18:08:07.436717+0000","last_peered":"2024-05-08T18:08:07.436717+0000","last_clean":"2024-05-08T18:08:07.436717+0000","last_became_active":"2024-05-08T18:04:51.831149+0000","last_became_peered":"2024-05-08T18:04:51.831149+0000","last_unstale":"2024-05-08T18:08:07.436717+0000","last_undegraded":"2024-05-08T18:08:07.436717+0000","last_fullsized":"2024-05-08T18:08:07.436717+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:07.436627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:07.436627+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.436627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:56:38.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"29'5","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:55.013107+0000","last_change":"2024-05-08T18:08:55.013107+0000","last_active":"2024-05-08T18:08:55.013107+0000","last_peered":"2024-05-08T18:08:55.013107+0000","last_clean":"2024-05-08T18:08:55.013107+0000","last_became_active":"2024-05-08T18:04:53.840652+0000","last_became_peered":"2024-05-08T18:04:53.840652+0000","last_unstale":"2024-05-08T18:08:55.013107+0000","last_undegraded":"2024-05-08T18:08:55.013107+0000","last_fullsized":"2024-05-08T18:08:55.013107+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'5","last_scrub_stamp":"2024-05-08T18:08:55.012991+0000","last_deep_scrub":"29'5","last_deep_scrub_stamp":"2024-05-08T18:08:55.012991+0000","last_clean_scrub_stamp":"2024-05-08T18:08:55.012991+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:55.220779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"27'4","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:38.499682+0000","last_change":"2024-05-08T18:08:38.499721+0000","last_active":"2024-05-08T18:08:38.499682+0000","last_peered":"2024-05-08T18:08:38.499682+0000","last_clean":"2024-05-08T18:08:38.499682+0000","last_became_active":"2024-05-08T18:04:45.804124+0000","last_became_peered":"2024-05-08T18:04:45.804124+0000","last_unstale":"2024-05-08T18:08:38.499682+0000","last_undegraded":"2024-05-08T18:08:38.499682+0000","last_fullsized":"2024-05-08T18:08:38.499682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-08T18:08:38.499539+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-08T18:08:38.499539+0000","last_clean_scrub_stamp":"2024-05-08T18:08:38.499539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:58.286733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:36.190485+0000","last_change":"2024-05-08T18:08:36.190511+0000","last_active":"2024-05-08T18:08:36.190485+0000","last_peered":"2024-05-08T18:08:36.190485+0000","last_clean":"2024-05-08T18:08:36.190485+0000","last_became_active":"2024-05-08T18:04:43.842126+0000","last_became_peered":"2024-05-08T18:04:43.842126+0000","last_unstale":"2024-05-08T18:08:36.190485+0000","last_undegraded":"2024-05-08T18:08:36.190485+0000","last_fullsized":"2024-05-08T18:08:36.190485+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:36.190391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:36.190391+0000","last_clean_scrub_stamp":"2024-05-08T18:08:36.190391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:10:38.441913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.583926+0000","last_change":"2024-05-08T18:08:22.583963+0000","last_active":"2024-05-08T18:08:22.583926+0000","last_peered":"2024-05-08T18:08:22.583926+0000","last_clean":"2024-05-08T18:08:22.583926+0000","last_became_active":"2024-05-08T18:04:47.822688+0000","last_became_peered":"2024-05-08T18:04:47.822688+0000","last_unstale":"2024-05-08T18:08:22.583926+0000","last_undegraded":"2024-05-08T18:08:22.583926+0000","last_fullsized":"2024-05-08T18:08:22.583926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.583822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.583822+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.583822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:22.094540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"29'36","reported_seq":91,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.839677+0000","last_change":"2024-05-08T18:08:16.839747+0000","last_active":"2024-05-08T18:08:16.839677+0000","last_peered":"2024-05-08T18:08:16.839677+0000","last_clean":"2024-05-08T18:08:16.839677+0000","last_became_active":"2024-05-08T18:04:49.829892+0000","last_became_peered":"2024-05-08T18:04:49.829892+0000","last_unstale":"2024-05-08T18:08:16.839677+0000","last_undegraded":"2024-05-08T18:08:16.839677+0000","last_fullsized":"2024-05-08T18:08:16.839677+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:08:16.839580+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:08:16.839580+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.839580+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:53.387986+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:04.455783+0000","last_change":"2024-05-08T18:08:04.455804+0000","last_active":"2024-05-08T18:08:04.455783+0000","last_peered":"2024-05-08T18:08:04.455783+0000","last_clean":"2024-05-08T18:08:04.455783+0000","last_became_active":"2024-05-08T18:04:51.832810+0000","last_became_peered":"2024-05-08T18:04:51.832810+0000","last_unstale":"2024-05-08T18:08:04.455783+0000","last_undegraded":"2024-05-08T18:08:04.455783+0000","last_fullsized":"2024-05-08T18:08:04.455783+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:04.455699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:04.455699+0000","last_clean_scrub_stamp":"2024-05-08T18:08:04.455699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:21.455723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"29'4","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:29.520481+0000","last_change":"2024-05-08T18:08:29.520481+0000","last_active":"2024-05-08T18:08:29.520481+0000","last_peered":"2024-05-08T18:08:29.520481+0000","last_clean":"2024-05-08T18:08:29.520481+0000","last_became_active":"2024-05-08T18:04:53.840894+0000","last_became_peered":"2024-05-08T18:04:53.840894+0000","last_unstale":"2024-05-08T18:08:29.520481+0000","last_undegraded":"2024-05-08T18:08:29.520481+0000","last_fullsized":"2024-05-08T18:08:29.520481+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:29.520364+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:29.520364+0000","last_clean_scrub_stamp":"2024-05-08T18:08:29.520364+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:53.170014+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"29'10","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:37.218745+0000","last_change":"2024-05-08T18:08:37.218777+0000","last_active":"2024-05-08T18:08:37.218745+0000","last_peered":"2024-05-08T18:08:37.218745+0000","last_clean":"2024-05-08T18:08:37.218745+0000","last_became_active":"2024-05-08T18:04:45.816817+0000","last_became_peered":"2024-05-08T18:04:45.816817+0000","last_unstale":"2024-05-08T18:08:37.218745+0000","last_undegraded":"2024-05-08T18:08:37.218745+0000","last_fullsized":"2024-05-08T18:08:37.218745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:37.218630+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:37.218630+0000","last_clean_scrub_stamp":"2024-05-08T18:08:37.218630+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:39.786476+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"29'4","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:03.027328+0000","last_change":"2024-05-08T18:09:03.027396+0000","last_active":"2024-05-08T18:09:03.027328+0000","last_peered":"2024-05-08T18:09:03.027328+0000","last_clean":"2024-05-08T18:09:03.027328+0000","last_became_active":"2024-05-08T18:04:43.838781+0000","last_became_peered":"2024-05-08T18:04:43.838781+0000","last_unstale":"2024-05-08T18:09:03.027328+0000","last_undegraded":"2024-05-08T18:09:03.027328+0000","last_fullsized":"2024-05-08T18:09:03.027328+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:09:03.027231+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:09:03.027231+0000","last_clean_scrub_stamp":"2024-05-08T18:09:03.027231+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:57:33.005457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:30.495270+0000","last_change":"2024-05-08T18:08:30.495329+0000","last_active":"2024-05-08T18:08:30.495270+0000","last_peered":"2024-05-08T18:08:30.495270+0000","last_clean":"2024-05-08T18:08:30.495270+0000","last_became_active":"2024-05-08T18:04:47.818871+0000","last_became_peered":"2024-05-08T18:04:47.818871+0000","last_unstale":"2024-05-08T18:08:30.495270+0000","last_undegraded":"2024-05-08T18:08:30.495270+0000","last_fullsized":"2024-05-08T18:08:30.495270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:30.495168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:30.495168+0000","last_clean_scrub_stamp":"2024-05-08T18:08:30.495168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:26.778212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"29'45","reported_seq":103,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.892863+0000","last_change":"2024-05-08T18:08:08.892899+0000","last_active":"2024-05-08T18:08:08.892863+0000","last_peered":"2024-05-08T18:08:08.892863+0000","last_clean":"2024-05-08T18:08:08.892863+0000","last_became_active":"2024-05-08T18:04:49.829270+0000","last_became_peered":"2024-05-08T18:04:49.829270+0000","last_unstale":"2024-05-08T18:08:08.892863+0000","last_undegraded":"2024-05-08T18:08:08.892863+0000","last_fullsized":"2024-05-08T18:08:08.892863+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'45","last_scrub_stamp":"2024-05-08T18:08:08.892763+0000","last_deep_scrub":"29'45","last_deep_scrub_stamp":"2024-05-08T18:08:08.892763+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.892763+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:11.460610+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"29'285","reported_seq":360,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.575979+0000","last_change":"2024-05-08T18:08:13.576007+0000","last_active":"2024-05-08T18:08:13.575979+0000","last_peered":"2024-05-08T18:08:13.575979+0000","last_clean":"2024-05-08T18:08:13.575979+0000","last_became_active":"2024-05-08T18:04:51.844529+0000","last_became_peered":"2024-05-08T18:04:51.844529+0000","last_unstale":"2024-05-08T18:08:13.575979+0000","last_undegraded":"2024-05-08T18:08:13.575979+0000","last_fullsized":"2024-05-08T18:08:13.575979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:13.575886+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:13.575886+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.575886+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:46.724169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"29'2","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:01.474432+0000","last_change":"2024-05-08T18:09:01.474503+0000","last_active":"2024-05-08T18:09:01.474432+0000","last_peered":"2024-05-08T18:09:01.474432+0000","last_clean":"2024-05-08T18:09:01.474432+0000","last_became_active":"2024-05-08T18:04:53.840767+0000","last_became_peered":"2024-05-08T18:04:53.840767+0000","last_unstale":"2024-05-08T18:09:01.474432+0000","last_undegraded":"2024-05-08T18:09:01.474432+0000","last_fullsized":"2024-05-08T18:09:01.474432+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:09:01.474327+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:09:01.474327+0000","last_clean_scrub_stamp":"2024-05-08T18:09:01.474327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:50.275478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"29'6","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:35.195828+0000","last_change":"2024-05-08T18:08:35.195855+0000","last_active":"2024-05-08T18:08:35.195828+0000","last_peered":"2024-05-08T18:08:35.195828+0000","last_clean":"2024-05-08T18:08:35.195828+0000","last_became_active":"2024-05-08T18:04:45.813735+0000","last_became_peered":"2024-05-08T18:04:45.813735+0000","last_unstale":"2024-05-08T18:08:35.195828+0000","last_undegraded":"2024-05-08T18:08:35.195828+0000","last_fullsized":"2024-05-08T18:08:35.195828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:35.195741+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:35.195741+0000","last_clean_scrub_stamp":"2024-05-08T18:08:35.195741+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:44:58.936221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"27'4","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:56.007642+0000","last_change":"2024-05-08T18:08:56.007674+0000","last_active":"2024-05-08T18:08:56.007642+0000","last_peered":"2024-05-08T18:08:56.007642+0000","last_clean":"2024-05-08T18:08:56.007642+0000","last_became_active":"2024-05-08T18:04:43.839047+0000","last_became_peered":"2024-05-08T18:04:43.839047+0000","last_unstale":"2024-05-08T18:08:56.007642+0000","last_undegraded":"2024-05-08T18:08:56.007642+0000","last_fullsized":"2024-05-08T18:08:56.007642+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-08T18:08:56.007537+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-08T18:08:56.007537+0000","last_clean_scrub_stamp":"2024-05-08T18:08:56.007537+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:37.060769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:28.472742+0000","last_change":"2024-05-08T18:08:28.472766+0000","last_active":"2024-05-08T18:08:28.472742+0000","last_peered":"2024-05-08T18:08:28.472742+0000","last_clean":"2024-05-08T18:08:28.472742+0000","last_became_active":"2024-05-08T18:04:47.818972+0000","last_became_peered":"2024-05-08T18:04:47.818972+0000","last_unstale":"2024-05-08T18:08:28.472742+0000","last_undegraded":"2024-05-08T18:08:28.472742+0000","last_fullsized":"2024-05-08T18:08:28.472742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:28.472654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:28.472654+0000","last_clean_scrub_stamp":"2024-05-08T18:08:28.472654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:31.040500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"29'63","reported_seq":136,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.861664+0000","last_change":"2024-05-08T18:08:12.861715+0000","last_active":"2024-05-08T18:08:12.861664+0000","last_peered":"2024-05-08T18:08:12.861664+0000","last_clean":"2024-05-08T18:08:12.861664+0000","last_became_active":"2024-05-08T18:04:49.830424+0000","last_became_peered":"2024-05-08T18:04:49.830424+0000","last_unstale":"2024-05-08T18:08:12.861664+0000","last_undegraded":"2024-05-08T18:08:12.861664+0000","last_fullsized":"2024-05-08T18:08:12.861664+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'63","last_scrub_stamp":"2024-05-08T18:08:12.861577+0000","last_deep_scrub":"29'63","last_deep_scrub_stamp":"2024-05-08T18:08:12.861577+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.861577+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:02.260578+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.479280+0000","last_change":"2024-05-08T18:08:10.479304+0000","last_active":"2024-05-08T18:08:10.479280+0000","last_peered":"2024-05-08T18:08:10.479280+0000","last_clean":"2024-05-08T18:08:10.479280+0000","last_became_active":"2024-05-08T18:04:51.830383+0000","last_became_peered":"2024-05-08T18:04:51.830383+0000","last_unstale":"2024-05-08T18:08:10.479280+0000","last_undegraded":"2024-05-08T18:08:10.479280+0000","last_fullsized":"2024-05-08T18:08:10.479280+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:10.479187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:10.479187+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.479187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:04.216069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"29'20","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.161729+0000","last_change":"2024-05-08T18:04:53.848105+0000","last_active":"2024-05-08T18:07:54.161729+0000","last_peered":"2024-05-08T18:07:54.161729+0000","last_clean":"2024-05-08T18:07:54.161729+0000","last_became_active":"2024-05-08T18:04:53.847842+0000","last_became_peered":"2024-05-08T18:04:53.847842+0000","last_unstale":"2024-05-08T18:07:54.161729+0000","last_undegraded":"2024-05-08T18:07:54.161729+0000","last_fullsized":"2024-05-08T18:07:54.161729+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":48,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.511443+0000","last_change":"2024-05-08T18:08:23.511472+0000","last_active":"2024-05-08T18:08:23.511443+0000","last_peered":"2024-05-08T18:08:23.511443+0000","last_clean":"2024-05-08T18:08:23.511443+0000","last_became_active":"2024-05-08T18:04:45.821412+0000","last_became_peered":"2024-05-08T18:04:45.821412+0000","last_unstale":"2024-05-08T18:08:23.511443+0000","last_undegraded":"2024-05-08T18:08:23.511443+0000","last_fullsized":"2024-05-08T18:08:23.511443+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:23.511346+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:23.511346+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.511346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:04.193921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:44.361805+0000","last_change":"2024-05-08T18:08:44.361833+0000","last_active":"2024-05-08T18:08:44.361805+0000","last_peered":"2024-05-08T18:08:44.361805+0000","last_clean":"2024-05-08T18:08:44.361805+0000","last_became_active":"2024-05-08T18:04:43.834723+0000","last_became_peered":"2024-05-08T18:04:43.834723+0000","last_unstale":"2024-05-08T18:08:44.361805+0000","last_undegraded":"2024-05-08T18:08:44.361805+0000","last_fullsized":"2024-05-08T18:08:44.361805+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:44.361707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:44.361707+0000","last_clean_scrub_stamp":"2024-05-08T18:08:44.361707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:54.804858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.295215+0000","last_change":"2024-05-08T18:08:22.295265+0000","last_active":"2024-05-08T18:08:22.295215+0000","last_peered":"2024-05-08T18:08:22.295215+0000","last_clean":"2024-05-08T18:08:22.295215+0000","last_became_active":"2024-05-08T18:04:47.822865+0000","last_became_peered":"2024-05-08T18:04:47.822865+0000","last_unstale":"2024-05-08T18:08:22.295215+0000","last_undegraded":"2024-05-08T18:08:22.295215+0000","last_fullsized":"2024-05-08T18:08:22.295215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.295113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.295113+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.295113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:31:07.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"29'40","reported_seq":96,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.481554+0000","last_change":"2024-05-08T18:08:11.481605+0000","last_active":"2024-05-08T18:08:11.481554+0000","last_peered":"2024-05-08T18:08:11.481554+0000","last_clean":"2024-05-08T18:08:11.481554+0000","last_became_active":"2024-05-08T18:04:49.824610+0000","last_became_peered":"2024-05-08T18:04:49.824610+0000","last_unstale":"2024-05-08T18:08:11.481554+0000","last_undegraded":"2024-05-08T18:08:11.481554+0000","last_fullsized":"2024-05-08T18:08:11.481554+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'40","last_scrub_stamp":"2024-05-08T18:08:11.481461+0000","last_deep_scrub":"29'40","last_deep_scrub_stamp":"2024-05-08T18:08:11.481461+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.481461+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:15.729540+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:28.829222+0000","last_change":"2024-05-08T18:08:28.829264+0000","last_active":"2024-05-08T18:08:28.829222+0000","last_peered":"2024-05-08T18:08:28.829222+0000","last_clean":"2024-05-08T18:08:28.829222+0000","last_became_active":"2024-05-08T18:04:51.848793+0000","last_became_peered":"2024-05-08T18:04:51.848793+0000","last_unstale":"2024-05-08T18:08:28.829222+0000","last_undegraded":"2024-05-08T18:08:28.829222+0000","last_fullsized":"2024-05-08T18:08:28.829222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:28.829111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:28.829111+0000","last_clean_scrub_stamp":"2024-05-08T18:08:28.829111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:35.508902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"29'5","reported_seq":24,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.082276+0000","last_change":"2024-05-08T18:04:53.849242+0000","last_active":"2024-05-08T18:08:05.082276+0000","last_peered":"2024-05-08T18:08:05.082276+0000","last_clean":"2024-05-08T18:08:05.082276+0000","last_became_active":"2024-05-08T18:04:53.848945+0000","last_became_peered":"2024-05-08T18:04:53.848945+0000","last_unstale":"2024-05-08T18:08:05.082276+0000","last_undegraded":"2024-05-08T18:08:05.082276+0000","last_fullsized":"2024-05-08T18:08:05.082276+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"29'10","reported_seq":84,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.836191+0000","last_change":"2024-05-08T18:08:23.836220+0000","last_active":"2024-05-08T18:08:23.836191+0000","last_peered":"2024-05-08T18:08:23.836191+0000","last_clean":"2024-05-08T18:08:23.836191+0000","last_became_active":"2024-05-08T18:04:45.829120+0000","last_became_peered":"2024-05-08T18:04:45.829120+0000","last_unstale":"2024-05-08T18:08:23.836191+0000","last_undegraded":"2024-05-08T18:08:23.836191+0000","last_fullsized":"2024-05-08T18:08:23.836191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:23.836088+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:23.836088+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.836088+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:15.906999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":57,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.080785+0000","last_change":"2024-05-08T18:04:43.835329+0000","last_active":"2024-05-08T18:08:05.080785+0000","last_peered":"2024-05-08T18:08:05.080785+0000","last_clean":"2024-05-08T18:08:05.080785+0000","last_became_active":"2024-05-08T18:04:43.834536+0000","last_became_peered":"2024-05-08T18:04:43.834536+0000","last_unstale":"2024-05-08T18:08:05.080785+0000","last_undegraded":"2024-05-08T18:08:05.080785+0000","last_fullsized":"2024-05-08T18:08:05.080785+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"29'5","reported_seq":238,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:26.539732+0000","last_change":"2024-05-08T18:08:26.539766+0000","last_active":"2024-05-08T18:08:26.539732+0000","last_peered":"2024-05-08T18:08:26.539732+0000","last_clean":"2024-05-08T18:08:26.539732+0000","last_became_active":"2024-05-08T18:04:47.819085+0000","last_became_peered":"2024-05-08T18:04:47.819085+0000","last_unstale":"2024-05-08T18:08:26.539732+0000","last_undegraded":"2024-05-08T18:08:26.539732+0000","last_fullsized":"2024-05-08T18:08:26.539732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'5","last_scrub_stamp":"2024-05-08T18:08:26.539641+0000","last_deep_scrub":"29'5","last_deep_scrub_stamp":"2024-05-08T18:08:26.539641+0000","last_clean_scrub_stamp":"2024-05-08T18:08:26.539641+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:50.189291+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":301,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"29'45","reported_seq":103,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.280826+0000","last_change":"2024-05-08T18:08:16.280897+0000","last_active":"2024-05-08T18:08:16.280826+0000","last_peered":"2024-05-08T18:08:16.280826+0000","last_clean":"2024-05-08T18:08:16.280826+0000","last_became_active":"2024-05-08T18:04:49.825613+0000","last_became_peered":"2024-05-08T18:04:49.825613+0000","last_unstale":"2024-05-08T18:08:16.280826+0000","last_undegraded":"2024-05-08T18:08:16.280826+0000","last_fullsized":"2024-05-08T18:08:16.280826+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'45","last_scrub_stamp":"2024-05-08T18:08:16.280719+0000","last_deep_scrub":"29'45","last_deep_scrub_stamp":"2024-05-08T18:08:16.280719+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.280719+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:29:19.130403+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"29'285","reported_seq":443,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.137858+0000","last_change":"2024-05-08T18:08:06.137883+0000","last_active":"2024-05-08T18:08:06.137858+0000","last_peered":"2024-05-08T18:08:06.137858+0000","last_clean":"2024-05-08T18:08:06.137858+0000","last_became_active":"2024-05-08T18:04:51.830924+0000","last_became_peered":"2024-05-08T18:04:51.830924+0000","last_unstale":"2024-05-08T18:08:06.137858+0000","last_undegraded":"2024-05-08T18:08:06.137858+0000","last_fullsized":"2024-05-08T18:08:06.137858+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:06.137767+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:06.137767+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.137767+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:43.622614+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"29'15","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:02.458510+0000","last_change":"2024-05-08T18:09:02.458510+0000","last_active":"2024-05-08T18:09:02.458510+0000","last_peered":"2024-05-08T18:09:02.458510+0000","last_clean":"2024-05-08T18:09:02.458510+0000","last_became_active":"2024-05-08T18:04:53.840708+0000","last_became_peered":"2024-05-08T18:04:53.840708+0000","last_unstale":"2024-05-08T18:09:02.458510+0000","last_undegraded":"2024-05-08T18:09:02.458510+0000","last_fullsized":"2024-05-08T18:09:02.458510+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:09:02.458318+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:09:02.458318+0000","last_clean_scrub_stamp":"2024-05-08T18:09:02.458318+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:49.790407+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":180,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"29'6","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:31.301310+0000","last_change":"2024-05-08T18:08:31.301337+0000","last_active":"2024-05-08T18:08:31.301310+0000","last_peered":"2024-05-08T18:08:31.301310+0000","last_clean":"2024-05-08T18:08:31.301310+0000","last_became_active":"2024-05-08T18:04:45.811926+0000","last_became_peered":"2024-05-08T18:04:45.811926+0000","last_unstale":"2024-05-08T18:08:31.301310+0000","last_undegraded":"2024-05-08T18:08:31.301310+0000","last_fullsized":"2024-05-08T18:08:31.301310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:31.301216+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:31.301216+0000","last_clean_scrub_stamp":"2024-05-08T18:08:31.301216+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:32:41.104191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"29'2","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:01.003015+0000","last_change":"2024-05-08T18:09:01.003015+0000","last_active":"2024-05-08T18:09:01.003015+0000","last_peered":"2024-05-08T18:09:01.003015+0000","last_clean":"2024-05-08T18:09:01.003015+0000","last_became_active":"2024-05-08T18:04:43.843934+0000","last_became_peered":"2024-05-08T18:04:43.843934+0000","last_unstale":"2024-05-08T18:09:01.003015+0000","last_undegraded":"2024-05-08T18:09:01.003015+0000","last_fullsized":"2024-05-08T18:09:01.003015+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:09:01.002893+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:09:01.002893+0000","last_clean_scrub_stamp":"2024-05-08T18:09:01.002893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:41:23.689098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"29'5","reported_seq":428,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.550034+0000","last_change":"2024-05-08T18:08:20.550071+0000","last_active":"2024-05-08T18:08:20.550034+0000","last_peered":"2024-05-08T18:08:20.550034+0000","last_clean":"2024-05-08T18:08:20.550034+0000","last_became_active":"2024-05-08T18:04:47.817851+0000","last_became_peered":"2024-05-08T18:04:47.817851+0000","last_unstale":"2024-05-08T18:08:20.550034+0000","last_undegraded":"2024-05-08T18:08:20.550034+0000","last_fullsized":"2024-05-08T18:08:20.550034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'5","last_scrub_stamp":"2024-05-08T18:08:20.549943+0000","last_deep_scrub":"29'5","last_deep_scrub_stamp":"2024-05-08T18:08:20.549943+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.549943+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:31.792211+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"29'15","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.800413+0000","last_change":"2024-05-08T18:08:18.800470+0000","last_active":"2024-05-08T18:08:18.800413+0000","last_peered":"2024-05-08T18:08:18.800413+0000","last_clean":"2024-05-08T18:08:18.800413+0000","last_became_active":"2024-05-08T18:04:49.830714+0000","last_became_peered":"2024-05-08T18:04:49.830714+0000","last_unstale":"2024-05-08T18:08:18.800413+0000","last_undegraded":"2024-05-08T18:08:18.800413+0000","last_fullsized":"2024-05-08T18:08:18.800413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:18.800324+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:18.800324+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.800324+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:41.821593+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"29'285","reported_seq":480,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.542263+0000","last_change":"2024-05-08T18:08:05.542295+0000","last_active":"2024-05-08T18:08:05.542263+0000","last_peered":"2024-05-08T18:08:05.542263+0000","last_clean":"2024-05-08T18:08:05.542263+0000","last_became_active":"2024-05-08T18:04:51.831508+0000","last_became_peered":"2024-05-08T18:04:51.831508+0000","last_unstale":"2024-05-08T18:08:05.542263+0000","last_undegraded":"2024-05-08T18:08:05.542263+0000","last_fullsized":"2024-05-08T18:08:05.542263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:05.542169+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:05.542169+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.542169+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:49:28.084669+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'4","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:52.956968+0000","last_change":"2024-05-08T18:08:52.956968+0000","last_active":"2024-05-08T18:08:52.956968+0000","last_peered":"2024-05-08T18:08:52.956968+0000","last_clean":"2024-05-08T18:08:52.956968+0000","last_became_active":"2024-05-08T18:04:53.840215+0000","last_became_peered":"2024-05-08T18:04:53.840215+0000","last_unstale":"2024-05-08T18:08:52.956968+0000","last_undegraded":"2024-05-08T18:08:52.956968+0000","last_fullsized":"2024-05-08T18:08:52.956968+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:52.956850+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:52.956850+0000","last_clean_scrub_stamp":"2024-05-08T18:08:52.956850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:44:16.337663+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'2","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:30.526060+0000","last_change":"2024-05-08T18:08:30.526084+0000","last_active":"2024-05-08T18:08:30.526060+0000","last_peered":"2024-05-08T18:08:30.526060+0000","last_clean":"2024-05-08T18:08:30.526060+0000","last_became_active":"2024-05-08T18:04:45.818533+0000","last_became_peered":"2024-05-08T18:04:45.818533+0000","last_unstale":"2024-05-08T18:08:30.526060+0000","last_undegraded":"2024-05-08T18:08:30.526060+0000","last_fullsized":"2024-05-08T18:08:30.526060+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:30.525970+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:30.525970+0000","last_clean_scrub_stamp":"2024-05-08T18:08:30.525970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:27.414733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"29'2","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:55.032953+0000","last_change":"2024-05-08T18:08:55.032953+0000","last_active":"2024-05-08T18:08:55.032953+0000","last_peered":"2024-05-08T18:08:55.032953+0000","last_clean":"2024-05-08T18:08:55.032953+0000","last_became_active":"2024-05-08T18:04:43.846125+0000","last_became_peered":"2024-05-08T18:04:43.846125+0000","last_unstale":"2024-05-08T18:08:55.032953+0000","last_undegraded":"2024-05-08T18:08:55.032953+0000","last_fullsized":"2024-05-08T18:08:55.032953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:55.032817+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:55.032817+0000","last_clean_scrub_stamp":"2024-05-08T18:08:55.032817+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:29:54.513234+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.895797+0000","last_change":"2024-05-08T18:08:05.895821+0000","last_active":"2024-05-08T18:08:05.895797+0000","last_peered":"2024-05-08T18:08:05.895797+0000","last_clean":"2024-05-08T18:08:05.895797+0000","last_became_active":"2024-05-08T18:04:47.822242+0000","last_became_peered":"2024-05-08T18:04:47.822242+0000","last_unstale":"2024-05-08T18:08:05.895797+0000","last_undegraded":"2024-05-08T18:08:05.895797+0000","last_fullsized":"2024-05-08T18:08:05.895797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:05.895706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:05.895706+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.895706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:00:55.969119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"29'48","reported_seq":107,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.545377+0000","last_change":"2024-05-08T18:08:08.545419+0000","last_active":"2024-05-08T18:08:08.545377+0000","last_peered":"2024-05-08T18:08:08.545377+0000","last_clean":"2024-05-08T18:08:08.545377+0000","last_became_active":"2024-05-08T18:04:49.825314+0000","last_became_peered":"2024-05-08T18:04:49.825314+0000","last_unstale":"2024-05-08T18:08:08.545377+0000","last_undegraded":"2024-05-08T18:08:08.545377+0000","last_fullsized":"2024-05-08T18:08:08.545377+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'48","last_scrub_stamp":"2024-05-08T18:08:08.545269+0000","last_deep_scrub":"29'48","last_deep_scrub_stamp":"2024-05-08T18:08:08.545269+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.545269+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:19.079352+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.136687+0000","last_change":"2024-05-08T18:08:05.136723+0000","last_active":"2024-05-08T18:08:05.136687+0000","last_peered":"2024-05-08T18:08:05.136687+0000","last_clean":"2024-05-08T18:08:05.136687+0000","last_became_active":"2024-05-08T18:04:51.832808+0000","last_became_peered":"2024-05-08T18:04:51.832808+0000","last_unstale":"2024-05-08T18:08:05.136687+0000","last_undegraded":"2024-05-08T18:08:05.136687+0000","last_fullsized":"2024-05-08T18:08:05.136687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:05.136592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:05.136592+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.136592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:30.377482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"29'9","reported_seq":37,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:25.506016+0000","last_change":"2024-05-08T18:08:25.506016+0000","last_active":"2024-05-08T18:08:25.506016+0000","last_peered":"2024-05-08T18:08:25.506016+0000","last_clean":"2024-05-08T18:08:25.506016+0000","last_became_active":"2024-05-08T18:04:53.837292+0000","last_became_peered":"2024-05-08T18:04:53.837292+0000","last_unstale":"2024-05-08T18:08:25.506016+0000","last_undegraded":"2024-05-08T18:08:25.506016+0000","last_fullsized":"2024-05-08T18:08:25.506016+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'9","last_scrub_stamp":"2024-05-08T18:08:25.505900+0000","last_deep_scrub":"29'9","last_deep_scrub_stamp":"2024-05-08T18:08:25.505900+0000","last_clean_scrub_stamp":"2024-05-08T18:08:25.505900+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:25.016619+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'4","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:25.876183+0000","last_change":"2024-05-08T18:08:25.876183+0000","last_active":"2024-05-08T18:08:25.876183+0000","last_peered":"2024-05-08T18:08:25.876183+0000","last_clean":"2024-05-08T18:08:25.876183+0000","last_became_active":"2024-05-08T18:04:45.828674+0000","last_became_peered":"2024-05-08T18:04:45.828674+0000","last_unstale":"2024-05-08T18:08:25.876183+0000","last_undegraded":"2024-05-08T18:08:25.876183+0000","last_fullsized":"2024-05-08T18:08:25.876183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:25.876052+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:25.876052+0000","last_clean_scrub_stamp":"2024-05-08T18:08:25.876052+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:30:56.782177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'74","reported_seq":149,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:51.419922+0000","last_change":"2024-05-08T18:08:51.419959+0000","last_active":"2024-05-08T18:08:51.419922+0000","last_peered":"2024-05-08T18:08:51.419922+0000","last_clean":"2024-05-08T18:08:51.419922+0000","last_became_active":"2024-05-08T18:04:35.551437+0000","last_became_peered":"2024-05-08T18:04:35.551437+0000","last_unstale":"2024-05-08T18:08:51.419922+0000","last_undegraded":"2024-05-08T18:08:51.419922+0000","last_fullsized":"2024-05-08T18:08:51.419922+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'74","last_scrub_stamp":"2024-05-08T18:08:51.419824+0000","last_deep_scrub":"29'74","last_deep_scrub_stamp":"2024-05-08T18:08:51.419824+0000","last_clean_scrub_stamp":"2024-05-08T18:08:51.419824+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:39:27.870934+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.080658+0000","last_change":"2024-05-08T18:04:35.552341+0000","last_active":"2024-05-08T18:08:05.080658+0000","last_peered":"2024-05-08T18:08:05.080658+0000","last_clean":"2024-05-08T18:08:05.080658+0000","last_became_active":"2024-05-08T18:04:35.552017+0000","last_became_peered":"2024-05-08T18:04:35.552017+0000","last_unstale":"2024-05-08T18:08:05.080658+0000","last_undegraded":"2024-05-08T18:08:05.080658+0000","last_fullsized":"2024-05-08T18:08:05.080658+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.080686+0000","last_change":"2024-05-08T18:04:43.837405+0000","last_active":"2024-05-08T18:08:05.080686+0000","last_peered":"2024-05-08T18:08:05.080686+0000","last_clean":"2024-05-08T18:08:05.080686+0000","last_became_active":"2024-05-08T18:04:43.837019+0000","last_became_peered":"2024-05-08T18:04:43.837019+0000","last_unstale":"2024-05-08T18:08:05.080686+0000","last_undegraded":"2024-05-08T18:08:05.080686+0000","last_fullsized":"2024-05-08T18:08:05.080686+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:04.168023+0000","last_change":"2024-05-08T18:05:30.871131+0000","last_active":"2024-05-08T18:08:04.168023+0000","last_peered":"2024-05-08T18:08:04.168023+0000","last_clean":"2024-05-08T18:08:04.168023+0000","last_became_active":"2024-05-08T18:05:30.870647+0000","last_became_peered":"2024-05-08T18:05:30.870647+0000","last_unstale":"2024-05-08T18:08:04.168023+0000","last_undegraded":"2024-05-08T18:08:04.168023+0000","last_fullsized":"2024-05-08T18:08:04.168023+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"29'91","reported_seq":181,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:34.946616+0000","last_change":"2024-05-08T18:08:34.946658+0000","last_active":"2024-05-08T18:08:34.946616+0000","last_peered":"2024-05-08T18:08:34.946616+0000","last_clean":"2024-05-08T18:08:34.946616+0000","last_became_active":"2024-05-08T18:04:49.830636+0000","last_became_peered":"2024-05-08T18:04:49.830636+0000","last_unstale":"2024-05-08T18:08:34.946616+0000","last_undegraded":"2024-05-08T18:08:34.946616+0000","last_fullsized":"2024-05-08T18:08:34.946616+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'91","last_scrub_stamp":"2024-05-08T18:08:34.946526+0000","last_deep_scrub":"29'91","last_deep_scrub_stamp":"2024-05-08T18:08:34.946526+0000","last_clean_scrub_stamp":"2024-05-08T18:08:34.946526+0000","objects_scrubbed":5,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:49.081723+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.600189+0000","last_change":"2024-05-08T18:08:21.600230+0000","last_active":"2024-05-08T18:08:21.600189+0000","last_peered":"2024-05-08T18:08:21.600189+0000","last_clean":"2024-05-08T18:08:21.600189+0000","last_became_active":"2024-05-08T18:04:51.844737+0000","last_became_peered":"2024-05-08T18:04:51.844737+0000","last_unstale":"2024-05-08T18:08:21.600189+0000","last_undegraded":"2024-05-08T18:08:21.600189+0000","last_fullsized":"2024-05-08T18:08:21.600189+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:21.600069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:21.600069+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.600069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:48.902927+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"29'86","reported_seq":176,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:25.255787+0000","last_change":"2024-05-08T18:08:25.255820+0000","last_active":"2024-05-08T18:08:25.255787+0000","last_peered":"2024-05-08T18:08:25.255787+0000","last_clean":"2024-05-08T18:08:25.255787+0000","last_became_active":"2024-05-08T18:04:53.838957+0000","last_became_peered":"2024-05-08T18:04:53.838957+0000","last_unstale":"2024-05-08T18:08:25.255787+0000","last_undegraded":"2024-05-08T18:08:25.255787+0000","last_fullsized":"2024-05-08T18:08:25.255787+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'86","last_scrub_stamp":"2024-05-08T18:08:25.255680+0000","last_deep_scrub":"29'86","last_deep_scrub_stamp":"2024-05-08T18:08:25.255680+0000","last_clean_scrub_stamp":"2024-05-08T18:08:25.255680+0000","objects_scrubbed":1,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:14:28.008294+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":216,"num_read_kb":214,"num_write":167,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"29'10","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:48.083593+0000","last_change":"2024-05-08T18:08:48.083623+0000","last_active":"2024-05-08T18:08:48.083593+0000","last_peered":"2024-05-08T18:08:48.083593+0000","last_clean":"2024-05-08T18:08:48.083593+0000","last_became_active":"2024-05-08T18:04:45.825843+0000","last_became_peered":"2024-05-08T18:04:45.825843+0000","last_unstale":"2024-05-08T18:08:48.083593+0000","last_undegraded":"2024-05-08T18:08:48.083593+0000","last_fullsized":"2024-05-08T18:08:48.083593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:48.083503+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:48.083503+0000","last_clean_scrub_stamp":"2024-05-08T18:08:48.083503+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:05:27.160237+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.080708+0000","last_change":"2024-05-08T18:04:35.552553+0000","last_active":"2024-05-08T18:08:05.080708+0000","last_peered":"2024-05-08T18:08:05.080708+0000","last_clean":"2024-05-08T18:08:05.080708+0000","last_became_active":"2024-05-08T18:04:35.552214+0000","last_became_peered":"2024-05-08T18:04:35.552214+0000","last_unstale":"2024-05-08T18:08:05.080708+0000","last_undegraded":"2024-05-08T18:08:05.080708+0000","last_fullsized":"2024-05-08T18:08:05.080708+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:54.429770+0000","last_change":"2024-05-08T18:08:54.429815+0000","last_active":"2024-05-08T18:08:54.429770+0000","last_peered":"2024-05-08T18:08:54.429770+0000","last_clean":"2024-05-08T18:08:54.429770+0000","last_became_active":"2024-05-08T18:04:43.820369+0000","last_became_peered":"2024-05-08T18:04:43.820369+0000","last_unstale":"2024-05-08T18:08:54.429770+0000","last_undegraded":"2024-05-08T18:08:54.429770+0000","last_fullsized":"2024-05-08T18:08:54.429770+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:54.429672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:54.429672+0000","last_clean_scrub_stamp":"2024-05-08T18:08:54.429672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:12.348107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:06:58.745186+0000","last_change":"2024-05-08T18:05:30.870035+0000","last_active":"2024-05-08T18:06:58.745186+0000","last_peered":"2024-05-08T18:06:58.745186+0000","last_clean":"2024-05-08T18:06:58.745186+0000","last_became_active":"2024-05-08T18:05:30.869578+0000","last_became_peered":"2024-05-08T18:05:30.869578+0000","last_unstale":"2024-05-08T18:06:58.745186+0000","last_undegraded":"2024-05-08T18:06:58.745186+0000","last_fullsized":"2024-05-08T18:06:58.745186+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"29'290","reported_seq":516,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.566330+0000","last_change":"2024-05-08T18:08:07.566330+0000","last_active":"2024-05-08T18:08:07.566330+0000","last_peered":"2024-05-08T18:08:07.566330+0000","last_clean":"2024-05-08T18:08:07.566330+0000","last_became_active":"2024-05-08T18:04:49.833053+0000","last_became_peered":"2024-05-08T18:04:49.833053+0000","last_unstale":"2024-05-08T18:08:07.566330+0000","last_undegraded":"2024-05-08T18:08:07.566330+0000","last_fullsized":"2024-05-08T18:08:07.566330+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'290","last_scrub_stamp":"2024-05-08T18:08:07.566185+0000","last_deep_scrub":"29'290","last_deep_scrub_stamp":"2024-05-08T18:08:07.566185+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.566185+0000","objects_scrubbed":10,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:24.566210+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":378,"num_read_kb":363,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:32.883526+0000","last_change":"2024-05-08T18:08:32.883526+0000","last_active":"2024-05-08T18:08:32.883526+0000","last_peered":"2024-05-08T18:08:32.883526+0000","last_clean":"2024-05-08T18:08:32.883526+0000","last_became_active":"2024-05-08T18:04:51.848066+0000","last_became_peered":"2024-05-08T18:04:51.848066+0000","last_unstale":"2024-05-08T18:08:32.883526+0000","last_undegraded":"2024-05-08T18:08:32.883526+0000","last_fullsized":"2024-05-08T18:08:32.883526+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:32.883401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:32.883401+0000","last_clean_scrub_stamp":"2024-05-08T18:08:32.883401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:44.412793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"29'3","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:00.515406+0000","last_change":"2024-05-08T18:09:00.515445+0000","last_active":"2024-05-08T18:09:00.515406+0000","last_peered":"2024-05-08T18:09:00.515406+0000","last_clean":"2024-05-08T18:09:00.515406+0000","last_became_active":"2024-05-08T18:04:53.842899+0000","last_became_peered":"2024-05-08T18:04:53.842899+0000","last_unstale":"2024-05-08T18:09:00.515406+0000","last_undegraded":"2024-05-08T18:09:00.515406+0000","last_fullsized":"2024-05-08T18:09:00.515406+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3","last_scrub_stamp":"2024-05-08T18:09:00.515314+0000","last_deep_scrub":"29'3","last_deep_scrub_stamp":"2024-05-08T18:09:00.515314+0000","last_clean_scrub_stamp":"2024-05-08T18:09:00.515314+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:36:11.806797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"29'8","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:59.474877+0000","last_change":"2024-05-08T18:08:59.474909+0000","last_active":"2024-05-08T18:08:59.474877+0000","last_peered":"2024-05-08T18:08:59.474877+0000","last_clean":"2024-05-08T18:08:59.474877+0000","last_became_active":"2024-05-08T18:04:45.812091+0000","last_became_peered":"2024-05-08T18:04:45.812091+0000","last_unstale":"2024-05-08T18:08:59.474877+0000","last_undegraded":"2024-05-08T18:08:59.474877+0000","last_fullsized":"2024-05-08T18:08:59.474877+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:59.474782+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:59.474782+0000","last_clean_scrub_stamp":"2024-05-08T18:08:59.474782+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:57.356263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:43.042058+0000","last_change":"2024-05-08T18:08:43.042058+0000","last_active":"2024-05-08T18:08:43.042058+0000","last_peered":"2024-05-08T18:08:43.042058+0000","last_clean":"2024-05-08T18:08:43.042058+0000","last_became_active":"2024-05-08T18:04:35.551313+0000","last_became_peered":"2024-05-08T18:04:35.551313+0000","last_unstale":"2024-05-08T18:08:43.042058+0000","last_undegraded":"2024-05-08T18:08:43.042058+0000","last_fullsized":"2024-05-08T18:08:43.042058+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:43.041942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:43.041942+0000","last_clean_scrub_stamp":"2024-05-08T18:08:43.041942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:53.026181+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:56.472181+0000","last_change":"2024-05-08T18:08:56.472209+0000","last_active":"2024-05-08T18:08:56.472181+0000","last_peered":"2024-05-08T18:08:56.472181+0000","last_clean":"2024-05-08T18:08:56.472181+0000","last_became_active":"2024-05-08T18:04:43.820335+0000","last_became_peered":"2024-05-08T18:04:43.820335+0000","last_unstale":"2024-05-08T18:08:56.472181+0000","last_undegraded":"2024-05-08T18:08:56.472181+0000","last_fullsized":"2024-05-08T18:08:56.472181+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:56.472080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:56.472080+0000","last_clean_scrub_stamp":"2024-05-08T18:08:56.472080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:36:07.763563+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"29'2","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.343008+0000","last_change":"2024-05-08T18:08:20.343043+0000","last_active":"2024-05-08T18:08:20.343008+0000","last_peered":"2024-05-08T18:08:20.343008+0000","last_clean":"2024-05-08T18:08:20.343008+0000","last_became_active":"2024-05-08T18:04:47.823631+0000","last_became_peered":"2024-05-08T18:04:47.823631+0000","last_unstale":"2024-05-08T18:08:20.343008+0000","last_undegraded":"2024-05-08T18:08:20.343008+0000","last_fullsized":"2024-05-08T18:08:20.343008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:20.342916+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:20.342916+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.342916+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:26:02.075842+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"29'24","reported_seq":102,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.233395+0000","last_change":"2024-05-08T18:08:15.233395+0000","last_active":"2024-05-08T18:08:15.233395+0000","last_peered":"2024-05-08T18:08:15.233395+0000","last_clean":"2024-05-08T18:08:15.233395+0000","last_became_active":"2024-05-08T18:04:49.827735+0000","last_became_peered":"2024-05-08T18:04:49.827735+0000","last_unstale":"2024-05-08T18:08:15.233395+0000","last_undegraded":"2024-05-08T18:08:15.233395+0000","last_fullsized":"2024-05-08T18:08:15.233395+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'24","last_scrub_stamp":"2024-05-08T18:08:15.233271+0000","last_deep_scrub":"29'24","last_deep_scrub_stamp":"2024-05-08T18:08:15.233271+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.233271+0000","objects_scrubbed":8,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:25.767355+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.442677+0000","last_change":"2024-05-08T18:08:09.442677+0000","last_active":"2024-05-08T18:08:09.442677+0000","last_peered":"2024-05-08T18:08:09.442677+0000","last_clean":"2024-05-08T18:08:09.442677+0000","last_became_active":"2024-05-08T18:04:51.843940+0000","last_became_peered":"2024-05-08T18:04:51.843940+0000","last_unstale":"2024-05-08T18:08:09.442677+0000","last_undegraded":"2024-05-08T18:08:09.442677+0000","last_fullsized":"2024-05-08T18:08:09.442677+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:09.442556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:09.442556+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.442556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:48:00.984508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"29'4","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.000331+0000","last_change":"2024-05-08T18:04:53.848676+0000","last_active":"2024-05-08T18:07:54.000331+0000","last_peered":"2024-05-08T18:07:54.000331+0000","last_clean":"2024-05-08T18:07:54.000331+0000","last_became_active":"2024-05-08T18:04:53.848255+0000","last_became_peered":"2024-05-08T18:04:53.848255+0000","last_unstale":"2024-05-08T18:07:54.000331+0000","last_undegraded":"2024-05-08T18:07:54.000331+0000","last_fullsized":"2024-05-08T18:07:54.000331+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"29'10","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:39.382370+0000","last_change":"2024-05-08T18:08:39.382370+0000","last_active":"2024-05-08T18:08:39.382370+0000","last_peered":"2024-05-08T18:08:39.382370+0000","last_clean":"2024-05-08T18:08:39.382370+0000","last_became_active":"2024-05-08T18:04:45.807085+0000","last_became_peered":"2024-05-08T18:04:45.807085+0000","last_unstale":"2024-05-08T18:08:39.382370+0000","last_undegraded":"2024-05-08T18:08:39.382370+0000","last_fullsized":"2024-05-08T18:08:39.382370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:39.382233+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:39.382233+0000","last_clean_scrub_stamp":"2024-05-08T18:08:39.382233+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:03.089676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.080639+0000","last_change":"2024-05-08T18:04:35.551456+0000","last_active":"2024-05-08T18:08:05.080639+0000","last_peered":"2024-05-08T18:08:05.080639+0000","last_clean":"2024-05-08T18:08:05.080639+0000","last_became_active":"2024-05-08T18:04:35.551120+0000","last_became_peered":"2024-05-08T18:04:35.551120+0000","last_unstale":"2024-05-08T18:08:05.080639+0000","last_undegraded":"2024-05-08T18:08:05.080639+0000","last_fullsized":"2024-05-08T18:08:05.080639+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:48.977118+0000","last_change":"2024-05-08T18:08:48.977118+0000","last_active":"2024-05-08T18:08:48.977118+0000","last_peered":"2024-05-08T18:08:48.977118+0000","last_clean":"2024-05-08T18:08:48.977118+0000","last_became_active":"2024-05-08T18:04:43.840036+0000","last_became_peered":"2024-05-08T18:04:43.840036+0000","last_unstale":"2024-05-08T18:08:48.977118+0000","last_undegraded":"2024-05-08T18:08:48.977118+0000","last_fullsized":"2024-05-08T18:08:48.977118+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:48.976992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:48.976992+0000","last_clean_scrub_stamp":"2024-05-08T18:08:48.976992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:39:35.647911+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0,3],"acting":[2,1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.607246+0000","last_change":"2024-05-08T18:08:20.607246+0000","last_active":"2024-05-08T18:08:20.607246+0000","last_peered":"2024-05-08T18:08:20.607246+0000","last_clean":"2024-05-08T18:08:20.607246+0000","last_became_active":"2024-05-08T18:04:47.823093+0000","last_became_peered":"2024-05-08T18:04:47.823093+0000","last_unstale":"2024-05-08T18:08:20.607246+0000","last_undegraded":"2024-05-08T18:08:20.607246+0000","last_fullsized":"2024-05-08T18:08:20.607246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:20.607136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:20.607136+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.607136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:01.289711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"29'20","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:24.842547+0000","last_change":"2024-05-08T18:08:24.842630+0000","last_active":"2024-05-08T18:08:24.842547+0000","last_peered":"2024-05-08T18:08:24.842547+0000","last_clean":"2024-05-08T18:08:24.842547+0000","last_became_active":"2024-05-08T18:04:49.830586+0000","last_became_peered":"2024-05-08T18:04:49.830586+0000","last_unstale":"2024-05-08T18:08:24.842547+0000","last_undegraded":"2024-05-08T18:08:24.842547+0000","last_fullsized":"2024-05-08T18:08:24.842547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'20","last_scrub_stamp":"2024-05-08T18:08:24.842450+0000","last_deep_scrub":"29'20","last_deep_scrub_stamp":"2024-05-08T18:08:24.842450+0000","last_clean_scrub_stamp":"2024-05-08T18:08:24.842450+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:35.285600+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":394,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:37.005677+0000","last_change":"2024-05-08T18:08:37.005716+0000","last_active":"2024-05-08T18:08:37.005677+0000","last_peered":"2024-05-08T18:08:37.005677+0000","last_clean":"2024-05-08T18:08:37.005677+0000","last_became_active":"2024-05-08T18:04:51.849137+0000","last_became_peered":"2024-05-08T18:04:51.849137+0000","last_unstale":"2024-05-08T18:08:37.005677+0000","last_undegraded":"2024-05-08T18:08:37.005677+0000","last_fullsized":"2024-05-08T18:08:37.005677+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:37.005585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:37.005585+0000","last_clean_scrub_stamp":"2024-05-08T18:08:37.005585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:35:48.297068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'4","reported_seq":22,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745230+0000","last_change":"2024-05-08T18:04:53.851345+0000","last_active":"2024-05-08T18:07:03.745230+0000","last_peered":"2024-05-08T18:07:03.745230+0000","last_clean":"2024-05-08T18:07:03.745230+0000","last_became_active":"2024-05-08T18:04:53.851008+0000","last_became_peered":"2024-05-08T18:04:53.851008+0000","last_unstale":"2024-05-08T18:07:03.745230+0000","last_undegraded":"2024-05-08T18:07:03.745230+0000","last_fullsized":"2024-05-08T18:07:03.745230+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'2","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:27.488724+0000","last_change":"2024-05-08T18:08:27.488724+0000","last_active":"2024-05-08T18:08:27.488724+0000","last_peered":"2024-05-08T18:08:27.488724+0000","last_clean":"2024-05-08T18:08:27.488724+0000","last_became_active":"2024-05-08T18:04:45.819326+0000","last_became_peered":"2024-05-08T18:04:45.819326+0000","last_unstale":"2024-05-08T18:08:27.488724+0000","last_undegraded":"2024-05-08T18:08:27.488724+0000","last_fullsized":"2024-05-08T18:08:27.488724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:27.488574+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:27.488574+0000","last_clean_scrub_stamp":"2024-05-08T18:08:27.488574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:06.323416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:49.957730+0000","last_change":"2024-05-08T18:08:49.957730+0000","last_active":"2024-05-08T18:08:49.957730+0000","last_peered":"2024-05-08T18:08:49.957730+0000","last_clean":"2024-05-08T18:08:49.957730+0000","last_became_active":"2024-05-08T18:04:43.840236+0000","last_became_peered":"2024-05-08T18:04:43.840236+0000","last_unstale":"2024-05-08T18:08:49.957730+0000","last_undegraded":"2024-05-08T18:08:49.957730+0000","last_fullsized":"2024-05-08T18:08:49.957730+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:49.957583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:49.957583+0000","last_clean_scrub_stamp":"2024-05-08T18:08:49.957583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:10.553789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:52.369564+0000","last_change":"2024-05-08T18:08:52.369589+0000","last_active":"2024-05-08T18:08:52.369564+0000","last_peered":"2024-05-08T18:08:52.369564+0000","last_clean":"2024-05-08T18:08:52.369564+0000","last_became_active":"2024-05-08T18:04:35.552173+0000","last_became_peered":"2024-05-08T18:04:35.552173+0000","last_unstale":"2024-05-08T18:08:52.369564+0000","last_undegraded":"2024-05-08T18:08:52.369564+0000","last_fullsized":"2024-05-08T18:08:52.369564+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:52.369478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:52.369478+0000","last_clean_scrub_stamp":"2024-05-08T18:08:52.369478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:03.898871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:24.599665+0000","last_change":"2024-05-08T18:08:24.599665+0000","last_active":"2024-05-08T18:08:24.599665+0000","last_peered":"2024-05-08T18:08:24.599665+0000","last_clean":"2024-05-08T18:08:24.599665+0000","last_became_active":"2024-05-08T18:04:47.819910+0000","last_became_peered":"2024-05-08T18:04:47.819910+0000","last_unstale":"2024-05-08T18:08:24.599665+0000","last_undegraded":"2024-05-08T18:08:24.599665+0000","last_fullsized":"2024-05-08T18:08:24.599665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:24.599549+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:24.599549+0000","last_clean_scrub_stamp":"2024-05-08T18:08:24.599549+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:03.434391+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"29'73","reported_seq":150,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.598754+0000","last_change":"2024-05-08T18:08:15.598754+0000","last_active":"2024-05-08T18:08:15.598754+0000","last_peered":"2024-05-08T18:08:15.598754+0000","last_clean":"2024-05-08T18:08:15.598754+0000","last_became_active":"2024-05-08T18:04:49.831360+0000","last_became_peered":"2024-05-08T18:04:49.831360+0000","last_unstale":"2024-05-08T18:08:15.598754+0000","last_undegraded":"2024-05-08T18:08:15.598754+0000","last_fullsized":"2024-05-08T18:08:15.598754+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'73","last_scrub_stamp":"2024-05-08T18:08:15.598616+0000","last_deep_scrub":"29'73","last_deep_scrub_stamp":"2024-05-08T18:08:15.598616+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.598616+0000","objects_scrubbed":9,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:23:01.195330+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":336,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.250201+0000","last_change":"2024-05-08T18:08:13.250201+0000","last_active":"2024-05-08T18:08:13.250201+0000","last_peered":"2024-05-08T18:08:13.250201+0000","last_clean":"2024-05-08T18:08:13.250201+0000","last_became_active":"2024-05-08T18:04:51.836441+0000","last_became_peered":"2024-05-08T18:04:51.836441+0000","last_unstale":"2024-05-08T18:08:13.250201+0000","last_undegraded":"2024-05-08T18:08:13.250201+0000","last_fullsized":"2024-05-08T18:08:13.250201+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:13.250078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:13.250078+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.250078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:13.374535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"29'9","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:29.474467+0000","last_change":"2024-05-08T18:08:29.474491+0000","last_active":"2024-05-08T18:08:29.474467+0000","last_peered":"2024-05-08T18:08:29.474467+0000","last_clean":"2024-05-08T18:08:29.474467+0000","last_became_active":"2024-05-08T18:04:53.837920+0000","last_became_peered":"2024-05-08T18:04:53.837920+0000","last_unstale":"2024-05-08T18:08:29.474467+0000","last_undegraded":"2024-05-08T18:08:29.474467+0000","last_fullsized":"2024-05-08T18:08:29.474467+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'9","last_scrub_stamp":"2024-05-08T18:08:29.474349+0000","last_deep_scrub":"29'9","last_deep_scrub_stamp":"2024-05-08T18:08:29.474349+0000","last_clean_scrub_stamp":"2024-05-08T18:08:29.474349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:17.638910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"29'16","reported_seq":88,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:24.470006+0000","last_change":"2024-05-08T18:08:24.470006+0000","last_active":"2024-05-08T18:08:24.470006+0000","last_peered":"2024-05-08T18:08:24.470006+0000","last_clean":"2024-05-08T18:08:24.470006+0000","last_became_active":"2024-05-08T18:04:45.821107+0000","last_became_peered":"2024-05-08T18:04:45.821107+0000","last_unstale":"2024-05-08T18:08:24.470006+0000","last_undegraded":"2024-05-08T18:08:24.470006+0000","last_fullsized":"2024-05-08T18:08:24.470006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'16","last_scrub_stamp":"2024-05-08T18:08:24.469872+0000","last_deep_scrub":"29'16","last_deep_scrub_stamp":"2024-05-08T18:08:24.469872+0000","last_clean_scrub_stamp":"2024-05-08T18:08:24.469872+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:51.772730+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:57.481987+0000","last_change":"2024-05-08T18:08:57.482019+0000","last_active":"2024-05-08T18:08:57.481987+0000","last_peered":"2024-05-08T18:08:57.481987+0000","last_clean":"2024-05-08T18:08:57.481987+0000","last_became_active":"2024-05-08T18:04:43.827920+0000","last_became_peered":"2024-05-08T18:04:43.827920+0000","last_unstale":"2024-05-08T18:08:57.481987+0000","last_undegraded":"2024-05-08T18:08:57.481987+0000","last_fullsized":"2024-05-08T18:08:57.481987+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:57.481903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:57.481903+0000","last_clean_scrub_stamp":"2024-05-08T18:08:57.481903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:05.904037+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.080660+0000","last_change":"2024-05-08T18:04:35.551253+0000","last_active":"2024-05-08T18:08:05.080660+0000","last_peered":"2024-05-08T18:08:05.080660+0000","last_clean":"2024-05-08T18:08:05.080660+0000","last_became_active":"2024-05-08T18:04:35.550912+0000","last_became_peered":"2024-05-08T18:04:35.550912+0000","last_unstale":"2024-05-08T18:08:05.080660+0000","last_undegraded":"2024-05-08T18:08:05.080660+0000","last_fullsized":"2024-05-08T18:08:05.080660+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.811076+0000","last_change":"2024-05-08T18:08:15.811076+0000","last_active":"2024-05-08T18:08:15.811076+0000","last_peered":"2024-05-08T18:08:15.811076+0000","last_clean":"2024-05-08T18:08:15.811076+0000","last_became_active":"2024-05-08T18:04:47.813717+0000","last_became_peered":"2024-05-08T18:04:47.813717+0000","last_unstale":"2024-05-08T18:08:15.811076+0000","last_undegraded":"2024-05-08T18:08:15.811076+0000","last_fullsized":"2024-05-08T18:08:15.811076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:15.810959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:15.810959+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.810959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:35.598153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"29'21","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.551971+0000","last_change":"2024-05-08T18:08:06.551971+0000","last_active":"2024-05-08T18:08:06.551971+0000","last_peered":"2024-05-08T18:08:06.551971+0000","last_clean":"2024-05-08T18:08:06.551971+0000","last_became_active":"2024-05-08T18:04:49.835451+0000","last_became_peered":"2024-05-08T18:04:49.835451+0000","last_unstale":"2024-05-08T18:08:06.551971+0000","last_undegraded":"2024-05-08T18:08:06.551971+0000","last_fullsized":"2024-05-08T18:08:06.551971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'21","last_scrub_stamp":"2024-05-08T18:08:06.551844+0000","last_deep_scrub":"29'21","last_deep_scrub_stamp":"2024-05-08T18:08:06.551844+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.551844+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:05.040309+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"29'285","reported_seq":382,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:04.536905+0000","last_change":"2024-05-08T18:08:04.536905+0000","last_active":"2024-05-08T18:08:04.536905+0000","last_peered":"2024-05-08T18:08:04.536905+0000","last_clean":"2024-05-08T18:08:04.536905+0000","last_became_active":"2024-05-08T18:04:51.832109+0000","last_became_peered":"2024-05-08T18:04:51.832109+0000","last_unstale":"2024-05-08T18:08:04.536905+0000","last_undegraded":"2024-05-08T18:08:04.536905+0000","last_fullsized":"2024-05-08T18:08:04.536905+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:04.536786+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:04.536786+0000","last_clean_scrub_stamp":"2024-05-08T18:08:04.536786+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:42.101932+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:49.039204+0000","last_change":"2024-05-08T18:08:49.039245+0000","last_active":"2024-05-08T18:08:49.039204+0000","last_peered":"2024-05-08T18:08:49.039204+0000","last_clean":"2024-05-08T18:08:49.039204+0000","last_became_active":"2024-05-08T18:04:53.842998+0000","last_became_peered":"2024-05-08T18:04:53.842998+0000","last_unstale":"2024-05-08T18:08:49.039204+0000","last_undegraded":"2024-05-08T18:08:49.039204+0000","last_fullsized":"2024-05-08T18:08:49.039204+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:49.039104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:49.039104+0000","last_clean_scrub_stamp":"2024-05-08T18:08:49.039104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:33.981256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'4","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.838524+0000","last_change":"2024-05-08T18:08:19.838524+0000","last_active":"2024-05-08T18:08:19.838524+0000","last_peered":"2024-05-08T18:08:19.838524+0000","last_clean":"2024-05-08T18:08:19.838524+0000","last_became_active":"2024-05-08T18:04:45.829257+0000","last_became_peered":"2024-05-08T18:04:45.829257+0000","last_unstale":"2024-05-08T18:08:19.838524+0000","last_undegraded":"2024-05-08T18:08:19.838524+0000","last_fullsized":"2024-05-08T18:08:19.838524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:19.838313+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:19.838313+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.838313+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:43.545756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"29'3","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.080721+0000","last_change":"2024-05-08T18:04:43.835453+0000","last_active":"2024-05-08T18:08:05.080721+0000","last_peered":"2024-05-08T18:08:05.080721+0000","last_clean":"2024-05-08T18:08:05.080721+0000","last_became_active":"2024-05-08T18:04:43.834610+0000","last_became_peered":"2024-05-08T18:04:43.834610+0000","last_unstale":"2024-05-08T18:08:05.080721+0000","last_undegraded":"2024-05-08T18:08:05.080721+0000","last_fullsized":"2024-05-08T18:08:05.080721+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:51.327283+0000","last_change":"2024-05-08T18:08:51.327283+0000","last_active":"2024-05-08T18:08:51.327283+0000","last_peered":"2024-05-08T18:08:51.327283+0000","last_clean":"2024-05-08T18:08:51.327283+0000","last_became_active":"2024-05-08T18:04:35.551366+0000","last_became_peered":"2024-05-08T18:04:35.551366+0000","last_unstale":"2024-05-08T18:08:51.327283+0000","last_undegraded":"2024-05-08T18:08:51.327283+0000","last_fullsized":"2024-05-08T18:08:51.327283+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:51.327154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:51.327154+0000","last_clean_scrub_stamp":"2024-05-08T18:08:51.327154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:17:16.185628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.923265+0000","last_change":"2024-05-08T18:08:06.923265+0000","last_active":"2024-05-08T18:08:06.923265+0000","last_peered":"2024-05-08T18:08:06.923265+0000","last_clean":"2024-05-08T18:08:06.923265+0000","last_became_active":"2024-05-08T18:04:47.819866+0000","last_became_peered":"2024-05-08T18:04:47.819866+0000","last_unstale":"2024-05-08T18:08:06.923265+0000","last_undegraded":"2024-05-08T18:08:06.923265+0000","last_fullsized":"2024-05-08T18:08:06.923265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:06.923149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:06.923149+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.923149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:30.572799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"29'15","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:35.962385+0000","last_change":"2024-05-08T18:08:35.962385+0000","last_active":"2024-05-08T18:08:35.962385+0000","last_peered":"2024-05-08T18:08:35.962385+0000","last_clean":"2024-05-08T18:08:35.962385+0000","last_became_active":"2024-05-08T18:04:49.830452+0000","last_became_peered":"2024-05-08T18:04:49.830452+0000","last_unstale":"2024-05-08T18:08:35.962385+0000","last_undegraded":"2024-05-08T18:08:35.962385+0000","last_fullsized":"2024-05-08T18:08:35.962385+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:35.962267+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:35.962267+0000","last_clean_scrub_stamp":"2024-05-08T18:08:35.962267+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:45.951593+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"29'285","reported_seq":412,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.471786+0000","last_change":"2024-05-08T18:08:06.471786+0000","last_active":"2024-05-08T18:08:06.471786+0000","last_peered":"2024-05-08T18:08:06.471786+0000","last_clean":"2024-05-08T18:08:06.471786+0000","last_became_active":"2024-05-08T18:04:51.831155+0000","last_became_peered":"2024-05-08T18:04:51.831155+0000","last_unstale":"2024-05-08T18:08:06.471786+0000","last_undegraded":"2024-05-08T18:08:06.471786+0000","last_fullsized":"2024-05-08T18:08:06.471786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:06.471664+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:06.471664+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.471664+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:29:09.321348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'5","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:01.531601+0000","last_change":"2024-05-08T18:09:01.531628+0000","last_active":"2024-05-08T18:09:01.531601+0000","last_peered":"2024-05-08T18:09:01.531601+0000","last_clean":"2024-05-08T18:09:01.531601+0000","last_became_active":"2024-05-08T18:04:53.848862+0000","last_became_peered":"2024-05-08T18:04:53.848862+0000","last_unstale":"2024-05-08T18:09:01.531601+0000","last_undegraded":"2024-05-08T18:09:01.531601+0000","last_fullsized":"2024-05-08T18:09:01.531601+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'5","last_scrub_stamp":"2024-05-08T18:09:01.531514+0000","last_deep_scrub":"29'5","last_deep_scrub_stamp":"2024-05-08T18:09:01.531514+0000","last_clean_scrub_stamp":"2024-05-08T18:09:01.531514+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:09.953648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"29'6","reported_seq":61,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:41.133464+0000","last_change":"2024-05-08T18:08:41.133464+0000","last_active":"2024-05-08T18:08:41.133464+0000","last_peered":"2024-05-08T18:08:41.133464+0000","last_clean":"2024-05-08T18:08:41.133464+0000","last_became_active":"2024-05-08T18:04:45.822824+0000","last_became_peered":"2024-05-08T18:04:45.822824+0000","last_unstale":"2024-05-08T18:08:41.133464+0000","last_undegraded":"2024-05-08T18:08:41.133464+0000","last_fullsized":"2024-05-08T18:08:41.133464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:41.133339+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:41.133339+0000","last_clean_scrub_stamp":"2024-05-08T18:08:41.133339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:17.740267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:44.501238+0000","last_change":"2024-05-08T18:08:44.501238+0000","last_active":"2024-05-08T18:08:44.501238+0000","last_peered":"2024-05-08T18:08:44.501238+0000","last_clean":"2024-05-08T18:08:44.501238+0000","last_became_active":"2024-05-08T18:04:43.839240+0000","last_became_peered":"2024-05-08T18:04:43.839240+0000","last_unstale":"2024-05-08T18:08:44.501238+0000","last_undegraded":"2024-05-08T18:08:44.501238+0000","last_fullsized":"2024-05-08T18:08:44.501238+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:44.501120+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:44.501120+0000","last_clean_scrub_stamp":"2024-05-08T18:08:44.501120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:27.037641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.080680+0000","last_change":"2024-05-08T18:04:35.551765+0000","last_active":"2024-05-08T18:08:05.080680+0000","last_peered":"2024-05-08T18:08:05.080680+0000","last_clean":"2024-05-08T18:08:05.080680+0000","last_became_active":"2024-05-08T18:04:35.551412+0000","last_became_peered":"2024-05-08T18:04:35.551412+0000","last_unstale":"2024-05-08T18:08:05.080680+0000","last_undegraded":"2024-05-08T18:08:05.080680+0000","last_fullsized":"2024-05-08T18:08:05.080680+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:34.543271+0000","last_clean_scrub_stamp":"2024-05-08T18:04:34.543271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.843881+0000","last_change":"2024-05-08T18:08:13.843881+0000","last_active":"2024-05-08T18:08:13.843881+0000","last_peered":"2024-05-08T18:08:13.843881+0000","last_clean":"2024-05-08T18:08:13.843881+0000","last_became_active":"2024-05-08T18:04:47.815947+0000","last_became_peered":"2024-05-08T18:04:47.815947+0000","last_unstale":"2024-05-08T18:08:13.843881+0000","last_undegraded":"2024-05-08T18:08:13.843881+0000","last_fullsized":"2024-05-08T18:08:13.843881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:13.843666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:13.843666+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.843666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:56.027223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"29'96","reported_seq":190,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.153072+0000","last_change":"2024-05-08T18:08:10.153072+0000","last_active":"2024-05-08T18:08:10.153072+0000","last_peered":"2024-05-08T18:08:10.153072+0000","last_clean":"2024-05-08T18:08:10.153072+0000","last_became_active":"2024-05-08T18:04:49.834586+0000","last_became_peered":"2024-05-08T18:04:49.834586+0000","last_unstale":"2024-05-08T18:08:10.153072+0000","last_undegraded":"2024-05-08T18:08:10.153072+0000","last_fullsized":"2024-05-08T18:08:10.153072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'96","last_scrub_stamp":"2024-05-08T18:08:10.152888+0000","last_deep_scrub":"29'96","last_deep_scrub_stamp":"2024-05-08T18:08:10.152888+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.152888+0000","objects_scrubbed":8,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:16.511455+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.455904+0000","last_change":"2024-05-08T18:08:08.455904+0000","last_active":"2024-05-08T18:08:08.455904+0000","last_peered":"2024-05-08T18:08:08.455904+0000","last_clean":"2024-05-08T18:08:08.455904+0000","last_became_active":"2024-05-08T18:04:51.830760+0000","last_became_peered":"2024-05-08T18:04:51.830760+0000","last_unstale":"2024-05-08T18:08:08.455904+0000","last_undegraded":"2024-05-08T18:08:08.455904+0000","last_fullsized":"2024-05-08T18:08:08.455904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:08.455773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:08.455773+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.455773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:12.703852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.342682+0000","last_change":"2024-05-08T18:08:21.342717+0000","last_active":"2024-05-08T18:08:21.342682+0000","last_peered":"2024-05-08T18:08:21.342682+0000","last_clean":"2024-05-08T18:08:21.342682+0000","last_became_active":"2024-05-08T18:04:53.838600+0000","last_became_peered":"2024-05-08T18:04:53.838600+0000","last_unstale":"2024-05-08T18:08:21.342682+0000","last_undegraded":"2024-05-08T18:08:21.342682+0000","last_fullsized":"2024-05-08T18:08:21.342682+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:21.342590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:21.342590+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.342590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:44.919676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"27'2","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:51.909643+0000","last_change":"2024-05-08T18:08:51.909643+0000","last_active":"2024-05-08T18:08:51.909643+0000","last_peered":"2024-05-08T18:08:51.909643+0000","last_clean":"2024-05-08T18:08:51.909643+0000","last_became_active":"2024-05-08T18:04:53.839500+0000","last_became_peered":"2024-05-08T18:04:53.839500+0000","last_unstale":"2024-05-08T18:08:51.909643+0000","last_undegraded":"2024-05-08T18:08:51.909643+0000","last_fullsized":"2024-05-08T18:08:51.909643+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:51.909525+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:51.909525+0000","last_clean_scrub_stamp":"2024-05-08T18:08:51.909525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:05:28.466923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"29'8","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:39.209848+0000","last_change":"2024-05-08T18:08:39.209919+0000","last_active":"2024-05-08T18:08:39.209848+0000","last_peered":"2024-05-08T18:08:39.209848+0000","last_clean":"2024-05-08T18:08:39.209848+0000","last_became_active":"2024-05-08T18:04:45.819508+0000","last_became_peered":"2024-05-08T18:04:45.819508+0000","last_unstale":"2024-05-08T18:08:39.209848+0000","last_undegraded":"2024-05-08T18:08:39.209848+0000","last_fullsized":"2024-05-08T18:08:39.209848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:39.209750+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:39.209750+0000","last_clean_scrub_stamp":"2024-05-08T18:08:39.209750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:48.744139+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.746021+0000","last_change":"2024-05-08T18:04:43.844864+0000","last_active":"2024-05-08T18:07:03.746021+0000","last_peered":"2024-05-08T18:07:03.746021+0000","last_clean":"2024-05-08T18:07:03.746021+0000","last_became_active":"2024-05-08T18:04:43.837929+0000","last_became_peered":"2024-05-08T18:04:43.837929+0000","last_unstale":"2024-05-08T18:07:03.746021+0000","last_undegraded":"2024-05-08T18:07:03.746021+0000","last_fullsized":"2024-05-08T18:07:03.746021+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.504440+0000","last_change":"2024-05-08T18:08:19.504467+0000","last_active":"2024-05-08T18:08:19.504440+0000","last_peered":"2024-05-08T18:08:19.504440+0000","last_clean":"2024-05-08T18:08:19.504440+0000","last_became_active":"2024-05-08T18:04:47.818443+0000","last_became_peered":"2024-05-08T18:04:47.818443+0000","last_unstale":"2024-05-08T18:08:19.504440+0000","last_undegraded":"2024-05-08T18:08:19.504440+0000","last_fullsized":"2024-05-08T18:08:19.504440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:19.504350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:19.504350+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.504350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:24.197683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"29'40","reported_seq":96,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:30.848309+0000","last_change":"2024-05-08T18:08:30.848380+0000","last_active":"2024-05-08T18:08:30.848309+0000","last_peered":"2024-05-08T18:08:30.848309+0000","last_clean":"2024-05-08T18:08:30.848309+0000","last_became_active":"2024-05-08T18:04:49.833599+0000","last_became_peered":"2024-05-08T18:04:49.833599+0000","last_unstale":"2024-05-08T18:08:30.848309+0000","last_undegraded":"2024-05-08T18:08:30.848309+0000","last_fullsized":"2024-05-08T18:08:30.848309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'40","last_scrub_stamp":"2024-05-08T18:08:30.848207+0000","last_deep_scrub":"29'40","last_deep_scrub_stamp":"2024-05-08T18:08:30.848207+0000","last_clean_scrub_stamp":"2024-05-08T18:08:30.848207+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:31:32.685411+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:26.837175+0000","last_change":"2024-05-08T18:08:26.837221+0000","last_active":"2024-05-08T18:08:26.837175+0000","last_peered":"2024-05-08T18:08:26.837175+0000","last_clean":"2024-05-08T18:08:26.837175+0000","last_became_active":"2024-05-08T18:04:51.844526+0000","last_became_peered":"2024-05-08T18:04:51.844526+0000","last_unstale":"2024-05-08T18:08:26.837175+0000","last_undegraded":"2024-05-08T18:08:26.837175+0000","last_fullsized":"2024-05-08T18:08:26.837175+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:26.837073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:26.837073+0000","last_clean_scrub_stamp":"2024-05-08T18:08:26.837073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:03.923893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"29'11","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:03.708494+0000","last_change":"2024-05-08T18:04:53.850911+0000","last_active":"2024-05-08T18:08:03.708494+0000","last_peered":"2024-05-08T18:08:03.708494+0000","last_clean":"2024-05-08T18:08:03.708494+0000","last_became_active":"2024-05-08T18:04:53.850593+0000","last_became_peered":"2024-05-08T18:04:53.850593+0000","last_unstale":"2024-05-08T18:08:03.708494+0000","last_undegraded":"2024-05-08T18:08:03.708494+0000","last_fullsized":"2024-05-08T18:08:03.708494+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":62,"num_write":31,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"29'8","reported_seq":82,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:31.489072+0000","last_change":"2024-05-08T18:08:31.489126+0000","last_active":"2024-05-08T18:08:31.489072+0000","last_peered":"2024-05-08T18:08:31.489072+0000","last_clean":"2024-05-08T18:08:31.489072+0000","last_became_active":"2024-05-08T18:04:45.818979+0000","last_became_peered":"2024-05-08T18:04:45.818979+0000","last_unstale":"2024-05-08T18:08:31.489072+0000","last_undegraded":"2024-05-08T18:08:31.489072+0000","last_fullsized":"2024-05-08T18:08:31.489072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:31.488942+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:31.488942+0000","last_clean_scrub_stamp":"2024-05-08T18:08:31.488942+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:34.056789+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:46.971371+0000","last_change":"2024-05-08T18:08:46.971417+0000","last_active":"2024-05-08T18:08:46.971371+0000","last_peered":"2024-05-08T18:08:46.971371+0000","last_clean":"2024-05-08T18:08:46.971371+0000","last_became_active":"2024-05-08T18:04:43.836614+0000","last_became_peered":"2024-05-08T18:04:43.836614+0000","last_unstale":"2024-05-08T18:08:46.971371+0000","last_undegraded":"2024-05-08T18:08:46.971371+0000","last_fullsized":"2024-05-08T18:08:46.971371+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:46.971249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:46.971249+0000","last_clean_scrub_stamp":"2024-05-08T18:08:46.971249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:02:46.938192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.821717+0000","last_change":"2024-05-08T18:08:14.821740+0000","last_active":"2024-05-08T18:08:14.821717+0000","last_peered":"2024-05-08T18:08:14.821717+0000","last_clean":"2024-05-08T18:08:14.821717+0000","last_became_active":"2024-05-08T18:04:47.814561+0000","last_became_peered":"2024-05-08T18:04:47.814561+0000","last_unstale":"2024-05-08T18:08:14.821717+0000","last_undegraded":"2024-05-08T18:08:14.821717+0000","last_fullsized":"2024-05-08T18:08:14.821717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:14.821626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:14.821626+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.821626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:24.805865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"29'10","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.572538+0000","last_change":"2024-05-08T18:08:18.572575+0000","last_active":"2024-05-08T18:08:18.572538+0000","last_peered":"2024-05-08T18:08:18.572538+0000","last_clean":"2024-05-08T18:08:18.572538+0000","last_became_active":"2024-05-08T18:04:49.825347+0000","last_became_peered":"2024-05-08T18:04:49.825347+0000","last_unstale":"2024-05-08T18:08:18.572538+0000","last_undegraded":"2024-05-08T18:08:18.572538+0000","last_fullsized":"2024-05-08T18:08:18.572538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:18.572458+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:18.572458+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.572458+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:21.293654+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.319006+0000","last_change":"2024-05-08T18:08:17.319068+0000","last_active":"2024-05-08T18:08:17.319006+0000","last_peered":"2024-05-08T18:08:17.319006+0000","last_clean":"2024-05-08T18:08:17.319006+0000","last_became_active":"2024-05-08T18:04:51.843002+0000","last_became_peered":"2024-05-08T18:04:51.843002+0000","last_unstale":"2024-05-08T18:08:17.319006+0000","last_undegraded":"2024-05-08T18:08:17.319006+0000","last_fullsized":"2024-05-08T18:08:17.319006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:17.318901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:17.318901+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.318901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:07.004932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:00.468674+0000","last_change":"2024-05-08T18:09:00.468698+0000","last_active":"2024-05-08T18:09:00.468674+0000","last_peered":"2024-05-08T18:09:00.468674+0000","last_clean":"2024-05-08T18:09:00.468674+0000","last_became_active":"2024-05-08T18:04:53.840437+0000","last_became_peered":"2024-05-08T18:04:53.840437+0000","last_unstale":"2024-05-08T18:09:00.468674+0000","last_undegraded":"2024-05-08T18:09:00.468674+0000","last_fullsized":"2024-05-08T18:09:00.468674+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:00.468582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:00.468582+0000","last_clean_scrub_stamp":"2024-05-08T18:09:00.468582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:35:34.723489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"29'8","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:53.867017+0000","last_change":"2024-05-08T18:05:30.868442+0000","last_active":"2024-05-08T18:07:53.867017+0000","last_peered":"2024-05-08T18:07:53.867017+0000","last_clean":"2024-05-08T18:07:53.867017+0000","last_became_active":"2024-05-08T18:05:30.868156+0000","last_became_peered":"2024-05-08T18:05:30.868156+0000","last_unstale":"2024-05-08T18:07:53.867017+0000","last_undegraded":"2024-05-08T18:07:53.867017+0000","last_fullsized":"2024-05-08T18:07:53.867017+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:44.789644+0000","last_clean_scrub_stamp":"2024-05-08T18:04:44.789644+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:45.035572+0000","last_change":"2024-05-08T18:08:45.035600+0000","last_active":"2024-05-08T18:08:45.035572+0000","last_peered":"2024-05-08T18:08:45.035572+0000","last_clean":"2024-05-08T18:08:45.035572+0000","last_became_active":"2024-05-08T18:04:43.835435+0000","last_became_peered":"2024-05-08T18:04:43.835435+0000","last_unstale":"2024-05-08T18:08:45.035572+0000","last_undegraded":"2024-05-08T18:08:45.035572+0000","last_fullsized":"2024-05-08T18:08:45.035572+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:45.035484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:45.035484+0000","last_clean_scrub_stamp":"2024-05-08T18:08:45.035484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:06:21.992378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.385584+0000","last_change":"2024-05-08T18:08:19.385611+0000","last_active":"2024-05-08T18:08:19.385584+0000","last_peered":"2024-05-08T18:08:19.385584+0000","last_clean":"2024-05-08T18:08:19.385584+0000","last_became_active":"2024-05-08T18:04:47.824140+0000","last_became_peered":"2024-05-08T18:04:47.824140+0000","last_unstale":"2024-05-08T18:08:19.385584+0000","last_undegraded":"2024-05-08T18:08:19.385584+0000","last_fullsized":"2024-05-08T18:08:19.385584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:19.385463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:19.385463+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.385463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:15.297455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"29'48","reported_seq":120,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.582586+0000","last_change":"2024-05-08T18:08:19.582659+0000","last_active":"2024-05-08T18:08:19.582586+0000","last_peered":"2024-05-08T18:08:19.582586+0000","last_clean":"2024-05-08T18:08:19.582586+0000","last_became_active":"2024-05-08T18:04:49.824600+0000","last_became_peered":"2024-05-08T18:04:49.824600+0000","last_unstale":"2024-05-08T18:08:19.582586+0000","last_undegraded":"2024-05-08T18:08:19.582586+0000","last_fullsized":"2024-05-08T18:08:19.582586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'48","last_scrub_stamp":"2024-05-08T18:08:19.582484+0000","last_deep_scrub":"29'48","last_deep_scrub_stamp":"2024-05-08T18:08:19.582484+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.582484+0000","objects_scrubbed":6,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:25.159113+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":706,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.521105+0000","last_change":"2024-05-08T18:08:09.521130+0000","last_active":"2024-05-08T18:08:09.521105+0000","last_peered":"2024-05-08T18:08:09.521105+0000","last_clean":"2024-05-08T18:08:09.521105+0000","last_became_active":"2024-05-08T18:04:51.829937+0000","last_became_peered":"2024-05-08T18:04:51.829937+0000","last_unstale":"2024-05-08T18:08:09.521105+0000","last_undegraded":"2024-05-08T18:08:09.521105+0000","last_fullsized":"2024-05-08T18:08:09.521105+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:09.521013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:09.521013+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.521013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:29:12.370696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:45.103397+0000","last_change":"2024-05-08T18:08:45.103421+0000","last_active":"2024-05-08T18:08:45.103397+0000","last_peered":"2024-05-08T18:08:45.103397+0000","last_clean":"2024-05-08T18:08:45.103397+0000","last_became_active":"2024-05-08T18:04:53.842610+0000","last_became_peered":"2024-05-08T18:04:53.842610+0000","last_unstale":"2024-05-08T18:08:45.103397+0000","last_undegraded":"2024-05-08T18:08:45.103397+0000","last_fullsized":"2024-05-08T18:08:45.103397+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:45.103296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:45.103296+0000","last_clean_scrub_stamp":"2024-05-08T18:08:45.103296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:29.008024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.866944+0000","last_change":"2024-05-08T18:08:22.866966+0000","last_active":"2024-05-08T18:08:22.866944+0000","last_peered":"2024-05-08T18:08:22.866944+0000","last_clean":"2024-05-08T18:08:22.866944+0000","last_became_active":"2024-05-08T18:04:45.829458+0000","last_became_peered":"2024-05-08T18:04:45.829458+0000","last_unstale":"2024-05-08T18:08:22.866944+0000","last_undegraded":"2024-05-08T18:08:22.866944+0000","last_fullsized":"2024-05-08T18:08:22.866944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.866853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.866853+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.866853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:05.403374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"29'4","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:56.481494+0000","last_change":"2024-05-08T18:08:56.481530+0000","last_active":"2024-05-08T18:08:56.481494+0000","last_peered":"2024-05-08T18:08:56.481494+0000","last_clean":"2024-05-08T18:08:56.481494+0000","last_became_active":"2024-05-08T18:04:43.836633+0000","last_became_peered":"2024-05-08T18:04:43.836633+0000","last_unstale":"2024-05-08T18:08:56.481494+0000","last_undegraded":"2024-05-08T18:08:56.481494+0000","last_fullsized":"2024-05-08T18:08:56.481494+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:56.481386+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:56.481386+0000","last_clean_scrub_stamp":"2024-05-08T18:08:56.481386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:32.379503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"29'2","reported_seq":230,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.909561+0000","last_change":"2024-05-08T18:08:07.909590+0000","last_active":"2024-05-08T18:08:07.909561+0000","last_peered":"2024-05-08T18:08:07.909561+0000","last_clean":"2024-05-08T18:08:07.909561+0000","last_became_active":"2024-05-08T18:04:47.819771+0000","last_became_peered":"2024-05-08T18:04:47.819771+0000","last_unstale":"2024-05-08T18:08:07.909561+0000","last_undegraded":"2024-05-08T18:08:07.909561+0000","last_fullsized":"2024-05-08T18:08:07.909561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:07.909472+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:07.909472+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.909472+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:04.798235+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"29'36","reported_seq":91,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.344301+0000","last_change":"2024-05-08T18:08:18.344360+0000","last_active":"2024-05-08T18:08:18.344301+0000","last_peered":"2024-05-08T18:08:18.344301+0000","last_clean":"2024-05-08T18:08:18.344301+0000","last_became_active":"2024-05-08T18:04:49.824022+0000","last_became_peered":"2024-05-08T18:04:49.824022+0000","last_unstale":"2024-05-08T18:08:18.344301+0000","last_undegraded":"2024-05-08T18:08:18.344301+0000","last_fullsized":"2024-05-08T18:08:18.344301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:08:18.344202+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:08:18.344202+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.344202+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:36:47.464601+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.198536+0000","last_change":"2024-05-08T18:08:08.198575+0000","last_active":"2024-05-08T18:08:08.198536+0000","last_peered":"2024-05-08T18:08:08.198536+0000","last_clean":"2024-05-08T18:08:08.198536+0000","last_became_active":"2024-05-08T18:04:51.826424+0000","last_became_peered":"2024-05-08T18:04:51.826424+0000","last_unstale":"2024-05-08T18:08:08.198536+0000","last_undegraded":"2024-05-08T18:08:08.198536+0000","last_fullsized":"2024-05-08T18:08:08.198536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:08.198405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:08.198405+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.198405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:15.614149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"29'20","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:57.043843+0000","last_change":"2024-05-08T18:08:57.043872+0000","last_active":"2024-05-08T18:08:57.043843+0000","last_peered":"2024-05-08T18:08:57.043843+0000","last_clean":"2024-05-08T18:08:57.043843+0000","last_became_active":"2024-05-08T18:05:30.868409+0000","last_became_peered":"2024-05-08T18:05:30.868409+0000","last_unstale":"2024-05-08T18:08:57.043843+0000","last_undegraded":"2024-05-08T18:08:57.043843+0000","last_fullsized":"2024-05-08T18:08:57.043843+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'20","last_scrub_stamp":"2024-05-08T18:08:57.043740+0000","last_deep_scrub":"29'20","last_deep_scrub_stamp":"2024-05-08T18:08:57.043740+0000","last_clean_scrub_stamp":"2024-05-08T18:08:57.043740+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:03.723531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:08.746752+0000","last_change":"2024-05-08T18:04:43.844983+0000","last_active":"2024-05-08T18:07:08.746752+0000","last_peered":"2024-05-08T18:07:08.746752+0000","last_clean":"2024-05-08T18:07:08.746752+0000","last_became_active":"2024-05-08T18:04:43.838957+0000","last_became_peered":"2024-05-08T18:04:43.838957+0000","last_unstale":"2024-05-08T18:07:08.746752+0000","last_undegraded":"2024-05-08T18:07:08.746752+0000","last_fullsized":"2024-05-08T18:07:08.746752+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.606122+0000","last_change":"2024-05-08T18:08:23.606122+0000","last_active":"2024-05-08T18:08:23.606122+0000","last_peered":"2024-05-08T18:08:23.606122+0000","last_clean":"2024-05-08T18:08:23.606122+0000","last_became_active":"2024-05-08T18:04:47.820256+0000","last_became_peered":"2024-05-08T18:04:47.820256+0000","last_unstale":"2024-05-08T18:08:23.606122+0000","last_undegraded":"2024-05-08T18:08:23.606122+0000","last_fullsized":"2024-05-08T18:08:23.606122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:23.606002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:23.606002+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.606002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:04.849646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"29'63","reported_seq":136,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.582374+0000","last_change":"2024-05-08T18:08:16.582414+0000","last_active":"2024-05-08T18:08:16.582374+0000","last_peered":"2024-05-08T18:08:16.582374+0000","last_clean":"2024-05-08T18:08:16.582374+0000","last_became_active":"2024-05-08T18:04:49.825366+0000","last_became_peered":"2024-05-08T18:04:49.825366+0000","last_unstale":"2024-05-08T18:08:16.582374+0000","last_undegraded":"2024-05-08T18:08:16.582374+0000","last_fullsized":"2024-05-08T18:08:16.582374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'63","last_scrub_stamp":"2024-05-08T18:08:16.582295+0000","last_deep_scrub":"29'63","last_deep_scrub_stamp":"2024-05-08T18:08:16.582295+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.582295+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:21.275627+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.157727+0000","last_change":"2024-05-08T18:08:07.157775+0000","last_active":"2024-05-08T18:08:07.157727+0000","last_peered":"2024-05-08T18:08:07.157727+0000","last_clean":"2024-05-08T18:08:07.157727+0000","last_became_active":"2024-05-08T18:04:51.827127+0000","last_became_peered":"2024-05-08T18:04:51.827127+0000","last_unstale":"2024-05-08T18:08:07.157727+0000","last_undegraded":"2024-05-08T18:08:07.157727+0000","last_fullsized":"2024-05-08T18:08:07.157727+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:07.157606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:07.157606+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.157606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:37:53.105588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"29'14","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:03.477837+0000","last_change":"2024-05-08T18:09:03.477873+0000","last_active":"2024-05-08T18:09:03.477837+0000","last_peered":"2024-05-08T18:09:03.477837+0000","last_clean":"2024-05-08T18:09:03.477837+0000","last_became_active":"2024-05-08T18:04:53.843772+0000","last_became_peered":"2024-05-08T18:04:53.843772+0000","last_unstale":"2024-05-08T18:09:03.477837+0000","last_undegraded":"2024-05-08T18:09:03.477837+0000","last_fullsized":"2024-05-08T18:09:03.477837+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'14","last_scrub_stamp":"2024-05-08T18:09:03.477748+0000","last_deep_scrub":"29'14","last_deep_scrub_stamp":"2024-05-08T18:09:03.477748+0000","last_clean_scrub_stamp":"2024-05-08T18:09:03.477748+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:41:23.206546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"29'2","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:43.351032+0000","last_change":"2024-05-08T18:08:43.351032+0000","last_active":"2024-05-08T18:08:43.351032+0000","last_peered":"2024-05-08T18:08:43.351032+0000","last_clean":"2024-05-08T18:08:43.351032+0000","last_became_active":"2024-05-08T18:04:45.811517+0000","last_became_peered":"2024-05-08T18:04:45.811517+0000","last_unstale":"2024-05-08T18:08:43.351032+0000","last_undegraded":"2024-05-08T18:08:43.351032+0000","last_fullsized":"2024-05-08T18:08:43.351032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:43.350844+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:43.350844+0000","last_clean_scrub_stamp":"2024-05-08T18:08:43.350844+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:35.421755+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:54.459528+0000","last_change":"2024-05-08T18:08:54.459559+0000","last_active":"2024-05-08T18:08:54.459528+0000","last_peered":"2024-05-08T18:08:54.459528+0000","last_clean":"2024-05-08T18:08:54.459528+0000","last_became_active":"2024-05-08T18:04:43.834767+0000","last_became_peered":"2024-05-08T18:04:43.834767+0000","last_unstale":"2024-05-08T18:08:54.459528+0000","last_undegraded":"2024-05-08T18:08:54.459528+0000","last_fullsized":"2024-05-08T18:08:54.459528+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:54.459439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:54.459439+0000","last_clean_scrub_stamp":"2024-05-08T18:08:54.459439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:45.537704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0,2],"acting":[1,3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:24.285019+0000","last_change":"2024-05-08T18:08:24.285019+0000","last_active":"2024-05-08T18:08:24.285019+0000","last_peered":"2024-05-08T18:08:24.285019+0000","last_clean":"2024-05-08T18:08:24.285019+0000","last_became_active":"2024-05-08T18:04:47.820662+0000","last_became_peered":"2024-05-08T18:04:47.820662+0000","last_unstale":"2024-05-08T18:08:24.285019+0000","last_undegraded":"2024-05-08T18:08:24.285019+0000","last_fullsized":"2024-05-08T18:08:24.285019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:24.284899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:24.284899+0000","last_clean_scrub_stamp":"2024-05-08T18:08:24.284899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:02:49.472387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"29'46","reported_seq":104,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.901855+0000","last_change":"2024-05-08T18:08:10.901892+0000","last_active":"2024-05-08T18:08:10.901855+0000","last_peered":"2024-05-08T18:08:10.901855+0000","last_clean":"2024-05-08T18:08:10.901855+0000","last_became_active":"2024-05-08T18:04:49.830476+0000","last_became_peered":"2024-05-08T18:04:49.830476+0000","last_unstale":"2024-05-08T18:08:10.901855+0000","last_undegraded":"2024-05-08T18:08:10.901855+0000","last_fullsized":"2024-05-08T18:08:10.901855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'46","last_scrub_stamp":"2024-05-08T18:08:10.901767+0000","last_deep_scrub":"29'46","last_deep_scrub_stamp":"2024-05-08T18:08:10.901767+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.901767+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:07.184811+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.490196+0000","last_change":"2024-05-08T18:08:17.490220+0000","last_active":"2024-05-08T18:08:17.490196+0000","last_peered":"2024-05-08T18:08:17.490196+0000","last_clean":"2024-05-08T18:08:17.490196+0000","last_became_active":"2024-05-08T18:04:51.844912+0000","last_became_peered":"2024-05-08T18:04:51.844912+0000","last_unstale":"2024-05-08T18:08:17.490196+0000","last_undegraded":"2024-05-08T18:08:17.490196+0000","last_fullsized":"2024-05-08T18:08:17.490196+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:17.490110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:17.490110+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.490110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:06:17.806415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"29'2","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.043126+0000","last_change":"2024-05-08T18:04:53.847736+0000","last_active":"2024-05-08T18:07:54.043126+0000","last_peered":"2024-05-08T18:07:54.043126+0000","last_clean":"2024-05-08T18:07:54.043126+0000","last_became_active":"2024-05-08T18:04:53.847413+0000","last_became_peered":"2024-05-08T18:04:53.847413+0000","last_unstale":"2024-05-08T18:07:54.043126+0000","last_undegraded":"2024-05-08T18:07:54.043126+0000","last_fullsized":"2024-05-08T18:07:54.043126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"29'12","reported_seq":69,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.835658+0000","last_change":"2024-05-08T18:08:17.835711+0000","last_active":"2024-05-08T18:08:17.835658+0000","last_peered":"2024-05-08T18:08:17.835658+0000","last_clean":"2024-05-08T18:08:17.835658+0000","last_became_active":"2024-05-08T18:04:45.830329+0000","last_became_peered":"2024-05-08T18:04:45.830329+0000","last_unstale":"2024-05-08T18:08:17.835658+0000","last_undegraded":"2024-05-08T18:08:17.835658+0000","last_fullsized":"2024-05-08T18:08:17.835658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:17.835540+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:17.835540+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.835540+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:36.036099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:52.463640+0000","last_change":"2024-05-08T18:08:52.463670+0000","last_active":"2024-05-08T18:08:52.463640+0000","last_peered":"2024-05-08T18:08:52.463640+0000","last_clean":"2024-05-08T18:08:52.463640+0000","last_became_active":"2024-05-08T18:04:43.835227+0000","last_became_peered":"2024-05-08T18:04:43.835227+0000","last_unstale":"2024-05-08T18:08:52.463640+0000","last_undegraded":"2024-05-08T18:08:52.463640+0000","last_fullsized":"2024-05-08T18:08:52.463640+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:52.463521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:52.463521+0000","last_clean_scrub_stamp":"2024-05-08T18:08:52.463521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:05.463037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3,2],"acting":[1,0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:56.000672+0000","last_change":"2024-05-08T18:08:56.000704+0000","last_active":"2024-05-08T18:08:56.000672+0000","last_peered":"2024-05-08T18:08:56.000672+0000","last_clean":"2024-05-08T18:08:56.000672+0000","last_became_active":"2024-05-08T18:05:30.870155+0000","last_became_peered":"2024-05-08T18:05:30.870155+0000","last_unstale":"2024-05-08T18:08:56.000672+0000","last_undegraded":"2024-05-08T18:08:56.000672+0000","last_fullsized":"2024-05-08T18:08:56.000672+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:56.000581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:56.000581+0000","last_clean_scrub_stamp":"2024-05-08T18:08:56.000581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:19.118117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"29'39","reported_seq":96,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.536541+0000","last_change":"2024-05-08T18:08:12.536541+0000","last_active":"2024-05-08T18:08:12.536541+0000","last_peered":"2024-05-08T18:08:12.536541+0000","last_clean":"2024-05-08T18:08:12.536541+0000","last_became_active":"2024-05-08T18:04:49.832649+0000","last_became_peered":"2024-05-08T18:04:49.832649+0000","last_unstale":"2024-05-08T18:08:12.536541+0000","last_undegraded":"2024-05-08T18:08:12.536541+0000","last_fullsized":"2024-05-08T18:08:12.536541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'39","last_scrub_stamp":"2024-05-08T18:08:12.536412+0000","last_deep_scrub":"29'39","last_deep_scrub_stamp":"2024-05-08T18:08:12.536412+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.536412+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:31.365352+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.444675+0000","last_change":"2024-05-08T18:08:05.444675+0000","last_active":"2024-05-08T18:08:05.444675+0000","last_peered":"2024-05-08T18:08:05.444675+0000","last_clean":"2024-05-08T18:08:05.444675+0000","last_became_active":"2024-05-08T18:04:51.833421+0000","last_became_peered":"2024-05-08T18:04:51.833421+0000","last_unstale":"2024-05-08T18:08:05.444675+0000","last_undegraded":"2024-05-08T18:08:05.444675+0000","last_fullsized":"2024-05-08T18:08:05.444675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:05.444585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:05.444585+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.444585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:15.978668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"29'5","reported_seq":24,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747669+0000","last_change":"2024-05-08T18:04:53.848762+0000","last_active":"2024-05-08T18:07:13.747669+0000","last_peered":"2024-05-08T18:07:13.747669+0000","last_clean":"2024-05-08T18:07:13.747669+0000","last_became_active":"2024-05-08T18:04:53.848430+0000","last_became_peered":"2024-05-08T18:04:53.848430+0000","last_unstale":"2024-05-08T18:07:13.747669+0000","last_undegraded":"2024-05-08T18:07:13.747669+0000","last_fullsized":"2024-05-08T18:07:13.747669+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"29'10","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:37.437418+0000","last_change":"2024-05-08T18:08:37.437444+0000","last_active":"2024-05-08T18:08:37.437418+0000","last_peered":"2024-05-08T18:08:37.437418+0000","last_clean":"2024-05-08T18:08:37.437418+0000","last_became_active":"2024-05-08T18:04:45.806808+0000","last_became_peered":"2024-05-08T18:04:45.806808+0000","last_unstale":"2024-05-08T18:08:37.437418+0000","last_undegraded":"2024-05-08T18:08:37.437418+0000","last_fullsized":"2024-05-08T18:08:37.437418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:37.437332+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:37.437332+0000","last_clean_scrub_stamp":"2024-05-08T18:08:37.437332+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:55.637891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747763+0000","last_change":"2024-05-08T18:04:43.833094+0000","last_active":"2024-05-08T18:07:13.747763+0000","last_peered":"2024-05-08T18:07:13.747763+0000","last_clean":"2024-05-08T18:07:13.747763+0000","last_became_active":"2024-05-08T18:04:43.832147+0000","last_became_peered":"2024-05-08T18:04:43.832147+0000","last_unstale":"2024-05-08T18:07:13.747763+0000","last_undegraded":"2024-05-08T18:07:13.747763+0000","last_fullsized":"2024-05-08T18:07:13.747763+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.517382+0000","last_change":"2024-05-08T18:08:22.517404+0000","last_active":"2024-05-08T18:08:22.517382+0000","last_peered":"2024-05-08T18:08:22.517382+0000","last_clean":"2024-05-08T18:08:22.517382+0000","last_became_active":"2024-05-08T18:04:47.815219+0000","last_became_peered":"2024-05-08T18:04:47.815219+0000","last_unstale":"2024-05-08T18:08:22.517382+0000","last_undegraded":"2024-05-08T18:08:22.517382+0000","last_fullsized":"2024-05-08T18:08:22.517382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.517292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.517292+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.517292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:28.093920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"29'15","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:38.037896+0000","last_change":"2024-05-08T18:08:38.037896+0000","last_active":"2024-05-08T18:08:38.037896+0000","last_peered":"2024-05-08T18:08:38.037896+0000","last_clean":"2024-05-08T18:08:38.037896+0000","last_became_active":"2024-05-08T18:04:49.830793+0000","last_became_peered":"2024-05-08T18:04:49.830793+0000","last_unstale":"2024-05-08T18:08:38.037896+0000","last_undegraded":"2024-05-08T18:08:38.037896+0000","last_fullsized":"2024-05-08T18:08:38.037896+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:38.037761+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:38.037761+0000","last_clean_scrub_stamp":"2024-05-08T18:08:38.037761+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:37:36.379302+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"29'285","reported_seq":379,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.160257+0000","last_change":"2024-05-08T18:08:09.160257+0000","last_active":"2024-05-08T18:08:09.160257+0000","last_peered":"2024-05-08T18:08:09.160257+0000","last_clean":"2024-05-08T18:08:09.160257+0000","last_became_active":"2024-05-08T18:04:51.833866+0000","last_became_peered":"2024-05-08T18:04:51.833866+0000","last_unstale":"2024-05-08T18:08:09.160257+0000","last_undegraded":"2024-05-08T18:08:09.160257+0000","last_fullsized":"2024-05-08T18:08:09.160257+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:09.160141+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:09.160141+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.160141+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:29:48.216550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747522+0000","last_change":"2024-05-08T18:04:53.848804+0000","last_active":"2024-05-08T18:07:13.747522+0000","last_peered":"2024-05-08T18:07:13.747522+0000","last_clean":"2024-05-08T18:07:13.747522+0000","last_became_active":"2024-05-08T18:04:53.848321+0000","last_became_peered":"2024-05-08T18:04:53.848321+0000","last_unstale":"2024-05-08T18:07:13.747522+0000","last_undegraded":"2024-05-08T18:07:13.747522+0000","last_fullsized":"2024-05-08T18:07:13.747522+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"29'4","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:33.249650+0000","last_change":"2024-05-08T18:08:33.249675+0000","last_active":"2024-05-08T18:08:33.249650+0000","last_peered":"2024-05-08T18:08:33.249650+0000","last_clean":"2024-05-08T18:08:33.249650+0000","last_became_active":"2024-05-08T18:04:45.813424+0000","last_became_peered":"2024-05-08T18:04:45.813424+0000","last_unstale":"2024-05-08T18:08:33.249650+0000","last_undegraded":"2024-05-08T18:08:33.249650+0000","last_fullsized":"2024-05-08T18:08:33.249650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:33.249561+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:33.249561+0000","last_clean_scrub_stamp":"2024-05-08T18:08:33.249561+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:14.493205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:42.532785+0000","last_change":"2024-05-08T18:08:42.532811+0000","last_active":"2024-05-08T18:08:42.532785+0000","last_peered":"2024-05-08T18:08:42.532785+0000","last_clean":"2024-05-08T18:08:42.532785+0000","last_became_active":"2024-05-08T18:04:43.840502+0000","last_became_peered":"2024-05-08T18:04:43.840502+0000","last_unstale":"2024-05-08T18:08:42.532785+0000","last_undegraded":"2024-05-08T18:08:42.532785+0000","last_fullsized":"2024-05-08T18:08:42.532785+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:42.532676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:42.532676+0000","last_clean_scrub_stamp":"2024-05-08T18:08:42.532676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:06.240119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.519639+0000","last_change":"2024-05-08T18:08:18.519666+0000","last_active":"2024-05-08T18:08:18.519639+0000","last_peered":"2024-05-08T18:08:18.519639+0000","last_clean":"2024-05-08T18:08:18.519639+0000","last_became_active":"2024-05-08T18:04:47.818839+0000","last_became_peered":"2024-05-08T18:04:47.818839+0000","last_unstale":"2024-05-08T18:08:18.519639+0000","last_undegraded":"2024-05-08T18:08:18.519639+0000","last_fullsized":"2024-05-08T18:08:18.519639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:18.519542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:18.519542+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.519542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:14:21.272156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"29'46","reported_seq":104,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.495547+0000","last_change":"2024-05-08T18:08:12.495592+0000","last_active":"2024-05-08T18:08:12.495547+0000","last_peered":"2024-05-08T18:08:12.495547+0000","last_clean":"2024-05-08T18:08:12.495547+0000","last_became_active":"2024-05-08T18:04:49.824405+0000","last_became_peered":"2024-05-08T18:04:49.824405+0000","last_unstale":"2024-05-08T18:08:12.495547+0000","last_undegraded":"2024-05-08T18:08:12.495547+0000","last_fullsized":"2024-05-08T18:08:12.495547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'46","last_scrub_stamp":"2024-05-08T18:08:12.495454+0000","last_deep_scrub":"29'46","last_deep_scrub_stamp":"2024-05-08T18:08:12.495454+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.495454+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:48:04.037407+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.562545+0000","last_change":"2024-05-08T18:08:17.562569+0000","last_active":"2024-05-08T18:08:17.562545+0000","last_peered":"2024-05-08T18:08:17.562545+0000","last_clean":"2024-05-08T18:08:17.562545+0000","last_became_active":"2024-05-08T18:04:51.844757+0000","last_became_peered":"2024-05-08T18:04:51.844757+0000","last_unstale":"2024-05-08T18:08:17.562545+0000","last_undegraded":"2024-05-08T18:08:17.562545+0000","last_fullsized":"2024-05-08T18:08:17.562545+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:17.562441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:17.562441+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.562441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:50:03.644980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:33.515849+0000","last_change":"2024-05-08T18:08:33.515879+0000","last_active":"2024-05-08T18:08:33.515849+0000","last_peered":"2024-05-08T18:08:33.515849+0000","last_clean":"2024-05-08T18:08:33.515849+0000","last_became_active":"2024-05-08T18:04:53.840956+0000","last_became_peered":"2024-05-08T18:04:53.840956+0000","last_unstale":"2024-05-08T18:08:33.515849+0000","last_undegraded":"2024-05-08T18:08:33.515849+0000","last_fullsized":"2024-05-08T18:08:33.515849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:33.515757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:33.515757+0000","last_clean_scrub_stamp":"2024-05-08T18:08:33.515757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:29.798801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"29'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:28.283892+0000","last_change":"2024-05-08T18:08:28.283933+0000","last_active":"2024-05-08T18:08:28.283892+0000","last_peered":"2024-05-08T18:08:28.283892+0000","last_clean":"2024-05-08T18:08:28.283892+0000","last_became_active":"2024-05-08T18:04:45.806560+0000","last_became_peered":"2024-05-08T18:04:45.806560+0000","last_unstale":"2024-05-08T18:08:28.283892+0000","last_undegraded":"2024-05-08T18:08:28.283892+0000","last_fullsized":"2024-05-08T18:08:28.283892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:28.283774+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:28.283774+0000","last_clean_scrub_stamp":"2024-05-08T18:08:28.283774+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:55:22.065846+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747834+0000","last_change":"2024-05-08T18:04:43.845626+0000","last_active":"2024-05-08T18:07:13.747834+0000","last_peered":"2024-05-08T18:07:13.747834+0000","last_clean":"2024-05-08T18:07:13.747834+0000","last_became_active":"2024-05-08T18:04:43.845214+0000","last_became_peered":"2024-05-08T18:04:43.845214+0000","last_unstale":"2024-05-08T18:07:13.747834+0000","last_undegraded":"2024-05-08T18:07:13.747834+0000","last_fullsized":"2024-05-08T18:07:13.747834+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.286599+0000","last_change":"2024-05-08T18:08:23.286625+0000","last_active":"2024-05-08T18:08:23.286599+0000","last_peered":"2024-05-08T18:08:23.286599+0000","last_clean":"2024-05-08T18:08:23.286599+0000","last_became_active":"2024-05-08T18:04:47.822748+0000","last_became_peered":"2024-05-08T18:04:47.822748+0000","last_unstale":"2024-05-08T18:08:23.286599+0000","last_undegraded":"2024-05-08T18:08:23.286599+0000","last_fullsized":"2024-05-08T18:08:23.286599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:23.286509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:23.286509+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.286509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:17:35.464618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"29'37","reported_seq":92,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.576260+0000","last_change":"2024-05-08T18:08:14.576300+0000","last_active":"2024-05-08T18:08:14.576260+0000","last_peered":"2024-05-08T18:08:14.576260+0000","last_clean":"2024-05-08T18:08:14.576260+0000","last_became_active":"2024-05-08T18:04:49.831440+0000","last_became_peered":"2024-05-08T18:04:49.831440+0000","last_unstale":"2024-05-08T18:08:14.576260+0000","last_undegraded":"2024-05-08T18:08:14.576260+0000","last_fullsized":"2024-05-08T18:08:14.576260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'37","last_scrub_stamp":"2024-05-08T18:08:14.576150+0000","last_deep_scrub":"29'37","last_deep_scrub_stamp":"2024-05-08T18:08:14.576150+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.576150+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:06:14.892455+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:27.822646+0000","last_change":"2024-05-08T18:08:27.822675+0000","last_active":"2024-05-08T18:08:27.822646+0000","last_peered":"2024-05-08T18:08:27.822646+0000","last_clean":"2024-05-08T18:08:27.822646+0000","last_became_active":"2024-05-08T18:04:51.844125+0000","last_became_peered":"2024-05-08T18:04:51.844125+0000","last_unstale":"2024-05-08T18:08:27.822646+0000","last_undegraded":"2024-05-08T18:08:27.822646+0000","last_fullsized":"2024-05-08T18:08:27.822646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:27.822555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:27.822555+0000","last_clean_scrub_stamp":"2024-05-08T18:08:27.822555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:47:50.940092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.748549+0000","last_change":"2024-05-08T18:04:53.849962+0000","last_active":"2024-05-08T18:07:18.748549+0000","last_peered":"2024-05-08T18:07:18.748549+0000","last_clean":"2024-05-08T18:07:18.748549+0000","last_became_active":"2024-05-08T18:04:53.849514+0000","last_became_peered":"2024-05-08T18:04:53.849514+0000","last_unstale":"2024-05-08T18:07:18.748549+0000","last_undegraded":"2024-05-08T18:07:18.748549+0000","last_fullsized":"2024-05-08T18:07:18.748549+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"29'4","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:54.075447+0000","last_change":"2024-05-08T18:08:54.075473+0000","last_active":"2024-05-08T18:08:54.075447+0000","last_peered":"2024-05-08T18:08:54.075447+0000","last_clean":"2024-05-08T18:08:54.075447+0000","last_became_active":"2024-05-08T18:04:45.822204+0000","last_became_peered":"2024-05-08T18:04:45.822204+0000","last_unstale":"2024-05-08T18:08:54.075447+0000","last_undegraded":"2024-05-08T18:08:54.075447+0000","last_fullsized":"2024-05-08T18:08:54.075447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:54.075350+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:54.075350+0000","last_clean_scrub_stamp":"2024-05-08T18:08:54.075350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:04.754616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"29'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:48.395613+0000","last_change":"2024-05-08T18:08:48.395693+0000","last_active":"2024-05-08T18:08:48.395613+0000","last_peered":"2024-05-08T18:08:48.395613+0000","last_clean":"2024-05-08T18:08:48.395613+0000","last_became_active":"2024-05-08T18:04:43.834678+0000","last_became_peered":"2024-05-08T18:04:43.834678+0000","last_unstale":"2024-05-08T18:08:48.395613+0000","last_undegraded":"2024-05-08T18:08:48.395613+0000","last_fullsized":"2024-05-08T18:08:48.395613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:48.395512+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:48.395512+0000","last_clean_scrub_stamp":"2024-05-08T18:08:48.395512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:55.075303+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.501159+0000","last_change":"2024-05-08T18:08:16.501182+0000","last_active":"2024-05-08T18:08:16.501159+0000","last_peered":"2024-05-08T18:08:16.501159+0000","last_clean":"2024-05-08T18:08:16.501159+0000","last_became_active":"2024-05-08T18:04:47.820036+0000","last_became_peered":"2024-05-08T18:04:47.820036+0000","last_unstale":"2024-05-08T18:08:16.501159+0000","last_undegraded":"2024-05-08T18:08:16.501159+0000","last_fullsized":"2024-05-08T18:08:16.501159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:16.501065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:16.501065+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.501065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:49.649348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"29'32","reported_seq":85,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.857112+0000","last_change":"2024-05-08T18:08:20.857184+0000","last_active":"2024-05-08T18:08:20.857112+0000","last_peered":"2024-05-08T18:08:20.857112+0000","last_clean":"2024-05-08T18:08:20.857112+0000","last_became_active":"2024-05-08T18:04:49.830882+0000","last_became_peered":"2024-05-08T18:04:49.830882+0000","last_unstale":"2024-05-08T18:08:20.857112+0000","last_undegraded":"2024-05-08T18:08:20.857112+0000","last_fullsized":"2024-05-08T18:08:20.857112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'32","last_scrub_stamp":"2024-05-08T18:08:20.857010+0000","last_deep_scrub":"29'32","last_deep_scrub_stamp":"2024-05-08T18:08:20.857010+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.857010+0000","objects_scrubbed":2,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:06.682042+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":49,"num_write":14,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:29.815248+0000","last_change":"2024-05-08T18:08:29.815276+0000","last_active":"2024-05-08T18:08:29.815248+0000","last_peered":"2024-05-08T18:08:29.815248+0000","last_clean":"2024-05-08T18:08:29.815248+0000","last_became_active":"2024-05-08T18:04:51.842734+0000","last_became_peered":"2024-05-08T18:04:51.842734+0000","last_unstale":"2024-05-08T18:08:29.815248+0000","last_undegraded":"2024-05-08T18:08:29.815248+0000","last_fullsized":"2024-05-08T18:08:29.815248+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:29.815133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:29.815133+0000","last_clean_scrub_stamp":"2024-05-08T18:08:29.815133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:51:13.451798+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'10","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:57.037492+0000","last_change":"2024-05-08T18:08:57.037521+0000","last_active":"2024-05-08T18:08:57.037492+0000","last_peered":"2024-05-08T18:08:57.037492+0000","last_clean":"2024-05-08T18:08:57.037492+0000","last_became_active":"2024-05-08T18:04:53.844891+0000","last_became_peered":"2024-05-08T18:04:53.844891+0000","last_unstale":"2024-05-08T18:08:57.037492+0000","last_undegraded":"2024-05-08T18:08:57.037492+0000","last_fullsized":"2024-05-08T18:08:57.037492+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:57.037400+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:57.037400+0000","last_clean_scrub_stamp":"2024-05-08T18:08:57.037400+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:33:31.062988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"29'2","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:27.291722+0000","last_change":"2024-05-08T18:08:27.291750+0000","last_active":"2024-05-08T18:08:27.291722+0000","last_peered":"2024-05-08T18:08:27.291722+0000","last_clean":"2024-05-08T18:08:27.291722+0000","last_became_active":"2024-05-08T18:04:45.808936+0000","last_became_peered":"2024-05-08T18:04:45.808936+0000","last_unstale":"2024-05-08T18:08:27.291722+0000","last_undegraded":"2024-05-08T18:08:27.291722+0000","last_fullsized":"2024-05-08T18:08:27.291722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:27.291616+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:27.291616+0000","last_clean_scrub_stamp":"2024-05-08T18:08:27.291616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:50:13.374155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:58.481285+0000","last_change":"2024-05-08T18:08:58.481320+0000","last_active":"2024-05-08T18:08:58.481285+0000","last_peered":"2024-05-08T18:08:58.481285+0000","last_clean":"2024-05-08T18:08:58.481285+0000","last_became_active":"2024-05-08T18:04:43.837070+0000","last_became_peered":"2024-05-08T18:04:43.837070+0000","last_unstale":"2024-05-08T18:08:58.481285+0000","last_undegraded":"2024-05-08T18:08:58.481285+0000","last_fullsized":"2024-05-08T18:08:58.481285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:58.481195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:58.481195+0000","last_clean_scrub_stamp":"2024-05-08T18:08:58.481195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:16.399630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"29'3","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.481725+0000","last_change":"2024-05-08T18:08:15.481759+0000","last_active":"2024-05-08T18:08:15.481725+0000","last_peered":"2024-05-08T18:08:15.481725+0000","last_clean":"2024-05-08T18:08:15.481725+0000","last_became_active":"2024-05-08T18:04:47.824172+0000","last_became_peered":"2024-05-08T18:04:47.824172+0000","last_unstale":"2024-05-08T18:08:15.481725+0000","last_undegraded":"2024-05-08T18:08:15.481725+0000","last_fullsized":"2024-05-08T18:08:15.481725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3","last_scrub_stamp":"2024-05-08T18:08:15.481627+0000","last_deep_scrub":"29'3","last_deep_scrub_stamp":"2024-05-08T18:08:15.481627+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.481627+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:31:00.387286+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"29'40","reported_seq":97,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:33.898989+0000","last_change":"2024-05-08T18:08:33.899046+0000","last_active":"2024-05-08T18:08:33.898989+0000","last_peered":"2024-05-08T18:08:33.898989+0000","last_clean":"2024-05-08T18:08:33.898989+0000","last_became_active":"2024-05-08T18:04:49.831803+0000","last_became_peered":"2024-05-08T18:04:49.831803+0000","last_unstale":"2024-05-08T18:08:33.898989+0000","last_undegraded":"2024-05-08T18:08:33.898989+0000","last_fullsized":"2024-05-08T18:08:33.898989+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'40","last_scrub_stamp":"2024-05-08T18:08:33.898897+0000","last_deep_scrub":"29'40","last_deep_scrub_stamp":"2024-05-08T18:08:33.898897+0000","last_clean_scrub_stamp":"2024-05-08T18:08:33.898897+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:05:09.280810+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1270,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:25.577300+0000","last_change":"2024-05-08T18:08:25.577348+0000","last_active":"2024-05-08T18:08:25.577300+0000","last_peered":"2024-05-08T18:08:25.577300+0000","last_clean":"2024-05-08T18:08:25.577300+0000","last_became_active":"2024-05-08T18:04:51.846225+0000","last_became_peered":"2024-05-08T18:04:51.846225+0000","last_unstale":"2024-05-08T18:08:25.577300+0000","last_undegraded":"2024-05-08T18:08:25.577300+0000","last_fullsized":"2024-05-08T18:08:25.577300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:25.577162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:25.577162+0000","last_clean_scrub_stamp":"2024-05-08T18:08:25.577162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:15.650574+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"29'4","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:53.121061+0000","last_change":"2024-05-08T18:08:53.121087+0000","last_active":"2024-05-08T18:08:53.121061+0000","last_peered":"2024-05-08T18:08:53.121061+0000","last_clean":"2024-05-08T18:08:53.121061+0000","last_became_active":"2024-05-08T18:04:53.842429+0000","last_became_peered":"2024-05-08T18:04:53.842429+0000","last_unstale":"2024-05-08T18:08:53.121061+0000","last_undegraded":"2024-05-08T18:08:53.121061+0000","last_fullsized":"2024-05-08T18:08:53.121061+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:53.120970+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:53.120970+0000","last_clean_scrub_stamp":"2024-05-08T18:08:53.120970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:15.271281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"27'2","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:32.506118+0000","last_change":"2024-05-08T18:08:32.506149+0000","last_active":"2024-05-08T18:08:32.506118+0000","last_peered":"2024-05-08T18:08:32.506118+0000","last_clean":"2024-05-08T18:08:32.506118+0000","last_became_active":"2024-05-08T18:04:45.818161+0000","last_became_peered":"2024-05-08T18:04:45.818161+0000","last_unstale":"2024-05-08T18:08:32.506118+0000","last_undegraded":"2024-05-08T18:08:32.506118+0000","last_fullsized":"2024-05-08T18:08:32.506118+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:32.506023+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:32.506023+0000","last_clean_scrub_stamp":"2024-05-08T18:08:32.506023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:20.670584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:38.182450+0000","last_change":"2024-05-08T18:08:38.182450+0000","last_active":"2024-05-08T18:08:38.182450+0000","last_peered":"2024-05-08T18:08:38.182450+0000","last_clean":"2024-05-08T18:08:38.182450+0000","last_became_active":"2024-05-08T18:04:43.842483+0000","last_became_peered":"2024-05-08T18:04:43.842483+0000","last_unstale":"2024-05-08T18:08:38.182450+0000","last_undegraded":"2024-05-08T18:08:38.182450+0000","last_fullsized":"2024-05-08T18:08:38.182450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:38.182336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:38.182336+0000","last_clean_scrub_stamp":"2024-05-08T18:08:38.182336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:54:29.772068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"29'36","reported_seq":115,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:41.377102+0000","last_change":"2024-05-08T18:08:41.377160+0000","last_active":"2024-05-08T18:08:41.377102+0000","last_peered":"2024-05-08T18:08:41.377102+0000","last_clean":"2024-05-08T18:08:41.377102+0000","last_became_active":"2024-05-08T18:04:45.807847+0000","last_became_peered":"2024-05-08T18:04:45.807847+0000","last_unstale":"2024-05-08T18:08:41.377102+0000","last_undegraded":"2024-05-08T18:08:41.377102+0000","last_fullsized":"2024-05-08T18:08:41.377102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:08:41.376990+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:08:41.376990+0000","last_clean_scrub_stamp":"2024-05-08T18:08:41.376990+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:19:16.885027+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":44,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:36.472685+0000","last_change":"2024-05-08T18:08:36.472685+0000","last_active":"2024-05-08T18:08:36.472685+0000","last_peered":"2024-05-08T18:08:36.472685+0000","last_clean":"2024-05-08T18:08:36.472685+0000","last_became_active":"2024-05-08T18:04:43.836045+0000","last_became_peered":"2024-05-08T18:04:43.836045+0000","last_unstale":"2024-05-08T18:08:36.472685+0000","last_undegraded":"2024-05-08T18:08:36.472685+0000","last_fullsized":"2024-05-08T18:08:36.472685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:36.472507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:36.472507+0000","last_clean_scrub_stamp":"2024-05-08T18:08:36.472507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:13.020912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3,2],"acting":[0,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"29'12","reported_seq":78,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:47.107065+0000","last_change":"2024-05-08T18:08:47.107135+0000","last_active":"2024-05-08T18:08:47.107065+0000","last_peered":"2024-05-08T18:08:47.107065+0000","last_clean":"2024-05-08T18:08:47.107065+0000","last_became_active":"2024-05-08T18:04:45.825506+0000","last_became_peered":"2024-05-08T18:04:45.825506+0000","last_unstale":"2024-05-08T18:08:47.107065+0000","last_undegraded":"2024-05-08T18:08:47.107065+0000","last_fullsized":"2024-05-08T18:08:47.107065+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:47.106962+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:47.106962+0000","last_clean_scrub_stamp":"2024-05-08T18:08:47.106962+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:30.261074+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":46,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"29'6","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.081032+0000","last_change":"2024-05-08T18:04:43.832854+0000","last_active":"2024-05-08T18:08:05.081032+0000","last_peered":"2024-05-08T18:08:05.081032+0000","last_clean":"2024-05-08T18:08:05.081032+0000","last_became_active":"2024-05-08T18:04:43.832406+0000","last_became_peered":"2024-05-08T18:04:43.832406+0000","last_unstale":"2024-05-08T18:08:05.081032+0000","last_undegraded":"2024-05-08T18:08:05.081032+0000","last_fullsized":"2024-05-08T18:08:05.081032+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":8,"num_write":30,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"29'2","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:35.470944+0000","last_change":"2024-05-08T18:08:35.470976+0000","last_active":"2024-05-08T18:08:35.470944+0000","last_peered":"2024-05-08T18:08:35.470944+0000","last_clean":"2024-05-08T18:08:35.470944+0000","last_became_active":"2024-05-08T18:04:45.806646+0000","last_became_peered":"2024-05-08T18:04:45.806646+0000","last_unstale":"2024-05-08T18:08:35.470944+0000","last_undegraded":"2024-05-08T18:08:35.470944+0000","last_fullsized":"2024-05-08T18:08:35.470944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:35.470848+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:35.470848+0000","last_clean_scrub_stamp":"2024-05-08T18:08:35.470848+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:55.258041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"29'6","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:40.377297+0000","last_change":"2024-05-08T18:08:40.377297+0000","last_active":"2024-05-08T18:08:40.377297+0000","last_peered":"2024-05-08T18:08:40.377297+0000","last_clean":"2024-05-08T18:08:40.377297+0000","last_became_active":"2024-05-08T18:04:43.835163+0000","last_became_peered":"2024-05-08T18:04:43.835163+0000","last_unstale":"2024-05-08T18:08:40.377297+0000","last_undegraded":"2024-05-08T18:08:40.377297+0000","last_fullsized":"2024-05-08T18:08:40.377297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:40.377108+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:40.377108+0000","last_clean_scrub_stamp":"2024-05-08T18:08:40.377108+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:26.202140+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":4746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,1],"acting":[0,3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"29'2","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:34.514234+0000","last_change":"2024-05-08T18:08:34.514234+0000","last_active":"2024-05-08T18:08:34.514234+0000","last_peered":"2024-05-08T18:08:34.514234+0000","last_clean":"2024-05-08T18:08:34.514234+0000","last_became_active":"2024-05-08T18:04:45.816391+0000","last_became_peered":"2024-05-08T18:04:45.816391+0000","last_unstale":"2024-05-08T18:08:34.514234+0000","last_undegraded":"2024-05-08T18:08:34.514234+0000","last_fullsized":"2024-05-08T18:08:34.514234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:34.514122+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:34.514122+0000","last_clean_scrub_stamp":"2024-05-08T18:08:34.514122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:11.960319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'4","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:42.343533+0000","last_change":"2024-05-08T18:08:42.343533+0000","last_active":"2024-05-08T18:08:42.343533+0000","last_peered":"2024-05-08T18:08:42.343533+0000","last_clean":"2024-05-08T18:08:42.343533+0000","last_became_active":"2024-05-08T18:04:43.835268+0000","last_became_peered":"2024-05-08T18:04:43.835268+0000","last_unstale":"2024-05-08T18:08:42.343533+0000","last_undegraded":"2024-05-08T18:08:42.343533+0000","last_fullsized":"2024-05-08T18:08:42.343533+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-08T18:08:42.343414+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-08T18:08:42.343414+0000","last_clean_scrub_stamp":"2024-05-08T18:08:42.343414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:24.879935+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"29'12","reported_seq":87,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:28.537351+0000","last_change":"2024-05-08T18:08:28.537351+0000","last_active":"2024-05-08T18:08:28.537351+0000","last_peered":"2024-05-08T18:08:28.537351+0000","last_clean":"2024-05-08T18:08:28.537351+0000","last_became_active":"2024-05-08T18:04:45.819699+0000","last_became_peered":"2024-05-08T18:04:45.819699+0000","last_unstale":"2024-05-08T18:08:28.537351+0000","last_undegraded":"2024-05-08T18:08:28.537351+0000","last_fullsized":"2024-05-08T18:08:28.537351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:28.537227+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:28.537227+0000","last_clean_scrub_stamp":"2024-05-08T18:08:28.537227+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:18.610639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:57.463007+0000","last_change":"2024-05-08T18:08:57.463007+0000","last_active":"2024-05-08T18:08:57.463007+0000","last_peered":"2024-05-08T18:08:57.463007+0000","last_clean":"2024-05-08T18:08:57.463007+0000","last_became_active":"2024-05-08T18:04:43.836735+0000","last_became_peered":"2024-05-08T18:04:43.836735+0000","last_unstale":"2024-05-08T18:08:57.463007+0000","last_undegraded":"2024-05-08T18:08:57.463007+0000","last_fullsized":"2024-05-08T18:08:57.463007+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:57.462845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:57.462845+0000","last_clean_scrub_stamp":"2024-05-08T18:08:57.462845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:44.500274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"29'14","reported_seq":80,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:47.377267+0000","last_change":"2024-05-08T18:08:47.377267+0000","last_active":"2024-05-08T18:08:47.377267+0000","last_peered":"2024-05-08T18:08:47.377267+0000","last_clean":"2024-05-08T18:08:47.377267+0000","last_became_active":"2024-05-08T18:04:45.811953+0000","last_became_peered":"2024-05-08T18:04:45.811953+0000","last_unstale":"2024-05-08T18:08:47.377267+0000","last_undegraded":"2024-05-08T18:08:47.377267+0000","last_fullsized":"2024-05-08T18:08:47.377267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'14","last_scrub_stamp":"2024-05-08T18:08:47.377155+0000","last_deep_scrub":"29'14","last_deep_scrub_stamp":"2024-05-08T18:08:47.377155+0000","last_clean_scrub_stamp":"2024-05-08T18:08:47.377155+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:10.494691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:34.213484+0000","last_change":"2024-05-08T18:08:34.213484+0000","last_active":"2024-05-08T18:08:34.213484+0000","last_peered":"2024-05-08T18:08:34.213484+0000","last_clean":"2024-05-08T18:08:34.213484+0000","last_became_active":"2024-05-08T18:04:43.842179+0000","last_became_peered":"2024-05-08T18:04:43.842179+0000","last_unstale":"2024-05-08T18:08:34.213484+0000","last_undegraded":"2024-05-08T18:08:34.213484+0000","last_fullsized":"2024-05-08T18:08:34.213484+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:34.213370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:34.213370+0000","last_clean_scrub_stamp":"2024-05-08T18:08:34.213370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:13.048212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,0],"acting":[2,3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"29'8","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:59.502471+0000","last_change":"2024-05-08T18:08:59.502501+0000","last_active":"2024-05-08T18:08:59.502471+0000","last_peered":"2024-05-08T18:08:59.502471+0000","last_clean":"2024-05-08T18:08:59.502471+0000","last_became_active":"2024-05-08T18:04:45.812637+0000","last_became_peered":"2024-05-08T18:04:45.812637+0000","last_unstale":"2024-05-08T18:08:59.502471+0000","last_undegraded":"2024-05-08T18:08:59.502471+0000","last_fullsized":"2024-05-08T18:08:59.502471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:59.502380+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:59.502380+0000","last_clean_scrub_stamp":"2024-05-08T18:08:59.502380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:08:00.350733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"29'6","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:58.521154+0000","last_change":"2024-05-08T18:08:58.521154+0000","last_active":"2024-05-08T18:08:58.521154+0000","last_peered":"2024-05-08T18:08:58.521154+0000","last_clean":"2024-05-08T18:08:58.521154+0000","last_became_active":"2024-05-08T18:04:43.834491+0000","last_became_peered":"2024-05-08T18:04:43.834491+0000","last_unstale":"2024-05-08T18:08:58.521154+0000","last_undegraded":"2024-05-08T18:08:58.521154+0000","last_fullsized":"2024-05-08T18:08:58.521154+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:58.520991+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:58.520991+0000","last_clean_scrub_stamp":"2024-05-08T18:08:58.520991+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:32:07.150914+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"29'8","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:34.558842+0000","last_change":"2024-05-08T18:08:34.558889+0000","last_active":"2024-05-08T18:08:34.558842+0000","last_peered":"2024-05-08T18:08:34.558842+0000","last_clean":"2024-05-08T18:08:34.558842+0000","last_became_active":"2024-05-08T18:04:45.815677+0000","last_became_peered":"2024-05-08T18:04:45.815677+0000","last_unstale":"2024-05-08T18:08:34.558842+0000","last_undegraded":"2024-05-08T18:08:34.558842+0000","last_fullsized":"2024-05-08T18:08:34.558842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:34.558754+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:34.558754+0000","last_clean_scrub_stamp":"2024-05-08T18:08:34.558754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:30.175682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.081047+0000","last_change":"2024-05-08T18:04:43.846054+0000","last_active":"2024-05-08T18:08:05.081047+0000","last_peered":"2024-05-08T18:08:05.081047+0000","last_clean":"2024-05-08T18:08:05.081047+0000","last_became_active":"2024-05-08T18:04:43.843826+0000","last_became_peered":"2024-05-08T18:04:43.843826+0000","last_unstale":"2024-05-08T18:08:05.081047+0000","last_undegraded":"2024-05-08T18:08:05.081047+0000","last_fullsized":"2024-05-08T18:08:05.081047+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"29'6","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:43.071536+0000","last_change":"2024-05-08T18:08:43.071567+0000","last_active":"2024-05-08T18:08:43.071536+0000","last_peered":"2024-05-08T18:08:43.071536+0000","last_clean":"2024-05-08T18:08:43.071536+0000","last_became_active":"2024-05-08T18:04:45.827252+0000","last_became_peered":"2024-05-08T18:04:45.827252+0000","last_unstale":"2024-05-08T18:08:43.071536+0000","last_undegraded":"2024-05-08T18:08:43.071536+0000","last_fullsized":"2024-05-08T18:08:43.071536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:43.071451+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:43.071451+0000","last_clean_scrub_stamp":"2024-05-08T18:08:43.071451+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:41:02.800248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:55.463777+0000","last_change":"2024-05-08T18:08:55.463831+0000","last_active":"2024-05-08T18:08:55.463777+0000","last_peered":"2024-05-08T18:08:55.463777+0000","last_clean":"2024-05-08T18:08:55.463777+0000","last_became_active":"2024-05-08T18:04:43.820360+0000","last_became_peered":"2024-05-08T18:04:43.820360+0000","last_unstale":"2024-05-08T18:08:55.463777+0000","last_undegraded":"2024-05-08T18:08:55.463777+0000","last_fullsized":"2024-05-08T18:08:55.463777+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:55.463676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:55.463676+0000","last_clean_scrub_stamp":"2024-05-08T18:08:55.463676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:43:05.453001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:31.445820+0000","last_change":"2024-05-08T18:08:31.445845+0000","last_active":"2024-05-08T18:08:31.445820+0000","last_peered":"2024-05-08T18:08:31.445820+0000","last_clean":"2024-05-08T18:08:31.445820+0000","last_became_active":"2024-05-08T18:04:45.823385+0000","last_became_peered":"2024-05-08T18:04:45.823385+0000","last_unstale":"2024-05-08T18:08:31.445820+0000","last_undegraded":"2024-05-08T18:08:31.445820+0000","last_fullsized":"2024-05-08T18:08:31.445820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:31.445729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:31.445729+0000","last_clean_scrub_stamp":"2024-05-08T18:08:31.445729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:27.062658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'6","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:55.454002+0000","last_change":"2024-05-08T18:08:55.454031+0000","last_active":"2024-05-08T18:08:55.454002+0000","last_peered":"2024-05-08T18:08:55.454002+0000","last_clean":"2024-05-08T18:08:55.454002+0000","last_became_active":"2024-05-08T18:04:43.835838+0000","last_became_peered":"2024-05-08T18:04:43.835838+0000","last_unstale":"2024-05-08T18:08:55.454002+0000","last_undegraded":"2024-05-08T18:08:55.454002+0000","last_fullsized":"2024-05-08T18:08:55.454002+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:55.453911+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:55.453911+0000","last_clean_scrub_stamp":"2024-05-08T18:08:55.453911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:47:45.526995+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8,"num_write":30,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"29'10","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:41.515456+0000","last_change":"2024-05-08T18:08:41.515484+0000","last_active":"2024-05-08T18:08:41.515456+0000","last_peered":"2024-05-08T18:08:41.515456+0000","last_clean":"2024-05-08T18:08:41.515456+0000","last_became_active":"2024-05-08T18:04:45.810703+0000","last_became_peered":"2024-05-08T18:04:45.810703+0000","last_unstale":"2024-05-08T18:08:41.515456+0000","last_undegraded":"2024-05-08T18:08:41.515456+0000","last_fullsized":"2024-05-08T18:08:41.515456+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:41.515361+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:41.515361+0000","last_clean_scrub_stamp":"2024-05-08T18:08:41.515361+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:54:04.536630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"29'4","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:53.458279+0000","last_change":"2024-05-08T18:08:53.458323+0000","last_active":"2024-05-08T18:08:53.458279+0000","last_peered":"2024-05-08T18:08:53.458279+0000","last_clean":"2024-05-08T18:08:53.458279+0000","last_became_active":"2024-05-08T18:04:43.835518+0000","last_became_peered":"2024-05-08T18:04:43.835518+0000","last_unstale":"2024-05-08T18:08:53.458279+0000","last_undegraded":"2024-05-08T18:08:53.458279+0000","last_fullsized":"2024-05-08T18:08:53.458279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:53.458190+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:53.458190+0000","last_clean_scrub_stamp":"2024-05-08T18:08:53.458190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:31:55.295395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,3],"acting":[1,2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"29'4","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:26.522634+0000","last_change":"2024-05-08T18:08:26.522702+0000","last_active":"2024-05-08T18:08:26.522634+0000","last_peered":"2024-05-08T18:08:26.522634+0000","last_clean":"2024-05-08T18:08:26.522634+0000","last_became_active":"2024-05-08T18:04:45.820746+0000","last_became_peered":"2024-05-08T18:04:45.820746+0000","last_unstale":"2024-05-08T18:08:26.522634+0000","last_undegraded":"2024-05-08T18:08:26.522634+0000","last_fullsized":"2024-05-08T18:08:26.522634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:26.522525+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:26.522525+0000","last_clean_scrub_stamp":"2024-05-08T18:08:26.522525+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:07.766169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:58.037319+0000","last_change":"2024-05-08T18:08:58.037345+0000","last_active":"2024-05-08T18:08:58.037319+0000","last_peered":"2024-05-08T18:08:58.037319+0000","last_clean":"2024-05-08T18:08:58.037319+0000","last_became_active":"2024-05-08T18:04:43.844404+0000","last_became_peered":"2024-05-08T18:04:43.844404+0000","last_unstale":"2024-05-08T18:08:58.037319+0000","last_undegraded":"2024-05-08T18:08:58.037319+0000","last_fullsized":"2024-05-08T18:08:58.037319+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:58.037225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:58.037225+0000","last_clean_scrub_stamp":"2024-05-08T18:08:58.037225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:11:08.420812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"29'4","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:39.501832+0000","last_change":"2024-05-08T18:08:39.501858+0000","last_active":"2024-05-08T18:08:39.501832+0000","last_peered":"2024-05-08T18:08:39.501832+0000","last_clean":"2024-05-08T18:08:39.501832+0000","last_became_active":"2024-05-08T18:04:45.809271+0000","last_became_peered":"2024-05-08T18:04:45.809271+0000","last_unstale":"2024-05-08T18:08:39.501832+0000","last_undegraded":"2024-05-08T18:08:39.501832+0000","last_fullsized":"2024-05-08T18:08:39.501832+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:39.501726+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:39.501726+0000","last_clean_scrub_stamp":"2024-05-08T18:08:39.501726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:16.050132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:59.036072+0000","last_change":"2024-05-08T18:08:59.036095+0000","last_active":"2024-05-08T18:08:59.036072+0000","last_peered":"2024-05-08T18:08:59.036072+0000","last_clean":"2024-05-08T18:08:59.036072+0000","last_became_active":"2024-05-08T18:04:43.838125+0000","last_became_peered":"2024-05-08T18:04:43.838125+0000","last_unstale":"2024-05-08T18:08:59.036072+0000","last_undegraded":"2024-05-08T18:08:59.036072+0000","last_fullsized":"2024-05-08T18:08:59.036072+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:59.035976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:59.035976+0000","last_clean_scrub_stamp":"2024-05-08T18:08:59.035976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:24:17.825578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"29'15","reported_seq":84,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.868964+0000","last_change":"2024-05-08T18:08:21.868994+0000","last_active":"2024-05-08T18:08:21.868964+0000","last_peered":"2024-05-08T18:08:21.868964+0000","last_clean":"2024-05-08T18:08:21.868964+0000","last_became_active":"2024-05-08T18:04:45.828488+0000","last_became_peered":"2024-05-08T18:04:45.828488+0000","last_unstale":"2024-05-08T18:08:21.868964+0000","last_undegraded":"2024-05-08T18:08:21.868964+0000","last_fullsized":"2024-05-08T18:08:21.868964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:21.868870+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:21.868870+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.868870+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:18:57.376906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:32.274959+0000","last_change":"2024-05-08T18:08:32.274984+0000","last_active":"2024-05-08T18:08:32.274959+0000","last_peered":"2024-05-08T18:08:32.274959+0000","last_clean":"2024-05-08T18:08:32.274959+0000","last_became_active":"2024-05-08T18:04:43.842670+0000","last_became_peered":"2024-05-08T18:04:43.842670+0000","last_unstale":"2024-05-08T18:08:32.274959+0000","last_undegraded":"2024-05-08T18:08:32.274959+0000","last_fullsized":"2024-05-08T18:08:32.274959+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:32.274865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:32.274865+0000","last_clean_scrub_stamp":"2024-05-08T18:08:32.274865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:55.489696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"29'2","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:45.484718+0000","last_change":"2024-05-08T18:08:45.484747+0000","last_active":"2024-05-08T18:08:45.484718+0000","last_peered":"2024-05-08T18:08:45.484718+0000","last_clean":"2024-05-08T18:08:45.484718+0000","last_became_active":"2024-05-08T18:04:45.810562+0000","last_became_peered":"2024-05-08T18:04:45.810562+0000","last_unstale":"2024-05-08T18:08:45.484718+0000","last_undegraded":"2024-05-08T18:08:45.484718+0000","last_fullsized":"2024-05-08T18:08:45.484718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:45.484623+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:45.484623+0000","last_clean_scrub_stamp":"2024-05-08T18:08:45.484623+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:05:22.042022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:02.000655+0000","last_change":"2024-05-08T18:09:02.000689+0000","last_active":"2024-05-08T18:09:02.000655+0000","last_peered":"2024-05-08T18:09:02.000655+0000","last_clean":"2024-05-08T18:09:02.000655+0000","last_became_active":"2024-05-08T18:04:43.835943+0000","last_became_peered":"2024-05-08T18:04:43.835943+0000","last_unstale":"2024-05-08T18:09:02.000655+0000","last_undegraded":"2024-05-08T18:09:02.000655+0000","last_fullsized":"2024-05-08T18:09:02.000655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:09:02.000540+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:09:02.000540+0000","last_clean_scrub_stamp":"2024-05-08T18:09:02.000540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:57.173467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:03.745822+0000","last_change":"2024-05-08T18:04:43.836378+0000","last_active":"2024-05-08T18:07:03.745822+0000","last_peered":"2024-05-08T18:07:03.745822+0000","last_clean":"2024-05-08T18:07:03.745822+0000","last_became_active":"2024-05-08T18:04:43.836007+0000","last_became_peered":"2024-05-08T18:04:43.836007+0000","last_unstale":"2024-05-08T18:07:03.745822+0000","last_undegraded":"2024-05-08T18:07:03.745822+0000","last_fullsized":"2024-05-08T18:07:03.745822+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:42.782385+0000","last_clean_scrub_stamp":"2024-05-08T18:04:42.782385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"29'8","reported_seq":71,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:49.357517+0000","last_change":"2024-05-08T18:08:49.357546+0000","last_active":"2024-05-08T18:08:49.357517+0000","last_peered":"2024-05-08T18:08:49.357517+0000","last_clean":"2024-05-08T18:08:49.357517+0000","last_became_active":"2024-05-08T18:04:45.811648+0000","last_became_peered":"2024-05-08T18:04:45.811648+0000","last_unstale":"2024-05-08T18:08:49.357517+0000","last_undegraded":"2024-05-08T18:08:49.357517+0000","last_fullsized":"2024-05-08T18:08:49.357517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:49.357428+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:49.357428+0000","last_clean_scrub_stamp":"2024-05-08T18:08:49.357428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:51:32.994094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":31,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"29'6","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:38.419488+0000","last_change":"2024-05-08T18:08:38.419521+0000","last_active":"2024-05-08T18:08:38.419488+0000","last_peered":"2024-05-08T18:08:38.419488+0000","last_clean":"2024-05-08T18:08:38.419488+0000","last_became_active":"2024-05-08T18:04:43.835712+0000","last_became_peered":"2024-05-08T18:04:43.835712+0000","last_unstale":"2024-05-08T18:08:38.419488+0000","last_undegraded":"2024-05-08T18:08:38.419488+0000","last_fullsized":"2024-05-08T18:08:38.419488+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:38.419396+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:38.419396+0000","last_clean_scrub_stamp":"2024-05-08T18:08:38.419396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:54:01.440665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":22,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"29'10","reported_seq":64,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:47.497538+0000","last_change":"2024-05-08T18:08:47.497589+0000","last_active":"2024-05-08T18:08:47.497538+0000","last_peered":"2024-05-08T18:08:47.497538+0000","last_clean":"2024-05-08T18:08:47.497538+0000","last_became_active":"2024-05-08T18:04:45.811921+0000","last_became_peered":"2024-05-08T18:04:45.811921+0000","last_unstale":"2024-05-08T18:08:47.497538+0000","last_undegraded":"2024-05-08T18:08:47.497538+0000","last_fullsized":"2024-05-08T18:08:47.497538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:47.497436+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:47.497436+0000","last_clean_scrub_stamp":"2024-05-08T18:08:47.497436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:06:35.185255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":349,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":272,"ondisk_log_size":272,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2281,"ondisk_log_size":2281,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1600,"num_read_kb":1397,"num_write":597,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3000,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1527,"ondisk_log_size":1527,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":483,"ondisk_log_size":483,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":145,"num_read_kb":87,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":256,"acting":256,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705720,"num_pgs":179,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":438272,"data_stored":135643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705721,"num_pgs":153,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44604,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94327236,"statfs":{"total":96636764160,"available":96591089664,"internally_reserved":0,"allocated":970752,"data_stored":724153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":44693957},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705721,"num_pgs":157,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40464,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331376,"statfs":{"total":96636764160,"available":96595329024,"internally_reserved":0,"allocated":991232,"data_stored":724641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705721,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":409600,"data_stored":135609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T18:09:06.603 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.604 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.604 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.604 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.604 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.604 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.605 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.605 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.605 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.606 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.606 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.606 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.606 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.606 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.606 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.606 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.607 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.607 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.607 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.607 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-05-08T18:04:34.543271+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=34, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.607 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.607 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.608 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-05-08T18:04:44.789644+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.608 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.608 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.608 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.608 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.609 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.609 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.609 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.609 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.609 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.610 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-05-08T18:04:42.782385+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:06.610 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T18:09:26.611 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T18:09:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.728+0000 7fad8c9f2640 1 Processor -- start 2024-05-08T18:09:26.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.728+0000 7fad8c9f2640 1 -- start start 2024-05-08T18:09:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.729+0000 7fad8c9f2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad8812c530 0x7fad8812c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:09:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.729+0000 7fad8c9f2640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad8812c230 con 0x7fad88071d40 2024-05-08T18:09:26.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.729+0000 7fad8c9f2640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fad8809ad20 con 0x7fad8812c530 2024-05-08T18:09:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.729+0000 7fad8659c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fad8812c530 0x7fad8812c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:09:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.729+0000 7fad85d9b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fad88071d40 0x7fad88072150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:58978/0 (socket says 172.21.15.46:58978) 2024-05-08T18:09:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.729+0000 7fad85d9b640 1 -- 172.21.15.46:0/2773819423 learned_addr learned my addr 172.21.15.46:0/2773819423 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:09:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.729+0000 7fad8659c640 1 -- 172.21.15.46:0/2773819423 >> v1:172.21.15.46:6789/0 conn(0x7fad88071d40 legacy=0x7fad88072150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:09:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.729+0000 7fad8659c640 1 -- 172.21.15.46:0/2773819423 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad88073510 con 0x7fad8812c530 2024-05-08T18:09:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.729+0000 7fad8559a640 1 -- 172.21.15.46:0/2773819423 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 804191311 0 0) 0x7fad8812c230 con 0x7fad88071d40 2024-05-08T18:09:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.730+0000 7fad8659c640 1 --2- 172.21.15.46:0/2773819423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad8812c530 0x7fad8812c910 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fad78009720 tx=0x7fad7802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0cb05f2def4e5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:09:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.730+0000 7fad8559a640 1 -- 172.21.15.46:0/2773819423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad780316d0 con 0x7fad8812c530 2024-05-08T18:09:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.730+0000 7fad8559a640 1 -- 172.21.15.46:0/2773819423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad78037070 con 0x7fad8812c530 2024-05-08T18:09:26.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.730+0000 7fad8559a640 1 -- 172.21.15.46:0/2773819423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad780319c0 con 0x7fad8812c530 2024-05-08T18:09:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.731+0000 7fad8c9f2640 1 -- 172.21.15.46:0/2773819423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad8812c530 msgr2=0x7fad8812c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:09:26.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.731+0000 7fad8c9f2640 1 --2- 172.21.15.46:0/2773819423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad8812c530 0x7fad8812c910 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fad78009720 tx=0x7fad7802b840 comp rx=0 tx=0).stop 2024-05-08T18:09:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.731+0000 7fad8c9f2640 1 -- 172.21.15.46:0/2773819423 shutdown_connections 2024-05-08T18:09:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.731+0000 7fad8c9f2640 1 --2- 172.21.15.46:0/2773819423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad8812c530 0x7fad8812c910 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:09:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.731+0000 7fad8c9f2640 1 -- 172.21.15.46:0/2773819423 >> 172.21.15.46:0/2773819423 conn(0x7fad8809d7a0 msgr2=0x7fad8809dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:09:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.731+0000 7fad8c9f2640 1 -- 172.21.15.46:0/2773819423 shutdown_connections 2024-05-08T18:09:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.731+0000 7fad8c9f2640 1 -- 172.21.15.46:0/2773819423 wait complete. 2024-05-08T18:09:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.732+0000 7fad8c9f2640 1 Processor -- start 2024-05-08T18:09:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.732+0000 7fad8c9f2640 1 -- start start 2024-05-08T18:09:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.732+0000 7fad8c9f2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad88071d40 0x7fad88126c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:09:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.732+0000 7fad8c9f2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad8812cfd0 con 0x7fad88071d40 2024-05-08T18:09:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.732+0000 7fad8659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad88071d40 0x7fad88126c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:09:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.732+0000 7fad8659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad88071d40 0x7fad88126c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46078/0 (socket says 172.21.15.46:46078) 2024-05-08T18:09:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.732+0000 7fad8659c640 1 -- 172.21.15.46:0/2883118305 learned_addr learned my addr 172.21.15.46:0/2883118305 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:09:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.733+0000 7fad8659c640 1 -- 172.21.15.46:0/2883118305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad88070ea0 con 0x7fad88071d40 2024-05-08T18:09:26.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.733+0000 7fad8659c640 1 --2- 172.21.15.46:0/2883118305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad88071d40 0x7fad88126c80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fad78009720 tx=0x7fad780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:09:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.733+0000 7fad76ffd640 1 -- 172.21.15.46:0/2883118305 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad78004160 con 0x7fad88071d40 2024-05-08T18:09:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.733+0000 7fad76ffd640 1 -- 172.21.15.46:0/2883118305 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad78004320 con 0x7fad88071d40 2024-05-08T18:09:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.733+0000 7fad8c9f2640 1 -- 172.21.15.46:0/2883118305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad88129650 con 0x7fad88071d40 2024-05-08T18:09:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.733+0000 7fad76ffd640 1 -- 172.21.15.46:0/2883118305 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fad78032640 con 0x7fad88071d40 2024-05-08T18:09:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.733+0000 7fad8c9f2640 1 -- 172.21.15.46:0/2883118305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad88128790 con 0x7fad88071d40 2024-05-08T18:09:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.734+0000 7fad76ffd640 1 -- 172.21.15.46:0/2883118305 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fad780327e0 con 0x7fad88071d40 2024-05-08T18:09:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.734+0000 7fad76ffd640 1 --2- 172.21.15.46:0/2883118305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fad6803d2b0 0x7fad6803f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:09:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.734+0000 7fad8c9f2640 1 -- 172.21.15.46:0/2883118305 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad88128ad0 con 0x7fad88071d40 2024-05-08T18:09:26.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.734+0000 7fad76ffd640 1 -- 172.21.15.46:0/2883118305 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fad7802f050 con 0x7fad88071d40 2024-05-08T18:09:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.735+0000 7fad85d9b640 1 --2- 172.21.15.46:0/2883118305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fad6803d2b0 0x7fad6803f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:09:26.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.735+0000 7fad85d9b640 1 --2- 172.21.15.46:0/2883118305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fad6803d2b0 0x7fad6803f770 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fad7c000c00 tx=0x7fad7c0078d0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:09:26.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.739+0000 7fad76ffd640 1 -- 172.21.15.46:0/2883118305 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fad88128ad0 con 0x7fad88071d40 2024-05-08T18:09:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.926+0000 7fad8c9f2640 1 -- 172.21.15.46:0/2883118305 --> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fad88071a30 con 0x7fad6803d2b0 2024-05-08T18:09:26.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.942+0000 7fad76ffd640 1 -- 172.21.15.46:0/2883118305 <== mgr.4098 v2:172.21.15.50:6824/2602185226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665997 (secure 0 0 0) 0x7fad88071a30 con 0x7fad6803d2b0 2024-05-08T18:09:26.947 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:09:26.952 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T18:09:26.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.946+0000 7fad8c9f2640 1 -- 172.21.15.46:0/2883118305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fad6803d2b0 msgr2=0x7fad6803f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:09:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.946+0000 7fad8c9f2640 1 --2- 172.21.15.46:0/2883118305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fad6803d2b0 0x7fad6803f770 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fad7c000c00 tx=0x7fad7c0078d0 comp rx=0 tx=0).stop 2024-05-08T18:09:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.946+0000 7fad8c9f2640 1 -- 172.21.15.46:0/2883118305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad88071d40 msgr2=0x7fad88126c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:09:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.946+0000 7fad8c9f2640 1 --2- 172.21.15.46:0/2883118305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad88071d40 0x7fad88126c80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fad78009720 tx=0x7fad780046c0 comp rx=0 tx=0).stop 2024-05-08T18:09:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.946+0000 7fad8c9f2640 1 -- 172.21.15.46:0/2883118305 shutdown_connections 2024-05-08T18:09:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.946+0000 7fad8c9f2640 1 --2- 172.21.15.46:0/2883118305 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fad6803d2b0 0x7fad6803f770 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:09:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.946+0000 7fad8c9f2640 1 --2- 172.21.15.46:0/2883118305 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad88071d40 0x7fad88126c80 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:09:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.946+0000 7fad8c9f2640 1 -- 172.21.15.46:0/2883118305 >> 172.21.15.46:0/2883118305 conn(0x7fad8809d7a0 msgr2=0x7fad8812adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:09:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.947+0000 7fad8c9f2640 1 -- 172.21.15.46:0/2883118305 shutdown_connections 2024-05-08T18:09:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:26.947+0000 7fad8c9f2640 1 -- 172.21.15.46:0/2883118305 wait complete. 2024-05-08T18:09:26.956 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-05-08T18:09:25.756599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605383,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":5050,"num_read_kb":4636,"num_write":2119,"num_write_kb":60585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3349,"num_omap_keys":16,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4742,"ondisk_log_size":4742,"up":658,"acting":658,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":658,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":127448,"kb_used_data":2744,"kb_used_omap":6,"kb_used_meta":124665,"kb_avail":377359912,"statfs":{"total":386547056640,"available":386416549888,"internally_reserved":0,"allocated":2809856,"data_stored":1720046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":127657750},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003709"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:46.010735+0000","last_change":"2024-05-08T18:08:46.010735+0000","last_active":"2024-05-08T18:08:46.010735+0000","last_peered":"2024-05-08T18:08:46.010735+0000","last_clean":"2024-05-08T18:08:46.010735+0000","last_became_active":"2024-05-08T18:04:43.836496+0000","last_became_peered":"2024-05-08T18:04:43.836496+0000","last_unstale":"2024-05-08T18:08:46.010735+0000","last_undegraded":"2024-05-08T18:08:46.010735+0000","last_fullsized":"2024-05-08T18:08:46.010735+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:46.010599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:46.010599+0000","last_clean_scrub_stamp":"2024-05-08T18:08:46.010599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:07.142721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'6","reported_seq":60,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:49.481085+0000","last_change":"2024-05-08T18:08:49.481113+0000","last_active":"2024-05-08T18:08:49.481085+0000","last_peered":"2024-05-08T18:08:49.481085+0000","last_clean":"2024-05-08T18:08:49.481085+0000","last_became_active":"2024-05-08T18:04:45.811368+0000","last_became_peered":"2024-05-08T18:04:45.811368+0000","last_unstale":"2024-05-08T18:08:49.481085+0000","last_undegraded":"2024-05-08T18:08:49.481085+0000","last_fullsized":"2024-05-08T18:08:49.481085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:49.480997+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:49.480997+0000","last_clean_scrub_stamp":"2024-05-08T18:08:49.480997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:21:18.714592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"29'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:50.347475+0000","last_change":"2024-05-08T18:08:50.347475+0000","last_active":"2024-05-08T18:08:50.347475+0000","last_peered":"2024-05-08T18:08:50.347475+0000","last_clean":"2024-05-08T18:08:50.347475+0000","last_became_active":"2024-05-08T18:04:43.834612+0000","last_became_peered":"2024-05-08T18:04:43.834612+0000","last_unstale":"2024-05-08T18:08:50.347475+0000","last_undegraded":"2024-05-08T18:08:50.347475+0000","last_fullsized":"2024-05-08T18:08:50.347475+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:50.347354+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:50.347354+0000","last_clean_scrub_stamp":"2024-05-08T18:08:50.347354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:07:56.846928+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"29'2","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:40.037447+0000","last_change":"2024-05-08T18:08:40.037474+0000","last_active":"2024-05-08T18:08:40.037447+0000","last_peered":"2024-05-08T18:08:40.037447+0000","last_clean":"2024-05-08T18:08:40.037447+0000","last_became_active":"2024-05-08T18:04:45.827965+0000","last_became_peered":"2024-05-08T18:04:45.827965+0000","last_unstale":"2024-05-08T18:08:40.037447+0000","last_undegraded":"2024-05-08T18:08:40.037447+0000","last_fullsized":"2024-05-08T18:08:40.037447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:40.037352+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:40.037352+0000","last_clean_scrub_stamp":"2024-05-08T18:08:40.037352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:22:58.859282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:00.008391+0000","last_change":"2024-05-08T18:09:00.008419+0000","last_active":"2024-05-08T18:09:00.008391+0000","last_peered":"2024-05-08T18:09:00.008391+0000","last_clean":"2024-05-08T18:09:00.008391+0000","last_became_active":"2024-05-08T18:04:43.837456+0000","last_became_peered":"2024-05-08T18:04:43.837456+0000","last_unstale":"2024-05-08T18:09:00.008391+0000","last_undegraded":"2024-05-08T18:09:00.008391+0000","last_fullsized":"2024-05-08T18:09:00.008391+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:00.008279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:00.008279+0000","last_clean_scrub_stamp":"2024-05-08T18:09:00.008279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:38.600725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"29'12","reported_seq":66,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:50.085604+0000","last_change":"2024-05-08T18:08:50.085643+0000","last_active":"2024-05-08T18:08:50.085604+0000","last_peered":"2024-05-08T18:08:50.085604+0000","last_clean":"2024-05-08T18:08:50.085604+0000","last_became_active":"2024-05-08T18:04:45.825022+0000","last_became_peered":"2024-05-08T18:04:45.825022+0000","last_unstale":"2024-05-08T18:08:50.085604+0000","last_undegraded":"2024-05-08T18:08:50.085604+0000","last_fullsized":"2024-05-08T18:08:50.085604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:50.085506+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:50.085506+0000","last_clean_scrub_stamp":"2024-05-08T18:08:50.085506+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:14:46.319120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:40.176922+0000","last_change":"2024-05-08T18:08:40.176922+0000","last_active":"2024-05-08T18:08:40.176922+0000","last_peered":"2024-05-08T18:08:40.176922+0000","last_clean":"2024-05-08T18:08:40.176922+0000","last_became_active":"2024-05-08T18:04:43.842077+0000","last_became_peered":"2024-05-08T18:04:43.842077+0000","last_unstale":"2024-05-08T18:08:40.176922+0000","last_undegraded":"2024-05-08T18:08:40.176922+0000","last_fullsized":"2024-05-08T18:08:40.176922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:40.176797+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:40.176797+0000","last_clean_scrub_stamp":"2024-05-08T18:08:40.176797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:14.062529+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"29'16","reported_seq":79,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:51.114482+0000","last_change":"2024-05-08T18:08:51.114508+0000","last_active":"2024-05-08T18:08:51.114482+0000","last_peered":"2024-05-08T18:08:51.114482+0000","last_clean":"2024-05-08T18:08:51.114482+0000","last_became_active":"2024-05-08T18:04:45.824581+0000","last_became_peered":"2024-05-08T18:04:45.824581+0000","last_unstale":"2024-05-08T18:08:51.114482+0000","last_undegraded":"2024-05-08T18:08:51.114482+0000","last_fullsized":"2024-05-08T18:08:51.114482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'16","last_scrub_stamp":"2024-05-08T18:08:51.114374+0000","last_deep_scrub":"29'16","last_deep_scrub_stamp":"2024-05-08T18:08:51.114374+0000","last_clean_scrub_stamp":"2024-05-08T18:08:51.114374+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:34.341808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:22.029562+0000","last_change":"2024-05-08T18:09:22.029587+0000","last_active":"2024-05-08T18:09:22.029562+0000","last_peered":"2024-05-08T18:09:22.029562+0000","last_clean":"2024-05-08T18:09:22.029562+0000","last_became_active":"2024-05-08T18:04:43.832722+0000","last_became_peered":"2024-05-08T18:04:43.832722+0000","last_unstale":"2024-05-08T18:09:22.029562+0000","last_undegraded":"2024-05-08T18:09:22.029562+0000","last_fullsized":"2024-05-08T18:09:22.029562+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:22.029460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:22.029460+0000","last_clean_scrub_stamp":"2024-05-08T18:09:22.029460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:45:23.654040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"29'8","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:35.526800+0000","last_change":"2024-05-08T18:08:35.526800+0000","last_active":"2024-05-08T18:08:35.526800+0000","last_peered":"2024-05-08T18:08:35.526800+0000","last_clean":"2024-05-08T18:08:35.526800+0000","last_became_active":"2024-05-08T18:04:45.808383+0000","last_became_peered":"2024-05-08T18:04:45.808383+0000","last_unstale":"2024-05-08T18:08:35.526800+0000","last_undegraded":"2024-05-08T18:08:35.526800+0000","last_fullsized":"2024-05-08T18:08:35.526800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:35.526676+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:35.526676+0000","last_clean_scrub_stamp":"2024-05-08T18:08:35.526676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:24.925676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:42.091190+0000","last_change":"2024-05-08T18:08:42.091220+0000","last_active":"2024-05-08T18:08:42.091190+0000","last_peered":"2024-05-08T18:08:42.091190+0000","last_clean":"2024-05-08T18:08:42.091190+0000","last_became_active":"2024-05-08T18:04:43.840851+0000","last_became_peered":"2024-05-08T18:04:43.840851+0000","last_unstale":"2024-05-08T18:08:42.091190+0000","last_undegraded":"2024-05-08T18:08:42.091190+0000","last_fullsized":"2024-05-08T18:08:42.091190+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:42.091091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:42.091091+0000","last_clean_scrub_stamp":"2024-05-08T18:08:42.091091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:24.274648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:39.004566+0000","last_change":"2024-05-08T18:08:39.004566+0000","last_active":"2024-05-08T18:08:39.004566+0000","last_peered":"2024-05-08T18:08:39.004566+0000","last_clean":"2024-05-08T18:08:39.004566+0000","last_became_active":"2024-05-08T18:04:45.828334+0000","last_became_peered":"2024-05-08T18:04:45.828334+0000","last_unstale":"2024-05-08T18:08:39.004566+0000","last_undegraded":"2024-05-08T18:08:39.004566+0000","last_fullsized":"2024-05-08T18:08:39.004566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:39.004442+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:39.004442+0000","last_clean_scrub_stamp":"2024-05-08T18:08:39.004442+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:31:47.634365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"29'2","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:13.988298+0000","last_change":"2024-05-08T18:09:13.988375+0000","last_active":"2024-05-08T18:09:13.988298+0000","last_peered":"2024-05-08T18:09:13.988298+0000","last_clean":"2024-05-08T18:09:13.988298+0000","last_became_active":"2024-05-08T18:04:43.833437+0000","last_became_peered":"2024-05-08T18:04:43.833437+0000","last_unstale":"2024-05-08T18:09:13.988298+0000","last_undegraded":"2024-05-08T18:09:13.988298+0000","last_fullsized":"2024-05-08T18:09:13.988298+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:09:13.988182+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:09:13.988182+0000","last_clean_scrub_stamp":"2024-05-08T18:09:13.988182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:41:56.815735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:29.307581+0000","last_change":"2024-05-08T18:08:29.307581+0000","last_active":"2024-05-08T18:08:29.307581+0000","last_peered":"2024-05-08T18:08:29.307581+0000","last_clean":"2024-05-08T18:08:29.307581+0000","last_became_active":"2024-05-08T18:04:45.811884+0000","last_became_peered":"2024-05-08T18:04:45.811884+0000","last_unstale":"2024-05-08T18:08:29.307581+0000","last_undegraded":"2024-05-08T18:08:29.307581+0000","last_fullsized":"2024-05-08T18:08:29.307581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:29.307470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:29.307470+0000","last_clean_scrub_stamp":"2024-05-08T18:08:29.307470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:16:25.939226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:53.384389+0000","last_change":"2024-05-08T18:08:53.384464+0000","last_active":"2024-05-08T18:08:53.384389+0000","last_peered":"2024-05-08T18:08:53.384389+0000","last_clean":"2024-05-08T18:08:53.384389+0000","last_became_active":"2024-05-08T18:04:43.820312+0000","last_became_peered":"2024-05-08T18:04:43.820312+0000","last_unstale":"2024-05-08T18:08:53.384389+0000","last_undegraded":"2024-05-08T18:08:53.384389+0000","last_fullsized":"2024-05-08T18:08:53.384389+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:53.384283+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:53.384283+0000","last_clean_scrub_stamp":"2024-05-08T18:08:53.384283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:05:28.766196+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"29'8","reported_seq":72,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:50.932035+0000","last_change":"2024-05-08T18:08:50.932035+0000","last_active":"2024-05-08T18:08:50.932035+0000","last_peered":"2024-05-08T18:08:50.932035+0000","last_clean":"2024-05-08T18:08:50.932035+0000","last_became_active":"2024-05-08T18:04:45.822620+0000","last_became_peered":"2024-05-08T18:04:45.822620+0000","last_unstale":"2024-05-08T18:08:50.932035+0000","last_undegraded":"2024-05-08T18:08:50.932035+0000","last_fullsized":"2024-05-08T18:08:50.932035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:50.931921+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:50.931921+0000","last_clean_scrub_stamp":"2024-05-08T18:08:50.931921+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:50:39.324779+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"29'4","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:46.060404+0000","last_change":"2024-05-08T18:08:46.060430+0000","last_active":"2024-05-08T18:08:46.060404+0000","last_peered":"2024-05-08T18:08:46.060404+0000","last_clean":"2024-05-08T18:08:46.060404+0000","last_became_active":"2024-05-08T18:04:45.826493+0000","last_became_peered":"2024-05-08T18:04:45.826493+0000","last_unstale":"2024-05-08T18:08:46.060404+0000","last_undegraded":"2024-05-08T18:08:46.060404+0000","last_fullsized":"2024-05-08T18:08:46.060404+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:46.060304+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:46.060304+0000","last_clean_scrub_stamp":"2024-05-08T18:08:46.060304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:28:42.176968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:46.481335+0000","last_change":"2024-05-08T18:08:46.481360+0000","last_active":"2024-05-08T18:08:46.481335+0000","last_peered":"2024-05-08T18:08:46.481335+0000","last_clean":"2024-05-08T18:08:46.481335+0000","last_became_active":"2024-05-08T18:04:43.838332+0000","last_became_peered":"2024-05-08T18:04:43.838332+0000","last_unstale":"2024-05-08T18:08:46.481335+0000","last_undegraded":"2024-05-08T18:08:46.481335+0000","last_fullsized":"2024-05-08T18:08:46.481335+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:46.481227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:46.481227+0000","last_clean_scrub_stamp":"2024-05-08T18:08:46.481227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:44:09.862039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"29'4","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:36.520147+0000","last_change":"2024-05-08T18:08:36.520179+0000","last_active":"2024-05-08T18:08:36.520147+0000","last_peered":"2024-05-08T18:08:36.520147+0000","last_clean":"2024-05-08T18:08:36.520147+0000","last_became_active":"2024-05-08T18:04:45.807438+0000","last_became_peered":"2024-05-08T18:04:45.807438+0000","last_unstale":"2024-05-08T18:08:36.520147+0000","last_undegraded":"2024-05-08T18:08:36.520147+0000","last_fullsized":"2024-05-08T18:08:36.520147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:36.520045+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:36.520045+0000","last_clean_scrub_stamp":"2024-05-08T18:08:36.520045+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:18.703602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:30.298724+0000","last_change":"2024-05-08T18:08:30.298757+0000","last_active":"2024-05-08T18:08:30.298724+0000","last_peered":"2024-05-08T18:08:30.298724+0000","last_clean":"2024-05-08T18:08:30.298724+0000","last_became_active":"2024-05-08T18:04:43.842741+0000","last_became_peered":"2024-05-08T18:04:43.842741+0000","last_unstale":"2024-05-08T18:08:30.298724+0000","last_undegraded":"2024-05-08T18:08:30.298724+0000","last_fullsized":"2024-05-08T18:08:30.298724+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:30.298603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:30.298603+0000","last_clean_scrub_stamp":"2024-05-08T18:08:30.298603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:30:25.561243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"29'12","reported_seq":67,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:43.533148+0000","last_change":"2024-05-08T18:08:43.533178+0000","last_active":"2024-05-08T18:08:43.533148+0000","last_peered":"2024-05-08T18:08:43.533148+0000","last_clean":"2024-05-08T18:08:43.533148+0000","last_became_active":"2024-05-08T18:04:45.810827+0000","last_became_peered":"2024-05-08T18:04:45.810827+0000","last_unstale":"2024-05-08T18:08:43.533148+0000","last_undegraded":"2024-05-08T18:08:43.533148+0000","last_fullsized":"2024-05-08T18:08:43.533148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:43.533034+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:43.533034+0000","last_clean_scrub_stamp":"2024-05-08T18:08:43.533034+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:04.129239+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"29'4","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:50.465263+0000","last_change":"2024-05-08T18:08:50.465291+0000","last_active":"2024-05-08T18:08:50.465263+0000","last_peered":"2024-05-08T18:08:50.465263+0000","last_clean":"2024-05-08T18:08:50.465263+0000","last_became_active":"2024-05-08T18:04:43.837706+0000","last_became_peered":"2024-05-08T18:04:43.837706+0000","last_unstale":"2024-05-08T18:08:50.465263+0000","last_undegraded":"2024-05-08T18:08:50.465263+0000","last_fullsized":"2024-05-08T18:08:50.465263+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:50.465171+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:50.465171+0000","last_clean_scrub_stamp":"2024-05-08T18:08:50.465171+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:00:00.657178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"29'8","reported_seq":68,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:32.455142+0000","last_change":"2024-05-08T18:08:32.455190+0000","last_active":"2024-05-08T18:08:32.455142+0000","last_peered":"2024-05-08T18:08:32.455142+0000","last_clean":"2024-05-08T18:08:32.455142+0000","last_became_active":"2024-05-08T18:04:45.822000+0000","last_became_peered":"2024-05-08T18:04:45.822000+0000","last_unstale":"2024-05-08T18:08:32.455142+0000","last_undegraded":"2024-05-08T18:08:32.455142+0000","last_fullsized":"2024-05-08T18:08:32.455142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:32.455035+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:32.455035+0000","last_clean_scrub_stamp":"2024-05-08T18:08:32.455035+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:21.854035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:40.545746+0000","last_change":"2024-05-08T18:08:40.545772+0000","last_active":"2024-05-08T18:08:40.545746+0000","last_peered":"2024-05-08T18:08:40.545746+0000","last_clean":"2024-05-08T18:08:40.545746+0000","last_became_active":"2024-05-08T18:04:43.841063+0000","last_became_peered":"2024-05-08T18:04:43.841063+0000","last_unstale":"2024-05-08T18:08:40.545746+0000","last_undegraded":"2024-05-08T18:08:40.545746+0000","last_fullsized":"2024-05-08T18:08:40.545746+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:40.545644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:40.545644+0000","last_clean_scrub_stamp":"2024-05-08T18:08:40.545644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:55:34.045722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"29'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:26.277429+0000","last_change":"2024-05-08T18:08:26.277484+0000","last_active":"2024-05-08T18:08:26.277429+0000","last_peered":"2024-05-08T18:08:26.277429+0000","last_clean":"2024-05-08T18:08:26.277429+0000","last_became_active":"2024-05-08T18:04:45.823611+0000","last_became_peered":"2024-05-08T18:04:45.823611+0000","last_unstale":"2024-05-08T18:08:26.277429+0000","last_undegraded":"2024-05-08T18:08:26.277429+0000","last_fullsized":"2024-05-08T18:08:26.277429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:26.277327+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:26.277327+0000","last_clean_scrub_stamp":"2024-05-08T18:08:26.277327+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:13.871401+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:46.372006+0000","last_change":"2024-05-08T18:08:46.372029+0000","last_active":"2024-05-08T18:08:46.372006+0000","last_peered":"2024-05-08T18:08:46.372006+0000","last_clean":"2024-05-08T18:08:46.372006+0000","last_became_active":"2024-05-08T18:04:43.834456+0000","last_became_peered":"2024-05-08T18:04:43.834456+0000","last_unstale":"2024-05-08T18:08:46.372006+0000","last_undegraded":"2024-05-08T18:08:46.372006+0000","last_fullsized":"2024-05-08T18:08:46.372006+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:46.371925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:46.371925+0000","last_clean_scrub_stamp":"2024-05-08T18:08:46.371925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:23.458745+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"29'6","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:37.506169+0000","last_change":"2024-05-08T18:08:37.506219+0000","last_active":"2024-05-08T18:08:37.506169+0000","last_peered":"2024-05-08T18:08:37.506169+0000","last_clean":"2024-05-08T18:08:37.506169+0000","last_became_active":"2024-05-08T18:04:45.808416+0000","last_became_peered":"2024-05-08T18:04:45.808416+0000","last_unstale":"2024-05-08T18:08:37.506169+0000","last_undegraded":"2024-05-08T18:08:37.506169+0000","last_fullsized":"2024-05-08T18:08:37.506169+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:37.506056+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:37.506056+0000","last_clean_scrub_stamp":"2024-05-08T18:08:37.506056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:14.952253+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'4","reported_seq":60,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:10.945688+0000","last_change":"2024-05-08T18:09:10.945688+0000","last_active":"2024-05-08T18:09:10.945688+0000","last_peered":"2024-05-08T18:09:10.945688+0000","last_clean":"2024-05-08T18:09:10.945688+0000","last_became_active":"2024-05-08T18:04:43.844702+0000","last_became_peered":"2024-05-08T18:04:43.844702+0000","last_unstale":"2024-05-08T18:09:10.945688+0000","last_undegraded":"2024-05-08T18:09:10.945688+0000","last_fullsized":"2024-05-08T18:09:10.945688+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-08T18:09:10.945566+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-08T18:09:10.945566+0000","last_clean_scrub_stamp":"2024-05-08T18:09:10.945566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:16:52.494468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"29'36","reported_seq":109,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:02.553501+0000","last_change":"2024-05-08T18:09:02.553531+0000","last_active":"2024-05-08T18:09:02.553501+0000","last_peered":"2024-05-08T18:09:02.553501+0000","last_clean":"2024-05-08T18:09:02.553501+0000","last_became_active":"2024-05-08T18:04:53.849475+0000","last_became_peered":"2024-05-08T18:04:53.849475+0000","last_unstale":"2024-05-08T18:09:02.553501+0000","last_undegraded":"2024-05-08T18:09:02.553501+0000","last_fullsized":"2024-05-08T18:09:02.553501+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:09:02.553411+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:09:02.553411+0000","last_clean_scrub_stamp":"2024-05-08T18:09:02.553411+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:25:37.317356+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"29'12","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:52.160816+0000","last_change":"2024-05-08T18:08:52.160843+0000","last_active":"2024-05-08T18:08:52.160816+0000","last_peered":"2024-05-08T18:08:52.160816+0000","last_clean":"2024-05-08T18:08:52.160816+0000","last_became_active":"2024-05-08T18:04:45.824005+0000","last_became_peered":"2024-05-08T18:04:45.824005+0000","last_unstale":"2024-05-08T18:08:52.160816+0000","last_undegraded":"2024-05-08T18:08:52.160816+0000","last_fullsized":"2024-05-08T18:08:52.160816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:52.160704+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:52.160704+0000","last_clean_scrub_stamp":"2024-05-08T18:08:52.160704+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:13.667834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:11.930576+0000","last_change":"2024-05-08T18:09:11.930576+0000","last_active":"2024-05-08T18:09:11.930576+0000","last_peered":"2024-05-08T18:09:11.930576+0000","last_clean":"2024-05-08T18:09:11.930576+0000","last_became_active":"2024-05-08T18:04:43.833822+0000","last_became_peered":"2024-05-08T18:04:43.833822+0000","last_unstale":"2024-05-08T18:09:11.930576+0000","last_undegraded":"2024-05-08T18:09:11.930576+0000","last_fullsized":"2024-05-08T18:09:11.930576+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:09:11.930438+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:09:11.930438+0000","last_clean_scrub_stamp":"2024-05-08T18:09:11.930438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:47:50.587916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.924513+0000","last_change":"2024-05-08T18:08:09.924538+0000","last_active":"2024-05-08T18:08:09.924513+0000","last_peered":"2024-05-08T18:08:09.924513+0000","last_clean":"2024-05-08T18:08:09.924513+0000","last_became_active":"2024-05-08T18:04:47.819682+0000","last_became_peered":"2024-05-08T18:04:47.819682+0000","last_unstale":"2024-05-08T18:08:09.924513+0000","last_undegraded":"2024-05-08T18:08:09.924513+0000","last_fullsized":"2024-05-08T18:08:09.924513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:09.924406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:09.924406+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.924406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:58.088967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"29'39","reported_seq":94,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:31.864262+0000","last_change":"2024-05-08T18:08:31.864298+0000","last_active":"2024-05-08T18:08:31.864262+0000","last_peered":"2024-05-08T18:08:31.864262+0000","last_clean":"2024-05-08T18:08:31.864262+0000","last_became_active":"2024-05-08T18:04:49.832148+0000","last_became_peered":"2024-05-08T18:04:49.832148+0000","last_unstale":"2024-05-08T18:08:31.864262+0000","last_undegraded":"2024-05-08T18:08:31.864262+0000","last_fullsized":"2024-05-08T18:08:31.864262+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'39","last_scrub_stamp":"2024-05-08T18:08:31.864172+0000","last_deep_scrub":"29'39","last_deep_scrub_stamp":"2024-05-08T18:08:31.864172+0000","last_clean_scrub_stamp":"2024-05-08T18:08:31.864172+0000","objects_scrubbed":4,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:47:21.937257+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":40,"num_read_kb":42,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.510810+0000","last_change":"2024-05-08T18:08:21.510876+0000","last_active":"2024-05-08T18:08:21.510810+0000","last_peered":"2024-05-08T18:08:21.510810+0000","last_clean":"2024-05-08T18:08:21.510810+0000","last_became_active":"2024-05-08T18:04:51.844901+0000","last_became_peered":"2024-05-08T18:04:51.844901+0000","last_unstale":"2024-05-08T18:08:21.510810+0000","last_undegraded":"2024-05-08T18:08:21.510810+0000","last_fullsized":"2024-05-08T18:08:21.510810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:21.510687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:21.510687+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.510687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:55:15.292759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"29'11","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:03.522110+0000","last_change":"2024-05-08T18:09:03.522138+0000","last_active":"2024-05-08T18:09:03.522110+0000","last_peered":"2024-05-08T18:09:03.522110+0000","last_clean":"2024-05-08T18:09:03.522110+0000","last_became_active":"2024-05-08T18:04:53.849579+0000","last_became_peered":"2024-05-08T18:04:53.849579+0000","last_unstale":"2024-05-08T18:09:03.522110+0000","last_undegraded":"2024-05-08T18:09:03.522110+0000","last_fullsized":"2024-05-08T18:09:03.522110+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'11","last_scrub_stamp":"2024-05-08T18:09:03.522007+0000","last_deep_scrub":"29'11","last_deep_scrub_stamp":"2024-05-08T18:09:03.522007+0000","last_clean_scrub_stamp":"2024-05-08T18:09:03.522007+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:01.403489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"29'6","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:44.084234+0000","last_change":"2024-05-08T18:08:44.084259+0000","last_active":"2024-05-08T18:08:44.084234+0000","last_peered":"2024-05-08T18:08:44.084234+0000","last_clean":"2024-05-08T18:08:44.084234+0000","last_became_active":"2024-05-08T18:04:45.825162+0000","last_became_peered":"2024-05-08T18:04:45.825162+0000","last_unstale":"2024-05-08T18:08:44.084234+0000","last_undegraded":"2024-05-08T18:08:44.084234+0000","last_fullsized":"2024-05-08T18:08:44.084234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:44.084137+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:44.084137+0000","last_clean_scrub_stamp":"2024-05-08T18:08:44.084137+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:00:15.074875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:44.010708+0000","last_change":"2024-05-08T18:08:44.010708+0000","last_active":"2024-05-08T18:08:44.010708+0000","last_peered":"2024-05-08T18:08:44.010708+0000","last_clean":"2024-05-08T18:08:44.010708+0000","last_became_active":"2024-05-08T18:04:43.827374+0000","last_became_peered":"2024-05-08T18:04:43.827374+0000","last_unstale":"2024-05-08T18:08:44.010708+0000","last_undegraded":"2024-05-08T18:08:44.010708+0000","last_fullsized":"2024-05-08T18:08:44.010708+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:44.010568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:44.010568+0000","last_clean_scrub_stamp":"2024-05-08T18:08:44.010568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:59:57.344162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:05.547885+0000","last_change":"2024-05-08T18:09:05.547916+0000","last_active":"2024-05-08T18:09:05.547885+0000","last_peered":"2024-05-08T18:09:05.547885+0000","last_clean":"2024-05-08T18:09:05.547885+0000","last_became_active":"2024-05-08T18:05:30.868191+0000","last_became_peered":"2024-05-08T18:05:30.868191+0000","last_unstale":"2024-05-08T18:09:05.547885+0000","last_undegraded":"2024-05-08T18:09:05.547885+0000","last_fullsized":"2024-05-08T18:09:05.547885+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:05.547803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:05.547803+0000","last_clean_scrub_stamp":"2024-05-08T18:09:05.547803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:54.348953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"29'9","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.529952+0000","last_change":"2024-05-08T18:08:10.529988+0000","last_active":"2024-05-08T18:08:10.529952+0000","last_peered":"2024-05-08T18:08:10.529952+0000","last_clean":"2024-05-08T18:08:10.529952+0000","last_became_active":"2024-05-08T18:04:49.825265+0000","last_became_peered":"2024-05-08T18:04:49.825265+0000","last_unstale":"2024-05-08T18:08:10.529952+0000","last_undegraded":"2024-05-08T18:08:10.529952+0000","last_fullsized":"2024-05-08T18:08:10.529952+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'9","last_scrub_stamp":"2024-05-08T18:08:10.529858+0000","last_deep_scrub":"29'9","last_deep_scrub_stamp":"2024-05-08T18:08:10.529858+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.529858+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:00.215888+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.491557+0000","last_change":"2024-05-08T18:08:13.491631+0000","last_active":"2024-05-08T18:08:13.491557+0000","last_peered":"2024-05-08T18:08:13.491557+0000","last_clean":"2024-05-08T18:08:13.491557+0000","last_became_active":"2024-05-08T18:04:51.833523+0000","last_became_peered":"2024-05-08T18:04:51.833523+0000","last_unstale":"2024-05-08T18:08:13.491557+0000","last_undegraded":"2024-05-08T18:08:13.491557+0000","last_fullsized":"2024-05-08T18:08:13.491557+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:13.491458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:13.491458+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.491458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:07.228339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'2","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:33.477085+0000","last_change":"2024-05-08T18:08:33.477157+0000","last_active":"2024-05-08T18:08:33.477085+0000","last_peered":"2024-05-08T18:08:33.477085+0000","last_clean":"2024-05-08T18:08:33.477085+0000","last_became_active":"2024-05-08T18:04:53.840395+0000","last_became_peered":"2024-05-08T18:04:53.840395+0000","last_unstale":"2024-05-08T18:08:33.477085+0000","last_undegraded":"2024-05-08T18:08:33.477085+0000","last_fullsized":"2024-05-08T18:08:33.477085+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:33.476981+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:33.476981+0000","last_clean_scrub_stamp":"2024-05-08T18:08:33.476981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:15.660538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:58.075763+0000","last_change":"2024-05-08T18:08:58.075787+0000","last_active":"2024-05-08T18:08:58.075763+0000","last_peered":"2024-05-08T18:08:58.075763+0000","last_clean":"2024-05-08T18:08:58.075763+0000","last_became_active":"2024-05-08T18:05:30.869162+0000","last_became_peered":"2024-05-08T18:05:30.869162+0000","last_unstale":"2024-05-08T18:08:58.075763+0000","last_undegraded":"2024-05-08T18:08:58.075763+0000","last_fullsized":"2024-05-08T18:08:58.075763+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-05-08T18:08:58.075662+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-05-08T18:08:58.075662+0000","last_clean_scrub_stamp":"2024-05-08T18:08:58.075662+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:36:28.054883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":45,"num_write":24,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:05.011315+0000","last_change":"2024-05-08T18:09:05.011315+0000","last_active":"2024-05-08T18:09:05.011315+0000","last_peered":"2024-05-08T18:09:05.011315+0000","last_clean":"2024-05-08T18:09:05.011315+0000","last_became_active":"2024-05-08T18:04:43.838666+0000","last_became_peered":"2024-05-08T18:04:43.838666+0000","last_unstale":"2024-05-08T18:09:05.011315+0000","last_undegraded":"2024-05-08T18:09:05.011315+0000","last_fullsized":"2024-05-08T18:09:05.011315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:05.011187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:05.011187+0000","last_clean_scrub_stamp":"2024-05-08T18:09:05.011187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:10.074163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:27.491659+0000","last_change":"2024-05-08T18:08:27.491684+0000","last_active":"2024-05-08T18:08:27.491659+0000","last_peered":"2024-05-08T18:08:27.491659+0000","last_clean":"2024-05-08T18:08:27.491659+0000","last_became_active":"2024-05-08T18:04:47.819169+0000","last_became_peered":"2024-05-08T18:04:47.819169+0000","last_unstale":"2024-05-08T18:08:27.491659+0000","last_undegraded":"2024-05-08T18:08:27.491659+0000","last_fullsized":"2024-05-08T18:08:27.491659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:27.491574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:27.491574+0000","last_clean_scrub_stamp":"2024-05-08T18:08:27.491574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:24.380337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"29'17","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.168244+0000","last_change":"2024-05-08T18:08:11.168281+0000","last_active":"2024-05-08T18:08:11.168244+0000","last_peered":"2024-05-08T18:08:11.168244+0000","last_clean":"2024-05-08T18:08:11.168244+0000","last_became_active":"2024-05-08T18:04:49.831500+0000","last_became_peered":"2024-05-08T18:04:49.831500+0000","last_unstale":"2024-05-08T18:08:11.168244+0000","last_undegraded":"2024-05-08T18:08:11.168244+0000","last_fullsized":"2024-05-08T18:08:11.168244+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'17","last_scrub_stamp":"2024-05-08T18:08:11.168144+0000","last_deep_scrub":"29'17","last_deep_scrub_stamp":"2024-05-08T18:08:11.168144+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.168144+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:03.071434+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"29'286","reported_seq":394,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:41.051149+0000","last_change":"2024-05-08T18:08:41.051182+0000","last_active":"2024-05-08T18:08:41.051149+0000","last_peered":"2024-05-08T18:08:41.051149+0000","last_clean":"2024-05-08T18:08:41.051149+0000","last_became_active":"2024-05-08T18:04:51.849301+0000","last_became_peered":"2024-05-08T18:04:51.849301+0000","last_unstale":"2024-05-08T18:08:41.051149+0000","last_undegraded":"2024-05-08T18:08:41.051149+0000","last_fullsized":"2024-05-08T18:08:41.051149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'286","last_scrub_stamp":"2024-05-08T18:08:41.051045+0000","last_deep_scrub":"29'286","last_deep_scrub_stamp":"2024-05-08T18:08:41.051045+0000","last_clean_scrub_stamp":"2024-05-08T18:08:41.051045+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:35:15.383617+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:53.985162+0000","last_change":"2024-05-08T18:08:53.985191+0000","last_active":"2024-05-08T18:08:53.985162+0000","last_peered":"2024-05-08T18:08:53.985162+0000","last_clean":"2024-05-08T18:08:53.985162+0000","last_became_active":"2024-05-08T18:04:53.840559+0000","last_became_peered":"2024-05-08T18:04:53.840559+0000","last_unstale":"2024-05-08T18:08:53.985162+0000","last_undegraded":"2024-05-08T18:08:53.985162+0000","last_fullsized":"2024-05-08T18:08:53.985162+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:53.985043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:53.985043+0000","last_clean_scrub_stamp":"2024-05-08T18:08:53.985043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:06:41.672862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"29'6","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:45.404482+0000","last_change":"2024-05-08T18:08:45.404510+0000","last_active":"2024-05-08T18:08:45.404482+0000","last_peered":"2024-05-08T18:08:45.404482+0000","last_clean":"2024-05-08T18:08:45.404482+0000","last_became_active":"2024-05-08T18:04:45.811570+0000","last_became_peered":"2024-05-08T18:04:45.811570+0000","last_unstale":"2024-05-08T18:08:45.404482+0000","last_undegraded":"2024-05-08T18:08:45.404482+0000","last_fullsized":"2024-05-08T18:08:45.404482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:45.404379+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:45.404379+0000","last_clean_scrub_stamp":"2024-05-08T18:08:45.404379+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:31:16.310504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"29'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:48.487844+0000","last_change":"2024-05-08T18:08:48.487844+0000","last_active":"2024-05-08T18:08:48.487844+0000","last_peered":"2024-05-08T18:08:48.487844+0000","last_clean":"2024-05-08T18:08:48.487844+0000","last_became_active":"2024-05-08T18:04:43.836410+0000","last_became_peered":"2024-05-08T18:04:43.836410+0000","last_unstale":"2024-05-08T18:08:48.487844+0000","last_undegraded":"2024-05-08T18:08:48.487844+0000","last_fullsized":"2024-05-08T18:08:48.487844+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:48.487704+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:48.487704+0000","last_clean_scrub_stamp":"2024-05-08T18:08:48.487704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:10:35.537762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.849059+0000","last_change":"2024-05-08T18:08:11.849101+0000","last_active":"2024-05-08T18:08:11.849059+0000","last_peered":"2024-05-08T18:08:11.849059+0000","last_clean":"2024-05-08T18:08:11.849059+0000","last_became_active":"2024-05-08T18:04:47.818797+0000","last_became_peered":"2024-05-08T18:04:47.818797+0000","last_unstale":"2024-05-08T18:08:11.849059+0000","last_undegraded":"2024-05-08T18:08:11.849059+0000","last_fullsized":"2024-05-08T18:08:11.849059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:11.848881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:11.848881+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.848881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:19.374213+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"29'37","reported_seq":92,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.217059+0000","last_change":"2024-05-08T18:08:12.217092+0000","last_active":"2024-05-08T18:08:12.217059+0000","last_peered":"2024-05-08T18:08:12.217059+0000","last_clean":"2024-05-08T18:08:12.217059+0000","last_became_active":"2024-05-08T18:04:49.831379+0000","last_became_peered":"2024-05-08T18:04:49.831379+0000","last_unstale":"2024-05-08T18:08:12.217059+0000","last_undegraded":"2024-05-08T18:08:12.217059+0000","last_fullsized":"2024-05-08T18:08:12.217059+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'37","last_scrub_stamp":"2024-05-08T18:08:12.216965+0000","last_deep_scrub":"29'37","last_deep_scrub_stamp":"2024-05-08T18:08:12.216965+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.216965+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:49:34.759465+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"29'285","reported_seq":453,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.529055+0000","last_change":"2024-05-08T18:08:11.529087+0000","last_active":"2024-05-08T18:08:11.529055+0000","last_peered":"2024-05-08T18:08:11.529055+0000","last_clean":"2024-05-08T18:08:11.529055+0000","last_became_active":"2024-05-08T18:04:51.826926+0000","last_became_peered":"2024-05-08T18:04:51.826926+0000","last_unstale":"2024-05-08T18:08:11.529055+0000","last_undegraded":"2024-05-08T18:08:11.529055+0000","last_fullsized":"2024-05-08T18:08:11.529055+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:11.528959+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:11.528959+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.528959+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:44.650679+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'6","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:42.053554+0000","last_change":"2024-05-08T18:08:42.053554+0000","last_active":"2024-05-08T18:08:42.053554+0000","last_peered":"2024-05-08T18:08:42.053554+0000","last_clean":"2024-05-08T18:08:42.053554+0000","last_became_active":"2024-05-08T18:04:45.827351+0000","last_became_peered":"2024-05-08T18:04:45.827351+0000","last_unstale":"2024-05-08T18:08:42.053554+0000","last_undegraded":"2024-05-08T18:08:42.053554+0000","last_fullsized":"2024-05-08T18:08:42.053554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-05-08T18:08:42.053430+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-05-08T18:08:42.053430+0000","last_clean_scrub_stamp":"2024-05-08T18:08:42.053430+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:29.385518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:47.984304+0000","last_change":"2024-05-08T18:08:47.984304+0000","last_active":"2024-05-08T18:08:47.984304+0000","last_peered":"2024-05-08T18:08:47.984304+0000","last_clean":"2024-05-08T18:08:47.984304+0000","last_became_active":"2024-05-08T18:04:43.838840+0000","last_became_peered":"2024-05-08T18:04:43.838840+0000","last_unstale":"2024-05-08T18:08:47.984304+0000","last_undegraded":"2024-05-08T18:08:47.984304+0000","last_fullsized":"2024-05-08T18:08:47.984304+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:47.984130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:47.984130+0000","last_clean_scrub_stamp":"2024-05-08T18:08:47.984130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:31.557621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,0],"acting":[2,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.468441+0000","last_change":"2024-05-08T18:08:14.468441+0000","last_active":"2024-05-08T18:08:14.468441+0000","last_peered":"2024-05-08T18:08:14.468441+0000","last_clean":"2024-05-08T18:08:14.468441+0000","last_became_active":"2024-05-08T18:04:47.825215+0000","last_became_peered":"2024-05-08T18:04:47.825215+0000","last_unstale":"2024-05-08T18:08:14.468441+0000","last_undegraded":"2024-05-08T18:08:14.468441+0000","last_fullsized":"2024-05-08T18:08:14.468441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:14.468285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:14.468285+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.468285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:38.045371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"29'51","reported_seq":112,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.277853+0000","last_change":"2024-05-08T18:08:14.277853+0000","last_active":"2024-05-08T18:08:14.277853+0000","last_peered":"2024-05-08T18:08:14.277853+0000","last_clean":"2024-05-08T18:08:14.277853+0000","last_became_active":"2024-05-08T18:04:49.827954+0000","last_became_peered":"2024-05-08T18:04:49.827954+0000","last_unstale":"2024-05-08T18:08:14.277853+0000","last_undegraded":"2024-05-08T18:08:14.277853+0000","last_fullsized":"2024-05-08T18:08:14.277853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'51","last_scrub_stamp":"2024-05-08T18:08:14.277647+0000","last_deep_scrub":"29'51","last_deep_scrub_stamp":"2024-05-08T18:08:14.277647+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.277647+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:23:20.826582+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.436758+0000","last_change":"2024-05-08T18:08:07.436758+0000","last_active":"2024-05-08T18:08:07.436758+0000","last_peered":"2024-05-08T18:08:07.436758+0000","last_clean":"2024-05-08T18:08:07.436758+0000","last_became_active":"2024-05-08T18:04:51.831149+0000","last_became_peered":"2024-05-08T18:04:51.831149+0000","last_unstale":"2024-05-08T18:08:07.436758+0000","last_undegraded":"2024-05-08T18:08:07.436758+0000","last_fullsized":"2024-05-08T18:08:07.436758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:07.436627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:07.436627+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.436627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:56:38.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"29'5","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:55.013107+0000","last_change":"2024-05-08T18:08:55.013107+0000","last_active":"2024-05-08T18:08:55.013107+0000","last_peered":"2024-05-08T18:08:55.013107+0000","last_clean":"2024-05-08T18:08:55.013107+0000","last_became_active":"2024-05-08T18:04:53.840652+0000","last_became_peered":"2024-05-08T18:04:53.840652+0000","last_unstale":"2024-05-08T18:08:55.013107+0000","last_undegraded":"2024-05-08T18:08:55.013107+0000","last_fullsized":"2024-05-08T18:08:55.013107+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'5","last_scrub_stamp":"2024-05-08T18:08:55.012991+0000","last_deep_scrub":"29'5","last_deep_scrub_stamp":"2024-05-08T18:08:55.012991+0000","last_clean_scrub_stamp":"2024-05-08T18:08:55.012991+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:55.220779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"27'4","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:38.499721+0000","last_change":"2024-05-08T18:08:38.499721+0000","last_active":"2024-05-08T18:08:38.499721+0000","last_peered":"2024-05-08T18:08:38.499721+0000","last_clean":"2024-05-08T18:08:38.499721+0000","last_became_active":"2024-05-08T18:04:45.804124+0000","last_became_peered":"2024-05-08T18:04:45.804124+0000","last_unstale":"2024-05-08T18:08:38.499721+0000","last_undegraded":"2024-05-08T18:08:38.499721+0000","last_fullsized":"2024-05-08T18:08:38.499721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-08T18:08:38.499539+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-08T18:08:38.499539+0000","last_clean_scrub_stamp":"2024-05-08T18:08:38.499539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:58.286733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:36.190511+0000","last_change":"2024-05-08T18:08:36.190511+0000","last_active":"2024-05-08T18:08:36.190511+0000","last_peered":"2024-05-08T18:08:36.190511+0000","last_clean":"2024-05-08T18:08:36.190511+0000","last_became_active":"2024-05-08T18:04:43.842126+0000","last_became_peered":"2024-05-08T18:04:43.842126+0000","last_unstale":"2024-05-08T18:08:36.190511+0000","last_undegraded":"2024-05-08T18:08:36.190511+0000","last_fullsized":"2024-05-08T18:08:36.190511+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:36.190391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:36.190391+0000","last_clean_scrub_stamp":"2024-05-08T18:08:36.190391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:10:38.441913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.583963+0000","last_change":"2024-05-08T18:08:22.583963+0000","last_active":"2024-05-08T18:08:22.583963+0000","last_peered":"2024-05-08T18:08:22.583963+0000","last_clean":"2024-05-08T18:08:22.583963+0000","last_became_active":"2024-05-08T18:04:47.822688+0000","last_became_peered":"2024-05-08T18:04:47.822688+0000","last_unstale":"2024-05-08T18:08:22.583963+0000","last_undegraded":"2024-05-08T18:08:22.583963+0000","last_fullsized":"2024-05-08T18:08:22.583963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.583822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.583822+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.583822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:22.094540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"29'36","reported_seq":92,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.839747+0000","last_change":"2024-05-08T18:08:16.839747+0000","last_active":"2024-05-08T18:08:16.839747+0000","last_peered":"2024-05-08T18:08:16.839747+0000","last_clean":"2024-05-08T18:08:16.839747+0000","last_became_active":"2024-05-08T18:04:49.829892+0000","last_became_peered":"2024-05-08T18:04:49.829892+0000","last_unstale":"2024-05-08T18:08:16.839747+0000","last_undegraded":"2024-05-08T18:08:16.839747+0000","last_fullsized":"2024-05-08T18:08:16.839747+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:08:16.839580+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:08:16.839580+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.839580+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:53.387986+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:04.455804+0000","last_change":"2024-05-08T18:08:04.455804+0000","last_active":"2024-05-08T18:08:04.455804+0000","last_peered":"2024-05-08T18:08:04.455804+0000","last_clean":"2024-05-08T18:08:04.455804+0000","last_became_active":"2024-05-08T18:04:51.832810+0000","last_became_peered":"2024-05-08T18:04:51.832810+0000","last_unstale":"2024-05-08T18:08:04.455804+0000","last_undegraded":"2024-05-08T18:08:04.455804+0000","last_fullsized":"2024-05-08T18:08:04.455804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:04.455699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:04.455699+0000","last_clean_scrub_stamp":"2024-05-08T18:08:04.455699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:21.455723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"29'4","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:29.520481+0000","last_change":"2024-05-08T18:08:29.520481+0000","last_active":"2024-05-08T18:08:29.520481+0000","last_peered":"2024-05-08T18:08:29.520481+0000","last_clean":"2024-05-08T18:08:29.520481+0000","last_became_active":"2024-05-08T18:04:53.840894+0000","last_became_peered":"2024-05-08T18:04:53.840894+0000","last_unstale":"2024-05-08T18:08:29.520481+0000","last_undegraded":"2024-05-08T18:08:29.520481+0000","last_fullsized":"2024-05-08T18:08:29.520481+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:29.520364+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:29.520364+0000","last_clean_scrub_stamp":"2024-05-08T18:08:29.520364+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:53.170014+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"29'10","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:37.218777+0000","last_change":"2024-05-08T18:08:37.218777+0000","last_active":"2024-05-08T18:08:37.218777+0000","last_peered":"2024-05-08T18:08:37.218777+0000","last_clean":"2024-05-08T18:08:37.218777+0000","last_became_active":"2024-05-08T18:04:45.816817+0000","last_became_peered":"2024-05-08T18:04:45.816817+0000","last_unstale":"2024-05-08T18:08:37.218777+0000","last_undegraded":"2024-05-08T18:08:37.218777+0000","last_fullsized":"2024-05-08T18:08:37.218777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:37.218630+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:37.218630+0000","last_clean_scrub_stamp":"2024-05-08T18:08:37.218630+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:39.786476+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"29'4","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:03.027396+0000","last_change":"2024-05-08T18:09:03.027396+0000","last_active":"2024-05-08T18:09:03.027396+0000","last_peered":"2024-05-08T18:09:03.027396+0000","last_clean":"2024-05-08T18:09:03.027396+0000","last_became_active":"2024-05-08T18:04:43.838781+0000","last_became_peered":"2024-05-08T18:04:43.838781+0000","last_unstale":"2024-05-08T18:09:03.027396+0000","last_undegraded":"2024-05-08T18:09:03.027396+0000","last_fullsized":"2024-05-08T18:09:03.027396+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:09:03.027231+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:09:03.027231+0000","last_clean_scrub_stamp":"2024-05-08T18:09:03.027231+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:57:33.005457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:30.495329+0000","last_change":"2024-05-08T18:08:30.495329+0000","last_active":"2024-05-08T18:08:30.495329+0000","last_peered":"2024-05-08T18:08:30.495329+0000","last_clean":"2024-05-08T18:08:30.495329+0000","last_became_active":"2024-05-08T18:04:47.818871+0000","last_became_peered":"2024-05-08T18:04:47.818871+0000","last_unstale":"2024-05-08T18:08:30.495329+0000","last_undegraded":"2024-05-08T18:08:30.495329+0000","last_fullsized":"2024-05-08T18:08:30.495329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:30.495168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:30.495168+0000","last_clean_scrub_stamp":"2024-05-08T18:08:30.495168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:26.778212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"29'45","reported_seq":104,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.892899+0000","last_change":"2024-05-08T18:08:08.892899+0000","last_active":"2024-05-08T18:08:08.892899+0000","last_peered":"2024-05-08T18:08:08.892899+0000","last_clean":"2024-05-08T18:08:08.892899+0000","last_became_active":"2024-05-08T18:04:49.829270+0000","last_became_peered":"2024-05-08T18:04:49.829270+0000","last_unstale":"2024-05-08T18:08:08.892899+0000","last_undegraded":"2024-05-08T18:08:08.892899+0000","last_fullsized":"2024-05-08T18:08:08.892899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'45","last_scrub_stamp":"2024-05-08T18:08:08.892763+0000","last_deep_scrub":"29'45","last_deep_scrub_stamp":"2024-05-08T18:08:08.892763+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.892763+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:11.460610+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"29'285","reported_seq":361,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.576007+0000","last_change":"2024-05-08T18:08:13.576007+0000","last_active":"2024-05-08T18:08:13.576007+0000","last_peered":"2024-05-08T18:08:13.576007+0000","last_clean":"2024-05-08T18:08:13.576007+0000","last_became_active":"2024-05-08T18:04:51.844529+0000","last_became_peered":"2024-05-08T18:04:51.844529+0000","last_unstale":"2024-05-08T18:08:13.576007+0000","last_undegraded":"2024-05-08T18:08:13.576007+0000","last_fullsized":"2024-05-08T18:08:13.576007+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:13.575886+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:13.575886+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.575886+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:46.724169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"29'2","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:01.474503+0000","last_change":"2024-05-08T18:09:01.474503+0000","last_active":"2024-05-08T18:09:01.474503+0000","last_peered":"2024-05-08T18:09:01.474503+0000","last_clean":"2024-05-08T18:09:01.474503+0000","last_became_active":"2024-05-08T18:04:53.840767+0000","last_became_peered":"2024-05-08T18:04:53.840767+0000","last_unstale":"2024-05-08T18:09:01.474503+0000","last_undegraded":"2024-05-08T18:09:01.474503+0000","last_fullsized":"2024-05-08T18:09:01.474503+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:09:01.474327+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:09:01.474327+0000","last_clean_scrub_stamp":"2024-05-08T18:09:01.474327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:50.275478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"29'6","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:35.195855+0000","last_change":"2024-05-08T18:08:35.195855+0000","last_active":"2024-05-08T18:08:35.195855+0000","last_peered":"2024-05-08T18:08:35.195855+0000","last_clean":"2024-05-08T18:08:35.195855+0000","last_became_active":"2024-05-08T18:04:45.813735+0000","last_became_peered":"2024-05-08T18:04:45.813735+0000","last_unstale":"2024-05-08T18:08:35.195855+0000","last_undegraded":"2024-05-08T18:08:35.195855+0000","last_fullsized":"2024-05-08T18:08:35.195855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:35.195741+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:35.195741+0000","last_clean_scrub_stamp":"2024-05-08T18:08:35.195741+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:44:58.936221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"27'4","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:56.007674+0000","last_change":"2024-05-08T18:08:56.007674+0000","last_active":"2024-05-08T18:08:56.007674+0000","last_peered":"2024-05-08T18:08:56.007674+0000","last_clean":"2024-05-08T18:08:56.007674+0000","last_became_active":"2024-05-08T18:04:43.839047+0000","last_became_peered":"2024-05-08T18:04:43.839047+0000","last_unstale":"2024-05-08T18:08:56.007674+0000","last_undegraded":"2024-05-08T18:08:56.007674+0000","last_fullsized":"2024-05-08T18:08:56.007674+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-08T18:08:56.007537+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-08T18:08:56.007537+0000","last_clean_scrub_stamp":"2024-05-08T18:08:56.007537+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:37.060769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:28.472766+0000","last_change":"2024-05-08T18:08:28.472766+0000","last_active":"2024-05-08T18:08:28.472766+0000","last_peered":"2024-05-08T18:08:28.472766+0000","last_clean":"2024-05-08T18:08:28.472766+0000","last_became_active":"2024-05-08T18:04:47.818972+0000","last_became_peered":"2024-05-08T18:04:47.818972+0000","last_unstale":"2024-05-08T18:08:28.472766+0000","last_undegraded":"2024-05-08T18:08:28.472766+0000","last_fullsized":"2024-05-08T18:08:28.472766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:28.472654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:28.472654+0000","last_clean_scrub_stamp":"2024-05-08T18:08:28.472654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:31.040500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"29'63","reported_seq":137,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.861715+0000","last_change":"2024-05-08T18:08:12.861715+0000","last_active":"2024-05-08T18:08:12.861715+0000","last_peered":"2024-05-08T18:08:12.861715+0000","last_clean":"2024-05-08T18:08:12.861715+0000","last_became_active":"2024-05-08T18:04:49.830424+0000","last_became_peered":"2024-05-08T18:04:49.830424+0000","last_unstale":"2024-05-08T18:08:12.861715+0000","last_undegraded":"2024-05-08T18:08:12.861715+0000","last_fullsized":"2024-05-08T18:08:12.861715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'63","last_scrub_stamp":"2024-05-08T18:08:12.861577+0000","last_deep_scrub":"29'63","last_deep_scrub_stamp":"2024-05-08T18:08:12.861577+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.861577+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:02.260578+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.479304+0000","last_change":"2024-05-08T18:08:10.479304+0000","last_active":"2024-05-08T18:08:10.479304+0000","last_peered":"2024-05-08T18:08:10.479304+0000","last_clean":"2024-05-08T18:08:10.479304+0000","last_became_active":"2024-05-08T18:04:51.830383+0000","last_became_peered":"2024-05-08T18:04:51.830383+0000","last_unstale":"2024-05-08T18:08:10.479304+0000","last_undegraded":"2024-05-08T18:08:10.479304+0000","last_fullsized":"2024-05-08T18:08:10.479304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:10.479187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:10.479187+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.479187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:04.216069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"29'23","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:04.511456+0000","last_change":"2024-05-08T18:09:04.511456+0000","last_active":"2024-05-08T18:09:04.511456+0000","last_peered":"2024-05-08T18:09:04.511456+0000","last_clean":"2024-05-08T18:09:04.511456+0000","last_became_active":"2024-05-08T18:04:53.847842+0000","last_became_peered":"2024-05-08T18:04:53.847842+0000","last_unstale":"2024-05-08T18:09:04.511456+0000","last_undegraded":"2024-05-08T18:09:04.511456+0000","last_fullsized":"2024-05-08T18:09:04.511456+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'23","last_scrub_stamp":"2024-05-08T18:09:04.511321+0000","last_deep_scrub":"29'23","last_deep_scrub_stamp":"2024-05-08T18:09:04.511321+0000","last_clean_scrub_stamp":"2024-05-08T18:09:04.511321+0000","objects_scrubbed":3,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:49.098612+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":48,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"29'6","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.511472+0000","last_change":"2024-05-08T18:08:23.511472+0000","last_active":"2024-05-08T18:08:23.511472+0000","last_peered":"2024-05-08T18:08:23.511472+0000","last_clean":"2024-05-08T18:08:23.511472+0000","last_became_active":"2024-05-08T18:04:45.821412+0000","last_became_peered":"2024-05-08T18:04:45.821412+0000","last_unstale":"2024-05-08T18:08:23.511472+0000","last_undegraded":"2024-05-08T18:08:23.511472+0000","last_fullsized":"2024-05-08T18:08:23.511472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:23.511346+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:23.511346+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.511346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:04.193921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:44.361833+0000","last_change":"2024-05-08T18:08:44.361833+0000","last_active":"2024-05-08T18:08:44.361833+0000","last_peered":"2024-05-08T18:08:44.361833+0000","last_clean":"2024-05-08T18:08:44.361833+0000","last_became_active":"2024-05-08T18:04:43.834723+0000","last_became_peered":"2024-05-08T18:04:43.834723+0000","last_unstale":"2024-05-08T18:08:44.361833+0000","last_undegraded":"2024-05-08T18:08:44.361833+0000","last_fullsized":"2024-05-08T18:08:44.361833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:44.361707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:44.361707+0000","last_clean_scrub_stamp":"2024-05-08T18:08:44.361707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:54.804858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.295265+0000","last_change":"2024-05-08T18:08:22.295265+0000","last_active":"2024-05-08T18:08:22.295265+0000","last_peered":"2024-05-08T18:08:22.295265+0000","last_clean":"2024-05-08T18:08:22.295265+0000","last_became_active":"2024-05-08T18:04:47.822865+0000","last_became_peered":"2024-05-08T18:04:47.822865+0000","last_unstale":"2024-05-08T18:08:22.295265+0000","last_undegraded":"2024-05-08T18:08:22.295265+0000","last_fullsized":"2024-05-08T18:08:22.295265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.295113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.295113+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.295113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:31:07.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"29'40","reported_seq":97,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.481605+0000","last_change":"2024-05-08T18:08:11.481605+0000","last_active":"2024-05-08T18:08:11.481605+0000","last_peered":"2024-05-08T18:08:11.481605+0000","last_clean":"2024-05-08T18:08:11.481605+0000","last_became_active":"2024-05-08T18:04:49.824610+0000","last_became_peered":"2024-05-08T18:04:49.824610+0000","last_unstale":"2024-05-08T18:08:11.481605+0000","last_undegraded":"2024-05-08T18:08:11.481605+0000","last_fullsized":"2024-05-08T18:08:11.481605+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'40","last_scrub_stamp":"2024-05-08T18:08:11.481461+0000","last_deep_scrub":"29'40","last_deep_scrub_stamp":"2024-05-08T18:08:11.481461+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.481461+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:15.729540+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:28.829264+0000","last_change":"2024-05-08T18:08:28.829264+0000","last_active":"2024-05-08T18:08:28.829264+0000","last_peered":"2024-05-08T18:08:28.829264+0000","last_clean":"2024-05-08T18:08:28.829264+0000","last_became_active":"2024-05-08T18:04:51.848793+0000","last_became_peered":"2024-05-08T18:04:51.848793+0000","last_unstale":"2024-05-08T18:08:28.829264+0000","last_undegraded":"2024-05-08T18:08:28.829264+0000","last_fullsized":"2024-05-08T18:08:28.829264+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:28.829111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:28.829111+0000","last_clean_scrub_stamp":"2024-05-08T18:08:28.829111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:35.508902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"29'5","reported_seq":24,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.082276+0000","last_change":"2024-05-08T18:04:53.849242+0000","last_active":"2024-05-08T18:08:05.082276+0000","last_peered":"2024-05-08T18:08:05.082276+0000","last_clean":"2024-05-08T18:08:05.082276+0000","last_became_active":"2024-05-08T18:04:53.848945+0000","last_became_peered":"2024-05-08T18:04:53.848945+0000","last_unstale":"2024-05-08T18:08:05.082276+0000","last_undegraded":"2024-05-08T18:08:05.082276+0000","last_fullsized":"2024-05-08T18:08:05.082276+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"29'10","reported_seq":85,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.836220+0000","last_change":"2024-05-08T18:08:23.836220+0000","last_active":"2024-05-08T18:08:23.836220+0000","last_peered":"2024-05-08T18:08:23.836220+0000","last_clean":"2024-05-08T18:08:23.836220+0000","last_became_active":"2024-05-08T18:04:45.829120+0000","last_became_peered":"2024-05-08T18:04:45.829120+0000","last_unstale":"2024-05-08T18:08:23.836220+0000","last_undegraded":"2024-05-08T18:08:23.836220+0000","last_fullsized":"2024-05-08T18:08:23.836220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:23.836088+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:23.836088+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.836088+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:15.906999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":57,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:09.962858+0000","last_change":"2024-05-08T18:09:09.962886+0000","last_active":"2024-05-08T18:09:09.962858+0000","last_peered":"2024-05-08T18:09:09.962858+0000","last_clean":"2024-05-08T18:09:09.962858+0000","last_became_active":"2024-05-08T18:04:43.834536+0000","last_became_peered":"2024-05-08T18:04:43.834536+0000","last_unstale":"2024-05-08T18:09:09.962858+0000","last_undegraded":"2024-05-08T18:09:09.962858+0000","last_fullsized":"2024-05-08T18:09:09.962858+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:09.962738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:09.962738+0000","last_clean_scrub_stamp":"2024-05-08T18:09:09.962738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:14.961834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"29'5","reported_seq":239,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:26.539766+0000","last_change":"2024-05-08T18:08:26.539766+0000","last_active":"2024-05-08T18:08:26.539766+0000","last_peered":"2024-05-08T18:08:26.539766+0000","last_clean":"2024-05-08T18:08:26.539766+0000","last_became_active":"2024-05-08T18:04:47.819085+0000","last_became_peered":"2024-05-08T18:04:47.819085+0000","last_unstale":"2024-05-08T18:08:26.539766+0000","last_undegraded":"2024-05-08T18:08:26.539766+0000","last_fullsized":"2024-05-08T18:08:26.539766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'5","last_scrub_stamp":"2024-05-08T18:08:26.539641+0000","last_deep_scrub":"29'5","last_deep_scrub_stamp":"2024-05-08T18:08:26.539641+0000","last_clean_scrub_stamp":"2024-05-08T18:08:26.539641+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:50.189291+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":301,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"29'45","reported_seq":104,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.280897+0000","last_change":"2024-05-08T18:08:16.280897+0000","last_active":"2024-05-08T18:08:16.280897+0000","last_peered":"2024-05-08T18:08:16.280897+0000","last_clean":"2024-05-08T18:08:16.280897+0000","last_became_active":"2024-05-08T18:04:49.825613+0000","last_became_peered":"2024-05-08T18:04:49.825613+0000","last_unstale":"2024-05-08T18:08:16.280897+0000","last_undegraded":"2024-05-08T18:08:16.280897+0000","last_fullsized":"2024-05-08T18:08:16.280897+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'45","last_scrub_stamp":"2024-05-08T18:08:16.280719+0000","last_deep_scrub":"29'45","last_deep_scrub_stamp":"2024-05-08T18:08:16.280719+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.280719+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:29:19.130403+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"29'285","reported_seq":444,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.137883+0000","last_change":"2024-05-08T18:08:06.137883+0000","last_active":"2024-05-08T18:08:06.137883+0000","last_peered":"2024-05-08T18:08:06.137883+0000","last_clean":"2024-05-08T18:08:06.137883+0000","last_became_active":"2024-05-08T18:04:51.830924+0000","last_became_peered":"2024-05-08T18:04:51.830924+0000","last_unstale":"2024-05-08T18:08:06.137883+0000","last_undegraded":"2024-05-08T18:08:06.137883+0000","last_fullsized":"2024-05-08T18:08:06.137883+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:06.137767+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:06.137767+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.137767+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:43.622614+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"29'15","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:02.458510+0000","last_change":"2024-05-08T18:09:02.458510+0000","last_active":"2024-05-08T18:09:02.458510+0000","last_peered":"2024-05-08T18:09:02.458510+0000","last_clean":"2024-05-08T18:09:02.458510+0000","last_became_active":"2024-05-08T18:04:53.840708+0000","last_became_peered":"2024-05-08T18:04:53.840708+0000","last_unstale":"2024-05-08T18:09:02.458510+0000","last_undegraded":"2024-05-08T18:09:02.458510+0000","last_fullsized":"2024-05-08T18:09:02.458510+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:09:02.458318+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:09:02.458318+0000","last_clean_scrub_stamp":"2024-05-08T18:09:02.458318+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:49.790407+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":180,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"29'6","reported_seq":64,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:31.301337+0000","last_change":"2024-05-08T18:08:31.301337+0000","last_active":"2024-05-08T18:08:31.301337+0000","last_peered":"2024-05-08T18:08:31.301337+0000","last_clean":"2024-05-08T18:08:31.301337+0000","last_became_active":"2024-05-08T18:04:45.811926+0000","last_became_peered":"2024-05-08T18:04:45.811926+0000","last_unstale":"2024-05-08T18:08:31.301337+0000","last_undegraded":"2024-05-08T18:08:31.301337+0000","last_fullsized":"2024-05-08T18:08:31.301337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:31.301216+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:31.301216+0000","last_clean_scrub_stamp":"2024-05-08T18:08:31.301216+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:32:41.104191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"29'2","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:01.003015+0000","last_change":"2024-05-08T18:09:01.003015+0000","last_active":"2024-05-08T18:09:01.003015+0000","last_peered":"2024-05-08T18:09:01.003015+0000","last_clean":"2024-05-08T18:09:01.003015+0000","last_became_active":"2024-05-08T18:04:43.843934+0000","last_became_peered":"2024-05-08T18:04:43.843934+0000","last_unstale":"2024-05-08T18:09:01.003015+0000","last_undegraded":"2024-05-08T18:09:01.003015+0000","last_fullsized":"2024-05-08T18:09:01.003015+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:09:01.002893+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:09:01.002893+0000","last_clean_scrub_stamp":"2024-05-08T18:09:01.002893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:41:23.689098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"29'5","reported_seq":429,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.550071+0000","last_change":"2024-05-08T18:08:20.550071+0000","last_active":"2024-05-08T18:08:20.550071+0000","last_peered":"2024-05-08T18:08:20.550071+0000","last_clean":"2024-05-08T18:08:20.550071+0000","last_became_active":"2024-05-08T18:04:47.817851+0000","last_became_peered":"2024-05-08T18:04:47.817851+0000","last_unstale":"2024-05-08T18:08:20.550071+0000","last_undegraded":"2024-05-08T18:08:20.550071+0000","last_fullsized":"2024-05-08T18:08:20.550071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'5","last_scrub_stamp":"2024-05-08T18:08:20.549943+0000","last_deep_scrub":"29'5","last_deep_scrub_stamp":"2024-05-08T18:08:20.549943+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.549943+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:31.792211+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"29'15","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.800470+0000","last_change":"2024-05-08T18:08:18.800470+0000","last_active":"2024-05-08T18:08:18.800470+0000","last_peered":"2024-05-08T18:08:18.800470+0000","last_clean":"2024-05-08T18:08:18.800470+0000","last_became_active":"2024-05-08T18:04:49.830714+0000","last_became_peered":"2024-05-08T18:04:49.830714+0000","last_unstale":"2024-05-08T18:08:18.800470+0000","last_undegraded":"2024-05-08T18:08:18.800470+0000","last_fullsized":"2024-05-08T18:08:18.800470+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:18.800324+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:18.800324+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.800324+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:41.821593+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"29'285","reported_seq":481,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.542295+0000","last_change":"2024-05-08T18:08:05.542295+0000","last_active":"2024-05-08T18:08:05.542295+0000","last_peered":"2024-05-08T18:08:05.542295+0000","last_clean":"2024-05-08T18:08:05.542295+0000","last_became_active":"2024-05-08T18:04:51.831508+0000","last_became_peered":"2024-05-08T18:04:51.831508+0000","last_unstale":"2024-05-08T18:08:05.542295+0000","last_undegraded":"2024-05-08T18:08:05.542295+0000","last_fullsized":"2024-05-08T18:08:05.542295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:05.542169+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:05.542169+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.542169+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:49:28.084669+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'4","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:52.956968+0000","last_change":"2024-05-08T18:08:52.956968+0000","last_active":"2024-05-08T18:08:52.956968+0000","last_peered":"2024-05-08T18:08:52.956968+0000","last_clean":"2024-05-08T18:08:52.956968+0000","last_became_active":"2024-05-08T18:04:53.840215+0000","last_became_peered":"2024-05-08T18:04:53.840215+0000","last_unstale":"2024-05-08T18:08:52.956968+0000","last_undegraded":"2024-05-08T18:08:52.956968+0000","last_fullsized":"2024-05-08T18:08:52.956968+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:52.956850+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:52.956850+0000","last_clean_scrub_stamp":"2024-05-08T18:08:52.956850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:44:16.337663+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'2","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:30.526084+0000","last_change":"2024-05-08T18:08:30.526084+0000","last_active":"2024-05-08T18:08:30.526084+0000","last_peered":"2024-05-08T18:08:30.526084+0000","last_clean":"2024-05-08T18:08:30.526084+0000","last_became_active":"2024-05-08T18:04:45.818533+0000","last_became_peered":"2024-05-08T18:04:45.818533+0000","last_unstale":"2024-05-08T18:08:30.526084+0000","last_undegraded":"2024-05-08T18:08:30.526084+0000","last_fullsized":"2024-05-08T18:08:30.526084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:30.525970+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:30.525970+0000","last_clean_scrub_stamp":"2024-05-08T18:08:30.525970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:27.414733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"29'2","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:55.032953+0000","last_change":"2024-05-08T18:08:55.032953+0000","last_active":"2024-05-08T18:08:55.032953+0000","last_peered":"2024-05-08T18:08:55.032953+0000","last_clean":"2024-05-08T18:08:55.032953+0000","last_became_active":"2024-05-08T18:04:43.846125+0000","last_became_peered":"2024-05-08T18:04:43.846125+0000","last_unstale":"2024-05-08T18:08:55.032953+0000","last_undegraded":"2024-05-08T18:08:55.032953+0000","last_fullsized":"2024-05-08T18:08:55.032953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:55.032817+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:55.032817+0000","last_clean_scrub_stamp":"2024-05-08T18:08:55.032817+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:29:54.513234+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.895821+0000","last_change":"2024-05-08T18:08:05.895821+0000","last_active":"2024-05-08T18:08:05.895821+0000","last_peered":"2024-05-08T18:08:05.895821+0000","last_clean":"2024-05-08T18:08:05.895821+0000","last_became_active":"2024-05-08T18:04:47.822242+0000","last_became_peered":"2024-05-08T18:04:47.822242+0000","last_unstale":"2024-05-08T18:08:05.895821+0000","last_undegraded":"2024-05-08T18:08:05.895821+0000","last_fullsized":"2024-05-08T18:08:05.895821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:05.895706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:05.895706+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.895706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:00:55.969119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"29'48","reported_seq":108,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.545419+0000","last_change":"2024-05-08T18:08:08.545419+0000","last_active":"2024-05-08T18:08:08.545419+0000","last_peered":"2024-05-08T18:08:08.545419+0000","last_clean":"2024-05-08T18:08:08.545419+0000","last_became_active":"2024-05-08T18:04:49.825314+0000","last_became_peered":"2024-05-08T18:04:49.825314+0000","last_unstale":"2024-05-08T18:08:08.545419+0000","last_undegraded":"2024-05-08T18:08:08.545419+0000","last_fullsized":"2024-05-08T18:08:08.545419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'48","last_scrub_stamp":"2024-05-08T18:08:08.545269+0000","last_deep_scrub":"29'48","last_deep_scrub_stamp":"2024-05-08T18:08:08.545269+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.545269+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:19.079352+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.136723+0000","last_change":"2024-05-08T18:08:05.136723+0000","last_active":"2024-05-08T18:08:05.136723+0000","last_peered":"2024-05-08T18:08:05.136723+0000","last_clean":"2024-05-08T18:08:05.136723+0000","last_became_active":"2024-05-08T18:04:51.832808+0000","last_became_peered":"2024-05-08T18:04:51.832808+0000","last_unstale":"2024-05-08T18:08:05.136723+0000","last_undegraded":"2024-05-08T18:08:05.136723+0000","last_fullsized":"2024-05-08T18:08:05.136723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:05.136592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:05.136592+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.136592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:30.377482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"29'9","reported_seq":37,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:25.506016+0000","last_change":"2024-05-08T18:08:25.506016+0000","last_active":"2024-05-08T18:08:25.506016+0000","last_peered":"2024-05-08T18:08:25.506016+0000","last_clean":"2024-05-08T18:08:25.506016+0000","last_became_active":"2024-05-08T18:04:53.837292+0000","last_became_peered":"2024-05-08T18:04:53.837292+0000","last_unstale":"2024-05-08T18:08:25.506016+0000","last_undegraded":"2024-05-08T18:08:25.506016+0000","last_fullsized":"2024-05-08T18:08:25.506016+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'9","last_scrub_stamp":"2024-05-08T18:08:25.505900+0000","last_deep_scrub":"29'9","last_deep_scrub_stamp":"2024-05-08T18:08:25.505900+0000","last_clean_scrub_stamp":"2024-05-08T18:08:25.505900+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:25.016619+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'4","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:25.876183+0000","last_change":"2024-05-08T18:08:25.876183+0000","last_active":"2024-05-08T18:08:25.876183+0000","last_peered":"2024-05-08T18:08:25.876183+0000","last_clean":"2024-05-08T18:08:25.876183+0000","last_became_active":"2024-05-08T18:04:45.828674+0000","last_became_peered":"2024-05-08T18:04:45.828674+0000","last_unstale":"2024-05-08T18:08:25.876183+0000","last_undegraded":"2024-05-08T18:08:25.876183+0000","last_fullsized":"2024-05-08T18:08:25.876183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:25.876052+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:25.876052+0000","last_clean_scrub_stamp":"2024-05-08T18:08:25.876052+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:30:56.782177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'74","reported_seq":149,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:51.419922+0000","last_change":"2024-05-08T18:08:51.419959+0000","last_active":"2024-05-08T18:08:51.419922+0000","last_peered":"2024-05-08T18:08:51.419922+0000","last_clean":"2024-05-08T18:08:51.419922+0000","last_became_active":"2024-05-08T18:04:35.551437+0000","last_became_peered":"2024-05-08T18:04:35.551437+0000","last_unstale":"2024-05-08T18:08:51.419922+0000","last_undegraded":"2024-05-08T18:08:51.419922+0000","last_fullsized":"2024-05-08T18:08:51.419922+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'74","last_scrub_stamp":"2024-05-08T18:08:51.419824+0000","last_deep_scrub":"29'74","last_deep_scrub_stamp":"2024-05-08T18:08:51.419824+0000","last_clean_scrub_stamp":"2024-05-08T18:08:51.419824+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:39:27.870934+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:18.009610+0000","last_change":"2024-05-08T18:09:18.009637+0000","last_active":"2024-05-08T18:09:18.009610+0000","last_peered":"2024-05-08T18:09:18.009610+0000","last_clean":"2024-05-08T18:09:18.009610+0000","last_became_active":"2024-05-08T18:04:35.552017+0000","last_became_peered":"2024-05-08T18:04:35.552017+0000","last_unstale":"2024-05-08T18:09:18.009610+0000","last_undegraded":"2024-05-08T18:09:18.009610+0000","last_fullsized":"2024-05-08T18:09:18.009610+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:18.009505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:18.009505+0000","last_clean_scrub_stamp":"2024-05-08T18:09:18.009505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:38:44.258938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:04.006356+0000","last_change":"2024-05-08T18:09:04.006401+0000","last_active":"2024-05-08T18:09:04.006356+0000","last_peered":"2024-05-08T18:09:04.006356+0000","last_clean":"2024-05-08T18:09:04.006356+0000","last_became_active":"2024-05-08T18:04:43.837019+0000","last_became_peered":"2024-05-08T18:04:43.837019+0000","last_unstale":"2024-05-08T18:09:04.006356+0000","last_undegraded":"2024-05-08T18:09:04.006356+0000","last_fullsized":"2024-05-08T18:09:04.006356+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:04.006222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:04.006222+0000","last_clean_scrub_stamp":"2024-05-08T18:09:04.006222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:36.210550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:06.524219+0000","last_change":"2024-05-08T18:09:06.524244+0000","last_active":"2024-05-08T18:09:06.524219+0000","last_peered":"2024-05-08T18:09:06.524219+0000","last_clean":"2024-05-08T18:09:06.524219+0000","last_became_active":"2024-05-08T18:05:30.870647+0000","last_became_peered":"2024-05-08T18:05:30.870647+0000","last_unstale":"2024-05-08T18:09:06.524219+0000","last_undegraded":"2024-05-08T18:09:06.524219+0000","last_fullsized":"2024-05-08T18:09:06.524219+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:06.524126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:06.524126+0000","last_clean_scrub_stamp":"2024-05-08T18:09:06.524126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:53.856214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"29'91","reported_seq":182,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:34.946658+0000","last_change":"2024-05-08T18:08:34.946658+0000","last_active":"2024-05-08T18:08:34.946658+0000","last_peered":"2024-05-08T18:08:34.946658+0000","last_clean":"2024-05-08T18:08:34.946658+0000","last_became_active":"2024-05-08T18:04:49.830636+0000","last_became_peered":"2024-05-08T18:04:49.830636+0000","last_unstale":"2024-05-08T18:08:34.946658+0000","last_undegraded":"2024-05-08T18:08:34.946658+0000","last_fullsized":"2024-05-08T18:08:34.946658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'91","last_scrub_stamp":"2024-05-08T18:08:34.946526+0000","last_deep_scrub":"29'91","last_deep_scrub_stamp":"2024-05-08T18:08:34.946526+0000","last_clean_scrub_stamp":"2024-05-08T18:08:34.946526+0000","objects_scrubbed":5,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:49.081723+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.600230+0000","last_change":"2024-05-08T18:08:21.600230+0000","last_active":"2024-05-08T18:08:21.600230+0000","last_peered":"2024-05-08T18:08:21.600230+0000","last_clean":"2024-05-08T18:08:21.600230+0000","last_became_active":"2024-05-08T18:04:51.844737+0000","last_became_peered":"2024-05-08T18:04:51.844737+0000","last_unstale":"2024-05-08T18:08:21.600230+0000","last_undegraded":"2024-05-08T18:08:21.600230+0000","last_fullsized":"2024-05-08T18:08:21.600230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:21.600069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:21.600069+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.600069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:48.902927+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"29'86","reported_seq":177,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:25.255820+0000","last_change":"2024-05-08T18:08:25.255820+0000","last_active":"2024-05-08T18:08:25.255820+0000","last_peered":"2024-05-08T18:08:25.255820+0000","last_clean":"2024-05-08T18:08:25.255820+0000","last_became_active":"2024-05-08T18:04:53.838957+0000","last_became_peered":"2024-05-08T18:04:53.838957+0000","last_unstale":"2024-05-08T18:08:25.255820+0000","last_undegraded":"2024-05-08T18:08:25.255820+0000","last_fullsized":"2024-05-08T18:08:25.255820+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'86","last_scrub_stamp":"2024-05-08T18:08:25.255680+0000","last_deep_scrub":"29'86","last_deep_scrub_stamp":"2024-05-08T18:08:25.255680+0000","last_clean_scrub_stamp":"2024-05-08T18:08:25.255680+0000","objects_scrubbed":1,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:14:28.008294+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":216,"num_read_kb":214,"num_write":167,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"29'10","reported_seq":76,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:48.083623+0000","last_change":"2024-05-08T18:08:48.083623+0000","last_active":"2024-05-08T18:08:48.083623+0000","last_peered":"2024-05-08T18:08:48.083623+0000","last_clean":"2024-05-08T18:08:48.083623+0000","last_became_active":"2024-05-08T18:04:45.825843+0000","last_became_peered":"2024-05-08T18:04:45.825843+0000","last_unstale":"2024-05-08T18:08:48.083623+0000","last_undegraded":"2024-05-08T18:08:48.083623+0000","last_fullsized":"2024-05-08T18:08:48.083623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:48.083503+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:48.083503+0000","last_clean_scrub_stamp":"2024-05-08T18:08:48.083503+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:05:27.160237+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:18.964836+0000","last_change":"2024-05-08T18:09:18.964862+0000","last_active":"2024-05-08T18:09:18.964836+0000","last_peered":"2024-05-08T18:09:18.964836+0000","last_clean":"2024-05-08T18:09:18.964836+0000","last_became_active":"2024-05-08T18:04:35.552214+0000","last_became_peered":"2024-05-08T18:04:35.552214+0000","last_unstale":"2024-05-08T18:09:18.964836+0000","last_undegraded":"2024-05-08T18:09:18.964836+0000","last_fullsized":"2024-05-08T18:09:18.964836+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:18.964742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:18.964742+0000","last_clean_scrub_stamp":"2024-05-08T18:09:18.964742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:32.307203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:54.429770+0000","last_change":"2024-05-08T18:08:54.429815+0000","last_active":"2024-05-08T18:08:54.429770+0000","last_peered":"2024-05-08T18:08:54.429770+0000","last_clean":"2024-05-08T18:08:54.429770+0000","last_became_active":"2024-05-08T18:04:43.820369+0000","last_became_peered":"2024-05-08T18:04:43.820369+0000","last_unstale":"2024-05-08T18:08:54.429770+0000","last_undegraded":"2024-05-08T18:08:54.429770+0000","last_fullsized":"2024-05-08T18:08:54.429770+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:54.429672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:54.429672+0000","last_clean_scrub_stamp":"2024-05-08T18:08:54.429672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:12.348107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.081628+0000","last_change":"2024-05-08T18:05:30.870035+0000","last_active":"2024-05-08T18:08:05.081628+0000","last_peered":"2024-05-08T18:08:05.081628+0000","last_clean":"2024-05-08T18:08:05.081628+0000","last_became_active":"2024-05-08T18:05:30.869578+0000","last_became_peered":"2024-05-08T18:05:30.869578+0000","last_unstale":"2024-05-08T18:08:05.081628+0000","last_undegraded":"2024-05-08T18:08:05.081628+0000","last_fullsized":"2024-05-08T18:08:05.081628+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:46.796744+0000","last_clean_scrub_stamp":"2024-05-08T18:04:46.796744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"29'290","reported_seq":516,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.566330+0000","last_change":"2024-05-08T18:08:07.566330+0000","last_active":"2024-05-08T18:08:07.566330+0000","last_peered":"2024-05-08T18:08:07.566330+0000","last_clean":"2024-05-08T18:08:07.566330+0000","last_became_active":"2024-05-08T18:04:49.833053+0000","last_became_peered":"2024-05-08T18:04:49.833053+0000","last_unstale":"2024-05-08T18:08:07.566330+0000","last_undegraded":"2024-05-08T18:08:07.566330+0000","last_fullsized":"2024-05-08T18:08:07.566330+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'290","last_scrub_stamp":"2024-05-08T18:08:07.566185+0000","last_deep_scrub":"29'290","last_deep_scrub_stamp":"2024-05-08T18:08:07.566185+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.566185+0000","objects_scrubbed":10,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:24.566210+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":378,"num_read_kb":363,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:32.883526+0000","last_change":"2024-05-08T18:08:32.883526+0000","last_active":"2024-05-08T18:08:32.883526+0000","last_peered":"2024-05-08T18:08:32.883526+0000","last_clean":"2024-05-08T18:08:32.883526+0000","last_became_active":"2024-05-08T18:04:51.848066+0000","last_became_peered":"2024-05-08T18:04:51.848066+0000","last_unstale":"2024-05-08T18:08:32.883526+0000","last_undegraded":"2024-05-08T18:08:32.883526+0000","last_fullsized":"2024-05-08T18:08:32.883526+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:32.883401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:32.883401+0000","last_clean_scrub_stamp":"2024-05-08T18:08:32.883401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:44.412793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"29'3","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:00.515445+0000","last_change":"2024-05-08T18:09:00.515445+0000","last_active":"2024-05-08T18:09:00.515445+0000","last_peered":"2024-05-08T18:09:00.515445+0000","last_clean":"2024-05-08T18:09:00.515445+0000","last_became_active":"2024-05-08T18:04:53.842899+0000","last_became_peered":"2024-05-08T18:04:53.842899+0000","last_unstale":"2024-05-08T18:09:00.515445+0000","last_undegraded":"2024-05-08T18:09:00.515445+0000","last_fullsized":"2024-05-08T18:09:00.515445+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3","last_scrub_stamp":"2024-05-08T18:09:00.515314+0000","last_deep_scrub":"29'3","last_deep_scrub_stamp":"2024-05-08T18:09:00.515314+0000","last_clean_scrub_stamp":"2024-05-08T18:09:00.515314+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:36:11.806797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"29'8","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:59.474909+0000","last_change":"2024-05-08T18:08:59.474909+0000","last_active":"2024-05-08T18:08:59.474909+0000","last_peered":"2024-05-08T18:08:59.474909+0000","last_clean":"2024-05-08T18:08:59.474909+0000","last_became_active":"2024-05-08T18:04:45.812091+0000","last_became_peered":"2024-05-08T18:04:45.812091+0000","last_unstale":"2024-05-08T18:08:59.474909+0000","last_undegraded":"2024-05-08T18:08:59.474909+0000","last_fullsized":"2024-05-08T18:08:59.474909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:59.474782+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:59.474782+0000","last_clean_scrub_stamp":"2024-05-08T18:08:59.474782+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:57.356263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:43.042058+0000","last_change":"2024-05-08T18:08:43.042058+0000","last_active":"2024-05-08T18:08:43.042058+0000","last_peered":"2024-05-08T18:08:43.042058+0000","last_clean":"2024-05-08T18:08:43.042058+0000","last_became_active":"2024-05-08T18:04:35.551313+0000","last_became_peered":"2024-05-08T18:04:35.551313+0000","last_unstale":"2024-05-08T18:08:43.042058+0000","last_undegraded":"2024-05-08T18:08:43.042058+0000","last_fullsized":"2024-05-08T18:08:43.042058+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:43.041942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:43.041942+0000","last_clean_scrub_stamp":"2024-05-08T18:08:43.041942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:53.026181+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:56.472181+0000","last_change":"2024-05-08T18:08:56.472209+0000","last_active":"2024-05-08T18:08:56.472181+0000","last_peered":"2024-05-08T18:08:56.472181+0000","last_clean":"2024-05-08T18:08:56.472181+0000","last_became_active":"2024-05-08T18:04:43.820335+0000","last_became_peered":"2024-05-08T18:04:43.820335+0000","last_unstale":"2024-05-08T18:08:56.472181+0000","last_undegraded":"2024-05-08T18:08:56.472181+0000","last_fullsized":"2024-05-08T18:08:56.472181+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:56.472080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:56.472080+0000","last_clean_scrub_stamp":"2024-05-08T18:08:56.472080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:36:07.763563+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"29'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.343043+0000","last_change":"2024-05-08T18:08:20.343043+0000","last_active":"2024-05-08T18:08:20.343043+0000","last_peered":"2024-05-08T18:08:20.343043+0000","last_clean":"2024-05-08T18:08:20.343043+0000","last_became_active":"2024-05-08T18:04:47.823631+0000","last_became_peered":"2024-05-08T18:04:47.823631+0000","last_unstale":"2024-05-08T18:08:20.343043+0000","last_undegraded":"2024-05-08T18:08:20.343043+0000","last_fullsized":"2024-05-08T18:08:20.343043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:20.342916+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:20.342916+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.342916+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:26:02.075842+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"29'24","reported_seq":102,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.233395+0000","last_change":"2024-05-08T18:08:15.233395+0000","last_active":"2024-05-08T18:08:15.233395+0000","last_peered":"2024-05-08T18:08:15.233395+0000","last_clean":"2024-05-08T18:08:15.233395+0000","last_became_active":"2024-05-08T18:04:49.827735+0000","last_became_peered":"2024-05-08T18:04:49.827735+0000","last_unstale":"2024-05-08T18:08:15.233395+0000","last_undegraded":"2024-05-08T18:08:15.233395+0000","last_fullsized":"2024-05-08T18:08:15.233395+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'24","last_scrub_stamp":"2024-05-08T18:08:15.233271+0000","last_deep_scrub":"29'24","last_deep_scrub_stamp":"2024-05-08T18:08:15.233271+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.233271+0000","objects_scrubbed":8,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:25.767355+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.442677+0000","last_change":"2024-05-08T18:08:09.442677+0000","last_active":"2024-05-08T18:08:09.442677+0000","last_peered":"2024-05-08T18:08:09.442677+0000","last_clean":"2024-05-08T18:08:09.442677+0000","last_became_active":"2024-05-08T18:04:51.843940+0000","last_became_peered":"2024-05-08T18:04:51.843940+0000","last_unstale":"2024-05-08T18:08:09.442677+0000","last_undegraded":"2024-05-08T18:08:09.442677+0000","last_fullsized":"2024-05-08T18:08:09.442677+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:09.442556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:09.442556+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.442556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:48:00.984508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"29'4","reported_seq":23,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.082315+0000","last_change":"2024-05-08T18:04:53.848676+0000","last_active":"2024-05-08T18:08:05.082315+0000","last_peered":"2024-05-08T18:08:05.082315+0000","last_clean":"2024-05-08T18:08:05.082315+0000","last_became_active":"2024-05-08T18:04:53.848255+0000","last_became_peered":"2024-05-08T18:04:53.848255+0000","last_unstale":"2024-05-08T18:08:05.082315+0000","last_undegraded":"2024-05-08T18:08:05.082315+0000","last_fullsized":"2024-05-08T18:08:05.082315+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"29'10","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:39.382370+0000","last_change":"2024-05-08T18:08:39.382370+0000","last_active":"2024-05-08T18:08:39.382370+0000","last_peered":"2024-05-08T18:08:39.382370+0000","last_clean":"2024-05-08T18:08:39.382370+0000","last_became_active":"2024-05-08T18:04:45.807085+0000","last_became_peered":"2024-05-08T18:04:45.807085+0000","last_unstale":"2024-05-08T18:08:39.382370+0000","last_undegraded":"2024-05-08T18:08:39.382370+0000","last_fullsized":"2024-05-08T18:08:39.382370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:39.382233+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:39.382233+0000","last_clean_scrub_stamp":"2024-05-08T18:08:39.382233+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:03.089676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:16.003737+0000","last_change":"2024-05-08T18:09:16.003761+0000","last_active":"2024-05-08T18:09:16.003737+0000","last_peered":"2024-05-08T18:09:16.003737+0000","last_clean":"2024-05-08T18:09:16.003737+0000","last_became_active":"2024-05-08T18:04:35.551120+0000","last_became_peered":"2024-05-08T18:04:35.551120+0000","last_unstale":"2024-05-08T18:09:16.003737+0000","last_undegraded":"2024-05-08T18:09:16.003737+0000","last_fullsized":"2024-05-08T18:09:16.003737+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:16.003623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:16.003623+0000","last_clean_scrub_stamp":"2024-05-08T18:09:16.003623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:42:17.695869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:48.977118+0000","last_change":"2024-05-08T18:08:48.977118+0000","last_active":"2024-05-08T18:08:48.977118+0000","last_peered":"2024-05-08T18:08:48.977118+0000","last_clean":"2024-05-08T18:08:48.977118+0000","last_became_active":"2024-05-08T18:04:43.840036+0000","last_became_peered":"2024-05-08T18:04:43.840036+0000","last_unstale":"2024-05-08T18:08:48.977118+0000","last_undegraded":"2024-05-08T18:08:48.977118+0000","last_fullsized":"2024-05-08T18:08:48.977118+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:48.976992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:48.976992+0000","last_clean_scrub_stamp":"2024-05-08T18:08:48.976992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:39:35.647911+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0,3],"acting":[2,1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.607246+0000","last_change":"2024-05-08T18:08:20.607246+0000","last_active":"2024-05-08T18:08:20.607246+0000","last_peered":"2024-05-08T18:08:20.607246+0000","last_clean":"2024-05-08T18:08:20.607246+0000","last_became_active":"2024-05-08T18:04:47.823093+0000","last_became_peered":"2024-05-08T18:04:47.823093+0000","last_unstale":"2024-05-08T18:08:20.607246+0000","last_undegraded":"2024-05-08T18:08:20.607246+0000","last_fullsized":"2024-05-08T18:08:20.607246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:20.607136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:20.607136+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.607136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:01.289711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"29'20","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:24.842630+0000","last_change":"2024-05-08T18:08:24.842630+0000","last_active":"2024-05-08T18:08:24.842630+0000","last_peered":"2024-05-08T18:08:24.842630+0000","last_clean":"2024-05-08T18:08:24.842630+0000","last_became_active":"2024-05-08T18:04:49.830586+0000","last_became_peered":"2024-05-08T18:04:49.830586+0000","last_unstale":"2024-05-08T18:08:24.842630+0000","last_undegraded":"2024-05-08T18:08:24.842630+0000","last_fullsized":"2024-05-08T18:08:24.842630+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'20","last_scrub_stamp":"2024-05-08T18:08:24.842450+0000","last_deep_scrub":"29'20","last_deep_scrub_stamp":"2024-05-08T18:08:24.842450+0000","last_clean_scrub_stamp":"2024-05-08T18:08:24.842450+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:35.285600+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":394,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:37.005716+0000","last_change":"2024-05-08T18:08:37.005716+0000","last_active":"2024-05-08T18:08:37.005716+0000","last_peered":"2024-05-08T18:08:37.005716+0000","last_clean":"2024-05-08T18:08:37.005716+0000","last_became_active":"2024-05-08T18:04:51.849137+0000","last_became_peered":"2024-05-08T18:04:51.849137+0000","last_unstale":"2024-05-08T18:08:37.005716+0000","last_undegraded":"2024-05-08T18:08:37.005716+0000","last_fullsized":"2024-05-08T18:08:37.005716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:37.005585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:37.005585+0000","last_clean_scrub_stamp":"2024-05-08T18:08:37.005585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:35:48.297068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'4","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:04.571470+0000","last_change":"2024-05-08T18:09:04.571470+0000","last_active":"2024-05-08T18:09:04.571470+0000","last_peered":"2024-05-08T18:09:04.571470+0000","last_clean":"2024-05-08T18:09:04.571470+0000","last_became_active":"2024-05-08T18:04:53.851008+0000","last_became_peered":"2024-05-08T18:04:53.851008+0000","last_unstale":"2024-05-08T18:09:04.571470+0000","last_undegraded":"2024-05-08T18:09:04.571470+0000","last_fullsized":"2024-05-08T18:09:04.571470+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:09:04.571331+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:09:04.571331+0000","last_clean_scrub_stamp":"2024-05-08T18:09:04.571331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:35:38.826238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'2","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:27.488724+0000","last_change":"2024-05-08T18:08:27.488724+0000","last_active":"2024-05-08T18:08:27.488724+0000","last_peered":"2024-05-08T18:08:27.488724+0000","last_clean":"2024-05-08T18:08:27.488724+0000","last_became_active":"2024-05-08T18:04:45.819326+0000","last_became_peered":"2024-05-08T18:04:45.819326+0000","last_unstale":"2024-05-08T18:08:27.488724+0000","last_undegraded":"2024-05-08T18:08:27.488724+0000","last_fullsized":"2024-05-08T18:08:27.488724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:27.488574+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:27.488574+0000","last_clean_scrub_stamp":"2024-05-08T18:08:27.488574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:06.323416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:49.957730+0000","last_change":"2024-05-08T18:08:49.957730+0000","last_active":"2024-05-08T18:08:49.957730+0000","last_peered":"2024-05-08T18:08:49.957730+0000","last_clean":"2024-05-08T18:08:49.957730+0000","last_became_active":"2024-05-08T18:04:43.840236+0000","last_became_peered":"2024-05-08T18:04:43.840236+0000","last_unstale":"2024-05-08T18:08:49.957730+0000","last_undegraded":"2024-05-08T18:08:49.957730+0000","last_fullsized":"2024-05-08T18:08:49.957730+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:49.957583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:49.957583+0000","last_clean_scrub_stamp":"2024-05-08T18:08:49.957583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:10.553789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:52.369564+0000","last_change":"2024-05-08T18:08:52.369589+0000","last_active":"2024-05-08T18:08:52.369564+0000","last_peered":"2024-05-08T18:08:52.369564+0000","last_clean":"2024-05-08T18:08:52.369564+0000","last_became_active":"2024-05-08T18:04:35.552173+0000","last_became_peered":"2024-05-08T18:04:35.552173+0000","last_unstale":"2024-05-08T18:08:52.369564+0000","last_undegraded":"2024-05-08T18:08:52.369564+0000","last_fullsized":"2024-05-08T18:08:52.369564+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:52.369478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:52.369478+0000","last_clean_scrub_stamp":"2024-05-08T18:08:52.369478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:03.898871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:24.599665+0000","last_change":"2024-05-08T18:08:24.599665+0000","last_active":"2024-05-08T18:08:24.599665+0000","last_peered":"2024-05-08T18:08:24.599665+0000","last_clean":"2024-05-08T18:08:24.599665+0000","last_became_active":"2024-05-08T18:04:47.819910+0000","last_became_peered":"2024-05-08T18:04:47.819910+0000","last_unstale":"2024-05-08T18:08:24.599665+0000","last_undegraded":"2024-05-08T18:08:24.599665+0000","last_fullsized":"2024-05-08T18:08:24.599665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:24.599549+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:24.599549+0000","last_clean_scrub_stamp":"2024-05-08T18:08:24.599549+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:03.434391+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"29'73","reported_seq":150,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.598754+0000","last_change":"2024-05-08T18:08:15.598754+0000","last_active":"2024-05-08T18:08:15.598754+0000","last_peered":"2024-05-08T18:08:15.598754+0000","last_clean":"2024-05-08T18:08:15.598754+0000","last_became_active":"2024-05-08T18:04:49.831360+0000","last_became_peered":"2024-05-08T18:04:49.831360+0000","last_unstale":"2024-05-08T18:08:15.598754+0000","last_undegraded":"2024-05-08T18:08:15.598754+0000","last_fullsized":"2024-05-08T18:08:15.598754+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'73","last_scrub_stamp":"2024-05-08T18:08:15.598616+0000","last_deep_scrub":"29'73","last_deep_scrub_stamp":"2024-05-08T18:08:15.598616+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.598616+0000","objects_scrubbed":9,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:23:01.195330+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":336,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.250201+0000","last_change":"2024-05-08T18:08:13.250201+0000","last_active":"2024-05-08T18:08:13.250201+0000","last_peered":"2024-05-08T18:08:13.250201+0000","last_clean":"2024-05-08T18:08:13.250201+0000","last_became_active":"2024-05-08T18:04:51.836441+0000","last_became_peered":"2024-05-08T18:04:51.836441+0000","last_unstale":"2024-05-08T18:08:13.250201+0000","last_undegraded":"2024-05-08T18:08:13.250201+0000","last_fullsized":"2024-05-08T18:08:13.250201+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:13.250078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:13.250078+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.250078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:13.374535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"29'9","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:29.474491+0000","last_change":"2024-05-08T18:08:29.474491+0000","last_active":"2024-05-08T18:08:29.474491+0000","last_peered":"2024-05-08T18:08:29.474491+0000","last_clean":"2024-05-08T18:08:29.474491+0000","last_became_active":"2024-05-08T18:04:53.837920+0000","last_became_peered":"2024-05-08T18:04:53.837920+0000","last_unstale":"2024-05-08T18:08:29.474491+0000","last_undegraded":"2024-05-08T18:08:29.474491+0000","last_fullsized":"2024-05-08T18:08:29.474491+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'9","last_scrub_stamp":"2024-05-08T18:08:29.474349+0000","last_deep_scrub":"29'9","last_deep_scrub_stamp":"2024-05-08T18:08:29.474349+0000","last_clean_scrub_stamp":"2024-05-08T18:08:29.474349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:17.638910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"29'16","reported_seq":88,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:24.470006+0000","last_change":"2024-05-08T18:08:24.470006+0000","last_active":"2024-05-08T18:08:24.470006+0000","last_peered":"2024-05-08T18:08:24.470006+0000","last_clean":"2024-05-08T18:08:24.470006+0000","last_became_active":"2024-05-08T18:04:45.821107+0000","last_became_peered":"2024-05-08T18:04:45.821107+0000","last_unstale":"2024-05-08T18:08:24.470006+0000","last_undegraded":"2024-05-08T18:08:24.470006+0000","last_fullsized":"2024-05-08T18:08:24.470006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'16","last_scrub_stamp":"2024-05-08T18:08:24.469872+0000","last_deep_scrub":"29'16","last_deep_scrub_stamp":"2024-05-08T18:08:24.469872+0000","last_clean_scrub_stamp":"2024-05-08T18:08:24.469872+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:51.772730+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:57.481987+0000","last_change":"2024-05-08T18:08:57.482019+0000","last_active":"2024-05-08T18:08:57.481987+0000","last_peered":"2024-05-08T18:08:57.481987+0000","last_clean":"2024-05-08T18:08:57.481987+0000","last_became_active":"2024-05-08T18:04:43.827920+0000","last_became_peered":"2024-05-08T18:04:43.827920+0000","last_unstale":"2024-05-08T18:08:57.481987+0000","last_undegraded":"2024-05-08T18:08:57.481987+0000","last_fullsized":"2024-05-08T18:08:57.481987+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:57.481903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:57.481903+0000","last_clean_scrub_stamp":"2024-05-08T18:08:57.481903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:05.904037+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:14.981686+0000","last_change":"2024-05-08T18:09:14.981740+0000","last_active":"2024-05-08T18:09:14.981686+0000","last_peered":"2024-05-08T18:09:14.981686+0000","last_clean":"2024-05-08T18:09:14.981686+0000","last_became_active":"2024-05-08T18:04:35.550912+0000","last_became_peered":"2024-05-08T18:04:35.550912+0000","last_unstale":"2024-05-08T18:09:14.981686+0000","last_undegraded":"2024-05-08T18:09:14.981686+0000","last_fullsized":"2024-05-08T18:09:14.981686+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:14.981584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:14.981584+0000","last_clean_scrub_stamp":"2024-05-08T18:09:14.981584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:53:46.695624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.811076+0000","last_change":"2024-05-08T18:08:15.811076+0000","last_active":"2024-05-08T18:08:15.811076+0000","last_peered":"2024-05-08T18:08:15.811076+0000","last_clean":"2024-05-08T18:08:15.811076+0000","last_became_active":"2024-05-08T18:04:47.813717+0000","last_became_peered":"2024-05-08T18:04:47.813717+0000","last_unstale":"2024-05-08T18:08:15.811076+0000","last_undegraded":"2024-05-08T18:08:15.811076+0000","last_fullsized":"2024-05-08T18:08:15.811076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:15.810959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:15.810959+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.810959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:35.598153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"29'21","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.551971+0000","last_change":"2024-05-08T18:08:06.551971+0000","last_active":"2024-05-08T18:08:06.551971+0000","last_peered":"2024-05-08T18:08:06.551971+0000","last_clean":"2024-05-08T18:08:06.551971+0000","last_became_active":"2024-05-08T18:04:49.835451+0000","last_became_peered":"2024-05-08T18:04:49.835451+0000","last_unstale":"2024-05-08T18:08:06.551971+0000","last_undegraded":"2024-05-08T18:08:06.551971+0000","last_fullsized":"2024-05-08T18:08:06.551971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'21","last_scrub_stamp":"2024-05-08T18:08:06.551844+0000","last_deep_scrub":"29'21","last_deep_scrub_stamp":"2024-05-08T18:08:06.551844+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.551844+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:05.040309+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"29'285","reported_seq":382,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:04.536905+0000","last_change":"2024-05-08T18:08:04.536905+0000","last_active":"2024-05-08T18:08:04.536905+0000","last_peered":"2024-05-08T18:08:04.536905+0000","last_clean":"2024-05-08T18:08:04.536905+0000","last_became_active":"2024-05-08T18:04:51.832109+0000","last_became_peered":"2024-05-08T18:04:51.832109+0000","last_unstale":"2024-05-08T18:08:04.536905+0000","last_undegraded":"2024-05-08T18:08:04.536905+0000","last_fullsized":"2024-05-08T18:08:04.536905+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:04.536786+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:04.536786+0000","last_clean_scrub_stamp":"2024-05-08T18:08:04.536786+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:42.101932+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:49.039245+0000","last_change":"2024-05-08T18:08:49.039245+0000","last_active":"2024-05-08T18:08:49.039245+0000","last_peered":"2024-05-08T18:08:49.039245+0000","last_clean":"2024-05-08T18:08:49.039245+0000","last_became_active":"2024-05-08T18:04:53.842998+0000","last_became_peered":"2024-05-08T18:04:53.842998+0000","last_unstale":"2024-05-08T18:08:49.039245+0000","last_undegraded":"2024-05-08T18:08:49.039245+0000","last_fullsized":"2024-05-08T18:08:49.039245+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:49.039104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:49.039104+0000","last_clean_scrub_stamp":"2024-05-08T18:08:49.039104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:33.981256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'4","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.838524+0000","last_change":"2024-05-08T18:08:19.838524+0000","last_active":"2024-05-08T18:08:19.838524+0000","last_peered":"2024-05-08T18:08:19.838524+0000","last_clean":"2024-05-08T18:08:19.838524+0000","last_became_active":"2024-05-08T18:04:45.829257+0000","last_became_peered":"2024-05-08T18:04:45.829257+0000","last_unstale":"2024-05-08T18:08:19.838524+0000","last_undegraded":"2024-05-08T18:08:19.838524+0000","last_fullsized":"2024-05-08T18:08:19.838524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:19.838313+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:19.838313+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.838313+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:43.545756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"29'3","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:07.939354+0000","last_change":"2024-05-08T18:09:07.939382+0000","last_active":"2024-05-08T18:09:07.939354+0000","last_peered":"2024-05-08T18:09:07.939354+0000","last_clean":"2024-05-08T18:09:07.939354+0000","last_became_active":"2024-05-08T18:04:43.834610+0000","last_became_peered":"2024-05-08T18:04:43.834610+0000","last_unstale":"2024-05-08T18:09:07.939354+0000","last_undegraded":"2024-05-08T18:09:07.939354+0000","last_fullsized":"2024-05-08T18:09:07.939354+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3","last_scrub_stamp":"2024-05-08T18:09:07.939248+0000","last_deep_scrub":"29'3","last_deep_scrub_stamp":"2024-05-08T18:09:07.939248+0000","last_clean_scrub_stamp":"2024-05-08T18:09:07.939248+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:39.228713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:51.327283+0000","last_change":"2024-05-08T18:08:51.327283+0000","last_active":"2024-05-08T18:08:51.327283+0000","last_peered":"2024-05-08T18:08:51.327283+0000","last_clean":"2024-05-08T18:08:51.327283+0000","last_became_active":"2024-05-08T18:04:35.551366+0000","last_became_peered":"2024-05-08T18:04:35.551366+0000","last_unstale":"2024-05-08T18:08:51.327283+0000","last_undegraded":"2024-05-08T18:08:51.327283+0000","last_fullsized":"2024-05-08T18:08:51.327283+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:51.327154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:51.327154+0000","last_clean_scrub_stamp":"2024-05-08T18:08:51.327154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:17:16.185628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.923265+0000","last_change":"2024-05-08T18:08:06.923265+0000","last_active":"2024-05-08T18:08:06.923265+0000","last_peered":"2024-05-08T18:08:06.923265+0000","last_clean":"2024-05-08T18:08:06.923265+0000","last_became_active":"2024-05-08T18:04:47.819866+0000","last_became_peered":"2024-05-08T18:04:47.819866+0000","last_unstale":"2024-05-08T18:08:06.923265+0000","last_undegraded":"2024-05-08T18:08:06.923265+0000","last_fullsized":"2024-05-08T18:08:06.923265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:06.923149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:06.923149+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.923149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:30.572799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"29'15","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:35.962385+0000","last_change":"2024-05-08T18:08:35.962385+0000","last_active":"2024-05-08T18:08:35.962385+0000","last_peered":"2024-05-08T18:08:35.962385+0000","last_clean":"2024-05-08T18:08:35.962385+0000","last_became_active":"2024-05-08T18:04:49.830452+0000","last_became_peered":"2024-05-08T18:04:49.830452+0000","last_unstale":"2024-05-08T18:08:35.962385+0000","last_undegraded":"2024-05-08T18:08:35.962385+0000","last_fullsized":"2024-05-08T18:08:35.962385+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:35.962267+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:35.962267+0000","last_clean_scrub_stamp":"2024-05-08T18:08:35.962267+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:45.951593+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"29'285","reported_seq":412,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.471786+0000","last_change":"2024-05-08T18:08:06.471786+0000","last_active":"2024-05-08T18:08:06.471786+0000","last_peered":"2024-05-08T18:08:06.471786+0000","last_clean":"2024-05-08T18:08:06.471786+0000","last_became_active":"2024-05-08T18:04:51.831155+0000","last_became_peered":"2024-05-08T18:04:51.831155+0000","last_unstale":"2024-05-08T18:08:06.471786+0000","last_undegraded":"2024-05-08T18:08:06.471786+0000","last_fullsized":"2024-05-08T18:08:06.471786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:06.471664+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:06.471664+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.471664+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:29:09.321348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'5","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:01.531628+0000","last_change":"2024-05-08T18:09:01.531628+0000","last_active":"2024-05-08T18:09:01.531628+0000","last_peered":"2024-05-08T18:09:01.531628+0000","last_clean":"2024-05-08T18:09:01.531628+0000","last_became_active":"2024-05-08T18:04:53.848862+0000","last_became_peered":"2024-05-08T18:04:53.848862+0000","last_unstale":"2024-05-08T18:09:01.531628+0000","last_undegraded":"2024-05-08T18:09:01.531628+0000","last_fullsized":"2024-05-08T18:09:01.531628+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'5","last_scrub_stamp":"2024-05-08T18:09:01.531514+0000","last_deep_scrub":"29'5","last_deep_scrub_stamp":"2024-05-08T18:09:01.531514+0000","last_clean_scrub_stamp":"2024-05-08T18:09:01.531514+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:09.953648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"29'6","reported_seq":61,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:41.133464+0000","last_change":"2024-05-08T18:08:41.133464+0000","last_active":"2024-05-08T18:08:41.133464+0000","last_peered":"2024-05-08T18:08:41.133464+0000","last_clean":"2024-05-08T18:08:41.133464+0000","last_became_active":"2024-05-08T18:04:45.822824+0000","last_became_peered":"2024-05-08T18:04:45.822824+0000","last_unstale":"2024-05-08T18:08:41.133464+0000","last_undegraded":"2024-05-08T18:08:41.133464+0000","last_fullsized":"2024-05-08T18:08:41.133464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:41.133339+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:41.133339+0000","last_clean_scrub_stamp":"2024-05-08T18:08:41.133339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:17.740267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:44.501238+0000","last_change":"2024-05-08T18:08:44.501238+0000","last_active":"2024-05-08T18:08:44.501238+0000","last_peered":"2024-05-08T18:08:44.501238+0000","last_clean":"2024-05-08T18:08:44.501238+0000","last_became_active":"2024-05-08T18:04:43.839240+0000","last_became_peered":"2024-05-08T18:04:43.839240+0000","last_unstale":"2024-05-08T18:08:44.501238+0000","last_undegraded":"2024-05-08T18:08:44.501238+0000","last_fullsized":"2024-05-08T18:08:44.501238+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:44.501120+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:44.501120+0000","last_clean_scrub_stamp":"2024-05-08T18:08:44.501120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:27.037641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:16.972932+0000","last_change":"2024-05-08T18:09:16.972955+0000","last_active":"2024-05-08T18:09:16.972932+0000","last_peered":"2024-05-08T18:09:16.972932+0000","last_clean":"2024-05-08T18:09:16.972932+0000","last_became_active":"2024-05-08T18:04:35.551412+0000","last_became_peered":"2024-05-08T18:04:35.551412+0000","last_unstale":"2024-05-08T18:09:16.972932+0000","last_undegraded":"2024-05-08T18:09:16.972932+0000","last_fullsized":"2024-05-08T18:09:16.972932+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:16.972830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:16.972830+0000","last_clean_scrub_stamp":"2024-05-08T18:09:16.972830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:02:49.510804+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.843881+0000","last_change":"2024-05-08T18:08:13.843881+0000","last_active":"2024-05-08T18:08:13.843881+0000","last_peered":"2024-05-08T18:08:13.843881+0000","last_clean":"2024-05-08T18:08:13.843881+0000","last_became_active":"2024-05-08T18:04:47.815947+0000","last_became_peered":"2024-05-08T18:04:47.815947+0000","last_unstale":"2024-05-08T18:08:13.843881+0000","last_undegraded":"2024-05-08T18:08:13.843881+0000","last_fullsized":"2024-05-08T18:08:13.843881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:13.843666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:13.843666+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.843666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:56.027223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"29'96","reported_seq":190,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.153072+0000","last_change":"2024-05-08T18:08:10.153072+0000","last_active":"2024-05-08T18:08:10.153072+0000","last_peered":"2024-05-08T18:08:10.153072+0000","last_clean":"2024-05-08T18:08:10.153072+0000","last_became_active":"2024-05-08T18:04:49.834586+0000","last_became_peered":"2024-05-08T18:04:49.834586+0000","last_unstale":"2024-05-08T18:08:10.153072+0000","last_undegraded":"2024-05-08T18:08:10.153072+0000","last_fullsized":"2024-05-08T18:08:10.153072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'96","last_scrub_stamp":"2024-05-08T18:08:10.152888+0000","last_deep_scrub":"29'96","last_deep_scrub_stamp":"2024-05-08T18:08:10.152888+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.152888+0000","objects_scrubbed":8,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:16.511455+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.455904+0000","last_change":"2024-05-08T18:08:08.455904+0000","last_active":"2024-05-08T18:08:08.455904+0000","last_peered":"2024-05-08T18:08:08.455904+0000","last_clean":"2024-05-08T18:08:08.455904+0000","last_became_active":"2024-05-08T18:04:51.830760+0000","last_became_peered":"2024-05-08T18:04:51.830760+0000","last_unstale":"2024-05-08T18:08:08.455904+0000","last_undegraded":"2024-05-08T18:08:08.455904+0000","last_fullsized":"2024-05-08T18:08:08.455904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:08.455773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:08.455773+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.455773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:12.703852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.342717+0000","last_change":"2024-05-08T18:08:21.342717+0000","last_active":"2024-05-08T18:08:21.342717+0000","last_peered":"2024-05-08T18:08:21.342717+0000","last_clean":"2024-05-08T18:08:21.342717+0000","last_became_active":"2024-05-08T18:04:53.838600+0000","last_became_peered":"2024-05-08T18:04:53.838600+0000","last_unstale":"2024-05-08T18:08:21.342717+0000","last_undegraded":"2024-05-08T18:08:21.342717+0000","last_fullsized":"2024-05-08T18:08:21.342717+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:21.342590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:21.342590+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.342590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:44.919676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"27'2","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:51.909643+0000","last_change":"2024-05-08T18:08:51.909643+0000","last_active":"2024-05-08T18:08:51.909643+0000","last_peered":"2024-05-08T18:08:51.909643+0000","last_clean":"2024-05-08T18:08:51.909643+0000","last_became_active":"2024-05-08T18:04:53.839500+0000","last_became_peered":"2024-05-08T18:04:53.839500+0000","last_unstale":"2024-05-08T18:08:51.909643+0000","last_undegraded":"2024-05-08T18:08:51.909643+0000","last_fullsized":"2024-05-08T18:08:51.909643+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:51.909525+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:51.909525+0000","last_clean_scrub_stamp":"2024-05-08T18:08:51.909525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:05:28.466923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"29'8","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:39.209848+0000","last_change":"2024-05-08T18:08:39.209919+0000","last_active":"2024-05-08T18:08:39.209848+0000","last_peered":"2024-05-08T18:08:39.209848+0000","last_clean":"2024-05-08T18:08:39.209848+0000","last_became_active":"2024-05-08T18:04:45.819508+0000","last_became_peered":"2024-05-08T18:04:45.819508+0000","last_unstale":"2024-05-08T18:08:39.209848+0000","last_undegraded":"2024-05-08T18:08:39.209848+0000","last_fullsized":"2024-05-08T18:08:39.209848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:39.209750+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:39.209750+0000","last_clean_scrub_stamp":"2024-05-08T18:08:39.209750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:48.744139+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:06.972658+0000","last_change":"2024-05-08T18:09:06.972658+0000","last_active":"2024-05-08T18:09:06.972658+0000","last_peered":"2024-05-08T18:09:06.972658+0000","last_clean":"2024-05-08T18:09:06.972658+0000","last_became_active":"2024-05-08T18:04:43.837929+0000","last_became_peered":"2024-05-08T18:04:43.837929+0000","last_unstale":"2024-05-08T18:09:06.972658+0000","last_undegraded":"2024-05-08T18:09:06.972658+0000","last_fullsized":"2024-05-08T18:09:06.972658+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:06.972472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:06.972472+0000","last_clean_scrub_stamp":"2024-05-08T18:09:06.972472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:00:23.965126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.504440+0000","last_change":"2024-05-08T18:08:19.504467+0000","last_active":"2024-05-08T18:08:19.504440+0000","last_peered":"2024-05-08T18:08:19.504440+0000","last_clean":"2024-05-08T18:08:19.504440+0000","last_became_active":"2024-05-08T18:04:47.818443+0000","last_became_peered":"2024-05-08T18:04:47.818443+0000","last_unstale":"2024-05-08T18:08:19.504440+0000","last_undegraded":"2024-05-08T18:08:19.504440+0000","last_fullsized":"2024-05-08T18:08:19.504440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:19.504350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:19.504350+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.504350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:24.197683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"29'40","reported_seq":96,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:30.848309+0000","last_change":"2024-05-08T18:08:30.848380+0000","last_active":"2024-05-08T18:08:30.848309+0000","last_peered":"2024-05-08T18:08:30.848309+0000","last_clean":"2024-05-08T18:08:30.848309+0000","last_became_active":"2024-05-08T18:04:49.833599+0000","last_became_peered":"2024-05-08T18:04:49.833599+0000","last_unstale":"2024-05-08T18:08:30.848309+0000","last_undegraded":"2024-05-08T18:08:30.848309+0000","last_fullsized":"2024-05-08T18:08:30.848309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'40","last_scrub_stamp":"2024-05-08T18:08:30.848207+0000","last_deep_scrub":"29'40","last_deep_scrub_stamp":"2024-05-08T18:08:30.848207+0000","last_clean_scrub_stamp":"2024-05-08T18:08:30.848207+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:31:32.685411+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:26.837175+0000","last_change":"2024-05-08T18:08:26.837221+0000","last_active":"2024-05-08T18:08:26.837175+0000","last_peered":"2024-05-08T18:08:26.837175+0000","last_clean":"2024-05-08T18:08:26.837175+0000","last_became_active":"2024-05-08T18:04:51.844526+0000","last_became_peered":"2024-05-08T18:04:51.844526+0000","last_unstale":"2024-05-08T18:08:26.837175+0000","last_undegraded":"2024-05-08T18:08:26.837175+0000","last_fullsized":"2024-05-08T18:08:26.837175+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:26.837073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:26.837073+0000","last_clean_scrub_stamp":"2024-05-08T18:08:26.837073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:03.923893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"29'11","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:05.534334+0000","last_change":"2024-05-08T18:09:05.534359+0000","last_active":"2024-05-08T18:09:05.534334+0000","last_peered":"2024-05-08T18:09:05.534334+0000","last_clean":"2024-05-08T18:09:05.534334+0000","last_became_active":"2024-05-08T18:04:53.850593+0000","last_became_peered":"2024-05-08T18:04:53.850593+0000","last_unstale":"2024-05-08T18:09:05.534334+0000","last_undegraded":"2024-05-08T18:09:05.534334+0000","last_fullsized":"2024-05-08T18:09:05.534334+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'11","last_scrub_stamp":"2024-05-08T18:09:05.534221+0000","last_deep_scrub":"29'11","last_deep_scrub_stamp":"2024-05-08T18:09:05.534221+0000","last_clean_scrub_stamp":"2024-05-08T18:09:05.534221+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:32:51.310647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":62,"num_write":31,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"29'8","reported_seq":82,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:31.489072+0000","last_change":"2024-05-08T18:08:31.489126+0000","last_active":"2024-05-08T18:08:31.489072+0000","last_peered":"2024-05-08T18:08:31.489072+0000","last_clean":"2024-05-08T18:08:31.489072+0000","last_became_active":"2024-05-08T18:04:45.818979+0000","last_became_peered":"2024-05-08T18:04:45.818979+0000","last_unstale":"2024-05-08T18:08:31.489072+0000","last_undegraded":"2024-05-08T18:08:31.489072+0000","last_fullsized":"2024-05-08T18:08:31.489072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:31.488942+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:31.488942+0000","last_clean_scrub_stamp":"2024-05-08T18:08:31.488942+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:34.056789+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:46.971371+0000","last_change":"2024-05-08T18:08:46.971417+0000","last_active":"2024-05-08T18:08:46.971371+0000","last_peered":"2024-05-08T18:08:46.971371+0000","last_clean":"2024-05-08T18:08:46.971371+0000","last_became_active":"2024-05-08T18:04:43.836614+0000","last_became_peered":"2024-05-08T18:04:43.836614+0000","last_unstale":"2024-05-08T18:08:46.971371+0000","last_undegraded":"2024-05-08T18:08:46.971371+0000","last_fullsized":"2024-05-08T18:08:46.971371+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:46.971249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:46.971249+0000","last_clean_scrub_stamp":"2024-05-08T18:08:46.971249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:02:46.938192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.821717+0000","last_change":"2024-05-08T18:08:14.821740+0000","last_active":"2024-05-08T18:08:14.821717+0000","last_peered":"2024-05-08T18:08:14.821717+0000","last_clean":"2024-05-08T18:08:14.821717+0000","last_became_active":"2024-05-08T18:04:47.814561+0000","last_became_peered":"2024-05-08T18:04:47.814561+0000","last_unstale":"2024-05-08T18:08:14.821717+0000","last_undegraded":"2024-05-08T18:08:14.821717+0000","last_fullsized":"2024-05-08T18:08:14.821717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:14.821626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:14.821626+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.821626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:24.805865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"29'10","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.572538+0000","last_change":"2024-05-08T18:08:18.572575+0000","last_active":"2024-05-08T18:08:18.572538+0000","last_peered":"2024-05-08T18:08:18.572538+0000","last_clean":"2024-05-08T18:08:18.572538+0000","last_became_active":"2024-05-08T18:04:49.825347+0000","last_became_peered":"2024-05-08T18:04:49.825347+0000","last_unstale":"2024-05-08T18:08:18.572538+0000","last_undegraded":"2024-05-08T18:08:18.572538+0000","last_fullsized":"2024-05-08T18:08:18.572538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:18.572458+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:18.572458+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.572458+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:21.293654+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.319006+0000","last_change":"2024-05-08T18:08:17.319068+0000","last_active":"2024-05-08T18:08:17.319006+0000","last_peered":"2024-05-08T18:08:17.319006+0000","last_clean":"2024-05-08T18:08:17.319006+0000","last_became_active":"2024-05-08T18:04:51.843002+0000","last_became_peered":"2024-05-08T18:04:51.843002+0000","last_unstale":"2024-05-08T18:08:17.319006+0000","last_undegraded":"2024-05-08T18:08:17.319006+0000","last_fullsized":"2024-05-08T18:08:17.319006+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:17.318901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:17.318901+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.318901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:07.004932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:00.468674+0000","last_change":"2024-05-08T18:09:00.468698+0000","last_active":"2024-05-08T18:09:00.468674+0000","last_peered":"2024-05-08T18:09:00.468674+0000","last_clean":"2024-05-08T18:09:00.468674+0000","last_became_active":"2024-05-08T18:04:53.840437+0000","last_became_peered":"2024-05-08T18:04:53.840437+0000","last_unstale":"2024-05-08T18:09:00.468674+0000","last_undegraded":"2024-05-08T18:09:00.468674+0000","last_fullsized":"2024-05-08T18:09:00.468674+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:00.468582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:00.468582+0000","last_clean_scrub_stamp":"2024-05-08T18:09:00.468582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:35:34.723489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"29'8","reported_seq":67,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:06.547159+0000","last_change":"2024-05-08T18:09:06.547181+0000","last_active":"2024-05-08T18:09:06.547159+0000","last_peered":"2024-05-08T18:09:06.547159+0000","last_clean":"2024-05-08T18:09:06.547159+0000","last_became_active":"2024-05-08T18:05:30.868156+0000","last_became_peered":"2024-05-08T18:05:30.868156+0000","last_unstale":"2024-05-08T18:09:06.547159+0000","last_undegraded":"2024-05-08T18:09:06.547159+0000","last_fullsized":"2024-05-08T18:09:06.547159+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:09:06.547085+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:09:06.547085+0000","last_clean_scrub_stamp":"2024-05-08T18:09:06.547085+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:46:26.349909+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:45.035572+0000","last_change":"2024-05-08T18:08:45.035600+0000","last_active":"2024-05-08T18:08:45.035572+0000","last_peered":"2024-05-08T18:08:45.035572+0000","last_clean":"2024-05-08T18:08:45.035572+0000","last_became_active":"2024-05-08T18:04:43.835435+0000","last_became_peered":"2024-05-08T18:04:43.835435+0000","last_unstale":"2024-05-08T18:08:45.035572+0000","last_undegraded":"2024-05-08T18:08:45.035572+0000","last_fullsized":"2024-05-08T18:08:45.035572+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:45.035484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:45.035484+0000","last_clean_scrub_stamp":"2024-05-08T18:08:45.035484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:06:21.992378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.385584+0000","last_change":"2024-05-08T18:08:19.385611+0000","last_active":"2024-05-08T18:08:19.385584+0000","last_peered":"2024-05-08T18:08:19.385584+0000","last_clean":"2024-05-08T18:08:19.385584+0000","last_became_active":"2024-05-08T18:04:47.824140+0000","last_became_peered":"2024-05-08T18:04:47.824140+0000","last_unstale":"2024-05-08T18:08:19.385584+0000","last_undegraded":"2024-05-08T18:08:19.385584+0000","last_fullsized":"2024-05-08T18:08:19.385584+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:19.385463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:19.385463+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.385463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:15.297455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"29'48","reported_seq":120,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.582586+0000","last_change":"2024-05-08T18:08:19.582659+0000","last_active":"2024-05-08T18:08:19.582586+0000","last_peered":"2024-05-08T18:08:19.582586+0000","last_clean":"2024-05-08T18:08:19.582586+0000","last_became_active":"2024-05-08T18:04:49.824600+0000","last_became_peered":"2024-05-08T18:04:49.824600+0000","last_unstale":"2024-05-08T18:08:19.582586+0000","last_undegraded":"2024-05-08T18:08:19.582586+0000","last_fullsized":"2024-05-08T18:08:19.582586+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'48","last_scrub_stamp":"2024-05-08T18:08:19.582484+0000","last_deep_scrub":"29'48","last_deep_scrub_stamp":"2024-05-08T18:08:19.582484+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.582484+0000","objects_scrubbed":6,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:25.159113+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":706,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.521105+0000","last_change":"2024-05-08T18:08:09.521130+0000","last_active":"2024-05-08T18:08:09.521105+0000","last_peered":"2024-05-08T18:08:09.521105+0000","last_clean":"2024-05-08T18:08:09.521105+0000","last_became_active":"2024-05-08T18:04:51.829937+0000","last_became_peered":"2024-05-08T18:04:51.829937+0000","last_unstale":"2024-05-08T18:08:09.521105+0000","last_undegraded":"2024-05-08T18:08:09.521105+0000","last_fullsized":"2024-05-08T18:08:09.521105+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:09.521013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:09.521013+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.521013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:29:12.370696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:45.103397+0000","last_change":"2024-05-08T18:08:45.103421+0000","last_active":"2024-05-08T18:08:45.103397+0000","last_peered":"2024-05-08T18:08:45.103397+0000","last_clean":"2024-05-08T18:08:45.103397+0000","last_became_active":"2024-05-08T18:04:53.842610+0000","last_became_peered":"2024-05-08T18:04:53.842610+0000","last_unstale":"2024-05-08T18:08:45.103397+0000","last_undegraded":"2024-05-08T18:08:45.103397+0000","last_fullsized":"2024-05-08T18:08:45.103397+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:45.103296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:45.103296+0000","last_clean_scrub_stamp":"2024-05-08T18:08:45.103296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:29.008024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.866944+0000","last_change":"2024-05-08T18:08:22.866966+0000","last_active":"2024-05-08T18:08:22.866944+0000","last_peered":"2024-05-08T18:08:22.866944+0000","last_clean":"2024-05-08T18:08:22.866944+0000","last_became_active":"2024-05-08T18:04:45.829458+0000","last_became_peered":"2024-05-08T18:04:45.829458+0000","last_unstale":"2024-05-08T18:08:22.866944+0000","last_undegraded":"2024-05-08T18:08:22.866944+0000","last_fullsized":"2024-05-08T18:08:22.866944+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.866853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.866853+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.866853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:05.403374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"29'4","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:56.481494+0000","last_change":"2024-05-08T18:08:56.481530+0000","last_active":"2024-05-08T18:08:56.481494+0000","last_peered":"2024-05-08T18:08:56.481494+0000","last_clean":"2024-05-08T18:08:56.481494+0000","last_became_active":"2024-05-08T18:04:43.836633+0000","last_became_peered":"2024-05-08T18:04:43.836633+0000","last_unstale":"2024-05-08T18:08:56.481494+0000","last_undegraded":"2024-05-08T18:08:56.481494+0000","last_fullsized":"2024-05-08T18:08:56.481494+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:56.481386+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:56.481386+0000","last_clean_scrub_stamp":"2024-05-08T18:08:56.481386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:32.379503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"29'2","reported_seq":230,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.909561+0000","last_change":"2024-05-08T18:08:07.909590+0000","last_active":"2024-05-08T18:08:07.909561+0000","last_peered":"2024-05-08T18:08:07.909561+0000","last_clean":"2024-05-08T18:08:07.909561+0000","last_became_active":"2024-05-08T18:04:47.819771+0000","last_became_peered":"2024-05-08T18:04:47.819771+0000","last_unstale":"2024-05-08T18:08:07.909561+0000","last_undegraded":"2024-05-08T18:08:07.909561+0000","last_fullsized":"2024-05-08T18:08:07.909561+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:07.909472+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:07.909472+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.909472+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:04.798235+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"29'36","reported_seq":91,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.344301+0000","last_change":"2024-05-08T18:08:18.344360+0000","last_active":"2024-05-08T18:08:18.344301+0000","last_peered":"2024-05-08T18:08:18.344301+0000","last_clean":"2024-05-08T18:08:18.344301+0000","last_became_active":"2024-05-08T18:04:49.824022+0000","last_became_peered":"2024-05-08T18:04:49.824022+0000","last_unstale":"2024-05-08T18:08:18.344301+0000","last_undegraded":"2024-05-08T18:08:18.344301+0000","last_fullsized":"2024-05-08T18:08:18.344301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:08:18.344202+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:08:18.344202+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.344202+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:36:47.464601+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.198536+0000","last_change":"2024-05-08T18:08:08.198575+0000","last_active":"2024-05-08T18:08:08.198536+0000","last_peered":"2024-05-08T18:08:08.198536+0000","last_clean":"2024-05-08T18:08:08.198536+0000","last_became_active":"2024-05-08T18:04:51.826424+0000","last_became_peered":"2024-05-08T18:04:51.826424+0000","last_unstale":"2024-05-08T18:08:08.198536+0000","last_undegraded":"2024-05-08T18:08:08.198536+0000","last_fullsized":"2024-05-08T18:08:08.198536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:08.198405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:08.198405+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.198405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:15.614149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"29'20","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:57.043843+0000","last_change":"2024-05-08T18:08:57.043872+0000","last_active":"2024-05-08T18:08:57.043843+0000","last_peered":"2024-05-08T18:08:57.043843+0000","last_clean":"2024-05-08T18:08:57.043843+0000","last_became_active":"2024-05-08T18:05:30.868409+0000","last_became_peered":"2024-05-08T18:05:30.868409+0000","last_unstale":"2024-05-08T18:08:57.043843+0000","last_undegraded":"2024-05-08T18:08:57.043843+0000","last_fullsized":"2024-05-08T18:08:57.043843+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'20","last_scrub_stamp":"2024-05-08T18:08:57.043740+0000","last_deep_scrub":"29'20","last_deep_scrub_stamp":"2024-05-08T18:08:57.043740+0000","last_clean_scrub_stamp":"2024-05-08T18:08:57.043740+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:03.723531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:08.952684+0000","last_change":"2024-05-08T18:09:08.952757+0000","last_active":"2024-05-08T18:09:08.952684+0000","last_peered":"2024-05-08T18:09:08.952684+0000","last_clean":"2024-05-08T18:09:08.952684+0000","last_became_active":"2024-05-08T18:04:43.838957+0000","last_became_peered":"2024-05-08T18:04:43.838957+0000","last_unstale":"2024-05-08T18:09:08.952684+0000","last_undegraded":"2024-05-08T18:09:08.952684+0000","last_fullsized":"2024-05-08T18:09:08.952684+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:08.952571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:08.952571+0000","last_clean_scrub_stamp":"2024-05-08T18:09:08.952571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:03.568080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.606122+0000","last_change":"2024-05-08T18:08:23.606122+0000","last_active":"2024-05-08T18:08:23.606122+0000","last_peered":"2024-05-08T18:08:23.606122+0000","last_clean":"2024-05-08T18:08:23.606122+0000","last_became_active":"2024-05-08T18:04:47.820256+0000","last_became_peered":"2024-05-08T18:04:47.820256+0000","last_unstale":"2024-05-08T18:08:23.606122+0000","last_undegraded":"2024-05-08T18:08:23.606122+0000","last_fullsized":"2024-05-08T18:08:23.606122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:23.606002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:23.606002+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.606002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:04.849646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"29'63","reported_seq":136,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.582374+0000","last_change":"2024-05-08T18:08:16.582414+0000","last_active":"2024-05-08T18:08:16.582374+0000","last_peered":"2024-05-08T18:08:16.582374+0000","last_clean":"2024-05-08T18:08:16.582374+0000","last_became_active":"2024-05-08T18:04:49.825366+0000","last_became_peered":"2024-05-08T18:04:49.825366+0000","last_unstale":"2024-05-08T18:08:16.582374+0000","last_undegraded":"2024-05-08T18:08:16.582374+0000","last_fullsized":"2024-05-08T18:08:16.582374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'63","last_scrub_stamp":"2024-05-08T18:08:16.582295+0000","last_deep_scrub":"29'63","last_deep_scrub_stamp":"2024-05-08T18:08:16.582295+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.582295+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:21.275627+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.157727+0000","last_change":"2024-05-08T18:08:07.157775+0000","last_active":"2024-05-08T18:08:07.157727+0000","last_peered":"2024-05-08T18:08:07.157727+0000","last_clean":"2024-05-08T18:08:07.157727+0000","last_became_active":"2024-05-08T18:04:51.827127+0000","last_became_peered":"2024-05-08T18:04:51.827127+0000","last_unstale":"2024-05-08T18:08:07.157727+0000","last_undegraded":"2024-05-08T18:08:07.157727+0000","last_fullsized":"2024-05-08T18:08:07.157727+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:07.157606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:07.157606+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.157606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:37:53.105588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"29'14","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:03.477837+0000","last_change":"2024-05-08T18:09:03.477873+0000","last_active":"2024-05-08T18:09:03.477837+0000","last_peered":"2024-05-08T18:09:03.477837+0000","last_clean":"2024-05-08T18:09:03.477837+0000","last_became_active":"2024-05-08T18:04:53.843772+0000","last_became_peered":"2024-05-08T18:04:53.843772+0000","last_unstale":"2024-05-08T18:09:03.477837+0000","last_undegraded":"2024-05-08T18:09:03.477837+0000","last_fullsized":"2024-05-08T18:09:03.477837+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'14","last_scrub_stamp":"2024-05-08T18:09:03.477748+0000","last_deep_scrub":"29'14","last_deep_scrub_stamp":"2024-05-08T18:09:03.477748+0000","last_clean_scrub_stamp":"2024-05-08T18:09:03.477748+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:41:23.206546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"29'2","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:43.351032+0000","last_change":"2024-05-08T18:08:43.351032+0000","last_active":"2024-05-08T18:08:43.351032+0000","last_peered":"2024-05-08T18:08:43.351032+0000","last_clean":"2024-05-08T18:08:43.351032+0000","last_became_active":"2024-05-08T18:04:45.811517+0000","last_became_peered":"2024-05-08T18:04:45.811517+0000","last_unstale":"2024-05-08T18:08:43.351032+0000","last_undegraded":"2024-05-08T18:08:43.351032+0000","last_fullsized":"2024-05-08T18:08:43.351032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:43.350844+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:43.350844+0000","last_clean_scrub_stamp":"2024-05-08T18:08:43.350844+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:35.421755+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:54.459528+0000","last_change":"2024-05-08T18:08:54.459559+0000","last_active":"2024-05-08T18:08:54.459528+0000","last_peered":"2024-05-08T18:08:54.459528+0000","last_clean":"2024-05-08T18:08:54.459528+0000","last_became_active":"2024-05-08T18:04:43.834767+0000","last_became_peered":"2024-05-08T18:04:43.834767+0000","last_unstale":"2024-05-08T18:08:54.459528+0000","last_undegraded":"2024-05-08T18:08:54.459528+0000","last_fullsized":"2024-05-08T18:08:54.459528+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:54.459439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:54.459439+0000","last_clean_scrub_stamp":"2024-05-08T18:08:54.459439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:45.537704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0,2],"acting":[1,3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:24.285019+0000","last_change":"2024-05-08T18:08:24.285019+0000","last_active":"2024-05-08T18:08:24.285019+0000","last_peered":"2024-05-08T18:08:24.285019+0000","last_clean":"2024-05-08T18:08:24.285019+0000","last_became_active":"2024-05-08T18:04:47.820662+0000","last_became_peered":"2024-05-08T18:04:47.820662+0000","last_unstale":"2024-05-08T18:08:24.285019+0000","last_undegraded":"2024-05-08T18:08:24.285019+0000","last_fullsized":"2024-05-08T18:08:24.285019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:24.284899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:24.284899+0000","last_clean_scrub_stamp":"2024-05-08T18:08:24.284899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:02:49.472387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"29'46","reported_seq":104,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.901855+0000","last_change":"2024-05-08T18:08:10.901892+0000","last_active":"2024-05-08T18:08:10.901855+0000","last_peered":"2024-05-08T18:08:10.901855+0000","last_clean":"2024-05-08T18:08:10.901855+0000","last_became_active":"2024-05-08T18:04:49.830476+0000","last_became_peered":"2024-05-08T18:04:49.830476+0000","last_unstale":"2024-05-08T18:08:10.901855+0000","last_undegraded":"2024-05-08T18:08:10.901855+0000","last_fullsized":"2024-05-08T18:08:10.901855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'46","last_scrub_stamp":"2024-05-08T18:08:10.901767+0000","last_deep_scrub":"29'46","last_deep_scrub_stamp":"2024-05-08T18:08:10.901767+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.901767+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:07.184811+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.490196+0000","last_change":"2024-05-08T18:08:17.490220+0000","last_active":"2024-05-08T18:08:17.490196+0000","last_peered":"2024-05-08T18:08:17.490196+0000","last_clean":"2024-05-08T18:08:17.490196+0000","last_became_active":"2024-05-08T18:04:51.844912+0000","last_became_peered":"2024-05-08T18:04:51.844912+0000","last_unstale":"2024-05-08T18:08:17.490196+0000","last_undegraded":"2024-05-08T18:08:17.490196+0000","last_fullsized":"2024-05-08T18:08:17.490196+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:17.490110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:17.490110+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.490110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:06:17.806415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"29'2","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:54.043126+0000","last_change":"2024-05-08T18:04:53.847736+0000","last_active":"2024-05-08T18:07:54.043126+0000","last_peered":"2024-05-08T18:07:54.043126+0000","last_clean":"2024-05-08T18:07:54.043126+0000","last_became_active":"2024-05-08T18:04:53.847413+0000","last_became_peered":"2024-05-08T18:04:53.847413+0000","last_unstale":"2024-05-08T18:07:54.043126+0000","last_undegraded":"2024-05-08T18:07:54.043126+0000","last_fullsized":"2024-05-08T18:07:54.043126+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"29'12","reported_seq":69,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.835658+0000","last_change":"2024-05-08T18:08:17.835711+0000","last_active":"2024-05-08T18:08:17.835658+0000","last_peered":"2024-05-08T18:08:17.835658+0000","last_clean":"2024-05-08T18:08:17.835658+0000","last_became_active":"2024-05-08T18:04:45.830329+0000","last_became_peered":"2024-05-08T18:04:45.830329+0000","last_unstale":"2024-05-08T18:08:17.835658+0000","last_undegraded":"2024-05-08T18:08:17.835658+0000","last_fullsized":"2024-05-08T18:08:17.835658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:17.835540+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:17.835540+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.835540+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:36.036099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:52.463640+0000","last_change":"2024-05-08T18:08:52.463670+0000","last_active":"2024-05-08T18:08:52.463640+0000","last_peered":"2024-05-08T18:08:52.463640+0000","last_clean":"2024-05-08T18:08:52.463640+0000","last_became_active":"2024-05-08T18:04:43.835227+0000","last_became_peered":"2024-05-08T18:04:43.835227+0000","last_unstale":"2024-05-08T18:08:52.463640+0000","last_undegraded":"2024-05-08T18:08:52.463640+0000","last_fullsized":"2024-05-08T18:08:52.463640+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:52.463521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:52.463521+0000","last_clean_scrub_stamp":"2024-05-08T18:08:52.463521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:05.463037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3,2],"acting":[1,0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:56.000672+0000","last_change":"2024-05-08T18:08:56.000704+0000","last_active":"2024-05-08T18:08:56.000672+0000","last_peered":"2024-05-08T18:08:56.000672+0000","last_clean":"2024-05-08T18:08:56.000672+0000","last_became_active":"2024-05-08T18:05:30.870155+0000","last_became_peered":"2024-05-08T18:05:30.870155+0000","last_unstale":"2024-05-08T18:08:56.000672+0000","last_undegraded":"2024-05-08T18:08:56.000672+0000","last_fullsized":"2024-05-08T18:08:56.000672+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:56.000581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:56.000581+0000","last_clean_scrub_stamp":"2024-05-08T18:08:56.000581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:19.118117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"29'39","reported_seq":96,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.536541+0000","last_change":"2024-05-08T18:08:12.536541+0000","last_active":"2024-05-08T18:08:12.536541+0000","last_peered":"2024-05-08T18:08:12.536541+0000","last_clean":"2024-05-08T18:08:12.536541+0000","last_became_active":"2024-05-08T18:04:49.832649+0000","last_became_peered":"2024-05-08T18:04:49.832649+0000","last_unstale":"2024-05-08T18:08:12.536541+0000","last_undegraded":"2024-05-08T18:08:12.536541+0000","last_fullsized":"2024-05-08T18:08:12.536541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'39","last_scrub_stamp":"2024-05-08T18:08:12.536412+0000","last_deep_scrub":"29'39","last_deep_scrub_stamp":"2024-05-08T18:08:12.536412+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.536412+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:31.365352+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.444675+0000","last_change":"2024-05-08T18:08:05.444675+0000","last_active":"2024-05-08T18:08:05.444675+0000","last_peered":"2024-05-08T18:08:05.444675+0000","last_clean":"2024-05-08T18:08:05.444675+0000","last_became_active":"2024-05-08T18:04:51.833421+0000","last_became_peered":"2024-05-08T18:04:51.833421+0000","last_unstale":"2024-05-08T18:08:05.444675+0000","last_undegraded":"2024-05-08T18:08:05.444675+0000","last_fullsized":"2024-05-08T18:08:05.444675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:05.444585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:05.444585+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.444585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:15.978668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"29'5","reported_seq":24,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747669+0000","last_change":"2024-05-08T18:04:53.848762+0000","last_active":"2024-05-08T18:07:13.747669+0000","last_peered":"2024-05-08T18:07:13.747669+0000","last_clean":"2024-05-08T18:07:13.747669+0000","last_became_active":"2024-05-08T18:04:53.848430+0000","last_became_peered":"2024-05-08T18:04:53.848430+0000","last_unstale":"2024-05-08T18:07:13.747669+0000","last_undegraded":"2024-05-08T18:07:13.747669+0000","last_fullsized":"2024-05-08T18:07:13.747669+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"29'10","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:37.437418+0000","last_change":"2024-05-08T18:08:37.437444+0000","last_active":"2024-05-08T18:08:37.437418+0000","last_peered":"2024-05-08T18:08:37.437418+0000","last_clean":"2024-05-08T18:08:37.437418+0000","last_became_active":"2024-05-08T18:04:45.806808+0000","last_became_peered":"2024-05-08T18:04:45.806808+0000","last_unstale":"2024-05-08T18:08:37.437418+0000","last_undegraded":"2024-05-08T18:08:37.437418+0000","last_fullsized":"2024-05-08T18:08:37.437418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:37.437332+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:37.437332+0000","last_clean_scrub_stamp":"2024-05-08T18:08:37.437332+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:55.637891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:20.985641+0000","last_change":"2024-05-08T18:09:20.985689+0000","last_active":"2024-05-08T18:09:20.985641+0000","last_peered":"2024-05-08T18:09:20.985641+0000","last_clean":"2024-05-08T18:09:20.985641+0000","last_became_active":"2024-05-08T18:04:43.832147+0000","last_became_peered":"2024-05-08T18:04:43.832147+0000","last_unstale":"2024-05-08T18:09:20.985641+0000","last_undegraded":"2024-05-08T18:09:20.985641+0000","last_fullsized":"2024-05-08T18:09:20.985641+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:20.985513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:20.985513+0000","last_clean_scrub_stamp":"2024-05-08T18:09:20.985513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:03:07.156148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.517382+0000","last_change":"2024-05-08T18:08:22.517404+0000","last_active":"2024-05-08T18:08:22.517382+0000","last_peered":"2024-05-08T18:08:22.517382+0000","last_clean":"2024-05-08T18:08:22.517382+0000","last_became_active":"2024-05-08T18:04:47.815219+0000","last_became_peered":"2024-05-08T18:04:47.815219+0000","last_unstale":"2024-05-08T18:08:22.517382+0000","last_undegraded":"2024-05-08T18:08:22.517382+0000","last_fullsized":"2024-05-08T18:08:22.517382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.517292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.517292+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.517292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:28.093920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"29'15","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:38.037896+0000","last_change":"2024-05-08T18:08:38.037896+0000","last_active":"2024-05-08T18:08:38.037896+0000","last_peered":"2024-05-08T18:08:38.037896+0000","last_clean":"2024-05-08T18:08:38.037896+0000","last_became_active":"2024-05-08T18:04:49.830793+0000","last_became_peered":"2024-05-08T18:04:49.830793+0000","last_unstale":"2024-05-08T18:08:38.037896+0000","last_undegraded":"2024-05-08T18:08:38.037896+0000","last_fullsized":"2024-05-08T18:08:38.037896+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:38.037761+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:38.037761+0000","last_clean_scrub_stamp":"2024-05-08T18:08:38.037761+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:37:36.379302+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"29'285","reported_seq":379,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.160257+0000","last_change":"2024-05-08T18:08:09.160257+0000","last_active":"2024-05-08T18:08:09.160257+0000","last_peered":"2024-05-08T18:08:09.160257+0000","last_clean":"2024-05-08T18:08:09.160257+0000","last_became_active":"2024-05-08T18:04:51.833866+0000","last_became_peered":"2024-05-08T18:04:51.833866+0000","last_unstale":"2024-05-08T18:08:09.160257+0000","last_undegraded":"2024-05-08T18:08:09.160257+0000","last_fullsized":"2024-05-08T18:08:09.160257+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:09.160141+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:09.160141+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.160141+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:29:48.216550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:13.747522+0000","last_change":"2024-05-08T18:04:53.848804+0000","last_active":"2024-05-08T18:07:13.747522+0000","last_peered":"2024-05-08T18:07:13.747522+0000","last_clean":"2024-05-08T18:07:13.747522+0000","last_became_active":"2024-05-08T18:04:53.848321+0000","last_became_peered":"2024-05-08T18:04:53.848321+0000","last_unstale":"2024-05-08T18:07:13.747522+0000","last_undegraded":"2024-05-08T18:07:13.747522+0000","last_fullsized":"2024-05-08T18:07:13.747522+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"29'4","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:33.249650+0000","last_change":"2024-05-08T18:08:33.249675+0000","last_active":"2024-05-08T18:08:33.249650+0000","last_peered":"2024-05-08T18:08:33.249650+0000","last_clean":"2024-05-08T18:08:33.249650+0000","last_became_active":"2024-05-08T18:04:45.813424+0000","last_became_peered":"2024-05-08T18:04:45.813424+0000","last_unstale":"2024-05-08T18:08:33.249650+0000","last_undegraded":"2024-05-08T18:08:33.249650+0000","last_fullsized":"2024-05-08T18:08:33.249650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:33.249561+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:33.249561+0000","last_clean_scrub_stamp":"2024-05-08T18:08:33.249561+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:14.493205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:42.532785+0000","last_change":"2024-05-08T18:08:42.532811+0000","last_active":"2024-05-08T18:08:42.532785+0000","last_peered":"2024-05-08T18:08:42.532785+0000","last_clean":"2024-05-08T18:08:42.532785+0000","last_became_active":"2024-05-08T18:04:43.840502+0000","last_became_peered":"2024-05-08T18:04:43.840502+0000","last_unstale":"2024-05-08T18:08:42.532785+0000","last_undegraded":"2024-05-08T18:08:42.532785+0000","last_fullsized":"2024-05-08T18:08:42.532785+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:42.532676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:42.532676+0000","last_clean_scrub_stamp":"2024-05-08T18:08:42.532676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:06.240119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.519639+0000","last_change":"2024-05-08T18:08:18.519666+0000","last_active":"2024-05-08T18:08:18.519639+0000","last_peered":"2024-05-08T18:08:18.519639+0000","last_clean":"2024-05-08T18:08:18.519639+0000","last_became_active":"2024-05-08T18:04:47.818839+0000","last_became_peered":"2024-05-08T18:04:47.818839+0000","last_unstale":"2024-05-08T18:08:18.519639+0000","last_undegraded":"2024-05-08T18:08:18.519639+0000","last_fullsized":"2024-05-08T18:08:18.519639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:18.519542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:18.519542+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.519542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:14:21.272156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"29'46","reported_seq":104,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.495547+0000","last_change":"2024-05-08T18:08:12.495592+0000","last_active":"2024-05-08T18:08:12.495547+0000","last_peered":"2024-05-08T18:08:12.495547+0000","last_clean":"2024-05-08T18:08:12.495547+0000","last_became_active":"2024-05-08T18:04:49.824405+0000","last_became_peered":"2024-05-08T18:04:49.824405+0000","last_unstale":"2024-05-08T18:08:12.495547+0000","last_undegraded":"2024-05-08T18:08:12.495547+0000","last_fullsized":"2024-05-08T18:08:12.495547+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'46","last_scrub_stamp":"2024-05-08T18:08:12.495454+0000","last_deep_scrub":"29'46","last_deep_scrub_stamp":"2024-05-08T18:08:12.495454+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.495454+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:48:04.037407+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.562545+0000","last_change":"2024-05-08T18:08:17.562569+0000","last_active":"2024-05-08T18:08:17.562545+0000","last_peered":"2024-05-08T18:08:17.562545+0000","last_clean":"2024-05-08T18:08:17.562545+0000","last_became_active":"2024-05-08T18:04:51.844757+0000","last_became_peered":"2024-05-08T18:04:51.844757+0000","last_unstale":"2024-05-08T18:08:17.562545+0000","last_undegraded":"2024-05-08T18:08:17.562545+0000","last_fullsized":"2024-05-08T18:08:17.562545+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:17.562441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:17.562441+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.562441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:50:03.644980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:33.515849+0000","last_change":"2024-05-08T18:08:33.515879+0000","last_active":"2024-05-08T18:08:33.515849+0000","last_peered":"2024-05-08T18:08:33.515849+0000","last_clean":"2024-05-08T18:08:33.515849+0000","last_became_active":"2024-05-08T18:04:53.840956+0000","last_became_peered":"2024-05-08T18:04:53.840956+0000","last_unstale":"2024-05-08T18:08:33.515849+0000","last_undegraded":"2024-05-08T18:08:33.515849+0000","last_fullsized":"2024-05-08T18:08:33.515849+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:33.515757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:33.515757+0000","last_clean_scrub_stamp":"2024-05-08T18:08:33.515757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:29.798801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"29'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:28.283892+0000","last_change":"2024-05-08T18:08:28.283933+0000","last_active":"2024-05-08T18:08:28.283892+0000","last_peered":"2024-05-08T18:08:28.283892+0000","last_clean":"2024-05-08T18:08:28.283892+0000","last_became_active":"2024-05-08T18:04:45.806560+0000","last_became_peered":"2024-05-08T18:04:45.806560+0000","last_unstale":"2024-05-08T18:08:28.283892+0000","last_undegraded":"2024-05-08T18:08:28.283892+0000","last_fullsized":"2024-05-08T18:08:28.283892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:28.283774+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:28.283774+0000","last_clean_scrub_stamp":"2024-05-08T18:08:28.283774+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:55:22.065846+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":48,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:12.966105+0000","last_change":"2024-05-08T18:09:12.966165+0000","last_active":"2024-05-08T18:09:12.966105+0000","last_peered":"2024-05-08T18:09:12.966105+0000","last_clean":"2024-05-08T18:09:12.966105+0000","last_became_active":"2024-05-08T18:04:43.845214+0000","last_became_peered":"2024-05-08T18:04:43.845214+0000","last_unstale":"2024-05-08T18:09:12.966105+0000","last_undegraded":"2024-05-08T18:09:12.966105+0000","last_fullsized":"2024-05-08T18:09:12.966105+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:12.965986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:12.965986+0000","last_clean_scrub_stamp":"2024-05-08T18:09:12.965986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:31.443445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.286599+0000","last_change":"2024-05-08T18:08:23.286625+0000","last_active":"2024-05-08T18:08:23.286599+0000","last_peered":"2024-05-08T18:08:23.286599+0000","last_clean":"2024-05-08T18:08:23.286599+0000","last_became_active":"2024-05-08T18:04:47.822748+0000","last_became_peered":"2024-05-08T18:04:47.822748+0000","last_unstale":"2024-05-08T18:08:23.286599+0000","last_undegraded":"2024-05-08T18:08:23.286599+0000","last_fullsized":"2024-05-08T18:08:23.286599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:23.286509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:23.286509+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.286509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:17:35.464618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"29'37","reported_seq":92,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.576260+0000","last_change":"2024-05-08T18:08:14.576300+0000","last_active":"2024-05-08T18:08:14.576260+0000","last_peered":"2024-05-08T18:08:14.576260+0000","last_clean":"2024-05-08T18:08:14.576260+0000","last_became_active":"2024-05-08T18:04:49.831440+0000","last_became_peered":"2024-05-08T18:04:49.831440+0000","last_unstale":"2024-05-08T18:08:14.576260+0000","last_undegraded":"2024-05-08T18:08:14.576260+0000","last_fullsized":"2024-05-08T18:08:14.576260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'37","last_scrub_stamp":"2024-05-08T18:08:14.576150+0000","last_deep_scrub":"29'37","last_deep_scrub_stamp":"2024-05-08T18:08:14.576150+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.576150+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:06:14.892455+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:27.822646+0000","last_change":"2024-05-08T18:08:27.822675+0000","last_active":"2024-05-08T18:08:27.822646+0000","last_peered":"2024-05-08T18:08:27.822646+0000","last_clean":"2024-05-08T18:08:27.822646+0000","last_became_active":"2024-05-08T18:04:51.844125+0000","last_became_peered":"2024-05-08T18:04:51.844125+0000","last_unstale":"2024-05-08T18:08:27.822646+0000","last_undegraded":"2024-05-08T18:08:27.822646+0000","last_fullsized":"2024-05-08T18:08:27.822646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:27.822555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:27.822555+0000","last_clean_scrub_stamp":"2024-05-08T18:08:27.822555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:47:50.940092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:07:18.748549+0000","last_change":"2024-05-08T18:04:53.849962+0000","last_active":"2024-05-08T18:07:18.748549+0000","last_peered":"2024-05-08T18:07:18.748549+0000","last_clean":"2024-05-08T18:07:18.748549+0000","last_became_active":"2024-05-08T18:04:53.849514+0000","last_became_peered":"2024-05-08T18:04:53.849514+0000","last_unstale":"2024-05-08T18:07:18.748549+0000","last_undegraded":"2024-05-08T18:07:18.748549+0000","last_fullsized":"2024-05-08T18:07:18.748549+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:04:52.818692+0000","last_clean_scrub_stamp":"2024-05-08T18:04:52.818692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"29'4","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:54.075447+0000","last_change":"2024-05-08T18:08:54.075473+0000","last_active":"2024-05-08T18:08:54.075447+0000","last_peered":"2024-05-08T18:08:54.075447+0000","last_clean":"2024-05-08T18:08:54.075447+0000","last_became_active":"2024-05-08T18:04:45.822204+0000","last_became_peered":"2024-05-08T18:04:45.822204+0000","last_unstale":"2024-05-08T18:08:54.075447+0000","last_undegraded":"2024-05-08T18:08:54.075447+0000","last_fullsized":"2024-05-08T18:08:54.075447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:54.075350+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:54.075350+0000","last_clean_scrub_stamp":"2024-05-08T18:08:54.075350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:04.754616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"29'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:48.395613+0000","last_change":"2024-05-08T18:08:48.395693+0000","last_active":"2024-05-08T18:08:48.395613+0000","last_peered":"2024-05-08T18:08:48.395613+0000","last_clean":"2024-05-08T18:08:48.395613+0000","last_became_active":"2024-05-08T18:04:43.834678+0000","last_became_peered":"2024-05-08T18:04:43.834678+0000","last_unstale":"2024-05-08T18:08:48.395613+0000","last_undegraded":"2024-05-08T18:08:48.395613+0000","last_fullsized":"2024-05-08T18:08:48.395613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:48.395512+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:48.395512+0000","last_clean_scrub_stamp":"2024-05-08T18:08:48.395512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:55.075303+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.501159+0000","last_change":"2024-05-08T18:08:16.501182+0000","last_active":"2024-05-08T18:08:16.501159+0000","last_peered":"2024-05-08T18:08:16.501159+0000","last_clean":"2024-05-08T18:08:16.501159+0000","last_became_active":"2024-05-08T18:04:47.820036+0000","last_became_peered":"2024-05-08T18:04:47.820036+0000","last_unstale":"2024-05-08T18:08:16.501159+0000","last_undegraded":"2024-05-08T18:08:16.501159+0000","last_fullsized":"2024-05-08T18:08:16.501159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:16.501065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:16.501065+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.501065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:49.649348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"29'32","reported_seq":85,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.857112+0000","last_change":"2024-05-08T18:08:20.857184+0000","last_active":"2024-05-08T18:08:20.857112+0000","last_peered":"2024-05-08T18:08:20.857112+0000","last_clean":"2024-05-08T18:08:20.857112+0000","last_became_active":"2024-05-08T18:04:49.830882+0000","last_became_peered":"2024-05-08T18:04:49.830882+0000","last_unstale":"2024-05-08T18:08:20.857112+0000","last_undegraded":"2024-05-08T18:08:20.857112+0000","last_fullsized":"2024-05-08T18:08:20.857112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'32","last_scrub_stamp":"2024-05-08T18:08:20.857010+0000","last_deep_scrub":"29'32","last_deep_scrub_stamp":"2024-05-08T18:08:20.857010+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.857010+0000","objects_scrubbed":2,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:06.682042+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":49,"num_write":14,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:29.815248+0000","last_change":"2024-05-08T18:08:29.815276+0000","last_active":"2024-05-08T18:08:29.815248+0000","last_peered":"2024-05-08T18:08:29.815248+0000","last_clean":"2024-05-08T18:08:29.815248+0000","last_became_active":"2024-05-08T18:04:51.842734+0000","last_became_peered":"2024-05-08T18:04:51.842734+0000","last_unstale":"2024-05-08T18:08:29.815248+0000","last_undegraded":"2024-05-08T18:08:29.815248+0000","last_fullsized":"2024-05-08T18:08:29.815248+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:29.815133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:29.815133+0000","last_clean_scrub_stamp":"2024-05-08T18:08:29.815133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:51:13.451798+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'10","reported_seq":36,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:57.037492+0000","last_change":"2024-05-08T18:08:57.037521+0000","last_active":"2024-05-08T18:08:57.037492+0000","last_peered":"2024-05-08T18:08:57.037492+0000","last_clean":"2024-05-08T18:08:57.037492+0000","last_became_active":"2024-05-08T18:04:53.844891+0000","last_became_peered":"2024-05-08T18:04:53.844891+0000","last_unstale":"2024-05-08T18:08:57.037492+0000","last_undegraded":"2024-05-08T18:08:57.037492+0000","last_fullsized":"2024-05-08T18:08:57.037492+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:57.037400+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:57.037400+0000","last_clean_scrub_stamp":"2024-05-08T18:08:57.037400+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:33:31.062988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"29'2","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:27.291722+0000","last_change":"2024-05-08T18:08:27.291750+0000","last_active":"2024-05-08T18:08:27.291722+0000","last_peered":"2024-05-08T18:08:27.291722+0000","last_clean":"2024-05-08T18:08:27.291722+0000","last_became_active":"2024-05-08T18:04:45.808936+0000","last_became_peered":"2024-05-08T18:04:45.808936+0000","last_unstale":"2024-05-08T18:08:27.291722+0000","last_undegraded":"2024-05-08T18:08:27.291722+0000","last_fullsized":"2024-05-08T18:08:27.291722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:27.291616+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:27.291616+0000","last_clean_scrub_stamp":"2024-05-08T18:08:27.291616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:50:13.374155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:58.481285+0000","last_change":"2024-05-08T18:08:58.481320+0000","last_active":"2024-05-08T18:08:58.481285+0000","last_peered":"2024-05-08T18:08:58.481285+0000","last_clean":"2024-05-08T18:08:58.481285+0000","last_became_active":"2024-05-08T18:04:43.837070+0000","last_became_peered":"2024-05-08T18:04:43.837070+0000","last_unstale":"2024-05-08T18:08:58.481285+0000","last_undegraded":"2024-05-08T18:08:58.481285+0000","last_fullsized":"2024-05-08T18:08:58.481285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:58.481195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:58.481195+0000","last_clean_scrub_stamp":"2024-05-08T18:08:58.481195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:16.399630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"29'3","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.481725+0000","last_change":"2024-05-08T18:08:15.481759+0000","last_active":"2024-05-08T18:08:15.481725+0000","last_peered":"2024-05-08T18:08:15.481725+0000","last_clean":"2024-05-08T18:08:15.481725+0000","last_became_active":"2024-05-08T18:04:47.824172+0000","last_became_peered":"2024-05-08T18:04:47.824172+0000","last_unstale":"2024-05-08T18:08:15.481725+0000","last_undegraded":"2024-05-08T18:08:15.481725+0000","last_fullsized":"2024-05-08T18:08:15.481725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3","last_scrub_stamp":"2024-05-08T18:08:15.481627+0000","last_deep_scrub":"29'3","last_deep_scrub_stamp":"2024-05-08T18:08:15.481627+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.481627+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:31:00.387286+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"29'40","reported_seq":97,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:33.898989+0000","last_change":"2024-05-08T18:08:33.899046+0000","last_active":"2024-05-08T18:08:33.898989+0000","last_peered":"2024-05-08T18:08:33.898989+0000","last_clean":"2024-05-08T18:08:33.898989+0000","last_became_active":"2024-05-08T18:04:49.831803+0000","last_became_peered":"2024-05-08T18:04:49.831803+0000","last_unstale":"2024-05-08T18:08:33.898989+0000","last_undegraded":"2024-05-08T18:08:33.898989+0000","last_fullsized":"2024-05-08T18:08:33.898989+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'40","last_scrub_stamp":"2024-05-08T18:08:33.898897+0000","last_deep_scrub":"29'40","last_deep_scrub_stamp":"2024-05-08T18:08:33.898897+0000","last_clean_scrub_stamp":"2024-05-08T18:08:33.898897+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:05:09.280810+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1270,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:25.577300+0000","last_change":"2024-05-08T18:08:25.577348+0000","last_active":"2024-05-08T18:08:25.577300+0000","last_peered":"2024-05-08T18:08:25.577300+0000","last_clean":"2024-05-08T18:08:25.577300+0000","last_became_active":"2024-05-08T18:04:51.846225+0000","last_became_peered":"2024-05-08T18:04:51.846225+0000","last_unstale":"2024-05-08T18:08:25.577300+0000","last_undegraded":"2024-05-08T18:08:25.577300+0000","last_fullsized":"2024-05-08T18:08:25.577300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:25.577162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:25.577162+0000","last_clean_scrub_stamp":"2024-05-08T18:08:25.577162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:15.650574+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"29'4","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:53.121061+0000","last_change":"2024-05-08T18:08:53.121087+0000","last_active":"2024-05-08T18:08:53.121061+0000","last_peered":"2024-05-08T18:08:53.121061+0000","last_clean":"2024-05-08T18:08:53.121061+0000","last_became_active":"2024-05-08T18:04:53.842429+0000","last_became_peered":"2024-05-08T18:04:53.842429+0000","last_unstale":"2024-05-08T18:08:53.121061+0000","last_undegraded":"2024-05-08T18:08:53.121061+0000","last_fullsized":"2024-05-08T18:08:53.121061+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:53.120970+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:53.120970+0000","last_clean_scrub_stamp":"2024-05-08T18:08:53.120970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:15.271281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"27'2","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:32.506149+0000","last_change":"2024-05-08T18:08:32.506149+0000","last_active":"2024-05-08T18:08:32.506149+0000","last_peered":"2024-05-08T18:08:32.506149+0000","last_clean":"2024-05-08T18:08:32.506149+0000","last_became_active":"2024-05-08T18:04:45.818161+0000","last_became_peered":"2024-05-08T18:04:45.818161+0000","last_unstale":"2024-05-08T18:08:32.506149+0000","last_undegraded":"2024-05-08T18:08:32.506149+0000","last_fullsized":"2024-05-08T18:08:32.506149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:32.506023+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:32.506023+0000","last_clean_scrub_stamp":"2024-05-08T18:08:32.506023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:20.670584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:38.182450+0000","last_change":"2024-05-08T18:08:38.182450+0000","last_active":"2024-05-08T18:08:38.182450+0000","last_peered":"2024-05-08T18:08:38.182450+0000","last_clean":"2024-05-08T18:08:38.182450+0000","last_became_active":"2024-05-08T18:04:43.842483+0000","last_became_peered":"2024-05-08T18:04:43.842483+0000","last_unstale":"2024-05-08T18:08:38.182450+0000","last_undegraded":"2024-05-08T18:08:38.182450+0000","last_fullsized":"2024-05-08T18:08:38.182450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:38.182336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:38.182336+0000","last_clean_scrub_stamp":"2024-05-08T18:08:38.182336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:54:29.772068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"29'36","reported_seq":116,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:41.377160+0000","last_change":"2024-05-08T18:08:41.377160+0000","last_active":"2024-05-08T18:08:41.377160+0000","last_peered":"2024-05-08T18:08:41.377160+0000","last_clean":"2024-05-08T18:08:41.377160+0000","last_became_active":"2024-05-08T18:04:45.807847+0000","last_became_peered":"2024-05-08T18:04:45.807847+0000","last_unstale":"2024-05-08T18:08:41.377160+0000","last_undegraded":"2024-05-08T18:08:41.377160+0000","last_fullsized":"2024-05-08T18:08:41.377160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:08:41.376990+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:08:41.376990+0000","last_clean_scrub_stamp":"2024-05-08T18:08:41.376990+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:19:16.885027+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":44,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:36.472685+0000","last_change":"2024-05-08T18:08:36.472685+0000","last_active":"2024-05-08T18:08:36.472685+0000","last_peered":"2024-05-08T18:08:36.472685+0000","last_clean":"2024-05-08T18:08:36.472685+0000","last_became_active":"2024-05-08T18:04:43.836045+0000","last_became_peered":"2024-05-08T18:04:43.836045+0000","last_unstale":"2024-05-08T18:08:36.472685+0000","last_undegraded":"2024-05-08T18:08:36.472685+0000","last_fullsized":"2024-05-08T18:08:36.472685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:36.472507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:36.472507+0000","last_clean_scrub_stamp":"2024-05-08T18:08:36.472507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:13.020912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3,2],"acting":[0,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"29'12","reported_seq":79,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:47.107135+0000","last_change":"2024-05-08T18:08:47.107135+0000","last_active":"2024-05-08T18:08:47.107135+0000","last_peered":"2024-05-08T18:08:47.107135+0000","last_clean":"2024-05-08T18:08:47.107135+0000","last_became_active":"2024-05-08T18:04:45.825506+0000","last_became_peered":"2024-05-08T18:04:45.825506+0000","last_unstale":"2024-05-08T18:08:47.107135+0000","last_undegraded":"2024-05-08T18:08:47.107135+0000","last_fullsized":"2024-05-08T18:08:47.107135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:47.106962+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:47.106962+0000","last_clean_scrub_stamp":"2024-05-08T18:08:47.106962+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:30.261074+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":46,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"29'6","reported_seq":60,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:19.940669+0000","last_change":"2024-05-08T18:09:19.940707+0000","last_active":"2024-05-08T18:09:19.940669+0000","last_peered":"2024-05-08T18:09:19.940669+0000","last_clean":"2024-05-08T18:09:19.940669+0000","last_became_active":"2024-05-08T18:04:43.832406+0000","last_became_peered":"2024-05-08T18:04:43.832406+0000","last_unstale":"2024-05-08T18:09:19.940669+0000","last_undegraded":"2024-05-08T18:09:19.940669+0000","last_fullsized":"2024-05-08T18:09:19.940669+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:09:19.940563+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:09:19.940563+0000","last_clean_scrub_stamp":"2024-05-08T18:09:19.940563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:41:55.643608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":8,"num_write":30,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"29'2","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:35.470976+0000","last_change":"2024-05-08T18:08:35.470976+0000","last_active":"2024-05-08T18:08:35.470976+0000","last_peered":"2024-05-08T18:08:35.470976+0000","last_clean":"2024-05-08T18:08:35.470976+0000","last_became_active":"2024-05-08T18:04:45.806646+0000","last_became_peered":"2024-05-08T18:04:45.806646+0000","last_unstale":"2024-05-08T18:08:35.470976+0000","last_undegraded":"2024-05-08T18:08:35.470976+0000","last_fullsized":"2024-05-08T18:08:35.470976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:35.470848+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:35.470848+0000","last_clean_scrub_stamp":"2024-05-08T18:08:35.470848+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:55.258041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"29'6","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:40.377297+0000","last_change":"2024-05-08T18:08:40.377297+0000","last_active":"2024-05-08T18:08:40.377297+0000","last_peered":"2024-05-08T18:08:40.377297+0000","last_clean":"2024-05-08T18:08:40.377297+0000","last_became_active":"2024-05-08T18:04:43.835163+0000","last_became_peered":"2024-05-08T18:04:43.835163+0000","last_unstale":"2024-05-08T18:08:40.377297+0000","last_undegraded":"2024-05-08T18:08:40.377297+0000","last_fullsized":"2024-05-08T18:08:40.377297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:40.377108+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:40.377108+0000","last_clean_scrub_stamp":"2024-05-08T18:08:40.377108+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:26.202140+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":4746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,1],"acting":[0,3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"29'2","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:34.514234+0000","last_change":"2024-05-08T18:08:34.514234+0000","last_active":"2024-05-08T18:08:34.514234+0000","last_peered":"2024-05-08T18:08:34.514234+0000","last_clean":"2024-05-08T18:08:34.514234+0000","last_became_active":"2024-05-08T18:04:45.816391+0000","last_became_peered":"2024-05-08T18:04:45.816391+0000","last_unstale":"2024-05-08T18:08:34.514234+0000","last_undegraded":"2024-05-08T18:08:34.514234+0000","last_fullsized":"2024-05-08T18:08:34.514234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:34.514122+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:34.514122+0000","last_clean_scrub_stamp":"2024-05-08T18:08:34.514122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:11.960319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'4","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:42.343533+0000","last_change":"2024-05-08T18:08:42.343533+0000","last_active":"2024-05-08T18:08:42.343533+0000","last_peered":"2024-05-08T18:08:42.343533+0000","last_clean":"2024-05-08T18:08:42.343533+0000","last_became_active":"2024-05-08T18:04:43.835268+0000","last_became_peered":"2024-05-08T18:04:43.835268+0000","last_unstale":"2024-05-08T18:08:42.343533+0000","last_undegraded":"2024-05-08T18:08:42.343533+0000","last_fullsized":"2024-05-08T18:08:42.343533+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-08T18:08:42.343414+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-08T18:08:42.343414+0000","last_clean_scrub_stamp":"2024-05-08T18:08:42.343414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:24.879935+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"29'12","reported_seq":87,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:28.537351+0000","last_change":"2024-05-08T18:08:28.537351+0000","last_active":"2024-05-08T18:08:28.537351+0000","last_peered":"2024-05-08T18:08:28.537351+0000","last_clean":"2024-05-08T18:08:28.537351+0000","last_became_active":"2024-05-08T18:04:45.819699+0000","last_became_peered":"2024-05-08T18:04:45.819699+0000","last_unstale":"2024-05-08T18:08:28.537351+0000","last_undegraded":"2024-05-08T18:08:28.537351+0000","last_fullsized":"2024-05-08T18:08:28.537351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:28.537227+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:28.537227+0000","last_clean_scrub_stamp":"2024-05-08T18:08:28.537227+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:18.610639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:57.463007+0000","last_change":"2024-05-08T18:08:57.463007+0000","last_active":"2024-05-08T18:08:57.463007+0000","last_peered":"2024-05-08T18:08:57.463007+0000","last_clean":"2024-05-08T18:08:57.463007+0000","last_became_active":"2024-05-08T18:04:43.836735+0000","last_became_peered":"2024-05-08T18:04:43.836735+0000","last_unstale":"2024-05-08T18:08:57.463007+0000","last_undegraded":"2024-05-08T18:08:57.463007+0000","last_fullsized":"2024-05-08T18:08:57.463007+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:57.462845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:57.462845+0000","last_clean_scrub_stamp":"2024-05-08T18:08:57.462845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:44.500274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"29'14","reported_seq":80,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:47.377267+0000","last_change":"2024-05-08T18:08:47.377267+0000","last_active":"2024-05-08T18:08:47.377267+0000","last_peered":"2024-05-08T18:08:47.377267+0000","last_clean":"2024-05-08T18:08:47.377267+0000","last_became_active":"2024-05-08T18:04:45.811953+0000","last_became_peered":"2024-05-08T18:04:45.811953+0000","last_unstale":"2024-05-08T18:08:47.377267+0000","last_undegraded":"2024-05-08T18:08:47.377267+0000","last_fullsized":"2024-05-08T18:08:47.377267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'14","last_scrub_stamp":"2024-05-08T18:08:47.377155+0000","last_deep_scrub":"29'14","last_deep_scrub_stamp":"2024-05-08T18:08:47.377155+0000","last_clean_scrub_stamp":"2024-05-08T18:08:47.377155+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:10.494691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:34.213484+0000","last_change":"2024-05-08T18:08:34.213484+0000","last_active":"2024-05-08T18:08:34.213484+0000","last_peered":"2024-05-08T18:08:34.213484+0000","last_clean":"2024-05-08T18:08:34.213484+0000","last_became_active":"2024-05-08T18:04:43.842179+0000","last_became_peered":"2024-05-08T18:04:43.842179+0000","last_unstale":"2024-05-08T18:08:34.213484+0000","last_undegraded":"2024-05-08T18:08:34.213484+0000","last_fullsized":"2024-05-08T18:08:34.213484+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:34.213370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:34.213370+0000","last_clean_scrub_stamp":"2024-05-08T18:08:34.213370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:13.048212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,0],"acting":[2,3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"29'8","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:59.502471+0000","last_change":"2024-05-08T18:08:59.502501+0000","last_active":"2024-05-08T18:08:59.502471+0000","last_peered":"2024-05-08T18:08:59.502471+0000","last_clean":"2024-05-08T18:08:59.502471+0000","last_became_active":"2024-05-08T18:04:45.812637+0000","last_became_peered":"2024-05-08T18:04:45.812637+0000","last_unstale":"2024-05-08T18:08:59.502471+0000","last_undegraded":"2024-05-08T18:08:59.502471+0000","last_fullsized":"2024-05-08T18:08:59.502471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:59.502380+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:59.502380+0000","last_clean_scrub_stamp":"2024-05-08T18:08:59.502380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:08:00.350733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"29'6","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:58.521154+0000","last_change":"2024-05-08T18:08:58.521154+0000","last_active":"2024-05-08T18:08:58.521154+0000","last_peered":"2024-05-08T18:08:58.521154+0000","last_clean":"2024-05-08T18:08:58.521154+0000","last_became_active":"2024-05-08T18:04:43.834491+0000","last_became_peered":"2024-05-08T18:04:43.834491+0000","last_unstale":"2024-05-08T18:08:58.521154+0000","last_undegraded":"2024-05-08T18:08:58.521154+0000","last_fullsized":"2024-05-08T18:08:58.521154+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:58.520991+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:58.520991+0000","last_clean_scrub_stamp":"2024-05-08T18:08:58.520991+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:32:07.150914+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"29'8","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:34.558842+0000","last_change":"2024-05-08T18:08:34.558889+0000","last_active":"2024-05-08T18:08:34.558842+0000","last_peered":"2024-05-08T18:08:34.558842+0000","last_clean":"2024-05-08T18:08:34.558842+0000","last_became_active":"2024-05-08T18:04:45.815677+0000","last_became_peered":"2024-05-08T18:04:45.815677+0000","last_unstale":"2024-05-08T18:08:34.558842+0000","last_undegraded":"2024-05-08T18:08:34.558842+0000","last_fullsized":"2024-05-08T18:08:34.558842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:34.558754+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:34.558754+0000","last_clean_scrub_stamp":"2024-05-08T18:08:34.558754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:30.175682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:23.050897+0000","last_change":"2024-05-08T18:09:23.050922+0000","last_active":"2024-05-08T18:09:23.050897+0000","last_peered":"2024-05-08T18:09:23.050897+0000","last_clean":"2024-05-08T18:09:23.050897+0000","last_became_active":"2024-05-08T18:04:43.843826+0000","last_became_peered":"2024-05-08T18:04:43.843826+0000","last_unstale":"2024-05-08T18:09:23.050897+0000","last_undegraded":"2024-05-08T18:09:23.050897+0000","last_fullsized":"2024-05-08T18:09:23.050897+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:23.050805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:23.050805+0000","last_clean_scrub_stamp":"2024-05-08T18:09:23.050805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:53:29.990439+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"29'6","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:43.071536+0000","last_change":"2024-05-08T18:08:43.071567+0000","last_active":"2024-05-08T18:08:43.071536+0000","last_peered":"2024-05-08T18:08:43.071536+0000","last_clean":"2024-05-08T18:08:43.071536+0000","last_became_active":"2024-05-08T18:04:45.827252+0000","last_became_peered":"2024-05-08T18:04:45.827252+0000","last_unstale":"2024-05-08T18:08:43.071536+0000","last_undegraded":"2024-05-08T18:08:43.071536+0000","last_fullsized":"2024-05-08T18:08:43.071536+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:43.071451+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:43.071451+0000","last_clean_scrub_stamp":"2024-05-08T18:08:43.071451+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:41:02.800248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:55.463831+0000","last_change":"2024-05-08T18:08:55.463831+0000","last_active":"2024-05-08T18:08:55.463831+0000","last_peered":"2024-05-08T18:08:55.463831+0000","last_clean":"2024-05-08T18:08:55.463831+0000","last_became_active":"2024-05-08T18:04:43.820360+0000","last_became_peered":"2024-05-08T18:04:43.820360+0000","last_unstale":"2024-05-08T18:08:55.463831+0000","last_undegraded":"2024-05-08T18:08:55.463831+0000","last_fullsized":"2024-05-08T18:08:55.463831+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:55.463676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:55.463676+0000","last_clean_scrub_stamp":"2024-05-08T18:08:55.463676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:43:05.453001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:31.445845+0000","last_change":"2024-05-08T18:08:31.445845+0000","last_active":"2024-05-08T18:08:31.445845+0000","last_peered":"2024-05-08T18:08:31.445845+0000","last_clean":"2024-05-08T18:08:31.445845+0000","last_became_active":"2024-05-08T18:04:45.823385+0000","last_became_peered":"2024-05-08T18:04:45.823385+0000","last_unstale":"2024-05-08T18:08:31.445845+0000","last_undegraded":"2024-05-08T18:08:31.445845+0000","last_fullsized":"2024-05-08T18:08:31.445845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:31.445729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:31.445729+0000","last_clean_scrub_stamp":"2024-05-08T18:08:31.445729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:27.062658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'6","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:55.454031+0000","last_change":"2024-05-08T18:08:55.454031+0000","last_active":"2024-05-08T18:08:55.454031+0000","last_peered":"2024-05-08T18:08:55.454031+0000","last_clean":"2024-05-08T18:08:55.454031+0000","last_became_active":"2024-05-08T18:04:43.835838+0000","last_became_peered":"2024-05-08T18:04:43.835838+0000","last_unstale":"2024-05-08T18:08:55.454031+0000","last_undegraded":"2024-05-08T18:08:55.454031+0000","last_fullsized":"2024-05-08T18:08:55.454031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:55.453911+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:55.453911+0000","last_clean_scrub_stamp":"2024-05-08T18:08:55.453911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:47:45.526995+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8,"num_write":30,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"29'10","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:41.515484+0000","last_change":"2024-05-08T18:08:41.515484+0000","last_active":"2024-05-08T18:08:41.515484+0000","last_peered":"2024-05-08T18:08:41.515484+0000","last_clean":"2024-05-08T18:08:41.515484+0000","last_became_active":"2024-05-08T18:04:45.810703+0000","last_became_peered":"2024-05-08T18:04:45.810703+0000","last_unstale":"2024-05-08T18:08:41.515484+0000","last_undegraded":"2024-05-08T18:08:41.515484+0000","last_fullsized":"2024-05-08T18:08:41.515484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:41.515361+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:41.515361+0000","last_clean_scrub_stamp":"2024-05-08T18:08:41.515361+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:54:04.536630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"29'4","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:53.458323+0000","last_change":"2024-05-08T18:08:53.458323+0000","last_active":"2024-05-08T18:08:53.458323+0000","last_peered":"2024-05-08T18:08:53.458323+0000","last_clean":"2024-05-08T18:08:53.458323+0000","last_became_active":"2024-05-08T18:04:43.835518+0000","last_became_peered":"2024-05-08T18:04:43.835518+0000","last_unstale":"2024-05-08T18:08:53.458323+0000","last_undegraded":"2024-05-08T18:08:53.458323+0000","last_fullsized":"2024-05-08T18:08:53.458323+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:53.458190+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:53.458190+0000","last_clean_scrub_stamp":"2024-05-08T18:08:53.458190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:31:55.295395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,3],"acting":[1,2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"29'4","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:26.522702+0000","last_change":"2024-05-08T18:08:26.522702+0000","last_active":"2024-05-08T18:08:26.522702+0000","last_peered":"2024-05-08T18:08:26.522702+0000","last_clean":"2024-05-08T18:08:26.522702+0000","last_became_active":"2024-05-08T18:04:45.820746+0000","last_became_peered":"2024-05-08T18:04:45.820746+0000","last_unstale":"2024-05-08T18:08:26.522702+0000","last_undegraded":"2024-05-08T18:08:26.522702+0000","last_fullsized":"2024-05-08T18:08:26.522702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:26.522525+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:26.522525+0000","last_clean_scrub_stamp":"2024-05-08T18:08:26.522525+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:07.766169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:58.037345+0000","last_change":"2024-05-08T18:08:58.037345+0000","last_active":"2024-05-08T18:08:58.037345+0000","last_peered":"2024-05-08T18:08:58.037345+0000","last_clean":"2024-05-08T18:08:58.037345+0000","last_became_active":"2024-05-08T18:04:43.844404+0000","last_became_peered":"2024-05-08T18:04:43.844404+0000","last_unstale":"2024-05-08T18:08:58.037345+0000","last_undegraded":"2024-05-08T18:08:58.037345+0000","last_fullsized":"2024-05-08T18:08:58.037345+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:58.037225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:58.037225+0000","last_clean_scrub_stamp":"2024-05-08T18:08:58.037225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:11:08.420812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"29'4","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:39.501858+0000","last_change":"2024-05-08T18:08:39.501858+0000","last_active":"2024-05-08T18:08:39.501858+0000","last_peered":"2024-05-08T18:08:39.501858+0000","last_clean":"2024-05-08T18:08:39.501858+0000","last_became_active":"2024-05-08T18:04:45.809271+0000","last_became_peered":"2024-05-08T18:04:45.809271+0000","last_unstale":"2024-05-08T18:08:39.501858+0000","last_undegraded":"2024-05-08T18:08:39.501858+0000","last_fullsized":"2024-05-08T18:08:39.501858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:39.501726+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:39.501726+0000","last_clean_scrub_stamp":"2024-05-08T18:08:39.501726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:16.050132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:59.036095+0000","last_change":"2024-05-08T18:08:59.036095+0000","last_active":"2024-05-08T18:08:59.036095+0000","last_peered":"2024-05-08T18:08:59.036095+0000","last_clean":"2024-05-08T18:08:59.036095+0000","last_became_active":"2024-05-08T18:04:43.838125+0000","last_became_peered":"2024-05-08T18:04:43.838125+0000","last_unstale":"2024-05-08T18:08:59.036095+0000","last_undegraded":"2024-05-08T18:08:59.036095+0000","last_fullsized":"2024-05-08T18:08:59.036095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:59.035976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:59.035976+0000","last_clean_scrub_stamp":"2024-05-08T18:08:59.035976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:24:17.825578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"29'15","reported_seq":85,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.868994+0000","last_change":"2024-05-08T18:08:21.868994+0000","last_active":"2024-05-08T18:08:21.868994+0000","last_peered":"2024-05-08T18:08:21.868994+0000","last_clean":"2024-05-08T18:08:21.868994+0000","last_became_active":"2024-05-08T18:04:45.828488+0000","last_became_peered":"2024-05-08T18:04:45.828488+0000","last_unstale":"2024-05-08T18:08:21.868994+0000","last_undegraded":"2024-05-08T18:08:21.868994+0000","last_fullsized":"2024-05-08T18:08:21.868994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:21.868870+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:21.868870+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.868870+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:18:57.376906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:32.274984+0000","last_change":"2024-05-08T18:08:32.274984+0000","last_active":"2024-05-08T18:08:32.274984+0000","last_peered":"2024-05-08T18:08:32.274984+0000","last_clean":"2024-05-08T18:08:32.274984+0000","last_became_active":"2024-05-08T18:04:43.842670+0000","last_became_peered":"2024-05-08T18:04:43.842670+0000","last_unstale":"2024-05-08T18:08:32.274984+0000","last_undegraded":"2024-05-08T18:08:32.274984+0000","last_fullsized":"2024-05-08T18:08:32.274984+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:32.274865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:32.274865+0000","last_clean_scrub_stamp":"2024-05-08T18:08:32.274865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:55.489696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"29'2","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:45.484747+0000","last_change":"2024-05-08T18:08:45.484747+0000","last_active":"2024-05-08T18:08:45.484747+0000","last_peered":"2024-05-08T18:08:45.484747+0000","last_clean":"2024-05-08T18:08:45.484747+0000","last_became_active":"2024-05-08T18:04:45.810562+0000","last_became_peered":"2024-05-08T18:04:45.810562+0000","last_unstale":"2024-05-08T18:08:45.484747+0000","last_undegraded":"2024-05-08T18:08:45.484747+0000","last_fullsized":"2024-05-08T18:08:45.484747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:45.484623+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:45.484623+0000","last_clean_scrub_stamp":"2024-05-08T18:08:45.484623+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:05:22.042022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:02.000689+0000","last_change":"2024-05-08T18:09:02.000689+0000","last_active":"2024-05-08T18:09:02.000689+0000","last_peered":"2024-05-08T18:09:02.000689+0000","last_clean":"2024-05-08T18:09:02.000689+0000","last_became_active":"2024-05-08T18:04:43.835943+0000","last_became_peered":"2024-05-08T18:04:43.835943+0000","last_unstale":"2024-05-08T18:09:02.000689+0000","last_undegraded":"2024-05-08T18:09:02.000689+0000","last_fullsized":"2024-05-08T18:09:02.000689+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:09:02.000540+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:09:02.000540+0000","last_clean_scrub_stamp":"2024-05-08T18:09:02.000540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:57.173467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:05.971768+0000","last_change":"2024-05-08T18:09:05.971768+0000","last_active":"2024-05-08T18:09:05.971768+0000","last_peered":"2024-05-08T18:09:05.971768+0000","last_clean":"2024-05-08T18:09:05.971768+0000","last_became_active":"2024-05-08T18:04:43.836007+0000","last_became_peered":"2024-05-08T18:04:43.836007+0000","last_unstale":"2024-05-08T18:09:05.971768+0000","last_undegraded":"2024-05-08T18:09:05.971768+0000","last_fullsized":"2024-05-08T18:09:05.971768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:05.971607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:05.971607+0000","last_clean_scrub_stamp":"2024-05-08T18:09:05.971607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:00:19.071636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"29'8","reported_seq":71,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:49.357517+0000","last_change":"2024-05-08T18:08:49.357546+0000","last_active":"2024-05-08T18:08:49.357517+0000","last_peered":"2024-05-08T18:08:49.357517+0000","last_clean":"2024-05-08T18:08:49.357517+0000","last_became_active":"2024-05-08T18:04:45.811648+0000","last_became_peered":"2024-05-08T18:04:45.811648+0000","last_unstale":"2024-05-08T18:08:49.357517+0000","last_undegraded":"2024-05-08T18:08:49.357517+0000","last_fullsized":"2024-05-08T18:08:49.357517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:49.357428+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:49.357428+0000","last_clean_scrub_stamp":"2024-05-08T18:08:49.357428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:51:32.994094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":31,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"29'6","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:38.419521+0000","last_change":"2024-05-08T18:08:38.419521+0000","last_active":"2024-05-08T18:08:38.419521+0000","last_peered":"2024-05-08T18:08:38.419521+0000","last_clean":"2024-05-08T18:08:38.419521+0000","last_became_active":"2024-05-08T18:04:43.835712+0000","last_became_peered":"2024-05-08T18:04:43.835712+0000","last_unstale":"2024-05-08T18:08:38.419521+0000","last_undegraded":"2024-05-08T18:08:38.419521+0000","last_fullsized":"2024-05-08T18:08:38.419521+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:38.419396+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:38.419396+0000","last_clean_scrub_stamp":"2024-05-08T18:08:38.419396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:54:01.440665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":22,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"29'10","reported_seq":64,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:47.497538+0000","last_change":"2024-05-08T18:08:47.497589+0000","last_active":"2024-05-08T18:08:47.497538+0000","last_peered":"2024-05-08T18:08:47.497538+0000","last_clean":"2024-05-08T18:08:47.497538+0000","last_became_active":"2024-05-08T18:04:45.811921+0000","last_became_peered":"2024-05-08T18:04:45.811921+0000","last_unstale":"2024-05-08T18:08:47.497538+0000","last_undegraded":"2024-05-08T18:08:47.497538+0000","last_fullsized":"2024-05-08T18:08:47.497538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:47.497436+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:47.497436+0000","last_clean_scrub_stamp":"2024-05-08T18:08:47.497436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:06:35.185255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":349,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2281,"ondisk_log_size":2281,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1600,"num_read_kb":1397,"num_write":597,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3000,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1527,"ondisk_log_size":1527,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":483,"ondisk_log_size":483,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":145,"num_read_kb":87,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":256,"acting":256,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705724,"num_pgs":179,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":438272,"data_stored":135643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705725,"num_pgs":153,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":970752,"data_stored":724153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705725,"num_pgs":157,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40464,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331376,"statfs":{"total":96636764160,"available":96595329024,"internally_reserved":0,"allocated":991232,"data_stored":724641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705725,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":409600,"data_stored":135609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T18:09:26.967 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:26.968 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-05-08T18:04:46.796744+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=46, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:26.968 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:26.969 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:26.969 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:26.969 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:26.969 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-05-08T18:04:52.818692+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=4, tm_sec=52, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=18, tm_min=8, tm_sec=2, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T18:09:26.970 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T18:09:46.971 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T18:09:47.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.087+0000 7f3c533dd640 1 Processor -- start 2024-05-08T18:09:47.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.088+0000 7f3c533dd640 1 -- start start 2024-05-08T18:09:47.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.088+0000 7f3c533dd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3c4c129ea0 0x7f3c4c12a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:09:47.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.088+0000 7f3c533dd640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c4c1144f0 con 0x7f3c4c12a7c0 2024-05-08T18:09:47.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.088+0000 7f3c533dd640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7f3c4c129ca0 con 0x7f3c4c129ea0 2024-05-08T18:09:47.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.088+0000 7f3c51179640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3c4c129ea0 0x7f3c4c12a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:09:47.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.088+0000 7f3c51179640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f3c4c129ea0 0x7f3c4c12a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33502/0 (socket says 172.21.15.46:33502) 2024-05-08T18:09:47.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.088+0000 7f3c51179640 1 -- 172.21.15.46:0/519777709 learned_addr learned my addr 172.21.15.46:0/519777709 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:09:47.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.088+0000 7f3c50978640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3c4c12a7c0 0x7f3c4c12d110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60428/0 (socket says 172.21.15.46:60428) 2024-05-08T18:09:47.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.089+0000 7f3c51179640 1 -- 172.21.15.46:0/519777709 >> v1:172.21.15.46:6789/0 conn(0x7f3c4c12a7c0 legacy=0x7f3c4c12d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:09:47.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.089+0000 7f3c51179640 1 -- 172.21.15.46:0/519777709 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c09acc0 con 0x7f3c4c129ea0 2024-05-08T18:09:47.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.089+0000 7f3c51179640 1 --2- 172.21.15.46:0/519777709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c4c129ea0 0x7f3c4c12a280 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3c480097f0 tx=0x7f3c4802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8321886c13ffb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:09:47.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.090+0000 7f3c43fff640 1 -- 172.21.15.46:0/519777709 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c48035020 con 0x7f3c4c129ea0 2024-05-08T18:09:47.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.090+0000 7f3c43fff640 1 -- 172.21.15.46:0/519777709 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c480317a0 con 0x7f3c4c129ea0 2024-05-08T18:09:47.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.090+0000 7f3c43fff640 1 -- 172.21.15.46:0/519777709 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c48031aa0 con 0x7f3c4c129ea0 2024-05-08T18:09:47.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.091+0000 7f3c533dd640 1 -- 172.21.15.46:0/519777709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c4c129ea0 msgr2=0x7f3c4c12a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:09:47.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.091+0000 7f3c533dd640 1 --2- 172.21.15.46:0/519777709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c4c129ea0 0x7f3c4c12a280 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3c480097f0 tx=0x7f3c4802b910 comp rx=0 tx=0).stop 2024-05-08T18:09:47.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.091+0000 7f3c533dd640 1 -- 172.21.15.46:0/519777709 shutdown_connections 2024-05-08T18:09:47.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.091+0000 7f3c533dd640 1 --2- 172.21.15.46:0/519777709 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c4c129ea0 0x7f3c4c12a280 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:09:47.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.091+0000 7f3c533dd640 1 -- 172.21.15.46:0/519777709 >> 172.21.15.46:0/519777709 conn(0x7f3c4c09d7a0 msgr2=0x7f3c4c09dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:09:47.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.091+0000 7f3c533dd640 1 -- 172.21.15.46:0/519777709 shutdown_connections 2024-05-08T18:09:47.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.091+0000 7f3c533dd640 1 -- 172.21.15.46:0/519777709 wait complete. 2024-05-08T18:09:47.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.092+0000 7f3c533dd640 1 Processor -- start 2024-05-08T18:09:47.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.092+0000 7f3c533dd640 1 -- start start 2024-05-08T18:09:47.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.092+0000 7f3c533dd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c4c129ea0 0x7f3c4c115060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:09:47.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.092+0000 7f3c533dd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4c113d90 con 0x7f3c4c129ea0 2024-05-08T18:09:47.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.092+0000 7f3c51179640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c4c129ea0 0x7f3c4c115060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:09:47.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.092+0000 7f3c51179640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c4c129ea0 0x7f3c4c115060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33506/0 (socket says 172.21.15.46:33506) 2024-05-08T18:09:47.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.092+0000 7f3c51179640 1 -- 172.21.15.46:0/1027127102 learned_addr learned my addr 172.21.15.46:0/1027127102 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:09:47.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.093+0000 7f3c51179640 1 -- 172.21.15.46:0/1027127102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4c116190 con 0x7f3c4c129ea0 2024-05-08T18:09:47.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.093+0000 7f3c51179640 1 --2- 172.21.15.46:0/1027127102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c4c129ea0 0x7f3c4c115060 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f3c480090b0 tx=0x7f3c480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:09:47.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.093+0000 7f3c41ffb640 1 -- 172.21.15.46:0/1027127102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c4803acb0 con 0x7f3c4c129ea0 2024-05-08T18:09:47.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.093+0000 7f3c41ffb640 1 -- 172.21.15.46:0/1027127102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c4803ae50 con 0x7f3c4c129ea0 2024-05-08T18:09:47.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.093+0000 7f3c533dd640 1 -- 172.21.15.46:0/1027127102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c4c1163d0 con 0x7f3c4c129ea0 2024-05-08T18:09:47.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.093+0000 7f3c41ffb640 1 -- 172.21.15.46:0/1027127102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c48044560 con 0x7f3c4c129ea0 2024-05-08T18:09:47.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.093+0000 7f3c533dd640 1 -- 172.21.15.46:0/1027127102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c4c12cb40 con 0x7f3c4c129ea0 2024-05-08T18:09:47.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.094+0000 7f3c41ffb640 1 -- 172.21.15.46:0/1027127102 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7f3c4804b060 con 0x7f3c4c129ea0 2024-05-08T18:09:47.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.094+0000 7f3c533dd640 1 -- 172.21.15.46:0/1027127102 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c4c09a370 con 0x7f3c4c129ea0 2024-05-08T18:09:47.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.095+0000 7f3c41ffb640 1 --2- 172.21.15.46:0/1027127102 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3c3003d2b0 0x7f3c3003f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:09:47.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.095+0000 7f3c41ffb640 1 -- 172.21.15.46:0/1027127102 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7f3c48073f40 con 0x7f3c4c129ea0 2024-05-08T18:09:47.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.095+0000 7f3c50978640 1 --2- 172.21.15.46:0/1027127102 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3c3003d2b0 0x7f3c3003f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:09:47.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.096+0000 7f3c50978640 1 --2- 172.21.15.46:0/1027127102 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3c3003d2b0 0x7f3c3003f770 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0015e0 tx=0x7f3c3c007a40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:09:47.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.099+0000 7f3c41ffb640 1 -- 172.21.15.46:0/1027127102 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7f3c4c09a370 con 0x7f3c4c129ea0 2024-05-08T18:09:47.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.288+0000 7f3c533dd640 1 -- 172.21.15.46:0/1027127102 --> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3c4c0640a0 con 0x7f3c3003d2b0 2024-05-08T18:09:47.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.304+0000 7f3c41ffb640 1 -- 172.21.15.46:0/1027127102 <== mgr.4098 v2:172.21.15.50:6824/2602185226 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+666264 (secure 0 0 0) 0x7f3c4c0640a0 con 0x7f3c3003d2b0 2024-05-08T18:09:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-08T18:09:47.315 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-08T18:09:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.309+0000 7f3c533dd640 1 -- 172.21.15.46:0/1027127102 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3c3003d2b0 msgr2=0x7f3c3003f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:09:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.309+0000 7f3c533dd640 1 --2- 172.21.15.46:0/1027127102 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3c3003d2b0 0x7f3c3003f770 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0015e0 tx=0x7f3c3c007a40 comp rx=0 tx=0).stop 2024-05-08T18:09:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.309+0000 7f3c533dd640 1 -- 172.21.15.46:0/1027127102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c4c129ea0 msgr2=0x7f3c4c115060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:09:47.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.309+0000 7f3c533dd640 1 --2- 172.21.15.46:0/1027127102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c4c129ea0 0x7f3c4c115060 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f3c480090b0 tx=0x7f3c480046c0 comp rx=0 tx=0).stop 2024-05-08T18:09:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.309+0000 7f3c533dd640 1 -- 172.21.15.46:0/1027127102 shutdown_connections 2024-05-08T18:09:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.309+0000 7f3c533dd640 1 --2- 172.21.15.46:0/1027127102 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7f3c3003d2b0 0x7f3c3003f770 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:09:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.309+0000 7f3c533dd640 1 --2- 172.21.15.46:0/1027127102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c4c129ea0 0x7f3c4c115060 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:09:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.309+0000 7f3c533dd640 1 -- 172.21.15.46:0/1027127102 >> 172.21.15.46:0/1027127102 conn(0x7f3c4c09d7a0 msgr2=0x7f3c4c128900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:09:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.309+0000 7f3c533dd640 1 -- 172.21.15.46:0/1027127102 shutdown_connections 2024-05-08T18:09:47.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.309+0000 7f3c533dd640 1 -- 172.21.15.46:0/1027127102 wait complete. 2024-05-08T18:09:47.319 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-05-08T18:09:45.762788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605383,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":5050,"num_read_kb":4636,"num_write":2119,"num_write_kb":60585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3349,"num_omap_keys":16,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4743,"ondisk_log_size":4743,"up":658,"acting":658,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":658,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":127448,"kb_used_data":2744,"kb_used_omap":6,"kb_used_meta":124665,"kb_avail":377359912,"statfs":{"total":386547056640,"available":386416549888,"internally_reserved":0,"allocated":2809856,"data_stored":1720046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":127657750},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003480"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:46.010735+0000","last_change":"2024-05-08T18:08:46.010735+0000","last_active":"2024-05-08T18:08:46.010735+0000","last_peered":"2024-05-08T18:08:46.010735+0000","last_clean":"2024-05-08T18:08:46.010735+0000","last_became_active":"2024-05-08T18:04:43.836496+0000","last_became_peered":"2024-05-08T18:04:43.836496+0000","last_unstale":"2024-05-08T18:08:46.010735+0000","last_undegraded":"2024-05-08T18:08:46.010735+0000","last_fullsized":"2024-05-08T18:08:46.010735+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:46.010599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:46.010599+0000","last_clean_scrub_stamp":"2024-05-08T18:08:46.010599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:07.142721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"29'6","reported_seq":60,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:49.481085+0000","last_change":"2024-05-08T18:08:49.481113+0000","last_active":"2024-05-08T18:08:49.481085+0000","last_peered":"2024-05-08T18:08:49.481085+0000","last_clean":"2024-05-08T18:08:49.481085+0000","last_became_active":"2024-05-08T18:04:45.811368+0000","last_became_peered":"2024-05-08T18:04:45.811368+0000","last_unstale":"2024-05-08T18:08:49.481085+0000","last_undegraded":"2024-05-08T18:08:49.481085+0000","last_fullsized":"2024-05-08T18:08:49.481085+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:49.480997+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:49.480997+0000","last_clean_scrub_stamp":"2024-05-08T18:08:49.480997+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:21:18.714592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"29'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:50.347475+0000","last_change":"2024-05-08T18:08:50.347475+0000","last_active":"2024-05-08T18:08:50.347475+0000","last_peered":"2024-05-08T18:08:50.347475+0000","last_clean":"2024-05-08T18:08:50.347475+0000","last_became_active":"2024-05-08T18:04:43.834612+0000","last_became_peered":"2024-05-08T18:04:43.834612+0000","last_unstale":"2024-05-08T18:08:50.347475+0000","last_undegraded":"2024-05-08T18:08:50.347475+0000","last_fullsized":"2024-05-08T18:08:50.347475+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:50.347354+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:50.347354+0000","last_clean_scrub_stamp":"2024-05-08T18:08:50.347354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:07:56.846928+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"29'2","reported_seq":50,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:40.037447+0000","last_change":"2024-05-08T18:08:40.037474+0000","last_active":"2024-05-08T18:08:40.037447+0000","last_peered":"2024-05-08T18:08:40.037447+0000","last_clean":"2024-05-08T18:08:40.037447+0000","last_became_active":"2024-05-08T18:04:45.827965+0000","last_became_peered":"2024-05-08T18:04:45.827965+0000","last_unstale":"2024-05-08T18:08:40.037447+0000","last_undegraded":"2024-05-08T18:08:40.037447+0000","last_fullsized":"2024-05-08T18:08:40.037447+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:40.037352+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:40.037352+0000","last_clean_scrub_stamp":"2024-05-08T18:08:40.037352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:22:58.859282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:00.008391+0000","last_change":"2024-05-08T18:09:00.008419+0000","last_active":"2024-05-08T18:09:00.008391+0000","last_peered":"2024-05-08T18:09:00.008391+0000","last_clean":"2024-05-08T18:09:00.008391+0000","last_became_active":"2024-05-08T18:04:43.837456+0000","last_became_peered":"2024-05-08T18:04:43.837456+0000","last_unstale":"2024-05-08T18:09:00.008391+0000","last_undegraded":"2024-05-08T18:09:00.008391+0000","last_fullsized":"2024-05-08T18:09:00.008391+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:00.008279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:00.008279+0000","last_clean_scrub_stamp":"2024-05-08T18:09:00.008279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:38.600725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"29'12","reported_seq":66,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:50.085604+0000","last_change":"2024-05-08T18:08:50.085643+0000","last_active":"2024-05-08T18:08:50.085604+0000","last_peered":"2024-05-08T18:08:50.085604+0000","last_clean":"2024-05-08T18:08:50.085604+0000","last_became_active":"2024-05-08T18:04:45.825022+0000","last_became_peered":"2024-05-08T18:04:45.825022+0000","last_unstale":"2024-05-08T18:08:50.085604+0000","last_undegraded":"2024-05-08T18:08:50.085604+0000","last_fullsized":"2024-05-08T18:08:50.085604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:50.085506+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:50.085506+0000","last_clean_scrub_stamp":"2024-05-08T18:08:50.085506+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:14:46.319120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":33,"num_read_kb":29,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:40.176922+0000","last_change":"2024-05-08T18:08:40.176922+0000","last_active":"2024-05-08T18:08:40.176922+0000","last_peered":"2024-05-08T18:08:40.176922+0000","last_clean":"2024-05-08T18:08:40.176922+0000","last_became_active":"2024-05-08T18:04:43.842077+0000","last_became_peered":"2024-05-08T18:04:43.842077+0000","last_unstale":"2024-05-08T18:08:40.176922+0000","last_undegraded":"2024-05-08T18:08:40.176922+0000","last_fullsized":"2024-05-08T18:08:40.176922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:40.176797+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:40.176797+0000","last_clean_scrub_stamp":"2024-05-08T18:08:40.176797+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:40:14.062529+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"29'16","reported_seq":79,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:51.114482+0000","last_change":"2024-05-08T18:08:51.114508+0000","last_active":"2024-05-08T18:08:51.114482+0000","last_peered":"2024-05-08T18:08:51.114482+0000","last_clean":"2024-05-08T18:08:51.114482+0000","last_became_active":"2024-05-08T18:04:45.824581+0000","last_became_peered":"2024-05-08T18:04:45.824581+0000","last_unstale":"2024-05-08T18:08:51.114482+0000","last_undegraded":"2024-05-08T18:08:51.114482+0000","last_fullsized":"2024-05-08T18:08:51.114482+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'16","last_scrub_stamp":"2024-05-08T18:08:51.114374+0000","last_deep_scrub":"29'16","last_deep_scrub_stamp":"2024-05-08T18:08:51.114374+0000","last_clean_scrub_stamp":"2024-05-08T18:08:51.114374+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:10:34.341808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:22.029587+0000","last_change":"2024-05-08T18:09:22.029587+0000","last_active":"2024-05-08T18:09:22.029587+0000","last_peered":"2024-05-08T18:09:22.029587+0000","last_clean":"2024-05-08T18:09:22.029587+0000","last_became_active":"2024-05-08T18:04:43.832722+0000","last_became_peered":"2024-05-08T18:04:43.832722+0000","last_unstale":"2024-05-08T18:09:22.029587+0000","last_undegraded":"2024-05-08T18:09:22.029587+0000","last_fullsized":"2024-05-08T18:09:22.029587+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:22.029460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:22.029460+0000","last_clean_scrub_stamp":"2024-05-08T18:09:22.029460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:45:23.654040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"29'8","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:35.526800+0000","last_change":"2024-05-08T18:08:35.526800+0000","last_active":"2024-05-08T18:08:35.526800+0000","last_peered":"2024-05-08T18:08:35.526800+0000","last_clean":"2024-05-08T18:08:35.526800+0000","last_became_active":"2024-05-08T18:04:45.808383+0000","last_became_peered":"2024-05-08T18:04:45.808383+0000","last_unstale":"2024-05-08T18:08:35.526800+0000","last_undegraded":"2024-05-08T18:08:35.526800+0000","last_fullsized":"2024-05-08T18:08:35.526800+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:35.526676+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:35.526676+0000","last_clean_scrub_stamp":"2024-05-08T18:08:35.526676+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:24.925676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:42.091220+0000","last_change":"2024-05-08T18:08:42.091220+0000","last_active":"2024-05-08T18:08:42.091220+0000","last_peered":"2024-05-08T18:08:42.091220+0000","last_clean":"2024-05-08T18:08:42.091220+0000","last_became_active":"2024-05-08T18:04:43.840851+0000","last_became_peered":"2024-05-08T18:04:43.840851+0000","last_unstale":"2024-05-08T18:08:42.091220+0000","last_undegraded":"2024-05-08T18:08:42.091220+0000","last_fullsized":"2024-05-08T18:08:42.091220+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:42.091091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:42.091091+0000","last_clean_scrub_stamp":"2024-05-08T18:08:42.091091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:24.274648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"29'4","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:39.004566+0000","last_change":"2024-05-08T18:08:39.004566+0000","last_active":"2024-05-08T18:08:39.004566+0000","last_peered":"2024-05-08T18:08:39.004566+0000","last_clean":"2024-05-08T18:08:39.004566+0000","last_became_active":"2024-05-08T18:04:45.828334+0000","last_became_peered":"2024-05-08T18:04:45.828334+0000","last_unstale":"2024-05-08T18:08:39.004566+0000","last_undegraded":"2024-05-08T18:08:39.004566+0000","last_fullsized":"2024-05-08T18:08:39.004566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:39.004442+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:39.004442+0000","last_clean_scrub_stamp":"2024-05-08T18:08:39.004442+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:31:47.634365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"29'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:13.988375+0000","last_change":"2024-05-08T18:09:13.988375+0000","last_active":"2024-05-08T18:09:13.988375+0000","last_peered":"2024-05-08T18:09:13.988375+0000","last_clean":"2024-05-08T18:09:13.988375+0000","last_became_active":"2024-05-08T18:04:43.833437+0000","last_became_peered":"2024-05-08T18:04:43.833437+0000","last_unstale":"2024-05-08T18:09:13.988375+0000","last_undegraded":"2024-05-08T18:09:13.988375+0000","last_fullsized":"2024-05-08T18:09:13.988375+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:09:13.988182+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:09:13.988182+0000","last_clean_scrub_stamp":"2024-05-08T18:09:13.988182+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:41:56.815735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:29.307581+0000","last_change":"2024-05-08T18:08:29.307581+0000","last_active":"2024-05-08T18:08:29.307581+0000","last_peered":"2024-05-08T18:08:29.307581+0000","last_clean":"2024-05-08T18:08:29.307581+0000","last_became_active":"2024-05-08T18:04:45.811884+0000","last_became_peered":"2024-05-08T18:04:45.811884+0000","last_unstale":"2024-05-08T18:08:29.307581+0000","last_undegraded":"2024-05-08T18:08:29.307581+0000","last_fullsized":"2024-05-08T18:08:29.307581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:29.307470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:29.307470+0000","last_clean_scrub_stamp":"2024-05-08T18:08:29.307470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:16:25.939226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:53.384464+0000","last_change":"2024-05-08T18:08:53.384464+0000","last_active":"2024-05-08T18:08:53.384464+0000","last_peered":"2024-05-08T18:08:53.384464+0000","last_clean":"2024-05-08T18:08:53.384464+0000","last_became_active":"2024-05-08T18:04:43.820312+0000","last_became_peered":"2024-05-08T18:04:43.820312+0000","last_unstale":"2024-05-08T18:08:53.384464+0000","last_undegraded":"2024-05-08T18:08:53.384464+0000","last_fullsized":"2024-05-08T18:08:53.384464+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:53.384283+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:53.384283+0000","last_clean_scrub_stamp":"2024-05-08T18:08:53.384283+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:05:28.766196+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"29'8","reported_seq":72,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:50.932035+0000","last_change":"2024-05-08T18:08:50.932035+0000","last_active":"2024-05-08T18:08:50.932035+0000","last_peered":"2024-05-08T18:08:50.932035+0000","last_clean":"2024-05-08T18:08:50.932035+0000","last_became_active":"2024-05-08T18:04:45.822620+0000","last_became_peered":"2024-05-08T18:04:45.822620+0000","last_unstale":"2024-05-08T18:08:50.932035+0000","last_undegraded":"2024-05-08T18:08:50.932035+0000","last_fullsized":"2024-05-08T18:08:50.932035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:50.931921+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:50.931921+0000","last_clean_scrub_stamp":"2024-05-08T18:08:50.931921+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:50:39.324779+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"29'4","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:46.060430+0000","last_change":"2024-05-08T18:08:46.060430+0000","last_active":"2024-05-08T18:08:46.060430+0000","last_peered":"2024-05-08T18:08:46.060430+0000","last_clean":"2024-05-08T18:08:46.060430+0000","last_became_active":"2024-05-08T18:04:45.826493+0000","last_became_peered":"2024-05-08T18:04:45.826493+0000","last_unstale":"2024-05-08T18:08:46.060430+0000","last_undegraded":"2024-05-08T18:08:46.060430+0000","last_fullsized":"2024-05-08T18:08:46.060430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:46.060304+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:46.060304+0000","last_clean_scrub_stamp":"2024-05-08T18:08:46.060304+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:28:42.176968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:46.481360+0000","last_change":"2024-05-08T18:08:46.481360+0000","last_active":"2024-05-08T18:08:46.481360+0000","last_peered":"2024-05-08T18:08:46.481360+0000","last_clean":"2024-05-08T18:08:46.481360+0000","last_became_active":"2024-05-08T18:04:43.838332+0000","last_became_peered":"2024-05-08T18:04:43.838332+0000","last_unstale":"2024-05-08T18:08:46.481360+0000","last_undegraded":"2024-05-08T18:08:46.481360+0000","last_fullsized":"2024-05-08T18:08:46.481360+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:46.481227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:46.481227+0000","last_clean_scrub_stamp":"2024-05-08T18:08:46.481227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:44:09.862039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"29'4","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:36.520179+0000","last_change":"2024-05-08T18:08:36.520179+0000","last_active":"2024-05-08T18:08:36.520179+0000","last_peered":"2024-05-08T18:08:36.520179+0000","last_clean":"2024-05-08T18:08:36.520179+0000","last_became_active":"2024-05-08T18:04:45.807438+0000","last_became_peered":"2024-05-08T18:04:45.807438+0000","last_unstale":"2024-05-08T18:08:36.520179+0000","last_undegraded":"2024-05-08T18:08:36.520179+0000","last_fullsized":"2024-05-08T18:08:36.520179+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:36.520045+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:36.520045+0000","last_clean_scrub_stamp":"2024-05-08T18:08:36.520045+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:18.703602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:30.298757+0000","last_change":"2024-05-08T18:08:30.298757+0000","last_active":"2024-05-08T18:08:30.298757+0000","last_peered":"2024-05-08T18:08:30.298757+0000","last_clean":"2024-05-08T18:08:30.298757+0000","last_became_active":"2024-05-08T18:04:43.842741+0000","last_became_peered":"2024-05-08T18:04:43.842741+0000","last_unstale":"2024-05-08T18:08:30.298757+0000","last_undegraded":"2024-05-08T18:08:30.298757+0000","last_fullsized":"2024-05-08T18:08:30.298757+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:30.298603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:30.298603+0000","last_clean_scrub_stamp":"2024-05-08T18:08:30.298603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:30:25.561243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"29'12","reported_seq":68,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:43.533178+0000","last_change":"2024-05-08T18:08:43.533178+0000","last_active":"2024-05-08T18:08:43.533178+0000","last_peered":"2024-05-08T18:08:43.533178+0000","last_clean":"2024-05-08T18:08:43.533178+0000","last_became_active":"2024-05-08T18:04:45.810827+0000","last_became_peered":"2024-05-08T18:04:45.810827+0000","last_unstale":"2024-05-08T18:08:43.533178+0000","last_undegraded":"2024-05-08T18:08:43.533178+0000","last_fullsized":"2024-05-08T18:08:43.533178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:43.533034+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:43.533034+0000","last_clean_scrub_stamp":"2024-05-08T18:08:43.533034+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:04.129239+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"29'4","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:50.465291+0000","last_change":"2024-05-08T18:08:50.465291+0000","last_active":"2024-05-08T18:08:50.465291+0000","last_peered":"2024-05-08T18:08:50.465291+0000","last_clean":"2024-05-08T18:08:50.465291+0000","last_became_active":"2024-05-08T18:04:43.837706+0000","last_became_peered":"2024-05-08T18:04:43.837706+0000","last_unstale":"2024-05-08T18:08:50.465291+0000","last_undegraded":"2024-05-08T18:08:50.465291+0000","last_fullsized":"2024-05-08T18:08:50.465291+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:50.465171+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:50.465171+0000","last_clean_scrub_stamp":"2024-05-08T18:08:50.465171+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:00:00.657178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"29'8","reported_seq":69,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:32.455190+0000","last_change":"2024-05-08T18:08:32.455190+0000","last_active":"2024-05-08T18:08:32.455190+0000","last_peered":"2024-05-08T18:08:32.455190+0000","last_clean":"2024-05-08T18:08:32.455190+0000","last_became_active":"2024-05-08T18:04:45.822000+0000","last_became_peered":"2024-05-08T18:04:45.822000+0000","last_unstale":"2024-05-08T18:08:32.455190+0000","last_undegraded":"2024-05-08T18:08:32.455190+0000","last_fullsized":"2024-05-08T18:08:32.455190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:32.455035+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:32.455035+0000","last_clean_scrub_stamp":"2024-05-08T18:08:32.455035+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:21.854035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:40.545772+0000","last_change":"2024-05-08T18:08:40.545772+0000","last_active":"2024-05-08T18:08:40.545772+0000","last_peered":"2024-05-08T18:08:40.545772+0000","last_clean":"2024-05-08T18:08:40.545772+0000","last_became_active":"2024-05-08T18:04:43.841063+0000","last_became_peered":"2024-05-08T18:04:43.841063+0000","last_unstale":"2024-05-08T18:08:40.545772+0000","last_undegraded":"2024-05-08T18:08:40.545772+0000","last_fullsized":"2024-05-08T18:08:40.545772+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:40.545644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:40.545644+0000","last_clean_scrub_stamp":"2024-05-08T18:08:40.545644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:55:34.045722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"29'4","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:26.277484+0000","last_change":"2024-05-08T18:08:26.277484+0000","last_active":"2024-05-08T18:08:26.277484+0000","last_peered":"2024-05-08T18:08:26.277484+0000","last_clean":"2024-05-08T18:08:26.277484+0000","last_became_active":"2024-05-08T18:04:45.823611+0000","last_became_peered":"2024-05-08T18:04:45.823611+0000","last_unstale":"2024-05-08T18:08:26.277484+0000","last_undegraded":"2024-05-08T18:08:26.277484+0000","last_fullsized":"2024-05-08T18:08:26.277484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:26.277327+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:26.277327+0000","last_clean_scrub_stamp":"2024-05-08T18:08:26.277327+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:11:13.871401+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:46.372029+0000","last_change":"2024-05-08T18:08:46.372029+0000","last_active":"2024-05-08T18:08:46.372029+0000","last_peered":"2024-05-08T18:08:46.372029+0000","last_clean":"2024-05-08T18:08:46.372029+0000","last_became_active":"2024-05-08T18:04:43.834456+0000","last_became_peered":"2024-05-08T18:04:43.834456+0000","last_unstale":"2024-05-08T18:08:46.372029+0000","last_undegraded":"2024-05-08T18:08:46.372029+0000","last_fullsized":"2024-05-08T18:08:46.372029+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:46.371925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:46.371925+0000","last_clean_scrub_stamp":"2024-05-08T18:08:46.371925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:23.458745+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"29'6","reported_seq":64,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:37.506219+0000","last_change":"2024-05-08T18:08:37.506219+0000","last_active":"2024-05-08T18:08:37.506219+0000","last_peered":"2024-05-08T18:08:37.506219+0000","last_clean":"2024-05-08T18:08:37.506219+0000","last_became_active":"2024-05-08T18:04:45.808416+0000","last_became_peered":"2024-05-08T18:04:45.808416+0000","last_unstale":"2024-05-08T18:08:37.506219+0000","last_undegraded":"2024-05-08T18:08:37.506219+0000","last_fullsized":"2024-05-08T18:08:37.506219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:37.506056+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:37.506056+0000","last_clean_scrub_stamp":"2024-05-08T18:08:37.506056+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:14.952253+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'4","reported_seq":60,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:10.945688+0000","last_change":"2024-05-08T18:09:10.945688+0000","last_active":"2024-05-08T18:09:10.945688+0000","last_peered":"2024-05-08T18:09:10.945688+0000","last_clean":"2024-05-08T18:09:10.945688+0000","last_became_active":"2024-05-08T18:04:43.844702+0000","last_became_peered":"2024-05-08T18:04:43.844702+0000","last_unstale":"2024-05-08T18:09:10.945688+0000","last_undegraded":"2024-05-08T18:09:10.945688+0000","last_fullsized":"2024-05-08T18:09:10.945688+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-08T18:09:10.945566+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-08T18:09:10.945566+0000","last_clean_scrub_stamp":"2024-05-08T18:09:10.945566+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:16:52.494468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"29'36","reported_seq":110,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:02.553531+0000","last_change":"2024-05-08T18:09:02.553531+0000","last_active":"2024-05-08T18:09:02.553531+0000","last_peered":"2024-05-08T18:09:02.553531+0000","last_clean":"2024-05-08T18:09:02.553531+0000","last_became_active":"2024-05-08T18:04:53.849475+0000","last_became_peered":"2024-05-08T18:04:53.849475+0000","last_unstale":"2024-05-08T18:09:02.553531+0000","last_undegraded":"2024-05-08T18:09:02.553531+0000","last_fullsized":"2024-05-08T18:09:02.553531+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:09:02.553411+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:09:02.553411+0000","last_clean_scrub_stamp":"2024-05-08T18:09:02.553411+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:25:37.317356+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"29'12","reported_seq":76,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:52.160843+0000","last_change":"2024-05-08T18:08:52.160843+0000","last_active":"2024-05-08T18:08:52.160843+0000","last_peered":"2024-05-08T18:08:52.160843+0000","last_clean":"2024-05-08T18:08:52.160843+0000","last_became_active":"2024-05-08T18:04:45.824005+0000","last_became_peered":"2024-05-08T18:04:45.824005+0000","last_unstale":"2024-05-08T18:08:52.160843+0000","last_undegraded":"2024-05-08T18:08:52.160843+0000","last_fullsized":"2024-05-08T18:08:52.160843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:52.160704+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:52.160704+0000","last_clean_scrub_stamp":"2024-05-08T18:08:52.160704+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:28:13.667834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:11.930576+0000","last_change":"2024-05-08T18:09:11.930576+0000","last_active":"2024-05-08T18:09:11.930576+0000","last_peered":"2024-05-08T18:09:11.930576+0000","last_clean":"2024-05-08T18:09:11.930576+0000","last_became_active":"2024-05-08T18:04:43.833822+0000","last_became_peered":"2024-05-08T18:04:43.833822+0000","last_unstale":"2024-05-08T18:09:11.930576+0000","last_undegraded":"2024-05-08T18:09:11.930576+0000","last_fullsized":"2024-05-08T18:09:11.930576+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:09:11.930438+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:09:11.930438+0000","last_clean_scrub_stamp":"2024-05-08T18:09:11.930438+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:47:50.587916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.924538+0000","last_change":"2024-05-08T18:08:09.924538+0000","last_active":"2024-05-08T18:08:09.924538+0000","last_peered":"2024-05-08T18:08:09.924538+0000","last_clean":"2024-05-08T18:08:09.924538+0000","last_became_active":"2024-05-08T18:04:47.819682+0000","last_became_peered":"2024-05-08T18:04:47.819682+0000","last_unstale":"2024-05-08T18:08:09.924538+0000","last_undegraded":"2024-05-08T18:08:09.924538+0000","last_fullsized":"2024-05-08T18:08:09.924538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:09.924406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:09.924406+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.924406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:58.088967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"29'39","reported_seq":95,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:31.864298+0000","last_change":"2024-05-08T18:08:31.864298+0000","last_active":"2024-05-08T18:08:31.864298+0000","last_peered":"2024-05-08T18:08:31.864298+0000","last_clean":"2024-05-08T18:08:31.864298+0000","last_became_active":"2024-05-08T18:04:49.832148+0000","last_became_peered":"2024-05-08T18:04:49.832148+0000","last_unstale":"2024-05-08T18:08:31.864298+0000","last_undegraded":"2024-05-08T18:08:31.864298+0000","last_fullsized":"2024-05-08T18:08:31.864298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'39","last_scrub_stamp":"2024-05-08T18:08:31.864172+0000","last_deep_scrub":"29'39","last_deep_scrub_stamp":"2024-05-08T18:08:31.864172+0000","last_clean_scrub_stamp":"2024-05-08T18:08:31.864172+0000","objects_scrubbed":4,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:47:21.937257+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":40,"num_read_kb":42,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.510876+0000","last_change":"2024-05-08T18:08:21.510876+0000","last_active":"2024-05-08T18:08:21.510876+0000","last_peered":"2024-05-08T18:08:21.510876+0000","last_clean":"2024-05-08T18:08:21.510876+0000","last_became_active":"2024-05-08T18:04:51.844901+0000","last_became_peered":"2024-05-08T18:04:51.844901+0000","last_unstale":"2024-05-08T18:08:21.510876+0000","last_undegraded":"2024-05-08T18:08:21.510876+0000","last_fullsized":"2024-05-08T18:08:21.510876+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:21.510687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:21.510687+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.510687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:55:15.292759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"29'11","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:03.522138+0000","last_change":"2024-05-08T18:09:03.522138+0000","last_active":"2024-05-08T18:09:03.522138+0000","last_peered":"2024-05-08T18:09:03.522138+0000","last_clean":"2024-05-08T18:09:03.522138+0000","last_became_active":"2024-05-08T18:04:53.849579+0000","last_became_peered":"2024-05-08T18:04:53.849579+0000","last_unstale":"2024-05-08T18:09:03.522138+0000","last_undegraded":"2024-05-08T18:09:03.522138+0000","last_fullsized":"2024-05-08T18:09:03.522138+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'11","last_scrub_stamp":"2024-05-08T18:09:03.522007+0000","last_deep_scrub":"29'11","last_deep_scrub_stamp":"2024-05-08T18:09:03.522007+0000","last_clean_scrub_stamp":"2024-05-08T18:09:03.522007+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:05:01.403489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":18,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"29'6","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:44.084259+0000","last_change":"2024-05-08T18:08:44.084259+0000","last_active":"2024-05-08T18:08:44.084259+0000","last_peered":"2024-05-08T18:08:44.084259+0000","last_clean":"2024-05-08T18:08:44.084259+0000","last_became_active":"2024-05-08T18:04:45.825162+0000","last_became_peered":"2024-05-08T18:04:45.825162+0000","last_unstale":"2024-05-08T18:08:44.084259+0000","last_undegraded":"2024-05-08T18:08:44.084259+0000","last_fullsized":"2024-05-08T18:08:44.084259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:44.084137+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:44.084137+0000","last_clean_scrub_stamp":"2024-05-08T18:08:44.084137+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:00:15.074875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:44.010708+0000","last_change":"2024-05-08T18:08:44.010708+0000","last_active":"2024-05-08T18:08:44.010708+0000","last_peered":"2024-05-08T18:08:44.010708+0000","last_clean":"2024-05-08T18:08:44.010708+0000","last_became_active":"2024-05-08T18:04:43.827374+0000","last_became_peered":"2024-05-08T18:04:43.827374+0000","last_unstale":"2024-05-08T18:08:44.010708+0000","last_undegraded":"2024-05-08T18:08:44.010708+0000","last_fullsized":"2024-05-08T18:08:44.010708+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:44.010568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:44.010568+0000","last_clean_scrub_stamp":"2024-05-08T18:08:44.010568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:59:57.344162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:05.547885+0000","last_change":"2024-05-08T18:09:05.547916+0000","last_active":"2024-05-08T18:09:05.547885+0000","last_peered":"2024-05-08T18:09:05.547885+0000","last_clean":"2024-05-08T18:09:05.547885+0000","last_became_active":"2024-05-08T18:05:30.868191+0000","last_became_peered":"2024-05-08T18:05:30.868191+0000","last_unstale":"2024-05-08T18:09:05.547885+0000","last_undegraded":"2024-05-08T18:09:05.547885+0000","last_fullsized":"2024-05-08T18:09:05.547885+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:05.547803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:05.547803+0000","last_clean_scrub_stamp":"2024-05-08T18:09:05.547803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:54.348953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"29'9","reported_seq":47,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.529988+0000","last_change":"2024-05-08T18:08:10.529988+0000","last_active":"2024-05-08T18:08:10.529988+0000","last_peered":"2024-05-08T18:08:10.529988+0000","last_clean":"2024-05-08T18:08:10.529988+0000","last_became_active":"2024-05-08T18:04:49.825265+0000","last_became_peered":"2024-05-08T18:04:49.825265+0000","last_unstale":"2024-05-08T18:08:10.529988+0000","last_undegraded":"2024-05-08T18:08:10.529988+0000","last_fullsized":"2024-05-08T18:08:10.529988+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'9","last_scrub_stamp":"2024-05-08T18:08:10.529858+0000","last_deep_scrub":"29'9","last_deep_scrub_stamp":"2024-05-08T18:08:10.529858+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.529858+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:00.215888+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.491631+0000","last_change":"2024-05-08T18:08:13.491631+0000","last_active":"2024-05-08T18:08:13.491631+0000","last_peered":"2024-05-08T18:08:13.491631+0000","last_clean":"2024-05-08T18:08:13.491631+0000","last_became_active":"2024-05-08T18:04:51.833523+0000","last_became_peered":"2024-05-08T18:04:51.833523+0000","last_unstale":"2024-05-08T18:08:13.491631+0000","last_undegraded":"2024-05-08T18:08:13.491631+0000","last_fullsized":"2024-05-08T18:08:13.491631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:13.491458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:13.491458+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.491458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:07.228339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'2","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:33.477085+0000","last_change":"2024-05-08T18:08:33.477157+0000","last_active":"2024-05-08T18:08:33.477085+0000","last_peered":"2024-05-08T18:08:33.477085+0000","last_clean":"2024-05-08T18:08:33.477085+0000","last_became_active":"2024-05-08T18:04:53.840395+0000","last_became_peered":"2024-05-08T18:04:53.840395+0000","last_unstale":"2024-05-08T18:08:33.477085+0000","last_undegraded":"2024-05-08T18:08:33.477085+0000","last_fullsized":"2024-05-08T18:08:33.477085+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:33.476981+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:33.476981+0000","last_clean_scrub_stamp":"2024-05-08T18:08:33.476981+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:19:15.660538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":28,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:58.075763+0000","last_change":"2024-05-08T18:08:58.075787+0000","last_active":"2024-05-08T18:08:58.075763+0000","last_peered":"2024-05-08T18:08:58.075763+0000","last_clean":"2024-05-08T18:08:58.075763+0000","last_became_active":"2024-05-08T18:05:30.869162+0000","last_became_peered":"2024-05-08T18:05:30.869162+0000","last_unstale":"2024-05-08T18:08:58.075763+0000","last_undegraded":"2024-05-08T18:08:58.075763+0000","last_fullsized":"2024-05-08T18:08:58.075763+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-05-08T18:08:58.075662+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-05-08T18:08:58.075662+0000","last_clean_scrub_stamp":"2024-05-08T18:08:58.075662+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:36:28.054883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":45,"num_write":24,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:05.011315+0000","last_change":"2024-05-08T18:09:05.011315+0000","last_active":"2024-05-08T18:09:05.011315+0000","last_peered":"2024-05-08T18:09:05.011315+0000","last_clean":"2024-05-08T18:09:05.011315+0000","last_became_active":"2024-05-08T18:04:43.838666+0000","last_became_peered":"2024-05-08T18:04:43.838666+0000","last_unstale":"2024-05-08T18:09:05.011315+0000","last_undegraded":"2024-05-08T18:09:05.011315+0000","last_fullsized":"2024-05-08T18:09:05.011315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:05.011187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:05.011187+0000","last_clean_scrub_stamp":"2024-05-08T18:09:05.011187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:24:10.074163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:27.491684+0000","last_change":"2024-05-08T18:08:27.491684+0000","last_active":"2024-05-08T18:08:27.491684+0000","last_peered":"2024-05-08T18:08:27.491684+0000","last_clean":"2024-05-08T18:08:27.491684+0000","last_became_active":"2024-05-08T18:04:47.819169+0000","last_became_peered":"2024-05-08T18:04:47.819169+0000","last_unstale":"2024-05-08T18:08:27.491684+0000","last_undegraded":"2024-05-08T18:08:27.491684+0000","last_fullsized":"2024-05-08T18:08:27.491684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:27.491574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:27.491574+0000","last_clean_scrub_stamp":"2024-05-08T18:08:27.491574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:24.380337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"29'17","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.168281+0000","last_change":"2024-05-08T18:08:11.168281+0000","last_active":"2024-05-08T18:08:11.168281+0000","last_peered":"2024-05-08T18:08:11.168281+0000","last_clean":"2024-05-08T18:08:11.168281+0000","last_became_active":"2024-05-08T18:04:49.831500+0000","last_became_peered":"2024-05-08T18:04:49.831500+0000","last_unstale":"2024-05-08T18:08:11.168281+0000","last_undegraded":"2024-05-08T18:08:11.168281+0000","last_fullsized":"2024-05-08T18:08:11.168281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'17","last_scrub_stamp":"2024-05-08T18:08:11.168144+0000","last_deep_scrub":"29'17","last_deep_scrub_stamp":"2024-05-08T18:08:11.168144+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.168144+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:12:03.071434+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"29'286","reported_seq":395,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:41.051182+0000","last_change":"2024-05-08T18:08:41.051182+0000","last_active":"2024-05-08T18:08:41.051182+0000","last_peered":"2024-05-08T18:08:41.051182+0000","last_clean":"2024-05-08T18:08:41.051182+0000","last_became_active":"2024-05-08T18:04:51.849301+0000","last_became_peered":"2024-05-08T18:04:51.849301+0000","last_unstale":"2024-05-08T18:08:41.051182+0000","last_undegraded":"2024-05-08T18:08:41.051182+0000","last_fullsized":"2024-05-08T18:08:41.051182+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'286","last_scrub_stamp":"2024-05-08T18:08:41.051045+0000","last_deep_scrub":"29'286","last_deep_scrub_stamp":"2024-05-08T18:08:41.051045+0000","last_clean_scrub_stamp":"2024-05-08T18:08:41.051045+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:35:15.383617+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:53.985162+0000","last_change":"2024-05-08T18:08:53.985191+0000","last_active":"2024-05-08T18:08:53.985162+0000","last_peered":"2024-05-08T18:08:53.985162+0000","last_clean":"2024-05-08T18:08:53.985162+0000","last_became_active":"2024-05-08T18:04:53.840559+0000","last_became_peered":"2024-05-08T18:04:53.840559+0000","last_unstale":"2024-05-08T18:08:53.985162+0000","last_undegraded":"2024-05-08T18:08:53.985162+0000","last_fullsized":"2024-05-08T18:08:53.985162+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:53.985043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:53.985043+0000","last_clean_scrub_stamp":"2024-05-08T18:08:53.985043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:06:41.672862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"29'6","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:45.404510+0000","last_change":"2024-05-08T18:08:45.404510+0000","last_active":"2024-05-08T18:08:45.404510+0000","last_peered":"2024-05-08T18:08:45.404510+0000","last_clean":"2024-05-08T18:08:45.404510+0000","last_became_active":"2024-05-08T18:04:45.811570+0000","last_became_peered":"2024-05-08T18:04:45.811570+0000","last_unstale":"2024-05-08T18:08:45.404510+0000","last_undegraded":"2024-05-08T18:08:45.404510+0000","last_fullsized":"2024-05-08T18:08:45.404510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:45.404379+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:45.404379+0000","last_clean_scrub_stamp":"2024-05-08T18:08:45.404379+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:31:16.310504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"29'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:48.487844+0000","last_change":"2024-05-08T18:08:48.487844+0000","last_active":"2024-05-08T18:08:48.487844+0000","last_peered":"2024-05-08T18:08:48.487844+0000","last_clean":"2024-05-08T18:08:48.487844+0000","last_became_active":"2024-05-08T18:04:43.836410+0000","last_became_peered":"2024-05-08T18:04:43.836410+0000","last_unstale":"2024-05-08T18:08:48.487844+0000","last_undegraded":"2024-05-08T18:08:48.487844+0000","last_fullsized":"2024-05-08T18:08:48.487844+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:48.487704+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:48.487704+0000","last_clean_scrub_stamp":"2024-05-08T18:08:48.487704+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:10:35.537762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.849101+0000","last_change":"2024-05-08T18:08:11.849101+0000","last_active":"2024-05-08T18:08:11.849101+0000","last_peered":"2024-05-08T18:08:11.849101+0000","last_clean":"2024-05-08T18:08:11.849101+0000","last_became_active":"2024-05-08T18:04:47.818797+0000","last_became_peered":"2024-05-08T18:04:47.818797+0000","last_unstale":"2024-05-08T18:08:11.849101+0000","last_undegraded":"2024-05-08T18:08:11.849101+0000","last_fullsized":"2024-05-08T18:08:11.849101+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:11.848881+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:11.848881+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.848881+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:55:19.374213+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"29'37","reported_seq":93,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.217092+0000","last_change":"2024-05-08T18:08:12.217092+0000","last_active":"2024-05-08T18:08:12.217092+0000","last_peered":"2024-05-08T18:08:12.217092+0000","last_clean":"2024-05-08T18:08:12.217092+0000","last_became_active":"2024-05-08T18:04:49.831379+0000","last_became_peered":"2024-05-08T18:04:49.831379+0000","last_unstale":"2024-05-08T18:08:12.217092+0000","last_undegraded":"2024-05-08T18:08:12.217092+0000","last_fullsized":"2024-05-08T18:08:12.217092+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'37","last_scrub_stamp":"2024-05-08T18:08:12.216965+0000","last_deep_scrub":"29'37","last_deep_scrub_stamp":"2024-05-08T18:08:12.216965+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.216965+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:49:34.759465+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"29'285","reported_seq":454,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.529087+0000","last_change":"2024-05-08T18:08:11.529087+0000","last_active":"2024-05-08T18:08:11.529087+0000","last_peered":"2024-05-08T18:08:11.529087+0000","last_clean":"2024-05-08T18:08:11.529087+0000","last_became_active":"2024-05-08T18:04:51.826926+0000","last_became_peered":"2024-05-08T18:04:51.826926+0000","last_unstale":"2024-05-08T18:08:11.529087+0000","last_undegraded":"2024-05-08T18:08:11.529087+0000","last_fullsized":"2024-05-08T18:08:11.529087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:11.528959+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:11.528959+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.528959+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:55:44.650679+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'6","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:42.053554+0000","last_change":"2024-05-08T18:08:42.053554+0000","last_active":"2024-05-08T18:08:42.053554+0000","last_peered":"2024-05-08T18:08:42.053554+0000","last_clean":"2024-05-08T18:08:42.053554+0000","last_became_active":"2024-05-08T18:04:45.827351+0000","last_became_peered":"2024-05-08T18:04:45.827351+0000","last_unstale":"2024-05-08T18:08:42.053554+0000","last_undegraded":"2024-05-08T18:08:42.053554+0000","last_fullsized":"2024-05-08T18:08:42.053554+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-05-08T18:08:42.053430+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-05-08T18:08:42.053430+0000","last_clean_scrub_stamp":"2024-05-08T18:08:42.053430+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:29.385518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:47.984304+0000","last_change":"2024-05-08T18:08:47.984304+0000","last_active":"2024-05-08T18:08:47.984304+0000","last_peered":"2024-05-08T18:08:47.984304+0000","last_clean":"2024-05-08T18:08:47.984304+0000","last_became_active":"2024-05-08T18:04:43.838840+0000","last_became_peered":"2024-05-08T18:04:43.838840+0000","last_unstale":"2024-05-08T18:08:47.984304+0000","last_undegraded":"2024-05-08T18:08:47.984304+0000","last_fullsized":"2024-05-08T18:08:47.984304+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:47.984130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:47.984130+0000","last_clean_scrub_stamp":"2024-05-08T18:08:47.984130+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:31.557621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,0],"acting":[2,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.468441+0000","last_change":"2024-05-08T18:08:14.468441+0000","last_active":"2024-05-08T18:08:14.468441+0000","last_peered":"2024-05-08T18:08:14.468441+0000","last_clean":"2024-05-08T18:08:14.468441+0000","last_became_active":"2024-05-08T18:04:47.825215+0000","last_became_peered":"2024-05-08T18:04:47.825215+0000","last_unstale":"2024-05-08T18:08:14.468441+0000","last_undegraded":"2024-05-08T18:08:14.468441+0000","last_fullsized":"2024-05-08T18:08:14.468441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:14.468285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:14.468285+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.468285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:38.045371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"29'51","reported_seq":112,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.277853+0000","last_change":"2024-05-08T18:08:14.277853+0000","last_active":"2024-05-08T18:08:14.277853+0000","last_peered":"2024-05-08T18:08:14.277853+0000","last_clean":"2024-05-08T18:08:14.277853+0000","last_became_active":"2024-05-08T18:04:49.827954+0000","last_became_peered":"2024-05-08T18:04:49.827954+0000","last_unstale":"2024-05-08T18:08:14.277853+0000","last_undegraded":"2024-05-08T18:08:14.277853+0000","last_fullsized":"2024-05-08T18:08:14.277853+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'51","last_scrub_stamp":"2024-05-08T18:08:14.277647+0000","last_deep_scrub":"29'51","last_deep_scrub_stamp":"2024-05-08T18:08:14.277647+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.277647+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:23:20.826582+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.436758+0000","last_change":"2024-05-08T18:08:07.436758+0000","last_active":"2024-05-08T18:08:07.436758+0000","last_peered":"2024-05-08T18:08:07.436758+0000","last_clean":"2024-05-08T18:08:07.436758+0000","last_became_active":"2024-05-08T18:04:51.831149+0000","last_became_peered":"2024-05-08T18:04:51.831149+0000","last_unstale":"2024-05-08T18:08:07.436758+0000","last_undegraded":"2024-05-08T18:08:07.436758+0000","last_fullsized":"2024-05-08T18:08:07.436758+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:07.436627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:07.436627+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.436627+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:56:38.107730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"29'5","reported_seq":35,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:55.013107+0000","last_change":"2024-05-08T18:08:55.013107+0000","last_active":"2024-05-08T18:08:55.013107+0000","last_peered":"2024-05-08T18:08:55.013107+0000","last_clean":"2024-05-08T18:08:55.013107+0000","last_became_active":"2024-05-08T18:04:53.840652+0000","last_became_peered":"2024-05-08T18:04:53.840652+0000","last_unstale":"2024-05-08T18:08:55.013107+0000","last_undegraded":"2024-05-08T18:08:55.013107+0000","last_fullsized":"2024-05-08T18:08:55.013107+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'5","last_scrub_stamp":"2024-05-08T18:08:55.012991+0000","last_deep_scrub":"29'5","last_deep_scrub_stamp":"2024-05-08T18:08:55.012991+0000","last_clean_scrub_stamp":"2024-05-08T18:08:55.012991+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:55.220779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"27'4","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:38.499721+0000","last_change":"2024-05-08T18:08:38.499721+0000","last_active":"2024-05-08T18:08:38.499721+0000","last_peered":"2024-05-08T18:08:38.499721+0000","last_clean":"2024-05-08T18:08:38.499721+0000","last_became_active":"2024-05-08T18:04:45.804124+0000","last_became_peered":"2024-05-08T18:04:45.804124+0000","last_unstale":"2024-05-08T18:08:38.499721+0000","last_undegraded":"2024-05-08T18:08:38.499721+0000","last_fullsized":"2024-05-08T18:08:38.499721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-08T18:08:38.499539+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-08T18:08:38.499539+0000","last_clean_scrub_stamp":"2024-05-08T18:08:38.499539+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:58.286733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:36.190511+0000","last_change":"2024-05-08T18:08:36.190511+0000","last_active":"2024-05-08T18:08:36.190511+0000","last_peered":"2024-05-08T18:08:36.190511+0000","last_clean":"2024-05-08T18:08:36.190511+0000","last_became_active":"2024-05-08T18:04:43.842126+0000","last_became_peered":"2024-05-08T18:04:43.842126+0000","last_unstale":"2024-05-08T18:08:36.190511+0000","last_undegraded":"2024-05-08T18:08:36.190511+0000","last_fullsized":"2024-05-08T18:08:36.190511+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:36.190391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:36.190391+0000","last_clean_scrub_stamp":"2024-05-08T18:08:36.190391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:10:38.441913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.583963+0000","last_change":"2024-05-08T18:08:22.583963+0000","last_active":"2024-05-08T18:08:22.583963+0000","last_peered":"2024-05-08T18:08:22.583963+0000","last_clean":"2024-05-08T18:08:22.583963+0000","last_became_active":"2024-05-08T18:04:47.822688+0000","last_became_peered":"2024-05-08T18:04:47.822688+0000","last_unstale":"2024-05-08T18:08:22.583963+0000","last_undegraded":"2024-05-08T18:08:22.583963+0000","last_fullsized":"2024-05-08T18:08:22.583963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.583822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.583822+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.583822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:22.094540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.b","version":"29'36","reported_seq":92,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.839747+0000","last_change":"2024-05-08T18:08:16.839747+0000","last_active":"2024-05-08T18:08:16.839747+0000","last_peered":"2024-05-08T18:08:16.839747+0000","last_clean":"2024-05-08T18:08:16.839747+0000","last_became_active":"2024-05-08T18:04:49.829892+0000","last_became_peered":"2024-05-08T18:04:49.829892+0000","last_unstale":"2024-05-08T18:08:16.839747+0000","last_undegraded":"2024-05-08T18:08:16.839747+0000","last_fullsized":"2024-05-08T18:08:16.839747+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:08:16.839580+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:08:16.839580+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.839580+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:53.387986+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:04.455804+0000","last_change":"2024-05-08T18:08:04.455804+0000","last_active":"2024-05-08T18:08:04.455804+0000","last_peered":"2024-05-08T18:08:04.455804+0000","last_clean":"2024-05-08T18:08:04.455804+0000","last_became_active":"2024-05-08T18:04:51.832810+0000","last_became_peered":"2024-05-08T18:04:51.832810+0000","last_unstale":"2024-05-08T18:08:04.455804+0000","last_undegraded":"2024-05-08T18:08:04.455804+0000","last_fullsized":"2024-05-08T18:08:04.455804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:04.455699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:04.455699+0000","last_clean_scrub_stamp":"2024-05-08T18:08:04.455699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:21.455723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"29'4","reported_seq":34,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:29.520481+0000","last_change":"2024-05-08T18:08:29.520481+0000","last_active":"2024-05-08T18:08:29.520481+0000","last_peered":"2024-05-08T18:08:29.520481+0000","last_clean":"2024-05-08T18:08:29.520481+0000","last_became_active":"2024-05-08T18:04:53.840894+0000","last_became_peered":"2024-05-08T18:04:53.840894+0000","last_unstale":"2024-05-08T18:08:29.520481+0000","last_undegraded":"2024-05-08T18:08:29.520481+0000","last_fullsized":"2024-05-08T18:08:29.520481+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:29.520364+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:29.520364+0000","last_clean_scrub_stamp":"2024-05-08T18:08:29.520364+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:53.170014+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"29'10","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:37.218777+0000","last_change":"2024-05-08T18:08:37.218777+0000","last_active":"2024-05-08T18:08:37.218777+0000","last_peered":"2024-05-08T18:08:37.218777+0000","last_clean":"2024-05-08T18:08:37.218777+0000","last_became_active":"2024-05-08T18:04:45.816817+0000","last_became_peered":"2024-05-08T18:04:45.816817+0000","last_unstale":"2024-05-08T18:08:37.218777+0000","last_undegraded":"2024-05-08T18:08:37.218777+0000","last_fullsized":"2024-05-08T18:08:37.218777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:37.218630+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:37.218630+0000","last_clean_scrub_stamp":"2024-05-08T18:08:37.218630+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:39.786476+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"29'4","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:03.027396+0000","last_change":"2024-05-08T18:09:03.027396+0000","last_active":"2024-05-08T18:09:03.027396+0000","last_peered":"2024-05-08T18:09:03.027396+0000","last_clean":"2024-05-08T18:09:03.027396+0000","last_became_active":"2024-05-08T18:04:43.838781+0000","last_became_peered":"2024-05-08T18:04:43.838781+0000","last_unstale":"2024-05-08T18:09:03.027396+0000","last_undegraded":"2024-05-08T18:09:03.027396+0000","last_fullsized":"2024-05-08T18:09:03.027396+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:09:03.027231+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:09:03.027231+0000","last_clean_scrub_stamp":"2024-05-08T18:09:03.027231+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:57:33.005457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:30.495329+0000","last_change":"2024-05-08T18:08:30.495329+0000","last_active":"2024-05-08T18:08:30.495329+0000","last_peered":"2024-05-08T18:08:30.495329+0000","last_clean":"2024-05-08T18:08:30.495329+0000","last_became_active":"2024-05-08T18:04:47.818871+0000","last_became_peered":"2024-05-08T18:04:47.818871+0000","last_unstale":"2024-05-08T18:08:30.495329+0000","last_undegraded":"2024-05-08T18:08:30.495329+0000","last_fullsized":"2024-05-08T18:08:30.495329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:30.495168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:30.495168+0000","last_clean_scrub_stamp":"2024-05-08T18:08:30.495168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:26.778212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"29'45","reported_seq":104,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.892899+0000","last_change":"2024-05-08T18:08:08.892899+0000","last_active":"2024-05-08T18:08:08.892899+0000","last_peered":"2024-05-08T18:08:08.892899+0000","last_clean":"2024-05-08T18:08:08.892899+0000","last_became_active":"2024-05-08T18:04:49.829270+0000","last_became_peered":"2024-05-08T18:04:49.829270+0000","last_unstale":"2024-05-08T18:08:08.892899+0000","last_undegraded":"2024-05-08T18:08:08.892899+0000","last_fullsized":"2024-05-08T18:08:08.892899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'45","last_scrub_stamp":"2024-05-08T18:08:08.892763+0000","last_deep_scrub":"29'45","last_deep_scrub_stamp":"2024-05-08T18:08:08.892763+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.892763+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:11.460610+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"29'285","reported_seq":361,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.576007+0000","last_change":"2024-05-08T18:08:13.576007+0000","last_active":"2024-05-08T18:08:13.576007+0000","last_peered":"2024-05-08T18:08:13.576007+0000","last_clean":"2024-05-08T18:08:13.576007+0000","last_became_active":"2024-05-08T18:04:51.844529+0000","last_became_peered":"2024-05-08T18:04:51.844529+0000","last_unstale":"2024-05-08T18:08:13.576007+0000","last_undegraded":"2024-05-08T18:08:13.576007+0000","last_fullsized":"2024-05-08T18:08:13.576007+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:13.575886+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:13.575886+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.575886+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:46.724169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"29'2","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:01.474503+0000","last_change":"2024-05-08T18:09:01.474503+0000","last_active":"2024-05-08T18:09:01.474503+0000","last_peered":"2024-05-08T18:09:01.474503+0000","last_clean":"2024-05-08T18:09:01.474503+0000","last_became_active":"2024-05-08T18:04:53.840767+0000","last_became_peered":"2024-05-08T18:04:53.840767+0000","last_unstale":"2024-05-08T18:09:01.474503+0000","last_undegraded":"2024-05-08T18:09:01.474503+0000","last_fullsized":"2024-05-08T18:09:01.474503+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:09:01.474327+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:09:01.474327+0000","last_clean_scrub_stamp":"2024-05-08T18:09:01.474327+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:50:50.275478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"29'6","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:35.195855+0000","last_change":"2024-05-08T18:08:35.195855+0000","last_active":"2024-05-08T18:08:35.195855+0000","last_peered":"2024-05-08T18:08:35.195855+0000","last_clean":"2024-05-08T18:08:35.195855+0000","last_became_active":"2024-05-08T18:04:45.813735+0000","last_became_peered":"2024-05-08T18:04:45.813735+0000","last_unstale":"2024-05-08T18:08:35.195855+0000","last_undegraded":"2024-05-08T18:08:35.195855+0000","last_fullsized":"2024-05-08T18:08:35.195855+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:35.195741+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:35.195741+0000","last_clean_scrub_stamp":"2024-05-08T18:08:35.195741+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:44:58.936221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"27'4","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:56.007674+0000","last_change":"2024-05-08T18:08:56.007674+0000","last_active":"2024-05-08T18:08:56.007674+0000","last_peered":"2024-05-08T18:08:56.007674+0000","last_clean":"2024-05-08T18:08:56.007674+0000","last_became_active":"2024-05-08T18:04:43.839047+0000","last_became_peered":"2024-05-08T18:04:43.839047+0000","last_unstale":"2024-05-08T18:08:56.007674+0000","last_undegraded":"2024-05-08T18:08:56.007674+0000","last_fullsized":"2024-05-08T18:08:56.007674+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-08T18:08:56.007537+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-08T18:08:56.007537+0000","last_clean_scrub_stamp":"2024-05-08T18:08:56.007537+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:29:37.060769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":3,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:28.472766+0000","last_change":"2024-05-08T18:08:28.472766+0000","last_active":"2024-05-08T18:08:28.472766+0000","last_peered":"2024-05-08T18:08:28.472766+0000","last_clean":"2024-05-08T18:08:28.472766+0000","last_became_active":"2024-05-08T18:04:47.818972+0000","last_became_peered":"2024-05-08T18:04:47.818972+0000","last_unstale":"2024-05-08T18:08:28.472766+0000","last_undegraded":"2024-05-08T18:08:28.472766+0000","last_fullsized":"2024-05-08T18:08:28.472766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:28.472654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:28.472654+0000","last_clean_scrub_stamp":"2024-05-08T18:08:28.472654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:31.040500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.9","version":"29'63","reported_seq":137,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.861715+0000","last_change":"2024-05-08T18:08:12.861715+0000","last_active":"2024-05-08T18:08:12.861715+0000","last_peered":"2024-05-08T18:08:12.861715+0000","last_clean":"2024-05-08T18:08:12.861715+0000","last_became_active":"2024-05-08T18:04:49.830424+0000","last_became_peered":"2024-05-08T18:04:49.830424+0000","last_unstale":"2024-05-08T18:08:12.861715+0000","last_undegraded":"2024-05-08T18:08:12.861715+0000","last_fullsized":"2024-05-08T18:08:12.861715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'63","last_scrub_stamp":"2024-05-08T18:08:12.861577+0000","last_deep_scrub":"29'63","last_deep_scrub_stamp":"2024-05-08T18:08:12.861577+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.861577+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:55:02.260578+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.479304+0000","last_change":"2024-05-08T18:08:10.479304+0000","last_active":"2024-05-08T18:08:10.479304+0000","last_peered":"2024-05-08T18:08:10.479304+0000","last_clean":"2024-05-08T18:08:10.479304+0000","last_became_active":"2024-05-08T18:04:51.830383+0000","last_became_peered":"2024-05-08T18:04:51.830383+0000","last_unstale":"2024-05-08T18:08:10.479304+0000","last_undegraded":"2024-05-08T18:08:10.479304+0000","last_fullsized":"2024-05-08T18:08:10.479304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:10.479187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:10.479187+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.479187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:23:04.216069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"29'23","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:04.511456+0000","last_change":"2024-05-08T18:09:04.511456+0000","last_active":"2024-05-08T18:09:04.511456+0000","last_peered":"2024-05-08T18:09:04.511456+0000","last_clean":"2024-05-08T18:09:04.511456+0000","last_became_active":"2024-05-08T18:04:53.847842+0000","last_became_peered":"2024-05-08T18:04:53.847842+0000","last_unstale":"2024-05-08T18:09:04.511456+0000","last_undegraded":"2024-05-08T18:09:04.511456+0000","last_fullsized":"2024-05-08T18:09:04.511456+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'23","last_scrub_stamp":"2024-05-08T18:09:04.511321+0000","last_deep_scrub":"29'23","last_deep_scrub_stamp":"2024-05-08T18:09:04.511321+0000","last_clean_scrub_stamp":"2024-05-08T18:09:04.511321+0000","objects_scrubbed":3,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:49:49.098612+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":48,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"29'6","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.511472+0000","last_change":"2024-05-08T18:08:23.511472+0000","last_active":"2024-05-08T18:08:23.511472+0000","last_peered":"2024-05-08T18:08:23.511472+0000","last_clean":"2024-05-08T18:08:23.511472+0000","last_became_active":"2024-05-08T18:04:45.821412+0000","last_became_peered":"2024-05-08T18:04:45.821412+0000","last_unstale":"2024-05-08T18:08:23.511472+0000","last_undegraded":"2024-05-08T18:08:23.511472+0000","last_fullsized":"2024-05-08T18:08:23.511472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:23.511346+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:23.511346+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.511346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:04.193921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:44.361833+0000","last_change":"2024-05-08T18:08:44.361833+0000","last_active":"2024-05-08T18:08:44.361833+0000","last_peered":"2024-05-08T18:08:44.361833+0000","last_clean":"2024-05-08T18:08:44.361833+0000","last_became_active":"2024-05-08T18:04:43.834723+0000","last_became_peered":"2024-05-08T18:04:43.834723+0000","last_unstale":"2024-05-08T18:08:44.361833+0000","last_undegraded":"2024-05-08T18:08:44.361833+0000","last_fullsized":"2024-05-08T18:08:44.361833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:44.361707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:44.361707+0000","last_clean_scrub_stamp":"2024-05-08T18:08:44.361707+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:54.804858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1,2],"acting":[0,3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.295265+0000","last_change":"2024-05-08T18:08:22.295265+0000","last_active":"2024-05-08T18:08:22.295265+0000","last_peered":"2024-05-08T18:08:22.295265+0000","last_clean":"2024-05-08T18:08:22.295265+0000","last_became_active":"2024-05-08T18:04:47.822865+0000","last_became_peered":"2024-05-08T18:04:47.822865+0000","last_unstale":"2024-05-08T18:08:22.295265+0000","last_undegraded":"2024-05-08T18:08:22.295265+0000","last_fullsized":"2024-05-08T18:08:22.295265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.295113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.295113+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.295113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:31:07.200771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"29'40","reported_seq":97,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:11.481605+0000","last_change":"2024-05-08T18:08:11.481605+0000","last_active":"2024-05-08T18:08:11.481605+0000","last_peered":"2024-05-08T18:08:11.481605+0000","last_clean":"2024-05-08T18:08:11.481605+0000","last_became_active":"2024-05-08T18:04:49.824610+0000","last_became_peered":"2024-05-08T18:04:49.824610+0000","last_unstale":"2024-05-08T18:08:11.481605+0000","last_undegraded":"2024-05-08T18:08:11.481605+0000","last_fullsized":"2024-05-08T18:08:11.481605+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'40","last_scrub_stamp":"2024-05-08T18:08:11.481461+0000","last_deep_scrub":"29'40","last_deep_scrub_stamp":"2024-05-08T18:08:11.481461+0000","last_clean_scrub_stamp":"2024-05-08T18:08:11.481461+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:15.729540+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:28.829264+0000","last_change":"2024-05-08T18:08:28.829264+0000","last_active":"2024-05-08T18:08:28.829264+0000","last_peered":"2024-05-08T18:08:28.829264+0000","last_clean":"2024-05-08T18:08:28.829264+0000","last_became_active":"2024-05-08T18:04:51.848793+0000","last_became_peered":"2024-05-08T18:04:51.848793+0000","last_unstale":"2024-05-08T18:08:28.829264+0000","last_undegraded":"2024-05-08T18:08:28.829264+0000","last_fullsized":"2024-05-08T18:08:28.829264+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:28.829111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:28.829111+0000","last_clean_scrub_stamp":"2024-05-08T18:08:28.829111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:35.508902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"29'6","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:27.990110+0000","last_change":"2024-05-08T18:09:27.990140+0000","last_active":"2024-05-08T18:09:27.990110+0000","last_peered":"2024-05-08T18:09:27.990110+0000","last_clean":"2024-05-08T18:09:27.990110+0000","last_became_active":"2024-05-08T18:04:53.848945+0000","last_became_peered":"2024-05-08T18:04:53.848945+0000","last_unstale":"2024-05-08T18:09:27.990110+0000","last_undegraded":"2024-05-08T18:09:27.990110+0000","last_fullsized":"2024-05-08T18:09:27.990110+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:09:27.990017+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:09:27.990017+0000","last_clean_scrub_stamp":"2024-05-08T18:09:27.990017+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:30:09.312139+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"29'10","reported_seq":85,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.836220+0000","last_change":"2024-05-08T18:08:23.836220+0000","last_active":"2024-05-08T18:08:23.836220+0000","last_peered":"2024-05-08T18:08:23.836220+0000","last_clean":"2024-05-08T18:08:23.836220+0000","last_became_active":"2024-05-08T18:04:45.829120+0000","last_became_peered":"2024-05-08T18:04:45.829120+0000","last_unstale":"2024-05-08T18:08:23.836220+0000","last_undegraded":"2024-05-08T18:08:23.836220+0000","last_fullsized":"2024-05-08T18:08:23.836220+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:23.836088+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:23.836088+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.836088+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:15.906999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":61,"num_read_kb":57,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:09.962858+0000","last_change":"2024-05-08T18:09:09.962886+0000","last_active":"2024-05-08T18:09:09.962858+0000","last_peered":"2024-05-08T18:09:09.962858+0000","last_clean":"2024-05-08T18:09:09.962858+0000","last_became_active":"2024-05-08T18:04:43.834536+0000","last_became_peered":"2024-05-08T18:04:43.834536+0000","last_unstale":"2024-05-08T18:09:09.962858+0000","last_undegraded":"2024-05-08T18:09:09.962858+0000","last_fullsized":"2024-05-08T18:09:09.962858+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:09.962738+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:09.962738+0000","last_clean_scrub_stamp":"2024-05-08T18:09:09.962738+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:37:14.961834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"29'5","reported_seq":239,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:26.539766+0000","last_change":"2024-05-08T18:08:26.539766+0000","last_active":"2024-05-08T18:08:26.539766+0000","last_peered":"2024-05-08T18:08:26.539766+0000","last_clean":"2024-05-08T18:08:26.539766+0000","last_became_active":"2024-05-08T18:04:47.819085+0000","last_became_peered":"2024-05-08T18:04:47.819085+0000","last_unstale":"2024-05-08T18:08:26.539766+0000","last_undegraded":"2024-05-08T18:08:26.539766+0000","last_fullsized":"2024-05-08T18:08:26.539766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'5","last_scrub_stamp":"2024-05-08T18:08:26.539641+0000","last_deep_scrub":"29'5","last_deep_scrub_stamp":"2024-05-08T18:08:26.539641+0000","last_clean_scrub_stamp":"2024-05-08T18:08:26.539641+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:50.189291+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":301,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"29'45","reported_seq":104,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.280897+0000","last_change":"2024-05-08T18:08:16.280897+0000","last_active":"2024-05-08T18:08:16.280897+0000","last_peered":"2024-05-08T18:08:16.280897+0000","last_clean":"2024-05-08T18:08:16.280897+0000","last_became_active":"2024-05-08T18:04:49.825613+0000","last_became_peered":"2024-05-08T18:04:49.825613+0000","last_unstale":"2024-05-08T18:08:16.280897+0000","last_undegraded":"2024-05-08T18:08:16.280897+0000","last_fullsized":"2024-05-08T18:08:16.280897+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'45","last_scrub_stamp":"2024-05-08T18:08:16.280719+0000","last_deep_scrub":"29'45","last_deep_scrub_stamp":"2024-05-08T18:08:16.280719+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.280719+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:29:19.130403+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"29'285","reported_seq":444,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.137883+0000","last_change":"2024-05-08T18:08:06.137883+0000","last_active":"2024-05-08T18:08:06.137883+0000","last_peered":"2024-05-08T18:08:06.137883+0000","last_clean":"2024-05-08T18:08:06.137883+0000","last_became_active":"2024-05-08T18:04:51.830924+0000","last_became_peered":"2024-05-08T18:04:51.830924+0000","last_unstale":"2024-05-08T18:08:06.137883+0000","last_undegraded":"2024-05-08T18:08:06.137883+0000","last_fullsized":"2024-05-08T18:08:06.137883+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:06.137767+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:06.137767+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.137767+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:22:43.622614+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"29'15","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:02.458510+0000","last_change":"2024-05-08T18:09:02.458510+0000","last_active":"2024-05-08T18:09:02.458510+0000","last_peered":"2024-05-08T18:09:02.458510+0000","last_clean":"2024-05-08T18:09:02.458510+0000","last_became_active":"2024-05-08T18:04:53.840708+0000","last_became_peered":"2024-05-08T18:04:53.840708+0000","last_unstale":"2024-05-08T18:09:02.458510+0000","last_undegraded":"2024-05-08T18:09:02.458510+0000","last_fullsized":"2024-05-08T18:09:02.458510+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:09:02.458318+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:09:02.458318+0000","last_clean_scrub_stamp":"2024-05-08T18:09:02.458318+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:49.790407+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":180,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"29'6","reported_seq":64,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:31.301337+0000","last_change":"2024-05-08T18:08:31.301337+0000","last_active":"2024-05-08T18:08:31.301337+0000","last_peered":"2024-05-08T18:08:31.301337+0000","last_clean":"2024-05-08T18:08:31.301337+0000","last_became_active":"2024-05-08T18:04:45.811926+0000","last_became_peered":"2024-05-08T18:04:45.811926+0000","last_unstale":"2024-05-08T18:08:31.301337+0000","last_undegraded":"2024-05-08T18:08:31.301337+0000","last_fullsized":"2024-05-08T18:08:31.301337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:31.301216+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:31.301216+0000","last_clean_scrub_stamp":"2024-05-08T18:08:31.301216+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:32:41.104191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"29'2","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:01.003015+0000","last_change":"2024-05-08T18:09:01.003015+0000","last_active":"2024-05-08T18:09:01.003015+0000","last_peered":"2024-05-08T18:09:01.003015+0000","last_clean":"2024-05-08T18:09:01.003015+0000","last_became_active":"2024-05-08T18:04:43.843934+0000","last_became_peered":"2024-05-08T18:04:43.843934+0000","last_unstale":"2024-05-08T18:09:01.003015+0000","last_undegraded":"2024-05-08T18:09:01.003015+0000","last_fullsized":"2024-05-08T18:09:01.003015+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:09:01.002893+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:09:01.002893+0000","last_clean_scrub_stamp":"2024-05-08T18:09:01.002893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:41:23.689098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8,"num_read_kb":5,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"29'5","reported_seq":429,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.550071+0000","last_change":"2024-05-08T18:08:20.550071+0000","last_active":"2024-05-08T18:08:20.550071+0000","last_peered":"2024-05-08T18:08:20.550071+0000","last_clean":"2024-05-08T18:08:20.550071+0000","last_became_active":"2024-05-08T18:04:47.817851+0000","last_became_peered":"2024-05-08T18:04:47.817851+0000","last_unstale":"2024-05-08T18:08:20.550071+0000","last_undegraded":"2024-05-08T18:08:20.550071+0000","last_fullsized":"2024-05-08T18:08:20.550071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'5","last_scrub_stamp":"2024-05-08T18:08:20.549943+0000","last_deep_scrub":"29'5","last_deep_scrub_stamp":"2024-05-08T18:08:20.549943+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.549943+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:16:31.792211+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"29'15","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.800470+0000","last_change":"2024-05-08T18:08:18.800470+0000","last_active":"2024-05-08T18:08:18.800470+0000","last_peered":"2024-05-08T18:08:18.800470+0000","last_clean":"2024-05-08T18:08:18.800470+0000","last_became_active":"2024-05-08T18:04:49.830714+0000","last_became_peered":"2024-05-08T18:04:49.830714+0000","last_unstale":"2024-05-08T18:08:18.800470+0000","last_undegraded":"2024-05-08T18:08:18.800470+0000","last_fullsized":"2024-05-08T18:08:18.800470+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:18.800324+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:18.800324+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.800324+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:41.821593+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"29'285","reported_seq":481,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.542295+0000","last_change":"2024-05-08T18:08:05.542295+0000","last_active":"2024-05-08T18:08:05.542295+0000","last_peered":"2024-05-08T18:08:05.542295+0000","last_clean":"2024-05-08T18:08:05.542295+0000","last_became_active":"2024-05-08T18:04:51.831508+0000","last_became_peered":"2024-05-08T18:04:51.831508+0000","last_unstale":"2024-05-08T18:08:05.542295+0000","last_undegraded":"2024-05-08T18:08:05.542295+0000","last_fullsized":"2024-05-08T18:08:05.542295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:05.542169+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:05.542169+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.542169+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:49:28.084669+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"29'4","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:52.956968+0000","last_change":"2024-05-08T18:08:52.956968+0000","last_active":"2024-05-08T18:08:52.956968+0000","last_peered":"2024-05-08T18:08:52.956968+0000","last_clean":"2024-05-08T18:08:52.956968+0000","last_became_active":"2024-05-08T18:04:53.840215+0000","last_became_peered":"2024-05-08T18:04:53.840215+0000","last_unstale":"2024-05-08T18:08:52.956968+0000","last_undegraded":"2024-05-08T18:08:52.956968+0000","last_fullsized":"2024-05-08T18:08:52.956968+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:52.956850+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:52.956850+0000","last_clean_scrub_stamp":"2024-05-08T18:08:52.956850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:44:16.337663+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'2","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:30.526084+0000","last_change":"2024-05-08T18:08:30.526084+0000","last_active":"2024-05-08T18:08:30.526084+0000","last_peered":"2024-05-08T18:08:30.526084+0000","last_clean":"2024-05-08T18:08:30.526084+0000","last_became_active":"2024-05-08T18:04:45.818533+0000","last_became_peered":"2024-05-08T18:04:45.818533+0000","last_unstale":"2024-05-08T18:08:30.526084+0000","last_undegraded":"2024-05-08T18:08:30.526084+0000","last_fullsized":"2024-05-08T18:08:30.526084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:30.525970+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:30.525970+0000","last_clean_scrub_stamp":"2024-05-08T18:08:30.525970+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:27.414733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"29'2","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:55.032953+0000","last_change":"2024-05-08T18:08:55.032953+0000","last_active":"2024-05-08T18:08:55.032953+0000","last_peered":"2024-05-08T18:08:55.032953+0000","last_clean":"2024-05-08T18:08:55.032953+0000","last_became_active":"2024-05-08T18:04:43.846125+0000","last_became_peered":"2024-05-08T18:04:43.846125+0000","last_unstale":"2024-05-08T18:08:55.032953+0000","last_undegraded":"2024-05-08T18:08:55.032953+0000","last_fullsized":"2024-05-08T18:08:55.032953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:55.032817+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:55.032817+0000","last_clean_scrub_stamp":"2024-05-08T18:08:55.032817+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:29:54.513234+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.895821+0000","last_change":"2024-05-08T18:08:05.895821+0000","last_active":"2024-05-08T18:08:05.895821+0000","last_peered":"2024-05-08T18:08:05.895821+0000","last_clean":"2024-05-08T18:08:05.895821+0000","last_became_active":"2024-05-08T18:04:47.822242+0000","last_became_peered":"2024-05-08T18:04:47.822242+0000","last_unstale":"2024-05-08T18:08:05.895821+0000","last_undegraded":"2024-05-08T18:08:05.895821+0000","last_fullsized":"2024-05-08T18:08:05.895821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:05.895706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:05.895706+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.895706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:00:55.969119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"29'48","reported_seq":108,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.545419+0000","last_change":"2024-05-08T18:08:08.545419+0000","last_active":"2024-05-08T18:08:08.545419+0000","last_peered":"2024-05-08T18:08:08.545419+0000","last_clean":"2024-05-08T18:08:08.545419+0000","last_became_active":"2024-05-08T18:04:49.825314+0000","last_became_peered":"2024-05-08T18:04:49.825314+0000","last_unstale":"2024-05-08T18:08:08.545419+0000","last_undegraded":"2024-05-08T18:08:08.545419+0000","last_fullsized":"2024-05-08T18:08:08.545419+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'48","last_scrub_stamp":"2024-05-08T18:08:08.545269+0000","last_deep_scrub":"29'48","last_deep_scrub_stamp":"2024-05-08T18:08:08.545269+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.545269+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:19.079352+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.136723+0000","last_change":"2024-05-08T18:08:05.136723+0000","last_active":"2024-05-08T18:08:05.136723+0000","last_peered":"2024-05-08T18:08:05.136723+0000","last_clean":"2024-05-08T18:08:05.136723+0000","last_became_active":"2024-05-08T18:04:51.832808+0000","last_became_peered":"2024-05-08T18:04:51.832808+0000","last_unstale":"2024-05-08T18:08:05.136723+0000","last_undegraded":"2024-05-08T18:08:05.136723+0000","last_fullsized":"2024-05-08T18:08:05.136723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:05.136592+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:05.136592+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.136592+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:13:30.377482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"29'9","reported_seq":37,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:25.506016+0000","last_change":"2024-05-08T18:08:25.506016+0000","last_active":"2024-05-08T18:08:25.506016+0000","last_peered":"2024-05-08T18:08:25.506016+0000","last_clean":"2024-05-08T18:08:25.506016+0000","last_became_active":"2024-05-08T18:04:53.837292+0000","last_became_peered":"2024-05-08T18:04:53.837292+0000","last_unstale":"2024-05-08T18:08:25.506016+0000","last_undegraded":"2024-05-08T18:08:25.506016+0000","last_fullsized":"2024-05-08T18:08:25.506016+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'9","last_scrub_stamp":"2024-05-08T18:08:25.505900+0000","last_deep_scrub":"29'9","last_deep_scrub_stamp":"2024-05-08T18:08:25.505900+0000","last_clean_scrub_stamp":"2024-05-08T18:08:25.505900+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:33:25.016619+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":12,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"29'4","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:25.876183+0000","last_change":"2024-05-08T18:08:25.876183+0000","last_active":"2024-05-08T18:08:25.876183+0000","last_peered":"2024-05-08T18:08:25.876183+0000","last_clean":"2024-05-08T18:08:25.876183+0000","last_became_active":"2024-05-08T18:04:45.828674+0000","last_became_peered":"2024-05-08T18:04:45.828674+0000","last_unstale":"2024-05-08T18:08:25.876183+0000","last_undegraded":"2024-05-08T18:08:25.876183+0000","last_fullsized":"2024-05-08T18:08:25.876183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:25.876052+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:25.876052+0000","last_clean_scrub_stamp":"2024-05-08T18:08:25.876052+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:30:56.782177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"29'74","reported_seq":149,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:51.419922+0000","last_change":"2024-05-08T18:08:51.419959+0000","last_active":"2024-05-08T18:08:51.419922+0000","last_peered":"2024-05-08T18:08:51.419922+0000","last_clean":"2024-05-08T18:08:51.419922+0000","last_became_active":"2024-05-08T18:04:35.551437+0000","last_became_peered":"2024-05-08T18:04:35.551437+0000","last_unstale":"2024-05-08T18:08:51.419922+0000","last_undegraded":"2024-05-08T18:08:51.419922+0000","last_fullsized":"2024-05-08T18:08:51.419922+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'74","last_scrub_stamp":"2024-05-08T18:08:51.419824+0000","last_deep_scrub":"29'74","last_deep_scrub_stamp":"2024-05-08T18:08:51.419824+0000","last_clean_scrub_stamp":"2024-05-08T18:08:51.419824+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:39:27.870934+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:18.009610+0000","last_change":"2024-05-08T18:09:18.009637+0000","last_active":"2024-05-08T18:09:18.009610+0000","last_peered":"2024-05-08T18:09:18.009610+0000","last_clean":"2024-05-08T18:09:18.009610+0000","last_became_active":"2024-05-08T18:04:35.552017+0000","last_became_peered":"2024-05-08T18:04:35.552017+0000","last_unstale":"2024-05-08T18:09:18.009610+0000","last_undegraded":"2024-05-08T18:09:18.009610+0000","last_fullsized":"2024-05-08T18:09:18.009610+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:18.009505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:18.009505+0000","last_clean_scrub_stamp":"2024-05-08T18:09:18.009505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:38:44.258938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:04.006356+0000","last_change":"2024-05-08T18:09:04.006401+0000","last_active":"2024-05-08T18:09:04.006356+0000","last_peered":"2024-05-08T18:09:04.006356+0000","last_clean":"2024-05-08T18:09:04.006356+0000","last_became_active":"2024-05-08T18:04:43.837019+0000","last_became_peered":"2024-05-08T18:04:43.837019+0000","last_unstale":"2024-05-08T18:09:04.006356+0000","last_undegraded":"2024-05-08T18:09:04.006356+0000","last_fullsized":"2024-05-08T18:09:04.006356+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:04.006222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:04.006222+0000","last_clean_scrub_stamp":"2024-05-08T18:09:04.006222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:36.210550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:06.524219+0000","last_change":"2024-05-08T18:09:06.524244+0000","last_active":"2024-05-08T18:09:06.524219+0000","last_peered":"2024-05-08T18:09:06.524219+0000","last_clean":"2024-05-08T18:09:06.524219+0000","last_became_active":"2024-05-08T18:05:30.870647+0000","last_became_peered":"2024-05-08T18:05:30.870647+0000","last_unstale":"2024-05-08T18:09:06.524219+0000","last_undegraded":"2024-05-08T18:09:06.524219+0000","last_fullsized":"2024-05-08T18:09:06.524219+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:06.524126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:06.524126+0000","last_clean_scrub_stamp":"2024-05-08T18:09:06.524126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:53.856214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"29'91","reported_seq":182,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:34.946658+0000","last_change":"2024-05-08T18:08:34.946658+0000","last_active":"2024-05-08T18:08:34.946658+0000","last_peered":"2024-05-08T18:08:34.946658+0000","last_clean":"2024-05-08T18:08:34.946658+0000","last_became_active":"2024-05-08T18:04:49.830636+0000","last_became_peered":"2024-05-08T18:04:49.830636+0000","last_unstale":"2024-05-08T18:08:34.946658+0000","last_undegraded":"2024-05-08T18:08:34.946658+0000","last_fullsized":"2024-05-08T18:08:34.946658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'91","last_scrub_stamp":"2024-05-08T18:08:34.946526+0000","last_deep_scrub":"29'91","last_deep_scrub_stamp":"2024-05-08T18:08:34.946526+0000","last_clean_scrub_stamp":"2024-05-08T18:08:34.946526+0000","objects_scrubbed":5,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:54:49.081723+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":84,"num_read_kb":73,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.600230+0000","last_change":"2024-05-08T18:08:21.600230+0000","last_active":"2024-05-08T18:08:21.600230+0000","last_peered":"2024-05-08T18:08:21.600230+0000","last_clean":"2024-05-08T18:08:21.600230+0000","last_became_active":"2024-05-08T18:04:51.844737+0000","last_became_peered":"2024-05-08T18:04:51.844737+0000","last_unstale":"2024-05-08T18:08:21.600230+0000","last_undegraded":"2024-05-08T18:08:21.600230+0000","last_fullsized":"2024-05-08T18:08:21.600230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:21.600069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:21.600069+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.600069+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:48.902927+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"29'86","reported_seq":177,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:25.255820+0000","last_change":"2024-05-08T18:08:25.255820+0000","last_active":"2024-05-08T18:08:25.255820+0000","last_peered":"2024-05-08T18:08:25.255820+0000","last_clean":"2024-05-08T18:08:25.255820+0000","last_became_active":"2024-05-08T18:04:53.838957+0000","last_became_peered":"2024-05-08T18:04:53.838957+0000","last_unstale":"2024-05-08T18:08:25.255820+0000","last_undegraded":"2024-05-08T18:08:25.255820+0000","last_fullsized":"2024-05-08T18:08:25.255820+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'86","last_scrub_stamp":"2024-05-08T18:08:25.255680+0000","last_deep_scrub":"29'86","last_deep_scrub_stamp":"2024-05-08T18:08:25.255680+0000","last_clean_scrub_stamp":"2024-05-08T18:08:25.255680+0000","objects_scrubbed":1,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:14:28.008294+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":216,"num_read_kb":214,"num_write":167,"num_write_kb":54,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"29'10","reported_seq":76,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:48.083623+0000","last_change":"2024-05-08T18:08:48.083623+0000","last_active":"2024-05-08T18:08:48.083623+0000","last_peered":"2024-05-08T18:08:48.083623+0000","last_clean":"2024-05-08T18:08:48.083623+0000","last_became_active":"2024-05-08T18:04:45.825843+0000","last_became_peered":"2024-05-08T18:04:45.825843+0000","last_unstale":"2024-05-08T18:08:48.083623+0000","last_undegraded":"2024-05-08T18:08:48.083623+0000","last_fullsized":"2024-05-08T18:08:48.083623+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:48.083503+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:48.083503+0000","last_clean_scrub_stamp":"2024-05-08T18:08:48.083503+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:05:27.160237+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:18.964836+0000","last_change":"2024-05-08T18:09:18.964862+0000","last_active":"2024-05-08T18:09:18.964836+0000","last_peered":"2024-05-08T18:09:18.964836+0000","last_clean":"2024-05-08T18:09:18.964836+0000","last_became_active":"2024-05-08T18:04:35.552214+0000","last_became_peered":"2024-05-08T18:04:35.552214+0000","last_unstale":"2024-05-08T18:09:18.964836+0000","last_undegraded":"2024-05-08T18:09:18.964836+0000","last_fullsized":"2024-05-08T18:09:18.964836+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:18.964742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:18.964742+0000","last_clean_scrub_stamp":"2024-05-08T18:09:18.964742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:32.307203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:54.429770+0000","last_change":"2024-05-08T18:08:54.429815+0000","last_active":"2024-05-08T18:08:54.429770+0000","last_peered":"2024-05-08T18:08:54.429770+0000","last_clean":"2024-05-08T18:08:54.429770+0000","last_became_active":"2024-05-08T18:04:43.820369+0000","last_became_peered":"2024-05-08T18:04:43.820369+0000","last_unstale":"2024-05-08T18:08:54.429770+0000","last_undegraded":"2024-05-08T18:08:54.429770+0000","last_fullsized":"2024-05-08T18:08:54.429770+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:54.429672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:54.429672+0000","last_clean_scrub_stamp":"2024-05-08T18:08:54.429672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:12.348107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":44,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:29.944002+0000","last_change":"2024-05-08T18:09:29.944027+0000","last_active":"2024-05-08T18:09:29.944002+0000","last_peered":"2024-05-08T18:09:29.944002+0000","last_clean":"2024-05-08T18:09:29.944002+0000","last_became_active":"2024-05-08T18:05:30.869578+0000","last_became_peered":"2024-05-08T18:05:30.869578+0000","last_unstale":"2024-05-08T18:09:29.944002+0000","last_undegraded":"2024-05-08T18:09:29.944002+0000","last_fullsized":"2024-05-08T18:09:29.944002+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:29.943899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:29.943899+0000","last_clean_scrub_stamp":"2024-05-08T18:09:29.943899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:16:49.416325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"29'290","reported_seq":516,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.566330+0000","last_change":"2024-05-08T18:08:07.566330+0000","last_active":"2024-05-08T18:08:07.566330+0000","last_peered":"2024-05-08T18:08:07.566330+0000","last_clean":"2024-05-08T18:08:07.566330+0000","last_became_active":"2024-05-08T18:04:49.833053+0000","last_became_peered":"2024-05-08T18:04:49.833053+0000","last_unstale":"2024-05-08T18:08:07.566330+0000","last_undegraded":"2024-05-08T18:08:07.566330+0000","last_fullsized":"2024-05-08T18:08:07.566330+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'290","last_scrub_stamp":"2024-05-08T18:08:07.566185+0000","last_deep_scrub":"29'290","last_deep_scrub_stamp":"2024-05-08T18:08:07.566185+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.566185+0000","objects_scrubbed":10,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:24.566210+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":378,"num_read_kb":363,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:32.883526+0000","last_change":"2024-05-08T18:08:32.883526+0000","last_active":"2024-05-08T18:08:32.883526+0000","last_peered":"2024-05-08T18:08:32.883526+0000","last_clean":"2024-05-08T18:08:32.883526+0000","last_became_active":"2024-05-08T18:04:51.848066+0000","last_became_peered":"2024-05-08T18:04:51.848066+0000","last_unstale":"2024-05-08T18:08:32.883526+0000","last_undegraded":"2024-05-08T18:08:32.883526+0000","last_fullsized":"2024-05-08T18:08:32.883526+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:32.883401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:32.883401+0000","last_clean_scrub_stamp":"2024-05-08T18:08:32.883401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:44.412793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"29'3","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:00.515445+0000","last_change":"2024-05-08T18:09:00.515445+0000","last_active":"2024-05-08T18:09:00.515445+0000","last_peered":"2024-05-08T18:09:00.515445+0000","last_clean":"2024-05-08T18:09:00.515445+0000","last_became_active":"2024-05-08T18:04:53.842899+0000","last_became_peered":"2024-05-08T18:04:53.842899+0000","last_unstale":"2024-05-08T18:09:00.515445+0000","last_undegraded":"2024-05-08T18:09:00.515445+0000","last_fullsized":"2024-05-08T18:09:00.515445+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3","last_scrub_stamp":"2024-05-08T18:09:00.515314+0000","last_deep_scrub":"29'3","last_deep_scrub_stamp":"2024-05-08T18:09:00.515314+0000","last_clean_scrub_stamp":"2024-05-08T18:09:00.515314+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:36:11.806797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"29'8","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:59.474909+0000","last_change":"2024-05-08T18:08:59.474909+0000","last_active":"2024-05-08T18:08:59.474909+0000","last_peered":"2024-05-08T18:08:59.474909+0000","last_clean":"2024-05-08T18:08:59.474909+0000","last_became_active":"2024-05-08T18:04:45.812091+0000","last_became_peered":"2024-05-08T18:04:45.812091+0000","last_unstale":"2024-05-08T18:08:59.474909+0000","last_undegraded":"2024-05-08T18:08:59.474909+0000","last_fullsized":"2024-05-08T18:08:59.474909+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:59.474782+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:59.474782+0000","last_clean_scrub_stamp":"2024-05-08T18:08:59.474782+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:04:57.356263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:43.042058+0000","last_change":"2024-05-08T18:08:43.042058+0000","last_active":"2024-05-08T18:08:43.042058+0000","last_peered":"2024-05-08T18:08:43.042058+0000","last_clean":"2024-05-08T18:08:43.042058+0000","last_became_active":"2024-05-08T18:04:35.551313+0000","last_became_peered":"2024-05-08T18:04:35.551313+0000","last_unstale":"2024-05-08T18:08:43.042058+0000","last_undegraded":"2024-05-08T18:08:43.042058+0000","last_fullsized":"2024-05-08T18:08:43.042058+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:43.041942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:43.041942+0000","last_clean_scrub_stamp":"2024-05-08T18:08:43.041942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:53.026181+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:56.472181+0000","last_change":"2024-05-08T18:08:56.472209+0000","last_active":"2024-05-08T18:08:56.472181+0000","last_peered":"2024-05-08T18:08:56.472181+0000","last_clean":"2024-05-08T18:08:56.472181+0000","last_became_active":"2024-05-08T18:04:43.820335+0000","last_became_peered":"2024-05-08T18:04:43.820335+0000","last_unstale":"2024-05-08T18:08:56.472181+0000","last_undegraded":"2024-05-08T18:08:56.472181+0000","last_fullsized":"2024-05-08T18:08:56.472181+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:56.472080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:56.472080+0000","last_clean_scrub_stamp":"2024-05-08T18:08:56.472080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:36:07.763563+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"29'2","reported_seq":41,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.343043+0000","last_change":"2024-05-08T18:08:20.343043+0000","last_active":"2024-05-08T18:08:20.343043+0000","last_peered":"2024-05-08T18:08:20.343043+0000","last_clean":"2024-05-08T18:08:20.343043+0000","last_became_active":"2024-05-08T18:04:47.823631+0000","last_became_peered":"2024-05-08T18:04:47.823631+0000","last_unstale":"2024-05-08T18:08:20.343043+0000","last_undegraded":"2024-05-08T18:08:20.343043+0000","last_fullsized":"2024-05-08T18:08:20.343043+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:20.342916+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:20.342916+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.342916+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:26:02.075842+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"29'24","reported_seq":102,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.233395+0000","last_change":"2024-05-08T18:08:15.233395+0000","last_active":"2024-05-08T18:08:15.233395+0000","last_peered":"2024-05-08T18:08:15.233395+0000","last_clean":"2024-05-08T18:08:15.233395+0000","last_became_active":"2024-05-08T18:04:49.827735+0000","last_became_peered":"2024-05-08T18:04:49.827735+0000","last_unstale":"2024-05-08T18:08:15.233395+0000","last_undegraded":"2024-05-08T18:08:15.233395+0000","last_fullsized":"2024-05-08T18:08:15.233395+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'24","last_scrub_stamp":"2024-05-08T18:08:15.233271+0000","last_deep_scrub":"29'24","last_deep_scrub_stamp":"2024-05-08T18:08:15.233271+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.233271+0000","objects_scrubbed":8,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:25.767355+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.442677+0000","last_change":"2024-05-08T18:08:09.442677+0000","last_active":"2024-05-08T18:08:09.442677+0000","last_peered":"2024-05-08T18:08:09.442677+0000","last_clean":"2024-05-08T18:08:09.442677+0000","last_became_active":"2024-05-08T18:04:51.843940+0000","last_became_peered":"2024-05-08T18:04:51.843940+0000","last_unstale":"2024-05-08T18:08:09.442677+0000","last_undegraded":"2024-05-08T18:08:09.442677+0000","last_fullsized":"2024-05-08T18:08:09.442677+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:09.442556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:09.442556+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.442556+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:48:00.984508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"29'4","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:25.064236+0000","last_change":"2024-05-08T18:09:25.064262+0000","last_active":"2024-05-08T18:09:25.064236+0000","last_peered":"2024-05-08T18:09:25.064236+0000","last_clean":"2024-05-08T18:09:25.064236+0000","last_became_active":"2024-05-08T18:04:53.848255+0000","last_became_peered":"2024-05-08T18:04:53.848255+0000","last_unstale":"2024-05-08T18:09:25.064236+0000","last_undegraded":"2024-05-08T18:09:25.064236+0000","last_fullsized":"2024-05-08T18:09:25.064236+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:09:25.064149+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:09:25.064149+0000","last_clean_scrub_stamp":"2024-05-08T18:09:25.064149+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:57:03.417390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"29'10","reported_seq":62,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:39.382370+0000","last_change":"2024-05-08T18:08:39.382370+0000","last_active":"2024-05-08T18:08:39.382370+0000","last_peered":"2024-05-08T18:08:39.382370+0000","last_clean":"2024-05-08T18:08:39.382370+0000","last_became_active":"2024-05-08T18:04:45.807085+0000","last_became_peered":"2024-05-08T18:04:45.807085+0000","last_unstale":"2024-05-08T18:08:39.382370+0000","last_undegraded":"2024-05-08T18:08:39.382370+0000","last_fullsized":"2024-05-08T18:08:39.382370+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:39.382233+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:39.382233+0000","last_clean_scrub_stamp":"2024-05-08T18:08:39.382233+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:03.089676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":13,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:16.003737+0000","last_change":"2024-05-08T18:09:16.003761+0000","last_active":"2024-05-08T18:09:16.003737+0000","last_peered":"2024-05-08T18:09:16.003737+0000","last_clean":"2024-05-08T18:09:16.003737+0000","last_became_active":"2024-05-08T18:04:35.551120+0000","last_became_peered":"2024-05-08T18:04:35.551120+0000","last_unstale":"2024-05-08T18:09:16.003737+0000","last_undegraded":"2024-05-08T18:09:16.003737+0000","last_fullsized":"2024-05-08T18:09:16.003737+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:16.003623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:16.003623+0000","last_clean_scrub_stamp":"2024-05-08T18:09:16.003623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:42:17.695869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:48.977118+0000","last_change":"2024-05-08T18:08:48.977118+0000","last_active":"2024-05-08T18:08:48.977118+0000","last_peered":"2024-05-08T18:08:48.977118+0000","last_clean":"2024-05-08T18:08:48.977118+0000","last_became_active":"2024-05-08T18:04:43.840036+0000","last_became_peered":"2024-05-08T18:04:43.840036+0000","last_unstale":"2024-05-08T18:08:48.977118+0000","last_undegraded":"2024-05-08T18:08:48.977118+0000","last_fullsized":"2024-05-08T18:08:48.977118+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:48.976992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:48.976992+0000","last_clean_scrub_stamp":"2024-05-08T18:08:48.976992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:39:35.647911+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0,3],"acting":[2,1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.607246+0000","last_change":"2024-05-08T18:08:20.607246+0000","last_active":"2024-05-08T18:08:20.607246+0000","last_peered":"2024-05-08T18:08:20.607246+0000","last_clean":"2024-05-08T18:08:20.607246+0000","last_became_active":"2024-05-08T18:04:47.823093+0000","last_became_peered":"2024-05-08T18:04:47.823093+0000","last_unstale":"2024-05-08T18:08:20.607246+0000","last_undegraded":"2024-05-08T18:08:20.607246+0000","last_fullsized":"2024-05-08T18:08:20.607246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:20.607136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:20.607136+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.607136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:04:01.289711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"29'20","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:24.842630+0000","last_change":"2024-05-08T18:08:24.842630+0000","last_active":"2024-05-08T18:08:24.842630+0000","last_peered":"2024-05-08T18:08:24.842630+0000","last_clean":"2024-05-08T18:08:24.842630+0000","last_became_active":"2024-05-08T18:04:49.830586+0000","last_became_peered":"2024-05-08T18:04:49.830586+0000","last_unstale":"2024-05-08T18:08:24.842630+0000","last_undegraded":"2024-05-08T18:08:24.842630+0000","last_fullsized":"2024-05-08T18:08:24.842630+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'20","last_scrub_stamp":"2024-05-08T18:08:24.842450+0000","last_deep_scrub":"29'20","last_deep_scrub_stamp":"2024-05-08T18:08:24.842450+0000","last_clean_scrub_stamp":"2024-05-08T18:08:24.842450+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:36:35.285600+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":394,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:37.005716+0000","last_change":"2024-05-08T18:08:37.005716+0000","last_active":"2024-05-08T18:08:37.005716+0000","last_peered":"2024-05-08T18:08:37.005716+0000","last_clean":"2024-05-08T18:08:37.005716+0000","last_became_active":"2024-05-08T18:04:51.849137+0000","last_became_peered":"2024-05-08T18:04:51.849137+0000","last_unstale":"2024-05-08T18:08:37.005716+0000","last_undegraded":"2024-05-08T18:08:37.005716+0000","last_fullsized":"2024-05-08T18:08:37.005716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:37.005585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:37.005585+0000","last_clean_scrub_stamp":"2024-05-08T18:08:37.005585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:35:48.297068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"29'4","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:04.571470+0000","last_change":"2024-05-08T18:09:04.571470+0000","last_active":"2024-05-08T18:09:04.571470+0000","last_peered":"2024-05-08T18:09:04.571470+0000","last_clean":"2024-05-08T18:09:04.571470+0000","last_became_active":"2024-05-08T18:04:53.851008+0000","last_became_peered":"2024-05-08T18:04:53.851008+0000","last_unstale":"2024-05-08T18:09:04.571470+0000","last_undegraded":"2024-05-08T18:09:04.571470+0000","last_fullsized":"2024-05-08T18:09:04.571470+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:09:04.571331+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:09:04.571331+0000","last_clean_scrub_stamp":"2024-05-08T18:09:04.571331+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:35:38.826238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'2","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:27.488724+0000","last_change":"2024-05-08T18:08:27.488724+0000","last_active":"2024-05-08T18:08:27.488724+0000","last_peered":"2024-05-08T18:08:27.488724+0000","last_clean":"2024-05-08T18:08:27.488724+0000","last_became_active":"2024-05-08T18:04:45.819326+0000","last_became_peered":"2024-05-08T18:04:45.819326+0000","last_unstale":"2024-05-08T18:08:27.488724+0000","last_undegraded":"2024-05-08T18:08:27.488724+0000","last_fullsized":"2024-05-08T18:08:27.488724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:27.488574+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:27.488574+0000","last_clean_scrub_stamp":"2024-05-08T18:08:27.488574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:06.323416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:49.957730+0000","last_change":"2024-05-08T18:08:49.957730+0000","last_active":"2024-05-08T18:08:49.957730+0000","last_peered":"2024-05-08T18:08:49.957730+0000","last_clean":"2024-05-08T18:08:49.957730+0000","last_became_active":"2024-05-08T18:04:43.840236+0000","last_became_peered":"2024-05-08T18:04:43.840236+0000","last_unstale":"2024-05-08T18:08:49.957730+0000","last_undegraded":"2024-05-08T18:08:49.957730+0000","last_fullsized":"2024-05-08T18:08:49.957730+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:49.957583+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:49.957583+0000","last_clean_scrub_stamp":"2024-05-08T18:08:49.957583+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:33:10.553789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,3],"acting":[2,0,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:52.369564+0000","last_change":"2024-05-08T18:08:52.369589+0000","last_active":"2024-05-08T18:08:52.369564+0000","last_peered":"2024-05-08T18:08:52.369564+0000","last_clean":"2024-05-08T18:08:52.369564+0000","last_became_active":"2024-05-08T18:04:35.552173+0000","last_became_peered":"2024-05-08T18:04:35.552173+0000","last_unstale":"2024-05-08T18:08:52.369564+0000","last_undegraded":"2024-05-08T18:08:52.369564+0000","last_fullsized":"2024-05-08T18:08:52.369564+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:52.369478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:52.369478+0000","last_clean_scrub_stamp":"2024-05-08T18:08:52.369478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:03.898871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"29'2","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:24.599665+0000","last_change":"2024-05-08T18:08:24.599665+0000","last_active":"2024-05-08T18:08:24.599665+0000","last_peered":"2024-05-08T18:08:24.599665+0000","last_clean":"2024-05-08T18:08:24.599665+0000","last_became_active":"2024-05-08T18:04:47.819910+0000","last_became_peered":"2024-05-08T18:04:47.819910+0000","last_unstale":"2024-05-08T18:08:24.599665+0000","last_undegraded":"2024-05-08T18:08:24.599665+0000","last_fullsized":"2024-05-08T18:08:24.599665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:24.599549+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:24.599549+0000","last_clean_scrub_stamp":"2024-05-08T18:08:24.599549+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:03.434391+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"29'73","reported_seq":150,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.598754+0000","last_change":"2024-05-08T18:08:15.598754+0000","last_active":"2024-05-08T18:08:15.598754+0000","last_peered":"2024-05-08T18:08:15.598754+0000","last_clean":"2024-05-08T18:08:15.598754+0000","last_became_active":"2024-05-08T18:04:49.831360+0000","last_became_peered":"2024-05-08T18:04:49.831360+0000","last_unstale":"2024-05-08T18:08:15.598754+0000","last_undegraded":"2024-05-08T18:08:15.598754+0000","last_fullsized":"2024-05-08T18:08:15.598754+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'73","last_scrub_stamp":"2024-05-08T18:08:15.598616+0000","last_deep_scrub":"29'73","last_deep_scrub_stamp":"2024-05-08T18:08:15.598616+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.598616+0000","objects_scrubbed":9,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:23:01.195330+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":71,"num_read_kb":59,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":336,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.250201+0000","last_change":"2024-05-08T18:08:13.250201+0000","last_active":"2024-05-08T18:08:13.250201+0000","last_peered":"2024-05-08T18:08:13.250201+0000","last_clean":"2024-05-08T18:08:13.250201+0000","last_became_active":"2024-05-08T18:04:51.836441+0000","last_became_peered":"2024-05-08T18:04:51.836441+0000","last_unstale":"2024-05-08T18:08:13.250201+0000","last_undegraded":"2024-05-08T18:08:13.250201+0000","last_fullsized":"2024-05-08T18:08:13.250201+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:13.250078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:13.250078+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.250078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:10:13.374535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"29'9","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:29.474491+0000","last_change":"2024-05-08T18:08:29.474491+0000","last_active":"2024-05-08T18:08:29.474491+0000","last_peered":"2024-05-08T18:08:29.474491+0000","last_clean":"2024-05-08T18:08:29.474491+0000","last_became_active":"2024-05-08T18:04:53.837920+0000","last_became_peered":"2024-05-08T18:04:53.837920+0000","last_unstale":"2024-05-08T18:08:29.474491+0000","last_undegraded":"2024-05-08T18:08:29.474491+0000","last_fullsized":"2024-05-08T18:08:29.474491+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'9","last_scrub_stamp":"2024-05-08T18:08:29.474349+0000","last_deep_scrub":"29'9","last_deep_scrub_stamp":"2024-05-08T18:08:29.474349+0000","last_clean_scrub_stamp":"2024-05-08T18:08:29.474349+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:17.638910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":21,"num_write":15,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"29'16","reported_seq":88,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:24.470006+0000","last_change":"2024-05-08T18:08:24.470006+0000","last_active":"2024-05-08T18:08:24.470006+0000","last_peered":"2024-05-08T18:08:24.470006+0000","last_clean":"2024-05-08T18:08:24.470006+0000","last_became_active":"2024-05-08T18:04:45.821107+0000","last_became_peered":"2024-05-08T18:04:45.821107+0000","last_unstale":"2024-05-08T18:08:24.470006+0000","last_undegraded":"2024-05-08T18:08:24.470006+0000","last_fullsized":"2024-05-08T18:08:24.470006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'16","last_scrub_stamp":"2024-05-08T18:08:24.469872+0000","last_deep_scrub":"29'16","last_deep_scrub_stamp":"2024-05-08T18:08:24.469872+0000","last_clean_scrub_stamp":"2024-05-08T18:08:24.469872+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:04:51.772730+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:57.481987+0000","last_change":"2024-05-08T18:08:57.482019+0000","last_active":"2024-05-08T18:08:57.481987+0000","last_peered":"2024-05-08T18:08:57.481987+0000","last_clean":"2024-05-08T18:08:57.481987+0000","last_became_active":"2024-05-08T18:04:43.827920+0000","last_became_peered":"2024-05-08T18:04:43.827920+0000","last_unstale":"2024-05-08T18:08:57.481987+0000","last_undegraded":"2024-05-08T18:08:57.481987+0000","last_fullsized":"2024-05-08T18:08:57.481987+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:57.481903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:57.481903+0000","last_clean_scrub_stamp":"2024-05-08T18:08:57.481903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:05.904037+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:14.981686+0000","last_change":"2024-05-08T18:09:14.981740+0000","last_active":"2024-05-08T18:09:14.981686+0000","last_peered":"2024-05-08T18:09:14.981686+0000","last_clean":"2024-05-08T18:09:14.981686+0000","last_became_active":"2024-05-08T18:04:35.550912+0000","last_became_peered":"2024-05-08T18:04:35.550912+0000","last_unstale":"2024-05-08T18:09:14.981686+0000","last_undegraded":"2024-05-08T18:09:14.981686+0000","last_fullsized":"2024-05-08T18:09:14.981686+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:14.981584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:14.981584+0000","last_clean_scrub_stamp":"2024-05-08T18:09:14.981584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:53:46.695624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.811076+0000","last_change":"2024-05-08T18:08:15.811076+0000","last_active":"2024-05-08T18:08:15.811076+0000","last_peered":"2024-05-08T18:08:15.811076+0000","last_clean":"2024-05-08T18:08:15.811076+0000","last_became_active":"2024-05-08T18:04:47.813717+0000","last_became_peered":"2024-05-08T18:04:47.813717+0000","last_unstale":"2024-05-08T18:08:15.811076+0000","last_undegraded":"2024-05-08T18:08:15.811076+0000","last_fullsized":"2024-05-08T18:08:15.811076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:15.810959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:15.810959+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.810959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:35.598153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"29'21","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.551971+0000","last_change":"2024-05-08T18:08:06.551971+0000","last_active":"2024-05-08T18:08:06.551971+0000","last_peered":"2024-05-08T18:08:06.551971+0000","last_clean":"2024-05-08T18:08:06.551971+0000","last_became_active":"2024-05-08T18:04:49.835451+0000","last_became_peered":"2024-05-08T18:04:49.835451+0000","last_unstale":"2024-05-08T18:08:06.551971+0000","last_undegraded":"2024-05-08T18:08:06.551971+0000","last_fullsized":"2024-05-08T18:08:06.551971+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'21","last_scrub_stamp":"2024-05-08T18:08:06.551844+0000","last_deep_scrub":"29'21","last_deep_scrub_stamp":"2024-05-08T18:08:06.551844+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.551844+0000","objects_scrubbed":7,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:47:05.040309+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"29'285","reported_seq":382,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:04.536905+0000","last_change":"2024-05-08T18:08:04.536905+0000","last_active":"2024-05-08T18:08:04.536905+0000","last_peered":"2024-05-08T18:08:04.536905+0000","last_clean":"2024-05-08T18:08:04.536905+0000","last_became_active":"2024-05-08T18:04:51.832109+0000","last_became_peered":"2024-05-08T18:04:51.832109+0000","last_unstale":"2024-05-08T18:08:04.536905+0000","last_undegraded":"2024-05-08T18:08:04.536905+0000","last_fullsized":"2024-05-08T18:08:04.536905+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:04.536786+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:04.536786+0000","last_clean_scrub_stamp":"2024-05-08T18:08:04.536786+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:19:42.101932+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:49.039245+0000","last_change":"2024-05-08T18:08:49.039245+0000","last_active":"2024-05-08T18:08:49.039245+0000","last_peered":"2024-05-08T18:08:49.039245+0000","last_clean":"2024-05-08T18:08:49.039245+0000","last_became_active":"2024-05-08T18:04:53.842998+0000","last_became_peered":"2024-05-08T18:04:53.842998+0000","last_unstale":"2024-05-08T18:08:49.039245+0000","last_undegraded":"2024-05-08T18:08:49.039245+0000","last_fullsized":"2024-05-08T18:08:49.039245+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:49.039104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:49.039104+0000","last_clean_scrub_stamp":"2024-05-08T18:08:49.039104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:33.981256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"29'4","reported_seq":73,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.838524+0000","last_change":"2024-05-08T18:08:19.838524+0000","last_active":"2024-05-08T18:08:19.838524+0000","last_peered":"2024-05-08T18:08:19.838524+0000","last_clean":"2024-05-08T18:08:19.838524+0000","last_became_active":"2024-05-08T18:04:45.829257+0000","last_became_peered":"2024-05-08T18:04:45.829257+0000","last_unstale":"2024-05-08T18:08:19.838524+0000","last_undegraded":"2024-05-08T18:08:19.838524+0000","last_fullsized":"2024-05-08T18:08:19.838524+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:19.838313+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:19.838313+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.838313+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:38:43.545756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"29'3","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:07.939354+0000","last_change":"2024-05-08T18:09:07.939382+0000","last_active":"2024-05-08T18:09:07.939354+0000","last_peered":"2024-05-08T18:09:07.939354+0000","last_clean":"2024-05-08T18:09:07.939354+0000","last_became_active":"2024-05-08T18:04:43.834610+0000","last_became_peered":"2024-05-08T18:04:43.834610+0000","last_unstale":"2024-05-08T18:09:07.939354+0000","last_undegraded":"2024-05-08T18:09:07.939354+0000","last_fullsized":"2024-05-08T18:09:07.939354+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3","last_scrub_stamp":"2024-05-08T18:09:07.939248+0000","last_deep_scrub":"29'3","last_deep_scrub_stamp":"2024-05-08T18:09:07.939248+0000","last_clean_scrub_stamp":"2024-05-08T18:09:07.939248+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:52:39.228713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,2],"acting":[3,0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:51.327283+0000","last_change":"2024-05-08T18:08:51.327283+0000","last_active":"2024-05-08T18:08:51.327283+0000","last_peered":"2024-05-08T18:08:51.327283+0000","last_clean":"2024-05-08T18:08:51.327283+0000","last_became_active":"2024-05-08T18:04:35.551366+0000","last_became_peered":"2024-05-08T18:04:35.551366+0000","last_unstale":"2024-05-08T18:08:51.327283+0000","last_undegraded":"2024-05-08T18:08:51.327283+0000","last_fullsized":"2024-05-08T18:08:51.327283+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:51.327154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:51.327154+0000","last_clean_scrub_stamp":"2024-05-08T18:08:51.327154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:17:16.185628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.923265+0000","last_change":"2024-05-08T18:08:06.923265+0000","last_active":"2024-05-08T18:08:06.923265+0000","last_peered":"2024-05-08T18:08:06.923265+0000","last_clean":"2024-05-08T18:08:06.923265+0000","last_became_active":"2024-05-08T18:04:47.819866+0000","last_became_peered":"2024-05-08T18:04:47.819866+0000","last_unstale":"2024-05-08T18:08:06.923265+0000","last_undegraded":"2024-05-08T18:08:06.923265+0000","last_fullsized":"2024-05-08T18:08:06.923265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:06.923149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:06.923149+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.923149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:30.572799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"29'15","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:35.962385+0000","last_change":"2024-05-08T18:08:35.962385+0000","last_active":"2024-05-08T18:08:35.962385+0000","last_peered":"2024-05-08T18:08:35.962385+0000","last_clean":"2024-05-08T18:08:35.962385+0000","last_became_active":"2024-05-08T18:04:49.830452+0000","last_became_peered":"2024-05-08T18:04:49.830452+0000","last_unstale":"2024-05-08T18:08:35.962385+0000","last_undegraded":"2024-05-08T18:08:35.962385+0000","last_fullsized":"2024-05-08T18:08:35.962385+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:35.962267+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:35.962267+0000","last_clean_scrub_stamp":"2024-05-08T18:08:35.962267+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:45.951593+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"29'285","reported_seq":412,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:06.471786+0000","last_change":"2024-05-08T18:08:06.471786+0000","last_active":"2024-05-08T18:08:06.471786+0000","last_peered":"2024-05-08T18:08:06.471786+0000","last_clean":"2024-05-08T18:08:06.471786+0000","last_became_active":"2024-05-08T18:04:51.831155+0000","last_became_peered":"2024-05-08T18:04:51.831155+0000","last_unstale":"2024-05-08T18:08:06.471786+0000","last_undegraded":"2024-05-08T18:08:06.471786+0000","last_fullsized":"2024-05-08T18:08:06.471786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:06.471664+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:06.471664+0000","last_clean_scrub_stamp":"2024-05-08T18:08:06.471664+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:29:09.321348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"29'5","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:01.531628+0000","last_change":"2024-05-08T18:09:01.531628+0000","last_active":"2024-05-08T18:09:01.531628+0000","last_peered":"2024-05-08T18:09:01.531628+0000","last_clean":"2024-05-08T18:09:01.531628+0000","last_became_active":"2024-05-08T18:04:53.848862+0000","last_became_peered":"2024-05-08T18:04:53.848862+0000","last_unstale":"2024-05-08T18:09:01.531628+0000","last_undegraded":"2024-05-08T18:09:01.531628+0000","last_fullsized":"2024-05-08T18:09:01.531628+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'5","last_scrub_stamp":"2024-05-08T18:09:01.531514+0000","last_deep_scrub":"29'5","last_deep_scrub_stamp":"2024-05-08T18:09:01.531514+0000","last_clean_scrub_stamp":"2024-05-08T18:09:01.531514+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:09.953648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"29'6","reported_seq":61,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:41.133464+0000","last_change":"2024-05-08T18:08:41.133464+0000","last_active":"2024-05-08T18:08:41.133464+0000","last_peered":"2024-05-08T18:08:41.133464+0000","last_clean":"2024-05-08T18:08:41.133464+0000","last_became_active":"2024-05-08T18:04:45.822824+0000","last_became_peered":"2024-05-08T18:04:45.822824+0000","last_unstale":"2024-05-08T18:08:41.133464+0000","last_undegraded":"2024-05-08T18:08:41.133464+0000","last_fullsized":"2024-05-08T18:08:41.133464+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:41.133339+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:41.133339+0000","last_clean_scrub_stamp":"2024-05-08T18:08:41.133339+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:42:17.740267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"27'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:44.501238+0000","last_change":"2024-05-08T18:08:44.501238+0000","last_active":"2024-05-08T18:08:44.501238+0000","last_peered":"2024-05-08T18:08:44.501238+0000","last_clean":"2024-05-08T18:08:44.501238+0000","last_became_active":"2024-05-08T18:04:43.839240+0000","last_became_peered":"2024-05-08T18:04:43.839240+0000","last_unstale":"2024-05-08T18:08:44.501238+0000","last_undegraded":"2024-05-08T18:08:44.501238+0000","last_fullsized":"2024-05-08T18:08:44.501238+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:44.501120+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:44.501120+0000","last_clean_scrub_stamp":"2024-05-08T18:08:44.501120+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:27.037641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,0],"acting":[1,2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:16.972932+0000","last_change":"2024-05-08T18:09:16.972955+0000","last_active":"2024-05-08T18:09:16.972932+0000","last_peered":"2024-05-08T18:09:16.972932+0000","last_clean":"2024-05-08T18:09:16.972932+0000","last_became_active":"2024-05-08T18:04:35.551412+0000","last_became_peered":"2024-05-08T18:04:35.551412+0000","last_unstale":"2024-05-08T18:09:16.972932+0000","last_undegraded":"2024-05-08T18:09:16.972932+0000","last_fullsized":"2024-05-08T18:09:16.972932+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:16.972830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:16.972830+0000","last_clean_scrub_stamp":"2024-05-08T18:09:16.972830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:02:49.510804+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:13.843881+0000","last_change":"2024-05-08T18:08:13.843881+0000","last_active":"2024-05-08T18:08:13.843881+0000","last_peered":"2024-05-08T18:08:13.843881+0000","last_clean":"2024-05-08T18:08:13.843881+0000","last_became_active":"2024-05-08T18:04:47.815947+0000","last_became_peered":"2024-05-08T18:04:47.815947+0000","last_unstale":"2024-05-08T18:08:13.843881+0000","last_undegraded":"2024-05-08T18:08:13.843881+0000","last_fullsized":"2024-05-08T18:08:13.843881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:13.843666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:13.843666+0000","last_clean_scrub_stamp":"2024-05-08T18:08:13.843666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:18:56.027223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"29'96","reported_seq":190,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.153072+0000","last_change":"2024-05-08T18:08:10.153072+0000","last_active":"2024-05-08T18:08:10.153072+0000","last_peered":"2024-05-08T18:08:10.153072+0000","last_clean":"2024-05-08T18:08:10.153072+0000","last_became_active":"2024-05-08T18:04:49.834586+0000","last_became_peered":"2024-05-08T18:04:49.834586+0000","last_unstale":"2024-05-08T18:08:10.153072+0000","last_undegraded":"2024-05-08T18:08:10.153072+0000","last_fullsized":"2024-05-08T18:08:10.153072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'96","last_scrub_stamp":"2024-05-08T18:08:10.152888+0000","last_deep_scrub":"29'96","last_deep_scrub_stamp":"2024-05-08T18:08:10.152888+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.152888+0000","objects_scrubbed":8,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:22:16.511455+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":93,"num_read_kb":79,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.455904+0000","last_change":"2024-05-08T18:08:08.455904+0000","last_active":"2024-05-08T18:08:08.455904+0000","last_peered":"2024-05-08T18:08:08.455904+0000","last_clean":"2024-05-08T18:08:08.455904+0000","last_became_active":"2024-05-08T18:04:51.830760+0000","last_became_peered":"2024-05-08T18:04:51.830760+0000","last_unstale":"2024-05-08T18:08:08.455904+0000","last_undegraded":"2024-05-08T18:08:08.455904+0000","last_fullsized":"2024-05-08T18:08:08.455904+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:08.455773+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:08.455773+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.455773+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:11:12.703852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.342717+0000","last_change":"2024-05-08T18:08:21.342717+0000","last_active":"2024-05-08T18:08:21.342717+0000","last_peered":"2024-05-08T18:08:21.342717+0000","last_clean":"2024-05-08T18:08:21.342717+0000","last_became_active":"2024-05-08T18:04:53.838600+0000","last_became_peered":"2024-05-08T18:04:53.838600+0000","last_unstale":"2024-05-08T18:08:21.342717+0000","last_undegraded":"2024-05-08T18:08:21.342717+0000","last_fullsized":"2024-05-08T18:08:21.342717+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:21.342590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:21.342590+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.342590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:30:44.919676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"27'2","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:51.909643+0000","last_change":"2024-05-08T18:08:51.909643+0000","last_active":"2024-05-08T18:08:51.909643+0000","last_peered":"2024-05-08T18:08:51.909643+0000","last_clean":"2024-05-08T18:08:51.909643+0000","last_became_active":"2024-05-08T18:04:53.839500+0000","last_became_peered":"2024-05-08T18:04:53.839500+0000","last_unstale":"2024-05-08T18:08:51.909643+0000","last_undegraded":"2024-05-08T18:08:51.909643+0000","last_fullsized":"2024-05-08T18:08:51.909643+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:51.909525+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:51.909525+0000","last_clean_scrub_stamp":"2024-05-08T18:08:51.909525+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:05:28.466923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"29'8","reported_seq":63,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:39.209919+0000","last_change":"2024-05-08T18:08:39.209919+0000","last_active":"2024-05-08T18:08:39.209919+0000","last_peered":"2024-05-08T18:08:39.209919+0000","last_clean":"2024-05-08T18:08:39.209919+0000","last_became_active":"2024-05-08T18:04:45.819508+0000","last_became_peered":"2024-05-08T18:04:45.819508+0000","last_unstale":"2024-05-08T18:08:39.209919+0000","last_undegraded":"2024-05-08T18:08:39.209919+0000","last_fullsized":"2024-05-08T18:08:39.209919+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:39.209750+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:39.209750+0000","last_clean_scrub_stamp":"2024-05-08T18:08:39.209750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:26:48.744139+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:06.972658+0000","last_change":"2024-05-08T18:09:06.972658+0000","last_active":"2024-05-08T18:09:06.972658+0000","last_peered":"2024-05-08T18:09:06.972658+0000","last_clean":"2024-05-08T18:09:06.972658+0000","last_became_active":"2024-05-08T18:04:43.837929+0000","last_became_peered":"2024-05-08T18:04:43.837929+0000","last_unstale":"2024-05-08T18:09:06.972658+0000","last_undegraded":"2024-05-08T18:09:06.972658+0000","last_fullsized":"2024-05-08T18:09:06.972658+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:06.972472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:06.972472+0000","last_clean_scrub_stamp":"2024-05-08T18:09:06.972472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:00:23.965126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.504467+0000","last_change":"2024-05-08T18:08:19.504467+0000","last_active":"2024-05-08T18:08:19.504467+0000","last_peered":"2024-05-08T18:08:19.504467+0000","last_clean":"2024-05-08T18:08:19.504467+0000","last_became_active":"2024-05-08T18:04:47.818443+0000","last_became_peered":"2024-05-08T18:04:47.818443+0000","last_unstale":"2024-05-08T18:08:19.504467+0000","last_undegraded":"2024-05-08T18:08:19.504467+0000","last_fullsized":"2024-05-08T18:08:19.504467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:19.504350+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:19.504350+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.504350+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:24.197683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"29'40","reported_seq":97,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:30.848380+0000","last_change":"2024-05-08T18:08:30.848380+0000","last_active":"2024-05-08T18:08:30.848380+0000","last_peered":"2024-05-08T18:08:30.848380+0000","last_clean":"2024-05-08T18:08:30.848380+0000","last_became_active":"2024-05-08T18:04:49.833599+0000","last_became_peered":"2024-05-08T18:04:49.833599+0000","last_unstale":"2024-05-08T18:08:30.848380+0000","last_undegraded":"2024-05-08T18:08:30.848380+0000","last_fullsized":"2024-05-08T18:08:30.848380+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'40","last_scrub_stamp":"2024-05-08T18:08:30.848207+0000","last_deep_scrub":"29'40","last_deep_scrub_stamp":"2024-05-08T18:08:30.848207+0000","last_clean_scrub_stamp":"2024-05-08T18:08:30.848207+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:31:32.685411+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:26.837221+0000","last_change":"2024-05-08T18:08:26.837221+0000","last_active":"2024-05-08T18:08:26.837221+0000","last_peered":"2024-05-08T18:08:26.837221+0000","last_clean":"2024-05-08T18:08:26.837221+0000","last_became_active":"2024-05-08T18:04:51.844526+0000","last_became_peered":"2024-05-08T18:04:51.844526+0000","last_unstale":"2024-05-08T18:08:26.837221+0000","last_undegraded":"2024-05-08T18:08:26.837221+0000","last_fullsized":"2024-05-08T18:08:26.837221+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:26.837073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:26.837073+0000","last_clean_scrub_stamp":"2024-05-08T18:08:26.837073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:53:03.923893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"29'11","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:05.534334+0000","last_change":"2024-05-08T18:09:05.534359+0000","last_active":"2024-05-08T18:09:05.534334+0000","last_peered":"2024-05-08T18:09:05.534334+0000","last_clean":"2024-05-08T18:09:05.534334+0000","last_became_active":"2024-05-08T18:04:53.850593+0000","last_became_peered":"2024-05-08T18:04:53.850593+0000","last_unstale":"2024-05-08T18:09:05.534334+0000","last_undegraded":"2024-05-08T18:09:05.534334+0000","last_fullsized":"2024-05-08T18:09:05.534334+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'11","last_scrub_stamp":"2024-05-08T18:09:05.534221+0000","last_deep_scrub":"29'11","last_deep_scrub_stamp":"2024-05-08T18:09:05.534221+0000","last_clean_scrub_stamp":"2024-05-08T18:09:05.534221+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:32:51.310647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":62,"num_write":31,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"29'8","reported_seq":83,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:31.489126+0000","last_change":"2024-05-08T18:08:31.489126+0000","last_active":"2024-05-08T18:08:31.489126+0000","last_peered":"2024-05-08T18:08:31.489126+0000","last_clean":"2024-05-08T18:08:31.489126+0000","last_became_active":"2024-05-08T18:04:45.818979+0000","last_became_peered":"2024-05-08T18:04:45.818979+0000","last_unstale":"2024-05-08T18:08:31.489126+0000","last_undegraded":"2024-05-08T18:08:31.489126+0000","last_fullsized":"2024-05-08T18:08:31.489126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:31.488942+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:31.488942+0000","last_clean_scrub_stamp":"2024-05-08T18:08:31.488942+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:34.056789+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:46.971417+0000","last_change":"2024-05-08T18:08:46.971417+0000","last_active":"2024-05-08T18:08:46.971417+0000","last_peered":"2024-05-08T18:08:46.971417+0000","last_clean":"2024-05-08T18:08:46.971417+0000","last_became_active":"2024-05-08T18:04:43.836614+0000","last_became_peered":"2024-05-08T18:04:43.836614+0000","last_unstale":"2024-05-08T18:08:46.971417+0000","last_undegraded":"2024-05-08T18:08:46.971417+0000","last_fullsized":"2024-05-08T18:08:46.971417+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:46.971249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:46.971249+0000","last_clean_scrub_stamp":"2024-05-08T18:08:46.971249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:02:46.938192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.821740+0000","last_change":"2024-05-08T18:08:14.821740+0000","last_active":"2024-05-08T18:08:14.821740+0000","last_peered":"2024-05-08T18:08:14.821740+0000","last_clean":"2024-05-08T18:08:14.821740+0000","last_became_active":"2024-05-08T18:04:47.814561+0000","last_became_peered":"2024-05-08T18:04:47.814561+0000","last_unstale":"2024-05-08T18:08:14.821740+0000","last_undegraded":"2024-05-08T18:08:14.821740+0000","last_fullsized":"2024-05-08T18:08:14.821740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:14.821626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:14.821626+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.821626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:20:24.805865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"29'10","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.572575+0000","last_change":"2024-05-08T18:08:18.572575+0000","last_active":"2024-05-08T18:08:18.572575+0000","last_peered":"2024-05-08T18:08:18.572575+0000","last_clean":"2024-05-08T18:08:18.572575+0000","last_became_active":"2024-05-08T18:04:49.825347+0000","last_became_peered":"2024-05-08T18:04:49.825347+0000","last_unstale":"2024-05-08T18:08:18.572575+0000","last_undegraded":"2024-05-08T18:08:18.572575+0000","last_fullsized":"2024-05-08T18:08:18.572575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:18.572458+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:18.572458+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.572458+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:10:21.293654+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":294,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.319068+0000","last_change":"2024-05-08T18:08:17.319068+0000","last_active":"2024-05-08T18:08:17.319068+0000","last_peered":"2024-05-08T18:08:17.319068+0000","last_clean":"2024-05-08T18:08:17.319068+0000","last_became_active":"2024-05-08T18:04:51.843002+0000","last_became_peered":"2024-05-08T18:04:51.843002+0000","last_unstale":"2024-05-08T18:08:17.319068+0000","last_undegraded":"2024-05-08T18:08:17.319068+0000","last_fullsized":"2024-05-08T18:08:17.319068+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:17.318901+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:17.318901+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.318901+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:53:07.004932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:00.468674+0000","last_change":"2024-05-08T18:09:00.468698+0000","last_active":"2024-05-08T18:09:00.468674+0000","last_peered":"2024-05-08T18:09:00.468674+0000","last_clean":"2024-05-08T18:09:00.468674+0000","last_became_active":"2024-05-08T18:04:53.840437+0000","last_became_peered":"2024-05-08T18:04:53.840437+0000","last_unstale":"2024-05-08T18:09:00.468674+0000","last_undegraded":"2024-05-08T18:09:00.468674+0000","last_fullsized":"2024-05-08T18:09:00.468674+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:00.468582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:00.468582+0000","last_clean_scrub_stamp":"2024-05-08T18:09:00.468582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:35:34.723489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"29'8","reported_seq":68,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:06.547181+0000","last_change":"2024-05-08T18:09:06.547181+0000","last_active":"2024-05-08T18:09:06.547181+0000","last_peered":"2024-05-08T18:09:06.547181+0000","last_clean":"2024-05-08T18:09:06.547181+0000","last_became_active":"2024-05-08T18:05:30.868156+0000","last_became_peered":"2024-05-08T18:05:30.868156+0000","last_unstale":"2024-05-08T18:09:06.547181+0000","last_undegraded":"2024-05-08T18:09:06.547181+0000","last_fullsized":"2024-05-08T18:09:06.547181+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:09:06.547085+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:09:06.547085+0000","last_clean_scrub_stamp":"2024-05-08T18:09:06.547085+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:46:26.349909+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:45.035600+0000","last_change":"2024-05-08T18:08:45.035600+0000","last_active":"2024-05-08T18:08:45.035600+0000","last_peered":"2024-05-08T18:08:45.035600+0000","last_clean":"2024-05-08T18:08:45.035600+0000","last_became_active":"2024-05-08T18:04:43.835435+0000","last_became_peered":"2024-05-08T18:04:43.835435+0000","last_unstale":"2024-05-08T18:08:45.035600+0000","last_undegraded":"2024-05-08T18:08:45.035600+0000","last_fullsized":"2024-05-08T18:08:45.035600+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:45.035484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:45.035484+0000","last_clean_scrub_stamp":"2024-05-08T18:08:45.035484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:06:21.992378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0,1],"acting":[2,3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.385611+0000","last_change":"2024-05-08T18:08:19.385611+0000","last_active":"2024-05-08T18:08:19.385611+0000","last_peered":"2024-05-08T18:08:19.385611+0000","last_clean":"2024-05-08T18:08:19.385611+0000","last_became_active":"2024-05-08T18:04:47.824140+0000","last_became_peered":"2024-05-08T18:04:47.824140+0000","last_unstale":"2024-05-08T18:08:19.385611+0000","last_undegraded":"2024-05-08T18:08:19.385611+0000","last_fullsized":"2024-05-08T18:08:19.385611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:19.385463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:19.385463+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.385463+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:02:15.297455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"29'48","reported_seq":121,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:19.582659+0000","last_change":"2024-05-08T18:08:19.582659+0000","last_active":"2024-05-08T18:08:19.582659+0000","last_peered":"2024-05-08T18:08:19.582659+0000","last_clean":"2024-05-08T18:08:19.582659+0000","last_became_active":"2024-05-08T18:04:49.824600+0000","last_became_peered":"2024-05-08T18:04:49.824600+0000","last_unstale":"2024-05-08T18:08:19.582659+0000","last_undegraded":"2024-05-08T18:08:19.582659+0000","last_fullsized":"2024-05-08T18:08:19.582659+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'48","last_scrub_stamp":"2024-05-08T18:08:19.582484+0000","last_deep_scrub":"29'48","last_deep_scrub_stamp":"2024-05-08T18:08:19.582484+0000","last_clean_scrub_stamp":"2024-05-08T18:08:19.582484+0000","objects_scrubbed":6,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:25.159113+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":706,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.521130+0000","last_change":"2024-05-08T18:08:09.521130+0000","last_active":"2024-05-08T18:08:09.521130+0000","last_peered":"2024-05-08T18:08:09.521130+0000","last_clean":"2024-05-08T18:08:09.521130+0000","last_became_active":"2024-05-08T18:04:51.829937+0000","last_became_peered":"2024-05-08T18:04:51.829937+0000","last_unstale":"2024-05-08T18:08:09.521130+0000","last_undegraded":"2024-05-08T18:08:09.521130+0000","last_fullsized":"2024-05-08T18:08:09.521130+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:09.521013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:09.521013+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.521013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:29:12.370696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":26,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:45.103397+0000","last_change":"2024-05-08T18:08:45.103421+0000","last_active":"2024-05-08T18:08:45.103397+0000","last_peered":"2024-05-08T18:08:45.103397+0000","last_clean":"2024-05-08T18:08:45.103397+0000","last_became_active":"2024-05-08T18:04:53.842610+0000","last_became_peered":"2024-05-08T18:04:53.842610+0000","last_unstale":"2024-05-08T18:08:45.103397+0000","last_undegraded":"2024-05-08T18:08:45.103397+0000","last_fullsized":"2024-05-08T18:08:45.103397+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:45.103296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:45.103296+0000","last_clean_scrub_stamp":"2024-05-08T18:08:45.103296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:25:29.008024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.866966+0000","last_change":"2024-05-08T18:08:22.866966+0000","last_active":"2024-05-08T18:08:22.866966+0000","last_peered":"2024-05-08T18:08:22.866966+0000","last_clean":"2024-05-08T18:08:22.866966+0000","last_became_active":"2024-05-08T18:04:45.829458+0000","last_became_peered":"2024-05-08T18:04:45.829458+0000","last_unstale":"2024-05-08T18:08:22.866966+0000","last_undegraded":"2024-05-08T18:08:22.866966+0000","last_fullsized":"2024-05-08T18:08:22.866966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.866853+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.866853+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.866853+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:05.403374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"29'4","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:56.481530+0000","last_change":"2024-05-08T18:08:56.481530+0000","last_active":"2024-05-08T18:08:56.481530+0000","last_peered":"2024-05-08T18:08:56.481530+0000","last_clean":"2024-05-08T18:08:56.481530+0000","last_became_active":"2024-05-08T18:04:43.836633+0000","last_became_peered":"2024-05-08T18:04:43.836633+0000","last_unstale":"2024-05-08T18:08:56.481530+0000","last_undegraded":"2024-05-08T18:08:56.481530+0000","last_fullsized":"2024-05-08T18:08:56.481530+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:56.481386+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:56.481386+0000","last_clean_scrub_stamp":"2024-05-08T18:08:56.481386+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:29:32.379503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":12,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"29'2","reported_seq":231,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.909590+0000","last_change":"2024-05-08T18:08:07.909590+0000","last_active":"2024-05-08T18:08:07.909590+0000","last_peered":"2024-05-08T18:08:07.909590+0000","last_clean":"2024-05-08T18:08:07.909590+0000","last_became_active":"2024-05-08T18:04:47.819771+0000","last_became_peered":"2024-05-08T18:04:47.819771+0000","last_unstale":"2024-05-08T18:08:07.909590+0000","last_undegraded":"2024-05-08T18:08:07.909590+0000","last_fullsized":"2024-05-08T18:08:07.909590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:07.909472+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:07.909472+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.909472+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:02:04.798235+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"29'36","reported_seq":92,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.344360+0000","last_change":"2024-05-08T18:08:18.344360+0000","last_active":"2024-05-08T18:08:18.344360+0000","last_peered":"2024-05-08T18:08:18.344360+0000","last_clean":"2024-05-08T18:08:18.344360+0000","last_became_active":"2024-05-08T18:04:49.824022+0000","last_became_peered":"2024-05-08T18:04:49.824022+0000","last_unstale":"2024-05-08T18:08:18.344360+0000","last_undegraded":"2024-05-08T18:08:18.344360+0000","last_fullsized":"2024-05-08T18:08:18.344360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:08:18.344202+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:08:18.344202+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.344202+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:36:47.464601+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:08.198575+0000","last_change":"2024-05-08T18:08:08.198575+0000","last_active":"2024-05-08T18:08:08.198575+0000","last_peered":"2024-05-08T18:08:08.198575+0000","last_clean":"2024-05-08T18:08:08.198575+0000","last_became_active":"2024-05-08T18:04:51.826424+0000","last_became_peered":"2024-05-08T18:04:51.826424+0000","last_unstale":"2024-05-08T18:08:08.198575+0000","last_undegraded":"2024-05-08T18:08:08.198575+0000","last_fullsized":"2024-05-08T18:08:08.198575+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:08.198405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:08.198405+0000","last_clean_scrub_stamp":"2024-05-08T18:08:08.198405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:21:15.614149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"29'20","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:57.043872+0000","last_change":"2024-05-08T18:08:57.043872+0000","last_active":"2024-05-08T18:08:57.043872+0000","last_peered":"2024-05-08T18:08:57.043872+0000","last_clean":"2024-05-08T18:08:57.043872+0000","last_became_active":"2024-05-08T18:05:30.868409+0000","last_became_peered":"2024-05-08T18:05:30.868409+0000","last_unstale":"2024-05-08T18:08:57.043872+0000","last_undegraded":"2024-05-08T18:08:57.043872+0000","last_fullsized":"2024-05-08T18:08:57.043872+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'20","last_scrub_stamp":"2024-05-08T18:08:57.043740+0000","last_deep_scrub":"29'20","last_deep_scrub_stamp":"2024-05-08T18:08:57.043740+0000","last_clean_scrub_stamp":"2024-05-08T18:08:57.043740+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:03.723531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":55,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:08.952757+0000","last_change":"2024-05-08T18:09:08.952757+0000","last_active":"2024-05-08T18:09:08.952757+0000","last_peered":"2024-05-08T18:09:08.952757+0000","last_clean":"2024-05-08T18:09:08.952757+0000","last_became_active":"2024-05-08T18:04:43.838957+0000","last_became_peered":"2024-05-08T18:04:43.838957+0000","last_unstale":"2024-05-08T18:09:08.952757+0000","last_undegraded":"2024-05-08T18:09:08.952757+0000","last_fullsized":"2024-05-08T18:09:08.952757+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:08.952571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:08.952571+0000","last_clean_scrub_stamp":"2024-05-08T18:09:08.952571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:35:03.568080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.606122+0000","last_change":"2024-05-08T18:08:23.606122+0000","last_active":"2024-05-08T18:08:23.606122+0000","last_peered":"2024-05-08T18:08:23.606122+0000","last_clean":"2024-05-08T18:08:23.606122+0000","last_became_active":"2024-05-08T18:04:47.820256+0000","last_became_peered":"2024-05-08T18:04:47.820256+0000","last_unstale":"2024-05-08T18:08:23.606122+0000","last_undegraded":"2024-05-08T18:08:23.606122+0000","last_fullsized":"2024-05-08T18:08:23.606122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:23.606002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:23.606002+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.606002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:04.849646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"29'63","reported_seq":136,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.582374+0000","last_change":"2024-05-08T18:08:16.582414+0000","last_active":"2024-05-08T18:08:16.582374+0000","last_peered":"2024-05-08T18:08:16.582374+0000","last_clean":"2024-05-08T18:08:16.582374+0000","last_became_active":"2024-05-08T18:04:49.825366+0000","last_became_peered":"2024-05-08T18:04:49.825366+0000","last_unstale":"2024-05-08T18:08:16.582374+0000","last_undegraded":"2024-05-08T18:08:16.582374+0000","last_fullsized":"2024-05-08T18:08:16.582374+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'63","last_scrub_stamp":"2024-05-08T18:08:16.582295+0000","last_deep_scrub":"29'63","last_deep_scrub_stamp":"2024-05-08T18:08:16.582295+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.582295+0000","objects_scrubbed":5,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:01:21.275627+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":22,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:07.157727+0000","last_change":"2024-05-08T18:08:07.157775+0000","last_active":"2024-05-08T18:08:07.157727+0000","last_peered":"2024-05-08T18:08:07.157727+0000","last_clean":"2024-05-08T18:08:07.157727+0000","last_became_active":"2024-05-08T18:04:51.827127+0000","last_became_peered":"2024-05-08T18:04:51.827127+0000","last_unstale":"2024-05-08T18:08:07.157727+0000","last_undegraded":"2024-05-08T18:08:07.157727+0000","last_fullsized":"2024-05-08T18:08:07.157727+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:07.157606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:07.157606+0000","last_clean_scrub_stamp":"2024-05-08T18:08:07.157606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:37:53.105588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"29'14","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:03.477873+0000","last_change":"2024-05-08T18:09:03.477873+0000","last_active":"2024-05-08T18:09:03.477873+0000","last_peered":"2024-05-08T18:09:03.477873+0000","last_clean":"2024-05-08T18:09:03.477873+0000","last_became_active":"2024-05-08T18:04:53.843772+0000","last_became_peered":"2024-05-08T18:04:53.843772+0000","last_unstale":"2024-05-08T18:09:03.477873+0000","last_undegraded":"2024-05-08T18:09:03.477873+0000","last_fullsized":"2024-05-08T18:09:03.477873+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'14","last_scrub_stamp":"2024-05-08T18:09:03.477748+0000","last_deep_scrub":"29'14","last_deep_scrub_stamp":"2024-05-08T18:09:03.477748+0000","last_clean_scrub_stamp":"2024-05-08T18:09:03.477748+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:41:23.206546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":16,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"29'2","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:43.351032+0000","last_change":"2024-05-08T18:08:43.351032+0000","last_active":"2024-05-08T18:08:43.351032+0000","last_peered":"2024-05-08T18:08:43.351032+0000","last_clean":"2024-05-08T18:08:43.351032+0000","last_became_active":"2024-05-08T18:04:45.811517+0000","last_became_peered":"2024-05-08T18:04:45.811517+0000","last_unstale":"2024-05-08T18:08:43.351032+0000","last_undegraded":"2024-05-08T18:08:43.351032+0000","last_fullsized":"2024-05-08T18:08:43.351032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:43.350844+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:43.350844+0000","last_clean_scrub_stamp":"2024-05-08T18:08:43.350844+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:06:35.421755+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:54.459559+0000","last_change":"2024-05-08T18:08:54.459559+0000","last_active":"2024-05-08T18:08:54.459559+0000","last_peered":"2024-05-08T18:08:54.459559+0000","last_clean":"2024-05-08T18:08:54.459559+0000","last_became_active":"2024-05-08T18:04:43.834767+0000","last_became_peered":"2024-05-08T18:04:43.834767+0000","last_unstale":"2024-05-08T18:08:54.459559+0000","last_undegraded":"2024-05-08T18:08:54.459559+0000","last_fullsized":"2024-05-08T18:08:54.459559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:54.459439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:54.459439+0000","last_clean_scrub_stamp":"2024-05-08T18:08:54.459439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:10:45.537704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,0,2],"acting":[1,3,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:24.285019+0000","last_change":"2024-05-08T18:08:24.285019+0000","last_active":"2024-05-08T18:08:24.285019+0000","last_peered":"2024-05-08T18:08:24.285019+0000","last_clean":"2024-05-08T18:08:24.285019+0000","last_became_active":"2024-05-08T18:04:47.820662+0000","last_became_peered":"2024-05-08T18:04:47.820662+0000","last_unstale":"2024-05-08T18:08:24.285019+0000","last_undegraded":"2024-05-08T18:08:24.285019+0000","last_fullsized":"2024-05-08T18:08:24.285019+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:24.284899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:24.284899+0000","last_clean_scrub_stamp":"2024-05-08T18:08:24.284899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:02:49.472387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"29'46","reported_seq":104,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:10.901855+0000","last_change":"2024-05-08T18:08:10.901892+0000","last_active":"2024-05-08T18:08:10.901855+0000","last_peered":"2024-05-08T18:08:10.901855+0000","last_clean":"2024-05-08T18:08:10.901855+0000","last_became_active":"2024-05-08T18:04:49.830476+0000","last_became_peered":"2024-05-08T18:04:49.830476+0000","last_unstale":"2024-05-08T18:08:10.901855+0000","last_undegraded":"2024-05-08T18:08:10.901855+0000","last_fullsized":"2024-05-08T18:08:10.901855+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'46","last_scrub_stamp":"2024-05-08T18:08:10.901767+0000","last_deep_scrub":"29'46","last_deep_scrub_stamp":"2024-05-08T18:08:10.901767+0000","last_clean_scrub_stamp":"2024-05-08T18:08:10.901767+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:07.184811+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.490196+0000","last_change":"2024-05-08T18:08:17.490220+0000","last_active":"2024-05-08T18:08:17.490196+0000","last_peered":"2024-05-08T18:08:17.490196+0000","last_clean":"2024-05-08T18:08:17.490196+0000","last_became_active":"2024-05-08T18:04:51.844912+0000","last_became_peered":"2024-05-08T18:04:51.844912+0000","last_unstale":"2024-05-08T18:08:17.490196+0000","last_undegraded":"2024-05-08T18:08:17.490196+0000","last_fullsized":"2024-05-08T18:08:17.490196+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:17.490110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:17.490110+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.490110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:06:17.806415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"29'2","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:24.043989+0000","last_change":"2024-05-08T18:09:24.043989+0000","last_active":"2024-05-08T18:09:24.043989+0000","last_peered":"2024-05-08T18:09:24.043989+0000","last_clean":"2024-05-08T18:09:24.043989+0000","last_became_active":"2024-05-08T18:04:53.847413+0000","last_became_peered":"2024-05-08T18:04:53.847413+0000","last_unstale":"2024-05-08T18:09:24.043989+0000","last_undegraded":"2024-05-08T18:09:24.043989+0000","last_fullsized":"2024-05-08T18:09:24.043989+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:09:24.043852+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:09:24.043852+0000","last_clean_scrub_stamp":"2024-05-08T18:09:24.043852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:51:14.584121+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"29'12","reported_seq":69,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.835658+0000","last_change":"2024-05-08T18:08:17.835711+0000","last_active":"2024-05-08T18:08:17.835658+0000","last_peered":"2024-05-08T18:08:17.835658+0000","last_clean":"2024-05-08T18:08:17.835658+0000","last_became_active":"2024-05-08T18:04:45.830329+0000","last_became_peered":"2024-05-08T18:04:45.830329+0000","last_unstale":"2024-05-08T18:08:17.835658+0000","last_undegraded":"2024-05-08T18:08:17.835658+0000","last_fullsized":"2024-05-08T18:08:17.835658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:17.835540+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:17.835540+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.835540+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:36.036099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:52.463670+0000","last_change":"2024-05-08T18:08:52.463670+0000","last_active":"2024-05-08T18:08:52.463670+0000","last_peered":"2024-05-08T18:08:52.463670+0000","last_clean":"2024-05-08T18:08:52.463670+0000","last_became_active":"2024-05-08T18:04:43.835227+0000","last_became_peered":"2024-05-08T18:04:43.835227+0000","last_unstale":"2024-05-08T18:08:52.463670+0000","last_undegraded":"2024-05-08T18:08:52.463670+0000","last_fullsized":"2024-05-08T18:08:52.463670+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:52.463521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:52.463521+0000","last_clean_scrub_stamp":"2024-05-08T18:08:52.463521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:21:05.463037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3,2],"acting":[1,0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:56.000672+0000","last_change":"2024-05-08T18:08:56.000704+0000","last_active":"2024-05-08T18:08:56.000672+0000","last_peered":"2024-05-08T18:08:56.000672+0000","last_clean":"2024-05-08T18:08:56.000672+0000","last_became_active":"2024-05-08T18:05:30.870155+0000","last_became_peered":"2024-05-08T18:05:30.870155+0000","last_unstale":"2024-05-08T18:08:56.000672+0000","last_undegraded":"2024-05-08T18:08:56.000672+0000","last_fullsized":"2024-05-08T18:08:56.000672+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:56.000581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:56.000581+0000","last_clean_scrub_stamp":"2024-05-08T18:08:56.000581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:19.118117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"29'39","reported_seq":96,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.536541+0000","last_change":"2024-05-08T18:08:12.536541+0000","last_active":"2024-05-08T18:08:12.536541+0000","last_peered":"2024-05-08T18:08:12.536541+0000","last_clean":"2024-05-08T18:08:12.536541+0000","last_became_active":"2024-05-08T18:04:49.832649+0000","last_became_peered":"2024-05-08T18:04:49.832649+0000","last_unstale":"2024-05-08T18:08:12.536541+0000","last_undegraded":"2024-05-08T18:08:12.536541+0000","last_fullsized":"2024-05-08T18:08:12.536541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'39","last_scrub_stamp":"2024-05-08T18:08:12.536412+0000","last_deep_scrub":"29'39","last_deep_scrub_stamp":"2024-05-08T18:08:12.536412+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.536412+0000","objects_scrubbed":5,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:31.365352+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:05.444675+0000","last_change":"2024-05-08T18:08:05.444675+0000","last_active":"2024-05-08T18:08:05.444675+0000","last_peered":"2024-05-08T18:08:05.444675+0000","last_clean":"2024-05-08T18:08:05.444675+0000","last_became_active":"2024-05-08T18:04:51.833421+0000","last_became_peered":"2024-05-08T18:04:51.833421+0000","last_unstale":"2024-05-08T18:08:05.444675+0000","last_undegraded":"2024-05-08T18:08:05.444675+0000","last_fullsized":"2024-05-08T18:08:05.444675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:05.444585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:05.444585+0000","last_clean_scrub_stamp":"2024-05-08T18:08:05.444585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:15.978668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"29'5","reported_seq":33,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:26.027981+0000","last_change":"2024-05-08T18:09:26.027981+0000","last_active":"2024-05-08T18:09:26.027981+0000","last_peered":"2024-05-08T18:09:26.027981+0000","last_clean":"2024-05-08T18:09:26.027981+0000","last_became_active":"2024-05-08T18:04:53.848430+0000","last_became_peered":"2024-05-08T18:04:53.848430+0000","last_unstale":"2024-05-08T18:09:26.027981+0000","last_undegraded":"2024-05-08T18:09:26.027981+0000","last_fullsized":"2024-05-08T18:09:26.027981+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'5","last_scrub_stamp":"2024-05-08T18:09:26.027859+0000","last_deep_scrub":"29'5","last_deep_scrub_stamp":"2024-05-08T18:09:26.027859+0000","last_clean_scrub_stamp":"2024-05-08T18:09:26.027859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:45:35.552904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"29'10","reported_seq":75,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:37.437418+0000","last_change":"2024-05-08T18:08:37.437444+0000","last_active":"2024-05-08T18:08:37.437418+0000","last_peered":"2024-05-08T18:08:37.437418+0000","last_clean":"2024-05-08T18:08:37.437418+0000","last_became_active":"2024-05-08T18:04:45.806808+0000","last_became_peered":"2024-05-08T18:04:45.806808+0000","last_unstale":"2024-05-08T18:08:37.437418+0000","last_undegraded":"2024-05-08T18:08:37.437418+0000","last_fullsized":"2024-05-08T18:08:37.437418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:37.437332+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:37.437332+0000","last_clean_scrub_stamp":"2024-05-08T18:08:37.437332+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:55.637891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:20.985689+0000","last_change":"2024-05-08T18:09:20.985689+0000","last_active":"2024-05-08T18:09:20.985689+0000","last_peered":"2024-05-08T18:09:20.985689+0000","last_clean":"2024-05-08T18:09:20.985689+0000","last_became_active":"2024-05-08T18:04:43.832147+0000","last_became_peered":"2024-05-08T18:04:43.832147+0000","last_unstale":"2024-05-08T18:09:20.985689+0000","last_undegraded":"2024-05-08T18:09:20.985689+0000","last_fullsized":"2024-05-08T18:09:20.985689+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:20.985513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:20.985513+0000","last_clean_scrub_stamp":"2024-05-08T18:09:20.985513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:03:07.156148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:22.517382+0000","last_change":"2024-05-08T18:08:22.517404+0000","last_active":"2024-05-08T18:08:22.517382+0000","last_peered":"2024-05-08T18:08:22.517382+0000","last_clean":"2024-05-08T18:08:22.517382+0000","last_became_active":"2024-05-08T18:04:47.815219+0000","last_became_peered":"2024-05-08T18:04:47.815219+0000","last_unstale":"2024-05-08T18:08:22.517382+0000","last_undegraded":"2024-05-08T18:08:22.517382+0000","last_fullsized":"2024-05-08T18:08:22.517382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:22.517292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:22.517292+0000","last_clean_scrub_stamp":"2024-05-08T18:08:22.517292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:45:28.093920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"29'15","reported_seq":55,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:38.037896+0000","last_change":"2024-05-08T18:08:38.037896+0000","last_active":"2024-05-08T18:08:38.037896+0000","last_peered":"2024-05-08T18:08:38.037896+0000","last_clean":"2024-05-08T18:08:38.037896+0000","last_became_active":"2024-05-08T18:04:49.830793+0000","last_became_peered":"2024-05-08T18:04:49.830793+0000","last_unstale":"2024-05-08T18:08:38.037896+0000","last_undegraded":"2024-05-08T18:08:38.037896+0000","last_fullsized":"2024-05-08T18:08:38.037896+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:38.037761+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:38.037761+0000","last_clean_scrub_stamp":"2024-05-08T18:08:38.037761+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:37:36.379302+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"29'285","reported_seq":379,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:09.160257+0000","last_change":"2024-05-08T18:08:09.160257+0000","last_active":"2024-05-08T18:08:09.160257+0000","last_peered":"2024-05-08T18:08:09.160257+0000","last_clean":"2024-05-08T18:08:09.160257+0000","last_became_active":"2024-05-08T18:04:51.833866+0000","last_became_peered":"2024-05-08T18:04:51.833866+0000","last_unstale":"2024-05-08T18:08:09.160257+0000","last_undegraded":"2024-05-08T18:08:09.160257+0000","last_fullsized":"2024-05-08T18:08:09.160257+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'285","last_scrub_stamp":"2024-05-08T18:08:09.160141+0000","last_deep_scrub":"29'285","last_deep_scrub_stamp":"2024-05-08T18:08:09.160141+0000","last_clean_scrub_stamp":"2024-05-08T18:08:09.160141+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:29:48.216550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:26.983362+0000","last_change":"2024-05-08T18:09:26.983362+0000","last_active":"2024-05-08T18:09:26.983362+0000","last_peered":"2024-05-08T18:09:26.983362+0000","last_clean":"2024-05-08T18:09:26.983362+0000","last_became_active":"2024-05-08T18:04:53.848321+0000","last_became_peered":"2024-05-08T18:04:53.848321+0000","last_unstale":"2024-05-08T18:09:26.983362+0000","last_undegraded":"2024-05-08T18:09:26.983362+0000","last_fullsized":"2024-05-08T18:09:26.983362+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:26.983245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:26.983245+0000","last_clean_scrub_stamp":"2024-05-08T18:09:26.983245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:37.160081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"29'4","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:33.249650+0000","last_change":"2024-05-08T18:08:33.249675+0000","last_active":"2024-05-08T18:08:33.249650+0000","last_peered":"2024-05-08T18:08:33.249650+0000","last_clean":"2024-05-08T18:08:33.249650+0000","last_became_active":"2024-05-08T18:04:45.813424+0000","last_became_peered":"2024-05-08T18:04:45.813424+0000","last_unstale":"2024-05-08T18:08:33.249650+0000","last_undegraded":"2024-05-08T18:08:33.249650+0000","last_fullsized":"2024-05-08T18:08:33.249650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:33.249561+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:33.249561+0000","last_clean_scrub_stamp":"2024-05-08T18:08:33.249561+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:14.493205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:42.532811+0000","last_change":"2024-05-08T18:08:42.532811+0000","last_active":"2024-05-08T18:08:42.532811+0000","last_peered":"2024-05-08T18:08:42.532811+0000","last_clean":"2024-05-08T18:08:42.532811+0000","last_became_active":"2024-05-08T18:04:43.840502+0000","last_became_peered":"2024-05-08T18:04:43.840502+0000","last_unstale":"2024-05-08T18:08:42.532811+0000","last_undegraded":"2024-05-08T18:08:42.532811+0000","last_fullsized":"2024-05-08T18:08:42.532811+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:42.532676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:42.532676+0000","last_clean_scrub_stamp":"2024-05-08T18:08:42.532676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:39:06.240119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:18.519639+0000","last_change":"2024-05-08T18:08:18.519666+0000","last_active":"2024-05-08T18:08:18.519639+0000","last_peered":"2024-05-08T18:08:18.519639+0000","last_clean":"2024-05-08T18:08:18.519639+0000","last_became_active":"2024-05-08T18:04:47.818839+0000","last_became_peered":"2024-05-08T18:04:47.818839+0000","last_unstale":"2024-05-08T18:08:18.519639+0000","last_undegraded":"2024-05-08T18:08:18.519639+0000","last_fullsized":"2024-05-08T18:08:18.519639+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:18.519542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:18.519542+0000","last_clean_scrub_stamp":"2024-05-08T18:08:18.519542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:14:21.272156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"29'46","reported_seq":105,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:12.495592+0000","last_change":"2024-05-08T18:08:12.495592+0000","last_active":"2024-05-08T18:08:12.495592+0000","last_peered":"2024-05-08T18:08:12.495592+0000","last_clean":"2024-05-08T18:08:12.495592+0000","last_became_active":"2024-05-08T18:04:49.824405+0000","last_became_peered":"2024-05-08T18:04:49.824405+0000","last_unstale":"2024-05-08T18:08:12.495592+0000","last_undegraded":"2024-05-08T18:08:12.495592+0000","last_fullsized":"2024-05-08T18:08:12.495592+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'46","last_scrub_stamp":"2024-05-08T18:08:12.495454+0000","last_deep_scrub":"29'46","last_deep_scrub_stamp":"2024-05-08T18:08:12.495454+0000","last_clean_scrub_stamp":"2024-05-08T18:08:12.495454+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:48:04.037407+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:17.562569+0000","last_change":"2024-05-08T18:08:17.562569+0000","last_active":"2024-05-08T18:08:17.562569+0000","last_peered":"2024-05-08T18:08:17.562569+0000","last_clean":"2024-05-08T18:08:17.562569+0000","last_became_active":"2024-05-08T18:04:51.844757+0000","last_became_peered":"2024-05-08T18:04:51.844757+0000","last_unstale":"2024-05-08T18:08:17.562569+0000","last_undegraded":"2024-05-08T18:08:17.562569+0000","last_fullsized":"2024-05-08T18:08:17.562569+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:17.562441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:17.562441+0000","last_clean_scrub_stamp":"2024-05-08T18:08:17.562441+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:50:03.644980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:33.515879+0000","last_change":"2024-05-08T18:08:33.515879+0000","last_active":"2024-05-08T18:08:33.515879+0000","last_peered":"2024-05-08T18:08:33.515879+0000","last_clean":"2024-05-08T18:08:33.515879+0000","last_became_active":"2024-05-08T18:04:53.840956+0000","last_became_peered":"2024-05-08T18:04:53.840956+0000","last_unstale":"2024-05-08T18:08:33.515879+0000","last_undegraded":"2024-05-08T18:08:33.515879+0000","last_fullsized":"2024-05-08T18:08:33.515879+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:33.515757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:33.515757+0000","last_clean_scrub_stamp":"2024-05-08T18:08:33.515757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:29.798801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"29'4","reported_seq":46,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:28.283892+0000","last_change":"2024-05-08T18:08:28.283933+0000","last_active":"2024-05-08T18:08:28.283892+0000","last_peered":"2024-05-08T18:08:28.283892+0000","last_clean":"2024-05-08T18:08:28.283892+0000","last_became_active":"2024-05-08T18:04:45.806560+0000","last_became_peered":"2024-05-08T18:04:45.806560+0000","last_unstale":"2024-05-08T18:08:28.283892+0000","last_undegraded":"2024-05-08T18:08:28.283892+0000","last_fullsized":"2024-05-08T18:08:28.283892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:28.283774+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:28.283774+0000","last_clean_scrub_stamp":"2024-05-08T18:08:28.283774+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:55:22.065846+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:12.966165+0000","last_change":"2024-05-08T18:09:12.966165+0000","last_active":"2024-05-08T18:09:12.966165+0000","last_peered":"2024-05-08T18:09:12.966165+0000","last_clean":"2024-05-08T18:09:12.966165+0000","last_became_active":"2024-05-08T18:04:43.845214+0000","last_became_peered":"2024-05-08T18:04:43.845214+0000","last_unstale":"2024-05-08T18:09:12.966165+0000","last_undegraded":"2024-05-08T18:09:12.966165+0000","last_fullsized":"2024-05-08T18:09:12.966165+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:12.965986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:12.965986+0000","last_clean_scrub_stamp":"2024-05-08T18:09:12.965986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:49:31.443445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:23.286599+0000","last_change":"2024-05-08T18:08:23.286625+0000","last_active":"2024-05-08T18:08:23.286599+0000","last_peered":"2024-05-08T18:08:23.286599+0000","last_clean":"2024-05-08T18:08:23.286599+0000","last_became_active":"2024-05-08T18:04:47.822748+0000","last_became_peered":"2024-05-08T18:04:47.822748+0000","last_unstale":"2024-05-08T18:08:23.286599+0000","last_undegraded":"2024-05-08T18:08:23.286599+0000","last_fullsized":"2024-05-08T18:08:23.286599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:23.286509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:23.286509+0000","last_clean_scrub_stamp":"2024-05-08T18:08:23.286509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:17:35.464618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"29'37","reported_seq":93,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:14.576300+0000","last_change":"2024-05-08T18:08:14.576300+0000","last_active":"2024-05-08T18:08:14.576300+0000","last_peered":"2024-05-08T18:08:14.576300+0000","last_clean":"2024-05-08T18:08:14.576300+0000","last_became_active":"2024-05-08T18:04:49.831440+0000","last_became_peered":"2024-05-08T18:04:49.831440+0000","last_unstale":"2024-05-08T18:08:14.576300+0000","last_undegraded":"2024-05-08T18:08:14.576300+0000","last_fullsized":"2024-05-08T18:08:14.576300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'37","last_scrub_stamp":"2024-05-08T18:08:14.576150+0000","last_deep_scrub":"29'37","last_deep_scrub_stamp":"2024-05-08T18:08:14.576150+0000","last_clean_scrub_stamp":"2024-05-08T18:08:14.576150+0000","objects_scrubbed":4,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:06:14.892455+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:27.822675+0000","last_change":"2024-05-08T18:08:27.822675+0000","last_active":"2024-05-08T18:08:27.822675+0000","last_peered":"2024-05-08T18:08:27.822675+0000","last_clean":"2024-05-08T18:08:27.822675+0000","last_became_active":"2024-05-08T18:04:51.844125+0000","last_became_peered":"2024-05-08T18:04:51.844125+0000","last_unstale":"2024-05-08T18:08:27.822675+0000","last_undegraded":"2024-05-08T18:08:27.822675+0000","last_fullsized":"2024-05-08T18:08:27.822675+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:27.822555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:27.822555+0000","last_clean_scrub_stamp":"2024-05-08T18:08:27.822555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:47:50.940092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":27,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:28.943072+0000","last_change":"2024-05-08T18:09:28.943072+0000","last_active":"2024-05-08T18:09:28.943072+0000","last_peered":"2024-05-08T18:09:28.943072+0000","last_clean":"2024-05-08T18:09:28.943072+0000","last_became_active":"2024-05-08T18:04:53.849514+0000","last_became_peered":"2024-05-08T18:04:53.849514+0000","last_unstale":"2024-05-08T18:09:28.943072+0000","last_undegraded":"2024-05-08T18:09:28.943072+0000","last_fullsized":"2024-05-08T18:09:28.943072+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:28.942957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:28.942957+0000","last_clean_scrub_stamp":"2024-05-08T18:09:28.942957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:14:00.720988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"29'4","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:54.075473+0000","last_change":"2024-05-08T18:08:54.075473+0000","last_active":"2024-05-08T18:08:54.075473+0000","last_peered":"2024-05-08T18:08:54.075473+0000","last_clean":"2024-05-08T18:08:54.075473+0000","last_became_active":"2024-05-08T18:04:45.822204+0000","last_became_peered":"2024-05-08T18:04:45.822204+0000","last_unstale":"2024-05-08T18:08:54.075473+0000","last_undegraded":"2024-05-08T18:08:54.075473+0000","last_fullsized":"2024-05-08T18:08:54.075473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:54.075350+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:54.075350+0000","last_clean_scrub_stamp":"2024-05-08T18:08:54.075350+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:48:04.754616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"29'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:48.395613+0000","last_change":"2024-05-08T18:08:48.395693+0000","last_active":"2024-05-08T18:08:48.395613+0000","last_peered":"2024-05-08T18:08:48.395613+0000","last_clean":"2024-05-08T18:08:48.395613+0000","last_became_active":"2024-05-08T18:04:43.834678+0000","last_became_peered":"2024-05-08T18:04:43.834678+0000","last_unstale":"2024-05-08T18:08:48.395613+0000","last_undegraded":"2024-05-08T18:08:48.395613+0000","last_fullsized":"2024-05-08T18:08:48.395613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:48.395512+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:48.395512+0000","last_clean_scrub_stamp":"2024-05-08T18:08:48.395512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:53:55.075303+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:16.501182+0000","last_change":"2024-05-08T18:08:16.501182+0000","last_active":"2024-05-08T18:08:16.501182+0000","last_peered":"2024-05-08T18:08:16.501182+0000","last_clean":"2024-05-08T18:08:16.501182+0000","last_became_active":"2024-05-08T18:04:47.820036+0000","last_became_peered":"2024-05-08T18:04:47.820036+0000","last_unstale":"2024-05-08T18:08:16.501182+0000","last_undegraded":"2024-05-08T18:08:16.501182+0000","last_fullsized":"2024-05-08T18:08:16.501182+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:16.501065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:16.501065+0000","last_clean_scrub_stamp":"2024-05-08T18:08:16.501065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:25:49.649348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"29'32","reported_seq":85,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:20.857112+0000","last_change":"2024-05-08T18:08:20.857184+0000","last_active":"2024-05-08T18:08:20.857112+0000","last_peered":"2024-05-08T18:08:20.857112+0000","last_clean":"2024-05-08T18:08:20.857112+0000","last_became_active":"2024-05-08T18:04:49.830882+0000","last_became_peered":"2024-05-08T18:04:49.830882+0000","last_unstale":"2024-05-08T18:08:20.857112+0000","last_undegraded":"2024-05-08T18:08:20.857112+0000","last_fullsized":"2024-05-08T18:08:20.857112+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'32","last_scrub_stamp":"2024-05-08T18:08:20.857010+0000","last_deep_scrub":"29'32","last_deep_scrub_stamp":"2024-05-08T18:08:20.857010+0000","last_clean_scrub_stamp":"2024-05-08T18:08:20.857010+0000","objects_scrubbed":2,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:06.682042+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":49,"num_write":14,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:29.815248+0000","last_change":"2024-05-08T18:08:29.815276+0000","last_active":"2024-05-08T18:08:29.815248+0000","last_peered":"2024-05-08T18:08:29.815248+0000","last_clean":"2024-05-08T18:08:29.815248+0000","last_became_active":"2024-05-08T18:04:51.842734+0000","last_became_peered":"2024-05-08T18:04:51.842734+0000","last_unstale":"2024-05-08T18:08:29.815248+0000","last_undegraded":"2024-05-08T18:08:29.815248+0000","last_fullsized":"2024-05-08T18:08:29.815248+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:29.815133+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:29.815133+0000","last_clean_scrub_stamp":"2024-05-08T18:08:29.815133+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:51:13.451798+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"29'10","reported_seq":37,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:57.037521+0000","last_change":"2024-05-08T18:08:57.037521+0000","last_active":"2024-05-08T18:08:57.037521+0000","last_peered":"2024-05-08T18:08:57.037521+0000","last_clean":"2024-05-08T18:08:57.037521+0000","last_became_active":"2024-05-08T18:04:53.844891+0000","last_became_peered":"2024-05-08T18:04:53.844891+0000","last_unstale":"2024-05-08T18:08:57.037521+0000","last_undegraded":"2024-05-08T18:08:57.037521+0000","last_fullsized":"2024-05-08T18:08:57.037521+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:57.037400+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:57.037400+0000","last_clean_scrub_stamp":"2024-05-08T18:08:57.037400+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:33:31.062988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":27,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"29'2","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:27.291750+0000","last_change":"2024-05-08T18:08:27.291750+0000","last_active":"2024-05-08T18:08:27.291750+0000","last_peered":"2024-05-08T18:08:27.291750+0000","last_clean":"2024-05-08T18:08:27.291750+0000","last_became_active":"2024-05-08T18:04:45.808936+0000","last_became_peered":"2024-05-08T18:04:45.808936+0000","last_unstale":"2024-05-08T18:08:27.291750+0000","last_undegraded":"2024-05-08T18:08:27.291750+0000","last_fullsized":"2024-05-08T18:08:27.291750+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:27.291616+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:27.291616+0000","last_clean_scrub_stamp":"2024-05-08T18:08:27.291616+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:50:13.374155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:58.481285+0000","last_change":"2024-05-08T18:08:58.481320+0000","last_active":"2024-05-08T18:08:58.481285+0000","last_peered":"2024-05-08T18:08:58.481285+0000","last_clean":"2024-05-08T18:08:58.481285+0000","last_became_active":"2024-05-08T18:04:43.837070+0000","last_became_peered":"2024-05-08T18:04:43.837070+0000","last_unstale":"2024-05-08T18:08:58.481285+0000","last_undegraded":"2024-05-08T18:08:58.481285+0000","last_fullsized":"2024-05-08T18:08:58.481285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:58.481195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:58.481195+0000","last_clean_scrub_stamp":"2024-05-08T18:08:58.481195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:16.399630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"29'3","reported_seq":42,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:15.481759+0000","last_change":"2024-05-08T18:08:15.481759+0000","last_active":"2024-05-08T18:08:15.481759+0000","last_peered":"2024-05-08T18:08:15.481759+0000","last_clean":"2024-05-08T18:08:15.481759+0000","last_became_active":"2024-05-08T18:04:47.824172+0000","last_became_peered":"2024-05-08T18:04:47.824172+0000","last_unstale":"2024-05-08T18:08:15.481759+0000","last_undegraded":"2024-05-08T18:08:15.481759+0000","last_fullsized":"2024-05-08T18:08:15.481759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3","last_scrub_stamp":"2024-05-08T18:08:15.481627+0000","last_deep_scrub":"29'3","last_deep_scrub_stamp":"2024-05-08T18:08:15.481627+0000","last_clean_scrub_stamp":"2024-05-08T18:08:15.481627+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:31:00.387286+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"29'40","reported_seq":97,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:33.898989+0000","last_change":"2024-05-08T18:08:33.899046+0000","last_active":"2024-05-08T18:08:33.898989+0000","last_peered":"2024-05-08T18:08:33.898989+0000","last_clean":"2024-05-08T18:08:33.898989+0000","last_became_active":"2024-05-08T18:04:49.831803+0000","last_became_peered":"2024-05-08T18:04:49.831803+0000","last_unstale":"2024-05-08T18:08:33.898989+0000","last_undegraded":"2024-05-08T18:08:33.898989+0000","last_fullsized":"2024-05-08T18:08:33.898989+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'40","last_scrub_stamp":"2024-05-08T18:08:33.898897+0000","last_deep_scrub":"29'40","last_deep_scrub_stamp":"2024-05-08T18:08:33.898897+0000","last_clean_scrub_stamp":"2024-05-08T18:08:33.898897+0000","objects_scrubbed":5,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:05:09.280810+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":36,"num_write":20,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1270,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:25.577300+0000","last_change":"2024-05-08T18:08:25.577348+0000","last_active":"2024-05-08T18:08:25.577300+0000","last_peered":"2024-05-08T18:08:25.577300+0000","last_clean":"2024-05-08T18:08:25.577300+0000","last_became_active":"2024-05-08T18:04:51.846225+0000","last_became_peered":"2024-05-08T18:04:51.846225+0000","last_unstale":"2024-05-08T18:08:25.577300+0000","last_undegraded":"2024-05-08T18:08:25.577300+0000","last_fullsized":"2024-05-08T18:08:25.577300+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:25.577162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:25.577162+0000","last_clean_scrub_stamp":"2024-05-08T18:08:25.577162+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:15.650574+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"29'4","reported_seq":32,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:53.121087+0000","last_change":"2024-05-08T18:08:53.121087+0000","last_active":"2024-05-08T18:08:53.121087+0000","last_peered":"2024-05-08T18:08:53.121087+0000","last_clean":"2024-05-08T18:08:53.121087+0000","last_became_active":"2024-05-08T18:04:53.842429+0000","last_became_peered":"2024-05-08T18:04:53.842429+0000","last_unstale":"2024-05-08T18:08:53.121087+0000","last_undegraded":"2024-05-08T18:08:53.121087+0000","last_fullsized":"2024-05-08T18:08:53.121087+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:53.120970+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:53.120970+0000","last_clean_scrub_stamp":"2024-05-08T18:08:53.120970+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:25:15.271281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":9,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"27'2","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:32.506149+0000","last_change":"2024-05-08T18:08:32.506149+0000","last_active":"2024-05-08T18:08:32.506149+0000","last_peered":"2024-05-08T18:08:32.506149+0000","last_clean":"2024-05-08T18:08:32.506149+0000","last_became_active":"2024-05-08T18:04:45.818161+0000","last_became_peered":"2024-05-08T18:04:45.818161+0000","last_unstale":"2024-05-08T18:08:32.506149+0000","last_undegraded":"2024-05-08T18:08:32.506149+0000","last_fullsized":"2024-05-08T18:08:32.506149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:08:32.506023+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:08:32.506023+0000","last_clean_scrub_stamp":"2024-05-08T18:08:32.506023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:23:20.670584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:38.182450+0000","last_change":"2024-05-08T18:08:38.182450+0000","last_active":"2024-05-08T18:08:38.182450+0000","last_peered":"2024-05-08T18:08:38.182450+0000","last_clean":"2024-05-08T18:08:38.182450+0000","last_became_active":"2024-05-08T18:04:43.842483+0000","last_became_peered":"2024-05-08T18:04:43.842483+0000","last_unstale":"2024-05-08T18:08:38.182450+0000","last_undegraded":"2024-05-08T18:08:38.182450+0000","last_fullsized":"2024-05-08T18:08:38.182450+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:38.182336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:38.182336+0000","last_clean_scrub_stamp":"2024-05-08T18:08:38.182336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:54:29.772068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"29'36","reported_seq":116,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:41.377160+0000","last_change":"2024-05-08T18:08:41.377160+0000","last_active":"2024-05-08T18:08:41.377160+0000","last_peered":"2024-05-08T18:08:41.377160+0000","last_clean":"2024-05-08T18:08:41.377160+0000","last_became_active":"2024-05-08T18:04:45.807847+0000","last_became_peered":"2024-05-08T18:04:45.807847+0000","last_unstale":"2024-05-08T18:08:41.377160+0000","last_undegraded":"2024-05-08T18:08:41.377160+0000","last_fullsized":"2024-05-08T18:08:41.377160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'36","last_scrub_stamp":"2024-05-08T18:08:41.376990+0000","last_deep_scrub":"29'36","last_deep_scrub_stamp":"2024-05-08T18:08:41.376990+0000","last_clean_scrub_stamp":"2024-05-08T18:08:41.376990+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:19:16.885027+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":44,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:36.472685+0000","last_change":"2024-05-08T18:08:36.472685+0000","last_active":"2024-05-08T18:08:36.472685+0000","last_peered":"2024-05-08T18:08:36.472685+0000","last_clean":"2024-05-08T18:08:36.472685+0000","last_became_active":"2024-05-08T18:04:43.836045+0000","last_became_peered":"2024-05-08T18:04:43.836045+0000","last_unstale":"2024-05-08T18:08:36.472685+0000","last_undegraded":"2024-05-08T18:08:36.472685+0000","last_fullsized":"2024-05-08T18:08:36.472685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:36.472507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:36.472507+0000","last_clean_scrub_stamp":"2024-05-08T18:08:36.472507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:13.020912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,3,2],"acting":[0,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"29'12","reported_seq":79,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:47.107135+0000","last_change":"2024-05-08T18:08:47.107135+0000","last_active":"2024-05-08T18:08:47.107135+0000","last_peered":"2024-05-08T18:08:47.107135+0000","last_clean":"2024-05-08T18:08:47.107135+0000","last_became_active":"2024-05-08T18:04:45.825506+0000","last_became_peered":"2024-05-08T18:04:45.825506+0000","last_unstale":"2024-05-08T18:08:47.107135+0000","last_undegraded":"2024-05-08T18:08:47.107135+0000","last_fullsized":"2024-05-08T18:08:47.107135+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:47.106962+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:47.106962+0000","last_clean_scrub_stamp":"2024-05-08T18:08:47.106962+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:18:30.261074+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":50,"num_read_kb":46,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"29'6","reported_seq":60,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:19.940669+0000","last_change":"2024-05-08T18:09:19.940707+0000","last_active":"2024-05-08T18:09:19.940669+0000","last_peered":"2024-05-08T18:09:19.940669+0000","last_clean":"2024-05-08T18:09:19.940669+0000","last_became_active":"2024-05-08T18:04:43.832406+0000","last_became_peered":"2024-05-08T18:04:43.832406+0000","last_unstale":"2024-05-08T18:09:19.940669+0000","last_undegraded":"2024-05-08T18:09:19.940669+0000","last_fullsized":"2024-05-08T18:09:19.940669+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:09:19.940563+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:09:19.940563+0000","last_clean_scrub_stamp":"2024-05-08T18:09:19.940563+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:41:55.643608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":8,"num_write":30,"num_write_kb":4093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"29'2","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:35.470976+0000","last_change":"2024-05-08T18:08:35.470976+0000","last_active":"2024-05-08T18:08:35.470976+0000","last_peered":"2024-05-08T18:08:35.470976+0000","last_clean":"2024-05-08T18:08:35.470976+0000","last_became_active":"2024-05-08T18:04:45.806646+0000","last_became_peered":"2024-05-08T18:04:45.806646+0000","last_unstale":"2024-05-08T18:08:35.470976+0000","last_undegraded":"2024-05-08T18:08:35.470976+0000","last_fullsized":"2024-05-08T18:08:35.470976+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:35.470848+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:35.470848+0000","last_clean_scrub_stamp":"2024-05-08T18:08:35.470848+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:44:55.258041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"29'6","reported_seq":56,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:40.377297+0000","last_change":"2024-05-08T18:08:40.377297+0000","last_active":"2024-05-08T18:08:40.377297+0000","last_peered":"2024-05-08T18:08:40.377297+0000","last_clean":"2024-05-08T18:08:40.377297+0000","last_became_active":"2024-05-08T18:04:43.835163+0000","last_became_peered":"2024-05-08T18:04:43.835163+0000","last_unstale":"2024-05-08T18:08:40.377297+0000","last_undegraded":"2024-05-08T18:08:40.377297+0000","last_fullsized":"2024-05-08T18:08:40.377297+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:40.377108+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:40.377108+0000","last_clean_scrub_stamp":"2024-05-08T18:08:40.377108+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:50:26.202140+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":4746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,1],"acting":[0,3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"29'2","reported_seq":45,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:34.514234+0000","last_change":"2024-05-08T18:08:34.514234+0000","last_active":"2024-05-08T18:08:34.514234+0000","last_peered":"2024-05-08T18:08:34.514234+0000","last_clean":"2024-05-08T18:08:34.514234+0000","last_became_active":"2024-05-08T18:04:45.816391+0000","last_became_peered":"2024-05-08T18:04:45.816391+0000","last_unstale":"2024-05-08T18:08:34.514234+0000","last_undegraded":"2024-05-08T18:08:34.514234+0000","last_fullsized":"2024-05-08T18:08:34.514234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:34.514122+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:34.514122+0000","last_clean_scrub_stamp":"2024-05-08T18:08:34.514122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:11.960319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'4","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:42.343533+0000","last_change":"2024-05-08T18:08:42.343533+0000","last_active":"2024-05-08T18:08:42.343533+0000","last_peered":"2024-05-08T18:08:42.343533+0000","last_clean":"2024-05-08T18:08:42.343533+0000","last_became_active":"2024-05-08T18:04:43.835268+0000","last_became_peered":"2024-05-08T18:04:43.835268+0000","last_unstale":"2024-05-08T18:08:42.343533+0000","last_undegraded":"2024-05-08T18:08:42.343533+0000","last_fullsized":"2024-05-08T18:08:42.343533+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-05-08T18:08:42.343414+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-05-08T18:08:42.343414+0000","last_clean_scrub_stamp":"2024-05-08T18:08:42.343414+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:24.879935+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"29'12","reported_seq":87,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:28.537351+0000","last_change":"2024-05-08T18:08:28.537351+0000","last_active":"2024-05-08T18:08:28.537351+0000","last_peered":"2024-05-08T18:08:28.537351+0000","last_clean":"2024-05-08T18:08:28.537351+0000","last_became_active":"2024-05-08T18:04:45.819699+0000","last_became_peered":"2024-05-08T18:04:45.819699+0000","last_unstale":"2024-05-08T18:08:28.537351+0000","last_undegraded":"2024-05-08T18:08:28.537351+0000","last_fullsized":"2024-05-08T18:08:28.537351+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'12","last_scrub_stamp":"2024-05-08T18:08:28.537227+0000","last_deep_scrub":"29'12","last_deep_scrub_stamp":"2024-05-08T18:08:28.537227+0000","last_clean_scrub_stamp":"2024-05-08T18:08:28.537227+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:18.610639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:57.463007+0000","last_change":"2024-05-08T18:08:57.463007+0000","last_active":"2024-05-08T18:08:57.463007+0000","last_peered":"2024-05-08T18:08:57.463007+0000","last_clean":"2024-05-08T18:08:57.463007+0000","last_became_active":"2024-05-08T18:04:43.836735+0000","last_became_peered":"2024-05-08T18:04:43.836735+0000","last_unstale":"2024-05-08T18:08:57.463007+0000","last_undegraded":"2024-05-08T18:08:57.463007+0000","last_fullsized":"2024-05-08T18:08:57.463007+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:57.462845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:57.462845+0000","last_clean_scrub_stamp":"2024-05-08T18:08:57.462845+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:58:44.500274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"29'14","reported_seq":80,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:47.377267+0000","last_change":"2024-05-08T18:08:47.377267+0000","last_active":"2024-05-08T18:08:47.377267+0000","last_peered":"2024-05-08T18:08:47.377267+0000","last_clean":"2024-05-08T18:08:47.377267+0000","last_became_active":"2024-05-08T18:04:45.811953+0000","last_became_peered":"2024-05-08T18:04:45.811953+0000","last_unstale":"2024-05-08T18:08:47.377267+0000","last_undegraded":"2024-05-08T18:08:47.377267+0000","last_fullsized":"2024-05-08T18:08:47.377267+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'14","last_scrub_stamp":"2024-05-08T18:08:47.377155+0000","last_deep_scrub":"29'14","last_deep_scrub_stamp":"2024-05-08T18:08:47.377155+0000","last_clean_scrub_stamp":"2024-05-08T18:08:47.377155+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:48:10.494691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:34.213484+0000","last_change":"2024-05-08T18:08:34.213484+0000","last_active":"2024-05-08T18:08:34.213484+0000","last_peered":"2024-05-08T18:08:34.213484+0000","last_clean":"2024-05-08T18:08:34.213484+0000","last_became_active":"2024-05-08T18:04:43.842179+0000","last_became_peered":"2024-05-08T18:04:43.842179+0000","last_unstale":"2024-05-08T18:08:34.213484+0000","last_undegraded":"2024-05-08T18:08:34.213484+0000","last_fullsized":"2024-05-08T18:08:34.213484+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:34.213370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:34.213370+0000","last_clean_scrub_stamp":"2024-05-08T18:08:34.213370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:22:13.048212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,0],"acting":[2,3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"29'8","reported_seq":57,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:59.502471+0000","last_change":"2024-05-08T18:08:59.502501+0000","last_active":"2024-05-08T18:08:59.502471+0000","last_peered":"2024-05-08T18:08:59.502471+0000","last_clean":"2024-05-08T18:08:59.502471+0000","last_became_active":"2024-05-08T18:04:45.812637+0000","last_became_peered":"2024-05-08T18:04:45.812637+0000","last_unstale":"2024-05-08T18:08:59.502471+0000","last_undegraded":"2024-05-08T18:08:59.502471+0000","last_fullsized":"2024-05-08T18:08:59.502471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:59.502380+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:59.502380+0000","last_clean_scrub_stamp":"2024-05-08T18:08:59.502380+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:08:00.350733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"29'6","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:58.521154+0000","last_change":"2024-05-08T18:08:58.521154+0000","last_active":"2024-05-08T18:08:58.521154+0000","last_peered":"2024-05-08T18:08:58.521154+0000","last_clean":"2024-05-08T18:08:58.521154+0000","last_became_active":"2024-05-08T18:04:43.834491+0000","last_became_peered":"2024-05-08T18:04:43.834491+0000","last_unstale":"2024-05-08T18:08:58.521154+0000","last_undegraded":"2024-05-08T18:08:58.521154+0000","last_fullsized":"2024-05-08T18:08:58.521154+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:58.520991+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:58.520991+0000","last_clean_scrub_stamp":"2024-05-08T18:08:58.520991+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:32:07.150914+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":7,"num_write":30,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,3],"acting":[0,1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"29'8","reported_seq":60,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:34.558889+0000","last_change":"2024-05-08T18:08:34.558889+0000","last_active":"2024-05-08T18:08:34.558889+0000","last_peered":"2024-05-08T18:08:34.558889+0000","last_clean":"2024-05-08T18:08:34.558889+0000","last_became_active":"2024-05-08T18:04:45.815677+0000","last_became_peered":"2024-05-08T18:04:45.815677+0000","last_unstale":"2024-05-08T18:08:34.558889+0000","last_undegraded":"2024-05-08T18:08:34.558889+0000","last_fullsized":"2024-05-08T18:08:34.558889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:34.558754+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:34.558754+0000","last_clean_scrub_stamp":"2024-05-08T18:08:34.558754+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:30.175682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:23.050897+0000","last_change":"2024-05-08T18:09:23.050922+0000","last_active":"2024-05-08T18:09:23.050897+0000","last_peered":"2024-05-08T18:09:23.050897+0000","last_clean":"2024-05-08T18:09:23.050897+0000","last_became_active":"2024-05-08T18:04:43.843826+0000","last_became_peered":"2024-05-08T18:04:43.843826+0000","last_unstale":"2024-05-08T18:09:23.050897+0000","last_undegraded":"2024-05-08T18:09:23.050897+0000","last_fullsized":"2024-05-08T18:09:23.050897+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:23.050805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:23.050805+0000","last_clean_scrub_stamp":"2024-05-08T18:09:23.050805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:53:29.990439+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"29'6","reported_seq":64,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:43.071567+0000","last_change":"2024-05-08T18:08:43.071567+0000","last_active":"2024-05-08T18:08:43.071567+0000","last_peered":"2024-05-08T18:08:43.071567+0000","last_clean":"2024-05-08T18:08:43.071567+0000","last_became_active":"2024-05-08T18:04:45.827252+0000","last_became_peered":"2024-05-08T18:04:45.827252+0000","last_unstale":"2024-05-08T18:08:43.071567+0000","last_undegraded":"2024-05-08T18:08:43.071567+0000","last_fullsized":"2024-05-08T18:08:43.071567+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:43.071451+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:43.071451+0000","last_clean_scrub_stamp":"2024-05-08T18:08:43.071451+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:41:02.800248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:55.463831+0000","last_change":"2024-05-08T18:08:55.463831+0000","last_active":"2024-05-08T18:08:55.463831+0000","last_peered":"2024-05-08T18:08:55.463831+0000","last_clean":"2024-05-08T18:08:55.463831+0000","last_became_active":"2024-05-08T18:04:43.820360+0000","last_became_peered":"2024-05-08T18:04:43.820360+0000","last_unstale":"2024-05-08T18:08:55.463831+0000","last_undegraded":"2024-05-08T18:08:55.463831+0000","last_fullsized":"2024-05-08T18:08:55.463831+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:55.463676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:55.463676+0000","last_clean_scrub_stamp":"2024-05-08T18:08:55.463676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:43:05.453001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1,3],"acting":[0,2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":43,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:31.445845+0000","last_change":"2024-05-08T18:08:31.445845+0000","last_active":"2024-05-08T18:08:31.445845+0000","last_peered":"2024-05-08T18:08:31.445845+0000","last_clean":"2024-05-08T18:08:31.445845+0000","last_became_active":"2024-05-08T18:04:45.823385+0000","last_became_peered":"2024-05-08T18:04:45.823385+0000","last_unstale":"2024-05-08T18:08:31.445845+0000","last_undegraded":"2024-05-08T18:08:31.445845+0000","last_fullsized":"2024-05-08T18:08:31.445845+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:31.445729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:31.445729+0000","last_clean_scrub_stamp":"2024-05-08T18:08:31.445729+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:08:27.062658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"29'6","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:55.454031+0000","last_change":"2024-05-08T18:08:55.454031+0000","last_active":"2024-05-08T18:08:55.454031+0000","last_peered":"2024-05-08T18:08:55.454031+0000","last_clean":"2024-05-08T18:08:55.454031+0000","last_became_active":"2024-05-08T18:04:43.835838+0000","last_became_peered":"2024-05-08T18:04:43.835838+0000","last_unstale":"2024-05-08T18:08:55.454031+0000","last_undegraded":"2024-05-08T18:08:55.454031+0000","last_fullsized":"2024-05-08T18:08:55.454031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:55.453911+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:55.453911+0000","last_clean_scrub_stamp":"2024-05-08T18:08:55.453911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:47:45.526995+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":8,"num_write":30,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2,0],"acting":[1,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"29'10","reported_seq":59,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:41.515484+0000","last_change":"2024-05-08T18:08:41.515484+0000","last_active":"2024-05-08T18:08:41.515484+0000","last_peered":"2024-05-08T18:08:41.515484+0000","last_clean":"2024-05-08T18:08:41.515484+0000","last_became_active":"2024-05-08T18:04:45.810703+0000","last_became_peered":"2024-05-08T18:04:45.810703+0000","last_unstale":"2024-05-08T18:08:41.515484+0000","last_undegraded":"2024-05-08T18:08:41.515484+0000","last_fullsized":"2024-05-08T18:08:41.515484+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:41.515361+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:41.515361+0000","last_clean_scrub_stamp":"2024-05-08T18:08:41.515361+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:54:04.536630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":20,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"29'4","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:53.458323+0000","last_change":"2024-05-08T18:08:53.458323+0000","last_active":"2024-05-08T18:08:53.458323+0000","last_peered":"2024-05-08T18:08:53.458323+0000","last_clean":"2024-05-08T18:08:53.458323+0000","last_became_active":"2024-05-08T18:04:43.835518+0000","last_became_peered":"2024-05-08T18:04:43.835518+0000","last_unstale":"2024-05-08T18:08:53.458323+0000","last_undegraded":"2024-05-08T18:08:53.458323+0000","last_fullsized":"2024-05-08T18:08:53.458323+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:53.458190+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:53.458190+0000","last_clean_scrub_stamp":"2024-05-08T18:08:53.458190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:31:55.295395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0,3],"acting":[1,2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"29'4","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:26.522702+0000","last_change":"2024-05-08T18:08:26.522702+0000","last_active":"2024-05-08T18:08:26.522702+0000","last_peered":"2024-05-08T18:08:26.522702+0000","last_clean":"2024-05-08T18:08:26.522702+0000","last_became_active":"2024-05-08T18:04:45.820746+0000","last_became_peered":"2024-05-08T18:04:45.820746+0000","last_unstale":"2024-05-08T18:08:26.522702+0000","last_undegraded":"2024-05-08T18:08:26.522702+0000","last_fullsized":"2024-05-08T18:08:26.522702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:26.522525+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:26.522525+0000","last_clean_scrub_stamp":"2024-05-08T18:08:26.522525+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T23:51:07.766169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:58.037345+0000","last_change":"2024-05-08T18:08:58.037345+0000","last_active":"2024-05-08T18:08:58.037345+0000","last_peered":"2024-05-08T18:08:58.037345+0000","last_clean":"2024-05-08T18:08:58.037345+0000","last_became_active":"2024-05-08T18:04:43.844404+0000","last_became_peered":"2024-05-08T18:04:43.844404+0000","last_unstale":"2024-05-08T18:08:58.037345+0000","last_undegraded":"2024-05-08T18:08:58.037345+0000","last_fullsized":"2024-05-08T18:08:58.037345+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:58.037225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:58.037225+0000","last_clean_scrub_stamp":"2024-05-08T18:08:58.037225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:11:08.420812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0,2],"acting":[3,1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"29'4","reported_seq":53,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:39.501858+0000","last_change":"2024-05-08T18:08:39.501858+0000","last_active":"2024-05-08T18:08:39.501858+0000","last_peered":"2024-05-08T18:08:39.501858+0000","last_clean":"2024-05-08T18:08:39.501858+0000","last_became_active":"2024-05-08T18:04:45.809271+0000","last_became_peered":"2024-05-08T18:04:45.809271+0000","last_unstale":"2024-05-08T18:08:39.501858+0000","last_undegraded":"2024-05-08T18:08:39.501858+0000","last_fullsized":"2024-05-08T18:08:39.501858+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'4","last_scrub_stamp":"2024-05-08T18:08:39.501726+0000","last_deep_scrub":"29'4","last_deep_scrub_stamp":"2024-05-08T18:08:39.501726+0000","last_clean_scrub_stamp":"2024-05-08T18:08:39.501726+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:50:16.050132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:59.036095+0000","last_change":"2024-05-08T18:08:59.036095+0000","last_active":"2024-05-08T18:08:59.036095+0000","last_peered":"2024-05-08T18:08:59.036095+0000","last_clean":"2024-05-08T18:08:59.036095+0000","last_became_active":"2024-05-08T18:04:43.838125+0000","last_became_peered":"2024-05-08T18:04:43.838125+0000","last_unstale":"2024-05-08T18:08:59.036095+0000","last_undegraded":"2024-05-08T18:08:59.036095+0000","last_fullsized":"2024-05-08T18:08:59.036095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:59.035976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:59.035976+0000","last_clean_scrub_stamp":"2024-05-08T18:08:59.035976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:24:17.825578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,2,1],"acting":[3,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"29'15","reported_seq":85,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:21.868994+0000","last_change":"2024-05-08T18:08:21.868994+0000","last_active":"2024-05-08T18:08:21.868994+0000","last_peered":"2024-05-08T18:08:21.868994+0000","last_clean":"2024-05-08T18:08:21.868994+0000","last_became_active":"2024-05-08T18:04:45.828488+0000","last_became_peered":"2024-05-08T18:04:45.828488+0000","last_unstale":"2024-05-08T18:08:21.868994+0000","last_undegraded":"2024-05-08T18:08:21.868994+0000","last_fullsized":"2024-05-08T18:08:21.868994+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'15","last_scrub_stamp":"2024-05-08T18:08:21.868870+0000","last_deep_scrub":"29'15","last_deep_scrub_stamp":"2024-05-08T18:08:21.868870+0000","last_clean_scrub_stamp":"2024-05-08T18:08:21.868870+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:18:57.376906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":56,"num_read_kb":52,"num_write":17,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:32.274984+0000","last_change":"2024-05-08T18:08:32.274984+0000","last_active":"2024-05-08T18:08:32.274984+0000","last_peered":"2024-05-08T18:08:32.274984+0000","last_clean":"2024-05-08T18:08:32.274984+0000","last_became_active":"2024-05-08T18:04:43.842670+0000","last_became_peered":"2024-05-08T18:04:43.842670+0000","last_unstale":"2024-05-08T18:08:32.274984+0000","last_undegraded":"2024-05-08T18:08:32.274984+0000","last_fullsized":"2024-05-08T18:08:32.274984+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:08:32.274865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:08:32.274865+0000","last_clean_scrub_stamp":"2024-05-08T18:08:32.274865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:58:55.489696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3,1],"acting":[2,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"29'2","reported_seq":51,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:45.484747+0000","last_change":"2024-05-08T18:08:45.484747+0000","last_active":"2024-05-08T18:08:45.484747+0000","last_peered":"2024-05-08T18:08:45.484747+0000","last_clean":"2024-05-08T18:08:45.484747+0000","last_became_active":"2024-05-08T18:04:45.810562+0000","last_became_peered":"2024-05-08T18:04:45.810562+0000","last_unstale":"2024-05-08T18:08:45.484747+0000","last_undegraded":"2024-05-08T18:08:45.484747+0000","last_fullsized":"2024-05-08T18:08:45.484747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2","last_scrub_stamp":"2024-05-08T18:08:45.484623+0000","last_deep_scrub":"29'2","last_deep_scrub_stamp":"2024-05-08T18:08:45.484623+0000","last_clean_scrub_stamp":"2024-05-08T18:08:45.484623+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:05:22.042022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2","reported_seq":52,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:02.000689+0000","last_change":"2024-05-08T18:09:02.000689+0000","last_active":"2024-05-08T18:09:02.000689+0000","last_peered":"2024-05-08T18:09:02.000689+0000","last_clean":"2024-05-08T18:09:02.000689+0000","last_became_active":"2024-05-08T18:04:43.835943+0000","last_became_peered":"2024-05-08T18:04:43.835943+0000","last_unstale":"2024-05-08T18:09:02.000689+0000","last_undegraded":"2024-05-08T18:09:02.000689+0000","last_fullsized":"2024-05-08T18:09:02.000689+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-05-08T18:09:02.000540+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-05-08T18:09:02.000540+0000","last_clean_scrub_stamp":"2024-05-08T18:09:02.000540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:39:57.173467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1,0],"acting":[3,2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":49,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:09:05.971768+0000","last_change":"2024-05-08T18:09:05.971768+0000","last_active":"2024-05-08T18:09:05.971768+0000","last_peered":"2024-05-08T18:09:05.971768+0000","last_clean":"2024-05-08T18:09:05.971768+0000","last_became_active":"2024-05-08T18:04:43.836007+0000","last_became_peered":"2024-05-08T18:04:43.836007+0000","last_unstale":"2024-05-08T18:09:05.971768+0000","last_undegraded":"2024-05-08T18:09:05.971768+0000","last_fullsized":"2024-05-08T18:09:05.971768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T18:09:05.971607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T18:09:05.971607+0000","last_clean_scrub_stamp":"2024-05-08T18:09:05.971607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:00:19.071636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0,1],"acting":[3,2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"29'8","reported_seq":72,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:49.357546+0000","last_change":"2024-05-08T18:08:49.357546+0000","last_active":"2024-05-08T18:08:49.357546+0000","last_peered":"2024-05-08T18:08:49.357546+0000","last_clean":"2024-05-08T18:08:49.357546+0000","last_became_active":"2024-05-08T18:04:45.811648+0000","last_became_peered":"2024-05-08T18:04:45.811648+0000","last_unstale":"2024-05-08T18:08:49.357546+0000","last_undegraded":"2024-05-08T18:08:49.357546+0000","last_fullsized":"2024-05-08T18:08:49.357546+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'8","last_scrub_stamp":"2024-05-08T18:08:49.357428+0000","last_deep_scrub":"29'8","last_deep_scrub_stamp":"2024-05-08T18:08:49.357428+0000","last_clean_scrub_stamp":"2024-05-08T18:08:49.357428+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:51:32.994094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":31,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"29'6","reported_seq":58,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:38.419521+0000","last_change":"2024-05-08T18:08:38.419521+0000","last_active":"2024-05-08T18:08:38.419521+0000","last_peered":"2024-05-08T18:08:38.419521+0000","last_clean":"2024-05-08T18:08:38.419521+0000","last_became_active":"2024-05-08T18:04:43.835712+0000","last_became_peered":"2024-05-08T18:04:43.835712+0000","last_unstale":"2024-05-08T18:08:38.419521+0000","last_undegraded":"2024-05-08T18:08:38.419521+0000","last_fullsized":"2024-05-08T18:08:38.419521+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'6","last_scrub_stamp":"2024-05-08T18:08:38.419396+0000","last_deep_scrub":"29'6","last_deep_scrub_stamp":"2024-05-08T18:08:38.419396+0000","last_clean_scrub_stamp":"2024-05-08T18:08:38.419396+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:54:01.440665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":22,"num_write_kb":4094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,3,1],"acting":[0,2,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"29'10","reported_seq":65,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-05-08T18:08:47.497589+0000","last_change":"2024-05-08T18:08:47.497589+0000","last_active":"2024-05-08T18:08:47.497589+0000","last_peered":"2024-05-08T18:08:47.497589+0000","last_clean":"2024-05-08T18:08:47.497589+0000","last_became_active":"2024-05-08T18:04:45.811921+0000","last_became_peered":"2024-05-08T18:04:45.811921+0000","last_unstale":"2024-05-08T18:08:47.497589+0000","last_undegraded":"2024-05-08T18:08:47.497589+0000","last_fullsized":"2024-05-08T18:08:47.497589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'10","last_scrub_stamp":"2024-05-08T18:08:47.497436+0000","last_deep_scrub":"29'10","last_deep_scrub_stamp":"2024-05-08T18:08:47.497436+0000","last_clean_scrub_stamp":"2024-05-08T18:08:47.497436+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:06:35.185255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":349,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2281,"ondisk_log_size":2281,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":1600,"num_read_kb":1397,"num_write":597,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3000,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1527,"ondisk_log_size":1527,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":483,"ondisk_log_size":483,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":145,"num_read_kb":87,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":256,"acting":256,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705728,"num_pgs":179,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":438272,"data_stored":135643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705729,"num_pgs":153,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":970752,"data_stored":724153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705729,"num_pgs":157,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40464,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331376,"statfs":{"total":96636764160,"available":96595329024,"internally_reserved":0,"allocated":991232,"data_stored":724641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705729,"num_pgs":169,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31576,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340264,"statfs":{"total":96636764160,"available":96604430336,"internally_reserved":0,"allocated":409600,"data_stored":135609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T18:09:47.332 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-08T18:09:47.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.462+0000 7fef8ceed640 1 Processor -- start 2024-05-08T18:09:47.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.462+0000 7fef8ceed640 1 -- start start 2024-05-08T18:09:47.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.463+0000 7fef8ceed640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef8809e150 0x7fef8812a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:09:47.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.463+0000 7fef8ceed640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef88113d60 con 0x7fef8812a720 2024-05-08T18:09:47.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.463+0000 7fef8ceed640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 -- 0x7fef88129d20 con 0x7fef8809e150 2024-05-08T18:09:47.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.463+0000 7fef8659c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fef8809e150 0x7fef8812a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:09:47.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.463+0000 7fef85d9b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fef8812a720 0x7fef8812d200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:60442/0 (socket says 172.21.15.46:60442) 2024-05-08T18:09:47.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.463+0000 7fef85d9b640 1 -- 172.21.15.46:0/2557290123 learned_addr learned my addr 172.21.15.46:0/2557290123 (peer_addr_for_me v1:172.21.15.46:0/0) 2024-05-08T18:09:47.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.463+0000 7fef8659c640 1 -- 172.21.15.46:0/2557290123 >> v1:172.21.15.46:6789/0 conn(0x7fef8812a720 legacy=0x7fef8812d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:09:47.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.463+0000 7fef8659c640 1 -- 172.21.15.46:0/2557290123 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef88114490 con 0x7fef8809e150 2024-05-08T18:09:47.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.464+0000 7fef8659c640 1 --2- 172.21.15.46:0/2557290123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8809e150 0x7fef8812a1e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fef80009d90 tx=0x7fef8002bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a38ed4c7600b89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:09:47.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.464+0000 7fef8559a640 1 -- 172.21.15.46:0/2557290123 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef80039070 con 0x7fef8809e150 2024-05-08T18:09:47.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.464+0000 7fef8559a640 1 -- 172.21.15.46:0/2557290123 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8002cc30 con 0x7fef8809e150 2024-05-08T18:09:47.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.464+0000 7fef8559a640 1 -- 172.21.15.46:0/2557290123 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef80034440 con 0x7fef8809e150 2024-05-08T18:09:47.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.465+0000 7fef8ceed640 1 -- 172.21.15.46:0/2557290123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8809e150 msgr2=0x7fef8812a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:09:47.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.465+0000 7fef8ceed640 1 --2- 172.21.15.46:0/2557290123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8809e150 0x7fef8812a1e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fef80009d90 tx=0x7fef8002bfd0 comp rx=0 tx=0).stop 2024-05-08T18:09:47.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.465+0000 7fef8ceed640 1 -- 172.21.15.46:0/2557290123 shutdown_connections 2024-05-08T18:09:47.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.465+0000 7fef8ceed640 1 --2- 172.21.15.46:0/2557290123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8809e150 0x7fef8812a1e0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:09:47.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.465+0000 7fef8ceed640 1 -- 172.21.15.46:0/2557290123 >> 172.21.15.46:0/2557290123 conn(0x7fef8809d630 msgr2=0x7fef8809da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:09:47.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.465+0000 7fef8ceed640 1 -- 172.21.15.46:0/2557290123 shutdown_connections 2024-05-08T18:09:47.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.465+0000 7fef8ceed640 1 -- 172.21.15.46:0/2557290123 wait complete. 2024-05-08T18:09:47.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.466+0000 7fef8ceed640 1 Processor -- start 2024-05-08T18:09:47.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.466+0000 7fef8ceed640 1 -- start start 2024-05-08T18:09:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.466+0000 7fef8ceed640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8809e150 0x7fef88071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:09:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.466+0000 7fef8ceed640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fef88114ac0 con 0x7fef8809e150 2024-05-08T18:09:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.466+0000 7fef8659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8809e150 0x7fef88071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:09:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.466+0000 7fef8659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8809e150 0x7fef88071d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33526/0 (socket says 172.21.15.46:33526) 2024-05-08T18:09:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.466+0000 7fef8659c640 1 -- 172.21.15.46:0/2247027884 learned_addr learned my addr 172.21.15.46:0/2247027884 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-08T18:09:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.467+0000 7fef8659c640 1 -- 172.21.15.46:0/2247027884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef88119cb0 con 0x7fef8809e150 2024-05-08T18:09:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.467+0000 7fef8659c640 1 --2- 172.21.15.46:0/2247027884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8809e150 0x7fef88071d40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fef80000f80 tx=0x7fef80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:09:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.467+0000 7fef7effd640 1 -- 172.21.15.46:0/2247027884 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef80044020 con 0x7fef8809e150 2024-05-08T18:09:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.467+0000 7fef7effd640 1 -- 172.21.15.46:0/2247027884 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef8003f5a0 con 0x7fef8809e150 2024-05-08T18:09:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.467+0000 7fef7effd640 1 -- 172.21.15.46:0/2247027884 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef80039040 con 0x7fef8809e150 2024-05-08T18:09:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.467+0000 7fef8ceed640 1 -- 172.21.15.46:0/2247027884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef88072e70 con 0x7fef8809e150 2024-05-08T18:09:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.467+0000 7fef8ceed640 1 -- 172.21.15.46:0/2247027884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef88073b00 con 0x7fef8809e150 2024-05-08T18:09:47.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.468+0000 7fef8ceed640 1 -- 172.21.15.46:0/2247027884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef880640a0 con 0x7fef8809e150 2024-05-08T18:09:47.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.468+0000 7fef7effd640 1 -- 172.21.15.46:0/2247027884 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 49411+0+0 (secure 0 0 0) 0x7fef80047020 con 0x7fef8809e150 2024-05-08T18:09:47.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.468+0000 7fef7effd640 1 --2- 172.21.15.46:0/2247027884 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fef6803d2b0 0x7fef6803f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T18:09:47.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.468+0000 7fef7effd640 1 -- 172.21.15.46:0/2247027884 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6336+0+0 (secure 0 0 0) 0x7fef80074b00 con 0x7fef8809e150 2024-05-08T18:09:47.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.469+0000 7fef85d9b640 1 --2- 172.21.15.46:0/2247027884 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fef6803d2b0 0x7fef6803f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T18:09:47.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.469+0000 7fef85d9b640 1 --2- 172.21.15.46:0/2247027884 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fef6803d2b0 0x7fef6803f770 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fef70007fd0 tx=0x7fef70007aa0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T18:09:47.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.473+0000 7fef7effd640 1 -- 172.21.15.46:0/2247027884 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+190839 (secure 0 0 0) 0x7fef8004e030 con 0x7fef8809e150 2024-05-08T18:09:47.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.663+0000 7fef8ceed640 1 -- 172.21.15.46:0/2247027884 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fef88119a50 con 0x7fef8809e150 2024-05-08T18:09:47.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.665+0000 7fef7effd640 1 -- 172.21.15.46:0/2247027884 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7fef80048cd0 con 0x7fef8809e150 2024-05-08T18:09:47.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.666+0000 7fef7effd640 1 -- 172.21.15.46:0/2247027884 <== mon.0 v2:172.21.15.46:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fef8007a080 con 0x7fef8809e150 2024-05-08T18:09:47.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.668+0000 7fef8ceed640 1 -- 172.21.15.46:0/2247027884 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fef6803d2b0 msgr2=0x7fef6803f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:09:47.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.668+0000 7fef8ceed640 1 --2- 172.21.15.46:0/2247027884 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fef6803d2b0 0x7fef6803f770 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fef70007fd0 tx=0x7fef70007aa0 comp rx=0 tx=0).stop 2024-05-08T18:09:47.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.668+0000 7fef8ceed640 1 -- 172.21.15.46:0/2247027884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8809e150 msgr2=0x7fef88071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T18:09:47.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.668+0000 7fef8ceed640 1 --2- 172.21.15.46:0/2247027884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8809e150 0x7fef88071d40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fef80000f80 tx=0x7fef80004a40 comp rx=0 tx=0).stop 2024-05-08T18:09:47.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.668+0000 7fef8ceed640 1 -- 172.21.15.46:0/2247027884 shutdown_connections 2024-05-08T18:09:47.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.668+0000 7fef8ceed640 1 --2- 172.21.15.46:0/2247027884 >> [v2:172.21.15.50:6824/2602185226,v1:172.21.15.50:6825/2602185226] conn(0x7fef6803d2b0 0x7fef6803f770 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:09:47.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.668+0000 7fef8ceed640 1 --2- 172.21.15.46:0/2247027884 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fef8809e150 0x7fef88071d40 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T18:09:47.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.668+0000 7fef8ceed640 1 -- 172.21.15.46:0/2247027884 >> 172.21.15.46:0/2247027884 conn(0x7fef8809d630 msgr2=0x7fef881289f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T18:09:47.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.668+0000 7fef8ceed640 1 -- 172.21.15.46:0/2247027884 shutdown_connections 2024-05-08T18:09:47.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-08T18:09:47.668+0000 7fef8ceed640 1 -- 172.21.15.46:0/2247027884 wait complete. 2024-05-08T18:09:47.677 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-08T18:09:47.677 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-08T18:09:47.677 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-08T18:09:47.677 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T18:09:47.719 INFO:tasks.ceph.osd.0:Stopped 2024-05-08T18:09:47.719 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-08T18:09:47.719 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T18:09:47.751 INFO:tasks.ceph.osd.1:Stopped 2024-05-08T18:09:47.751 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-08T18:09:47.751 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T18:09:47.820 INFO:tasks.ceph.osd.2:Stopped 2024-05-08T18:09:47.820 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-08T18:09:47.820 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T18:09:47.896 INFO:tasks.ceph.osd.3:Stopped 2024-05-08T18:09:47.896 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-08T18:09:47.896 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-08T18:09:47.896 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T18:09:47.967 INFO:tasks.ceph.mgr.x:Stopped 2024-05-08T18:09:47.967 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-08T18:09:47.968 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T18:09:47.968 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T18:09:47.979 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T18:09:47.979 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-08T18:09:47.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'not have an application enabled' | head -n 1 2024-05-08T18:09:48.047 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T18:09:48.048 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-08T18:09:48.194 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi050.front.sepia.ceph.com 2024-05-08T18:09:48.194 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-08T18:09:48.376 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi050.front.sepia.ceph.com 2024-05-08T18:09:48.377 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-08T18:09:48.442 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi050.front.sepia.ceph.com 2024-05-08T18:09:48.443 DEBUG:teuthology.orchestra.run.smithi050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-08T18:09:48.523 INFO:tasks.ceph:Archiving mon data... 2024-05-08T18:09:48.524 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/ivancich-2024-05-08_17:43:43-rgw-wip-eric-testing-1-distro-default-smithi/7697857/data/mon.a.tgz 2024-05-08T18:09:48.524 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-05-08T18:09:48.541 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.tKkxDiSaa7 2024-05-08T18:09:48.542 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.tKkxDiSaa7 2024-05-08T18:09:48.687 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.tKkxDiSaa7 2024-05-08T18:09:48.773 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.tKkxDiSaa7 is 267KB 2024-05-08T18:09:48.839 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.tKkxDiSaa7 2024-05-08T18:09:48.856 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-08T18:09:48.856 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T18:09:48.898 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T18:09:48.977 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T18:09:48.995 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T18:09:48.995 INFO:tasks.ceph:Archiving crash dumps... 2024-05-08T18:09:48.997 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/crash to /home/teuthworker/archive/ivancich-2024-05-08_17:43:43-rgw-wip-eric-testing-1-distro-default-smithi/7697857/remote/smithi046/crash 2024-05-08T18:09:48.998 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T18:09:49.043 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/crash to /home/teuthworker/archive/ivancich-2024-05-08_17:43:43-rgw-wip-eric-testing-1-distro-default-smithi/7697857/remote/smithi050/crash 2024-05-08T18:09:49.044 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T18:09:49.079 INFO:tasks.ceph:Compressing logs... 2024-05-08T18:09:49.079 DEBUG:teuthology.orchestra.run.smithi046:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T18:09:49.087 DEBUG:teuthology.orchestra.run.smithi050:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T18:09:49.109 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30483.log 2024-05-08T18:09:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30760.log 2024-05-08T18:09:49.124 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30484.log 2024-05-08T18:09:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30483.log: /var/log/ceph/ceph-client.admin.30760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30940.log 2024-05-08T18:09:49.125 INFO:teuthology.orchestra.run.smithi046.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.30483.log.gz 2024-05-08T18:09:49.125 INFO:teuthology.orchestra.run.smithi046.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.30760.log.gz 2024-05-08T18:09:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30844.log 2024-05-08T18:09:49.125 INFO:teuthology.orchestra.run.smithi046.stderr: 86.7%/var/log/ceph/ceph-client.admin.30940.log: -- replaced with /var/log/ceph/ceph-client.admin.30484.log.gz 2024-05-08T18:09:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30316.log 2024-05-08T18:09:49.125 INFO:teuthology.orchestra.run.smithi046.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.30940.log.gz 2024-05-08T18:09:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30844.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T18:09:49.125 INFO:teuthology.orchestra.run.smithi046.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.30844.log.gz 2024-05-08T18:09:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30424.log 2024-05-08T18:09:49.125 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.30316.log.gz 2024-05-08T18:09:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28595.log 2024-05-08T18:09:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.30424.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T18:09:49.125 INFO:teuthology.orchestra.run.smithi046.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.30424.log.gz 2024-05-08T18:09:49.125 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30654.log 2024-05-08T18:09:49.125 INFO:teuthology.orchestra.run.smithi046.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.28595.log.gz 2024-05-08T18:09:49.126 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31444.log 2024-05-08T18:09:49.126 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29972.log 2024-05-08T18:09:49.126 INFO:teuthology.orchestra.run.smithi046.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.30654.log.gz 2024-05-08T18:09:49.126 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T18:09:49.126 INFO:teuthology.orchestra.run.smithi046.stderr: 86.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T18:09:49.126 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31570.log 2024-05-08T18:09:49.126 INFO:teuthology.orchestra.run.smithi046.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.31444.log.gz 2024-05-08T18:09:49.126 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30172.log 2024-05-08T18:09:49.126 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.29972.log: /var/log/ceph/ceph-client.admin.31570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31612.log 2024-05-08T18:09:49.126 INFO:teuthology.orchestra.run.smithi046.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.29972.log.gz 2024-05-08T18:09:49.126 INFO:teuthology.orchestra.run.smithi046.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.31570.log.gz 2024-05-08T18:09:49.126 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30898.log 2024-05-08T18:09:49.126 INFO:teuthology.orchestra.run.smithi046.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.30172.log.gz 2024-05-08T18:09:49.126 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30214.log 2024-05-08T18:09:49.126 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30898.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.31612.log.gz 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26891.log 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.30898.log.gz 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30982.log 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.30214.log.gz 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.tmp-client.admin.26891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30653.log 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26891.log.gz 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30072.log 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30982.log: /var/log/ceph/ceph-client.admin.30653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31150.log 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.30982.log.gz 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30072.log: 86.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.30653.log.gz 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.30072.log.gz 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30313.log 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.0.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.31150.log.gz 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30480.log 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31108.log 2024-05-08T18:09:49.127 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.30313.log.gz 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31486.log 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31108.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.30480.log.gz 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31318.log 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.31108.log.gz 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31486.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.31486.log.gz 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30114.log 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mon.a.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.31318.log.gz 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31652.log 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31360.log 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.30114.log.gz 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31276.log 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.31652.log.gz 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30030.log 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31360.log: /var/log/ceph/ceph-client.admin.31276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31402.log 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.31360.log.gz 2024-05-08T18:09:49.128 INFO:teuthology.orchestra.run.smithi046.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.31276.log.gz 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31192.log 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.30030.log.gz 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31024.log 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.31402.log.gz 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30802.log 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31192.log: /var/log/ceph/ceph-client.admin.31024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28713.log 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.31192.log.gz 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30802.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.31024.log.gz 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30592.log 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.30802.log.gz 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28753.log 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28713.log: /var/log/ceph/ceph-client.admin.30592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30315.log 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.28713.log.gz 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.30592.log.gz 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.28753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31528.log 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30314.log 2024-05-08T18:09:49.129 INFO:teuthology.orchestra.run.smithi046.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.28753.log.gz 2024-05-08T18:09:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30315.log: /var/log/ceph/ceph-client.admin.31528.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.30315.log.gz 2024-05-08T18:09:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31066.log 2024-05-08T18:09:49.130 INFO:teuthology.orchestra.run.smithi046.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.31528.log.gz 2024-05-08T18:09:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30612.log 2024-05-08T18:09:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30314.log: /var/log/ceph/ceph-client.admin.31066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31234.log 2024-05-08T18:09:49.130 INFO:teuthology.orchestra.run.smithi046.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.30314.log.gz 2024-05-08T18:09:49.130 INFO:teuthology.orchestra.run.smithi046.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.31066.log.gz 2024-05-08T18:09:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.30612.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.30612.log.gz 2024-05-08T18:09:49.130 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.31234.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.31234.log.gz 2024-05-08T18:09:49.151 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34386.log 2024-05-08T18:09:49.151 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85969.log 2024-05-08T18:09:49.152 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65423.log 2024-05-08T18:09:49.152 INFO:teuthology.orchestra.run.smithi050.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.34386.log.gz 2024-05-08T18:09:49.152 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78128.log 2024-05-08T18:09:49.152 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.85969.log: /var/log/ceph/ceph-client.0.65423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90803.log 2024-05-08T18:09:49.154 INFO:teuthology.orchestra.run.smithi050.stderr: 90.4%/var/log/ceph/ceph-client.0.78128.log: -- replaced with /var/log/ceph/ceph-client.0.85969.log.gz 2024-05-08T18:09:49.155 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72078.log 2024-05-08T18:09:49.155 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.90803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82968.log 2024-05-08T18:09:49.155 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.72078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.72683.log 2024-05-08T18:09:49.155 INFO:teuthology.orchestra.run.smithi050.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.65423.log.gz 2024-05-08T18:09:49.155 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.82968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31009.log 2024-05-08T18:09:49.155 INFO:teuthology.orchestra.run.smithi050.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.78128.log.gz 2024-05-08T18:09:49.155 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.72683.log: 90.9% -- replaced with /var/log/ceph/ceph-client.0.90803.log.gz 2024-05-08T18:09:49.155 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43643.log 2024-05-08T18:09:49.155 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.72078.log.gz 2024-05-08T18:09:49.155 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53928.log 2024-05-08T18:09:49.155 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.31009.log: /var/log/ceph/ceph-client.0.43643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67843.log 2024-05-08T18:09:49.155 INFO:teuthology.orchestra.run.smithi050.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.31009.log.gz 2024-05-08T18:09:49.160 INFO:teuthology.orchestra.run.smithi050.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.82968.log.gz 2024-05-08T18:09:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.53928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61793.log 2024-05-08T18:09:49.160 INFO:teuthology.orchestra.run.smithi050.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.72683.log.gz 2024-05-08T18:09:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35784.log 2024-05-08T18:09:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.67843.log: /var/log/ceph/ceph-client.0.61793.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.43643.log.gz 2024-05-08T18:09:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75103.log 2024-05-08T18:09:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.35784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47878.log 2024-05-08T18:09:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.75103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49088.log 2024-05-08T18:09:49.160 INFO:teuthology.orchestra.run.smithi050.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.53928.log.gz 2024-05-08T18:09:49.160 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.47878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77523.log 2024-05-08T18:09:49.161 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.61793.log.gz 2024-05-08T18:09:49.161 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.49088.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-08T18:09:49.161 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5/var/log/ceph/ceph-client.0.77523.log: --verbose 91.4% -- replaced with /var/log/ceph/ceph-client.0.35784.log.gz 2024-05-08T18:09:49.161 INFO:teuthology.orchestra.run.smithi050.stderr: -- /var/log/ceph/ceph-client.0.59978.log 2024-05-08T18:09:49.161 INFO:teuthology.orchestra.run.smithi050.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.75103.log.gz 2024-05-08T18:09:49.161 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-osd.2.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.47878.log.gz 2024-05-08T18:09:49.161 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 91.3% --verbose -- replaced with /var/log/ceph/ceph-client.0.49088.log.gz -- 2024-05-08T18:09:49.161 INFO:teuthology.orchestra.run.smithi050.stderr: /var/log/ceph/ceph-client.0.88383.log 2024-05-08T18:09:49.161 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.59978.log: 93.3% -- replaced with /var/log/ceph/ceph-client.0.67843.log.gz 2024-05-08T18:09:49.161 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64213.log 2024-05-08T18:09:49.161 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.88383.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.77523.log.gz 2024-05-08T18:09:49.161 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67238.log 2024-05-08T18:09:49.161 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30969.log 2024-05-08T18:09:49.161 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.64213.log: /var/log/ceph/ceph-client.0.67238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42433.log 2024-05-08T18:09:49.174 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.30969.log: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66028.log 2024-05-08T18:09:49.174 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.0.59978.log.gz 91.1% -- replaced with /var/log/ceph/ceph-client.0.88383.log.gz 2024-05-08T18:09:49.174 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-05-08T18:09:49.174 INFO:teuthology.orchestra.run.smithi050.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.30969.log.gz 2024-05-08T18:09:49.174 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.42433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.87778.log 2024-05-08T18:09:49.174 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.66028.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.64213.log.gz 2024-05-08T18:09:49.174 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58163.log 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44248.log 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.87778.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.42433.log.gz 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.58163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.89593.log 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr:gzip 92.5% -- replaced with /var/log/ceph/ceph-client.0.67238.log.gz/var/log/ceph/ceph-client.0.44248.log: 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.90198.log 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.89593.log: gzip -5 90.9% -- replaced with /var/log/ceph/ceph-client.0.66028.log.gz 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr: --verbose -- /var/log/ceph/ceph-client.0.63003.log 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.87778.log.gz 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.90198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79943.log 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.0.58163.log.gz 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.63003.log: gzip 91.0% -- replaced with /var/log/ceph/ceph-client.0.44248.log.gz 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.70263.log 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.79943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78733.log 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.89593.log.gz 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.70263.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.90198.log.gz 2024-05-08T18:09:49.175 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.78733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68448.log 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-osd.3.log: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64818.log 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.0.63003.log.gz 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.68448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70868.log 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.70263.log.gz 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ops-log-ceph-client.0.log 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.64818.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.78733.log.gz 91.5% -- replaced with /var/log/ceph/ceph-client.0.79943.log.gz 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34266.log 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.70868.log: /var/log/ceph/ops-log-ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56348.log 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57558.log 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr: 91.8% -- replaced with /var/log/ceph/ops-log-ceph-client.0.log.gz 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.56348.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.34266.log.gz 91.4% -- replaced with /var/log/ceph/ceph-client.0.64818.log.gz 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.0.68448.log.gz 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.92013.log 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.57558.log: gzip -5 --verbose -- /var/log/ceph/rgw.ceph.client.0.log 2024-05-08T18:09:49.176 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.92013.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.0.56348.log.gz 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/rgw.ceph.client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34346.log 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31049.log 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.70868.log.gz 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr: 91.2%/var/log/ceph/ceph-osd.1.log: -- replaced with /var/log/ceph/ceph-client.0.57558.log.gz 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69658.log 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34346.log: 90.9%/var/log/ceph/ceph-client.admin.31049.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.34346.log.gz 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.87173.log 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.0.92013.log.gz 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.31049.log.gz 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.69658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.94433.log 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.87173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36994.log 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.94433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40013.log 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0%/var/log/ceph/ceph-client.0.36994.log: -- replaced with /var/log/ceph/ceph-client.0.69658.log.gz 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.87173.log.gz 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74498.log 2024-05-08T18:09:49.177 INFO:teuthology.orchestra.run.smithi050.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.0.94433.log.gz/var/log/ceph/ceph-client.0.40013.log: 2024-05-08T18:09:49.178 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76918.log 2024-05-08T18:09:49.178 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.74498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79338.log 2024-05-08T18:09:49.182 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.76918.log: 91.5% -- replaced with /var/log/ceph/ceph-client.0.36994.log.gz 2024-05-08T18:09:49.182 INFO:teuthology.orchestra.run.smithi050.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.40013.log.gz 2024-05-08T18:09:49.182 INFO:teuthology.orchestra.run.smithi050.stderr:gzip 91.2% -5 --verbose -- /var/log/ceph/ceph-client.0.71473.log 2024-05-08T18:09:49.182 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.0.74498.log.gz 2024-05-08T18:09:49.182 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.79338.log: gzip 91.2% -- replaced with /var/log/ceph/ceph-client.0.76918.log.gz 2024-05-08T18:09:49.182 INFO:teuthology.orchestra.run.smithi050.stderr: -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-08T18:09:49.182 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.71473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38803.log 2024-05-08T18:09:49.183 INFO:teuthology.orchestra.run.smithi050.stderr: 91.2%/var/log/ceph/ceph-mgr.x.log: -- replaced with /var/log/ceph/ceph-client.0.79338.log.gz 2024-05-08T18:09:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.93828.log 2024-05-08T18:09:49.183 INFO:teuthology.orchestra.run.smithi050.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.71473.log.gz 2024-05-08T18:09:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62398.log 2024-05-08T18:09:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.38803.log: /var/log/ceph/ceph-client.0.93828.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.93828.log.gz 2024-05-08T18:09:49.183 INFO:teuthology.orchestra.run.smithi050.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.38803.log.gz 2024-05-08T18:09:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85370.log 2024-05-08T18:09:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.62398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56953.log 2024-05-08T18:09:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.85370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52113.log 2024-05-08T18:09:49.183 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.56953.log: 90.4% -- replaced with /var/log/ceph/ceph-client.0.85370.log.gz 2024-05-08T18:09:49.183 INFO:teuthology.orchestra.run.smithi050.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.62398.log.gz 2024-05-08T18:09:49.184 INFO:teuthology.orchestra.run.smithi050.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.56953.log.gz 2024-05-08T18:09:49.185 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66633.log 2024-05-08T18:09:49.185 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.52113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43038.log 2024-05-08T18:09:49.185 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.66633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.83573.log 2024-05-08T18:09:49.185 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.43038.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.52113.log.gz 2024-05-08T18:09:49.185 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.66633.log.gz 2024-05-08T18:09:49.186 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.43038.log.gz 2024-05-08T18:09:49.186 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.86568.log 2024-05-08T18:09:49.188 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.83573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44853.log 2024-05-08T18:09:49.188 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.86568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36389.log 2024-05-08T18:09:49.188 INFO:teuthology.orchestra.run.smithi050.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.0.83573.log.gz 2024-05-08T18:09:49.188 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.44853.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.86568.log.gz 2024-05-08T18:09:49.189 INFO:teuthology.orchestra.run.smithi050.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.44853.log.gz 2024-05-08T18:09:49.189 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49693.log 2024-05-08T18:09:49.190 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.36389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39408.log 2024-05-08T18:09:49.190 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.49693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52718.log 2024-05-08T18:09:49.192 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.39408.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.36389.log.gz 2024-05-08T18:09:49.192 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.49693.log.gz 2024-05-08T18:09:49.192 INFO:teuthology.orchestra.run.smithi050.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.39408.log.gz 2024-05-08T18:09:49.193 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40618.log 2024-05-08T18:09:49.194 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.52718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.92618.log 2024-05-08T18:09:49.194 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.40618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46063.log 2024-05-08T18:09:49.194 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.92618.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.52718.log.gz 2024-05-08T18:09:49.195 INFO:teuthology.orchestra.run.smithi050.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.40618.log.gz 2024-05-08T18:09:49.196 INFO:teuthology.orchestra.run.smithi050.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.0.92618.log.gz 2024-05-08T18:09:49.197 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84172.log 2024-05-08T18:09:49.197 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.46063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58768.log 2024-05-08T18:09:49.197 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.84172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35179.log 2024-05-08T18:09:49.198 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.58768.log: 91.5% -- replaced with /var/log/ceph/ceph-client.0.84172.log.gz 91.0% -- replaced with /var/log/ceph/ceph-client.0.46063.log.gz 2024-05-08T18:09:49.198 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-05-08T18:09:49.200 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51508.log 2024-05-08T18:09:49.200 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.35179.log: 91.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.91408.log 2024-05-08T18:09:49.200 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.0.58768.log.gz 2024-05-08T18:09:49.200 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.51508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41223.log 2024-05-08T18:09:49.205 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.91408.log: 92.0% -- replaced with /var/log/ceph/ceph-client.0.35179.log.gz 2024-05-08T18:09:49.205 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46668.log 2024-05-08T18:09:49.205 INFO:teuthology.orchestra.run.smithi050.stderr: 90.9%/var/log/ceph/ceph-client.0.41223.log: -- replaced with /var/log/ceph/ceph-client.0.91408.log.gz 2024-05-08T18:09:49.205 INFO:teuthology.orchestra.run.smithi050.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.51508.log.gz 2024-05-08T18:09:49.205 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82363.log 2024-05-08T18:09:49.205 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.46668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80548.log 2024-05-08T18:09:49.205 INFO:teuthology.orchestra.run.smithi050.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.41223.log.gz 2024-05-08T18:09:49.205 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.82363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55743.log 2024-05-08T18:09:49.205 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.80548.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.46668.log.gz 2024-05-08T18:09:49.205 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41828.log 2024-05-08T18:09:49.207 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.55743.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.55743.log.gz 2024-05-08T18:09:49.208 INFO:teuthology.orchestra.run.smithi050.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.82363.log.gz 2024-05-08T18:09:49.208 INFO:teuthology.orchestra.run.smithi050.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.80548.log.gz 2024-05-08T18:09:49.208 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.93223.log 2024-05-08T18:09:49.208 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.41828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81758.log 2024-05-08T18:09:49.208 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.93223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.95090.log 2024-05-08T18:09:49.209 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.81758.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.41828.log.gz 2024-05-08T18:09:49.210 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38204.log 2024-05-08T18:09:49.210 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.93223.log.gz 2024-05-08T18:09:49.210 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.95090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50903.log 2024-05-08T18:09:49.210 INFO:teuthology.orchestra.run.smithi050.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.81758.log.gz 2024-05-08T18:09:49.210 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.38204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48483.log 2024-05-08T18:09:49.211 INFO:teuthology.orchestra.run.smithi050.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.0.38204.log.gz 2024-05-08T18:09:49.211 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.50903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76313.log 2024-05-08T18:09:49.211 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.48483.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.50903.log.gz 2024-05-08T18:09:49.212 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47273.log 2024-05-08T18:09:49.213 INFO:teuthology.orchestra.run.smithi050.stderr: 91.1%/var/log/ceph/ceph-client.0.76313.log: 93.2% -- replaced with /var/log/ceph/ceph-client.0.95090.log.gz 2024-05-08T18:09:49.213 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.0.48483.log.gz 2024-05-08T18:09:49.214 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73288.log 2024-05-08T18:09:49.214 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.47273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45458.log 2024-05-08T18:09:49.215 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.73288.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.76313.log.gz 2024-05-08T18:09:49.215 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37599.log 2024-05-08T18:09:49.215 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.45458.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.47273.log.gz 2024-05-08T18:09:49.215 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30929.log 2024-05-08T18:09:49.216 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.37599.log: 91.6% -- replaced with /var/log/ceph/ceph-client.0.73288.log.gz 2024-05-08T18:09:49.219 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34306.log 2024-05-08T18:09:49.219 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.0.45458.log.gz 2024-05-08T18:09:49.219 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69053.log 2024-05-08T18:09:49.219 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.30929.log: /var/log/ceph/ceph-client.admin.34306.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.37599.log.gz 2024-05-08T18:09:49.219 INFO:teuthology.orchestra.run.smithi050.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.30929.log.gz 2024-05-08T18:09:49.219 INFO:teuthology.orchestra.run.smithi050.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.34306.log.gz 2024-05-08T18:09:49.219 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81153.log 2024-05-08T18:09:49.219 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.69053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.88988.log 2024-05-08T18:09:49.219 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.81153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54533.log 2024-05-08T18:09:49.220 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.88988.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.69053.log.gz 2024-05-08T18:09:49.220 INFO:teuthology.orchestra.run.smithi050.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.81153.log.gz 2024-05-08T18:09:49.220 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84771.log 2024-05-08T18:09:49.221 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.54533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53323.log 2024-05-08T18:09:49.221 INFO:teuthology.orchestra.run.smithi050.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.88988.log.gz 2024-05-08T18:09:49.221 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.84771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34226.log 2024-05-08T18:09:49.222 INFO:teuthology.orchestra.run.smithi050.stderr: 90.3%/var/log/ceph/ceph-client.0.53323.log: -- replaced with /var/log/ceph/ceph-client.0.84771.log.gz 2024-05-08T18:09:49.224 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60583.log 2024-05-08T18:09:49.224 INFO:teuthology.orchestra.run.smithi050.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.54533.log.gz 2024-05-08T18:09:49.224 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.admin.34226.log: 85.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50298.log 2024-05-08T18:09:49.224 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34226.log.gz 2024-05-08T18:09:49.224 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.60583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75708.log 2024-05-08T18:09:49.224 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.50298.log: 91.9% -- replaced with /var/log/ceph/ceph-client.0.53323.log.gz 2024-05-08T18:09:49.225 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61188.log 2024-05-08T18:09:49.225 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.75708.log: 92.1% -- replaced with /var/log/ceph/rgw.ceph.client.0.log.gz 2024-05-08T18:09:49.225 INFO:teuthology.orchestra.run.smithi050.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.60583.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.0.55138.log 90.9% 2024-05-08T18:09:49.225 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.0.50298.log.gz 2024-05-08T18:09:49.225 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-05-08T18:09:49.226 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.61188.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.75708.log.gz 2024-05-08T18:09:49.226 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63608.log 2024-05-08T18:09:49.227 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.55138.log: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59373.log 2024-05-08T18:09:49.227 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/ceph-client.0.61188.log.gz 2024-05-08T18:09:49.228 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.63608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73893.log 2024-05-08T18:09:49.228 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/ceph-client.0.59373.log: /var/log/ceph/ceph-client.0.73893.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.55138.log.gz 2024-05-08T18:09:49.229 INFO:teuthology.orchestra.run.smithi050.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.63608.log.gz 2024-05-08T18:09:49.229 INFO:teuthology.orchestra.run.smithi050.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.0.59373.log.gz 2024-05-08T18:09:49.229 INFO:teuthology.orchestra.run.smithi050.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.73893.log.gz 2024-05-08T18:09:49.239 INFO:teuthology.orchestra.run.smithi050.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-08T18:09:49.296 INFO:teuthology.orchestra.run.smithi046.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-08T18:09:50.069 INFO:teuthology.orchestra.run.smithi050.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-08T18:09:50.208 INFO:teuthology.orchestra.run.smithi046.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-08T18:09:50.210 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-05-08T18:09:50.210 INFO:teuthology.orchestra.run.smithi046.stderr:real 0m1.111s 2024-05-08T18:09:50.211 INFO:teuthology.orchestra.run.smithi046.stderr:user 0m1.256s 2024-05-08T18:09:50.211 INFO:teuthology.orchestra.run.smithi046.stderr:sys 0m0.077s 2024-05-08T18:09:50.253 INFO:teuthology.orchestra.run.smithi050.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-08T18:09:50.306 INFO:teuthology.orchestra.run.smithi050.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-08T18:09:50.309 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-05-08T18:09:50.309 INFO:teuthology.orchestra.run.smithi050.stderr:real 0m1.169s 2024-05-08T18:09:50.309 INFO:teuthology.orchestra.run.smithi050.stderr:user 0m3.320s 2024-05-08T18:09:50.309 INFO:teuthology.orchestra.run.smithi050.stderr:sys 0m0.203s 2024-05-08T18:09:50.309 INFO:tasks.ceph:Archiving logs... 2024-05-08T18:09:50.309 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/ivancich-2024-05-08_17:43:43-rgw-wip-eric-testing-1-distro-default-smithi/7697857/remote/smithi046/log 2024-05-08T18:09:50.310 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T18:09:50.542 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/log/ceph to /home/teuthworker/archive/ivancich-2024-05-08_17:43:43-rgw-wip-eric-testing-1-distro-default-smithi/7697857/remote/smithi050/log 2024-05-08T18:09:50.543 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T18:09:51.111 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T18:09:51.122 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T18:09:51.122 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T18:09:51.125 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T18:09:51.203 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T18:09:51.211 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T18:09:51.212 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T18:09:51.215 DEBUG:teuthology.orchestra.run.smithi050:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T18:09:51.229 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-05-08T18:09:51.230 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-05-08T18:09:51.234 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T18:09:51.234 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-05-08T18:09:51.234 INFO:teuthology.orchestra.run.smithi046.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 -2191us[-2182us] +/- 61ms 2024-05-08T18:09:51.234 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 44 +1985us[+1994us] +/- 64ms 2024-05-08T18:09:51.234 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +497us[ +506us] +/- 81ms 2024-05-08T18:09:51.234 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T18:09:51.236 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T18:09:51.236 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-05-08T18:09:51.236 INFO:teuthology.orchestra.run.smithi050.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 -2185us[-2173us] +/- 61ms 2024-05-08T18:09:51.236 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 43 +2002us[+2002us] +/- 64ms 2024-05-08T18:09:51.236 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +493us[ +505us] +/- 82ms 2024-05-08T18:09:51.236 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T18:09:51.236 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T18:09:51.245 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T18:09:51.245 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T18:09:51.286 DEBUG:teuthology.orchestra.run.smithi046:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T18:09:51.353 DEBUG:teuthology.orchestra.run.smithi050:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T18:09:51.417 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T18:09:51.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T18:09:51.448 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T18:09:51.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T18:09:51.477 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T18:09:51.487 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T18:09:51.495 INFO:teuthology.task.internal:Duration was 773.313952 seconds 2024-05-08T18:09:51.495 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T18:09:51.545 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T18:09:51.545 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T18:09:51.547 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T18:09:51.587 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T18:09:51.588 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T18:09:51.924 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T18:09:51.924 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-05-08T18:09:51.924 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T18:09:51.988 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi050.front.sepia.ceph.com 2024-05-08T18:09:51.988 DEBUG:teuthology.orchestra.run.smithi050:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T18:09:52.015 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T18:09:52.016 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T18:09:52.031 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T18:09:52.083 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T18:09:52.083 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T18:09:52.100 DEBUG:teuthology.orchestra.run.smithi050:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T18:09:52.221 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T18:09:52.233 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T18:09:52.233 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T18:09:52.259 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T18:09:52.284 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T18:09:52.294 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T18:09:52.302 DEBUG:teuthology.orchestra.run.smithi050:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T18:09:52.324 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-05-08T18:09:52.349 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = core 2024-05-08T18:09:52.372 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T18:09:52.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T18:09:52.413 DEBUG:teuthology.orchestra.run.smithi050:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T18:09:52.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T18:09:52.428 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T18:09:52.438 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T18:09:52.439 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/ivancich-2024-05-08_17:43:43-rgw-wip-eric-testing-1-distro-default-smithi/7697857/remote/smithi046 2024-05-08T18:09:52.439 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T18:09:52.487 DEBUG:teuthology.misc:Transferring archived files from smithi050:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/ivancich-2024-05-08_17:43:43-rgw-wip-eric-testing-1-distro-default-smithi/7697857/remote/smithi050 2024-05-08T18:09:52.487 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T18:09:52.520 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T18:09:52.520 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T18:09:52.524 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T18:09:52.572 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T18:09:52.582 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T18:09:52.582 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T18:09:52.591 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T18:09:52.592 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T18:09:52.594 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T18:09:52.607 INFO:teuthology.orchestra.run.smithi046.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 18:09 /home/ubuntu/cephtest 2024-05-08T18:09:52.627 INFO:teuthology.orchestra.run.smithi050.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 18:09 /home/ubuntu/cephtest 2024-05-08T18:09:52.628 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T18:09:52.638 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T18:09:52.659 INFO:teuthology.run:Summary data: description: rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{centos_latest}} duration: 773.3139519691467 flavor: default owner: scheduled_ivancich@teuthology success: true 2024-05-08T18:09:52.659 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T18:09:52.737 INFO:teuthology.run:pass